<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hello everbody,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-GB">finally we got Varnish 6.0 + Vmods + Hitch TLS running on Ubuntu. So far everything works fine, also HTTP/2 Support.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">But as soon as we are receiving some more requests (approx. > 500req/s) varnish does not deliver all requests anymore. The syslog is flooded with messages like:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Jul 4 08:04:05 cache1-vm hitch[5480]: 20180704T080405.289397 [ 5502] {backend-connect}: Connection refused<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Jul 4 08:04:05 cache1-vm hitch[5480]: 20180704T080405.290213 [ 5502] {backend-connect}: Connection refused<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Jul 4 07:57:29 cache1-vm hitch[5480]: 20180704T075729.837457 [ 5504] xxx.xxx.xxx.xxx:5835 :0 1469:1470 backend connect timeout<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Jul 4 07:57:29 cache1-vm hitch[5480]: 20180704T075729.851809 [ 5510] xxx.xxx.xxx.xxx:54396 :0 7190:7191 backend connect timeout<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Jul 4 07:57:31 cache1-vm hitch[5480]: 20180704T075731.488096 [ 5510] {backend} Socket error: Connection reset by peer</span><span lang="EN-GB" style="font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">We had some similar problems in the past with HTTPS. We could solve them by adding:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">net.ipv4.ip_local_port_range = 4096 64999<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">net.ipv4.tcp_tw_reuse = 1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">to /etc/sysctl.conf But this seems to be not the problem here.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">As soon as we remove </span><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">alpn-protos = "h2,http/1.1"</span><span lang="EN-GB"> from hitch.conf everything is working normally.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Are there any limitations regarding HTTP/2 within varnish?
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Our config: <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">[Service]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">Type=simple<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">LimitNOFILE=131072<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">LimitMEMLOCK=82000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -a '[::1]:6086,PROXY' -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p thread_pools=2
-p thread_pool_min=200 -p thread_pool_max=5000 -p timeout_idle=100 -p send_timeout=3600 -p feature=+http2 -s malloc,2g<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">ProtectSystem=full<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">ProtectHome=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">PrivateTmp=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">PrivateDevices=true<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Also DefaultLimitNOFILE is set to 250000 in /etc/systemd/system.conf.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Hitch config:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""># Listening<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">frontend = "[*]:443"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">ciphers = "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""># TLS 1.0 wird gebraucht, damit alte Server (intranet-srv) eine Verbindung aufbauen können<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">tls-protos = TLSv1.1 TLSv1.2 TLSv1.0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""># Send traffic to the Varnish backend using the PROXY protocol<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">backend = "[::1]:6086"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">write-proxy-v2 = on<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">alpn-protos = "h2,http/1.1"<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""># Number of processes<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New"">workers = 8<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:9.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">We already searched on the varnish github account for similar problem, but did not found anything...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Thomas<o:p></o:p></span></p>
</div>
<br>
<hr>
<font face="Verdana" color="Black" size="1">RADIO / TELE FFH GmbH & Co. Betriebs-KG<br>
FFH-Platz 1, 61111 Bad Vilbel<br>
HRA - Nr. 26092 Frankfurt/Main<br>
USt.IdNr. DE 112152620<br>
Geschäftsführer / Programmdirektor: Hans-Dieter Hillmoth<br>
</font>
</body>
</html>