<div dir="ltr"><div>
<p class=""><span class="">Hey guys,</span></p><p class=""><span class=""> I've setup my wiki to be delivered through varnish. My first breakthrough in getting this to work was to allow for http basic auth to pass through varnish so the user will be prompted to login. This is working quite well at this point. </span></p><p class=""><span class="">But my current problem is that I've been able to get the main page to the site cached, and a few others. But when I click to like the 4th or 5th page on the wiki that I want to cache through varnish, everything comes to a halt. I get a 503 - service unavailable error, instead of varnish going to the web server to retrieve the page I had clicked on.</span></p><p class=""><span class="">In my varnishlog, I'm seeing the following which results in an error:</span></p><p class=""><span class="">10 SessionOpen c 10.10.10.25 35852 :80</span></p><p class=""><span class=""> 10 ReqStart c 10.10.10.25 35852 1369002554</span></p><p class=""><span class=""> 10 RxRequest c GET</span></p><p class=""><span class=""> 10 RxURL c /index.php/Affluencers</span></p><p class=""><span class=""> 10 RxProtocol c HTTP/1.1</span></p><p class=""><span class=""> 10 RxHeader c Host: <a href="http://wiki.example.com/"><span class="">wiki.example.com</span></a></span></p><p class=""><span class=""> 10 RxHeader c Cache-Control: max-age=0</span></p><p class=""><span class=""> 10 RxHeader c Authorization: Basic YWRtaW46RHVrMzBmWmgwdQ==</span></p><p class=""><span class=""> 10 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8</span></p><p class=""><span class=""> 10 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36</span></p><p class=""><span class=""> 10 RxHeader c Referer: <a href="http://wiki.example.com/index.php/NBCUniversal"><span class="">http://wiki.example.com/index.php/NBCUniversal</span></a></span></p><p class=""><span class=""> 10 RxHeader c Accept-Encoding: gzip, deflate, sdch</span></p><p class=""><span class=""> 10 RxHeader c Accept-Language: en-US,en;q=0.8</span></p><p class=""><span class=""> 10 RxHeader c X-Forwarded-For: 47.18.111.100</span></p><p class=""><span class=""> 10 RxHeader c Connection: close</span></p><p class=""><span class=""> 10 VCL_call c recv lookup</span></p><p class=""><span class=""> 10 VCL_call c hash</span></p><p class=""><span class=""> 10 Hash c /index.php/Affluencers</span></p><p class=""><span class=""> 10 Hash c <a href="http://wiki.example.com/"><span class="">wiki.example.com</span></a></span></p><p class=""><span class=""> 10 VCL_return c hash</span></p><p class=""><span class=""> 10 VCL_call c miss fetch</span></p><p class=""><span class=""> 10 FetchError c no backend connection</span></p><p class=""><span class=""> 10 VCL_call c error deliver</span></p><p class=""><span class=""> 10 VCL_call c deliver deliver</span></p><p class=""><span class=""> 10 TxProtocol c HTTP/1.1</span></p><p class=""><span class=""> 10 TxStatus c 503</span></p><p class=""><span class=""> 10 TxResponse c Service Unavailable</span></p><p class=""><span class=""> 10 TxHeader c Server: Varnish</span></p><p class=""><span class=""> 10 TxHeader c Content-Type: text/html; charset=utf-8</span></p><p class=""><span class=""> 10 TxHeader c Retry-After: 5</span></p><p class=""><span class=""> 10 TxHeader c Content-Length: 419</span></p><p class=""><span class=""> 10 TxHeader c Accept-Ranges: bytes</span></p><p class=""><span class=""> 10 TxHeader c Date: Sun, 05 Jul 2015 00:42:16 GMT</span></p><p class=""><span class=""> 10 TxHeader c X-Varnish: 1369002554</span></p><p class=""><span class=""> 10 TxHeader c Age: 0</span></p><p class=""><span class=""> 10 TxHeader c Via: 1.1 varnish</span></p><p class=""><span class=""> 10 TxHeader c Connection: close</span></p><p class=""><span class=""> 10 TxHeader c X-Cache: MISS</span></p><p class=""><span class=""> 10 Length c 419</span></p><p class=""><span class=""> 10 ReqEnd c 1369002554 1436056936.048977137 1436056936.049291372 0.000086308 0.000152111 0.000162125</span></p><p class=""><span class=""> 10 SessionClose c error</span></p><p class=""><span class=""> 10 StatSess c 10.10.10.25 35852 0 1 1 0 0 0 272 419</span></p><p class=""><span class="">I'm not sure how to interpret that to be honest. But I'd like some help resolving this issue. :)</span></p><p class=""><span class="">One thing that I considered is that I may be running out of memory while trying to cache the site. </span></p><p class=""><span class="">The varnish host itself is only a 1GB VM with 40GB of disk space. </span></p><p class=""><span class="">Here's my varnish system config:</span></p><p class=""><span class="">grep -v '#' /etc/sysconfig/varnish</span></p><p class=""><span class="">NFILES=131072<br>
</span></p><p class=""><span class="">MEMLOCK=82000<br>
</span></p><p class=""><span class="">NPROCS="unlimited"<br>
</span></p><p class=""><span class="">RELOAD_VCL=1</span></p><p class=""><span class="">VARNISH_VCL_CONF=/etc/varnish/default.vcl<br>
</span></p><p class=""><span class="">VARNISH_LISTEN_PORT=80</span></p><p class=""><span class="">VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1</span></p><p class=""><span class="">VARNISH_ADMIN_LISTEN_PORT=6082</span></p><p class=""><span class="">VARNISH_SECRET_FILE=/etc/varnish/secret</span></p><p class=""><span class="">VARNISH_MIN_THREADS=50</span></p><p class=""><span class="">VARNISH_MAX_THREADS=1000</span></p><p class=""><span class="">VARNISH_THREAD_TIMEOUT=120</span></p><p class=""><span class="">VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin</span></p><p class=""><span class="">VARNISH_STORAGE_SIZE=10G</span></p><p class=""><span class="">VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"</span></p><p class=""><span class="">VARNISH_TTL=120</span></p><p class=""><span class="">DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \</span></p><p class=""><span class=""> -f ${VARNISH_VCL_CONF} \</span></p><p class=""><span class=""> -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \</span></p><p class=""><span class=""> -t ${VARNISH_TTL} \</span></p><p class=""><span class=""> -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \</span></p><p class=""><span class=""> -u varnish -g varnish \</span></p><p class=""><span class=""> -S ${VARNISH_SECRET_FILE} \</span></p><p class=""><span class=""></span><br></p><p class=""><span class=""> -s ${VARNISH_STORAGE}"</span></p><p class=""><span class="">And this is my very simple VCL file: </span></p><p class=""><span class="">backend web1 {</span></p><p class=""><span class=""> .host = "10.10.10.25";</span></p><p class=""><span class=""> .port = "80";</span></p><p class=""><span class=""> .connect_timeout = 30s;</span></p><p class=""><span class=""> .first_byte_timeout = 30s;</span></p><p class=""><span class=""> .between_bytes_timeout = 30s;</span></p><p class=""><span class=""> .max_connections = 70;</span></p><p class=""><span class=""> .probe = {</span></p><p class=""><span class=""> .request =</span></p><p class=""><span class=""> "GET /healthcheck.php HTTP/1.1"</span></p><p class=""><span class=""> "Host: <a href="http://wiki.example.com">wiki.example.com</a>"</span></p><p class=""><span class=""> "Connection: close";</span></p><p class=""><span class=""> .interval = 10m;</span></p><p class=""><span class=""> .timeout = 60s;</span></p><p class=""><span class=""> .window = 3;</span></p><p class=""><span class=""> .threshold = 2;</span></p><p class=""><span class=""> }</span></p><p class=""><span class="">}</span></p><p class=""><span class="">backend web2 {<br>
</span></p><p class=""><span class=""> .host = "10.10.10.26";</span></p><p class=""><span class=""> .port = "80";</span></p><p class=""><span class=""> .connect_timeout = 30s;</span></p><p class=""><span class=""> .first_byte_timeout = 30s;</span></p><p class=""><span class=""> .between_bytes_timeout = 30s;</span></p><p class=""><span class=""> .max_connections = 70;</span></p><p class=""><span class=""> .probe = {</span></p><p class=""><span class=""> .request =</span></p><p class=""><span class=""> "GET /healthcheck.php HTTP/1.1"</span></p><p class=""><span class=""> "Host: <a href="http://wiki.example.com">wiki.example.com</a>"</span></p><p class=""><span class=""> "Connection: close";</span></p><p class=""><span class=""> .interval = 10m;</span></p><p class=""><span class=""> .timeout = 60s;</span></p><p class=""><span class=""> .window = 3;</span></p><p class=""><span class=""> .threshold = 2;</span></p><p class=""><span class=""> }</span></p><p class=""><span class="">}</span></p><p class=""><span class=""><br>director www round-robin {</span></p><p class=""><span class=""> { .backend = web1; }</span></p><p class=""><span class=""> { .backend = web2; }</span></p><p class=""><span class=""> }</span></p><p class=""><span class=""><br>sub vcl_recv {</span></p><p class=""><span class=""> set req.backend = www;<br> return (lookup);</span></p><p class=""><span class="">}</span></p><p class=""><span class="">sub vcl_fetch {</span></p><p class=""><span class=""> set beresp.ttl = 3600s;</span></p><p class=""><span class=""> set beresp.grace = 4h;</span></p><p class=""><span class=""> return (deliver);</span></p><p class=""><span class="">}</span></p><p class=""><span class="">sub vcl_deliver {<br>
</span></p><p class=""><span class=""> if (obj.hits> 0) {</span></p><p class=""><span class=""> set resp.http.X-Cache = "HIT";</span></p><p class=""><span class=""> } else {</span></p><p class=""><span class=""> set resp.http.X-Cache = "MISS";</span></p><p class=""><span class=""> }</span></p><p class=""><span class=""> }</span></p><p class="">So I'm wondering if I'm anywhere near the mark that I could be running out of memory when trying to cache pages to my wiki site. <br><span class=""></span></p><p class="">I'd really appreciate any feedback you guys may have!</p><p class="">Thanks,</p><p class="">Tim</p></div>
</div>