<div dir="ltr">Guys,<div><br></div><div> I know I've been a pain in the ass about this. Sorry about that. But I just wanted to let you know that I got this problem sorted out. The problem was in fact that the web servers were too damn far away from the varnish caches. I spun up a few VMs in the new york data center where the varnish machines were located. And voila! Everything fell into place.</div><div><br></div><div>My setup is and always has been a little odd. I have two haproxy load balancers in AWS / northern Virginia. And I have two databases in the same location. Everything is done on the free tier. </div><div><br></div><div>But vanish would have trouble running on a t2 micro instance so I sprung for two low cost $10 a month hosts at digital ocean to be the varnish cashes. The web servers remained at amazon using the free tier until I could work out that the distance was the problem. <br><br>So now I've sprung for 3 low cost $5 a month web servers that only have 512MB of ram. But are definitely fine as web servers.</div><div><br></div><div>So even tho my LB's and Databases are in northern virginia on AWS and my web servers and varnish are in New York on Digital Ocean, everything turned out fine. The distance between the load balancers and the databases don't appear to be an issue the way it was for the varnish cache.<br><br>And another reason I had put varnish in New York at D.O. was because that's where I'm developing my app. On a 2GB web server in that location. But I also have a wiki that I wanted to cache on varnish. And that's what this project was about.<br><br>I'm glad that everything's resolved. <br><br>I appreciated your input and advice as always.</div><div><br></div><div>Thanks,</div><div>Tim</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 29, 2015 at 10:18 PM, Tim Dunphy <span dir="ltr"><<a href="mailto:bluethundr@gmail.com" target="_blank">bluethundr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey all,<div><br></div><div><div>I'm having trouble getting basic auth to work under varnish 4. I'm setting it up in front of a mediawiki site.</div><div><br></div><div>If I hit the page from either a web browser I get a 504 The server didn't respond in time error. This doesn't happen when basic auth is not enabled in either the apache config or the varnish config. Which makes me think I may be handling basic auth wrong somewhere in my setup.</div><div><br></div><div>I am able to curl the health check file through varnish. I'm doing this on the varnish server itself:</div><div><br></div><div> #time curl --user admin <a href="http://wiki.example.com/healthcheck.php" target="_blank">http://wiki.example.com/healthcheck.php</a></div><div> Enter host password for user 'admin':</div><div> good</div><div> </div><div> real 0m3.080s</div><div> user 0m0.003s</div><div> sys 0m0.004s</div><div><br></div><div>The health check file contains only the word 'good'.</div><div><br></div><div>On the web server, the healthcheck.php file is in the doc root of the wiki site and is readable by the apache user:</div><div><br></div><div> #ls -l /var/www/jf/wiki/healthcheck.php</div><div> -rw-r--r--. 1 apache ftpgroup 5 Jul 17 00:42 /var/www/jf/wiki/healthcheck.php</div><div><br></div><div>I've setup a no auth exception in the apache vhost for the site:</div><div><br></div><div> <VirtualHost *:80></div><div> ServerName <a href="http://wiki.example.com" target="_blank">wiki.example.com</a></div><div> ServerAlias <a href="http://www.wiki.example.com" target="_blank">www.wiki.example.com</a></div><div> Options -Indexes +FollowSymlinks</div><div> LogLevel debug</div><div> ErrorLog logs/wiki-error.log</div><div> LogFormat "%h %l %u %t \"%r\" %>s %b" common</div><div> CustomLog logs/wiki-access_log common</div><div> DocumentRoot /var/www/jf/wiki</div><div> SetEnvIf Request_URI ^/healthcheck.php noauth=1</div><div> </div><div> <Directory /var/www/jf/wiki></div><div> Options -Indexes</div><div> AuthType Basic</div><div> AuthName "JF Wiki Page"</div><div> AuthUserFile /etc/httpd/auth</div><div> Require valid-user</div><div> #equire all granted</div><div> Allow from env=noauth</div><div> </Directory></div><div> </div><div> <Directory /var/www/jf/wiki/images></div><div> Options -Indexes</div><div> </Directory></div><div> </VirtualHost></div><div><br></div><div><br></div><div>On the varnish end I installed the 'basicauth' vmod, and imported it. Then set it up in the VCL. Here's how my VCL is looking:</div><div><br></div><div> #egrep -v '#|^$' default.vcl</div><div> vcl 4.0;</div><div> import std;</div><div> import directors;</div><div> import basicauth;</div><div> backend web1 {</div><div> .host = "10.10.10.25"; # <-- not a real IP</div><div> .port = "80";</div><div> .connect_timeout = 45s;</div><div> .first_byte_timeout = 45s;</div><div> .between_bytes_timeout = 45s;</div><div> .max_connections = 800;</div><div> .probe = {</div><div> .request =</div><div> "GET /healthcheck.php HTTP/1.1"</div><div> "Host: <a href="http://wiki.example.com" target="_blank">wiki.example.com</a>"</div><div> "Authorization: Basic LongBasicAuthBase64Hash=="</div><div> "Connection: close";</div><div> .timeout = 10s;</div><div> .interval = 1s;</div><div> .window = 15;</div><div> .threshold = 8;</div><div> }</div><div> }</div><div> sub vcl_init {</div><div> new wiki = directors.round_robin();</div><div> wiki.add_backend(web1);</div><div> }</div><div> sub vcl_recv {</div><div> set req.backend_hint = wiki.backend();</div><div> if (!basicauth.match("/etc/httpd/auth", req.http.Authorization)) {</div><div> return(synth(401, "Authentication required"));</div><div> }</div><div> }</div><div> sub vcl_backend_response {</div><div> }</div><div> sub vcl_deliver {</div><div> }</div><div> sub vcl_synth {</div><div> if (resp.status == 401) {</div><div> set resp.http.WWW-Authenticate = "Basic";</div><div> }</div><div> }</div><div><br></div><div><br></div><div>You can see in my VCL that I'm attempting to pass basic auth headers to the healthcheck .probe.</div><div><br></div><div>In varnishlog, when I'm getting the 504 errors in the browser, I'm seeing the following:</div><div><br></div><div> - Timestamp Process: 1438220128.357217 5.381197 0.000029</div><div> - RespHeader Transfer-Encoding: chunked</div><div> - Debug "RES_MODE 8"</div><div> - RespHeader Connection: close</div><div> - RespHeader Accept-Ranges: bytes</div><div> - Debug "Write error, retval = -1, len = 14553, errno = Connection reset by peer"</div><div> - Timestamp Resp: 1438220128.357317 5.381297 0.000101</div><div> - Debug "XXX REF 1"</div><div> - ReqAcct 506 0 506 0 0 0</div><div> - End</div><div><br></div><div>And in the apache error log for the site I'm seeing this authorization error that corresponds with the time that I'm getting the 504 error:</div><div><br></div><div> [Thu Jul 30 01:37:43.197847 2015] [authz_core:debug] [pid 29441] mod_authz_core.c(809): [client <a href="http://10.10.10.19:47588" target="_blank">10.10.10.19:47588</a>] AH01626: authorization result of Require valid-user : denied (no authenticated user yet)</div><div><br></div><div>I'm hoping to get some suggestions that will get this to work!</div><div><br></div><div>Thanks</div><span class="HOEnZb"><font color="#888888"><div>Tim</div>-- <br><div>GPG me!!<br><br>gpg --keyserver <a href="http://pool.sks-keyservers.net" target="_blank">pool.sks-keyservers.net</a> --recv-keys F186197B<br><br></div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">GPG me!!<br><br>gpg --keyserver <a href="http://pool.sks-keyservers.net" target="_blank">pool.sks-keyservers.net</a> --recv-keys F186197B<br><br></div>
</div>