<meta charset="utf-8"><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; ">I could use some help troubleshooting why users sometimes receive a 503 error from Varnish. It does not happen every time, but it is fairly reproducible on the below example (<a href="http://gowalla.com/users/keeg" target="_blank" style="color: rgb(51, 102, 51); ">http://gowalla.com/users/keeg</a>). The backends are all configured as follows:<div>
<br></div><div><div><font face="'courier new', monospace">backend server1 {</font></div><div><font face="'courier new', monospace"> .host = "<internal_ip>";</font></div><div><font face="'courier new', monospace"> .port = "<internal_port>";</font></div>
<div><font face="'courier new', monospace"> .connect_timeout = 30s;</font></div><div><font face="'courier new', monospace"> .first_byte_timeout = 30s;</font></div><div><font face="'courier new', monospace"> .between_bytes_timeout = 30s;</font></div>
<div><font face="'courier new', monospace">}</font></div><div><br></div><div>The director configuration is:</div><div><br></div><div><div><font face="'courier new', monospace">director rr_director round-robin {</font></div>
<div><font face="'courier new', monospace"> { .backend = server1; }</font></div><div><font face="'courier new', monospace"> { .backend = server2; }</font></div><div><font face="'courier new', monospace"> { .backend = server3; }</font></div>
<div><font face="'courier new', monospace"> { .backend = server4; }</font></div><div><font face="'courier new', monospace"> { .backend = server5; }</font></div><div><font face="'courier new', monospace"> { .backend = server6; }</font></div>
<div><font face="'courier new', monospace"> { .backend = server7; }</font></div><div><font face="'courier new', monospace"> { .backend = server8; }</font></div><div><font face="'courier new', monospace"> { .backend = server9; }</font></div>
<div><font face="'courier new', monospace">}</font></div></div><div><font face="'courier new', monospace"><br></font></div><div><font face="arial, helvetica, sans-serif">Here is the varnishlog output of a 503 response. I've searched everywhere for information on the FetchError provided, but I can't find anything that seems to apply. Also, the request is not hitting any of the timeouts listed in the backend configuration. The 503 response usually returns in less than a second.</font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div><div><div><font face="'courier new', monospace"> 27 SessionOpen c 127.0.0.1 33235 :8080</font></div><div><font face="'courier new', monospace"> 27 ReqStart c 127.0.0.1 33235 1983217687</font></div>
<div><font face="'courier new', monospace"> 27 RxRequest c GET</font></div><div><font face="'courier new', monospace"> 27 RxURL c /users/keeg</font></div><div><font face="'courier new', monospace"> 27 RxProtocol c HTTP/1.0</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c X-Real-IP: 24.153.226.54</font></div><div><font face="'courier new', monospace"> 27 RxHeader c X-Forwarded-For: 24.153.226.54</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c Host: <a href="http://gowalla.com/" target="_blank" style="color: rgb(51, 102, 51); ">gowalla.com</a></font></div><div><font face="'courier new', monospace"> 27 RxHeader c Connection: close</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c Referer: <a href="http://gowalla.com/users/keeg" target="_blank" style="color: rgb(51, 102, 51); ">http://gowalla.com/users/keeg</a></font></div><div>
<font face="'courier new', monospace"> 27 RxHeader c Cache-Control: max-age=0</font></div><div><font face="'courier new', monospace"> 27 RxHeader c Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.102 Safari/534.13</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c Accept-Encoding: gzip,deflate,sdch</font></div><div><font face="'courier new', monospace"> 27 RxHeader c Accept-Language: en-US,en;q=0.8</font></div>
<div><font face="'courier new', monospace"> 27 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3</font></div><div><font face="'courier new', monospace"> 27 RxHeader c If-None-Match: "bf9753b17d60d033e7654ea22abfcbd8"</font></div>
<div><font face="'courier new', monospace"> 27 VCL_call c recv pass</font></div><div><font face="'courier new', monospace"> 27 VCL_call c hash hash</font></div><div><font face="'courier new', monospace"> 27 VCL_call c pass pass</font></div>
<div><font face="'courier new', monospace"> 27 Backend c 29 rr_director server9</font></div><div><font face="'courier new', monospace"> 27 FetchError c http first read error: -1 0 (Success)</font></div>
<div><font face="'courier new', monospace"> 27 VCL_call c error deliver</font></div><div><font face="'courier new', monospace"> 27 VCL_call c deliver deliver</font></div><div><font face="'courier new', monospace"> 27 TxProtocol c HTTP/1.1</font></div>
<div><font face="'courier new', monospace"> 27 TxStatus c 503</font></div><div><font face="'courier new', monospace"> 27 TxResponse c Service Unavailable</font></div><div><font face="'courier new', monospace"> 27 TxHeader c Server: Varnish</font></div>
<div><font face="'courier new', monospace"> 27 TxHeader c Retry-After: 0</font></div><div><font face="'courier new', monospace"> 27 TxHeader c Content-Type: text/html; charset=utf-8</font></div>
<div><font face="'courier new', monospace"> 27 TxHeader c Content-Length: 419</font></div><div><font face="'courier new', monospace"> 27 TxHeader c Date: Thu, 17 Feb 2011 20:44:21 GMT</font></div>
<div><font face="'courier new', monospace"> 27 TxHeader c X-Varnish: 1983217687</font></div><div><font face="'courier new', monospace"> 27 TxHeader c Age: 1</font></div><div><font face="'courier new', monospace"> 27 TxHeader c Via: 1.1 varnish</font></div>
<div><font face="'courier new', monospace"> 27 TxHeader c Connection: close</font></div><div><font face="'courier new', monospace"> 27 Length c 419</font></div><div><font face="'courier new', monospace"> 27 ReqEnd c 1983217687 1297975461.143035889 1297975461.773808956 0.000031948 0.630686998 0.000086069</font></div>
</div><div><br></div></div><div>I would appreciate any insights anyone might have.</div><div><br></div><div>Thanks,</div><div><br></div><div>--</div><div>Adam McManus</div></span>