Shouldnt saint mode return a 200 OK (stale cache item) if the backend fails and you are within the "Saint Window" for that object? I dont think turning off saint mode is going to correct this issue.<br><br><div class="gmail_quote">
On Fri, Nov 4, 2011 at 4:26 PM, Lei Wang <span dir="ltr"><<a href="mailto:leiwang@rhapsody.com">leiwang@rhapsody.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi August,<br><br>Thank for the info. I will definitely try that today.<br><br>Have a nice day.<br><font color="#888888"><br>Lei</font><div><div></div><div class="h5"><br><br><div class="gmail_quote">On Fri, Nov 4, 2011 at 12:48 PM, Gresens, August <span dir="ltr"><<a href="mailto:AGresens@scholastic.com" target="_blank">AGresens@scholastic.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="blue" lang="EN-US">
<div>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy">We were able to resolve this. Our Varnish
instance is pointing a load balanced pool of backend servers (a vip on a
hardware load balancer), so to Varnish there is only one backend. We had a
situation in which certain images were returning 500 errors from the backend.
Because saint mode was enabled, it invalidated the backend (the only back end)
after a certain number of failures within a particular window. We disabled
saint mode and this fixed this issue. Best, August<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" color="navy" face="Arial"><span style="font-size:10.0pt;font-family:Arial;color:navy"><u></u> <u></u></span></font></p>
<div>
<div class="MsoNormal" style="text-align:center" align="center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">From:</span></font></b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> Lei Wang
[mailto:<a href="mailto:leiwang@rhapsody.com" target="_blank">leiwang@rhapsody.com</a>] <br>
<b><span style="font-weight:bold">Sent:</span></b> Friday, November 04, 2011
3:40 PM<br>
<b><span style="font-weight:bold">To:</span></b> Rob S<br>
<b><span style="font-weight:bold">Cc:</span></b> Gresens, August;
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<b><span style="font-weight:bold">Subject:</span></b> Re: Intermittent 503's
from Varnish</span></font><u></u><u></u></p>
</div><div><div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">I had the same problem. Please help!<br>
<br>
/usr/sbin/varnishd -P /var/run/varnish80.pid -a :80 -f
/etc/varnish/varnish80.vcl -T <a href="http://127.0.0.1:6082" target="_blank">127.0.0.1:6082</a>
-t 120 -w 1,1000,120 -u varnish -g varnish -i varnish81 -S /etc/varnish/secret
-p thread_pool_add_delay 2 -p thread_pools 1 -p thread_pool_min 800 -p
thread_pool_max 2000 -p session_linger 50 -p sess_workspace 4096 -n varnish80
-s malloc,6.5G<br>
<br>
configuration related to this problem:<br>
backend eadrm {<br>
.host = "<a href="http://drm-ea.xxxxxxxx.com" target="_blank">drm-ea.xxxxxxxx.com</a>";<br>
.port = "80";<br>
.connect_timeout = 40s;<br>
.first_byte_timeout = 80s;<br>
.between_bytes_timeout = 40s;<br>
}<br>
<br>
} elsif (req.http.host ~ "drm-ea") {<br>
unset beresp.http.set-cookie;<br>
set beresp.ttl = 86400s;<br>
log "Adjusting EA-DRM TTL to 86400 seconds";<br>
<br>
Thanks,<br>
<br>
Lei<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Fri, Nov 4, 2011 at 11:52 AM, Rob S <<a href="mailto:rtshilston@gmail.com" target="_blank">rtshilston@gmail.com</a>> wrote:<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Can you send the varnishlog for one of these failing requests? Is<br>
this a POST or a GET request? Are you using default VCL, or can you<br>
send the VCL you are using?<br>
<br>
Rob<u></u><u></u></span></font></p>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Fri, Nov 4, 2011 at 6:44 PM, Gresens, August <<a href="mailto:AGresens@scholastic.com" target="_blank">AGresens@scholastic.com</a>> wrote:<br>
> Hello<br>
><br>
><br>
><br>
> We’re having a very nasty production issue. We’re getting intermittent 503<br>
> from Varnish in front of a healthy backend.<br>
><br>
><br>
><br>
> This seems to happen on cache-miss, but we’re not seeing requests make it
to<br>
> the backend. Varnish immediately returns the 503. After the first 503, the<br>
> page can usually be loaded by requesting it a few more times. This seems
to<br>
> indicate that there is some condition within Varnish that prevents it from<br>
> hitting the backend that clears up quickly.<br>
><br>
><br>
><br>
> Has anyone seen this issue before? Is there anything we can try?<br>
><br>
><br>
><br>
> This is our startup config:<br>
><br>
><br>
><br>
> DAEMON_OPTS="-a :80 \<br>
><br>
> -T localhost:6082 \<br>
><br>
> -f /etc/varnish/default.vcl \<br>
><br>
> -S /etc/varnish/secret \<br>
><br>
> -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G \<br>
><br>
> -p http_headers=128 \<br>
><br>
> -p sess_workspace=262144 \<br>
><br>
> -p connect_timeout=60 \<br>
><br>
> -p max_restarts=6 \<br>
><br>
> -p log_hashstring=on \<br>
><br>
> -p cli_timeout=60 \<br>
><br>
> -t 0"<br>
><br>
><br>
><br>
> We are on version 2.0.5<br>
><br>
><br>
><br>
> Thanks,<br>
><br>
><br>
><br>
> August<br>
><br>
><br>
><br>
><br>
><u></u><u></u></span></font></p>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">> _______________________________________________<br>
> varnish-misc mailing list<br>
> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
> <a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
><br>
<br>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><u></u><u></u></span></font></p>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div></div></div>
</div>
</blockquote></div><br>
</div></div><br>_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br></blockquote></div><br>