Yes, when the 503 error is happened, all my backend is in heavy load, the load is more than 40. the very strange thing is that I can wget the probe file hundreds times, and there is no fail. I can not explain it.<div><br>
</div><div>We use cookie to save customers information, when guest come back to our website, it helps to do auto login. our varnish server work load is very low, under 1.0 in most time, but web server is in heavy load. Your answer can explain it. </div>
<div><div><br></div><div>the hit raito sees terrible. that is the output of varnishstat:</div><div><div>varnishstat -n vcache-my</div><div>=========================================================================</div><div>
0+09:20:26 vcache-my</div><div>Hitrate ratio: 8 8 8</div><div>Hitrate avg: 0.6145 0.6145 0.6145</div>
<div><br></div><div> 4195411 61.92 124.77 client_conn - Client connections accepted</div><div> 5371966 65.92 159.76 client_req - Client requests received</div><div> 372757 7.99 11.09 cache_hit - Cache hits</div>
<div> 6820 1.00 0.20 cache_hitpass - Cache hits for pass</div><div> 63206 3.00 1.88 cache_miss - Cache misses</div><div> 4240160 71.91 126.10 backend_conn - Backend conn. success</div>
<div> 2277628 0.00 67.73 backend_fail - Backend conn. failures</div><div> 84 0.00 0.00 fetch_head - Fetch head</div><div> 3387280 52.93 100.73 fetch_length - Fetch with Length</div>
<div> 417030 3.99 12.40 fetch_chunked - Fetch chunked</div><div> 8669 1.00 0.26 fetch_close - Fetch wanted close</div><div> 4 0.00 0.00 fetch_failed - Fetch failed</div>
<div> 428006 0.00 12.73 fetch_304 - Fetch no body (304)</div><div> 919 . . n_sess_mem - N struct sess_mem</div><div> 339 . . n_sess - N struct sess</div>
<div> 7159 . . n_object - N struct object</div><div> 7633 . . n_objectcore - N struct objectcore</div><div> 7635 . . n_objecthead - N struct objecthead</div>
<div> 875 . . n_waitinglist - N struct waitinglist</div><div> 62 . . n_vbc - N struct vbc</div><div> 1000 . . n_wrk - N worker threads</div>
<div> 1000 0.00 0.03 n_wrk_create - N worker threads created</div><div> 219 0.00 0.01 n_wrk_queued - N queued work requests</div><div> 3 . . n_backend - N backends</div>
<div> 48739 . . n_expired - N expired objects</div><div> 2567 . . n_lru_moved - N LRU moved objects</div><div> 143 0.00 0.00 losthdr - HTTP header overflows</div>
</div><div><br></div><div>That is great, thanks Rangel, Raul, your information is import for me, I will rewrite the vcl_recv function.</div><div><br><br><div class="gmail_quote">On Tue, Jan 8, 2013 at 2:52 AM, Rangel, Raul <span dir="ltr"><<a href="mailto:Raul.Rangel@disney.com" target="_blank">Raul.Rangel@disney.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">What is the load on your backend servers? It seems like they are under heavy load. One thing I did notice in your vcl_recv is the cookie code. You are checking the existence of a specific cookie and passing if it exists. If that cookie does not exist you fall back to the default vcl_recv. Well the default vcl_recv also does a check on req.http.Cookie. If it finds one then it also does a pass. My guess would be that most of your incoming requests have some sort of cookie and thus all the requests are getting passed to the backend. Have you done a varnishstat and looked at your cache hit ratios?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If I were you I would add a unset req.http.cookie to the end of vcl_recv. This way the default vcl_recv won’t pass the request.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Raul<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:varnish-misc-bounces@varnish-cache.org" target="_blank">varnish-misc-bounces@varnish-cache.org</a> [mailto:<a href="mailto:varnish-misc-bounces@varnish-cache.org" target="_blank">varnish-misc-bounces@varnish-cache.org</a>] <b>On Behalf Of </b>Shaohui Zheng<br>
<b>Sent:</b> Monday, January 07, 2013 11:16 AM<br><b>To:</b> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br><b>Subject:</b> 503 Error was met frequently<u></u><u></u></span></p>
</div><div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Hello,<u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal"> After a few days efforts, I did not get my problem fixed, I almost exhaust every possible methods which I could do, so I am trying to get help from the community. <u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">I use varnish as web cache and load balancer to manage 3 web nodes, but recently, I get 503 errors frequently, <u></u><u></u></p></div><div>
<p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">My varnish configuration file:<u></u><u></u></p></div><div><p class="MsoNormal">=======================================================<u></u><u></u></p>
</div><div><div><p class="MsoNormal">backend nanjing {<u></u><u></u></p></div><div><p class="MsoNormal"> .host = "10.80.125.66";<u></u><u></u></p></div><div><p class="MsoNormal"> .port = "80";<u></u><u></u></p>
</div><div><p class="MsoNormal"> .connect_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"> .first_byte_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"> .between_bytes_timeout = 1800s;<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>.probe = {<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.url = "/live.html";<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>.interval = 1s;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.timeout = 3s;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.window = 10;<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>.threshold = 2;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span> }<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">backend hangzhou {<u></u><u></u></p></div><div><p class="MsoNormal"> .host = "10.80.125.68";<u></u><u></u></p></div><div><p class="MsoNormal">
#.host = "10.36.146.202";<u></u><u></u></p></div><div><p class="MsoNormal"> .port = "80";<u></u><u></u></p></div><div><p class="MsoNormal"> .connect_timeout = 1800s;<u></u><u></u></p></div>
<div><p class="MsoNormal"> .first_byte_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"> .between_bytes_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div>
<p class="MsoNormal"><span> </span>.probe = {<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.url = "/live.html";<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>.interval = 1s;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.timeout = 3s;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.window = 10;<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>.threshold = 2;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span> }<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p>
</div><div><p class="MsoNormal">backend chongqing {<u></u><u></u></p></div><div><p class="MsoNormal"> .host = "10.80.125.76";<u></u><u></u></p></div><div><p class="MsoNormal"> .port = "80";<u></u><u></u></p>
</div><div><p class="MsoNormal"> .connect_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"> .first_byte_timeout = 1800s;<u></u><u></u></p></div><div><p class="MsoNormal"> .between_bytes_timeout = 1800s;<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><span> </span>.probe = {<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.url = "/live.html";<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>.interval = 1s;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.timeout = 3s;<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>.window = 10;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.threshold = 2;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span> }<u></u><u></u></p>
</div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">
director proxy random {<u></u><u></u></p></div><div><p class="MsoNormal"> {<u></u><u></u></p></div><div><p class="MsoNormal"> .backend = chongqing;<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>.weight = 2;<u></u><u></u></p>
</div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"> {<u></u><u></u></p></div><div><p class="MsoNormal"> .backend = nanjing;<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>.weight = 4;<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"> {<u></u><u></u></p></div><div><p class="MsoNormal"> .backend = hangzhou;<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>.weight = 4;<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal">}<u></u><u></u></p></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal">acl purge {<u></u><u></u></p></div><div><p class="MsoNormal"> "localhost";<u></u><u></u></p></div><div><p class="MsoNormal"> "10.80.125.0"/24;<u></u><u></u></p>
</div><div><p class="MsoNormal">}<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">sub vcl_recv {<u></u><u></u></p></div><div><p class="MsoNormal"> set req.backend = proxy;<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"> if (req.request != "GET" && req.request != "HEAD") {<u></u><u></u></p></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal"> # POST - Logins and edits<u></u><u></u></p></div><div><p class="MsoNormal"> if (req.request == "POST") {<u></u><u></u></p></div><div>
<p class="MsoNormal"> return(pass);<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">
# PURGE - The CacheFu product can invalidate updated URLs<u></u><u></u></p></div><div><p class="MsoNormal"> if (req.request == "PURGE") {<u></u><u></u></p></div><div><p class="MsoNormal">
if (!client.ip ~ purge) {<u></u><u></u></p></div><div><p class="MsoNormal"> error 405 "Not allowed.";<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p>
</div><div><p class="MsoNormal"> return(lookup);<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div>
<div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"> # Don't cache authenticated requests<u></u><u></u></p></div><div><p class="MsoNormal"> if (req.http.Cookie && req.http.Cookie ~ "__ac(|_(name|password|persistent))=") {<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><span> </span># Force lookup of specific urls unlikely to need protection<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>if (req.url ~ "\.(js|css)") {<u></u><u></u></p></div><div><p class="MsoNormal"> remove req.http.cookie;<u></u><u></u></p></div><div><p class="MsoNormal">
return(lookup);<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal"> return(pass);<u></u><u></u></p></div><div><p class="MsoNormal">
}<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"> # The default vcl_recv is used from here.<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">sub vcl_hit {<u></u><u></u></p></div><div><p class="MsoNormal"> # if (req.request == "PURGE") {<u></u><u></u></p></div><div>
<p class="MsoNormal"> # purge('');<u></u><u></u></p></div><div><p class="MsoNormal"> # error 200 "Purged";<u></u><u></u></p></div><div><p class="MsoNormal"> # }<u></u><u></u></p>
</div><div><p class="MsoNormal">}<u></u><u></u></p></div><div><p class="MsoNormal">sub vcl_miss {<u></u><u></u></p></div><div><p class="MsoNormal"> # if (req.request == "PURGE") {<u></u><u></u></p></div><div>
<p class="MsoNormal"> # purge('');<u></u><u></u></p></div><div><p class="MsoNormal"> # error 200 "Purged";<u></u><u></u></p></div><div><p class="MsoNormal"> # }<u></u><u></u></p>
</div><div><p class="MsoNormal">}<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"># Enforce a minimum TTL, since we can PURGE changed objects actively<u></u><u></u></p></div>
<div><p class="MsoNormal"># from Zope by using the CacheFu product<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">sub vcl_fetch {<u></u><u></u></p></div><div><p class="MsoNormal">
if (beresp.ttl < 3600s) {<u></u><u></u></p></div><div><p class="MsoNormal"> set beresp.ttl = 3600s;<u></u><u></u></p></div><div><p class="MsoNormal"> }<u></u><u></u></p></div><div><p class="MsoNormal">
}<u></u><u></u></p></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Varnish boots up script<u></u><u></u></p></div><div><p class="MsoNormal">
==========================================<u></u><u></u></p></div><div><div><p class="MsoNormal"><span> </span>varnishd -f /etc/varnish/my.vcl -s malloc,8192M -a $ip:80 \<u></u><u></u></p></div><div>
<p class="MsoNormal"><span> </span>-T $ip:2048 \<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-n vcache-my\<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>-p thread_pools=2 \<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-p thread_pool_max=15000\<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-p thread_pool_min=500\<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>-p listen_depth=2048 \<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-p lru_interval=1800 \<u></u><u></u></p></div>
<div><p class="MsoNormal"><span> </span>-h classic,169313 \<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-p connect_timeout=1800 \<u></u><u></u></p></div><div>
<p class="MsoNormal"><span> </span>-p http_max_hdr=8192\<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>-p http_resp_hdr_len=18192\<u></u><u></u></p></div><div>
<p class="MsoNormal"><span> </span>-p max_restarts=6 <u></u><u></u></p></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">I try to the backend status:<u></u><u></u></p>
</div><div><div><p class="MsoNormal">[root@hongkong varnish]# varnishadm -n vcache-my backend.list<u></u><u></u></p></div><div><p class="MsoNormal">==============================================<u></u><u></u></p></div><div>
<p class="MsoNormal">Backend name Refs Admin Probe<u></u><u></u></p></div><div><p class="MsoNormal">nanjing(10.80.125.66,,80) 68 probe Healthy 8/10<u></u><u></u></p></div><div><p class="MsoNormal">
hangzhou(10.80.125.68,,80) 66 probe Healthy 7/10<u></u><u></u></p></div><div><p class="MsoNormal">chongqing(10.80.125.76,,80) 23 probe Healthy 9/10<u></u><u></u></p></div></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">I already downgrade the .threshold from 8 to 2, so it can make sure the all the node is in Healthy status, if I set the .threshold to 8,<u></u><u></u></p>
</div><div><p class="MsoNormal">most of the node will be Sick.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">I try to use a script to wget the probe page every 2 seconds, there is no failure, but it is always have failure in the command 'backend.list', <u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">I have to script to watch the status of my website:<u></u><u></u></p></div><div><p class="MsoNormal">----------------------------------------------------------------------------------<u></u><u></u></p>
</div><div><div><p class="MsoNormal">#!/bin/bash<u></u><u></u></p></div><div><p class="MsoNormal">pass=0<u></u><u></u></p></div><div><p class="MsoNormal">fail=0<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p>
</div><div><p class="MsoNormal">while [ 1 ]<u></u><u></u></p></div><div><p class="MsoNormal">do<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>wget <a href="http://mysite/live.html" target="_blank">http://mysite/live.html</a> -O /dev/null<u></u><u></u></p>
</div><div><p class="MsoNormal"><span> </span>if [ $? -eq 0 ];then<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>pass=$(expr $pass + 1)<u></u><u></u></p></div><div><p class="MsoNormal">
<span> </span>else<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>fail=$(expr $fail + 1)<u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>fi<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal"><span> </span>echo -e "pass: $pass\n fail: $fail" <u></u><u></u></p></div><div><p class="MsoNormal"><span> </span>sleep 5<u></u><u></u></p>
</div><div><p class="MsoNormal">done<u></u><u></u></p></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">25% failed, it is very strange thing, I have no clue about it, <u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Example result about the varnish log:<u></u><u></u></p></div><div><p class="MsoNormal">=======================================<u></u><u></u></p>
</div><div><p class="MsoNormal">varnishlog -n vcache-my| tee -a /var/log/varnish.log<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><div><p class="MsoNormal"> 977 RxHeader c Connection: keep-alive<u></u><u></u></p>
</div><div><p class="MsoNormal"> 977 RxHeader c User-Agent: Mozilla/5.0 (iPad; CPU OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A523 Safari/8536.25<u></u><u></u></p></div><div><p class="MsoNormal">
977 VCL_call c recv pass<u></u><u></u></p></div><div><p class="MsoNormal"> 977 VCL_call c hash<u></u><u></u></p></div><div><p class="MsoNormal"> 977 Hash c /<u></u><u></u></p></div><div><p class="MsoNormal">
977 Hash c <a href="http://www.mywebsite.com" target="_blank">www.mywebsite.com</a><u></u><u></u></p></div><div><p class="MsoNormal"> 977 VCL_return c hash<u></u><u></u></p></div><div><p class="MsoNormal"> 977 VCL_call c pass pass<u></u><u></u></p>
</div><div><p class="MsoNormal"> 977 FetchError c no backend connection<u></u><u></u></p></div><div><p class="MsoNormal"> 977 VCL_call c error deliver<u></u><u></u></p></div><div><p class="MsoNormal"> 977 VCL_call c deliver deliver<u></u><u></u></p>
</div><div><p class="MsoNormal"> 977 TxProtocol c HTTP/1.1<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxStatus c 503<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxResponse c Service Unavailable<u></u><u></u></p>
</div><div><p class="MsoNormal"> 977 TxHeader c Server: Varnish<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxHeader c Content-Type: text/html; charset=utf-8<u></u><u></u></p></div><div><p class="MsoNormal">
977 TxHeader c Retry-After: 5<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxHeader c Content-Length: 419<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxHeader c Accept-Ranges: bytes<u></u><u></u></p>
</div><div><p class="MsoNormal"> 977 TxHeader c Date: Mon, 07 Jan 2013 18:03:02 GMT<u></u><u></u></p></div><div><p class="MsoNormal"> 977 TxHeader c X-Varnish: 2122413499<u></u><u></u></p></div></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal">more varnish log:<u></u><u></u></p></div><div><p class="MsoNormal">shaohui dot org/downloads/varnish.tgz<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p>
</div><div><p class="MsoNormal">this 503 error make my website trap in troubles, my customers can not access my site, I did not have any clue, can some body provide some advices, thanks so much. <u></u><u></u></p></div><div>
<p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">-- <br>Best regards<br>Shaohui <u></u><u></u></p></div></div></div></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Best regards<br>
Shaohui
</div></div>