Varnishstat doesn't list any nuked objects and file storage and shmlog look like they have plenty of space:<br><br>df -h<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Filesystem Size Used Avail Use% Mounted on<br>tmpfs 150M 81M 70M 54% /usr/local/var/varnish<br>
/dev/sdc1 74G 11G 61G 16% /var/lib/varnish<br><br>top<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>top - 12:26:33 up 164 days, 22:21, 1 user, load average: 2.60, 3.26, 3.75<br>Tasks: 67 total, 1 running, 66 sleeping, 0 stopped, 0 zombie<br>
Cpu(s): 0.7%us, 0.3%sy, 0.0%ni, 97.0%id, 0.7%wa, 0.3%hi, 1.0%si, 0.0%st<br>Mem: 8183492k total, 7763100k used, 420392k free, 13424k buffers<br>Swap: 3148720k total, 56636k used, 3092084k free, 7317692k cached<br>
<br> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND<br> 7441 varnish 15 0 70.0g 6.4g 6.1g S 2 82.5 56:33.31 varnishd<br><br><br>Varnishstat:<br>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>Hitrate ratio: 8 8 8<br>
Hitrate avg: 0.9782 0.9782 0.9782<br><br> 36494404 219.98 160.57 Client connections accepted<br> 36494486 220.98 160.57 Client requests received<br> 35028477 212.98 154.12 Cache hits<br>
474091 4.00 2.09 Cache hits for pass<br> 988013 6.00 4.35 Cache misses<br> 1465955 10.00 6.45 Backend connections success<br> 9 0.00 0.00 Backend connections failures<br>
994 . . N struct sess_mem<br> 11 . . N struct sess<br> 274047 . . N struct object<br> 252063 . . N struct objecthead<br>
609018 . . N struct smf<br> 28720 . . N small free smf<br> 2 . . N large free smf<br> 2 . . N struct vbe_conn<br>
901 . . N struct bereq<br> 2000 . . N worker threads<br> 2000 0.00 0.01 N worker threads created<br> 143 0.00 0.00 N overflowed work requests<br>
1 . . N backends<br> 672670 . . N expired objects<br> 3514467 . . N LRU moved objects<br> 49 0.00 0.00 HTTP header overflows<br>
32124238 206.98 141.34 Objects sent with write<br> 36494396 224.98 160.57 Total Sessions<br> 36494484 224.98 160.57 Total Requests<br> 783 0.00 0.00 Total pipe<br>
518770 4.00 2.28 Total pass<br> 1464570 10.00 6.44 Total fetch<br> 14559014884 93563.69 64058.18 Total header bytes<br>168823109304 489874.04 742804.45 Total body bytes<br>
36494387 224.98 160.57 Session Closed<br> 203 0.00 0.00 Session herd<br> 1736767745 10880.80 7641.60 SHM records<br> 148079555 908.90 651.53 SHM writes<br> 15088 0.00 0.07 SHM flushes due to overflow<br>
10494 0.00 0.05 SHM MTX contention<br> 687 0.00 0.00 SHM cycles through buffer<br> 2988576 21.00 13.15 allocator requests<br> 580296 . . outstanding allocations<br>
8916353024 . . bytes allocated<br> 44770738176 . . bytes free<br> 656 0.00 0.00 SMS allocator requests<br> 303864 . . SMS bytes allocated<br>
303864 . . SMS bytes freed<br> 1465172 10.00 6.45 Backend requests made<br><br><br><br><div class="gmail_quote">On Thu, Apr 9, 2009 at 12:18 PM, Artur Bergman <span dir="ltr"><<a href="mailto:sky@crucially.net">sky@crucially.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style=""><div>For the file storage or for the shmlog?</div><div><br></div><div>When do you start nuking/expiring from disk? I suspect the load goes up when you run out of storage space?</div>
<div><br></div><div>Cheers</div><div>Artur</div><div><br></div><div><br><div><div><div></div><div class="h5"><div>On Apr 9, 2009, at 12:02 PM, Cloude Porteus wrote:</div><br></div></div><blockquote type="cite"><div><div></div>
<div class="h5">Has anyone experienced very high server load averages? We're running varnish on a dual core with 8gb of ram. It runs okay for a day or two and then I start seeing load averages in 6-10 range for an hour or so, drops down to 2-3, then goes back up.<br>
<br>This starts to happen once we have more items in the cache than our physical memory. Maybe increasing our lru_interval will help? It's currently set to 3600.<br><br>Right now we're running with a 50gb file storage option. There are 270k objects in the cache, 70gb virtual memory, 6.2gb of res memory used, 11gb of data on disk in the file storage. We have a 98% hit ratio.<br>
<br>We followed Artur's advice about setting a tmpfs and creating an ext2 partition for our file storage.<br><br>I also tried running with malloc as our storage type, but I had to set it at a little less than half of our physical ram in order for it to work well after the cache got full. I don't understand why the virtual memory is double when I am running in malloc mode. I was running it with 5gb and the virtual memory was about 10-12gb and once it got full it started using the swap memory.<br>
<br>Thanks for any help/insight.<br><br>best,<br>cloude<br>-- <br>VP of Product Development<br>Instructables.com<br><br><a href="http://www.instructables.com/member/lebowski" target="_blank">http://www.instructables.com/member/lebowski</a><br>
</div></div> _______________________________________________<br>varnish-dev mailing list<br><a href="mailto:varnish-dev@projects.linpro.no" target="_blank">varnish-dev@projects.linpro.no</a><br><a href="http://projects.linpro.no/mailman/listinfo/varnish-dev" target="_blank">http://projects.linpro.no/mailman/listinfo/varnish-dev</a><br>
</blockquote></div><br></div></div></blockquote></div><br><br clear="all"><br>-- <br>VP of Product Development<br>Instructables.com<br><br><a href="http://www.instructables.com/member/lebowski">http://www.instructables.com/member/lebowski</a><br>