<font size="2"><font face="georgia,serif">This is expected behavior with "-sfile" -- Varnish will use all available RAM to cache its much larger on-disk cache component. If you can get away with memory-only caching, you can use "-smalloc" to utilize a specific amount of RAM.</font></font><div>
<font size="2"><font face="georgia,serif"><br></font></font></div><div><font size="2"><font face="georgia,serif">If you do want to use "-sfile" but are having issue with swap or memory allocation, increasing /proc/sys/vm/min_free_kbytes (or vm.min_free_kbytes via sysctl) to something like 131072 will stabilize performance /in my experience/.</font></font></div>
<div><font size="2"><font face="georgia,serif"><br></font></font></div><div><font size="2"><font face="georgia,serif">Hope it helps,<br clear="all"></font></font><font face="georgia, serif">-- </font><div><font face="georgia, serif">kb</font></div>
<br>
<br><br><div class="gmail_quote">On Thu, Apr 7, 2011 at 03:38, Jean-Francois Laurens <span dir="ltr"><<a href="mailto:jean-francois.laurens@rts.ch">jean-francois.laurens@rts.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div>
<font size="5"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11.5pt">Hi there !<br>
<br>
I recently updated varnish from version 2.0.6 to 2.1.5 on a centos 5.4 server.<br>
<br>
After pretty much a day running 2.1.5, the system went out of memory. The server has 8G of memory and we specified a file of 50Gb as varnish cache file.<br>
<br>
I changed some options yesterday following Kristian Lyngstol recommendations ( <font color="#134FAE"><u><a href="http://kristianlyng.wordpress.com/2009/10/19/high-end-varnish-tuning/)" target="_blank">http://kristianlyng.wordpress.com/2009/10/19/high-end-varnish-tuning/)</a></u></font> and did a restart so the system is now running.<br>
<br>
I decreased the size of the cache file to 40Gb to.<br>
<br>
Here is a top output where you can see that varnish is consuming pretty much all the memory available:<br>
top -cbn 1<br>
top - 11:54:24 up 625 days, 19:51, 1 user, load average: 0.52, 0.41, 0.36<br>
Tasks: 110 total, 1 running, 109 sleeping, 0 stopped, 0 zombie<br>
Cpu(s): 0.1%us, 0.2%sy, 0.0%ni, 99.1%id, 0.5%wa, 0.0%hi, 0.1%si, 0.0%st<br>
Mem: 8177764k total, 8132712k used, 45052k free, 13100k buffers<br>
Swap: 4096564k total, 263324k used, 3833240k free, 7337568k cached<br>
<br>
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br>
3733 varnish 15 0 44.6g 7.1g 6.7g S 2.0 91.4 9:56.07 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/default.vcl -T <a href="http://0.0.0.0:6082" target="_blank">0.0.0.0:6082</a> -t 120 -w 100,3000,120 -u varnish -g varn<br>
....<br>
3732 root 15 0 106m 720 492 S 0.0 0.0 0:00.72 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/default.vcl -T <a href="http://0.0.0.0:6082" target="_blank">0.0.0.0:6082</a> -t 120 -w 100,3000,120 -u varnish -g varn<br>
<br>
Regularly it’s consuming more and more swap aswell:<br>
>From 15mn ago:<br>
free -m<br>
total used free shared buffers cached<br>
Mem: 7986 7941 44 0 12 7163<br>
-/+ buffers/cache: 765 7220<br>
Swap: 4000 258 3742<br>
<br>
>From 5 mn ago:<br>
free -m<br>
total used free shared buffers cached<br>
Mem: 7986 7940 45 0 15 7163<br>
-/+ buffers/cache: 761 7224<br>
Swap: 4000 260 3739<br>
<br>
>From 1mn ago:<br>
free -m<br>
total used free shared buffers cached<br>
Mem: 7986 7940 45 0 17 7160<br>
-/+ buffers/cache: 763 7222<br>
Swap: 4000 262 3737<br>
<br>
I understood that varnish relies on the system itself for memory allocation.<br>
Could we say that the bigger you file cache size is the more varnish need memory ?<br>
Or Am I facing an issue with my vcl that could have this side effect or ??<br>
<br>
In addition to this I've seen the percentage hit decreasing from 65% average with 2.0.6 to 40% with 2.1.5.<br>
Apart from changing obj to beresp in the vcl_fetch, I had to specify return(deliver) or return(pass) instead of only deliver or pass.<br>
Would it be a reason for this ?<br>
Would you be able to give me pointers for me to dig a bit more ?<br>
<br>
I Attached a plot from cacti wich would show the difference between 2.0.6 and 2.1.5 (the gap is due to the time to update the server, the trafic was sent to an other server ...)<br>
</span></font></font><font face="Helvetica, Verdana, Arial"><span style="font-size:9pt"><br>
</span></font><font size="5"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11.5pt"><br>
Thanks in advance for your advices ! <br>
</span></font></font><font face="Calibri, Verdana, Helvetica, Arial"><font size="4"><span style="font-size:11pt"><br>
</span></font></font><font color="#808080"><font face="Arial"><span style="font-size:9pt">Jean-Francois Laurens<br>
</span><font size="4"><span style="font-size:10pt">Ingénieur Système Unix<br>
Resources et Développement<br>
Secteur Backend<br>
</span></font><span style="font-size:9pt"><b>RTS - Radio Télévision Suisse<br>
</b>Quai Ernest-Ansermet 20 <br>
Case postale 234 <br>
CH - 1211 Genève 8<br>
T <a href="tel:%2B41%20%280%2958%20236%2081%2063" value="+41582368163" target="_blank">+41 (0)58 236 81 63</a><br>
</span></font></font><font size="4"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:10pt"><br>
</span></font></font>
</div>
<br>_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org">varnish-misc@varnish-cache.org</a><br>
<a href="http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br></blockquote></div><br></div>