Sure no problem. I'll show a few more details below too. For a quick rundown of my application, I'm placing varnish in front of a big mapping system. We have about 22TB in mapping tiles, and I have 4 varnish machines sitting in front of them, with 6GB of memory each. A load balancer distributes the zoom levels between two clusters (two members each) in an attempt to balance my hit ratio and hit traffic. It works pretty well, the LRU seems to keep my hit ratio at around 60%. Which still results in very quick map loads for most views. I was previously using the <b>file</b> storage, but it just doesn't make sense to stick with that in my application. The backend NFS server is quicker than the VM images that varnish is running on, so I decided to try out the malloc storage and ran into this problem.<br>
<br><span style="font-family:courier new,monospace"></span><br>Here's a screenshot of top. This machine was set to malloc max at 5500MB.<br><img alt="Inline image 1" src="cid:ii_138a101a33fc257a"><br><br>Here's a cacti graph of me trying to lower the malloc size the last few days. Even set at 4000MB it climbs up to 6GB and crashes the box<br>
<br><img alt="Inline image 1" src="cid:ii_138a1139620375ae"><br><br>Here's the output of varnishstat -1 on a machine during the time when it started to swap out. (IP addresses removed)<br>---------------------------------------------------------------------------------------------------------------------------------------------------<br>
<span style="font-family:courier new,monospace"># varnishstat -1</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">client_conn 4516253 24.35 Client connections accepted</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">client_drop 0 0.00 Connection dropped, no sess/wrk</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">client_req 9665270 52.10 Client requests received</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">cache_hit 8866434 47.80 Cache hits</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">cache_hitpass 0 0.00 Cache hits for pass</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">cache_miss 798835 4.31 Cache misses</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">backend_conn 96001 0.52 Backend conn. success</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">backend_unhealthy 0 0.00 Backend conn. not attempted</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">backend_busy 0 0.00 Backend conn. too many</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">backend_fail 5 0.00 Backend conn. failures</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">backend_reuse 702842 3.79 Backend conn. reuses</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">backend_toolate 95484 0.51 Backend conn. was closed</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">backend_recycle 798339 4.30 Backend conn. recycles</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">backend_retry 7 0.00 Backend conn. retry</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_head 0 0.00 Fetch head</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_length 638890 3.44 Fetch with Length</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_chunked 159945 0.86 Fetch chunked</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_eof 0 0.00 Fetch EOF</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_bad 0 0.00 Fetch had bad headers</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_close 0 0.00 Fetch wanted close</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_zero 0 0.00 Fetch zero len</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_failed 0 0.00 Fetch failed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_1xx 0 0.00 Fetch no body (1xx)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">fetch_204 0 0.00 Fetch no body (204)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">fetch_304 0 0.00 Fetch no body (304)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_sess_mem 512 . N struct sess_mem</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_sess 318 . N struct sess</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_object 798209 . N struct object</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_vampireobject 0 . N unresurrected objects</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_objectcore 798256 . N struct objectcore</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_objecthead 798256 . N struct objecthead</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_waitinglist 2017 . N struct waitinglist</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_vbc 15 . N struct vbc</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_wrk 49 . N worker threads</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_wrk_create 1039 0.01 N worker threads created</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_wrk_failed 0 0.00 N worker threads not created</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_wrk_max 3421 0.02 N worker threads limited</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_wrk_lqueue 0 0.00 work request queue length</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_wrk_queued 20376 0.11 N queued work requests</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_wrk_drop 0 0.00 N dropped work requests</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_backend 8 . N backends</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_expired 626 . N expired objects</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_lru_nuked 0 . N LRU nuked objects</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_lru_moved 2011016 . N LRU moved objects</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">losthdr 0 0.00 HTTP header overflows</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_objsendfile 0 0.00 Objects sent with sendfile</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_objwrite 9650020 52.02 Objects sent with write</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_objoverflow 0 0.00 Objects overflowing workspace</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">s_sess 4516229 24.35 Total Sessions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">s_req 9665270 52.10 Total Requests</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">s_pipe 1 0.00 Total pipe</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">s_pass 0 0.00 Total pass</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">s_fetch 798835 4.31 Total fetch</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">s_hdrbytes 3810296042 20540.68 Total header bytes</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">s_bodybytes 67134175568 361909.30 Total body bytes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sess_closed 121965 0.66 Session Closed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sess_pipeline 0 0.00 Session Pipeline</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sess_readahead 0 0.00 Session Read Ahead</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sess_linger 9634396 51.94 Session Linger</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sess_herd 9535551 51.40 Session herd</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">shm_records 459505182 2477.12 SHM records</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">shm_writes 35192359 189.72 SHM writes</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">shm_flushes 0 0.00 SHM flushes due to overflow</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">shm_cont 6447 0.03 SHM MTX contention</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">shm_cycles 208 0.00 SHM cycles through buffer</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sms_nreq 0 0.00 SMS allocator requests</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sms_nobj 0 . SMS outstanding allocations</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sms_nbytes 0 . SMS outstanding bytes</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sms_balloc 0 . SMS bytes allocated</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sms_bfree 0 . SMS bytes freed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">backend_req 798844 4.31 Backend requests made</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_vcl 1 0.00 N vcl total</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_vcl_avail 1 0.00 N vcl available</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_vcl_discard 0 0.00 N vcl discarded</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_ban 1 . N total active bans</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_ban_add 1 0.00 N new bans added</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_ban_retire 0 0.00 N old bans deleted</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_ban_obj_test 0 0.00 N objects tested</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_ban_re_test 0 0.00 N regexps tested against</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_ban_dups 0 0.00 N duplicate bans removed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">hcb_nolock 0 0.00 HCB Lookups without lock</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">hcb_lock 0 0.00 HCB Lookups with lock</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">hcb_insert 0 0.00 HCB Inserts</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">esi_errors 0 0.00 ESI parse errors (unlock)</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">esi_warnings 0 0.00 ESI parse warnings (unlock)</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">accept_fail 0 0.00 Accept failures</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">client_drop_late 0 0.00 Connection dropped late</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">uptime 185500 1.00 Client uptime</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">dir_dns_lookups 0 0.00 DNS director lookups</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">dir_dns_failed 0 0.00 DNS director failed lookups</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">dir_dns_hit 0 0.00 DNS director cached lookups hit</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">dir_dns_cache_full 0 0.00 DNS director full dnscache</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">vmods 0 . Loaded VMODs</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">n_gzip 0 0.00 Gzip operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">n_gunzip 0 0.00 Gunzip operations</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sms.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sms.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sms.locks 0 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sms.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.smp.creat 0 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.smp.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.smp.locks 0 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.smp.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sma.creat 2 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sma.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sma.locks 1759151 9.48 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sma.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.smf.creat 0 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.smf.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.smf.locks 0 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.smf.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hsl.creat 0 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hsl.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hsl.locks 0 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hsl.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hcb.creat 0 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hcb.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hcb.locks 0 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hcb.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hcl.creat 70001 0.38 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hcl.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.hcl.locks 18532340 99.90 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.hcl.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vcl.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vcl.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vcl.locks 3999 0.02 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vcl.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.stat.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.stat.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.stat.locks 512 0.00 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.stat.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sessmem.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sessmem.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.sessmem.locks 4533824 24.44 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.sessmem.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.wstat.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.wstat.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.wstat.locks 575635 3.10 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.wstat.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.herder.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.herder.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.herder.locks 17392 0.09 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.herder.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.wq.creat 4 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.wq.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.wq.locks 20053031 108.10 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.wq.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.objhdr.creat 799708 4.31 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.objhdr.destroy 1454 0.01 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.objhdr.locks 20929870 112.83 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.objhdr.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.exp.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.exp.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.exp.locks 984222 5.31 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.exp.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.lru.creat 2 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.lru.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.lru.locks 798835 4.31 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.lru.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.cli.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.cli.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.cli.locks 61755 0.33 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.cli.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.ban.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.ban.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.ban.locks 984223 5.31 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.ban.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vbp.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vbp.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vbp.locks 623084 3.36 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vbp.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vbe.creat 1 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vbe.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.vbe.locks 191999 1.04 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.vbe.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.backend.creat 8 0.00 Created locks</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.backend.destroy 0 0.00 Destroyed locks</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">LCK.backend.locks 2779529 14.98 Lock Operations</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">LCK.backend.colls 0 0.00 Collisions</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.s0.c_req 1596704 8.61 Allocator requests</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.s0.c_fail 0 0.00 Allocator failures</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.s0.c_bytes 23077754159 124408.38 Bytes allocated</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.s0.c_freed 17482598971 94245.82 Bytes freed</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.s0.g_alloc 1596704 . Allocations outstanding</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.s0.g_bytes 5595155188 . Bytes outstanding</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.s0.g_space 172012812 . Bytes available</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.Transient.c_req 1252 0.01 Allocator requests</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.Transient.c_fail 0 0.00 Allocator failures</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.Transient.c_bytes 556514 3.00 Bytes allocated</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.Transient.c_freed 556514 3.00 Bytes freed</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.Transient.g_alloc 0 . Allocations outstanding</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">SMA.Transient.g_bytes 0 . Bytes outstanding</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">SMA.Transient.g_space 0 . Bytes available</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb11(,,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb11(,,80).happy 0 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb1(,,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb1(,,80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb2(,,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb2(,80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb3(,,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb3(,,80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb7(,,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb7(,80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb8(,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb8(80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb9(,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb9(80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">VBE.mapweb10(,80).vcls 1 . VCL references</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">VBE.mapweb10(,80).happy 18446744073709551615 . Happy health probes</span><br style="font-family:courier new,monospace">---------------------------------------------------------------------------------------------------------------------------------------------------<br>
<br><br>mapping system VCL config. I've snipped the unimportant stuff, and changed my names around a bit<br>---------------------------------------------------------------------------------------------------------------------------------------------------<br>
<span style="font-family:courier new,monospace">backend mapweb11 {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">.host = "...";</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">.port = "80";</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">.probe = {</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> .url = "/ka-map/images/a_pixel.gif";</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> .interval = 5s;</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> .timeout = 500 ms;</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> .window = 5;</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> .threshold = 3;</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">}</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"># ... More backends defined here ...</span><br style="font-family:courier new,monospace">
<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">director maps round-robin</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">{</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> { .backend = mapweb1; }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> { .backend = mapweb2; }</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> { .backend = mapweb3; }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> { .backend = mapweb7; }</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> { .backend = mapweb8; }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> { .backend = mapweb9; }</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> { .backend = mapweb10; }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">}</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sub vcl_recv {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set req.backend = maps;</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> if (req.request != "GET" &&</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> req.request != "HEAD" &&</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> req.request != "PUT" &&</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> req.request != "POST" &&</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> req.request != "TRACE" &&</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> req.request != "OPTIONS" &&</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> req.request != "DELETE") {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* Non-RFC2616 or CONNECT which is weird. */</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> return (pipe);</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace">
<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> if (req.request != "GET" && req.request != "HEAD") {</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> /* We only deal with GET and HEAD by default */</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> return (pass);</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> # Don't cache any shapegen calls, they're dynamic</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> if ( req.url ~ "^/cgi-bin/shapegen" )</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> {</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> return(pass);</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> # Change anything *.<a href="http://mydomain.com">mydomain.com</a> to just <a href="http://mydomain.com">mydomain.com</a></span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> # to avoid duplications in the cache</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> if ( req.http.host ~ "<a href="http://mydomain.com">mydomain.com</a>" )</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set req.http.host = "<a href="http://mydomain.com">mydomain.com</a>";</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> # for mapping grace period</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> set req.grace = 30s;</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> return (lookup);</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">}</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">sub vcl_fetch {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> if (beresp.ttl <= 0s ||</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> beresp.http.Set-Cookie ||</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> beresp.http.Vary == "*")</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> return(hit_for_pass);</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* Remove Expires from backend, it's not long enough */</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> unset beresp.http.expires;</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* Set the clients TTL on this object */</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> set beresp.http.Cache-Control = "max-age=604800";</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* Set how long Varnish will keep it */</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> set beresp.ttl = 1w;</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* marker for vcl_deliver to reset Age: */</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> set beresp.http.magicmarker = "1";</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set beresp.grace = 30s;</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> return (deliver);</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">}</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace">sub vcl_deliver {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> # Added by Cal to easily see a HIT / MISS in the headers</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> if (obj.hits > 0) {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set resp.http.X-Cache = "HIT";</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> } else {</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set resp.http.X-Cache = "MISS";</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> if (resp.http.magicmarker) {</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> /* Remove the magic marker */</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> unset resp.http.magicmarker;</span><br style="font-family:courier new,monospace">
<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> /* By definition we have a fresh object */</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> set resp.http.age = "0";</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> }</span><br style="font-family:courier new,monospace"><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> # Set a nice header to show which varnish server this comes from</span><br style="font-family:courier new,monospace">
<span style="font-family:courier new,monospace"> set resp.http.X-Hostname = regsub(server.identity, ".<a href="http://mydomain.com">mydomain.com</a>", "");</span><br style="font-family:courier new,monospace">
<br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace"> return (deliver);</span><br style="font-family:courier new,monospace"><span style="font-family:courier new,monospace">}</span><br>
---------------------------------------------------------------------------------------------------------------------------------------------------<br><br clear="all">Thanks for any help!<br><br>--Cal<br><br><br><br><div class="gmail_quote">
On Thu, Jul 19, 2012 at 2:20 PM, Andreas Plesner Jacobsen <span dir="ltr"><<a href="mailto:apj@mutt.dk" target="_blank">apj@mutt.dk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Thu, Jul 19, 2012 at 02:08:20PM -0500, Cal Heldenbrand wrote:<br>
><br>
> I just upgraded to Varnish 3.0.2, and changed the storage method to malloc,<br>
> with a size limitation of 5GB on a machine with 6GB of physical memory.<br>
> Varnish doesn't appear to be using the size limitation, and is consuming<br>
> memory until the machine dips into swap and becomes unresponsive.<br>
<br>
</div>Please attach a varnishstat. A copy of your vcl will probably be helpful too.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Andreas<br>
<br>
_______________________________________________<br>
varnish-dev mailing list<br>
<a href="mailto:varnish-dev@varnish-cache.org">varnish-dev@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev</a><br>
</font></span></blockquote></div><br>