<div dir="ltr">Hi there,<div><br></div><div>I've got a problem with varnishstat and was hoping to find an answer here.</div><div><br></div><div>Basically, today, my varnish server (3.0.2) started acting very weirdly. I suspect a misconfiguration on my part was at the origin of some weirdness (like pages randomly being flushed from the cache, despite the cache being half empty), but, our website was under low traffic (nothing Varnish can't handle), and somehow everything went crazy:</div>
<div><br></div><div>- varnishadm would tell me that perfectly valid commands like ban.list, vcl.list, vcl.load were wrong, returning a 105 (?) error. Preventing me from loading a configuration without doing a varnish reload/restart.</div>
<div>- varnishstat would report a highly improbable high number of "duplicate bans removed", but the rest of the reported numbers (hit, miss, etc.) seemed pretty normal.</div><div><br></div><div>The panicked admin that I am decided that, if I was to restart varnish, I should at least use the occasion to upgrade it to 3.0.4.; my dev server and personal machine having been running 3.0.4 for the past few month with no problems.</div>
<div><br></div><div>So I run apt-get update varnish, and get the new version.</div><div><br></div><div>Unfortunately, this made varnishstat even weirder. It now report 0 hits, 0 miss, 0 hit for pass. I also have seen negative numbers for n_wrk_*.</div>
<div><br></div><div>I tried restarting varnishd, made sure that varnishstat is 3.0.4, everything is fine, my website is mostly in cache, but still varnishstat reports what I believe are wrong numbers. </div><div><br></div>
<div>Has someone ever run into this problem?</div><div><br></div><div>I can provide my configuration file if needed.</div><div><br></div><div>Here's a varnishstat -1:</div><div><br></div><div><div><font face="courier new, monospace">client_conn 0 0.00 Client connections accepted</font></div>
<div><font face="courier new, monospace">client_drop 0 0.00 Connection dropped, no sess/wrk</font></div><div><font face="courier new, monospace">client_req 0 0.00 Client requests received</font></div>
<div><font face="courier new, monospace">cache_hit 0 0.00 Cache hits</font></div><div><font face="courier new, monospace">cache_hitpass 0 0.00 Cache hits for pass</font></div>
<div><font face="courier new, monospace">cache_miss 0 0.00 Cache misses</font></div><div><font face="courier new, monospace">backend_conn 0 0.00 Backend conn. success</font></div>
<div><font face="courier new, monospace">backend_unhealthy 0 0.00 Backend conn. not attempted</font></div><div><font face="courier new, monospace">backend_busy 271072 50.69 Backend conn. too many</font></div>
<div><font face="courier new, monospace">backend_fail 0 0.00 Backend conn. failures</font></div><div><font face="courier new, monospace">backend_reuse 560452 104.80 Backend conn. reuses</font></div>
<div><font face="courier new, monospace">backend_toolate 656588 122.77 Backend conn. was closed</font></div><div><font face="courier new, monospace">backend_recycle 0 0.00 Backend conn. recycles</font></div>
<div><font face="courier new, monospace">backend_retry 53208 9.95 Backend conn. retry</font></div><div><font face="courier new, monospace">fetch_head 9701 1.81 Fetch head</font></div>
<div><font face="courier new, monospace">fetch_length 0 0.00 Fetch with Length</font></div><div><font face="courier new, monospace">fetch_chunked 0 0.00 Fetch chunked</font></div>
<div><font face="courier new, monospace">fetch_eof 0 0.00 Fetch EOF</font></div><div><font face="courier new, monospace">fetch_bad 66843 12.50 Fetch had bad headers</font></div>
<div><font face="courier new, monospace">fetch_close 8116 1.52 Fetch wanted close</font></div><div><font face="courier new, monospace">fetch_oldhttp 74960 14.02 Fetch pre HTTP/1.1 closed</font></div>
<div><font face="courier new, monospace">fetch_zero 16 0.00 Fetch zero len</font></div><div><font face="courier new, monospace">fetch_failed 12 0.00 Fetch failed</font></div>
<div><font face="courier new, monospace">fetch_1xx 75267 14.07 Fetch no body (1xx)</font></div><div><font face="courier new, monospace">fetch_204 114 0.02 Fetch no body (204)</font></div>
<div><font face="courier new, monospace">fetch_304 0 0.00 Fetch no body (304)</font></div><div><font face="courier new, monospace">n_sess_mem 0 . N struct sess_mem</font></div>
<div><font face="courier new, monospace">n_sess 81 . N struct sess</font></div><div><font face="courier new, monospace">n_object 0 . N struct object</font></div>
<div><font face="courier new, monospace">n_vampireobject 0 . N unresurrected objects</font></div><div><font face="courier new, monospace">n_objectcore 0 . N struct objectcore</font></div>
<div><font face="courier new, monospace">n_objecthead 0 . N struct objecthead</font></div><div><font face="courier new, monospace">n_waitinglist 0 . N struct waitinglist</font></div>
<div><font face="courier new, monospace">n_vbc 772 . N struct vbc</font></div><div><font face="courier new, monospace">n_wrk 299 . N worker threads</font></div>
<div><font face="courier new, monospace">n_wrk_create 28 0.01 N worker threads created</font></div><div><font face="courier new, monospace">n_wrk_failed 14365 2.69 N worker threads not created</font></div>
<div><font face="courier new, monospace">n_wrk_max 0 0.00 N worker threads limited</font></div><div><font face="courier new, monospace">n_wrk_lqueue 14394 2.69 work request queue length</font></div>
<div><font face="courier new, monospace">n_wrk_queued 14644 2.74 N queued work requests</font></div><div><font face="courier new, monospace">n_wrk_drop 29 0.01 N dropped work requests</font></div>
<div><font face="courier new, monospace">n_backend 1 . N backends</font></div><div><font face="courier new, monospace">n_expired 29 . N expired objects</font></div>
<div><font face="courier new, monospace">n_lru_nuked 485 . N LRU nuked objects</font></div><div><font face="courier new, monospace">n_lru_moved 0 . N LRU moved objects</font></div>
<div><font face="courier new, monospace">losthdr 0 0.00 HTTP header overflows</font></div><div><font face="courier new, monospace">n_objsendfile 0 0.00 Objects sent with sendfile</font></div>
<div><font face="courier new, monospace">n_objwrite 2751 0.51 Objects sent with write</font></div><div><font face="courier new, monospace">n_objoverflow 0 0.00 Objects overflowing workspace</font></div>
<div><font face="courier new, monospace">s_sess 1 0.00 Total Sessions</font></div><div><font face="courier new, monospace">s_req 38843 7.26 Total Requests</font></div>
<div><font face="courier new, monospace">s_pipe 0 0.00 Total pipe</font></div><div><font face="courier new, monospace">s_pass 530752 99.24 Total pass</font></div><div>
<font face="courier new, monospace">s_fetch 64 0.01 Total fetch</font></div><div><font face="courier new, monospace">s_hdrbytes 0 0.00 Total header bytes</font></div>
<div>
<font face="courier new, monospace">s_bodybytes 689741 128.97 Total body bytes</font></div><div><font face="courier new, monospace">sess_closed 0 0.00 Session Closed</font></div>
<div><font face="courier new, monospace">sess_pipeline 271072 50.69 Session Pipeline</font></div><div><font face="courier new, monospace">sess_readahead 560452 104.80 Session Read Ahead</font></div>
<div><font face="courier new, monospace">sess_linger 255 0.05 Session Linger</font></div><div><font face="courier new, monospace">sess_herd 23065 4.31 Session herd</font></div>
<div><font face="courier new, monospace">shm_records 76246 14.26 SHM records</font></div><div><font face="courier new, monospace">shm_writes 203311427 38016.35 SHM writes</font></div><div>
<font face="courier new, monospace">shm_flushes 34338009106 6420719.73 SHM flushes due to overflow</font></div><div><font face="courier new, monospace">shm_cont 26666 4.99 SHM MTX contention</font></div>
<div><font face="courier new, monospace">shm_cycles 1234 0.23 SHM cycles through buffer</font></div><div><font face="courier new, monospace">sms_nreq 235 0.04 SMS allocator requests</font></div>
<div><font face="courier new, monospace">sms_nobj 541285 . SMS outstanding allocations</font></div><div><font face="courier new, monospace">sms_nbytes 566363 . SMS outstanding bytes</font></div>
<div><font face="courier new, monospace">sms_balloc 30696332 . SMS bytes allocated</font></div><div><font face="courier new, monospace">sms_bfree 2279693 . SMS bytes freed</font></div>
<div><font face="courier new, monospace">backend_req 0 0.00 Backend requests made</font></div><div><font face="courier new, monospace">n_vcl 2756 0.52 N vcl total</font></div>
<div><font face="courier new, monospace">n_vcl_avail 13 0.00 N vcl available</font></div><div><font face="courier new, monospace">n_vcl_discard 1447 0.27 N vcl discarded</font></div>
<div><font face="courier new, monospace">n_ban 0 . N total active bans</font></div><div><font face="courier new, monospace">n_ban_add 0 0.00 N new bans added</font></div>
<div><font face="courier new, monospace">n_ban_retire 1883994 352.28 N old bans deleted</font></div><div><font face="courier new, monospace">n_ban_obj_test 1883994 352.28 N objects tested</font></div>
<div><font face="courier new, monospace">n_ban_re_test 76246 14.26 N regexps tested against</font></div><div><font face="courier new, monospace">n_ban_dups 6 0.00 N duplicate bans removed</font></div>
<div><font face="courier new, monospace">hcb_nolock 6 0.00 HCB Lookups without lock</font></div><div><font face="courier new, monospace">hcb_lock 0 0.00 HCB Lookups with lock</font></div>
<div><font face="courier new, monospace">hcb_insert 11 0.00 HCB Inserts</font></div><div><font face="courier new, monospace">esi_errors 6 0.00 ESI parse errors (unlock)</font></div>
<div><font face="courier new, monospace">esi_warnings 6723 1.26 ESI parse warnings (unlock)</font></div><div><font face="courier new, monospace">accept_fail 6712 1.26 Accept failures</font></div>
<div><font face="courier new, monospace">client_drop_late 2054111 384.09 Connection dropped late</font></div><div><font face="courier new, monospace">uptime 23573960 4408.00 Client uptime</font></div>
<div><font face="courier new, monospace">dir_dns_lookups 5348 1.00 DNS director lookups</font></div><div><font face="courier new, monospace">dir_dns_failed 709796 132.72 DNS director failed lookups</font></div>
<div><font face="courier new, monospace">dir_dns_hit 50845 9.51 DNS director cached lookups hit</font></div><div><font face="courier new, monospace">dir_dns_cache_full 50844 9.51 DNS director full dnscache</font></div>
<div><font face="courier new, monospace">vmods 0 . Loaded VMODs</font></div><div><font face="courier new, monospace">n_gzip 0 0.00 Gzip operations</font></div>
<div><font face="courier new, monospace">n_gunzip 0 0.00 Gunzip operations</font></div><div><font face="courier new, monospace">LCK.sms.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.sms.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.sms.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.sms.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.smp.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.smp.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.smp.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.smp.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.sma.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.sma.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.sma.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.sma.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.smf.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.smf.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.smf.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.smf.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.hsl.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.hsl.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.hsl.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.hsl.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.hcb.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.hcb.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.hcb.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.hcb.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.hcl.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.hcl.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.hcl.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.hcl.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.vcl.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.vcl.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.vcl.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.vcl.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.stat.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.stat.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.stat.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.stat.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.sessmem.creat 0 0.00 Created locks</font></div><div>
<font face="courier new, monospace">LCK.sessmem.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.sessmem.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.sessmem.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.wstat.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.wstat.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.wstat.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.wstat.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.herder.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.herder.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.herder.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.herder.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.wq.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.wq.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.wq.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.wq.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.objhdr.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.objhdr.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.objhdr.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.objhdr.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.exp.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.exp.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.exp.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.exp.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.lru.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.lru.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.lru.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.lru.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.cli.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.cli.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.cli.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.cli.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.ban.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.ban.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.ban.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.ban.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.vbp.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.vbp.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.vbp.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.vbp.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.vbe.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.vbe.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.vbe.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.vbe.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">LCK.backend.creat 0 0.00 Created locks</font></div>
<div><font face="courier new, monospace">LCK.backend.destroy 0 0.00 Destroyed locks</font></div><div><font face="courier new, monospace">LCK.backend.locks 0 0.00 Lock Operations</font></div>
<div><font face="courier new, monospace">LCK.backend.colls 0 0.00 Collisions</font></div><div><font face="courier new, monospace">SMA.memory.c_req 0 0.00 Allocator requests</font></div>
<div><font face="courier new, monospace">SMA.memory.c_fail 0 0.00 Allocator failures</font></div><div><font face="courier new, monospace">SMA.memory.c_bytes 0 0.00 Bytes allocated</font></div>
<div><font face="courier new, monospace">SMA.memory.c_freed 0 0.00 Bytes freed</font></div><div><font face="courier new, monospace">SMA.memory.g_alloc 0 . Allocations outstanding</font></div>
<div><font face="courier new, monospace">SMA.memory.g_bytes 0 . Bytes outstanding</font></div><div><font face="courier new, monospace">SMA.memory.g_space 0 . Bytes available</font></div>
<div><font face="courier new, monospace">SMA.Transient.c_req 0 0.00 Allocator requests</font></div><div><font face="courier new, monospace">SMA.Transient.c_fail 0 0.00 Allocator failures</font></div>
<div><font face="courier new, monospace">SMA.Transient.c_bytes 0 0.00 Bytes allocated</font></div><div><font face="courier new, monospace">SMA.Transient.c_freed 0 0.00 Bytes freed</font></div>
<div><font face="courier new, monospace">SMA.Transient.g_alloc 0 . Allocations outstanding</font></div><div><font face="courier new, monospace">SMA.Transient.g_bytes 0 . Bytes outstanding</font></div>
<div><font face="courier new, monospace">SMA.Transient.g_space 0 . Bytes available</font></div><div><font face="courier new, monospace">VBE.default(127.0.0.1,,8080).vcls 0 . VCL references</font></div>
<div><font face="courier new, monospace">VBE.default(127.0.0.1,,8080).happy 0 . Happy health probes</font></div></div><div><br></div><div><br></div><div>Thanks for any help!</div><div><br></div><div>
-Hugues</div>
</div>