<div dir="ltr">Hi, <br><br>I have some questions regarding Varnish 4 and some problems we have with it's memory usage. <br><br>We are using Varnish 4 on a virtual server (CentOS 6.5) with 14 GB memory and 9 GB malloc. The TTL in use is 5m and the grace time is 60m. Varnish sits in front of an API that is responding with JSON only.<br>
<br>On the API servers we are using Nginx, and according to the body_bytes_sent variable in Nginx we can see that the average response size is 4.4 Kbyte and in 1 hour the API is sending around 300 Mbytes of data. Nginx is also compressing the response before sending it to Varnish. The number of requests to the API servers in 1 hour is between 60 000 and 70 000.<br>
<br>The problem we see is that when we reach around 73 000 objects in Varnish all the memory (9G) is used and the lru_nuked counter starts to increase. When Varnish is restarted most of the memory is used after 1 to 1.5 hour. As mentioned before the TTL is 5 minutes and the grace time is 60 minutes. I am really confused why we are seeing this behaviour, especially when the API servers is only sending around 300 Mbytes of data in 1 hour time. <br>
<br>I hope somebody can shed some light into this, thanks!<br><br><br>## Backend request ##<br><br>* << BeReq >> 808484921 <br>- Begin bereq 808484920 fetch <br>- Timestamp Start: 1403174559.768064 0.000000 0.000000<br>
- BereqMethod GET<br>- BereqURL request_url<br>- BereqProtocol HTTP/1.1<br>- BereqHeader Content-Type: application/hal+json<br>- BereqHeader Accept: application/hal+json<br>- BereqHeader Cneonction: close<br>
- BereqHeader Host: backend-servers:8080<br>- BereqHeader X-Forwarded-For: x.x.x.x<br>- BereqHeader Accept-Encoding: gzip<br>- BereqHeader X-Varnish: 808484921<br>- VCL_call BACKEND_FETCH<br>- VCL_return fetch<br>
- Backend 46 backend backend3(x.x.x.x,,8080)<br>- Timestamp Bereq: 1403174559.768100 0.000036 0.000036<br>- Timestamp Beresp: 1403174560.085275 0.317211 0.317176<br>- BerespProtocol HTTP/1.1<br>- BerespStatus 200<br>
- BerespResponse OK<br>- BerespHeader Server: nginx/1.6.0<br>- BerespHeader Date: Thu, 19 Jun 2014 10:42:46 GMT<br>- BerespHeader Content-Type: application/hal+json; charset=utf-8<br>- BerespHeader Transfer-Encoding: chunked<br>
- BerespHeader Connection: keep-alive<br>- BerespHeader Vary: Accept-Encoding<br>- BerespHeader Status: 200 OK<br>- BerespHeader X-Frame-Options: SAMEORIGIN<br>- BerespHeader X-XSS-Protection: 1; mode=block<br>
- BerespHeader X-Content-Type-Options: nosniff<br>- BerespHeader X-UA-Compatible: chrome=1<br>- BerespHeader Cache-Control: max-age=0, private, must-revalidate<br>- BerespHeader X-Runtime: 0.182126<br>- BerespHeader Content-Encoding: gzip<br>
- TTL RFC 0 -1 -1 1403174560 1403174560 1403174566 0 0<br>- VCL_call BACKEND_RESPONSE<br>- TTL VCL 300 10 0 1403174560<br>- TTL VCL 300 3600 0 1403174560<br>- VCL_return deliver<br>
- Storage malloc s0<br>- ObjProtocol HTTP/1.1<br>- ObjStatus 200<br>- ObjResponse OK<br>- ObjHeader Server: nginx/1.6.0<br>- ObjHeader Date: Thu, 19 Jun 2014 10:42:46 GMT<br>- ObjHeader Content-Type: application/hal+json; charset=utf-8<br>
- ObjHeader Vary: Accept-Encoding<br>- ObjHeader Status: 200 OK<br>- ObjHeader X-Frame-Options: SAMEORIGIN<br>- ObjHeader X-XSS-Protection: 1; mode=block<br>- ObjHeader X-Content-Type-Options: nosniff<br>
- ObjHeader X-UA-Compatible: chrome=1<br>- ObjHeader Cache-Control: max-age=0, private, must-revalidate<br>- ObjHeader X-Runtime: 0.182126<br>- ObjHeader Content-Encoding: gzip<br>- Fetch_Body 2 chunked stream<br>
- Gzip u F - 5876 21006 80 80 46942<br>- Timestamp BerespBody: 1403174560.085712 0.317648 0.000436<br>- BackendReuse 46 backend3(x.x.x.x,,8080)<br>- Length 5876<br>- BereqAcct 471 0 471 657 5888 6545<br>
- End<br><br><br>## default_vcl ##<br><br># This is a Varnish 4.x VCL file<br>vcl 4.0;<br>import directors;<br>import std;<br><br>backend backend1 {<br> .host = "x.x.x.x";<br> .port = "8080";<br>
.probe = {<br> .url = "/ping";<br> .timeout = 4s;<br> .interval = 5s;<br> .window = 5;<br> .threshold = 4;<br> }<br>}<br>backend backend2 {<br> .host = "x.x.x.x";<br>
.port = "8080";<br> .probe = {<br> .url = "/ping";<br> .timeout = 4s;<br> .interval = 5s;<br> .window = 5;<br> .threshold = 4;<br> }<br>}<br>backend backend3 {<br>
.host = "x.x.x.x";<br> .port = "8080";<br> .probe = {<br> .url = "/ping";<br> .timeout = 4s;<br> .interval = 5s;<br> .window = 5;<br> .threshold = 4;<br>
}<br>}<br><br># Below is an example redirector based on round-robin requests<br>sub vcl_init {<br> new sservice = directors.round_robin();<br> sservice.add_backend(backend1);<br> sservice.add_backend(backend2);<br> sservice.add_backend(backend3);<br>
}<br><br>acl local<br>{<br> "x.x.x.x"/24;<br>}<br><br># Handle the HTTP request received by the client<br>sub vcl_recv {<br> set req.backend_hint = sservice.backend();<br><br> if (req.url ~ "^/ping$")<br>
{<br> if (client.ip ~ local)<br> {<br> return (synth(900, "pong"));<br> }<br> }<br><br> if (! req.http.Authorization ~ "apitokens")<br> {<br> return (synth(401, "Restricted"));<br>
}<br><br> return (hash);<br>}<br><br>sub vcl_backend_response {<br> set beresp.ttl = 5m;<br> set beresp.grace = 60m;<br> return (deliver);<br>}<br><br>sub vcl_deliver {<br> if (obj.hits > 0) {<br> set resp.http.X-Cache = "HIT";<br>
} else {<br> set resp.http.X-Cache = "MISS";<br> }<br> return (deliver);<br>}<br><br>sub vcl_synth {<br> if (resp.status == 900)<br> {<br> set resp.status = 200;<br> set resp.http.Content-Type = "text/plain; charset=utf-8";<br>
synthetic("pong");<br> return (deliver);<br> }<br><br> set resp.http.Content-Type = "application/hal+json";<br> return (deliver);<br>}<br><br>sub vcl_backend_error {<br> set beresp.http.Content-Type = "application/hal+json";<br>
return (deliver);<br>}<br><br><br>## varnishstat -1 ##<br><br>MAIN.uptime 236760 1.00 Child process uptime<br>MAIN.sess_conn 217727 0.92 Sessions accepted<br>MAIN.sess_drop 0 0.00 Sessions dropped<br>
MAIN.sess_fail 0 0.00 Session accept failures<br>MAIN.sess_pipe_overflow 0 0.00 Session pipe overflow<br>MAIN.client_req_400 0 0.00 Client requests received, subject to 400 errors<br>
MAIN.client_req_411 0 0.00 Client requests received, subject to 411 errors<br>MAIN.client_req_413 0 0.00 Client requests received, subject to 413 errors<br>MAIN.client_req_417 0 0.00 Client requests received, subject to 417 errors<br>
MAIN.client_req 5316027 22.45 Good Client requests received<br>MAIN.cache_hit 2061461 8.71 Cache hits<br>MAIN.cache_hitpass 0 0.00 Cache hits for pass<br>MAIN.cache_miss 3204801 13.54 Cache misses<br>
MAIN.backend_conn 107148 0.45 Backend conn. success<br>MAIN.backend_unhealthy 0 0.00 Backend conn. not attempted<br>MAIN.backend_busy 0 0.00 Backend conn. too many<br>
MAIN.backend_fail 0 0.00 Backend conn. failures<br>MAIN.backend_reuse 4253086 17.96 Backend conn. reuses<br>MAIN.backend_toolate 82192 0.35 Backend conn. was closed<br>
MAIN.backend_recycle 4335308 18.31 Backend conn. recycles<br>MAIN.backend_retry 0 0.00 Backend conn. retry<br>MAIN.fetch_head 0 0.00 Fetch no body (HEAD)<br>
MAIN.fetch_length 305 0.00 Fetch with Length<br>MAIN.fetch_chunked 4359871 18.41 Fetch chunked<br>MAIN.fetch_eof 0 0.00 Fetch EOF<br>MAIN.fetch_bad 0 0.00 Fetch bad T-E<br>
MAIN.fetch_close 0 0.00 Fetch wanted close<br>MAIN.fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed<br>MAIN.fetch_zero 0 0.00 Fetch zero len body<br>
MAIN.fetch_1xx 0 0.00 Fetch no body (1xx)<br>MAIN.fetch_204 0 0.00 Fetch no body (204)<br>MAIN.fetch_304 0 0.00 Fetch no body (304)<br>MAIN.fetch_failed 4994 0.02 Fetch body failed<br>
MAIN.pools 2 . Number of thread pools<br>MAIN.threads 125 . Total number of threads<br>MAIN.threads_limited 0 0.00 Threads hit max<br>
MAIN.threads_created 24761 0.10 Threads created<br>MAIN.threads_destroyed 24636 0.10 Threads destoryed<br>MAIN.threads_failed 0 0.00 Thread creation failed<br>MAIN.thread_queue_len 0 . Length of session queue<br>
MAIN.busy_sleep 44391 0.19 Number of requests sent to sleep on busy objhdr<br>MAIN.busy_wakeup 44390 0.19 Number of requests woken after sleep on busy objhdr<br>MAIN.sess_queued 22381 0.09 Sessions queued for thread<br>
MAIN.sess_dropped 0 0.00 Sessions dropped for thread<br>MAIN.n_object 72729 . N struct object<br>MAIN.n_vampireobject 0 . N unresurrected objects<br>
MAIN.n_objectcore 79071 . N struct objectcore<br>MAIN.n_objecthead 69256 . N struct objecthead<br>MAIN.n_waitinglist 21819 . N struct waitinglist<br>MAIN.n_backend 3 . N backends<br>
MAIN.n_expired 2977375 . N expired objects<br>MAIN.n_lru_nuked 1336131 . N LRU nuked objects<br>MAIN.n_lru_moved 1928525 . N LRU moved objects<br>MAIN.losthdr 0 0.00 HTTP header overflows<br>
MAIN.s_sess 217727 0.92 Total Sessions<br>MAIN.s_req 5316027 22.45 Total Requests<br>MAIN.s_pipe 0 0.00 Total pipe<br>MAIN.s_pass 0 0.00 Total pass<br>
MAIN.s_fetch 3204801 13.54 Total fetch<br>MAIN.s_synth 53251 0.22 Total synth<br>MAIN.s_req_hdrbytes 2351626120 9932.53 Request header bytes<br>MAIN.s_req_bodybytes 0 0.00 Request body bytes<br>
MAIN.s_resp_hdrbytes 3914065512 16531.79 Response header bytes<br>MAIN.s_resp_bodybytes 78724096530 332505.90 Reponse body bytes<br>MAIN.s_pipe_hdrbytes 0 0.00 Pipe request header bytes<br>
MAIN.s_pipe_in 0 0.00 Piped bytes from client<br>MAIN.s_pipe_out 0 0.00 Piped bytes to client<br>MAIN.sess_closed 47598 0.20 Session Closed<br>MAIN.sess_pipeline 0 0.00 Session Pipeline<br>
MAIN.sess_readahead 0 0.00 Session Read Ahead<br>MAIN.sess_herd 2769201 11.70 Session herd<br>MAIN.shm_records 666122457 2813.49 SHM records<br>MAIN.shm_writes 27933605 117.98 SHM writes<br>
MAIN.shm_flushes 0 0.00 SHM flushes due to overflow<br>MAIN.shm_cont 1780 0.01 SHM MTX contention<br>MAIN.shm_cycles 315 0.00 SHM cycles through buffer<br>
MAIN.sms_nreq 0 0.00 SMS allocator requests<br>MAIN.sms_nobj 0 . SMS outstanding allocations<br>MAIN.sms_nbytes 0 . SMS outstanding bytes<br>
MAIN.sms_balloc 0 . SMS bytes allocated<br>MAIN.sms_bfree 0 . SMS bytes freed<br>MAIN.backend_req 4360234 18.42 Backend requests made<br>MAIN.n_vcl 4 0.00 N vcl total<br>
MAIN.n_vcl_avail 4 0.00 N vcl available<br>MAIN.n_vcl_discard 0 0.00 N vcl discarded<br>MAIN.bans 1 . Count of bans<br>MAIN.bans_completed 1 . Number of bans marked 'completed'<br>
MAIN.bans_obj 0 . Number of bans using obj.*<br>MAIN.bans_req 0 . Number of bans using req.*<br>MAIN.bans_added 1 0.00 Bans added<br>
MAIN.bans_deleted 0 0.00 Bans deleted<br>MAIN.bans_tested 0 0.00 Bans tested against objects (lookup)<br>MAIN.bans_obj_killed 0 0.00 Objects killed by bans (lookup)<br>
MAIN.bans_lurker_tested 0 0.00 Bans tested against objects (lurker)<br>MAIN.bans_tests_tested 0 0.00 Ban tests tested against objects (lookup)<br>MAIN.bans_lurker_tests_tested 0 0.00 Ban tests tested against objects (lurker)<br>
MAIN.bans_lurker_obj_killed 0 0.00 Objects killed by bans (lurker)<br>MAIN.bans_dups 0 0.00 Bans superseded by other bans<br>MAIN.bans_lurker_contention 0 0.00 Lurker gave way for lookup<br>
MAIN.bans_persisted_bytes 13 . Bytes used by the persisted ban lists<br>MAIN.bans_persisted_fragmentation 0 . Extra bytes in persisted ban lists due to fragmentation<br>MAIN.exp_mailed 5722364 24.17 Number of objects mailed to expiry thread<br>
MAIN.exp_received 5722364 24.17 Number of objects received by expiry thread<br>MAIN.hcb_nolock 5266297 22.24 HCB Lookups without lock<br>MAIN.hcb_lock 3204827 13.54 HCB Lookups with lock<br>
MAIN.hcb_insert 3204827 13.54 HCB Inserts<br>MAIN.esi_errors 0 0.00 ESI parse errors (unlock)<br>MAIN.esi_warnings 0 0.00 ESI parse warnings (unlock)<br>
MAIN.vmods 2 . Loaded VMODs<br>MAIN.n_gzip 4083400 17.25 Gzip operations<br>MAIN.n_gunzip 5065871 21.40 Gunzip operations<br>
MAIN.vsm_free 972096 . Free VSM space<br>MAIN.vsm_used 83962512 . Used VSM space<br>MAIN.vsm_cooling 0 . Cooling VSM space<br>
MAIN.vsm_overflow 0 . Overflow VSM space<br>MAIN.vsm_overflowed 0 0.00 Overflowed VSM space<br>MGT.uptime 307850 1.30 Management process uptime<br>
MGT.child_start 2 0.00 Child process started<br>MGT.child_exit 0 0.00 Child process normal exit<br>MGT.child_stop 0 0.00 Child process unexpected exit<br>
MGT.child_died 1 0.00 Child process died (signal)<br>MGT.child_dump 0 0.00 Child process core dumped<br>MGT.child_panic 1 0.00 Child process panic<br>
MEMPOOL.vbc.live 88 . In use<br>MEMPOOL.vbc.pool 15 . In Pool<br>MEMPOOL.vbc.sz_wanted 88 . Size requested<br>MEMPOOL.vbc.sz_needed 120 . Size allocated<br>
MEMPOOL.vbc.allocs 107148 0.45 Allocations<br>MEMPOOL.vbc.frees 107060 0.45 Frees<br>MEMPOOL.vbc.recycle 105673 0.45 Recycled from pool<br>
MEMPOOL.vbc.timeout 75842 0.32 Timed out from pool<br>MEMPOOL.vbc.toosmall 0 0.00 Too small to recycle<br>MEMPOOL.vbc.surplus 2032 0.01 Too many for pool<br>
MEMPOOL.vbc.randry 1475 0.01 Pool ran dry<br>MEMPOOL.busyobj.live 58 . In use<br>MEMPOOL.busyobj.pool 11 . In Pool<br>MEMPOOL.busyobj.sz_wanted 65536 . Size requested<br>
MEMPOOL.busyobj.sz_needed 65568 . Size allocated<br>MEMPOOL.busyobj.allocs 4391285 18.55 Allocations<br>MEMPOOL.busyobj.frees 4391227 18.55 Frees<br>
MEMPOOL.busyobj.recycle 4380692 18.50 Recycled from pool<br>MEMPOOL.busyobj.timeout 231100 0.98 Timed out from pool<br>MEMPOOL.busyobj.toosmall 0 0.00 Too small to recycle<br>
MEMPOOL.busyobj.surplus 916 0.00 Too many for pool<br>MEMPOOL.busyobj.randry 10593 0.04 Pool ran dry<br>MEMPOOL.req0.live 14 . In use<br>
MEMPOOL.req0.pool 10 . In Pool<br>MEMPOOL.req0.sz_wanted 65536 . Size requested<br>MEMPOOL.req0.sz_needed 65568 . Size allocated<br>
MEMPOOL.req0.allocs 1408010 5.95 Allocations<br>MEMPOOL.req0.frees 1407996 5.95 Frees<br>MEMPOOL.req0.recycle 1407941 5.95 Recycled from pool<br>
MEMPOOL.req0.timeout 167174 0.71 Timed out from pool<br>MEMPOOL.req0.toosmall 0 0.00 Too small to recycle<br>MEMPOOL.req0.surplus 0 0.00 Too many for pool<br>
MEMPOOL.req0.randry 69 0.00 Pool ran dry<br>MEMPOOL.sess0.live 19 . In use<br>MEMPOOL.sess0.pool 10 . In Pool<br>MEMPOOL.sess0.sz_wanted 384 . Size requested<br>
MEMPOOL.sess0.sz_needed 416 . Size allocated<br>MEMPOOL.sess0.allocs 108788 0.46 Allocations<br>MEMPOOL.sess0.frees 108769 0.46 Frees<br>
MEMPOOL.sess0.recycle 108735 0.46 Recycled from pool<br>MEMPOOL.sess0.timeout 76175 0.32 Timed out from pool<br>MEMPOOL.sess0.toosmall 0 0.00 Too small to recycle<br>
MEMPOOL.sess0.surplus 0 0.00 Too many for pool<br>MEMPOOL.sess0.randry 53 0.00 Pool ran dry<br>MEMPOOL.req1.live 14 . In use<br>
MEMPOOL.req1.pool 11 . In Pool<br>MEMPOOL.req1.sz_wanted 65536 . Size requested<br>MEMPOOL.req1.sz_needed 65568 . Size allocated<br>
MEMPOOL.req1.allocs 1409105 5.95 Allocations<br>MEMPOOL.req1.frees 1409091 5.95 Frees<br>MEMPOOL.req1.recycle 1408955 5.95 Recycled from pool<br>
MEMPOOL.req1.timeout 168231 0.71 Timed out from pool<br>MEMPOOL.req1.toosmall 0 0.00 Too small to recycle<br>MEMPOOL.req1.surplus 0 0.00 Too many for pool<br>
MEMPOOL.req1.randry 150 0.00 Pool ran dry<br>MEMPOOL.sess1.live 19 . In use<br>MEMPOOL.sess1.pool 13 . In Pool<br>MEMPOOL.sess1.sz_wanted 384 . Size requested<br>
MEMPOOL.sess1.sz_needed 416 . Size allocated<br>MEMPOOL.sess1.allocs 108939 0.46 Allocations<br>MEMPOOL.sess1.frees 108920 0.46 Frees<br>
MEMPOOL.sess1.recycle 108788 0.46 Recycled from pool<br>MEMPOOL.sess1.timeout 76251 0.32 Timed out from pool<br>MEMPOOL.sess1.toosmall 0 0.00 Too small to recycle<br>
MEMPOOL.sess1.surplus 0 0.00 Too many for pool<br>MEMPOOL.sess1.randry 151 0.00 Pool ran dry<br>SMA.s0.c_req 10059622 42.49 Allocator requests<br>
SMA.s0.c_fail 1339270 5.66 Allocator failures<br>SMA.s0.c_bytes 576850225314 2436434.47 Bytes allocated<br>SMA.s0.c_freed 567228141680 2395793.81 Bytes freed<br>
SMA.s0.g_alloc 145458 . Allocations outstanding<br>SMA.s0.g_bytes 9622083634 . Bytes outstanding<br>SMA.s0.g_space 41592782 . Bytes available<br>
SMA.Transient.c_req 26057 0.11 Allocator requests<br>SMA.Transient.c_fail 0 0.00 Allocator failures<br>SMA.Transient.c_bytes 11256624 47.54 Bytes allocated<br>
SMA.Transient.c_freed 11256624 47.54 Bytes freed<br>SMA.Transient.g_alloc 0 . Allocations outstanding<br>SMA.Transient.g_bytes 0 . Bytes outstanding<br>
SMA.Transient.g_space 0 . Bytes available<br>VBE.backend1(x.x.x.x,,8080).vcls 4 . VCL references<br>VBE.backend1(x.x.x.x,,8080).happy 11493186249049505791 . Happy health probes<br>
VBE.backend1(x.x.x.x,,8080).bereq_hdrbytes 715578294 3022.38 Request header bytes<br>VBE.backend1(x.x.x.x,,8080).bereq_bodybytes 0 0.00 Request body bytes<br>VBE.backend1(x.x.x.x,,8080).beresp_hdrbytes 941776370 3977.77 Response header bytes<br>
VBE.backend1(x.x.x.x,,8080).beresp_bodybytes 17197789137 72638.07 Response body bytes<br>VBE.backend1(x.x.x.x,,8080).pipe_hdrbytes 0 0.00 Pipe request header bytes<br>VBE.backend1(x.x.x.x,,8080).pipe_out 0 0.00 Piped bytes to backend<br>
VBE.backend1(x.x.x.x,,8080).pipe_in 0 0.00 Piped bytes from backend<br>VBE.backend2(x.x.x.x,,8080).vcls 4 . VCL references<br>VBE.backend2(x.x.x.x,,8080).happy 14987979559889002495 . Happy health probes<br>
VBE.backend2(x.x.x.x,,8080).bereq_hdrbytes 715272478 3021.09 Request header bytes<br>VBE.backend2(x.x.x.x,,8080).bereq_bodybytes 0 0.00 Request body bytes<br>VBE.backend2(x.x.x.x,,8080).beresp_hdrbytes 941339831 3975.92 Response header bytes<br>
VBE.backend2(x.x.x.x,,8080).beresp_bodybytes 17174050588 72537.80 Response body bytes<br>VBE.backend2(x.x.x.x,,8080).pipe_hdrbytes 0 0.00 Pipe request header bytes<br>VBE.backend2(x.x.x.x,,8080).pipe_out 0 0.00 Piped bytes to backend<br>
VBE.backend2(x.x.x.x,,8080).pipe_in 0 0.00 Piped bytes from backend<br>VBE.backend3(x.x.x.x,,8080).vcls 4 . VCL references<br>VBE.backend3(x.x.x.x,,8080).happy 4611686018427363327 . Happy health probes<br>
VBE.backend3(x.x.x.x,,8080).bereq_hdrbytes 707848949 2989.73 Request header bytes<br>VBE.backend3(x.x.x.x,,8080).bereq_bodybytes 0 0.00 Request body bytes<br>VBE.backend3(x.x.x.x,,8080).beresp_hdrbytes 931448534 3934.15 Response header bytes<br>
VBE.backend3(x.x.x.x,,8080).beresp_bodybytes 16984021399 71735.18 Response body bytes<br>VBE.backend3(x.x.x.x,,8080).pipe_hdrbytes 0 0.00 Pipe request header bytes<br>VBE.backend3(x.x.x.x,,8080).pipe_out 0 0.00 Piped bytes to backend<br>
VBE.backend3(x.x.x.x,,8080).pipe_in 0 0.00 Piped bytes from backend<br>LCK.sms.creat 0 0.00 Created locks<br>LCK.sms.destroy 0 0.00 Destroyed locks<br>
LCK.sms.locks 0 0.00 Lock Operations<br>LCK.smp.creat 0 0.00 Created locks<br>LCK.smp.destroy 0 0.00 Destroyed locks<br>
LCK.smp.locks 0 0.00 Lock Operations<br>LCK.sma.creat 2 0.00 Created locks<br>LCK.sma.destroy 0 0.00 Destroyed locks<br>
LCK.sma.locks 18686630 78.93 Lock Operations<br>LCK.smf.creat 0 0.00 Created locks<br>LCK.smf.destroy 0 0.00 Destroyed locks<br>
LCK.smf.locks 0 0.00 Lock Operations<br>LCK.hsl.creat 0 0.00 Created locks<br>LCK.hsl.destroy 0 0.00 Destroyed locks<br>
LCK.hsl.locks 0 0.00 Lock Operations<br>LCK.hcb.creat 1 0.00 Created locks<br>LCK.hcb.destroy 0 0.00 Destroyed locks<br>
LCK.hcb.locks 6356203 26.85 Lock Operations<br>LCK.hcl.creat 0 0.00 Created locks<br>LCK.hcl.destroy 0 0.00 Destroyed locks<br>
LCK.hcl.locks 0 0.00 Lock Operations<br>LCK.vcl.creat 1 0.00 Created locks<br>LCK.vcl.destroy 0 0.00 Destroyed locks<br>
LCK.vcl.locks 8976880 37.92 Lock Operations<br>LCK.sessmem.creat 0 0.00 Created locks<br>LCK.sessmem.destroy 0 0.00 Destroyed locks<br>
LCK.sessmem.locks 0 0.00 Lock Operations<br>LCK.sess.creat 217727 0.92 Created locks<br>LCK.sess.destroy 217689 0.92 Destroyed locks<br>
LCK.sess.locks 0 0.00 Lock Operations<br>LCK.wstat.creat 1 0.00 Created locks<br>LCK.wstat.destroy 0 0.00 Destroyed locks<br>
LCK.wstat.locks 19771639 83.51 Lock Operations<br>LCK.herder.creat 0 0.00 Created locks<br>LCK.herder.destroy 0 0.00 Destroyed locks<br>
LCK.herder.locks 0 0.00 Lock Operations<br>LCK.wq.creat 3 0.00 Created locks<br>LCK.wq.destroy 0 0.00 Destroyed locks<br>
LCK.wq.locks 14797256 62.50 Lock Operations<br>LCK.objhdr.creat 3216824 13.59 Created locks<br>LCK.objhdr.destroy 3147552 13.29 Destroyed locks<br>
LCK.objhdr.locks 60262894 254.53 Lock Operations<br>LCK.exp.creat 1 0.00 Created locks<br>LCK.exp.destroy 0 0.00 Destroyed locks<br>
LCK.exp.locks 26069460 110.11 Lock Operations<br>LCK.lru.creat 2 0.00 Created locks<br>LCK.lru.destroy 0 0.00 Destroyed locks<br>
LCK.lru.locks 18404226 77.73 Lock Operations<br>LCK.cli.creat 1 0.00 Created locks<br>LCK.cli.destroy 0 0.00 Destroyed locks<br>
LCK.cli.locks 78936 0.33 Lock Operations<br>LCK.ban.creat 1 0.00 Created locks<br>LCK.ban.destroy 0 0.00 Destroyed locks<br>
LCK.ban.locks 9477160 40.03 Lock Operations<br>LCK.vbp.creat 1 0.00 Created locks<br>LCK.vbp.destroy 0 0.00 Destroyed locks<br>
LCK.vbp.locks 124145 0.52 Lock Operations<br>LCK.backend.creat 3 0.00 Created locks<br>LCK.backend.destroy 0 0.00 Destroyed locks<br>
LCK.backend.locks 8991937 37.98 Lock Operations<br>LCK.vcapace.creat 1 0.00 Created locks<br>LCK.vcapace.destroy 0 0.00 Destroyed locks<br>
LCK.vcapace.locks 0 0.00 Lock Operations<br>LCK.nbusyobj.creat 0 0.00 Created locks<br>LCK.nbusyobj.destroy 0 0.00 Destroyed locks<br>
LCK.nbusyobj.locks 0 0.00 Lock Operations<br>LCK.busyobj.creat 4391285 18.55 Created locks<br>LCK.busyobj.destroy 4391227 18.55 Destroyed locks<br>
LCK.busyobj.locks 33222445 140.32 Lock Operations<br>LCK.mempool.creat 6 0.00 Created locks<br>LCK.mempool.destroy 0 0.00 Destroyed locks<br>
LCK.mempool.locks 17553267 74.14 Lock Operations<br>LCK.vxid.creat 1 0.00 Created locks<br>LCK.vxid.destroy 0 0.00 Destroyed locks<br>
LCK.vxid.locks 22801 0.10 Lock Operations<br>LCK.pipestat.creat 1 0.00 Created locks<br>LCK.pipestat.destroy 0 0.00 Destroyed locks<br>
LCK.pipestat.locks 0 0.00 Lock Operations<br><br></div>