<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">There is a large discrepancy between Varnish resident memory reported by top vs reported by varnishstat. Varnish is configured with both malloc and file storage; 20G malloc and 75G file storage.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">These are the startup parameters-<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">VARNISH_STORAGE="memcache=malloc,20G -s filecache=file,/mnt/xvdf1/varnish/varnish_storage.bin,75G"<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">After running for a few days top is reporting more than twice amount of memory used for varnishd process than varnishstat.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">From top-<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> 1083 varnish 20 0 82.235g 0.014t 7.765g S 75.7 51.4 2159:07 varnishd <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">From varnishstat-<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMA.memcache.g_bytes 5.20G 39.65K . 5.20G 5.20G 5.20G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMA.memcache.g_space 14.80G -39.65K . 14.80G 14.80G 14.80G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMF.filecache.g_bytes 7.78G 27.97K . 7.78G 7.78G 7.78G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMF.filecache.g_space 67.22G -27.97K . 67.22G 67.22G 67.22G<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This is the relevant part of the VCL regarding storage backend selection-<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">sub vcl_backend_response {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> # define separate cache storage groups<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (bereq.http.host ~ "^(encore|thereal|static)\.(beachcamera|buydig)\.com") {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.storage_hint = "filecache";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.http.X-Cache-Storage = "disk";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> } elsif (bereq.url ~ "(?i)\.(jpg|jpeg|gif|ico|pdf|swf|png|zip)") {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.storage_hint = "filecache";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.http.X-Cache-Storage = "disk";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> } elsif (bereq.url ~ "(?i)product\-image\.aspx") {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.storage_hint = "filecache";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.http.X-Cache-Storage = "disk";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> } else {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.storage_hint = "memcache";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> set beresp.http.X-Cache-Storage = "memory";<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Can post entire VCL if needed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Would think that since varnishstat reports 5.20G RAM used the resident memory should be around 6-7G, 14G seems excessively high. File storage should use minimal resident memory, correct?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Varnish was installed from Varnish Cache 4.1 repo. No VMODs loaded except std and directors. Using latest 4.1.7<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Anything else you need please let me know.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
</body>
</html>