<div dir="ltr">You misundestood me, but that's probably my fault :-)<div><br></div><div>The file used by the file storage will be fully allocated, ie. you;ll have a 75G file on your disk. Then varnish will mmap it to memory, which means the kernel will give Varnish a memory space corresponding to the file content. The trick is that the whole file doesn't need to be in memory, only the "active" parts are.</div><div><br></div><div>What happens is that we let the kernel manage that space, and it will leverage the unused memory to do so. So, true, it you had 200G of RAM, the file storage would effectively take 75G, because they would be available. That's not your case, so the kernel will only use whatever amount is unused.</div><div><br></div><div>That being said, outside of storage, Varnish uses roughly 1K per object stored, that's probably not impacting right now, but that's good to keep in mind.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>-- <br></div>Guillaume Quintard<br></div></div></div>
<br><div class="gmail_quote">On Mon, Jul 10, 2017 at 2:42 PM, Bender, Charles <span dir="ltr"><<a href="mailto:charles@beachcamera.com" target="_blank">charles@beachcamera.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">



<div dir="auto">
<div></div>
<div>Hi Guillaume,</div>
<div><br>
</div>
<div>Thank you for replying so quickly. I think i'm misunderstanding what file storage does. My goal is to have some objects stored in memory (malloc) and others stored on disk (file)</div>
<div><br>
</div>
<div>From what you're saying file method still uses resident memory for each object, so with my configuration (20G malloc, 75G file) i would need 95G RAM if all storage is used? (without swap being used)</div>
<div><br>
</div>
<div>If this is the case i'm curious what the use cases are for file vs only malloc. </div><div><div class="h5">
<div><br>
On Jul 10, 2017, at 1:55 PM, Guillaume Quintard <<a href="mailto:guillaume@varnish-software.com" target="_blank">guillaume@varnish-software.<wbr>com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">Hi Charles,
<div><br>
</div>
<div>So, if I'm reading this right, there's no discrepancy. Varnish will malloc the full storage (malloc), and will mmap a file the size of the full storage (file). So even though the storage is not used, it's allocated.</div>
<div><br>
</div>
<div>Does it make sense, or did I miss something?</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div class="m_-1823304827062197160gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>-- <br>
</div>
Guillaume Quintard<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Mon, Jul 10, 2017 at 1:23 PM, Bender, Charles <span dir="ltr">
<<a href="mailto:charles@beachcamera.com" target="_blank">charles@beachcamera.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="white" lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="m_-1823304827062197160m_-682086389115876189WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">There is a large discrepancy between Varnish resident memory reported by top vs reported by varnishstat. Varnish is configured with both malloc and file storage; 20G malloc and 75G file storage.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">These are the startup parameters-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">VARNISH_STORAGE="memcache=mall<wbr>oc,20G -s filecache=file,/mnt/xvdf1/varn<wbr>ish/varnish_storage.bin,75G"<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">After running for a few days top is reporting more than twice amount of memory used for varnishd process than varnishstat.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">From top-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                       <wbr>                              <wbr>                    <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> 1083 varnish   20   0 82.235g 0.014t 7.765g S  75.7 51.4   2159:07 varnishd                      <wbr>                              <wbr>                    <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">From varnishstat-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMA.memcache.g_bytes          <wbr>                              <wbr>               5.20G        39.65K          .            5.20G         5.20G   5.20G<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMA.memcache.g_space          <wbr>                              <wbr>              14.80G       -39.65K          .           14.80G        14.80G              14.80G<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMF.filecache.g_bytes         <wbr>                              <wbr>               7.78G        27.97K          .         7.78G         7.78G         7.78G<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">SMF.filecache.g_space         <wbr>                              <wbr>              67.22G       -27.97K          .      67.22G        67.22G        67.22G<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This is the relevant part of the VCL regarding storage backend selection-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">sub vcl_backend_response {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                # define separate cache storage groups<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                if (bereq.http.host ~ "^(encore|thereal|static)\.(be<wbr>achcamera|buydig)\.com") {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.storage_hint = "filecache";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.http.X-Cache-Storage = "disk";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                } elsif (bereq.url ~ "(?i)\.(jpg|jpeg|gif|ico|pdf|s<wbr>wf|png|zip)") {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.storage_hint = "filecache";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.http.X-Cache-Storage = "disk";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                } elsif (bereq.url ~ "(?i)product\-image\.aspx") {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.storage_hint = "filecache";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.http.X-Cache-Storage = "disk";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                } else {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.storage_hint = "memcache";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                              <wbr>  set beresp.http.X-Cache-Storage = "memory";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">                }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Can post entire VCL if needed.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Would think that since varnishstat reports 5.20G RAM used the resident memory should be around 6-7G, 14G seems excessively high. File storage should use minimal resident memory, correct?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Varnish was installed from Varnish Cache 4.1 repo. No VMODs loaded except std and directors. Using latest 4.1.7<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Anything else you need please let me know.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><u></u> <u></u></span></p>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" rel="noreferrer" target="_blank">https://www.varnish-cache.org/<wbr>lists/mailman/listinfo/varnish<wbr>-misc</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div></div></div>

</blockquote></div><br></div>