<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi,</p>
<p>Could you please suggest what would be the optimal configuration
for varnishd in my scenario - in terms of memory usage and
performance?</p>
<p>Thanks,</p>
<p>Pinakee<br>
</p>
<br>
<div class="moz-cite-prefix">On 07/02/18 7:19 pm, Pinakee BIswas
wrote:<br>
</div>
<blockquote
cite="mid:4f298fd0-fd0f-282e-6cae-107a2832bfd0@waltzz.com"
type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<p>I agree but the reason being we have other applications like DB
servers, backend server, cache server etc running on the same
machine (as our scale is not that big). Hence, memory is a
sought after resource and needs to be optimized as much as
possible. </p>
<p>So, if you suggest malloc would be a better storage still, I
can change the storage to the same. <br>
</p>
<div class="moz-cite-prefix">On 07/02/18 7:09 pm, Guillaume
Quintard wrote:<br>
</div>
<blockquote
cite="mid:CAJ6ZYQzC-dOp69kMNz3iRFTF_N9y4-ExeTgAW3WGW9kVWNhoxQ@mail.gmail.com"
type="cite">
<div dir="auto">So, it's probably not related to your issue, but
it seems your cached data easily fits in memory, why use the
file storage?<br>
<br>
<div data-smartmail="gmail_signature">-- <br>
Guillaume Quintard </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Feb 7, 2018 14:35, "Pinakee
BIswas" <<a moz-do-not-send="true"
href="mailto:pinakee@waltzz.com">pinakee@waltzz.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Following is the data from varnishstat:</p>
<p>SMA.Transient.g_bytes <wbr> <wbr>
15.76K -2.19K .
18.53K 15.50K 15.24K</p>
<p>SMF.s0.g_bytes <wbr> <wbr>
673.26M 47.95K .
672.87M 669.89M 668.90M<br>
</p>
<p><br>
</p>
<div class="m_-5598368564391915570moz-cite-prefix">On
07/02/18 6:51 pm, Guillaume Quintard wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">Amount of storage used is just the sum
of all the g_bytes fields<br>
<br>
<div data-smartmail="gmail_signature">-- <br>
Guillaume Quintard </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Feb 7, 2018 14:12,
"Pinakee BIswas" <<a moz-do-not-send="true"
href="mailto:pinakee@waltzz.com" target="_blank">pinakee@waltzz.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hi Guillaume,</p>
<p>Thanks for your response and details.</p>
<p>Please find my comments inline:<br>
</p>
<br>
<div
class="m_-5598368564391915570m_-8343487515810236383moz-cite-prefix">On
07/02/18 6:21 pm, Guillaume Quintard wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi there,
<div><br>
</div>
<div>So, varnish memory usage will mostly
come from three directions:</div>
<div>- storage, include Transient, so
check the g_bytes fields in varnishstat.
Passes and shortlived objects will use
Transient, so you can either reduce
those, or limit the Transient storage
(unbounded by defaukt).</div>
</div>
</blockquote>
Checked the g_bytes for transient in
varnishstat. It's in Kbs. <br>
<blockquote type="cite">
<div dir="ltr">
<div>- thread workspaces, one thread
typically uses one workspace, so you can
limit the number of threads, or reduce
the workspace size</div>
<div>- memory fragmentation: jemalloc will
fragment up to 25%, not much you can do
here.</div>
<div><br>
</div>
<div>And no, no way to modify the mmap
without restarting varnish.</div>
<div><br>
</div>
<div>May I ask how much data you are
caching?</div>
</div>
</blockquote>
Could you please let me know how to figure out
the amount of data cached?<br>
<blockquote type="cite">
<div class="gmail_extra"><br clear="all">
<div>
<div
class="m_-5598368564391915570m_-8343487515810236383gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>-- <br>
</div>
Guillaume Quintard<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Wed, Feb 7,
2018 at 1:34 PM, Pinakee BIswas <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:pinakee@waltzz.com"
target="_blank">pinakee@waltzz.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Hi,</p>
<p>We have been using varnish since
more than a year for our ecommerce
site. <br>
</p>
<p>Current version is 4.1.8.</p>
<p>OS is Linux. RAM is 8GB. <br>
</p>
<p>I am observing that the varnish
memory usage is increasing (right
now 7%) and so is resident memory
usage increasing. <br>
</p>
<p>The storage being used is file:</p>
<p>-s file,/tmp/varnish/,${storage}</p>
<p>I am not sure about the reason
for the increase in varnish memory
usage:</p>
<ul>
<li>Is there a way to limit the
varnish memory usage?</li>
<li>How can I diagnose what is
consuming memory?</li>
<li>Most of our web pages have
maximum 2 days of cache. Also,
some of the pages might be least
visited.</li>
<li>Is there a way to manipulate
varnish mmap (when file storage
is used)?</li>
</ul>
<p>Would appreciate any help on the
above for efficient use of
varnish.</p>
<p>Thanks,</p>
<p>Pinakee<br>
</p>
<p><br>
</p>
</div>
<br>
______________________________<wbr>_________________<br>
varnish-misc mailing list<br>
<a moz-do-not-send="true"
href="mailto:varnish-misc@varnish-cache.org"
target="_blank">varnish-misc@varnish-cache.org</a><br>
<a moz-do-not-send="true"
href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc"
rel="noreferrer" target="_blank">https://www.varnish-cache.org/<wbr>lists/mailman/listinfo/varnish<wbr>-misc</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>