<div dir="ltr">There's no optimal configuration, only compromises, but I can point you there: <a href="https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage">https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage</a></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>-- <br></div>Guillaume Quintard<br></div></div></div>
<br><div class="gmail_quote">On Thu, Feb 8, 2018 at 7:26 AM, Pinakee BIswas <span dir="ltr"><<a href="mailto:pinakee@waltzz.com" target="_blank">pinakee@waltzz.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <p>Hi,</p>
    <p>Could you please suggest what would be the optimal configuration
      for varnishd in my scenario - in terms of memory usage and
      performance?</p>
    <p>Thanks,</p>
    <p>Pinakee<br>
    </p><div><div class="h5">
    <br>
    <div class="m_7939043173025163298moz-cite-prefix">On 07/02/18 7:19 pm, Pinakee BIswas
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <p>I agree but the reason being we have other applications like DB
        servers, backend server, cache server etc running on the same
        machine (as our scale is not that big). Hence, memory is a
        sought after resource and needs to be optimized as much as
        possible. </p>
      <p>So, if you suggest malloc would be a better storage still, I
        can change the storage to the same. <br>
      </p>
      <div class="m_7939043173025163298moz-cite-prefix">On 07/02/18 7:09 pm, Guillaume
        Quintard wrote:<br>
      </div>
      <blockquote type="cite">
        <div dir="auto">So, it's probably not related to your issue, but
          it seems your cached data easily fits in memory, why use the
          file storage?<br>
          <br>
          <div data-smartmail="gmail_signature">-- <br>
            Guillaume Quintard </div>
        </div>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Feb 7, 2018 14:35, "Pinakee
            BIswas" <<a href="mailto:pinakee@waltzz.com" target="_blank">pinakee@waltzz.com</a>>
            wrote:<br type="attribution">
            <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div bgcolor="#FFFFFF" text="#000000">
                <p>Following is the data from varnishstat:</p>
                <p>SMA.Transient.g_bytes         <wbr>                              <wbr>                              
                  15.76K        -2.19K          .          
                  18.53K        15.50K        15.24K</p>
                <p>SMF.s0.g_bytes                <wbr>                              <wbr>                             
                  673.26M        47.95K          .         
                  672.87M       669.89M       668.90M<br>
                </p>
                <p><br>
                </p>
                <div class="m_7939043173025163298m_-5598368564391915570moz-cite-prefix">On
                  07/02/18 6:51 pm, Guillaume Quintard wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="auto">Amount of storage used is just the sum
                    of all the g_bytes fields<br>
                    <br>
                    <div data-smartmail="gmail_signature">-- <br>
                      Guillaume Quintard </div>
                  </div>
                  <div class="gmail_extra"><br>
                    <div class="gmail_quote">On Feb 7, 2018 14:12,
                      "Pinakee BIswas" <<a href="mailto:pinakee@waltzz.com" target="_blank">pinakee@waltzz.com</a>>
                      wrote:<br type="attribution">
                      <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                        <div bgcolor="#FFFFFF" text="#000000">
                          <p>Hi Guillaume,</p>
                          <p>Thanks for your response and details.</p>
                          <p>Please find my comments inline:<br>
                          </p>
                          <br>
                          <div class="m_7939043173025163298m_-5598368564391915570m_-8343487515810236383moz-cite-prefix">On
                            07/02/18 6:21 pm, Guillaume Quintard wrote:<br>
                          </div>
                          <blockquote type="cite">
                            <div dir="ltr">Hi there,
                              <div><br>
                              </div>
                              <div>So, varnish memory usage will mostly
                                come from three directions:</div>
                              <div>- storage, include Transient, so
                                check the g_bytes fields in varnishstat.
                                Passes and shortlived objects will use
                                Transient, so you can either reduce
                                those, or limit the Transient storage
                                (unbounded by defaukt).</div>
                            </div>
                          </blockquote>
                          Checked the g_bytes for transient in
                          varnishstat. It's in Kbs. <br>
                          <blockquote type="cite">
                            <div dir="ltr">
                              <div>- thread workspaces, one thread
                                typically uses one workspace, so you can
                                limit the number of threads, or reduce
                                the workspace size</div>
                              <div>- memory fragmentation: jemalloc will
                                fragment up to 25%, not much you can do
                                here.</div>
                              <div><br>
                              </div>
                              <div>And no, no way to modify the mmap
                                without restarting varnish.</div>
                              <div><br>
                              </div>
                              <div>May I ask how much data you are
                                caching?</div>
                            </div>
                          </blockquote>
                          Could you please let me know how to figure out
                          the amount of data cached?<br>
                          <blockquote type="cite">
                            <div class="gmail_extra"><br clear="all">
                              <div>
                                <div class="m_7939043173025163298m_-5598368564391915570m_-8343487515810236383gmail_signature" data-smartmail="gmail_signature">
                                  <div dir="ltr">
                                    <div>-- <br>
                                    </div>
                                    Guillaume Quintard<br>
                                  </div>
                                </div>
                              </div>
                              <br>
                              <div class="gmail_quote">On Wed, Feb 7,
                                2018 at 1:34 PM, Pinakee BIswas <span dir="ltr"><<a href="mailto:pinakee@waltzz.com" target="_blank">pinakee@waltzz.com</a>></span>
                                wrote:<br>
                                <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                                  <div bgcolor="#FFFFFF" text="#000000">
                                    <p>Hi,</p>
                                    <p>We have been using varnish since
                                      more than a year for our ecommerce
                                      site. <br>
                                    </p>
                                    <p>Current version is 4.1.8.</p>
                                    <p>OS is Linux. RAM is 8GB. <br>
                                    </p>
                                    <p>I am observing that the varnish
                                      memory usage is increasing (right
                                      now 7%) and so is resident memory
                                      usage increasing. <br>
                                    </p>
                                    <p>The storage being used is file:</p>
                                    <p>-s file,/tmp/varnish/,${storage}</p>
                                    <p>I am not sure about the reason
                                      for the increase in varnish memory
                                      usage:</p>
                                    <ul>
                                      <li>Is there a way to limit the
                                        varnish memory usage?</li>
                                      <li>How can I diagnose what is
                                        consuming memory?</li>
                                      <li>Most of our web pages have
                                        maximum 2 days of cache. Also,
                                        some of the pages might be least
                                        visited.</li>
                                      <li>Is there a way to manipulate
                                        varnish mmap (when file storage
                                        is used)?</li>
                                    </ul>
                                    <p>Would appreciate any help on the
                                      above for efficient use of
                                      varnish.</p>
                                    <p>Thanks,</p>
                                    <p>Pinakee<br>
                                    </p>
                                    <p><br>
                                    </p>
                                  </div>
                                  <br>
                                  ______________________________<wbr>_________________<br>
                                  varnish-misc mailing list<br>
                                  <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
                                  <a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" rel="noreferrer" target="_blank">https://www.varnish-cache.org/<wbr>lists/mailman/listinfo/varnish<wbr>-misc</a><br>
                                </blockquote>
                              </div>
                              <br>
                            </div>
                          </blockquote>
                          <br>
                        </div>
                      </blockquote>
                    </div>
                  </div>
                </blockquote>
                <br>
              </div>
            </blockquote>
          </div>
        </div>
      </blockquote>
      <br>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div>