<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:D="DAV:" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal>Hey,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Sorry if this is the wrong place to be posting this, but I
didn’t see any other lists that seemed to fit the question.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Basically, we are currently using varnish for some our photo
caching, and it’s working perfectly. Here’s the relevant portion of
varnishstat from a photo cache:<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>client_conn
4478824 49.86 Client connections
accepted<o:p></o:p></p>
<p class=MsoNormal>client_req
10146386 112.96 Client requests received<o:p></o:p></p>
<p class=MsoNormal>cache_hit
9101426 101.32 Cache hits<o:p></o:p></p>
<p class=MsoNormal>cache_hitpass
27 0.00 Cache hits for pass<o:p></o:p></p>
<p class=MsoNormal>cache_miss
1044704 11.63 Cache misses<o:p></o:p></p>
<p class=MsoNormal>backend_conn
1044954 11.63 Backend connections
success<o:p></o:p></p>
<p class=MsoNormal>backend_fail
0 0.00 Backend connections
failures<o:p></o:p></p>
<p class=MsoNormal>backend_reuse
979195 10.90 Backend connections
reuses<o:p></o:p></p>
<p class=MsoNormal>backend_recycle
985646 10.97 Backend connections
recycles<o:p></o:p></p>
<p class=MsoNormal>backend_unused
39 0.00 Backend connections
unused<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>We start varnishd up with this commandline:<o:p></o:p></p>
<p class=MsoNormal>/usr/local/sbin/varnishd -a :80 -f /etc/varnish/default.vcl
-T 127.0.0.1:6082 -t 120 -w 1,1000,120 -s
file,/c01/varnish/varnish_storage.bin,40% -u varnish -g varnish -n /var/varnish
-P /var/run/varnish.pid<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Varnish is working exactly as we expected with photos, and
exceptionally better than squid (which we also use). We have a problem when we
use varnish for serving audio and video files though. Using the same
commandline, varnish eats up all the memory in the system, and then starts to
eat into swap, which in turn makes the system unusable. During our test, the
load rose on the machine to ~950 (!), at which point we shut it off and went
back to using squid, where the load hovers around ~6.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>The machine running squid is a dell pe1950, 8gb of ram, 1
xeon 2.33 ghz. The pe1950 has 4 drives in the system formatted as a raid 10,
and is connected to a md1000 (external storage) consisting of 15 73gb 15k scsi
drives formatted in a raid 10+HS. This is formatted at /c01, which, as you can
see above, is where we keep the backend storage for varnish.<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Unfortunately, I don’t have the output from varnish on
the audio/video system. I’m just wondering if anyone has any suggestions
on anything I may be doing wrong. Is 40% (~190GB) too much for a backend
storage? The default.vcl file on the audio/video system is about as simple as
you get, with something added for TTL’s. I know varnish is being used on
some pretty big sites out there, so I’m hoping people have some pointers…<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Thanks,<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>--<o:p></o:p></p>
<p class=MsoNormal><b>Andrew Knapp</b> | systems engineer | imeem, inc. |
415.762.0135 o | 415.994.5752 m | 415.836.0088 f<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>