<div dir="ltr"><div>Hi all,<br><br>I am testing using Varnish 4.0.3 in our datacenter. The test setup has 24 servers (48<br>core, 128GB, Ubuntu 14.10, Intel ixgbe) running HAProxy, with thousand backends,<br>each of which are VM's created on different servers. The HAProxy servers all run<br>either HAProxy only, or HAProxy + Varnish only, the two testing configurations, with no<br>other load.<br><br>Model 1 (Direct): Clients --> [ HAProxy ] --> 2300 Backends<br>Model 2 (Caching): Clients --> [ HAProxy --> 1 Varnish backend ] --> 2300 Backends<br><br>The Square brackets above indicate that this item runs on a single baremetal as described<br>above. Diagrammatically:<br> Clients (1000 VM's)<br><br>________ _______ ________ ________ (24 baremetals, each<br>|HAProxy| |HAProxy| |HAProxy| |HAProxy| of which runs either<br>|Varnish | |Varnish | |Varnish | |Varnish | HAProxy or HAProxy+Varnish)<br>-------------- -------------- -------------- --------------<br> | | | |<br> v v v v<br>B1-B2300 B1-B2300 B1-B2300 B1-B2300 (2300 Backends, VM's running nginx)<br><br>For the Caching setup, Varnish is configured to run on each baremetal on the<br>'lo' interface (on the same system). The test is admittedly primitive, each of the<br>2300 clients simply gets the same 128 byte or 128K byte for a 15 minute run,<br>using wrk with "-t100 -c800". The following table shows the RPS and BW for 128<br>bytes and 128K bytes, for both "Direct" and "Caching" scenarios, and the % change:<br><br>|--------|--------------------------------------|-------------------------------------|---------------------------------|<br>| I/O | Req Per Sec | BW | Load |<br>| Size | Direct Cache % | Direct Cache % | Direct Cache % |<br>|--------|--------------------------------------|--------------------------------------|---------------------------------|<br>| 128 | 5559603 2112749 -62% | 1770 780 -56% | 629 197 -69% |<br>| 128K | 118294 138990 18% | 15158 17429 15% | 143 389 172% |<br>|---------|--------------------------------------|-------------------------------------|---------------------------------|<br><br></div>(hope it is readable)<br><br><div>For small packets, Varnish gets much lower RPS and BW (about half), but also takes<br>lesser system load (measured by top). For large packets, Varnish gets about 15-18%<br>increase in RPS and BW, but significantly increased system load. Varnishstat from a<br>typical server:<br> MAIN.cache_hit 73512277 14655.56 Cache hits<br> MAIN.cache_miss 2 0.00 Cache misses<br><br></div><div>/etc/default/varnish has:<br></div><div><br>NFILES=262144<br>MEMLOCK=82000<br>DAEMON_OPTS="-a :6081 -T localhost:6082 -f /etc/varnish/default.vcl \<br> -p thread_pools=2 -p thread_pool_min=500 -p thread_pool_max=750 \<br> -p thread_pool_add_delay=2 -S /etc/varnish/secret -s malloc,64G"<br><br>/etc/varnish/default.vcl is created with the 2300 VM ip addresses as:<br><br>backend varnish_server_1 {<br> .host = "10.32.118.14";<br> .port = "80";<br> .connect_timeout = 5s;<br> .first_byte_timeout = 5s;<br> .between_bytes_timeout = 2s;<br>}<br>...<br>(and so on till varnish_server_2300)<br><br>sub vcl_init {<br> new varnish_cluster = directors.round_robin();<br> varnish_cluster.add_backend(varnish_server_1);<br> varnish_cluster.add_backend(varnish_server_2);<br> ...<br> varnish_cluster.add_backend(varnish_server_2300);<br>}<br><br>sub vcl_recv {<br> set req.backend_hint = varnish_cluster.backend();<br>}<br><br>Could someone suggest any improvement in my setup for better performance?<br><br>Thanks,<br>Krishna Kumar<br><br></div></div>
<br>
<p>------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------------------<wbr>------------------</p><p>This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Although Flipkart has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments</p>