<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I've been using Varnish in an "intranet" application. The picture
is roughly:<br>
<br>
origin <-> Varnish <-- 10G channel ---> switch <--
1G channel --> client<br>
<br>
The machine running Varnish is a high-performance server. It can<br>
easily saturate a 10Gbit channel. The machine running the client is
a<br>
more modest desktop workstation, but it's fully capable of
saturating<br>
a 1Gbit channel.<br>
<br>
The client makes HTTP requests for objects of size 128kB.<br>
<br>
When the client makes those requests serially, "useful" data is<br>
transferred at about 80% of the channel bandwidth of the Gigabit<br>
link, which seems perfectly reasonable.<br>
<br>
But when the client makes the requests in parallel (typically<br>
4-at-a-time, but it can vary), *total* throughput drops to about 25%<br>
of the channel bandwidth, i.e., about 30Mbyte/sec.<br>
<br>
After looking at traces and doing a fair amount of experimentation,
we<br>
have reached the tentative conclusion that we're seeing "TCP Incast<br>
Throughput Collapse" (see references below)<br>
<br>
The literature on "TCP Incast Throughput Collapse" typically
describes<br>
scenarios where a large number of servers overwhelm a single inbound<br>
port. I haven't found any discussion of incast collapse with only
one<br>
server, but it seems like a natural consequence of a
10Gigabit-capable<br>
server feeding a 1-Gigabit downlink.<br>
<br>
Has anybody else seen anything similar? With Varnish or other
single<br>
servers on 10Gbit to 1Gbit links.<br>
<br>
The literature offers a variety of mitigation strategies, but there
are<br>
non-trivial tradeoffs and none appears to be a silver bullet.<br>
<br>
If anyone has seen TCP Incast Collapse with Varnish, were you able
to work<br>
around it, and if so, how?<br>
<br>
Thanks,<br>
John Salmon<br>
<br>
References:<br>
<br>
<a class="moz-txt-link-freetext" href="http://www.pdl.cmu.edu/Incast/">http://www.pdl.cmu.edu/Incast/</a><br>
<br>
Annotated Bibliography in:<br>
<a class="moz-txt-link-freetext" href="https://lists.freebsd.org/pipermail/freebsd-net/2015-November/043926.html">https://lists.freebsd.org/pipermail/freebsd-net/2015-November/043926.html</a><br>
<br>
<div class="moz-signature">-- <br>
<b>.</b></div>
</body>
</html>