Bug in 1.1.2: Multiple headers stripped (oops)
Adrian Otto
aotto at mosso.com
Thu Aug 14 18:58:19 CEST 2008
Hello,
I've found, and reproduced an apparent bug in varnish 1.1.2. It
happens when handling a Connection header sent from the client
browser. The HTTP/1.1 protocol requires that any header names listed
in the Connection header be removed before the proxy forwards the
request to the backend web server. That works. It removes the
specified header. However, it also removes the NEXT header that
follows the specified one. This can lead to substantial trouble if
the header following the specified one is the Host header. It results
in a 404 result from the backend web server because varnish sends a
Host header with the IP address of the backend web server rather than
the original Host name that was sent by the client. This results in
the incorrect document being fetched (and cached) from the backend
server.
I have provided two examples of the problem below. I need to do some
additional work to set up my dev environment to start working on a
patch, but if anyone else already has a patch for this, or can easily
produce one, I's certainly appreciate any assistance you are willing
to offer in the mean time.
Thanks,
Adrian Otto
Here is an example:
Notice that the Connection header from the client browser specifies
that the TE header should be removed. However, once the request is
proxied to the backend web server, the Host header has changed.
13 SessionOpen c 10.3.0.30 49408
0 ExpBan 121045864 was banned
13 ReqStart c 10.3.0.30 49408 121045866
13 RxRequest c GET
13 RxURL c /spacer_thin.gif
13 RxProtocol c HTTP/1.1
13 RxHeader c Referer: http://www.example.com/carchart/index.htm
13 RxHeader c Accept-Language: en-US,en;q=0.9
13 RxHeader c Accept-Encoding: deflate, gzip, x-gzip,
identity, *;q=0
13 RxHeader c Cookie: X-Mapping-
caklakng=BB7BF84977E04EAA5A5B5A914FFEE25F
13 RxHeader c X-Cluster-Client-Ip: 10.2.0.200
13 RxHeader c Cookie2: $Version=1
13 RxHeader c Connection: Keep-Alive, TE
13 RxHeader c Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-
xbitmap, */*;q=0.1
13 RxHeader c TE: deflate, gzip, chunked, identity, trailers
13 RxHeader c Host: www.example.com
13 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
13 RxHeader c User-Agent: Opera/9.51 (Windows NT 5.1; U; en)
13 VCL_call c recv
13 VCL_return c lookup
13 VCL_call c hash
13 VCL_return c hash
13 VCL_call c miss
13 VCL_return c fetch
16 BackendOpen b default 10.0.0.100 34140 10.4.0.40 80
16 BackendXID b 121045866
13 Backend c 16 default
16 TxRequest b GET
16 TxURL b /spacer_thin.gif
16 TxProtocol b HTTP/1.1
16 TxHeader b Referer: http://www.example.com/carchart/index.htm
16 TxHeader b Accept-Language: en-US,en;q=0.9
16 TxHeader b Accept-Encoding: deflate, gzip, x-gzip,
identity, *;q=0
16 TxHeader b X-Cluster-Client-Ip: 10.2.0.200
16 TxHeader b Cookie2: $Version=1
16 TxHeader b Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-
xbitmap, */*;q=0.1
16 TxHeader b Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
16 TxHeader b User-Agent: Opera/9.51 (Windows NT 5.1; U; en)
16 TxHeader b X-Varnish: 121045866
16 TxHeader b X-Forwarded-for: 10.3.0.30
16 TxHeader b Host: 10.4.0.40
16 RxProtocol b HTTP/1.1
16 RxStatus b 404
16 RxResponse b Not Found
16 RxHeader b Date: Thu, 14 Aug 2008 16:41:57 GMT
16 RxHeader b Server: Apache/2.0.52 (CentOS)
16 RxHeader b Content-Length: 290
16 RxHeader b Connection: close
16 RxHeader b Content-Type: text/html; charset=iso-8859-1
13 ObjProtocol c HTTP/1.1
13 ObjStatus c 404
13 ObjResponse c Not Found
13 ObjHeader c Date: Thu, 14 Aug 2008 16:41:57 GMT
13 ObjHeader c Server: Apache/2.0.52 (CentOS)
13 ObjHeader c Content-Type: text/html; charset=iso-8859-1
16 BackendClose b default
13 TTL c 121045866 RFC 1800 1218732117 1218732117 0 0 0
13 VCL_call c fetch
13 VCL_return c insert
13 Length c 290
13 VCL_call c deliver
13 VCL_return c deliver
13 TxProtocol c HTTP/1.1
13 TxStatus c 404
13 TxResponse c Not Found
13 TxHeader c Server: Apache/2.0.52 (CentOS)
13 TxHeader c Content-Type: text/html; charset=iso-8859-1
13 TxHeader c Content-Length: 290
13 TxHeader c Date: Thu, 14 Aug 2008 16:41:57 GMT
13 TxHeader c X-Varnish: 121045866
13 TxHeader c Age: 0
13 TxHeader c Via: 1.1 varnish
13 TxHeader c Connection: keep-alive
13 ReqEnd c 121045866 1218732117.032603979
1218732117.123506069 0.003006935 0.090833902 0.000068188
0 StatAddr 10.3.0.30 0 84 2 2 0 0 1 522 333
0 ExpKill 121045864 -1218732117
13 SessionClose c timeout
13 StatSess c 10.3.0.30 49408 0 1 1 0 0 1 233 290
13 SessionOpen c 10.3.0.30 49409
13 ReqStart c 10.3.0.30 49409 121045867
13 RxRequest c GET
13 RxURL c /spacer_thin.gif
13 RxProtocol c HTTP/1.1
13 RxHeader c Referer: http://www.example.com/carchart/index.htm
13 RxHeader c Accept-Language: en-US,en;q=0.9
13 RxHeader c Accept-Encoding: deflate, gzip, x-gzip,
identity, *;q=0
13 RxHeader c Cookie: X-Mapping-
caklakng=BB7BF84977E04EAA5A5B5A914FFEE25F
13 RxHeader c X-Cluster-Client-Ip: 10.2.0.200
13 RxHeader c Cookie2: $Version=1
13 RxHeader c Connection: Keep-Alive, TE
13 RxHeader c Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-
xbitmap, */*;q=0.1
13 RxHeader c Host: www.example.com
13 RxHeader c TE: deflate, gzip, chunked, identity, trailers
13 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
13 RxHeader c User-Agent: Opera/9.51 (Windows NT 5.1; U; en)
13 VCL_call c recv
13 VCL_return c lookup
13 VCL_call c hash
13 VCL_return c hash
13 Hit c 121045866
13 VCL_call c hit
13 VCL_return c deliver
13 Length c 290
13 VCL_call c deliver
13 VCL_return c deliver
13 TxProtocol c HTTP/1.1
13 TxStatus c 404
13 TxResponse c Not Found
13 TxHeader c Server: Apache/2.0.52 (CentOS)
13 TxHeader c Content-Type: text/html; charset=iso-8859-1
13 TxHeader c Content-Length: 290
13 TxHeader c Date: Thu, 14 Aug 2008 16:43:18 GMT
13 TxHeader c X-Varnish: 121045867 121045866
13 TxHeader c Age: 81
13 TxHeader c Via: 1.1 varnish
13 TxHeader c Connection: keep-alive
13 ReqEnd c 121045867 1218732198.320590973
1218732198.320696115 0.003068924 0.000036001 0.000069141
0 StatAddr 10.3.0.30 0 165 3 3 0 0 1 766 623
13 SessionClose c timeout
13 StatSess c 10.3.0.30 49409 0 1 1 0 0 0 244 290
0 CLI Rd "url.purge" "/spacer_thin.gif"
0 CLI Wr 0 200 PURGE /spacer_thin.gif
Here is another example that illustrates the same problem, but
actually fetches the correct document from the backend web server. If
the Host header is relocated to BEFORE the TE header, the Accept-
Charset header vanishes instead, and the Host header goes through
unmodified. This actually works, but we don't want to strip headers
that should still be there.
13 SessionOpen c 10.3.0.30 49410
0 ExpBan 121045866 was banned
13 ReqStart c 10.3.0.30 49410 121045868
13 RxRequest c GET
13 RxURL c /spacer_thin.gif
13 RxProtocol c HTTP/1.1
13 RxHeader c Referer: http://www.example.com/carchart/index.htm
13 RxHeader c Accept-Language: en-US,en;q=0.9
13 RxHeader c Accept-Encoding: deflate, gzip, x-gzip,
identity, *;q=0
13 RxHeader c Cookie: X-Mapping-
caklakng=BB7BF84977E04EAA5A5B5A914FFEE25F
13 RxHeader c X-Cluster-Client-Ip: 10.2.0.200
13 RxHeader c Cookie2: $Version=1
13 RxHeader c Connection: Keep-Alive, TE
13 RxHeader c Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-
xbitmap, */*;q=0.1
13 RxHeader c Host: www.example.com
13 RxHeader c TE: deflate, gzip, chunked, identity, trailers
13 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
13 RxHeader c User-Agent: Opera/9.51 (Windows NT 5.1; U; en)
13 VCL_call c recv
13 VCL_return c lookup
13 VCL_call c hash
13 VCL_return c hash
13 VCL_call c miss
13 VCL_return c fetch
16 BackendOpen b default 10.0.0.100 39798 10.4.0.40 80
16 BackendXID b 121045868
13 Backend c 16 default
16 TxRequest b GET
16 TxURL b /spacer_thin.gif
16 TxProtocol b HTTP/1.1
16 TxHeader b Referer: http://www.example.com/carchart/index.htm
16 TxHeader b Accept-Language: en-US,en;q=0.9
16 TxHeader b Accept-Encoding: deflate, gzip, x-gzip,
identity, *;q=0
16 TxHeader b X-Cluster-Client-Ip: 10.2.0.200
16 TxHeader b Cookie2: $Version=1
16 TxHeader b Accept: text/html, application/xml;q=0.9,
application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-
xbitmap, */*;q=0.1
16 TxHeader b Host: www.example.com
16 TxHeader b User-Agent: Opera/9.51 (Windows NT 5.1; U; en)
16 TxHeader b X-Varnish: 121045868
16 TxHeader b X-Forwarded-for: 10.3.0.30
16 RxProtocol b HTTP/1.1
16 RxStatus b 200
16 RxResponse b OK
16 RxHeader b Date: Thu, 14 Aug 2008 16:43:35 GMT
16 RxHeader b Server: Apache/2.0.52 (CentOS)
16 RxHeader b Last-Modified: Fri, 02 Feb 2007 02:57:20 GMT
16 RxHeader b ETag: "40d5-2b-808f4400"
16 RxHeader b Accept-Ranges: bytes
16 RxHeader b Content-Length: 43
16 RxHeader b Connection: close
16 RxHeader b Content-Type: image/gif
13 ObjProtocol c HTTP/1.1
13 ObjStatus c 200
13 ObjResponse c OK
13 ObjHeader c Date: Thu, 14 Aug 2008 16:43:35 GMT
13 ObjHeader c Server: Apache/2.0.52 (CentOS)
13 ObjHeader c Last-Modified: Fri, 02 Feb 2007 02:57:20 GMT
13 ObjHeader c ETag: "40d5-2b-808f4400"
13 ObjHeader c Content-Type: image/gif
16 BackendClose b default
13 TTL c 121045868 RFC 1800 1218732215 1218732215 0 0 0
13 VCL_call c fetch
13 VCL_return c insert
13 Length c 43
13 VCL_call c deliver
13 VCL_return c deliver
13 TxProtocol c HTTP/1.1
13 TxStatus c 200
13 TxResponse c OK
13 TxHeader c Server: Apache/2.0.52 (CentOS)
13 TxHeader c Last-Modified: Fri, 02 Feb 2007 02:57:20 GMT
13 TxHeader c ETag: "40d5-2b-808f4400"
13 TxHeader c Content-Type: image/gif
13 TxHeader c Content-Length: 43
13 TxHeader c Date: Thu, 14 Aug 2008 16:43:35 GMT
13 TxHeader c X-Varnish: 121045868
13 TxHeader c Age: 0
13 TxHeader c Via: 1.1 varnish
13 TxHeader c Connection: keep-alive
13 ReqEnd c 121045868 1218732215.300252914
1218732215.390639067 0.005183935 0.090335131 0.000051022
0 StatAddr 10.3.0.30 0 182 4 4 0 0 2 1043 666
0 ExpKill 121045866 -1218732216
13 SessionClose c timeout
13 StatSess c 10.3.0.30 49410 0 1 1 0 0 1 277 43
More information about the varnish-dev
mailing list