compression-at-edge, in front of varnish reverse proxy, gzip-compresses some content, but not other ?
Laurence Rowe
l at lrowe.co.uk
Wed Apr 28 03:40:39 CEST 2010
2010/4/28 Ben DJ <bendj095124367913213465 at gmail.com>:
> nginx compression-at-edge (in front of a reverse proxy) compresses
> some content, not other ?
>
> I've an nginx + varnish + apache2 stack. Nginx serves as redirector,
> ssl handshake, and gzip compression. Varnish serves as a
> reverse-proxy, and apache 'just' hosts apps & serves content.
>
> I'm trying to get gzip compression behaving properly. I'm getting
> intermittent results -- some content seems to be gzipped, some not.
>
> I'm looking for some help figuring out WHERE the problem lies, and
> what to do to fix it.
>
> In nginx conf, I've,
>
> ...
> http {
> ...
> gzip on;
> gzip_http_version 1.0;
> gzip_comp_level 9;
> gzip_proxied any;
> gzip_buffers 16 8k;
> gzip_min_length 0;
> gzip_types text/plain text/css text/xml text/javascript
> application/x-javascript;
> gzip_disable "MSIE [1-6].(?!.*SV1)";
> gzip_vary on;
>
> upstream varnish {
> server 127.0.0.1:8090 weight=10 max_fails=3 fail_timeout=15s;
> }
>
> server {
> listen x.y.z.w:443;
> ...
> location / {
> proxy_pass http://varnish;
> proxy_redirect off;
> proxy_set_header Host $host;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Client-Verify SUCCESS;
> proxy_set_header X-SSL-Subject $ssl_client_s_dn;
> proxy_set_header X-SSL-Issuer $ssl_client_i_dn;
> }
> }
> ...
>
>
> In varnish config, per chat @ #irc, I've replaced,
>
> if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
> # No point in compressing these
> remove req.http.Accept-Encoding;
> } elsif (req.http.Accept-Encoding ~ "gzip") {
> # if the browser supports it, we'll use gzip
> set req.http.Accept-Encoding = "gzip";
> } elsif (req.http.Accept-Encoding ~ "deflate") {
> # next, try deflate if it is supported
> set req.http.Accept-Encoding = "deflate";
> } else {
> # unknown algorithm. Probably junk, remove it
> remove req.http.Accept-Encoding;
> }
>
> with a,
>
> ...
> if (req.http.Accept-Encoding) {
> remove req.http.Accept-Encoding;
> }
> ...
>
> clause, since the compression is to be done only at the nginx 'edge'.
>
> Apache's compression (gzip, default, or otherwise), is completely disabled.
>
> Atm, a check of my test site with YSlow complains that .js's are NOT
> being compressed. Checking with LiveHTTPHeaders Firefox plugin,
> shows, e.g., the .js in question NOT being gzipped, but a .gif *is*,
> e.g.
>
> ...
> ----------------------------------------------------------
>
> https://my.site.com/main/apostrophePlugin/js/jquery.keycodes-0.2.js
>
>
>
> GET /main/apostrophePlugin/js/jquery.keycodes-0.2.js HTTP/1.1
> Host: my.site.com
> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.4)
> Gecko/20100417
> Accept: */*
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Keep-Alive: 115
> Connection: keep-alive
> Referer: https://my.site.com/main/
> Cookie: SESS6fa8cdc2d7064704bbda0c83e2c2588c=94889db68945e19ed6f666b7e00cdd36;
> symfony=3KOH8Qk0hV%2C%2CvXLi0PK5YmdenP1
> If-Modified-Since: Tue, 27 Apr 2010 18:18:18 GMT
> If-None-Match: "362e4-1008-4853810064180"
> Authorization: Digest username="admin", realm="AUTH my.site.com",
> nonce="5GopBDmFBAA=99f7be8796e018dde459a07178393d235366ecd9",
> uri="/main/apostrophePlugin/js/jquery.keycodes-0.2.js", algorithm=MD5,
> response="b04cb995cd1f86a67197aab3b5a5dbc9", qop=auth, nc=000001c9,
> cnonce="9dbeaefee4d57b12"
> Cache-Control: max-age=0
>
>
>
> HTTP/1.1 304 Not Modified
> Server: nginx/0.8.35
> Date: Wed, 28 Apr 2010 00:39:48 GMT
> Connection: keep-alive
> Etag: "362e4-1008-4853beaefee80"
> Expires: Sat, 01 May 2010 00:39:48 GMT
> Cache-Control: max-age=259200
> Content-Length: 0
> X-Varnish: 940462008
> Age: 0
> Via: 1.1 varnish
>
> ----------------------------------------------------------
>
> https://my.site.com/apostrophePlugin/images/a-special-blank.gif
>
> GET /apostrophePlugin/images/a-special-blank.gif HTTP/1.1
> Host: my.site.com
> User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.4)
> Gecko/20100417
> Accept: image/png,image/*;q=0.8,*/*;q=0.5
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Keep-Alive: 115
> Connection: keep-alive
> Referer: https://my.site.com/main/
> Cookie: SESS6fa8cdc2d7064704bbda0c83e2c2588c=94889db68945e19ed6f666b7e00cdd36;
> symfony=3KOH8Qk0hV%2C%2CvXLi0PK5YmdenP1
>
> HTTP/1.1 404 Not Found
> Server: nginx/0.8.35
> Date: Wed, 28 Apr 2010 00:39:48 GMT
> Content-Type: text/html; charset=iso-8859-1
> Transfer-Encoding: chunked
> Connection: keep-alive
> Vary: Accept-Encoding, accept-language,accept-charset
> Content-Language: en
> X-Varnish: 940462009 940461993
> Age: 87
> Via: 1.1 varnish
> Content-Encoding: gzip
> ----------------------------------------------------------
> ...
>
>
> where, iiuc, the presence/absence of "Content-Encoding: gzip" defines
> whether or not the item in question was succesfully gzipped.
That's a 404 Not Found error message being gzipped because it is
text/html (I think Nginx compresses that by default).
> Any ideas why/where the .js is not getting gzipped?
That is a 304 Not Modified response, which has no body, so nothing to
be gzipped.
Try doing a shift-reload in firefox to see what you get from the
original request (which should return 200 OK)
Laurence
More information about the varnish-misc
mailing list