source: man/vcl.7so @ eacd8c3

Revision eacd8c3, 20.3 KB checked in by Tollef Fog Heen <tfheen@…>, 4 years ago (diff)

Document that TTL is no longer taken into account for beresp.cacheable

git-svn-id:  http://www.varnish-cache.org/svn/trunk/varnish-cache@4669 d4fa192b-c00b-0410-8231-f00ffab90ce4

  • Property mode set to 100644
Line 
1.\"-
2.\" Copyright (c) 2006 Verdens Gang AS
3.\" Copyright (c) 2006-2009 Linpro AS
4.\" All rights reserved.
5.\"
6.\" Author: Dag-Erling Smørgrav <des@des.no>
7.\"
8.\" Redistribution and use in source and binary forms, with or without
9.\" modification, are permitted provided that the following conditions
10.\" are met:
11.\" 1. Redistributions of source code must retain the above copyright
12.\"    notice, this list of conditions and the following disclaimer.
13.\" 2. Redistributions in binary form must reproduce the above copyright
14.\"    notice, this list of conditions and the following disclaimer in the
15.\"    documentation and/or other materials provided with the distribution.
16.\"
17.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
18.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
20.\" ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
21.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
22.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
23.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
24.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
25.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
26.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
27.\" SUCH DAMAGE.
28.\"
29.\" $Id$
30.\"
31.Dd August 10, 2007
32.Dt VCL 7
33.Os
34.Sh NAME
35.Nm VCL
36.Nd Varnish Configuration Language
37.Sh DESCRIPTION
38The
39.Nm
40language is a small domain-specific language designed to be used to
41define request handling and document caching policies for the Varnish
42HTTP accelerator.
43.Pp
44When a new configuration is loaded, the
45.Nm varnishd
46management process translates the VCL code to C and compiles it to a
47shared object which is then dynamically linked into the server
48process.
49.Ss Syntax
50The VCL syntax is very simple, and deliberately similar to C and Perl.
51Blocks are delimited by curly braces, statements end with semicolons,
52and comments may be written as in C, C++ or Perl according to your own
53preferences.
54.Pp
55In addition to the C-like assignment (=), comparison (==) and boolean
56(!, && and ||) operators, VCL supports regular expression and ACL
57matching using the ~ operator.
58.Pp
59Unlike C and Perl, the backslash (\e) character has no special meaning
60in strings in VCL, which use the (%xx) escape mechanism just like URLs,
61so it can be freely used in regular expressions without doubling.
62.Pp
63Strings are concatenated by just putting them one after each other
64without any operator in between.
65.Pp
66Assignments are introduced with the
67.Cm set
68keyword.
69There are no user-defined variables; values can only be assigned to
70variables attached to backend, request or document objects.
71Most of these are typed, and the values assigned to them must have a
72compatible unit suffix.
73.Pp
74VCL has
75.Cm if
76tests, but no loops.
77.Pp
78The contents of another VCL file may be inserted at any point in the
79code by using the
80.Cm include
81keyword followed by the name of the other file as a quoted string.
82.Ss Backend declarations
83A backend declaration creates and initializes a named backend object:
84.Bd -literal -offset 4n
85backend www {
86    .host = "www.example.com";
87    .port = "http";
88}
89.Ed
90.Pp
91The backend object can later be used to select a backend at request
92time:
93.Bd -literal -offset 4n
94if (req.http.host ~ "^(www\.)?example.com$") {
95    set req.backend = www;
96}
97.Ed
98.Pp
99To avoid overloading backend servers,
100.Fa .max_connections
101can be set to limit the maximum number of concurrent backend connections.
102.Pp
103The timeout parameters can be overridden in the backend declaration.
104The timeout parameters are
105.Fa .connect_timeout
106for the time to wait for a backend connection,
107.Fa .first_byte_timeout
108for the time to wait for the first byte from the backend and
109.Fa .between_bytes_timeout
110for time to wait between each received byte.
111.Pp
112These can be set in the declaration like this:
113.Bd -literal -offset 4n
114backend www {
115    .host = "www.example.com";
116    .port = "http";
117    .connect_timeout = 1s;
118    .first_byte_timeout = 5s;
119    .between_bytes_timeout = 2s;
120}
121.Ed
122.Pp
123To mark a backend as unhealthy after number of items have been added to
124it's saintmode list
125.Fa .saintmode_threshold
126can be set to the maximum list size. Setting a value of 0 disables
127saintmode checking entirely for that backend. The value in the backend
128declaration overrides the parameter.
129
130.Ss Directors
131Directors choose from different backends based on health status and a
132per-director algorithm.
133There currently exists a round-robin and a random director.
134.Pp
135Directors are defined using:
136.Bd -literal -offset 4n
137director b2 random {
138    .retries = 5;
139    {
140        /* We can refer to named backends */
141        .backend        = b1;
142        .weight         = 7;
143    }
144    {
145        /* Or define them inline */
146        .backend        = {
147            .host = "fs2";
148        }
149        .weight         = 3;
150    }
151}
152.Ed
153.Ss The random director
154The random director takes one per-director option
155.Fa .retries .
156This specifies how many tries it will use to find a working backend.
157The default is the same as the number of backends defined for the
158director.
159.Pp
160There is also a per-backend option: weight which defines the portion
161of traffic to send to the particular backend.
162.Ss The round-robin director
163The round-robin does not take any options.
164.Ss Backend probes
165Backends can be probed to see whether they should be considered
166healthy or not.  The return status can also be checked by using
167.Fa req.backend.healthy
168.
169.Fa .window
170is how many of the latest polls we examine, while
171.Fa .threshold
172is how many of those must have succeeded for us to consider the
173backend healthy.
174.Fa .initial
175is how many of the probes are considered good when Varnish starts -
176defaults to the same amount as the threshold.
177.Bd -literal -offset 4n
178backend www {
179    .host = "www.example.com";
180    .port = "http";
181    .probe = {
182        .url = "/test.jpg";
183        .timeout = 0.3 s;
184        .window = 8;
185        .threshold = 3;
186        .initial = 3;
187    }
188}
189.Ed
190It is also possible to specify the raw HTTP request.
191.Bd -literal -offset 4n
192backend www {
193    .host = "www.example.com";
194    .port = "http";
195    .probe = {
196        # NB: \er\en automatically inserted after each string!
197        .request =
198            "GET / HTTP/1.1"
199            "Host: www.foo.bar"
200            "Connection: close";
201    }
202}
203.Ed
204.Ss ACLs
205An ACL declaration creates and initializes a named access control list
206which can later be used to match client addresses:
207.Bd -literal -offset 4n
208acl local {
209    "localhost";         /* myself */
210    "192.0.2.0"/24;      /* and everyone on the local network */
211    ! "192.0.2.23";      /* except for the dialin router */
212}
213.Ed
214.Pp
215If an ACL entry specifies a host name which Varnish is unable to
216resolve, it will match any address it is compared to.
217Consequently, if it is preceded by a negation mark, it will reject any
218address it is compared to, which may not be what you intended.
219If the entry is enclosed in parentheses, however, it will simply be
220ignored.
221.Pp
222To match an IP address against an ACL, simply use the match operator:
223.Bd -literal -offset 4n
224if (client.ip ~ local) {
225    pipe;
226}
227.Ed
228.Ss Grace
229If the backend takes a long time to generate an object there is a risk
230of a thread pile up.
231In order to prevent this you can enable grace.
232This allows varnish to serve an expired version of the object while a
233fresh object is being generated by the backend.
234.Pp
235The following vcl code will make Varnish serve expired objects.
236All object will be kept up to two minutes past their expiration time
237or a fresh object is generated.
238.Bd -literal -offset 4n
239sub vcl_recv {
240    set req.grace = 2m;
241}
242sub vcl_fetch {
243    set obj.grace = 2m;
244}
245.Ed
246.Ss Functions
247The following built-in functions are available:
248.Bl -tag -width indent
249.It Fn regsub "str" "regex" "sub"
250Returns a copy of
251.Fa str
252with the first occurrence of the regular expression
253.Fa regex
254replaced with
255.Fa sub .
256Within
257.Fa sub ,
258.Va \e0
259(which can also be spelled
260.Va & )
261is replaced with the entire matched string, and
262.Va \en
263is replaced with the contents of subgroup
264.Ar n
265in the matched string.
266.It Fn regsuball "str" "regex" "sub"
267As
268.Fn regsuball
269but this replaces all occurrences.
270.It Fn purge_url "regex"
271Purge all objects in cache whose URLs match
272.Fa regex .
273.El
274.Ss Subroutines
275A subroutine is used to group code for legibility or reusability:
276.Bd -literal -offset 4n
277sub pipe_if_local {
278    if (client.ip ~ local) {
279        pipe;
280    }
281}
282.Ed
283.Pp
284Subroutines in VCL do not take arguments, nor do they return values.
285.Pp
286To call a subroutine, use the
287.Cm call
288keyword followed by the subroutine's name:
289.Bd -literal -offset 4n
290call pipe_if_local;
291.Ed
292.Pp
293There are a number of special subroutines which hook into the Varnish
294workflow.
295These subroutines may inspect and manipulate HTTP headers and various
296other aspects of each request, and to a certain extent decide how the
297request should be handled.
298Each subroutine terminates by calling one of a small number of
299keywords which indicates the desired outcome.
300.Bl -tag -width indent
301.\" vcl_recv
302.It Cm vcl_recv
303Called at the beginning of a request, after the complete request has
304been received and parsed.
305Its purpose is to decide whether or not to serve the request, how to
306do it, and, if applicable, which backend to use.
307.Pp
308The
309.Cm vcl_recv
310subroutine may terminate with one of the following keywords:
311.Bl -tag -width indent
312.It Cm error Ar code Op Ar reason
313Return the specified error code to the client and abandon the
314request.
315.It Cm pass
316Switch to pass mode.
317Control will eventually pass to
318.Cm vcl_pass .
319.It Cm pipe
320Switch to pipe mode.
321Control will eventually pass to
322.Cm vcl_pipe .
323.It Cm lookup
324Look up the requested object in the cache.
325Control will eventually pass to
326.Cm vcl_hit
327or
328.Cm vcl_miss ,
329depending on whether the object is in the cache.
330.El
331.\" vcl_pipe
332.It Cm vcl_pipe
333Called upon entering pipe mode.
334In this mode, the request is passed on to the backend, and any further
335data from either client or backend is passed on unaltered until either
336end closes the connection.
337.Pp
338The
339.Cm vcl_pipe
340subroutine may terminate with one of the following keywords:
341.Bl -tag -width indent
342.It Cm error Ar code Op Ar reason
343Return the specified error code to the client and abandon the
344request.
345.It Cm pipe
346Proceed with pipe mode.
347.El
348.\" vcl_pass
349.It Cm vcl_pass
350Called upon entering pass mode.
351In this mode, the request is passed on to the backend, and the
352backend's response is passed on to the client, but is not entered into
353the cache.
354Subsequent requests submitted over the same client connection are
355handled normally.
356.Pp
357The
358.Cm vcl_pass
359subroutine may terminate with one of the following keywords:
360.Bl -tag -width indent
361.It Cm error Ar code Op Ar reason
362Return the specified error code to the client and abandon the
363request.
364.It Cm pass
365Proceed with pass mode.
366.El
367.\" vcl_hash
368.It Cm vcl_hash
369Use
370.Cm req.hash += req.http.Cookie
371or similar to include the Cookie HTTP header in the hash string.
372The
373.Cm vcl_hash
374subroutine may terminate with one of the following keywords:
375.Bl -tag -width indent
376.It Cm hash
377Proceed.
378.El
379.\" vcl_hit
380.It Cm vcl_hit
381Called after a cache lookup if the requested document was found in the
382cache.
383.Pp
384The
385.Cm vcl_hit
386subroutine may terminate with one of the following keywords:
387.Bl -tag -width indent
388.It Cm error Ar code Op Ar reason
389Return the specified error code to the client and abandon the
390request.
391.It Cm pass
392Switch to pass mode.
393Control will eventually pass to
394.Cm vcl_pass .
395.It Cm deliver
396Deliver the cached object to the client.
397Control will eventually pass to
398.Cm vcl_deliver .
399.El
400.\" vcl_miss
401.It Cm vcl_miss
402Called after a cache lookup if the requested document was not found in
403the cache.
404Its purpose is to decide whether or not to attempt to retrieve the
405document from the backend, and which backend to use.
406.Pp
407The
408.Cm vcl_miss
409subroutine may terminate with one of the following keywords:
410.Bl -tag -width indent
411.It Cm error Ar code Op Ar reason
412Return the specified error code to the client and abandon the
413request.
414.It Cm pass
415Switch to pass mode.
416Control will eventually pass to
417.Cm vcl_pass .
418.It Cm fetch
419Retrieve the requested object from the backend.
420Control will eventually pass to
421.Cm vcl_fetch .
422.El
423.\" vcl_fetch
424.It Cm vcl_fetch
425Called after a document has been successfully retrieved from the
426backend.
427.Pp
428The
429.Cm vcl_fetch
430subroutine may terminate with one of the following keywords:
431.Bl -tag -width indent
432.It Cm error Ar code Op Ar reason
433Return the specified error code to the client and abandon the
434request.
435.It Cm pass
436Switch to pass mode.
437Control will eventually pass to
438.Cm vcl_pass .
439.It Cm deliver
440Possibly insert the object into the cache, then deliver it to the client.
441Control will eventually pass to
442.Cm vcl_deliver .
443.It Cm esi
444ESI-process the document which has just been fetched.
445.El
446.\" vcl_deliver
447.It Cm vcl_deliver
448Called before a cached object is delivered to the client.
449.Pp
450The
451.Cm vcl_deliver
452subroutine may terminate with one of the following keywords:
453.Bl -tag -width indent
454.It Cm error Ar code Op Ar reason
455Return the specified error code to the client and abandon the
456request.
457.It Cm deliver
458Deliver the object to the client.
459.El
460.El
461.Pp
462If one of these subroutines is left undefined or terminates without
463reaching a handling decision, control will be handed over to the
464builtin default.
465See the
466.Sx EXAMPLES
467section for a listing of the default code.
468.Ss Multiple subroutines
469If multiple subroutines with the same name are defined, they are
470concatenated in the order in which the appear in the source.
471.Pp
472Example:
473.Bd -literal -offset 4n
474# in file "main.vcl"
475include "backends.vcl";
476include "purge.vcl";
477
478# in file "backends.vcl"
479sub vcl_recv {
480  if (req.http.host ~ "example.com") {
481    set req.backend = foo;
482  } elsif (req.http.host ~ "example.org") {
483    set req.backend = bar;
484  }
485}
486
487# in file "purge.vcl"
488sub vcl_recv {
489  if (client.ip ~ admin_network) {
490    if (req.http.Cache-Control ~ "no-cache") {
491      purge_url(req.url);
492    }
493  }
494}
495.Ed
496.Pp
497The builtin default subroutines are implicitly appended in this way.
498.Ss Variables
499Although subroutines take no arguments, the necessary information is
500made available to the handler subroutines through global variables.
501.Pp
502The following variables are always available:
503.Bl -tag -width 4n
504.It Va now
505The current time, in seconds since the epoch.
506.El
507.Pp
508The following variables are available in backend declarations:
509.Bl -tag -width 4n
510.It Va .host
511Host name or IP address of a backend.
512.It Va .port
513Service name or port number of a backend.
514.El
515.Pp
516The following variables are available while processing a request:
517.Bl -tag -width 4n
518.It Va client.ip
519The client's IP address.
520.It Va server.hostname
521The host name of the server.
522.It Va server.identity
523The identity of the server, as set by the
524.Fl i
525parameter.
526If the
527.Fl i
528parameter is not passed to
529.Nm varnishd ,
530.Va server.identity
531will be set to the name of the instance, as specified by the
532.Fl n
533parameter.
534.It Va server.ip
535The IP address of the socket on which the client connection was
536received.
537.It Va server.port
538The port number of the socket on which the client connection was
539received.
540.It Va req.request
541The request type (e.g. "GET", "HEAD").
542.It Va req.url
543The requested URL.
544.It Va req.proto
545The HTTP protocol version used by the client.
546.It Va req.backend
547The backend to use to service the request.
548.It Va req.backend.healthy
549Whether the backend is healthy or not.
550.It Va req.http. Ns Ar header
551The corresponding HTTP
552.Ar header .
553.El
554.Pp
555The following variables are available while preparing a backend
556request (either for a cache miss or for pass or pipe mode):
557.Bl -tag -width 4n
558.It Va bereq.request
559The request type (e.g. "GET", "HEAD").
560.It Va bereq.url
561The requested URL.
562.It Va bereq.proto
563The HTTP protocol version used to talk to the server.
564.It Va bereq.http. Ns Ar header
565The corresponding HTTP
566.Ar header .
567.It Va bereq.connect_timeout
568The time in seconds to wait for a backend connection.
569.It Va bereq.first_byte_timeout
570The time in seconds to wait for the first byte from the backend.
571Not available in pipe mode.
572.It Va bereq.between_bytes_timeout
573The time in seconds to wait between each received byte from the backend.
574Not available in pipe mode.
575.El
576.Pp
577The following variables are available after the requested object has
578been retrieved from cache or from the backend:
579.Bl -tag -width 4n
580.It Va obj.proto
581The HTTP protocol version used when the object was retrieved.
582.It Va obj.status
583The HTTP status code returned by the server.
584.It Va obj.response
585The HTTP status message returned by the server.
586.It Va obj.cacheable
587True if the request resulted in a cacheable response.
588.\" see cache_center.c and rfc2616.c for details
589A response is considered cacheable if it is valid (see above), and the
590HTTP status code is 200, 203, 300, 301, 302, 404 or 410.
591.It Va obj.ttl
592The object's remaining time to live, in seconds.
593.It Va obj.lastuse
594The approximate time elapsed since the object was last requests, in
595seconds.
596.It Va obj.hits
597The approximate number of times the object has been delivered. A value of 0
598indicates a cache miss.
599.El
600.Pp
601The following variables are available while determining the hash key
602of an object:
603.Bl -tag -width 4n
604.It Va req.hash
605The hash key used to refer to an object in the cache.  Used when both
606reading from and writing to the cache.
607.El
608.Pp
609The following variables are available while preparing a response to
610the client:
611.Bl -tag -width 4n
612.It Va resp.proto
613The HTTP protocol version to use for the response.
614.It Va resp.status
615The HTTP status code that will be returned.
616.It Va resp.response
617The HTTP status message that will be returned.
618.It Va resp.http. Ns Ar header
619The corresponding HTTP
620.Ar header .
621.El
622.Pp
623Values may be assigned to variables using the
624.Cm set
625keyword:
626.Bd -literal -offset 4n
627sub vcl_recv {
628    # Normalize the Host: header
629    if (req.http.host ~ "^(www\.)?example\.com$") {
630        set req.http.host = "www.example.com";
631    }
632}
633.Ed
634.Pp
635HTTP headers can be removed entirely using the
636.Cm remove
637keyword:
638.Bd -literal -offset 4n
639sub vcl_fetch {
640    # Don't cache cookies
641    remove obj.http.Set-Cookie;
642}
643.Ed
644.Sh EXAMPLES
645The following code is the equivalent of the default configuration with
646the backend address set to "backend.example.com" and no backend port
647specified.
648.\" Keep this in synch with bin/varnishd/mgt_vcc.c and etc/default.vcl
649.Bd -literal -offset 4n
650backend default {
651    .host = "backend.example.com";
652    .port = "http";
653}
654
655.so default.vcl
656.Ed
657.Pp
658The following example shows how to support multiple sites running on
659separate backends in the same Varnish instance, by selecting backends
660based on the request URL.
661.Bd -literal -offset 4n
662backend www {
663    .host = "www.example.com";
664    .port = "80";
665}
666
667backend images {
668    .host = "images.example.com";
669    .port = "80";
670}
671
672sub vcl_recv {
673    if (req.http.host ~ "^(www\.)?example\.com$") {
674        set req.http.host = "www.example.com";
675        set req.backend = www;
676    } elsif (req.http.host ~ "^images\.example\.com$") {
677        set req.backend = images;
678    } else {
679        error 404 "Unknown virtual host";
680    }
681}
682.Ed
683.Pp
684The following snippet demonstrates how to force a minimum TTL for all
685documents.
686Note that this is not the same as setting the
687.Va default_ttl
688run-time parameter, as that only affects document for which the
689backend did not specify a TTL.
690.Bd -literal -offset 4n
691sub vcl_fetch {
692    if (obj.ttl < 120s) {
693        set obj.ttl = 120s;
694    }
695}
696.Ed
697.Pp
698The following snippet demonstrates how to force Varnish to cache
699documents even when cookies are present.
700.Bd -literal -offset 4n
701sub vcl_recv {
702    if (req.request == "GET" && req.http.cookie) {
703        lookup;
704    }
705}
706
707sub vcl_fetch {
708    if (obj.http.Set-Cookie) {
709        deliver;
710    }
711}
712.Ed
713.Pp
714The following code implements the HTTP PURGE method as used by Squid
715for object invalidation:
716.Bd -literal -offset 4n
717acl purge {
718        "localhost";
719        "192.0.2.1"/24;
720}
721
722sub vcl_recv {
723    if (req.request == "PURGE") {
724        if (!client.ip ~ purge) {
725            error 405 "Not allowed.";
726        }
727        lookup;
728    }
729}
730
731sub vcl_hit {
732    if (req.request == "PURGE") {
733        set obj.ttl = 0s;
734        error 200 "Purged.";
735    }
736}
737
738sub vcl_miss {
739    if (req.request == "PURGE") {
740        error 404 "Not in cache.";
741    }
742}
743.Ed
744.Sh SEE ALSO
745.Xr varnishd 1
746.Sh HISTORY
747The
748.Nm
749language was developed by
750.An Poul-Henning Kamp Aq phk@phk.freebsd.dk
751in cooperation with Verdens Gang AS and Linpro AS.
752This manual page was written by
753.An Dag-Erling Sm\(/orgrav Aq des@des.no .
Note: See TracBrowser for help on using the repository browser.