The Varnish Command Line Interface

Varnish has a Command Line Interface, a "CLI", which can control and change most of the operational parameters of varnish, without interrupting service.

If you change your VCL code for instance, you can load the new version using the CLI:

vcl.load test_vcl "/etc/test_vcl.vcl"
200 13
VCL compiled.

vcl.use test_vcl
200 0

Varnish response to a CLI command always has the same format, a integer status, a count of characters, and the explanation they count.

Like HTTP, a status of "200" means OK.

Configuring the CLI

The Varnishd CLI can be accessed multiple ways and by multiple connections at the same time, but the commands will executed sequentially so that only one command is ever being processed at the same time.

Debug CLI

If varnishd is started with -d option, the standard input/output of the manager process will be a CLI connection, into which you can type CLI commands from your keyboard.

Server CLI

If varnishd is started with -T <address>:<port> argument, it will listen for TCP connections at the given port, and handle CLI commands sent over these connections.

Master CLI

Varnishd can also be started with -M <address>:<port> argument, in which case varnishd will try to open a TCP connection to the address given, in order to receive CLI commands from there. The intended use for this, is to have a central "cache cluster controller" taking care of a bunch of varnish instances, for instance making sure they all receive the same VCL, purge requests etc. (So far, we have still not gotten around to write code for that).

Securing CLI

Obviously, you want to secure your CLI connections, so that not every Tom, Dick & Harry can amuse themselves with sending CLI commands to your varnishd process.

The way this works is very simple and very flexible: You give varnishd a -S somefile argument, and varnish will issue a cryptographic challenge on Server & Master CLI connections, which can only be replied correctly to, if you know the contents of that file. (Don't worry about the cryptographic calculations, the varnishadm program will do that for you.)

This principle is called "shared secret" authentication, and it is probably the most efficient form of authentication in the world, provided you can keep the secret a secret.

The contents of the secret file is only read at the moment somebody tries to authenticate, so you can change it as often as you want, without restarting varnishd, and the contents can be any sequence of bytes you may care for, from "foo" to the Illiad in the original ancient greek, using utf8.

On the local system, you can use the UNIX mode/owner/group on the secret file to control who gets access, for instance by creating a special group "varnish" of people who has CLI priviledges, and letting that group own the file with read permission. Obviously, the varnishd manager process needs to be able to read the file as well.

CLI access from a remote system, can be done many ways, but the two most obvious are:

If you have a copy of the secret file on the remote machine, you can use varnishadm directly. This gives you an authenticated connection, but it is not a private connection: anybody who can snoop on your packets can see which CLI commands you send. Thanks to the cryptographic challenge, an eavesdropper can not see what the secret file contains, for its content is never sent across the CLI connection, only a calculated cryptographic hash is sent, which proves that you know what the file contains. and the challeng issued by varnishd is unique for each attempt, so replays are not possible.

The other way, is to use ssh(1) or similar, to run varnishadm on the local machine, this obviously gives you all of the advantages of ssh, including password-less authentication, privacy etc. etc. etc.

Scripting the CLI

If you are going to write a script that talks CLI to varnishd, the include/cli.h contains the relevant magic numbers.

One particular magic number to know, is that the line with the status code and length field always is exactly 13 characters long, including the NL character.

For your reference the sourcefile lib/libvarnish/cli_common.h contains the functions varnish code uses to read and write CLI response.

Authentication: The gory details

If the -S secret-file is given as argument to varnishd, all network CLI connections must authenticate, by proving they know the contents of that file.

The file is read at the time the auth command is issued and the contents is not cached in varnishd, so it is possible to update the file on the fly.

Use the unix file permissions to control access to the file.

An authenticated session looks like this:

critter phk> telnet localhost 1234
Trying ::1...
Connected to localhost.
Escape character is '^]'.
107 59      

Authentication required.

auth 455ce847f0073c7ab3b1465f74507b75d3dc064c1e7de3b71e00de9092fdc89a
200 193     
Varnish HTTP accelerator CLI.
Type 'help' for command list.
Type 'quit' to close CLI session.
Type 'start' to launch worker process.

The CLI status of 107 indicates that authentication is necessary. The first 32 characters of the reponse text is the challenge "ixsl...mpg". The challenge is randomly generated for each CLI connection, and changes each time a 107 is emitted.

The most recently emitted challenge must be used for calculating the authenticator "455c...c89a".

The authenticator is calculated by applying the SHA256 function to the following byte sequence:

  • Challenge string
  • Newline (0x0a) character.
  • Contents of the secret file
  • Challenge string
  • Newline (0x0a) character.

and dumping the resulting digest in lower-case hex.

In the above example, the secret file contained foo\n and thus:

critter phk> cat > _
critter phk> hexdump -C _
00000000  69 78 73 6c 76 76 78 72  67 6b 6a 70 74 78 6d 63  |ixslvvxrgkjptxmc|
00000010  67 6e 6e 73 64 78 73 76  64 6d 76 66 79 6d 70 67  |gnnsdxsvdmvfympg|
00000020  0a 66 6f 6f 0a 69 78 73  6c 76 76 78 72 67 6b 6a  |.foo.ixslvvxrgkj|
00000030  70 74 78 6d 63 67 6e 6e  73 64 78 73 76 64 6d 76  |ptxmcgnnsdxsvdmv|
00000040  66 79 6d 70 67 0a                                 |fympg.|
critter phk> sha256 _ 
SHA256 (_) = 455ce847f0073c7ab3b1465f74507b75d3dc064c1e7de3b71e00de9092fdc89a
critter phk> openssl dgst -sha256 < _

The sourcefile lib/libvarnish/cli_auth.c contains a useful function which calculates the response, given an open filedescriptor to the secret file, and the challenge string.

Example implementation in python:

import hashlib

challenge = "ixslvvxrgkjptxmcgnnsdxsvdmvfympg"
secret = "foo\n"
print hashlib.sha256(challenge + "\n" + secret + challenge + "\n").hexdigest()