This is the first beta release of the upcoming 5.0 release.
The list of changes are numerous and will not be expanded on in detail.
The release notes contain more background information and are highly recommended reading before using any of the new features.
Major items:
Changes between 4.0 and 4.1 are numerous. Please read the upgrade section in the documentation for a general overview.
New since 4.0.2-rc1:
New since 4.0.1:
New since 4.0.0:
New since 4.0.0-beta1:
New since TP2:
varnishsizes -----------~
Persistent storage is now experimentally supported using the persistent stevedore. It has the same command line arguments as the file stevedore.
obj.* is now called beresp.* in vcl_fetch, and obj.* is now read-only.
The regular expression engine is now PCRE instead of POSIX regular expressions.
req.* is now available in vcl_deliver.
Add saint mode where we can attempt to grace an object if we don't like the backend response for some reason.
Related, add saintmode_threshold which is the threshold for the number of objects to be added to the trouble list before the backend is considered sick.
Add a new hashing method called critbit. This autoscales and should work better on large object workloads than the classic hash. Critbit has been made the default hash algorithm.
When closing connections, we experimented with sending RST to free up load balancers and free up threads more quickly. This caused some problems with NAT routers and so has been reverted for now.
Add thread that checks objects against ban list in order to prevent ban list from growing forever. Note that this needs purges to be written so they don't depend on req.*. Enabled by setting ban_lurker_sleep to a nonzero value.
The shared memory log file format was limited to maximum 64k simultaneous connections. This is now a 32 bit field which removes this limitation.
Remove obj_workspace, this is now sized automatically.
Rename acceptors to waiters
vcl_prefetch has been removed. It was never fully implemented.
Add support for authenticating CLI connections.
Add hash director that chooses which backend to use depending on req.hash.
Add client director that chooses which backend to use depending on the client's IP address. Note that this ignores the X-Forwarded-For header.
varnishd now displays a banner by default when you connect to the CLI.
Increase performance somewhat by moving statistics gathering into a per-worker structure that is regularly flushed to the global stats.
Make sure we store the header and body of object together. This may in some cases improve performance and is needed for persistence.
Remove client-side address accounting. It was never used for anything and presented a performance problem.
Add a timestamp to bans, so you can know how old they are.
Quite a few people got confused over the warning about not being able to lock the shared memory log into RAM, so stop warning about that.
Change the default CLI timeout to 10 seconds.
We previously forced all inserts into the cache to be GET requests. This has been changed to allow POST as well in order to be able to implement purge-on-POST semantics.
The CLI command stats now only lists non-zero values.
The CLI command stats now only lists non-zero values.
Use daemon(3) from libcompat on Darwin.
Remove vcl_discard as it causes too much complexity and never actually worked particularly well.
Remove vcl_timeout as it causes too much complexity and never actually worked particularly well.
Update the documentation so it refers to sess_workspace, not http_workspace.
Document the -i switch to varnishd as well as the server.identity and server.hostname VCL variables.
purge.hash is now deprecated and no longer shown in help listings.
When processing ESI, replace the five mandatory XML entities when we encounter them.
Add string representations of time and relative time.
Add locking for n_vbe_conn to make it stop underflowing.
When ESI-processing content, check for illegal XML character entities.
Varnish can now connect its CLI to a remote instance when starting up, rather than just being connected to.
It is no longer needed to specify the maximum number of HTTP headers to allow from backends. This is now a run-time parameter.
The X-Forwarded-For header is now generated by vcl_recv rather than the C code.
It is now possible to not send all CLI traffic to syslog.
It is now possible to not send all CLI traffic to syslog.
In the case of varnish crashing, it now outputs a identifying string with the OS, OS revision, architecture and storage parameters together with the backtrace.
Use exponential backoff when we run out of file descriptors or sessions.
Allow setting backend timeouts to zero.
Count uptime in the shared memory log.
Try to detect the case of two running varnishes with the same shmlog and storage by writing the master and child process ids to the shmlog and refusing to start if they are still running.
Make sure to use EOF mode when serving ESI content to HTTP/1.0 clients.
Make sure we close the connection if it either sends Connection: close or it is a HTTP/1.0 backend that does not send Connection: keep-alive.
Increase the default session workspace to 64k on 64-bit systems.
Make the epoll waiter use level triggering, not edge triggering as edge triggering caused problems on very busy servers.
Handle unforeseen client disconnections better on Solaris.
Make session lingering apply to new sessions, not just reused sessions.
VCL Manual page --------------~
Red Hat spec file ----------------~
Build system -----------~
Build system -----------~
Build system -----------~
The request workflow has been redesigned to simplify request processing and eliminate code duplication. All codepaths which need to speak HTTP now share a single implementation of the protocol. Some new VCL hooks have been added, though they aren't much use yet. The only real user-visible change should be that Varnish now handles persistent backend connections correctly (see ticket #56).
Support for multiple listen addresses has been added.
An "include" facility has been added to VCL, allowing VCL code to pull in code fragments from multiple files.
Multiple definitions of the same VCL function are now concatenated into one in the order in which they appear in the source. This simplifies the mechanism for falling back to the built-in default for cases which aren't handled in custom code, and facilitates modularization.
The code used to format management command arguments before passing them on to the child process would underestimate the amount of space needed to hold each argument once quotes and special characters were properly escaped, resulting in a buffer overflow. This has been corrected.
The VCL compiler has been overhauled. Several memory leaks have been plugged, and error detection and reporting has been improved throughout. Parts of the compiler have been refactored to simplify future extension of the language.
A bug in the VCL compiler which resulted in incorrect parsing of the decrement (-=) operator has been fixed.
A new -C command-line option has been added which causes varnishd to compile the VCL code (either from a file specified with -f or the built-in default), print the resulting C code and exit.
When processing a backend response using chunked encoding, if a chunk header crosses a read buffer boundary, read additional bytes from the backend connection until the chunk header is complete.
A new ping_interval run-time parameter controls how often the management process checks that the worker process is alive.
A bug which would cause the worker process to dereference a NULL pointer and crash if the backend did not respond has been fixed.
In some cases, such as when they are used by AJAX applications to circumvent Internet Explorer's over-eager disk cache, it may be desirable to cache POST requests. However, the code path responsible for delivering objects from cache would only transmit the response body when replying to a GET request. This has been extended to also apply to POST.
This should be revisited at a later date to allow VCL code to control whether the body is delivered.
Varnish now respects Cache-control: s-maxage, and prefers it to Cache-control: max-age if both are present.
This should be revisited at a later date to allow VCL code to control which headers are used and how they are interpreted.
When loading a new VCL script, the management process will now load the compiled object to verify that it links correctly before instructing the worker process to load it.
A new -P command-line options has been added which causes varnishd to create a PID file.
The sendfile_threshold run-time parameter's default value has been set to infinity after a variety of sendfile()-related bugs were discovered on several platforms.
The formatting callback has been largely rewritten for clarity, robustness and efficiency.
If a request included a Host: header, construct and output an absolute URL. This makes varnishncsa output from servers which handle multiple virtual hosts far more useful.
The flag that is raised upon reception of a SIGHUP has been marked volatile so it will not be optimized away by the compiler.