mod_deflate and Apache 2.0.x

In a previous paper, the use of mod_gzip to dynamically compress the output from an Apache server was described. With the growing use of the Apache 2.0.x family of Web servers, the question arises of how to perform a similar GZIP-encoding function within this server. The developers of the Apache 2.0.x servers have included a module in the codebase for the server to perform just this task.

mod_deflate is included in the Apache 2.0.x source package, and compiling it in is a simple matter of adding it to the configure command.

./configure --enable-modules=all --enable-mods-shared=all --enable-deflate

When the server is made and installed, the GZIP-encoding of documents can be enabled in one of two ways: explicit exclusion of files by extension; or by explcit inclusion of files by MIME type. These methods are specified in the httpd.conf file.

Explicit Exclusion

SetOutputFilter DEFLATE
DeflateFilterNote ratio
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

Explicit Inclusion

DeflateFilterNote ratio
AddOutputFilterByType DEFLATE text/*
AddOutputFilterByType DEFLATE application/ms* application/vnd* application/postscript

Both methods enable the automatic GZIP-encoding of all MIME-types, except image and PDF files, as they leave the server. Image files and PDF files are excluded as they are already in a highly compressed format. In fact, PDFs become unreadable by Adobe's Acrobat Reader if they are further compressed by mod_deflate or mod_gzip.

On the server used for testing mod_deflate for this article, no Windows executables or compressed files are served to visitors. However, for safety's sake, please ensure that compressed files and binaries are not GZIP-encoded by your Web server application.

For the file-types indicated in the exclude statements, the server is told explicitly not to send the Vary header. The Vary header indicates to any proxy or cache server which particular condition(s) will cause this response to Vary from other responses to the same request.

If a client sends a request which does not include the Accept-Encoding: gzip header, then the item which is stored in the cache cannot be returned to the requesting client if the Accept-Encoding headers do not match. The request must then be passed directly to the origin server to obtain a non-encoded version. In effect, proxy servers may store 2 or more copies of the same file, depending on the client request conditions which cause the server response to Vary.

Removing the Vary response requirement for objects not handled means that if the objects do not vary due to any other directives on the server (browser type, for example), then the cached object can be served up without any additional requests until the Time-To-Live (TTL) of the cached object has expired.

In examing the performance of mod_deflate against mod_gzip, the one item that seems to distinguish mod_deflate from mod_gzip is the compression algorithm used. The mod_deflate algorithm uses the ZLIB compression library and the functions in this library do not seem to be as effective at compressing files in real time as the GZIP functions used for mod_gzip. The examples below demonstrate that the compression algorithm for mod_gzip produces between 4-6% more compression than mod_deflate for the same file.[1]

Table 1 — /compress/homepage2.html

Table 2 — /documents/spierzchala-resume.ps

Attempts to increase the compression ratio of mod_deflate using the directives provided for this module produced no further decrease in transferred file size. A comment from one of the authors mod_deflate states that the module was written specifically to ensure that server performance was not degraded by using this compression method. With future releases of this module, the authors of mod_deflate may want to compare their algorithm to the one used in mod_gzip to see if there are ways to improve the achieved compression ratio in mod_deflate without compromising server performance.

Despite the fact that the compression algorithm is not as effective as the one found in mod_gzip, using mod_deflate for Apache 2.0.x is still a quick and effective way to decrease the size of the files that sent to clients. Anything that can produce between 50% and 80% in bandwidth savings with so little effort should definitely be considered for any and all Apache 2.0.x deployments wishing to use the default Apache codebase.

A note on the compression in mod_deflate:[1]

The level of compression can be modified by changing the ZLIB compression setting in mod_deflate.c from Z_BEST_SPEED (equivalent to "gzip -1") to Z_BEST_COMPRESSION (equivalent to "gzip -9"). These defaults can also be replaced with a numeric value between 1 and 9. A "hacked" version of the mod_deflate.c code is available here; the compression level has been set to "6", which is regarded as a good balance between speed and compression.

More info on hacking mod_deflate for Apache 2.0.x can be found here.