HTTP compression selects a coded representation, not a new media type. Gzipped text remains text after decompression. A useful check therefore asks three questions: was the coding acceptable, did the transfer become smaller, and did decoding recover the intended bytes? A compression checkbox or a successful status code cannot answer all three.

Absence, exclusion, and preference differ

RFC 9110 considers any content coding acceptable when Accept-Encoding is absent. An explicitly empty value instead requests no content coding. Identity means no content coding and is normally acceptable unless excluded by identity;q=0 or an applicable *;q=0. A wildcard can make an unlisted coding acceptable; the server is not limited to tokens explicitly named individually.

Quality zero means unacceptable. Positive quality values rank acceptable codings, but the server still needs an available encoder and suitable content. An unsupported Brotli preference does not install Brotli. When identity remains acceptable, an uncoded response can be appropriate. If everything is excluded, do not assume every server implements a strict 406 policy: RFC 9110 also allows disregarding negotiation preferences. Validate the policy required by your application.

When it breaks

A proxy that compresses a body without updating Content-Encoding makes the client interpret compressed bytes as text. A proxy that retains the field after decompression invites a second decode. Content-Length, when present, describes the coded content bytes, not the original length. Streaming compression may omit that field instead of knowing the final length in advance. Transfer framing is separate from this content coding.

Vary must describe Accept-Encoding when it selects reusable variants, including uncoded responses. Otherwise, a cached gzip representation can reach a client that excluded gzip. Freshness and storage eligibility remain separate questions. Correct selection also does not guarantee useful savings: tiny payloads have compression overhead, and already-compressed media often gain little. Measure representative content and CPU cost rather than quoting one universal compression percentage.

Check it quickly

Use Python 3.11 or newer, curl with gzip support, and Caddy 2.11.4 on PATH. Obtain the platform binary from Caddy’s official release and verify its published checksum before running it. The program creates an isolated file server on loopback port 18765; choose another unused port in the code if needed. It disables the admin endpoint, automatic HTTPS, and configuration persistence, and cleans up only its temporary files and child process.

import gzip
import os
import shutil
import socket
import subprocess
import tempfile
import time
from http.client import HTTPConnection, HTTPException
from pathlib import Path

caddy = shutil.which("caddy")
if caddy is None:
    raise SystemExit("Put Caddy 2.11.4 on PATH first")
print(subprocess.check_output([caddy, "version"], text=True).split()[0])
port = 18765
with socket.socket() as probe:
    probe.bind(("127.0.0.1", port))


def get(name, accept):
    connection = HTTPConnection("127.0.0.1", port, timeout=2)
    try:
        connection.putrequest("GET", "/" + name, skip_accept_encoding=True)
        if accept is not None:
            connection.putheader("Accept-Encoding", accept)
        connection.endheaders()
        response = connection.getresponse()
        body = response.read()
        assert response.status == 200
        return response.headers, body
    finally:
        connection.close()


with tempfile.TemporaryDirectory(prefix="compression-lab-") as temporary:
    directory = Path(temporary)
    site = directory / "site"
    site.mkdir()
    plain = b"HTTP content coding changes bytes, not the media type.\n" * 200
    (site / "large.txt").write_bytes(plain)
    (site / "tiny.txt").write_bytes(b"small\n")
    config = directory / "Caddyfile"
    config.write_text(
        "{\n admin off\n auto_https off\n persist_config off\n}\n"
        f"http://127.0.0.1:{port} {{\n root * {site}\n encode gzip\n file_server\n}}\n"
    )
    environment = dict(os.environ, XDG_DATA_HOME=str(directory / "data"),
                       XDG_CONFIG_HOME=str(directory / "config"))
    with (directory / "server.log").open("w+") as log:
        process = subprocess.Popen(
            [caddy, "run", "--config", str(config), "--adapter", "caddyfile"],
            stdout=log, stderr=log, env=environment,
        )
        try:
            deadline = time.monotonic() + 5
            while True:
                if process.poll() is not None or time.monotonic() > deadline:
                    log.seek(0)
                    raise RuntimeError(log.read() or "Caddy startup timed out")
                try:
                    _, body = get("large.txt", "identity")
                    assert body == plain
                    break
                except (OSError, HTTPException):
                    time.sleep(0.05)
            cases = [(None, None), ("", None), ("gzip", "gzip"),
                     ("gzip;q=0", None), ("br", None),
                     ("identity;q=0, gzip;q=1", "gzip"), ("*;q=0", None)]
            for accept, expected in cases:
                headers, body = get("large.txt", accept)
                coding = headers.get("Content-Encoding")
                assert coding == expected
                assert headers.get_content_type() == "text/plain"
                assert headers["Vary"] == "Accept-Encoding"
                decoded = gzip.decompress(body) if coding == "gzip" else body
                assert decoded == plain
                if coding == "gzip":
                    assert len(body) < len(plain)
                if headers.get("Content-Length") is not None:
                    assert int(headers["Content-Length"]) == len(body)
                print(repr(accept), coding, len(body), headers.get("Vary"))
            headers, body = get("tiny.txt", "gzip")
            assert body == b"small\n" and headers.get("Content-Encoding") is None
            print("tiny", len(body), headers.get("Vary"))
            decoded = subprocess.check_output([
                "curl", "-q", "--noproxy", "*", "-fsS", "--max-time", "3",
                "--compressed", "-H", "Accept-Encoding: gzip",
                f"http://127.0.0.1:{port}/large.txt",
            ])
            assert decoded == plain
            print("curl decoded:", len(decoded))
        finally:
            process.terminate()
            try:
                process.wait(timeout=5)
            except subprocess.TimeoutExpired:
                process.kill()
                process.wait(timeout=5)

The repeated fixture contains 11,000 bytes. In the checked run, gzip transferred 122 content bytes and decompression recovered all 11,000. This comparison excludes HTTP headers and transfer framing; it is not a bandwidth or latency benchmark. skip_accept_encoding=True matters because Python otherwise adds an identity preference, making a supposed missing-header test misleading.

Missing, empty, gzip-excluded, and unsupported-Brotli requests receive uncoded content in this configuration. Absence permits compression but does not require it. The two gzip-accepting cases receive gzip. The six-byte file stays uncoded despite accepting gzip because it is below Caddy’s default 512-byte threshold. All large-file responses are checked for Vary, media type, and decoded-byte equality, not just a compression label.

The *;q=0 case also receives uncoded content with 200 in this tested Caddy configuration, even though identity was excluded. That observation demonstrates why a strict rejection contract cannot be inferred from enabling compression. The assertion records this implementation behavior; it does not declare that the client’s exclusion was satisfied. Test or implement the required rejection policy at the responsible endpoint.

Finally, curl’s --compressed returns the decoded 11,000 bytes. Its saved response headers can still contain Content-Encoding, so do not decompress that output again based only on a header dump. The test bypasses curl configuration files and proxies to isolate loopback behavior. Production verification must repeat selection and byte checks through the actual proxy or CDN, where compression can be added, removed, or applied twice.

Further reading

Sources