Compression · XB Field Notes

gzip, HTTP/2, and Brotli need separate verification

Fri, 28 Jan 2028 00:00:00 +0000

HTTP/2 changes HTTP transport framing and multiplexing; gzip and Brotli encode representation data. Enabling HTTP/2 does not automatically compress response bodies, and enabling gzip does not prove that a client negotiated HTTP/2. HTTP/2 header compression is another mechanism again. A reliable check records the negotiated protocol, response Content-Encoding, transferred body size, and the bytes obtained after decoding.

When it breaks

Requesting Accept-Encoding: gzip, br and saving the result as .br does not prove Brotli was used. The server may choose gzip or send an acceptable identity representation. The response header names the actual coding. Guessing from the request list or output filename can lead to applying the wrong decoder and reporting compression results that never occurred.

Brotli support is also a deployment capability, not an implication of nginx being installed. Separate modules provide Brotli filtering or precompressed-file handling. Their availability and build compatibility must be established for the actual binary and configuration. Adding an unsupported directive can prevent configuration validation, while merely advertising Brotli from the client cannot install server functionality.

Automatic client decoding creates another measurement trap. curl --compressed can save a large decoded file after receiving a much smaller encoded body. Comparing that file’s size with a raw compressed output compares different things. Conversely, using --http2 without checking the negotiated version can mistake an allowed protocol fallback for a successful HTTP/2 test.

Check it quickly

Save the script as check-h2-coding.py and run it with Docker, Python, OpenSSL, and a curl build supporting HTTP/2. The recorded client was curl 8.7.1 with nghttp2; the server was nginx 1.28.0 on Linux arm64. It generates a short-lived local certificate and explicitly trusts that certificate with --cacert, while publishing only a loopback port:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
import gzip,hashlib
payload=(b'{"message":"'+b'HTTP content coding and transport are separate. '*1000+b'"}\n')
with TemporaryDirectory(prefix='xbcatm-local-tls-') as directory:
 root=Path(directory);cert=root/'cert.pem';key=root/'key.pem'
 result=subprocess.run(['openssl','req','-x509','-newkey','ec','-pkeyopt','ec_paramgen_curve:prime256v1',
 '-nodes','-keyout',str(key),'-out',str(cert),'-days','1','-subj','/CN=localhost',
 '-addext','subjectAltName=DNS:localhost,IP:127.0.0.1'],capture_output=True,text=True)
 assert result.returncode==0,result.stderr
 configuration=config('location = /app.json {} location / { return 404; }',
 'gzip on; gzip_types application/json; gzip_vary on; types { application/json json; }')
 configuration=configuration.replace('listen 8080;', 'listen 8080 ssl; http2 on; ssl_certificate /srv/cert.pem; ssl_certificate_key /srv/key.pem;')
 files={'cert.pem':cert.read_bytes(),'key.pem':key.read_bytes(),'app.json':payload,
 'brotli.conf':'events {} http { brotli on; server { listen 8080; } }'}
 with lab(configuration,files) as (proxy,request):
 probe=docker('exec',proxy,'nginx','-t','-c','/srv/brotli.conf',check=False)
 assert probe.returncode!=0 and 'unknown directive "brotli"' in probe.stderr
 results={'brotli_directive_supported':False}
 for name,version,accept,decode in [
 ('h2_gzip','2','gzip, br',False),('h1_gzip','1.1','gzip',False),
 ('h2_br_only','2','br',False),('h2_identity','2','identity',False),
 ('h2_decoded','2','gzip',True)]:
 output=root/(name+'.body');header=root/(name+'.headers')
 args=['curl','--silent','--show-error','--fail','--noproxy','*','--cacert',str(cert),
 '--http2' if version=='2' else '--http1.1','--header','Accept-Encoding: '+accept,
 '--output',str(output),'--dump-header',str(header),'--write-out','%{http_version}\n%{size_download}\n',
 f'https://127.0.0.1:{request.port}/app.json']
 if decode:args.insert(1,'--compressed')
 run=subprocess.run(args,capture_output=True,text=True,timeout=15);assert run.returncode==0,run.stderr
 actual,download=run.stdout.strip().splitlines();assert actual==version,(name,run.stdout)
 headers={k.lower():v.strip() for line in header.read_text().splitlines() if ':' in line for k,v in [line.split(':',1)]}
 body=output.read_bytes();encoded='gzip' in accept
 assert headers.get('content-encoding')==('gzip' if encoded else None)
 assert (gzip.decompress(body) if encoded and not decode else body)==payload
 assert headers['vary']=='Accept-Encoding'
 results[name]={'protocol':actual,'content_encoding':headers.get('content-encoding','identity'),
 'download_body_bytes':int(download),'saved_file_bytes':len(body)}
 assert results['h2_gzip']['download_body_bytes']<len(payload)
 assert results['h2_decoded']['saved_file_bytes']==len(payload)
 rejected={}
 for path in ['/key.pem','/cert.pem','/brotli.conf']:
 run=subprocess.run(['curl','--silent','--show-error','--noproxy','*','--cacert',str(cert),
 '--http2','--output',str(root/'rejected.body'),'--write-out','%{http_code}',
 f'https://127.0.0.1:{request.port}'+path],capture_output=True,text=True,timeout=15)
 assert run.returncode==0 and run.stdout=='404',(path,run.stdout,run.stderr)
 rejected[path]=404
 results['non_asset_statuses']=rejected
 results['original_sha256']=hashlib.sha256(payload).hexdigest()
 print(json.dumps(results,sort_keys=True))

The certificate includes the loopback IP in its subject alternative names, so the local HTTPS request can validate both trust and the requested identity. The fixture does not disable certificate verification. Only the JSON asset is routed for serving; requests for the key, certificate, and probe configuration must return 404. Temporary key material stays within the disposable test files and image, and the experiment removes its containers, image tags, network, and temporary files afterward.

The synthetic JSON representation contains 48,015 bytes. Over negotiated HTTP/2, a request advertising gzip and Brotli receives Content-Encoding: gzip and a 381-byte body in the recorded run. Decoding reproduces every original byte. Explicit HTTP/1.1 produces gzip too. These outcomes show that the body coding works across both tested transports rather than depending on HTTP/2 being selected.

An HTTP/2 identity request receives 48,015 unencoded bytes. A request advertising only br also receives identity in this configuration, which remains acceptable under that header. The fixture separately tests a configuration containing brotli on and requires nginx to reject it as an unknown directive. Together, those observations establish that this tested setup did not serve Brotli.

With --compressed, curl still reports a 381-byte downloaded body, but the saved file contains the original 48,015 decoded bytes. The script checks both measurements and the content itself. The downloaded-body figure excludes other transport overhead; it is not a complete packet capture or a measurement of all TLS and HTTP/2 framing bytes on the connection.

Verify capabilities before comparing performance

The example enables HTTP/2 explicitly on a TLS listener and checks curl’s resulting protocol value. It serves static content locally, so it makes no claim about an upstream application’s transport. A reverse proxy can negotiate one protocol with clients and use another with its backend; inspect each connection separately when that distinction affects capacity or behavior.

For Brotli deployment, establish the module or precompressed serving path, then require an actual br response and a successful Brotli decode before measuring its savings. This experiment provides a negative capability check, not a Brotli benchmark. Installing a module or rebuilding nginx is a separate configuration change with its own compatibility and serving checks.

Finally, compare representative content with consistent measurements. This highly repetitive test body demonstrates correct coding and decoding, not a universal compression ratio or page-speed improvement. Record cache selection through Vary, encoded body bytes, decoding correctness, CPU use, and client latency as distinct observations. Neither a transport version nor a codec name alone establishes that a page became faster.

Sources

nginx gzip module, response content coding and Vary configuration, checked 2026-09-12.
nginx HTTP/2 module, HTTP/2 enablement and TLS ALPN requirements, checked 2026-09-12.
Google ngx_brotli module README, the separate Brotli filter/static modules and build requirements, checked 2026-09-12.
RFC 9110 HTTP Semantics, content-coding negotiation and identity acceptability, checked 2026-09-12.

Enable gzip explicitly, then verify negotiation and decoded bytes

Sun, 29 Aug 2027 00:00:00 +0000

gzip is a response content coding: nginx can send a compressed representation that an accepting client decodes. Enabling it requires gzip on; gzip_types extends the eligible MIME types but does not activate compression by itself. The result depends on the response, request capabilities, and effective configuration. A smaller body is useful only if it still decodes to the intended content.

When it breaks

A MIME list beneath gzip off changes no response into gzip. Another common omission is gzip_vary on. nginx does not unconditionally add Vary: Accept-Encoding merely because compression occurred; the directive controls that behavior. A shared cache selecting representations needs the relevant request-header dimensions, otherwise clients with different coding capabilities can receive an inappropriate stored response.

The minimum-length setting has a narrower contract than its name suggests. gzip_min_length determines length from the response’s Content-Length field. It does not necessarily know the final size of a response whose length is unspecified. A tiny dynamically streamed response can therefore be compressed even when an equally small static file falls below the configured threshold.

Measurements can also be misleading. The size of a decompressed output file is not the compressed transfer-body size, and compressed body bytes do not include all HTTP framing, headers, or TLS traffic. File extensions such as .gz prove nothing by themselves. Check the actual content coding and decode the bytes before interpreting any apparent bandwidth saving.

Check it quickly

Save this as check-gzip.py and run it with Python, Docker, and the pinned images. The recorded setup used nginx 1.28.0 on Linux arm64 and a small Python backend. nginx enables gzip at level three, allows JSON in addition to its built-in HTML eligibility, sets a 1,024-byte floor, and explicitly enables Vary:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
import gzip,hashlib
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def log_message(self,*args):pass
 def do_GET(self):
 self.send_response(200);self.send_header('Content-Type','application/json')
 self.send_header('Connection','close');self.end_headers();self.wfile.write(b'{"ok":true}\n');self.close_connection=True
ThreadingHTTPServer(('0.0.0.0',8081),Handler).serve_forever()
'''
payload=json.dumps({'message':'compressible nginx response '*1000}).encode()
locations='''
location = /disabled.json { gzip off; gzip_types application/json; }
location = /novary.json { gzip_vary off; }
location = /stream { proxy_pass http://backend:8081; }
'''
extra='''
types { application/json json; text/html html; application/octet-stream bin; }
gzip on; gzip_types application/json; gzip_vary on; gzip_min_length 1024; gzip_comp_level 3;
'''
files={name:payload for name in ['large.json','other.bin','page.html','disabled.json','novary.json']};files['tiny.json']=b'{"ok":true}\n'
with lab(config(locations,extra),files,backend) as (_,request):
 results={}
 for encoding in ['gzip','identity','gzip;q=0','br']:
 status,h,b=request('/large.json',headers={'Accept-Encoding':encoding});assert status==200
 if encoding=='gzip':assert h['content-encoding']=='gzip' and gzip.decompress(b)==payload and len(b)<len(payload)
 else:assert 'content-encoding' not in h and b==payload
 assert h['vary']=='Accept-Encoding'
 results[encoding]={'encoding':h.get('content-encoding','identity'),'body_bytes':len(b)}
 for path,encoded in [('/tiny.json',False),('/other.bin',False),('/page.html',True),('/disabled.json',False),('/novary.json',True),('/stream',True)]:
 status,h,b=request(path,headers={'Accept-Encoding':'gzip'});assert status==200
 assert (h.get('content-encoding')=='gzip')==encoded,(path,h)
 decoded=gzip.decompress(b) if encoded else b
 assert decoded==(files['tiny.json'] if path=='/stream' else files[path[1:]])
 if path=='/novary.json':assert 'vary' not in h
 results[path]={'encoding':h.get('content-encoding','identity'),'body_bytes':len(b),'vary':h.get('vary')}
 results['decoded_sha256']=hashlib.sha256(payload).hexdigest()
 print(json.dumps(results,sort_keys=True))

The large synthetic JSON representation is 28,015 bytes. With Accept-Encoding: gzip, its transferred response body is 246 bytes in the recorded run, and Python’s gzip decoder reproduces the original bytes exactly. This intentionally repetitive payload demonstrates operation and integrity; its compression ratio is not a prediction for production JSON, JavaScript, or arbitrary text.

Requests accepting identity, setting gzip;q=0, or advertising only br receive the original bytes without a Content-Encoding field. Zero quality disallows gzip. Advertising Brotli does not enable a Brotli compressor in this nginx configuration, and identity remains acceptable in that request. The eligible large responses carry Vary: Accept-Encoding, including the uncompressed variants.

The same synthetic payload served as an ineligible binary type remains uncompressed, while the HTML type is eligible without appearing in gzip_types. A location explicitly setting gzip off also remains uncompressed despite listing JSON. These controls separate type eligibility from the enable switch, rather than treating every absent coding as evidence that negotiation failed.

The 12-byte static JSON response stays uncompressed because its known length is below the floor. The backend returns the same 12 bytes without Content-Length, ending the response by closing its connection. nginx compresses that response into 32 bytes. The floor therefore does not guarantee that all tiny responses avoid compression, and this particular tiny payload actually grows.

A final location keeps gzip active but turns gzip_vary off. Its body still arrives gzip encoded, with no Vary field. This makes the missing-cache-dimension failure observable without requiring a real shared cache to misdeliver content. The fixture checks the header policy; a deployed cache’s representation selection should also be tested through that cache.

Tune the policy using representative responses

Select content types whose actual bytes compress well. Already compressed media or archives usually offer little benefit from another coding layer, while repetitive text may shrink substantially. Do not infer CPU cost from this byte-count experiment. Compare representative sizes, compression settings, server utilization, and client latency before choosing a level for a busy service.

Keep the response metadata honest through proxies. Content-Encoding tells the recipient how to decode the representation, while Vary tells caches which request fields affect selection. If an application already sends an encoded response, review that path separately rather than adding a misleading header to unencoded bytes. The decisive check remains negotiated headers plus a successful byte-for-byte decode.

Sources

nginx gzip module, enabling gzip, MIME eligibility, Content-Length thresholds, and Vary, checked 2026-09-12.
RFC 9110 HTTP Semantics, Accept-Encoding, Content-Encoding, and representation selection, checked 2026-09-12.

Compression negotiation is content negotiation

Tue, 21 Jul 2026 00:00:00 +0000

HTTP compression selects a coded representation, not a new media type. Gzipped text remains text after decompression. A useful check therefore asks three questions: was the coding acceptable, did the transfer become smaller, and did decoding recover the intended bytes? A compression checkbox or a successful status code cannot answer all three.

Absence, exclusion, and preference differ

RFC 9110 considers any content coding acceptable when Accept-Encoding is absent. An explicitly empty value instead requests no content coding. Identity means no content coding and is normally acceptable unless excluded by identity;q=0 or an applicable *;q=0. A wildcard can make an unlisted coding acceptable; the server is not limited to tokens explicitly named individually.

Quality zero means unacceptable. Positive quality values rank acceptable codings, but the server still needs an available encoder and suitable content. An unsupported Brotli preference does not install Brotli. When identity remains acceptable, an uncoded response can be appropriate. If everything is excluded, do not assume every server implements a strict 406 policy: RFC 9110 also allows disregarding negotiation preferences. Validate the policy required by your application.

When it breaks

A proxy that compresses a body without updating Content-Encoding makes the client interpret compressed bytes as text. A proxy that retains the field after decompression invites a second decode. Content-Length, when present, describes the coded content bytes, not the original length. Streaming compression may omit that field instead of knowing the final length in advance. Transfer framing is separate from this content coding.

Vary must describe Accept-Encoding when it selects reusable variants, including uncoded responses. Otherwise, a cached gzip representation can reach a client that excluded gzip. Freshness and storage eligibility remain separate questions. Correct selection also does not guarantee useful savings: tiny payloads have compression overhead, and already-compressed media often gain little. Measure representative content and CPU cost rather than quoting one universal compression percentage.

Check it quickly

Use Python 3.11 or newer, curl with gzip support, and Caddy 2.11.4 on PATH. Obtain the platform binary from Caddy’s official release and verify its published checksum before running it. The program creates an isolated file server on loopback port 18765; choose another unused port in the code if needed. It disables the admin endpoint, automatic HTTPS, and configuration persistence, and cleans up only its temporary files and child process.

import gzip
import os
import shutil
import socket
import subprocess
import tempfile
import time
from http.client import HTTPConnection, HTTPException
from pathlib import Path

caddy = shutil.which("caddy")
if caddy is None:
 raise SystemExit("Put Caddy 2.11.4 on PATH first")
print(subprocess.check_output([caddy, "version"], text=True).split()[0])
port = 18765
with socket.socket() as probe:
 probe.bind(("127.0.0.1", port))


def get(name, accept):
 connection = HTTPConnection("127.0.0.1", port, timeout=2)
 try:
 connection.putrequest("GET", "/" + name, skip_accept_encoding=True)
 if accept is not None:
 connection.putheader("Accept-Encoding", accept)
 connection.endheaders()
 response = connection.getresponse()
 body = response.read()
 assert response.status == 200
 return response.headers, body
 finally:
 connection.close()


with tempfile.TemporaryDirectory(prefix="compression-lab-") as temporary:
 directory = Path(temporary)
 site = directory / "site"
 site.mkdir()
 plain = b"HTTP content coding changes bytes, not the media type.\n" * 200
 (site / "large.txt").write_bytes(plain)
 (site / "tiny.txt").write_bytes(b"small\n")
 config = directory / "Caddyfile"
 config.write_text(
 "{\n admin off\n auto_https off\n persist_config off\n}\n"
 f"http://127.0.0.1:{port} {{\n root * {site}\n encode gzip\n file_server\n}}\n"
 )
 environment = dict(os.environ, XDG_DATA_HOME=str(directory / "data"),
 XDG_CONFIG_HOME=str(directory / "config"))
 with (directory / "server.log").open("w+") as log:
 process = subprocess.Popen(
 [caddy, "run", "--config", str(config), "--adapter", "caddyfile"],
 stdout=log, stderr=log, env=environment,
 )
 try:
 deadline = time.monotonic() + 5
 while True:
 if process.poll() is not None or time.monotonic() > deadline:
 log.seek(0)
 raise RuntimeError(log.read() or "Caddy startup timed out")
 try:
 _, body = get("large.txt", "identity")
 assert body == plain
 break
 except (OSError, HTTPException):
 time.sleep(0.05)
 cases = [(None, None), ("", None), ("gzip", "gzip"),
 ("gzip;q=0", None), ("br", None),
 ("identity;q=0, gzip;q=1", "gzip"), ("*;q=0", None)]
 for accept, expected in cases:
 headers, body = get("large.txt", accept)
 coding = headers.get("Content-Encoding")
 assert coding == expected
 assert headers.get_content_type() == "text/plain"
 assert headers["Vary"] == "Accept-Encoding"
 decoded = gzip.decompress(body) if coding == "gzip" else body
 assert decoded == plain
 if coding == "gzip":
 assert len(body) < len(plain)
 if headers.get("Content-Length") is not None:
 assert int(headers["Content-Length"]) == len(body)
 print(repr(accept), coding, len(body), headers.get("Vary"))
 headers, body = get("tiny.txt", "gzip")
 assert body == b"small\n" and headers.get("Content-Encoding") is None
 print("tiny", len(body), headers.get("Vary"))
 decoded = subprocess.check_output([
 "curl", "-q", "--noproxy", "*", "-fsS", "--max-time", "3",
 "--compressed", "-H", "Accept-Encoding: gzip",
 f"http://127.0.0.1:{port}/large.txt",
 ])
 assert decoded == plain
 print("curl decoded:", len(decoded))
 finally:
 process.terminate()
 try:
 process.wait(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.wait(timeout=5)

The repeated fixture contains 11,000 bytes. In the checked run, gzip transferred 122 content bytes and decompression recovered all 11,000. This comparison excludes HTTP headers and transfer framing; it is not a bandwidth or latency benchmark. skip_accept_encoding=True matters because Python otherwise adds an identity preference, making a supposed missing-header test misleading.

Missing, empty, gzip-excluded, and unsupported-Brotli requests receive uncoded content in this configuration. Absence permits compression but does not require it. The two gzip-accepting cases receive gzip. The six-byte file stays uncoded despite accepting gzip because it is below Caddy’s default 512-byte threshold. All large-file responses are checked for Vary, media type, and decoded-byte equality, not just a compression label.

The *;q=0 case also receives uncoded content with 200 in this tested Caddy configuration, even though identity was excluded. That observation demonstrates why a strict rejection contract cannot be inferred from enabling compression. The assertion records this implementation behavior; it does not declare that the client’s exclusion was satisfied. Test or implement the required rejection policy at the responsible endpoint.

Finally, curl’s --compressed returns the decoded 11,000 bytes. Its saved response headers can still contain Content-Encoding, so do not decompress that output again based only on a header dump. The test bypasses curl configuration files and proxies to isolate loopback behavior. Production verification must repeat selection and byte checks through the actual proxy or CDN, where compression can be added, removed, or applied twice.

Sources

RFC 9110: Accept-Encoding, absence, identity, exclusions, and content coding, checked 2026-09-07.
Caddy encode directive, encoder selection and the default size threshold, checked 2026-09-07.
Caddy 2.11.4 release, the tested binary and published checksums, checked 2026-09-07.
curl compressed mode, automatic decompression and unchanged saved response headers, checked 2026-09-07.

Compression · XB Field Notes

gzip, HTTP/2, and Brotli need separate verification

When it breaks

Check it quickly

Verify capabilities before comparing performance

Sources

Enable gzip explicitly, then verify negotiation and decoded bytes

When it breaks

Check it quickly

Tune the policy using representative responses

Sources

Compression negotiation is content negotiation

Absence, exclusion, and preference differ

When it breaks

Check it quickly

Further reading

Sources