A microcache serves fresh entries briefly and coalesces misses explicitly

Sun, 12 Mar 2028 00:00:00 +0000

A microcache keeps reusable responses for a short freshness lifetime, often a few seconds. Requests for the same selected representation can reuse a fresh entry during that window. The response might lag a new origin update, but it is not HTTP-stale merely because it came from a cache. Expiration, permission to serve stale content, and concurrent cache filling are separate policies.

When it breaks

A short TTL does not make a shared response safe for every user. If personalized requests read or populate a generic entry, one user’s representation can be served in another context. proxy_cache_bypass prevents reading from cache under its conditions; proxy_no_cache prevents saving the response. Using only one side can leave the other path inconsistent with the intended privacy boundary.

The cache key also defines which requests share a response. Ignoring a meaningful query or host dimension can merge different representations. Including arbitrary varying data can instead destroy reuse. The fixture below uses one backend authority and preserves the complete request URI, including arguments. A service varying by tenant, language, or other inputs must establish its own representation identity.

Caching alone does not guarantee one origin request during a cold burst. Several requests can miss the same entry before any response is stored. proxy_cache_lock coordinates filling a new entry, with finite timeout and age controls. That lock is not the TTL, and a brief freshness lifetime does not implicitly turn lock behavior on.

Check it quickly

Save the script as check-microcache.py and run it with Python, Docker, and the pinned images. The recorded environment used nginx 1.28.0 on Linux arm64. A disposable backend reports per-path request counts, and nginx uses a two-second validity setting with explicit cookie and authorization bypass conditions. The listener is restricted to host loopback:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
from concurrent.futures import ThreadPoolExecutor
import threading
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import json,threading
lock=threading.Lock();counts={};gates={'/lock':threading.Event(),'/nolock':threading.Event()}
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def log_message(self,*args):pass
 def do_GET(self):
 parsed=urlsplit(self.path)
 if parsed.path=='/control/state':
 with lock:data=dict(counts)
 elif parsed.path=='/control/release':
 gates[parse_qs(parsed.query)['path'][0]].set();data={'released':True}
 else:
 with lock:counts[self.path]=counts.get(self.path,0)+1;count=counts[self.path]
 if parsed.path in gates:assert gates[parsed.path].wait(15)
 data={'path':self.path,'count':count,'cookie':self.headers.get('Cookie'),'authorization_present':bool(self.headers.get('Authorization'))}
 body=json.dumps(data,sort_keys=True).encode();self.send_response(200)
 if parsed.path=='/cookie':self.send_header('Set-Cookie','fixture=public-example; Path=/')
 if parsed.path=='/private':self.send_header('Cache-Control','private')
 if parsed.path=='/no-store':self.send_header('Cache-Control','no-store')
 self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
ThreadingHTTPServer(('0.0.0.0',8081),Handler).serve_forever()
'''
extra='''
proxy_cache_path /tmp/cache levels=1:2 keys_zone=pages:1m inactive=30s max_size=10m;
map $http_cookie $skip_cookie { default 1; "" 0; }
map $http_authorization $skip_auth { default 1; "" 0; }
proxy_cache pages; proxy_cache_key "$scheme|$proxy_host|$request_uri"; proxy_cache_valid 200 2s;
proxy_cache_bypass $skip_cookie $skip_auth; proxy_no_cache $skip_cookie $skip_auth;
proxy_cache_lock on; proxy_cache_lock_timeout 5s; proxy_cache_lock_age 5s;
add_header X-Cache $upstream_cache_status always;
'''
locations='''
location /control/ { proxy_cache off; proxy_pass http://backend:8081; }
location = /nolock { proxy_cache_lock off; proxy_pass http://backend:8081; }
location / { proxy_pass http://backend:8081; }
'''
with lab(config(locations,extra),backend=backend) as (_,request):
 def get(path,headers=None):
 status,h,b=request(path,headers=headers);assert status==200;return h.get('x-cache'),json.loads(b)
 results={}
 first=get('/item?x=1');second=get('/item?x=1');assert first[0]=='MISS' and second==('HIT',first[1])
 other=get('/item?x=2');assert other[0]=='MISS' and other[1]['path']!=first[1]['path']
 results['fresh']=[first[0],second[0]];results['different_query']=other[0]
 for name,headers in [('cookie',{'Cookie':'session=public-fixture'}),('authorization',{'Authorization':'Bearer public-fixture'})]:
 private=get('/item?x=1',headers);assert private[0]=='BYPASS' and private[1]!=first[1]
 assert get('/item?x=1')==('HIT',first[1])
 new='/new-'+name;assert get(new,headers)[0]=='BYPASS'
 public=get(new);assert public[0]=='MISS' and public[1]['cookie'] is None and not public[1]['authorization_present']
 results[name]={'warm_key':'BYPASS','public_after_private_miss':public[0]}
 for path in ['/cookie','private','no-store']:
 path='/'+path.lstrip('/');a=get(path);b=get(path)
 assert a[0]==b[0]=='MISS' and a[1]['count']==1 and b[1]['count']==2
 results[path]=[a[0],b[0]]
 deadline=time.monotonic()+6
 while time.monotonic()<deadline:
 value=get('/item?x=2')
 if value[0]=='EXPIRED':break
 assert value[0]=='HIT';time.sleep(.15)
 else:raise AssertionError('TTL did not expire')
 assert value[1]['count']==2;results['after_ttl']={'status':value[0],'count':value[1]['count']}
 for path,number,origin_count in [('/lock',6,1),('/nolock',4,4)]:
 barrier=threading.Barrier(number+1)
 def client():barrier.wait();return get(path)
 with ThreadPoolExecutor(max_workers=number) as pool:
 futures=[pool.submit(client) for _ in range(number)];barrier.wait()
 wait(lambda:get('/control/state')[1].get(path)==origin_count)
 time.sleep(.3);before=get('/control/state')[1][path]
 assert before==origin_count and not any(f.done() for f in futures)
 get('/control/release?path='+path);responses=[f.result(timeout=10) for f in futures]
 assert get('/control/state')[1][path]==origin_count
 statuses=sorted(r[0] for r in responses)
 if path=='/lock':assert statuses==['HIT']*5+['MISS'] and all(r[1]==responses[0][1] for r in responses)
 else:assert statuses==['MISS']*4
 results[path]={'origin_requests_while_held':before,'cache_statuses':statuses}
 print(json.dumps(results,sort_keys=True))

The first public request to /item?x=1 reports MISS, and the next reports HIT with the same backend result. /item?x=2 has its own MISS and distinct path in the response. Polling that second entry eventually produces EXPIRED and a backend count of two. This observes actual expiration using the server’s clock; it does not assume an exact subsecond transition time.

Requests carrying the example session cookie or authorization marker report BYPASS even when a public entry is already warm. A later public request still returns the original public HIT. A second control first visits an unused key with those headers, then visits it publicly: the public request is MISS, proving that the preceding private-context response did not populate a reusable entry.

Backend responses containing Set-Cookie, Cache-Control: private, or Cache-Control: no-store each produce two successive MISS results and increasing origin counts. The configuration does not ignore those response controls. This distinguishes client-side bypass policy from origin-provided storage restrictions; the short TTL is not used as an excuse to override either privacy signal.

For a cold locked entry, six clients start together while the backend deliberately withholds the response. The backend sees one request before release; afterward, one client reports MISS and five report HIT with the same result. For a separate entry with the lock disabled, four held clients create four origin requests and all report MISS. The gate makes overlapping misses directly observable.

Choose freshness and overload behavior separately

The two-second setting applies to eligible 200 responses in this experiment. Relevant origin cache headers can take priority over configured validity. Decide which responses are shareable before selecting the lifetime, and keep the cache key aligned with the inputs that actually change their content. A high hit ratio is not evidence that the selected representation was correct.

Lock waiters exchange extra waiting for reduced origin work. This fixture releases the origin within the configured lock limits; it does not establish a permanent single-request guarantee when those limits expire or a backend fails. Review lock timeout, lock age, and any stale-serving policy together with the service’s acceptable latency and failure behavior.

Observe cache status alongside origin counts and response identity. MISS means the requested representation was not served as a cache hit, while BYPASS identifies the explicit read exclusion shown here. Fresh HIT, EXPIRED refill, and intentionally stale service should not be collapsed into one “cached” result. Their different effects on correctness and backend load are the reason to measure them separately.

Sources

nginx proxy module, cache validity, cache locks, bypass, no-cache, and response header controls, checked 2026-09-12.
nginx content caching guide, cache keys, status observation, and caching behavior, checked 2026-09-12.
RFC 9111 HTTP Caching, fresh versus stale responses and private/no-store directives, checked 2026-09-12.

Microcaching · XB Field Notes

A microcache serves fresh entries briefly and coalesces misses explicitly

When it breaks

Check it quickly

Choose freshness and overload behavior separately

Sources