An nginx upstream keepalive cache preserves eligible idle backend connections for later requests. In the tested configuration, keepalive 2 permits two idle connections in that upstream cache for each worker. It does not restrict the worker to two active backend connections. Reuse reduces repeated connection setup; it does not itself impose an application concurrency budget or guarantee lower latency for every request.

When it breaks

An idle-cache size mistaken for a connection cap leaves the backend exposed to more simultaneous work than expected. A busy worker can open additional active connections while its retained idle cache remains small. With several workers, each worker’s idle cache also matters. Count active and idle resources separately when comparing nginx settings with a backend’s actual connection capacity.

Connection reuse requires a reusable protocol exchange. For this nginx 1.28.0 HTTP example, upstream HTTP/1.1 and clearing the default Connection: close header are explicit configuration choices. Merely setting Host does not establish persistence. A backend can still close connections, and incomplete or incorrectly framed messages cannot be treated as clean reusable exchanges just because a cache is configured.

Client-side persistence is a different connection relationship. A fresh client connection can be served using a previously cached upstream connection, while a long-lived client can trigger several backend connections. Assuming those lifetimes match leads to misleading measurements. Connection setup savings also differ between a plain HTTP upstream and a TLS upstream; this experiment uses plain HTTP behind nginx.

Check it quickly

Save the script as check-upstream-keepalive.py and run it with Python, Docker, and the pinned images. It uses one nginx 1.28.0 worker and a Python HTTP/1.1 backend on a private network. Each accepted backend connection receives an identity, and each response has an explicit length. Client requests enter through host loopback:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def wait(predicate):
    deadline=time.monotonic()+15
    while time.monotonic()<deadline:
        try:
            if predicate():return
        except OSError:pass
        time.sleep(.1)
    raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None):
    name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
    try:
        with TemporaryDirectory(prefix=name+'-') as directory:
            root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
            for path,content in (files or {}).items():
                target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
                target.write_bytes(content if isinstance(content,bytes) else content.encode())
            docker('network','create',name)
            if backend is not None:
                (root/'backend.py').write_text(backend)
                (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
                docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
                docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
                wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
            (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
            docker('build','--network','none','-t',tags[0],str(root))
            docker('run','--rm','--network',name,tags[0],'nginx','-t')
            docker('run','-d','--name',containers[0],'--network',name,'-p','127.0.0.1::8080',tags[0])
            details=json.loads(docker('inspect',containers[0]).stdout)[0]
            binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
            def ready():
                connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
            wait(ready)
            def request(path,method='GET',headers=None,body=None):
                connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
                try:
                    connection.request(method,path,body=body,headers=headers or {})
                    response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
                finally:connection.close()
            request.port=port
            yield containers[0],request
    finally:
        docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
    return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
from concurrent.futures import ThreadPoolExecutor
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import itertools,json,threading
ids=itertools.count(1);lock=threading.Lock();opened=set();active=set();release=threading.Event()
class Handler(BaseHTTPRequestHandler):
    protocol_version='HTTP/1.1'
    def setup(self):
        super().setup();self.ident=next(ids)
        with lock:opened.add(self.ident)
    def finish(self):
        try:super().finish()
        finally:
            with lock:opened.discard(self.ident)
    def log_message(self,*args):pass
    def do_GET(self):
        if self.path=='/hold':
            with lock:active.add(self.ident)
            assert release.wait(15)
            with lock:active.remove(self.ident)
        elif self.path=='/release':release.set()
        with lock:data=json.dumps({'connection':self.ident,'active':sorted(active),'other_open':sorted(opened-{self.ident}),
            'host':self.headers.get('Host'),'connection_header':self.headers.get('Connection')}).encode()
        self.send_response(200);self.send_header('Content-Length',str(len(data)));self.end_headers();self.wfile.write(data)
ThreadingHTTPServer(('0.0.0.0',8081),Handler).serve_forever()
'''
locations='''
location /reuse/ { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass http://pool/; }
location /close/ { proxy_http_version 1.1; proxy_set_header Connection close; proxy_pass http://pool/; }
location /host-only/ { proxy_http_version 1.1; proxy_set_header Host service.example; proxy_pass http://pool/; }
location /control/ { proxy_set_header Connection close; proxy_pass http://backend:8081/; }
'''
extra='upstream pool { server backend:8081; keepalive 2; keepalive_timeout 30s; }'
with lab(config(locations,extra),backend=backend) as (_,request):
    def get(path):
        status,_,body=request(path);assert status==200;return json.loads(body)
    results={}
    for mode in ['reuse','close','host-only']:
        first=get('/'+mode+'/id');second=get('/'+mode+'/id')
        assert (first['connection']==second['connection'])==(mode=='reuse')
        if mode=='host-only':assert first['host']=='service.example' and first['connection_header']=='close'
        results[mode]={'connections':[first['connection'],second['connection']],'host':first['host'],'connection_header':first['connection_header']}
    with ThreadPoolExecutor(max_workers=4) as pool:
        futures=[pool.submit(get,'/reuse/hold') for _ in range(4)]
        wait(lambda:len(get('/control/state')['active'])==4)
        active=get('/control/state')['active'];assert len(set(active))==4
        get('/control/release');replies=[f.result(timeout=10) for f in futures]
        assert len({r['connection'] for r in replies})==4
    wait(lambda:len(get('/control/state')['other_open'])==2)
    results['concurrent_active']=active;results['idle_after_burst']=get('/control/state')['other_open']
    print(json.dumps(results,sort_keys=True))

Two sequential /reuse/id requests return the same backend connection identity. The client helper opens and closes its own HTTP connection for each request, so this is upstream reuse across distinct downstream connections. The location explicitly uses upstream HTTP/1.1 and sends no Connection header. Reading each complete response leaves the backend exchange eligible to become idle.

The /close/id control sends Connection: close and returns different backend identities for successive requests. The /host-only/id control changes Host but leaves the closing behavior intact; its responses also use different connections. These comparisons show why a routing header is not a substitute for correct persistence settings, even when all paths contact the same application.

Next, four clients request responses that the backend deliberately holds open. The controller waits until the backend reports four active connection identities before releasing any response. That directly demonstrates four simultaneous upstream connections under keepalive 2. It does not infer concurrency from timing or from a count of completed requests that might have been processed sequentially.

After release, all four replies complete. The fixture observes exactly two remaining idle backend connections, excluding the short-lived control connection used to inspect state. Which identities survive can vary with completion order; their count is the assertion. This distinguishes the configured idle retention limit from the larger set of active connections used during the burst.

Match reuse settings to backend behavior

Size idle retention using connection setup cost, expected reuse, backend limits, and the number of nginx workers. A retained connection consumes resources even while it carries no request. A very large cache can therefore compete with new useful connections, while a very small one may repeatedly discard connections that would soon have been reused.

Request-count, idle-time, and total connection-lifetime controls can recycle persistent connections for different reasons. Do not respond to every reconnection by increasing all of them. Observe backend closes, configuration, completed response framing, and actual connection identities first. The tested idle timeout is thirty seconds; the short burst does not benchmark long-running connection aging or TLS session behavior.

Finally, use an actual concurrency or admission policy when limiting backend work is the objective. Keepalive is reuse policy. The fixture chooses its protocol and cache settings explicitly because rolling documentation includes later nginx defaults and features. Apply the semantics of the deployed version, and verify both reuse and a concurrent burst instead of assuming one successful repeated request proves the capacity boundary.

Sources