Load-Balancing · XB Field Notes

Least connections measures occupied connections, not every kind of load

Thu, 11 Nov 2027 00:00:00 +0000

Least-connections balancing chooses a peer using active connection counts, with the implementation’s weighting and tie rules. In nginx, least_conn accounts for server weights and uses weighted round robin among equally suitable peers. This can help when requests occupy connections for different durations. It is still a connection-based measurement, not direct knowledge of CPU, memory pressure, queued jobs, or database contention.

When it breaks

Calling the selected peer the “emptiest server” overstates the signal. One connection may carry an expensive operation while another carries a cheap wait. Background work can continue after an HTTP response closes, and multiplexed protocols can place several operations on one connection. A low connection count therefore does not prove that the backend has spare capacity for the next costly request.

The count is also the balancer’s observed upstream state, not an omniscient census of every backend client. Work arriving through another proxy or a direct internal path may be outside that view. In nginx, worker scope and upstream shared-zone configuration affect which runtime state is shared. Independent balancers do not automatically merge their observations into one global least-connections decision.

Weights make raw-count comparisons insufficient. A peer with weight two can appropriately carry more active connections than one with weight one. Changing a weight expresses a capacity assumption; it does not measure whether that assumption is true. An idle keepalive cache is another different quantity and should not be confused with the active connections used for this selection.

Check it quickly

Save this as check-least-connections.py and run it with Python, Docker, and the pinned images. The recorded setup uses one nginx 1.28.0 worker with no upstream keepalive cache and two HTTP peers on separate ports. The backend provides controlled response gates and active counters, making held work visible without depending on a guessed sleep duration:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
 docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port;request.network=name
 request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import itertools,json,threading
lock=threading.Lock();ids=itertools.count(1);counts={'A':0,'B':0};active={'A':0,'B':0};faults={'A':200,'B':200};gates={}
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def setup(self):super().setup();self.ident=next(ids)
 def log_message(self,*args):pass
 def do_GET(self):
 parsed=urlsplit(self.path);label=self.server.label;code=200
 if label=='control':
 query=parse_qs(parsed.query)
 with lock:
 if parsed.path=='/fail':faults[query['peer'][0]]=int(query['status'][0])
 if parsed.path=='/release':gates[query['path'][0]].set()
 data={'counts':dict(counts),'active':dict(active),'faults':dict(faults),'held':list(gates)}
 else:
 with lock:
 counts[label]+=1;active[label]+=1;code=faults[label]
 if parsed.path.startswith('/hold/'):gates[parsed.path]=threading.Event()
 if parsed.path.startswith('/hold/'):assert gates[parsed.path].wait(20)
 data={'peer':label,'path':self.path,'connection':self.ident}
 body=json.dumps(data,sort_keys=True).encode();self.send_response(code)
 self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)))
 self.end_headers();self.wfile.write(body)
 if label!='control':
 with lock:active[label]-=1
for port,label in [(8081,'A'),(8082,'B'),(8083,'control')]:
 server=ThreadingHTTPServer(('0.0.0.0',port),Handler);server.label=label
 threading.Thread(target=server.serve_forever,daemon=True).start()
threading.Event().wait()
'''
from concurrent.futures import ThreadPoolExecutor
extra='''
upstream least { least_conn; server backend:8081; server backend:8082; }
upstream weighted_least { least_conn; server backend:8081 weight=2; server backend:8082; }
'''
locations='''
location /least/ { proxy_pass http://least/; }
location /weighted/ { proxy_pass http://weighted_least/; }
location /control/ { proxy_pass http://backend:8083/; }
'''
with lab(config(locations,extra),backend=backend) as (_,request):
 def get(path):
 status,_,body=request(path);assert status==200;return json.loads(body)
 def state():return get('/control/state')
 results={}
 with ThreadPoolExecutor(max_workers=3) as pool:
 held=pool.submit(get,'/least/hold/first');wait(lambda:sum(state()['active'].values())==1)
 occupied=next(peer for peer,value in state()['active'].items() if value==1)
 quick=[get('/least/quick')['peer'] for _ in range(6)]
 assert all(peer!=occupied for peer in quick),quick
 get('/control/release?path=/hold/first');assert held.result()['peer']==occupied
 wait(lambda:sum(state()['active'].values())==0)
 results['held_peer']=occupied;results['quick_while_held']=quick
 futures=[]
 for index in range(3):
 futures.append(pool.submit(get,'/weighted/hold/w'+str(index)))
 wait(lambda:sum(state()['active'].values())==index+1)
 snapshot=state()['active'];assert snapshot=={'A':2,'B':1},snapshot
 for index in range(3):get('/control/release?path=/hold/w'+str(index))
 results['weighted_active']=snapshot;results['weighted_completed']=[f.result()['peer'] for f in futures]
 wait(lambda:sum(state()['active'].values())==0)
 print(json.dumps(results,sort_keys=True))

The first request enters the equally weighted least-connections group and waits inside one backend handler. The controller observes exactly one active request and identifies the occupied peer before sending anything else. Which peer wins the initial tie is not the important assertion; the fixture discovers that identity from the backend’s actual state.

Six short requests then complete through the same upstream group. All choose the other peer while the first request remains held. Only afterward does the controller release the original response and verify that it completes on its original peer. This demonstrates selection based on ongoing occupancy, rather than merely counting how many requests each server has answered over its lifetime.

The second group assigns weight two to A and weight one to B. Three requests are started one at a time, with each left unfinished and its arrival observed before the next starts. The active snapshot becomes A=2 and B=1. All three are then released and complete successfully, showing how weights alter the effective comparison instead of always preferring the smallest raw number.

The experiment’s HTTP exchanges keep one active request on each selected upstream connection. The two peer endpoints share a disposable container, so their names do not represent separate hardware capacity. The gates demonstrate occupancy and placement; they do not create a CPU benchmark or validate a particular weight for production machines.

Compare the observed metric with the resource you need to protect

Ask what keeps an upstream connection active in the real path. A slow client may hold it longer when response buffering is disabled, while a buffered proxy can sometimes finish reading upstream before that client finishes downloading. Changing buffering can therefore change the count seen by least connections without changing the application’s intrinsic computational cost.

Measure per-peer latency, errors, queueing, and resource use alongside connection counts. If one peer is slow because a dependency is failing, directing fewer new connections there can reduce pressure but does not repair it. Health eligibility and retry policy remain separate decisions. Least connections is not an active health probe or a gradual recovery ramp for a newly returned peer.

Finally, validate the scope and protocol of the deployed implementation. A method named least connections in another product can have different weighting, persistence, and state-sharing rules. Test a held request, release, and weighted competition through the actual balancer, and use the results to justify the selected metric rather than treating the algorithm’s name as a general capacity guarantee.

Sources

nginx upstream module, least_conn weighting, ties, shared zones, and active connection state, checked 2026-09-12.
nginx HTTP load balancing guide, least-connected selection and its workload rationale, checked 2026-09-12.

Choose an nginx balancing method by the decision it actually makes

Tue, 18 May 2027 00:00:00 +0000

An upstream group names candidate backend endpoints, and its balancing method chooses among eligible peers. Round robin distributes selections by weight; least connections considers active connections and weights; hash methods use a selected key for affinity. These decisions describe routing policy. They do not automatically measure CPU utilization, guarantee equal response times, or make local session state durable when a peer disappears.

When it breaks

A common measurement error is inspecting the wrong address. curl’s remote IP identifies the server that curl connected to, which is usually the proxy in this architecture. An access log’s client-address field is likewise not a backend distribution report. To assess peer selection, inspect actual upstream addresses or a controlled backend identity, and retain attempt details when retries can contact several peers.

Even request counts do not imply even work. A long report and a tiny cached response each count as a request but can occupy very different resources. A configured weight expresses an intended relative share; it is not an automatic measurement of machine capacity. Doubling a weight does not prove that per-core load or latency became equal across different backends.

Affinity has different failure modes. Many users can share one source address, so IP hashing can concentrate traffic rather than distribute individual users. A failed or changed peer set can alter placement, and local session state does not move merely because requests do. ip_hash should not be described as nginx’s configurable consistent hash ring; the explicit hash ... consistent method is separate.

Check it quickly

Save this as check-balancing.py and run it with Python, Docker, and the pinned images. The recorded setup uses one nginx 1.28.0 worker and two identifiable HTTP peers on separate ports inside one disposable backend container. The shared machine is intentional: the test measures selection, not comparative hardware capacity. All client access uses host loopback:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
 docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port;request.network=name
 request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import itertools,json,threading
lock=threading.Lock();ids=itertools.count(1);counts={'A':0,'B':0};active={'A':0,'B':0};faults={'A':200,'B':200};gates={}
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def setup(self):super().setup();self.ident=next(ids)
 def log_message(self,*args):pass
 def do_GET(self):
 parsed=urlsplit(self.path);label=self.server.label;code=200
 if label=='control':
 query=parse_qs(parsed.query)
 with lock:
 if parsed.path=='/fail':faults[query['peer'][0]]=int(query['status'][0])
 if parsed.path=='/release':gates[query['path'][0]].set()
 data={'counts':dict(counts),'active':dict(active),'faults':dict(faults),'held':list(gates)}
 else:
 with lock:
 counts[label]+=1;active[label]+=1;code=faults[label]
 if parsed.path.startswith('/hold/'):gates[parsed.path]=threading.Event()
 if parsed.path.startswith('/hold/'):assert gates[parsed.path].wait(20)
 data={'peer':label,'path':self.path,'connection':self.ident}
 body=json.dumps(data,sort_keys=True).encode();self.send_response(code)
 self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)))
 self.end_headers();self.wfile.write(body)
 if label!='control':
 with lock:active[label]-=1
for port,label in [(8081,'A'),(8082,'B'),(8083,'control')]:
 server=ThreadingHTTPServer(('0.0.0.0',port),Handler);server.label=label
 threading.Thread(target=server.serve_forever,daemon=True).start()
threading.Event().wait()
'''
from collections import Counter
extra='''
upstream round { server backend:8081; server backend:8082; }
upstream weighted { server backend:8081 weight=2; server backend:8082; }
upstream by_ip { ip_hash; server backend:8081; server backend:8082; }
upstream by_key { hash $arg_key consistent; server backend:8081; server backend:8082; }
add_header X-Upstream $upstream_addr always; add_header X-Peer $remote_addr always;
'''
locations='\n'.join(f'location /{path}/ {{ proxy_pass http://{group}/; }}' for path,group in
 [('round','round'),('weighted','weighted'),('sticky','by_ip'),('key','by_key')])
with lab(config(locations,extra),backend=backend) as (_,request):
 def get(path,headers=None):
 status,h,b=request(path,headers=headers);assert status==200;data=json.loads(b)
 assert h['x-upstream'].endswith(':8081' if data['peer']=='A' else ':8082')
 return data['peer'],h
 results={}
 for mode,expected in [('round',{'A':6,'B':6}),('weighted',{'A':8,'B':4})]:
 chosen=[get('/'+mode+'/item')[0] for _ in range(12)]
 assert dict(Counter(chosen))==expected,chosen;results[mode]={'sequence':chosen,'counts':dict(Counter(chosen))}
 sticky=[get('/sticky/item',{'X-Forwarded-For':'192.0.2.'+str(i)}) for i in range(1,9)]
 assert len({r[0] for r in sticky})==len({r[1]['x-peer'] for r in sticky})==1
 results['ip_hash']={'peers':[r[0] for r in sticky],'socket_peer':sticky[0][1]['x-peer']}
 groups={}
 for key in ['alpha','beta','gamma','delta','epsilon','zeta']:
 pair=[get('/key/item?key='+key)[0] for _ in range(2)];assert pair[0]==pair[1];groups[key]=pair[0]
 assert set(groups.values())=={'A','B'},groups;results['consistent_key_hash']=groups
 print(json.dumps(results,sort_keys=True))

Twelve sequential requests through the equally weighted round-robin group produce six responses from A and six from B. In this controlled run, they alternate. Each response’s backend marker is checked against the port in nginx’s upstream-address header, so the distribution is observed at the selected peer rather than inferred from the client’s connection to nginx.

With A assigned weight two and B weight one, twelve requests produce eight A responses and four B responses. This demonstrates the configured relative share for healthy peers under these serial requests. It does not promise that every arbitrary prefix of a traffic stream, a retried request set, or a fleet with unavailable peers will have exactly that ratio.

The IP-hash group receives eight requests from the same actual socket peer. Their fabricated X-Forwarded-For values vary, but every response still comes from one backend. No real-IP trust configuration converts those untrusted headers into the client address. The observed socket-peer field stays constant, making clear which identity the method is actually using in this setup.

The separate consistent-hash group uses a public query key. Each of six keys is requested twice, and both requests for a key choose the same backend; the chosen keys cover both peers. This checks stable placement for the current group. It does not test membership changes or prove that an arbitrary client-controlled key is an authenticated user identity.

Match the method to the workload and its identity

For similar independent requests, weighted round robin offers a simple relative distribution. For variable connection occupancy, least connections can steer around peers with more active connections, subject to its own metric limitations. For affinity, select a stable, appropriate key and decide what should happen when placement changes. Routing stickiness cannot replace replication, shared state, or an explicit session-loss policy.

nginx IP hashing uses the first three octets of an IPv4 address or the full IPv6 address. That documented granularity matters when estimating distribution; it is not always a full-address hash with one independent outcome per IPv4 host. Proxy address handling and NAT can further change how many distinct identities the balancer actually observes.

Finally, evaluate selection together with eligibility, retries, and per-peer response behavior. Compare upstream attempt addresses and timing with returned content, rather than assuming a slower peer proves the algorithm is wrong. A method change can redistribute work, but it does not repair a broken backend or make an unsafe request safe to retry on another node.

Sources

nginx upstream module, weights, ip_hash, consistent key hashing, and peer selection, checked 2026-09-12.
nginx HTTP load balancing guide, round robin, least connections, affinity, and passive failure handling, checked 2026-09-12.
nginx proxy module, upstream forwarding and retry conditions, checked 2026-09-12.

Passive upstream health follows failed traffic, not a probe timer

Tue, 05 Jan 2027 00:00:00 +0000

nginx passive upstream health uses outcomes of real requests to decide whether a peer should temporarily stop receiving traffic. max_fails sets an unsuccessful-attempt threshold, while fail_timeout supplies both the counting interval and the unavailability period. These settings do not schedule periodic application probes. Active checks require a supported checking capability and configuration beyond this passive request-outcome policy.

When it breaks

Not every error-looking HTTP response counts as an unsuccessful upstream attempt. The proxy retry configuration helps define that classification. In the tested setup, HTTP 503 only contributes to passive failure handling when http_503 is included. Treating every 5xx as an automatic health failure can leave an operator expecting exclusion while nginx continues selecting the peer according to its actual rules.

Another mistake is treating the timeout window as proof of recovery. When the unavailable period ends, a peer can become eligible for real traffic again without a successful independent health probe first. A still-broken backend can make that request fail. least_conn changes selection based on active connections; it is not a gradual recovery ramp or a replacement for a supported slow-start mechanism.

Retries have boundaries as well. Switching peers cannot transparently repair an error after response data has already been sent to the client. Replaying an operation with side effects can create duplicates. The following fixture uses GET requests and failures returned before a successful body, with an explicit attempt limit; it does not establish safe retry semantics for every application method.

Check it quickly

Save this as check-passive-health.py and run it with Python, Docker, and the pinned images. The recorded server is nginx 1.28.0 with one worker. Two HTTP endpoints in a disposable container report their identities and can be instructed to return 503. A separate control listener exposes counts without generating requests through the balanced groups:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
 docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port;request.network=name
 request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import itertools,json,threading
lock=threading.Lock();ids=itertools.count(1);counts={'A':0,'B':0};active={'A':0,'B':0};faults={'A':200,'B':200};gates={}
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def setup(self):super().setup();self.ident=next(ids)
 def log_message(self,*args):pass
 def do_GET(self):
 parsed=urlsplit(self.path);label=self.server.label;code=200
 if label=='control':
 query=parse_qs(parsed.query)
 with lock:
 if parsed.path=='/fail':faults[query['peer'][0]]=int(query['status'][0])
 if parsed.path=='/release':gates[query['path'][0]].set()
 data={'counts':dict(counts),'active':dict(active),'faults':dict(faults),'held':list(gates)}
 else:
 with lock:
 counts[label]+=1;active[label]+=1;code=faults[label]
 if parsed.path.startswith('/hold/'):gates[parsed.path]=threading.Event()
 if parsed.path.startswith('/hold/'):assert gates[parsed.path].wait(20)
 data={'peer':label,'path':self.path,'connection':self.ident}
 body=json.dumps(data,sort_keys=True).encode();self.send_response(code)
 self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)))
 self.end_headers();self.wfile.write(body)
 if label!='control':
 with lock:active[label]-=1
for port,label in [(8081,'A'),(8082,'B'),(8083,'control')]:
 server=ThreadingHTTPServer(('0.0.0.0',port),Handler);server.label=label
 threading.Thread(target=server.serve_forever,daemon=True).start()
threading.Event().wait()
'''
extra='''
upstream passive { server backend:8081 max_fails=1 fail_timeout=2s; server backend:8082; }
upstream plain { server backend:8081 max_fails=1 fail_timeout=30s; server backend:8082; }
upstream single { server backend:8081 max_fails=1 fail_timeout=30s; }
add_header X-Upstream $upstream_addr always; add_header X-Attempts $upstream_status always;
'''
locations='''
location /passive { proxy_next_upstream error timeout http_503; proxy_next_upstream_tries 2; proxy_pass http://passive; }
location /plain { proxy_next_upstream error timeout; proxy_pass http://plain; }
location /single { proxy_next_upstream error timeout http_503; proxy_pass http://single; }
location /control/ { proxy_pass http://backend:8083/; }
'''
active_config=config('location / { proxy_pass http://peers; health_check; }',
 'upstream peers { zone peers 64k; server backend:8081; server backend:8082; }')
with lab(config(locations,extra),{'active.conf':active_config},backend) as (proxy,request):
 def control(path):
 status,_,body=request('/control/'+path);assert status==200;return json.loads(body)
 def call(path):
 status,h,b=request(path);return {'status':status,'attempts':h['x-attempts'],'upstream':h['x-upstream'],'peer':json.loads(b)['peer']}
 control('fail?peer=A&status=503');results={}
 plain=[call('/plain') for _ in range(6)]
 assert [r['status'] for r in plain].count(503)==3 and all(',' not in r['attempts'] for r in plain)
 results['unlisted_http_503']=plain
 for _ in range(2):
 first=call('/passive')
 if first['attempts']=='503, 200':break
 assert first['status']==200 and first['attempts']=='503, 200' and first['peer']=='B',first
 results['failed_attempt_then_success']=first
 after_failure=control('state')['counts']['A']
 excluded=[call('/passive') for _ in range(4)]
 assert all(r['peer']=='B' and r['attempts']=='200' for r in excluded)
 assert control('state')['counts']['A']==after_failure
 time.sleep(.4);assert control('state')['counts']['A']==after_failure
 results['during_exclusion']=excluded;results['background_probe_count']=0
 control('fail?peer=A&status=200');deadline=time.monotonic()+6;recovered=None
 while time.monotonic()<deadline:
 observed=call('/passive')
 if observed['peer']=='A':recovered=observed;break
 time.sleep(.1)
 assert recovered and recovered['status']==200;results['live_request_after_window']=recovered
 control('fail?peer=A&status=503');before=control('state')['counts']['A']
 single=[call('/single') for _ in range(2)]
 assert all(r['status']==503 and r['attempts']=='503' for r in single)
 assert control('state')['counts']['A']==before+2;results['single_peer_still_attempted']=single
 probe=docker('exec',proxy,'nginx','-t','-c','/srv/active.conf',check=False)
 assert probe.returncode!=0 and 'unknown directive "health_check"' in probe.stderr
 results['active_health_check_directive_supported']=False
 print(json.dumps(results,sort_keys=True))

First, peer A returns actual HTTP 503 while B returns 200. A comparison group retries only connection errors and timeouts. Six requests still select A three times and B three times, with no multi-attempt response. A’s max_fails=1 does not remove it for that unlisted HTTP outcome. The fixture inspects statuses and selected upstream addresses, not only the final response body.

The passive group includes http_503, allows two tries, and uses max_fails=1 fail_timeout=2s for A. A request encounters A’s 503, then succeeds on B with final 200. nginx reports upstream statuses 503, 200 and both addresses. Those values are real backend HTTP responses in this experiment, not synthetic labels inferred from a connection refusal.

Four subsequent requests use only B during A’s exclusion period. A’s request count remains unchanged, including through a controlled pause, demonstrating that this configuration is not sending background health probes. The backend is then made healthy. After the configured window, a live request reaches A and succeeds; the test waits for that observed selection instead of assuming an exact clock boundary.

A separate group contains only A. Even with max_fails=1 fail_timeout=30s, two requests still reach that failing peer and return 503. nginx ignores those passive-unavailability parameters for a single-server group. There is no alternative peer to select, so the multi-peer exclusion behavior cannot be assumed to create availability where the topology provides none.

Finally, a disposable candidate configuration includes health_check. The tested binary rejects the directive as unknown. This verifies the installed capability boundary directly: the passive settings did not enable active checking. The valid running configuration is retained; no unverified module or commercial capability is represented as available merely because it appears in general documentation.

Design failure detection with the service’s response contract

Choose which outcomes indicate an unusable peer and which are legitimate application responses. An authorization failure or missing object need not mean the server is unhealthy. Passive detection also depends on receiving traffic, so a quiet but broken peer may remain untested until a user request arrives. Active checking, where available, answers a different timing question.

Define a meaningful health endpoint if deploying active checks, and understand which dependencies it exercises. A TCP accept alone does not prove a useful application response. Conversely, an overly broad dependency check can remove every peer for a shared downstream problem. Keep the check’s success condition aligned with the traffic that removing a peer would actually protect.

Observe final client status together with attempt statuses, peer addresses, latency, and exclusion state. A final 200 can conceal a failed first attempt and extra delay. Verify the deployed version’s capabilities, retry limits, and recovery path so the availability claim matches what clients experience rather than what the word “health” suggests.

Sources

nginx upstream module, max_fails, both fail_timeout meanings, single-peer behavior, and slow start, checked 2026-09-12.
nginx proxy module, unsuccessful-attempt classification, retry conditions, and retry limits, checked 2026-09-12.
nginx HTTP health check guide, passive versus active checks and product capabilities, checked 2026-09-12.

Transport balancing keeps a connection together; HTTP routing can split its requests

Wed, 28 Oct 2026 00:00:00 +0000

The useful distinction between transport and HTTP balancing is where a routing decision can be made. A TCP relay can select a backend for a connection and forward its byte stream without interpreting HTTP requests. An HTTP proxy parses requests and can choose an upstream using their paths or headers. Several requests on one client connection can therefore have different routing outcomes.

When it breaks

Expecting a TCP connection choice to route each embedded URL separately leads to requests reaching the wrong application. Once the stream is relayed to one backend, a later HTTP path does not automatically trigger a new peer selection. Conversely, treating HTTP routing as permanent connection affinity can surprise an application that assumes every request from that client connection reaches the same local state.

The layer label does not fully describe TLS handling. TLS termination and HTTP interpretation are separate capabilities. A stream proxy can terminate TLS without becoming an HTTP path router, or inspect selected ClientHello metadata without decrypting application data. SNI-based selection is not visibility into encrypted URLs, headers, or response bodies. Describe the configured behavior instead of assuming a product label proves all those properties.

Performance claims need the same care. A userspace TCP relay and kernel packet forwarding have different resource paths, even when both are called transport balancing. An HTTP proxy performs parsing and policy work, but its relative throughput depends on implementation and workload. No universal near-wire-speed or fixed-overhead conclusion follows merely from choosing the terms L4 and L7.

Check it quickly

Save this as check-routing-layers.py and run it with Python, Docker, and the pinned images. The recorded server is nginx 1.28.0 with its stream module available. Two HTTP peers expose distinct identities and backend connection numbers. nginx publishes separate HTTP-routing and TCP-relay listeners only on host loopback, allowing the same request sequence through both:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def wait(predicate):
 deadline=time.monotonic()+15
 while time.monotonic()<deadline:
 try:
 if predicate():return
 except OSError:pass
 time.sleep(.1)
 raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
 name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
 try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
 for path,content in (files or {}).items():
 target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
 target.write_bytes(content if isinstance(content,bytes) else content.encode())
 docker('network','create',name)
 if backend is not None:
 (root/'backend.py').write_text(backend)
 (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
 docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
 docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
 wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
 (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
 docker('build','--network','none','-t',tags[0],str(root))
 docker('run','--rm','--network',name,tags[0],'nginx','-t')
 published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
 docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
 details=json.loads(docker('inspect',containers[0]).stdout)[0]
 binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
 def ready():
 connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
 wait(ready)
 def request(path,method='GET',headers=None,body=None):
 connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
 try:
 connection.request(method,path,body=body,headers=headers or {})
 response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
 finally:connection.close()
 request.port=port;request.network=name
 request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
 yield containers[0],request
 finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
 return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import itertools,json,threading
lock=threading.Lock();ids=itertools.count(1);counts={'A':0,'B':0};active={'A':0,'B':0};faults={'A':200,'B':200};gates={}
class Handler(BaseHTTPRequestHandler):
 protocol_version='HTTP/1.1'
 def setup(self):super().setup();self.ident=next(ids)
 def log_message(self,*args):pass
 def do_GET(self):
 parsed=urlsplit(self.path);label=self.server.label;code=200
 if label=='control':
 query=parse_qs(parsed.query)
 with lock:
 if parsed.path=='/fail':faults[query['peer'][0]]=int(query['status'][0])
 if parsed.path=='/release':gates[query['path'][0]].set()
 data={'counts':dict(counts),'active':dict(active),'faults':dict(faults),'held':list(gates)}
 else:
 with lock:
 counts[label]+=1;active[label]+=1;code=faults[label]
 if parsed.path.startswith('/hold/'):gates[parsed.path]=threading.Event()
 if parsed.path.startswith('/hold/'):assert gates[parsed.path].wait(20)
 data={'peer':label,'path':self.path,'connection':self.ident}
 body=json.dumps(data,sort_keys=True).encode();self.send_response(code)
 self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)))
 self.end_headers();self.wfile.write(body)
 if label!='control':
 with lock:active[label]-=1
for port,label in [(8081,'A'),(8082,'B'),(8083,'control')]:
 server=ThreadingHTTPServer(('0.0.0.0',port),Handler);server.label=label
 threading.Thread(target=server.serve_forever,daemon=True).start()
threading.Event().wait()
'''
from collections import Counter
locations='''
location /a { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass http://backend:8081; }
location /b { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass http://backend:8082; }
'''
configuration=config(locations)+'''
stream {
 upstream transport { server backend:8081; server backend:8082; }
 server { listen 8084; proxy_pass transport; }
}
'''
with lab(configuration,backend=backend,extra_ports=(8084,)) as (_,request):
 def exchange(connection,path):
 connection.request('GET',path);response=connection.getresponse();data=json.loads(response.read())
 assert response.status==200;return data
 l7_connection=http.client.HTTPConnection('127.0.0.1',request.port,timeout=5)
 a=exchange(l7_connection,'/a');client_socket=l7_connection.sock;assert client_socket is not None
 b=exchange(l7_connection,'/b')
 assert a['peer']=='A' and b['peer']=='B' and l7_connection.sock is client_socket;l7_connection.close()
 results={'http_one_client_connection':[a,b]};transport=[]
 for _ in range(4):
 raw=http.client.HTTPConnection('127.0.0.1',request.ports[8084],timeout=5)
 a=exchange(raw,'/a');client_socket=raw.sock;assert client_socket is not None
 b=exchange(raw,'/b')
 assert a['peer']==b['peer'] and a['connection']==b['connection'] and raw.sock is client_socket
 transport.append([a,b]);raw.close()
 assert dict(Counter(pair[0]['peer'] for pair in transport))=={'A':2,'B':2}
 results['tcp_connection_choices']=transport
 print(json.dumps(results,sort_keys=True))

On the HTTP listener, the client sends /a and then /b over one persistent HTTP/1.1 connection. The /a location selects peer A, while /b selects peer B. The fixture confirms that the client socket object remains the same across both exchanges. Changing the upstream target therefore did not require the client to establish another connection to nginx.

On the TCP listener, each new client connection is assigned one peer by the stream upstream group. The client again sends /a followed by /b, but both responses come from the same peer and carry the same backend connection identity. The HTTP paths still reach the application; they simply are not used as routing input by this TCP proxy configuration.

Four separate TCP client connections choose A twice and B twice in the controlled run. Within every connection, both requests stay together. The example therefore distinguishes distribution across connections from distribution across HTTP requests. A balanced count of TCP connections does not imply that the number, cost, or latency of requests inside those connections is equally distributed.

The peers share a disposable container on different ports. This keeps identities and routing easy to observe without pretending to compare two machines. The experiment uses plain persistent HTTP/1.1 carried over TCP; it does not test TLS negotiation, UDP flow handling, HTTP/2 multiplexing, or HTTP/3. Those protocol capabilities require their own configured paths and observations.

Place policy where its required information is available

Use HTTP routing when the decision depends on a path, method, or HTTP header. For HTTPS traffic, the component applying such policy needs access to the decrypted requests. Use a transport path when preserving the connection’s stream is the required contract and per-request HTTP inspection is unnecessary. Keep certificate ownership and encryption boundaries explicit rather than treating either choice as automatically secure.

When layering proxies, document each hop’s role: which component accepts client connections, which decrypts, which interprets HTTP, and which selects the application peer. Timeouts, source-address propagation, health eligibility, and retry behavior are not guaranteed by the diagram’s layer names. In particular, moving an established byte stream between backends is not equivalent to retrying a new HTTP request.

Finally, choose observability that matches the decision unit. Record connection identities for transport routing and request plus upstream identities for HTTP routing. Reuse one client connection during tests; otherwise repeated fresh connections can conceal the distinction. Verify that the routing behavior preserves the application’s state assumptions before using an even-looking distribution as evidence of correctness.

Sources

nginx stream proxy module, TCP connection forwarding and stream proxy behavior, checked 2026-09-12.
nginx HTTP proxy module, HTTP request forwarding and location-specific upstream selection, checked 2026-09-12.
nginx stream SSL preread module, reading ClientHello metadata without TLS termination, checked 2026-09-12.
nginx stream SSL module, TLS termination as a separate stream capability, checked 2026-09-12.

Load-Balancing · XB Field Notes

Least connections measures occupied connections, not every kind of load

When it breaks

Check it quickly

Compare the observed metric with the resource you need to protect

Sources

Choose an nginx balancing method by the decision it actually makes

When it breaks

Check it quickly

Match the method to the workload and its identity

Sources

Passive upstream health follows failed traffic, not a probe timer

When it breaks

Check it quickly

Design failure detection with the service’s response contract

Sources

Transport balancing keeps a connection together; HTTP routing can split its requests

When it breaks

Check it quickly

Place policy where its required information is available

Further reading

Sources