The useful distinction between transport and HTTP balancing is where a routing decision can be made. A TCP relay can select a backend for a connection and forward its byte stream without interpreting HTTP requests. An HTTP proxy parses requests and can choose an upstream using their paths or headers. Several requests on one client connection can therefore have different routing outcomes.

When it breaks

Expecting a TCP connection choice to route each embedded URL separately leads to requests reaching the wrong application. Once the stream is relayed to one backend, a later HTTP path does not automatically trigger a new peer selection. Conversely, treating HTTP routing as permanent connection affinity can surprise an application that assumes every request from that client connection reaches the same local state.

The layer label does not fully describe TLS handling. TLS termination and HTTP interpretation are separate capabilities. A stream proxy can terminate TLS without becoming an HTTP path router, or inspect selected ClientHello metadata without decrypting application data. SNI-based selection is not visibility into encrypted URLs, headers, or response bodies. Describe the configured behavior instead of assuming a product label proves all those properties.

Performance claims need the same care. A userspace TCP relay and kernel packet forwarding have different resource paths, even when both are called transport balancing. An HTTP proxy performs parsing and policy work, but its relative throughput depends on implementation and workload. No universal near-wire-speed or fixed-overhead conclusion follows merely from choosing the terms L4 and L7.

Check it quickly

Save this as check-routing-layers.py and run it with Python, Docker, and the pinned images. The recorded server is nginx 1.28.0 with its stream module available. Two HTTP peers expose distinct identities and backend connection numbers. nginx publishes separate HTTP-routing and TCP-relay listeners only on host loopback, allowing the same request sequence through both:

from contextlib import contextmanager
from pathlib import Path
from tempfile import TemporaryDirectory
import http.client,json,socket,subprocess,time,uuid

NGINX='nginx@sha256:552e7481ca93ffccd046aa658dbbed22caefbc09c66fa7cd247cbb90b8a5c609'
PYTHON='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def wait(predicate):
    deadline=time.monotonic()+15
    while time.monotonic()<deadline:
        try:
            if predicate():return
        except OSError:pass
        time.sleep(.1)
    raise AssertionError('fixture did not become ready')

@contextmanager
def lab(configuration,files=None,backend=None,extra_ports=()):
    name='xbcatm-nginx-'+uuid.uuid4().hex[:10];tags=[name+':proxy',name+':backend'];containers=[name+'-proxy',name+'-backend']
    try:
        with TemporaryDirectory(prefix=name+'-') as directory:
            root=Path(directory);(root/'files').mkdir();(root/'nginx.conf').write_text(configuration)
            for path,content in (files or {}).items():
                target=root/'files'/path;target.parent.mkdir(parents=True,exist_ok=True)
                target.write_bytes(content if isinstance(content,bytes) else content.encode())
            docker('network','create',name)
            if backend is not None:
                (root/'backend.py').write_text(backend)
                (root/'backend.Dockerfile').write_text(f'FROM {PYTHON}\nCOPY backend.py /backend.py\nCMD ["python", "/backend.py"]\n')
                docker('build','--network','none','-f',str(root/'backend.Dockerfile'),'-t',tags[1],str(root))
                docker('run','-d','--name',containers[1],'--network',name,'--network-alias','backend',tags[1])
                wait(lambda:docker('exec',containers[1],'python','-c','import socket;s=socket.create_connection(("127.0.0.1",8081),timeout=.5);s.close()',check=False).returncode==0)
            (root/'Dockerfile').write_text(f'FROM {NGINX}\nCOPY nginx.conf /etc/nginx/nginx.conf\nCOPY files /srv\n')
            docker('build','--network','none','-t',tags[0],str(root))
            docker('run','--rm','--network',name,tags[0],'nginx','-t')
            published=[arg for value in (8080,*extra_ports) for arg in ('-p','127.0.0.1::'+str(value))]
            docker('run','-d','--name',containers[0],'--network',name,*published,tags[0])
            details=json.loads(docker('inspect',containers[0]).stdout)[0]
            binding=details['NetworkSettings']['Ports']['8080/tcp'][0];assert binding['HostIp']=='127.0.0.1';port=int(binding['HostPort'])
            def ready():
                connection=socket.create_connection(('127.0.0.1',port),timeout=.5);connection.close();return True
            wait(ready)
            def request(path,method='GET',headers=None,body=None):
                connection=http.client.HTTPConnection('127.0.0.1',port,timeout=5)
                try:
                    connection.request(method,path,body=body,headers=headers or {})
                    response=connection.getresponse();return response.status,{k.lower():v for k,v in response.getheaders()},response.read()
                finally:connection.close()
            request.port=port;request.network=name
            request.ports={int(key.split('/')[0]):int(value[0]['HostPort']) for key,value in details['NetworkSettings']['Ports'].items() if value}
            yield containers[0],request
    finally:
        docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False);docker('network','rm',name,check=False)

def config(locations,extra=''):
    return 'worker_processes 1;\nerror_log /tmp/error.log notice;\nevents {}\nhttp {\naccess_log off;\n'+extra+'\nserver { listen 8080; root /srv;\n'+locations+'\n}\n}\n'
backend=r'''
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
from urllib.parse import urlsplit,parse_qs
import itertools,json,threading
lock=threading.Lock();ids=itertools.count(1);counts={'A':0,'B':0};active={'A':0,'B':0};faults={'A':200,'B':200};gates={}
class Handler(BaseHTTPRequestHandler):
    protocol_version='HTTP/1.1'
    def setup(self):super().setup();self.ident=next(ids)
    def log_message(self,*args):pass
    def do_GET(self):
        parsed=urlsplit(self.path);label=self.server.label;code=200
        if label=='control':
            query=parse_qs(parsed.query)
            with lock:
                if parsed.path=='/fail':faults[query['peer'][0]]=int(query['status'][0])
                if parsed.path=='/release':gates[query['path'][0]].set()
                data={'counts':dict(counts),'active':dict(active),'faults':dict(faults),'held':list(gates)}
        else:
            with lock:
                counts[label]+=1;active[label]+=1;code=faults[label]
                if parsed.path.startswith('/hold/'):gates[parsed.path]=threading.Event()
            if parsed.path.startswith('/hold/'):assert gates[parsed.path].wait(20)
            data={'peer':label,'path':self.path,'connection':self.ident}
        body=json.dumps(data,sort_keys=True).encode();self.send_response(code)
        self.send_header('Content-Type','application/json');self.send_header('Content-Length',str(len(body)))
        self.end_headers();self.wfile.write(body)
        if label!='control':
            with lock:active[label]-=1
for port,label in [(8081,'A'),(8082,'B'),(8083,'control')]:
    server=ThreadingHTTPServer(('0.0.0.0',port),Handler);server.label=label
    threading.Thread(target=server.serve_forever,daemon=True).start()
threading.Event().wait()
'''
from collections import Counter
locations='''
location /a { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass http://backend:8081; }
location /b { proxy_http_version 1.1; proxy_set_header Connection ""; proxy_pass http://backend:8082; }
'''
configuration=config(locations)+'''
stream {
    upstream transport { server backend:8081; server backend:8082; }
    server { listen 8084; proxy_pass transport; }
}
'''
with lab(configuration,backend=backend,extra_ports=(8084,)) as (_,request):
    def exchange(connection,path):
        connection.request('GET',path);response=connection.getresponse();data=json.loads(response.read())
        assert response.status==200;return data
    l7_connection=http.client.HTTPConnection('127.0.0.1',request.port,timeout=5)
    a=exchange(l7_connection,'/a');client_socket=l7_connection.sock;assert client_socket is not None
    b=exchange(l7_connection,'/b')
    assert a['peer']=='A' and b['peer']=='B' and l7_connection.sock is client_socket;l7_connection.close()
    results={'http_one_client_connection':[a,b]};transport=[]
    for _ in range(4):
        raw=http.client.HTTPConnection('127.0.0.1',request.ports[8084],timeout=5)
        a=exchange(raw,'/a');client_socket=raw.sock;assert client_socket is not None
        b=exchange(raw,'/b')
        assert a['peer']==b['peer'] and a['connection']==b['connection'] and raw.sock is client_socket
        transport.append([a,b]);raw.close()
    assert dict(Counter(pair[0]['peer'] for pair in transport))=={'A':2,'B':2}
    results['tcp_connection_choices']=transport
    print(json.dumps(results,sort_keys=True))

On the HTTP listener, the client sends /a and then /b over one persistent HTTP/1.1 connection. The /a location selects peer A, while /b selects peer B. The fixture confirms that the client socket object remains the same across both exchanges. Changing the upstream target therefore did not require the client to establish another connection to nginx.

On the TCP listener, each new client connection is assigned one peer by the stream upstream group. The client again sends /a followed by /b, but both responses come from the same peer and carry the same backend connection identity. The HTTP paths still reach the application; they simply are not used as routing input by this TCP proxy configuration.

Four separate TCP client connections choose A twice and B twice in the controlled run. Within every connection, both requests stay together. The example therefore distinguishes distribution across connections from distribution across HTTP requests. A balanced count of TCP connections does not imply that the number, cost, or latency of requests inside those connections is equally distributed.

The peers share a disposable container on different ports. This keeps identities and routing easy to observe without pretending to compare two machines. The experiment uses plain persistent HTTP/1.1 carried over TCP; it does not test TLS negotiation, UDP flow handling, HTTP/2 multiplexing, or HTTP/3. Those protocol capabilities require their own configured paths and observations.

Place policy where its required information is available

Use HTTP routing when the decision depends on a path, method, or HTTP header. For HTTPS traffic, the component applying such policy needs access to the decrypted requests. Use a transport path when preserving the connection’s stream is the required contract and per-request HTTP inspection is unnecessary. Keep certificate ownership and encryption boundaries explicit rather than treating either choice as automatically secure.

When layering proxies, document each hop’s role: which component accepts client connections, which decrypts, which interprets HTTP, and which selects the application peer. Timeouts, source-address propagation, health eligibility, and retry behavior are not guaranteed by the diagram’s layer names. In particular, moving an established byte stream between backends is not equivalent to retrying a new HTTP request.

Finally, choose observability that matches the decision unit. Record connection identities for transport routing and request plus upstream identities for HTTP routing. Reuse one client connection during tests; otherwise repeated fresh connections can conceal the distinction. Verify that the routing behavior preserves the application’s state assumptions before using an even-looking distribution as evidence of correctness.

Further reading

Sources