Multiplexing shares the transport and keeps ordering within each stream, which is why long-lived pushes fit this model better than request-reply websockets.

One connection, many streams

HTTP/2 multiplexes many logical streams over a single TCP connection, and gRPC makes each call a stream, so a chatty microservice stops reopening sockets per request. That one shared connection means connection setup, TLS handshake, and idle bookkeeping happen once instead of per call. Streams are independent in ordering and cancellation, so a slow, long-lived stream does not block a short one on the same pipe. The practical effect is a low army of connection state, with per-call overhead shrinking to a frame header.

Streaming beats HTTP/1.1 fan-out

Because gRPC is streaming in both directions, a server can begin sending a partial result before the client finishes sending the request, and a client can keep pushing frames over the same call. That fits search, large-file streaming, log shipping, and interactive feeds where a single reply would force the whole payload to buffer first. It also makes fan-out cheaper than HTTP/1.1, where a burst of calls would each need its own connection and its own handshake. Everything that needs a back-and-forth loop maps naturally onto one long-lived stream instead of a series of separate requests.

Deadlines share the same pipe

The elegance of sharing one connection is real, but it is still bounded by the transport. A retry requires a new stream, not a replay on the same one, and backpressure still matters, since a slow consumer can stall the shared connection for everyone else. Deadlines therefore cut across the whole pipe, so one hair-loss deadline on a stream can surface as latency for every stream on that connection. The practical habit is to give each call a real deadline and a way to cancel, rather than letting the shared pipe silently donate time to someone else’s stuck call.

When it breaks

A stalled flow-control window looks like a healthy connection with zero throughput. HTTP/2 keeps a stream-level and a connection-level window, and only the receiver advances them with WINDOW_UPDATE. A slow consumer first parks its own stream; once the connection window fills, every stream behind it stalls too. Separate the two signals in dashboards, and give each RPC a bounded buffer and a consumer timeout.

Oversize metadata fails fast and retries cannot help. gRPC sends initial metadata as HTTP/2 headers with caps near eight kilobytes. An auth token plus tracing baggage plus tenant tags can cross the limit, and the call dies with RESOURCE_EXHAUSTED before business logic runs. Keep metadata small, move correlation blobs out of band, and remember binary metadata pays base64 inflation.

Long-lived connections defeat naive balancing. Clients that each hold one multiplexed connection pin all streams to the backend chosen at dial time, leaving new pods idle while old ones saturate. Fix it with client-side balancing or proxyless xDS, periodic MaxConnAge reconnects, and a cap on concurrent streams per connection.

Lost deadline propagation turns one slow leaf into whole-tree latency. gRPC carries the budget as grpc-timeout, but a gateway that fans out with a fresh timeout or a background context silently drops it. Derive downstream calls from the incoming context, keep each attempt shorter than the end-to-end deadline, and cancel siblings after the first failure.

Check it quickly

Save the following as mux_sim.py and run python3 mux_sim.py. It uses only the standard library to model one connection carrying three logical streams as interleaved frames. Stream 2 sends one frame and then stalls to simulate an exhausted window, while the scheduler must finish streams 1 and 3. Assertions check per-stream order and wire interleaving.

from collections import deque

# One shared connection, three logical streams. Each stream is an
# ordered queue of application messages; the wire is a single log of
# interleaved (stream_id, payload) frames.
streams = {
    1: deque(["a1", "a2", "a3"]),
    2: deque(["b1", "b2", "b3"]),
    3: deque(["c1", "c2"]),
}
stalled = set()  # streams whose flow-control window is exhausted
wire = []  # shared transport log
delivered = {sid: [] for sid in streams}


def pump():
    """One round-robin pass: each ready stream emits at most one frame."""
    progressed = False
    for sid in sorted(streams):
        if sid in stalled:
            continue
        if streams[sid]:
            payload = streams[sid].popleft()
            wire.append((sid, payload))
            delivered[sid].append(payload)
            progressed = True
    return progressed


def drain():
    while pump():
        pass


# Round one: every stream places its opening frame on the wire.
assert pump() is True
assert wire == [(1, "a1"), (2, "b1"), (3, "c1")], wire

# Stream 2 exhausts its window and stops sending WINDOW_UPDATE.
stalled.add(2)
drain()

# Siblings must complete while stream 2 is stuck on one frame.
assert delivered[1] == ["a1", "a2", "a3"], delivered[1]
assert delivered[3] == ["c1", "c2"], delivered[3]
assert delivered[2] == ["b1"], delivered[2]
assert len(wire) == 6, wire
assert list(streams[2]) == ["b2", "b3"]

# The window reopens: stream 2 resumes on the same connection.
stalled.discard(2)
drain()

assert delivered[2] == ["b1", "b2", "b3"], delivered[2]
for sid, items in delivered.items():
    assert items == sorted(items), (sid, items)  # per-stream order preserved
assert len(wire) == 8, wire
assert all(not q for q in streams.values())

print("wire:", wire)
print("PASS: 3 streams, 8 frames, one shared wire; stall of stream 2 did not block 1 and 3")

The script pumps one round so each stream places its opening frame, then stalls stream 2. The next drain must complete streams 1 and 3 while stream 2 stays at one frame, proving one blocked stream does not block its siblings. Reopening the window drains the rest, and the final check confirms in-order arrival on one log. This models scheduling only, without HPACK, TLS, or real flow-control bytes, so confirm window and deadline behavior on a gRPC stack before tuning production.

Further reading

Sources