The cluster schedules on requests but enforces limits, so a container that sets only one side leaves a half-open contract. Requests-only pods schedule honestly but burst without bound, free to starve neighbors when they spike. Limits-only pods are capped at runtime yet scheduled as if they cost nothing, so the scheduler packs them densely until the node saturates. Either half-contract fails under pressure; the two numbers only work as a pair.

Think of requests as the booking and limits as the house rules: the booking decides which node has room, the rules decide what happens when the guest gets loud. Set both from measurement and the cluster stays predictable.

Two numbers, two jobs

A CPU request pledges what the scheduler reserves on a node, while the limit caps the share a container can hit before the cgroup throttles it. Both matter: requests decide placement, limits decide ceiling. CPU is compressible, so breaching the CPU limit only slows the container — throttling, not death. Memory is not: breaching the memory limit kills the process via OOM, with no slowdown phase in between. That asymmetry drives every sane policy: CPU limits can sit well above requests to absorb bursts, while memory limits should hug the measured working set plus headroom.

Omissions have defaults with teeth, and admission policy can change the exact result. Without a CPU request the scheduler accounts for zero requested CPU unless a LimitRange or an explicit limit supplies a default; the Pod may then be packed too densely and receive little share under contention. Without a memory limit the process can grow until node pressure makes the kubelet reclaim or evict work. Inspect the admitted Pod, not only the source YAML, because namespace defaults may have filled in values you did not write.

The OOM corner

An untuned memory limit is the classic cause of the kernel reclaiming the workload when a spike crosses the advertised cap. If you set a memory limit, give it headroom above the measured working set. The working set is not the average — it is the p95-to-max range across a full cycle including deploys, batch windows, and traffic peaks. Set the limit at average and every routine spike becomes a kill; set it at 2x max and the reservation waste shows up as Pending pods elsewhere. The honest window is max plus twenty to fifty percent, tuned per workload, rechecked after every scaling change.

Java and other runtime-managed workloads deserve special care: heap plus metaspace plus native plus off-heap must all fit under the limit, not just the -Xmx you remember setting. Size the container limit from the observed process RSS, then work backwards to heap flags — never the reverse.

Start near reality

Measure the steady state, set requests near it, keep limits a defensive step above, and revisit whenever the workload shape shifts. Iterating from real usage beats guessing a number that protects nothing. The practical loop: run a week on honest estimates, read p95 usage per container from the metrics server or Prometheus, set requests at that line and limits above max with headroom, then watch throttling and OOM counters for a week. Zero throttling with huge headroom means requests can drop; any throttling on latency-sensitive paths means limits must rise.

Encode the ratio as team policy rather than folklore: 1:1 memory (Guaranteed QoS) for critical stateful services, 1:2 for web tiers, looser for batch — written down with the workload evidence linked, so the next engineer copies the reasoning along with the numbers.

When it breaks

The signature failure is the OOMKilled loop: restarts climbing, exit code 137, last week’s deploy the only change — and the change was a ten-percent traffic lift against a limit set exactly at old max. Diagnose with the usage-versus-limit graph, not the logs: the process did nothing wrong, it simply outgrew its cap. The fix is headroom, and the prevention is alerting on usage-to-limit ratio crossing eighty percent, which pages days before the kill instead of after.

The mirror failure is the throttled-to-death service: p99 latency doubled, CPU flat at the limit line, dashboard blaming the application. CPU throttling is invisible to in-container metrics — the app sees idle time it cannot use. When latency degrades with CPU pinned exactly at the limit, raise the limit before profiling the code.

Check it quickly

Compare booked versus real per container:

kubectl top pods -n prod --sort-by=memory | head
kubectl get pods -n prod -o custom-columns=NAME:.metadata.name,REQ:.spec.containers[*].resources.requests.memory,LIM:.spec.containers[*].resources.limits.memory --no-headers | head -n 15
kubectl get events -n prod --field-selector reason=OOMKilling --sort-by=.lastTimestamp | tail -n 5

Usage shows reality, the manifest dump shows the contract, and the event tail shows where the contract already broke.

Sources