cgroup v2 organizes resource control in a unified hierarchy. A workload’s processes can share aggregate budgets, with limits inherited through the hierarchy rather than applied only to one particular PID. CPU bandwidth, memory usage, and task count are separate controllers with different enforcement behavior. Reading the actual cgroup files and failure counters is more useful than treating every configured limit as the same kind of hard stop.

When it breaks

cpu.max limits CPU bandwidth over a period. It can throttle runnable work after the group consumes its quota, but it does not reserve a dedicated processor or guarantee a particular request latency. Multiple threads can consume a shared quota quickly. CPU weights instead influence relative sharing under contention; confusing weights, quotas, and CPU affinity leads to incorrect capacity expectations.

Memory limits do not universally turn a large allocation into a clean application-level error. At memory.max, the kernel can reclaim memory and may invoke the cgroup OOM mechanism if usage cannot be reduced. Some allocation paths have other failure behavior. memory.high applies reclaim pressure and throttling rather than itself invoking OOM, so the two settings should not be described interchangeably.

Task limits also cover threads, not just the number of application processes a dashboard happens to display. Reaching pids.max can make fork or clone fail even when CPU and memory still look available. Hierarchical restrictions mean a generous child value cannot simply override a tighter ancestor budget. The relevant path and controller delegation matter when interpreting visible files.

Check it quickly

Run the script with Python and a Docker daemon using cgroup v2, after obtaining the pinned image. The recorded environment used Docker 29.3.1, Linux 6.12.76, and Python 3.11.16. Every limit applies only to a disposable experiment container. The script performs a bounded memory exhaustion test, removes its containers, and never changes host-wide cgroup settings:

import json,subprocess,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(['docker',*args],text=True,capture_output=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def inspect(name):return json.loads(docker('inspect',name).stdout)[0]

name='xbcatm-budgets-'+uuid.uuid4().hex[:10];containers=[name+'-limits',name+'-oom',name+'-killed']
limits_code='''from pathlib import Path
import errno,json,subprocess,time
root=Path('/sys/fs/cgroup')
assert (root/'cgroup.controllers').exists()
quota,period=(root/'cpu.max').read_text().split();assert int(quota)*2==int(period)
assert int((root/'memory.max').read_text())==64*1024*1024
assert int((root/'pids.max').read_text())==12
def stats():return dict(line.split() for line in (root/'cpu.stat').read_text().splitlines())
before=stats();end=time.monotonic()+2
while time.monotonic()<end:pass
after=stats();assert int(after['nr_throttled'])>int(before['nr_throttled'])
children=[]
try:
 while True:
  try:children.append(subprocess.Popen(['sleep','30']))
  except OSError as error:
   assert error.errno==errno.EAGAIN;break
 assert children and int((root/'pids.current').read_text())==12
 assert int(dict(line.split() for line in (root/'pids.events').read_text().splitlines())['max'])>0
 print(json.dumps({'cpu.max':[quota,period],'memory.max':(root/'memory.max').read_text().strip(),'pids.max':12,'children':len(children),'throttled_periods':int(after['nr_throttled'])-int(before['nr_throttled'])}),flush=True)
finally:
 for child in children:child.terminate()
 for child in children:child.wait()
'''
try:
    result=docker('run','--name',containers[0],'--network','none','--cpus','.5','--memory','64m','--memory-swap','64m','--pids-limit','12',BASE,'python','-c',limits_code)
    print(result.stdout.strip())
    oom=docker('run','--name',containers[1],'--network','none','--memory','64m','--memory-swap','64m',BASE,'python','-c','print("allocating",flush=True);data=bytearray(256*1024*1024)',check=False)
    state=inspect(containers[1])['State'];assert oom.returncode==137 and state['OOMKilled'] and state['ExitCode']==137
    docker('run','-d','--name',containers[2],'--network','none',BASE,'python','-c','import time;time.sleep(60)')
    docker('kill','--signal','KILL',containers[2]);docker('wait',containers[2])
    state=inspect(containers[2])['State'];assert state['ExitCode']==137 and not state['OOMKilled']
    print('a 64 MiB memory budget caused actual OOM termination; an explicit SIGKILL also returned 137 but OOMKilled was false')
finally:docker('rm','-f',*containers,check=False)

The first container reads cpu.max as 50000 100000, memory.max as 67,108,864 bytes, and pids.max as twelve. The CPU values represent fifty milliseconds of bandwidth per hundred-millisecond period. Setting Docker’s memory-plus-swap limit equal to its memory limit prevents this fixture from extending its budget through swap; the memory test remains deliberately small.

A two-second busy loop increases cpu.stat’s nr_throttled counter. The recorded increase was twenty periods, while the assertion requires only a positive increase. This confirms actual enforcement rather than merely checking that a Docker flag was accepted. It is not a benchmark proving an exact wall-clock slowdown for every workload or scheduler environment.

The process then starts sleeping children until task creation raises EAGAIN. In the recorded run, eleven children plus the controlling Python process reach the limit of twelve, and the pids controller’s rejection counter increases. The children are terminated and reaped. The refusal comes from the group’s task budget even though the children are mostly idle.

Another container receives a 64 MiB memory limit and attempts to initialize a 256 MiB bytearray. It exits with status 137, and Docker records OOMKilled as true. This validates an actual memory-limit failure for the tested allocation, rather than asserting that every possible allocation must produce the same exception or termination pattern.

A control container is explicitly sent SIGKILL. It also exits with status 137, but OOMKilled is false. That comparison is essential: the exit status alone cannot establish an OOM diagnosis. Correlate runtime state with memory events and system evidence when investigating production incidents, especially if several supervisors can terminate a workload.

Size and observe the whole workload

Budget for the application’s workers and relevant charged memory, not just a single process’s heap estimate. Page cache and other accounted memory can contribute to pressure. Leave room for startup, dependency changes, and recovery operations, then measure the workload under representative concurrency. A tight limit can change latency or failure rates before it produces an obvious crash.

Monitor current usage, pressure signals, and controller events alongside application behavior. Pressure indicators can reveal sustained contention, but a sudden allocation spike may outrun a periodic alert. Monitoring should explain failures and guide sizing without promising that every limit breach will be predicted in advance.

Change resource policy through the manager that owns the workload, such as Docker or systemd, and verify the effective controller values afterward. Container-local cgroup paths may reflect a namespace view rather than the host’s full hierarchy. Correct diagnosis connects that view, inherited constraints, and the specific enforcement mechanism that actually acted.

Sources