Latency, traffic, errors, and saturation are four complementary ways to ask whether a service is serving users well. They are a starting set of questions, not four universal counters that prove health by themselves. Their usefulness depends on where they are measured, which work they describe, and how their time windows align. A fast response can still be a failed operation.

Separate experience from demand

Latency measures how long work takes at a chosen boundary. Track successful and failed operations separately so that immediate failures do not make the overall distribution appear healthier. Include useful tail information, but also preserve the observation count: a p99 calculated from a tiny population does not establish a stable view of rare behavior. Client latency and server processing time describe different portions of the experience.

Traffic describes demand or completed work in an appropriate unit, such as requests per second or bytes per second. Define whether the measurement counts arrivals, completions, or retries. A completion rate can fall while demand continues rising if a queue is stuck. Concurrent requests are a count, not a request rate; use them as additional context rather than swapping their units into an otherwise identical traffic panel.

Explain failure and resource pressure

Errors measure work that fails the promised outcome. HTTP status failures are one signal, but a nominally successful response containing incorrect data can also violate the contract. Decide which outcomes and latency violations belong in the indicator, and use a denominator with the same scope. Display counts as well as ratios so that one error in a quiet period is not mistaken for thousands of affected requests.

Saturation measures how close a constrained resource is to its usable limit. CPU, memory, connection pools, worker occupancy, and queue age may reveal different bottlenecks. A thread pool with every worker busy is not automatically unhealthy, but a growing queue and aging work show why that occupancy matters. A snapshot is not an interval average, and a low average can conceal brief overload. Capacity thresholds need workload context and measurement, not a decorative red line.

When it breaks

Mixing populations can create an impossible-looking story: latency from successful application requests, errors from an edge proxy, and traffic from another region may change independently. Name boundaries and filters explicitly before correlating them. Fast rejections can improve an all-request latency average while errors rise. Measuring only completions can also omit stalled requests, making both latency and throughput miss the work users are still waiting for.

None of the four signals proves the root cause of a deviation. CPU can be quiet while a database connection pool is exhausted, and queue growth can follow either more demand or slower downstream work. Use the signals to choose a next investigation, then inspect traces, logs, dependencies, and change evidence. Alert on actionable user impact or impending exhaustion; a momentary deviation in any one gauge need not wake a human.

Check it quickly

Save the following as golden.py and run Python 3. It uses one synthetic ten-second interval of completed requests and a separately labeled end-of-interval worker snapshot. The fixture verifies aggregation and interpretation of the four signals. It neither generates load nor measures a live server, and its exact nearest-rank percentile is not an estimate from histogram buckets.

from math import ceil
from statistics import mean

# Synthetic completed requests in one 10-second observation interval.
records = [(True, 0.05)] * 90 + [(True, 0.8)] * 5 + [(False, 0.005)] * 5
success = sorted(duration for ok, duration in records if ok)
failure = [duration for ok, duration in records if not ok]
p99_success = success[ceil(0.99 * len(success)) - 1]
traffic_rps = len(records) / 10
error_ratio = len(failure) / len(records)
# These are end-of-interval gauges, not interval averages.
busy_workers, total_workers = 4, 4
queued_requests, oldest_queue_seconds = 12, 1.2

assert p99_success == 0.8 and traffic_rps == 10 and error_ratio == 0.05
assert mean(failure) < mean(success)
assert busy_workers / total_workers == 1
print(f"traffic={traffic_rps:.1f} completed requests/s; errors={error_ratio:.1%}")
print(f"success p99={p99_success:.3f}s; failed mean={mean(failure):.3f}s")
print(f"success mean={mean(success):.6f}s; all-request mean={mean(d for _, d in records):.6f}s")
print(f"worker snapshot={busy_workers}/{total_workers}; queue={queued_requests}; oldest={oldest_queue_seconds}s")

The output shows ten completed requests per second, a 5% error ratio, successful-request p99 of 0.800 seconds, and failed-request mean of 0.005 seconds. The successful mean is about 0.089474 seconds, while including the fast errors lowers the all-request mean to 0.085250 seconds. A lower combined mean therefore does not imply more useful service.

All four workers are busy in the snapshot, with twelve queued requests and an oldest wait of 1.2 seconds. Those gauges provide pressure context, not proof that occupancy stayed at 100% throughout the interval. Read them with the latency, demand definition, and subsequent queue trend before deciding whether to scale, shed load, or investigate a slower dependency.

Further reading

Sources