A metric costs more than its short name suggests. Prometheus identifies a time series by its metric name and complete label set. A new observed label combination creates a different series, which must be ingested, indexed, retained, and queried. Cardinality budgeting asks which distinctions are worth paying for and whether their number stays bounded as the service grows.

Count combinations that actually exist

Label dimensions multiply the possible combinations, but the product of their distinct values is an upper bound unless every combination occurs. Two routes, two methods, and two status categories allow eight combinations. A service that only accepts one method on each route may emit fewer. Count observed combinations as well as theoretical limits, and include target labels such as instance when estimating what the server will ingest.

The value set matters more than the label’s friendly name. A normalized route such as /orders/{order_id} groups operations; a raw path containing each order identifier creates a growing dimension. Request IDs, timestamps, user identifiers, and arbitrary query strings can grow with event volume. A label named route is not bounded merely because the developer intended it to describe a route. Check the values produced on real traffic, including errors and uncommon paths.

Include the metric representation

A counter usually contributes one series for each observed label combination, before optional companion metrics. A classic histogram expands the same combination into its bucket, sum, and count series. With six finite bucket boundaries, one positive-infinity bucket, a sum, and a count, that is nine core series per combination. Some clients expose an additional creation-time series; include it in a real inventory rather than assuming this simplified count is complete.

Replicas add their own instance dimension. Scrape interval changes samples per series, while retention changes how long historical samples remain; neither is the same as the number of distinct series. Rapidly changing identifiers also create churn: yesterday’s short-lived combinations may remain in storage even after their producers disappear. Monitor active series, creation rate, ingestion volume, memory, and query behavior instead of using one fixed bytes-per-series estimate for every deployment.

When it breaks

An innocent-looking diagnostic label can turn an aggregate metric into a per-request event store. Removing that label from a dashboard query does not undo the cost already paid to ingest and retain it. Hashing each identifier still preserves its distinctness, and truncating a random identifier generally only postpones the problem while introducing collisions. Choose a deliberately bounded category when aggregation is the actual requirement.

Do not collapse away distinctions needed to detect customer impact. Keep useful bounded outcome or service dimensions, and move request-level investigation to appropriately controlled logs or traces. If unwanted labels are removed during collection, check for duplicate resulting series within a scrape and for lost diagnostic meaning. Prefer fixing the producer where possible, then verify the new label set and plan how historical series age out.

Check it quickly

Save this as cardinality.py and run Python 3. The synthetic events exercise every route, method, and status combination fifty times. One version retains only those bounded dimensions; another also includes a unique request ID. The calculation assumes twenty replicas each expose the same combinations as classic histograms with six finite boundaries.

from itertools import product

routes = ["/orders/{order_id}", "/products/{product_id}"]
methods = ["GET", "POST"]
statuses = ["200", "500"]
events = list(product(routes, methods, statuses, range(50)))
bounded = {(route, method, status) for route, method, status, _ in events}
with_request_id = {(route, method, status, f"request-{i}")
                   for i, (route, method, status, _) in enumerate(events)}

finite_buckets = 6
# Core classic-histogram series: finite buckets, +Inf, _sum, _count.
# Optional _created series are deliberately excluded from this calculation.
series_per_combination = finite_buckets + 3
replicas = 20
bounded_series = len(bounded) * series_per_combination * replicas
request_id_series = len(with_request_id) * series_per_combination * replicas

assert len(events) == 400 and len(bounded) == 8
assert len(with_request_id) == 400
assert bounded_series == 1440 and request_id_series == 72000
print(f"observed label sets: bounded={len(bounded)}, request_id={len(with_request_id)}")
print(f"core classic-histogram series: bounded={bounded_series}, request_id={request_id_series}")
print(f"multiplier={request_id_series // bounded_series}x")

Four hundred events collapse into eight bounded combinations. Adding request IDs keeps four hundred distinct combinations, giving 1,440 versus 72,000 core histogram series across the twenty replicas: a fiftyfold increase. Optional creation-time series and other metric families are excluded explicitly, so these are modeled series counts rather than a measurement of Prometheus memory or storage use.

The comparison demonstrates why aggregating events at instrumentation time changes the cost profile. Before approving another label, state the operational question it answers, its maximum expected value set, the combinations actually emitted, and the effect on each metric family. Then validate that estimate against the collector and server as the workload changes.

Sources