An active journal file and archived files with different removal rules.

The systemd journal needs an explicit retention contract

The systemd journal stores messages with structured metadata in journal files. That makes service output searchable without first inventing a parser for every application’s text format. Keeping those messages for a useful period is a separate requirement. Storage location, size policy, rotation, and forwarding determine which evidence remains available when an incident is investigated later. ...

October 26, 2027 · 4 min · XB Field Notes
Journal fields narrow two service invocations into an exact result set.

Query the journal with explicit field and priority rules

A useful journal query selects the records that answer a specific incident question. Unit identity, invocation, time, priority, and application fields provide different dimensions. Combining them correctly matters more than finding a familiar word somewhere in the machine’s history. journald can attach process metadata, but it does not automatically collect every file an application writes or infer the business meaning of each message. ...

July 30, 2027 · 5 min · XB Field Notes
JSON escaping protects string content, while quoted optional fields preserve valid records when a value is absent.

Structured nginx logs need escaping and a value-type contract

A log_format that resembles JSON is not necessarily a reliable structured event format. nginx substitutes variable values into the text, and those values can contain quotes, backslashes, control characters, or no data at all. Correct string escaping and correct value types are separate requirements. Testing emitted records with a real parser catches failures that a visually convincing configuration or successful nginx -t cannot establish. ...

July 15, 2027 · 6 min · XB Field Notes
Three connected blocks labeled Observe, Decide, and Act.

Structured logs make their query contract explicit

Structured logging assigns stable meaning to fields instead of asking each query to infer everything from message text. A record might contain an event name, request identifier, numeric duration, severity, and a human-readable message. Free text remains useful and can also be searched or parsed. The practical gain is that routine questions can use a documented schema whose meaning survives changes to explanatory wording. ...

March 9, 2027 · 5 min · XB Field Notes
One client response can summarize several upstream attempts, while original and final URI fields describe different stages.

An access log separates client outcomes from upstream attempts

An nginx access log records request information when logging runs at the end of processing. The selected fields can distinguish what the client requested, which URI ultimately handled it, what response was sent, and what happened during upstream attempts. These are different observations. A useful format keeps them separate instead of assuming that one status, duration, or path explains the whole request by itself. ...

March 1, 2027 · 6 min · XB Field Notes
One instant has several offset representations while a repeated local time has two meanings.

Timestamps need explicit time semantics

A wall-clock label such as 2026-11-01 01:30 does not identify one instant by itself. Its interpretation depends on a timezone and sometimes on which occurrence was intended during a clock transition. Conversely, an explicitly defined Unix timestamp already carries epoch and unit semantics without displaying a zone name. Decide which kind of value your system is storing before choosing a format. ...

November 29, 2026 · 4 min · XB Field Notes
Logrotate creates a new active file and a documented reopen hook moves the writer onto it.

Logrotate applies a rotation policy when it runs

Logrotate evaluates configuration when invoked, rotates eligible logs, and applies the selected retention and compression actions. A timer or another scheduler determines when that evaluation happens. The tool does not continuously cap file size, and renaming a log does not automatically redirect an application’s open descriptor. A working setup combines evaluation frequency, file policy, and the writer’s actual reopening behavior. ...

November 21, 2026 · 5 min · XB Field Notes
Three connected blocks labeled Observe, Decide, and Act.

Metrics and logs need an explicit population

Metrics and logs are different representations of observed behavior, not competing definitions of truth. A metric usually summarizes measurements under a set of dimensions. A log record describes an event and can carry detailed context. Both can answer aggregate questions if their coverage and semantics support the calculation. Neither becomes authoritative merely because its dashboard looks cleaner or its records contain more fields. ...

October 16, 2026 · 5 min · XB Field Notes