Consistency · XB Field Notes

Vector clocks order concurrent writes

Sat, 18 Sep 2027 00:00:00 +0000

Each node keeps a counter per participant. Comparing vectors reveals a causal edge or a true concurrent write, which decides whether to merge or to ask.

One timestamp is not enough

A single timestamp cannot order concurrent events, because a clock is not a cause, so two nodes that each wrote something at the same label may still disagree about which came first. Vector clocks solve that by giving every participant its own counter: a process bumps its own slot whenever it does something, and merges the sender’s vector into its own whenever it receives a message. The result is a vector that grows as events happen, and comparing two vectors tells you whether one event provably preceded the other, or whether neither did.

Compare vectors to see causality

To compare vectors, look element by element: if every value in one vector is less than or equal to the matching value in the other, the first event happened before the second; otherwise the two events are concurrent. Because a causal order accumulates everywhere the two paths crossed, a vector captures the transitive and direct edges that a single scalar time forfeits. The moment a message passes from A to B, B’s vector reflects both B’s own progress and everything A had seen, which is how the model traces causality instead of guessing it.

Concurrent means someone decides

Concurrency in the vector sense does not produce a winner; it only says two events are unrelated or tied, so someone else must decide the order when the application needs one. That decision is usually made by comparing a key or by ordering on the vector values themselves, and it must be deterministic so every replica agrees. The design is honest: vector clocks do not fabricate a total order among unrelated writes, they only expose that no such order exists; the final ordering decision belongs to the rule above them.

When it breaks

Vector clocks fail first by growing. One entry exists per participant, so a key written by hundreds of short-lived clients carries hundreds of counters on every read, and every write ships them back. Dynamo reported exactly this pressure: reconciliation metadata grows with the writer set, and Riak operators see the same symptom as oversized causal context that slows reads and multiplies siblings. Evicting the oldest entries feels like relief, but it deletes causality.

Once node C is pruned from [A:3, B:2, C:5], later comparisons read the missing slot as zero, so an event C had already observed looks brand new, and a write that should have stayed subsumed resurrects as a false sibling.

The second failure misreads concurrent vectors as ordered. Comparing only the sum, the maximum, or a wall-clock timestamp beside the vector elects a winner between [A:2, B:1] and [A:1, B:2], yet neither dominates the other element by element, so the honest answer is concurrent, not before or after. The discarded sibling is a lost update the clock had flagged correctly. The mirror mistake treats one equal slot as equal history: [A:2] seen twice is the same event replayed, but [A:2] against [A:2, B:1] is a strict before relation, and overwriting the longer vector with the shorter one rewinds history.

The guard is strict: return causal context on every write, merge with element-wise maximum, increment only your own slot, and prune only with a dominance-preserving scheme such as dotted version vectors. A write without its read vector forks a sibling every time.

Check it quickly

Save this as check_vector_clocks.py and run python3 check_vector_clocks.py. It stores vectors as plain dicts with missing entries read as zero, then asserts all three outcomes: happens-before with equality in both directions, plus a concurrent pair whose merge dominates both parents.

def compare(a, b):
 """Return 'equal', 'before', 'after', or 'concurrent'."""
 nodes = set(a) | set(b)
 le = all(a.get(n, 0) <= b.get(n, 0) for n in nodes)
 ge = all(a.get(n, 0) >= b.get(n, 0) for n in nodes)
 if le and ge:
 return "equal"
 if le:
 return "before" # a happens-before b
 if ge:
 return "after" # b happens-before a
 return "concurrent"


def merge(a, b):
 return {n: max(a.get(n, 0), b.get(n, 0)) for n in set(a) | set(b)}


def tick(clock, node):
 nxt = dict(clock)
 nxt[node] = nxt.get(node, 0) + 1
 return nxt


# 1. Happens-before: b extends a with one new event from B.
a = {"A": 1}
b = {"A": 1, "B": 1}
assert compare(a, b) == "before"
assert compare(b, a) == "after"

# 2. Equal: the same causal history seen twice.
assert compare({"A": 2}, {"A": 2}) == "equal"

# 3. Concurrent: each side saw something the other missed.
c = {"A": 2, "B": 1}
d = {"A": 1, "B": 2}
assert compare(c, d) == "concurrent"

# Merge of concurrent histories dominates both parents.
m = merge(c, d)
assert m == {"A": 2, "B": 2}
assert compare(c, m) == "before"
assert compare(d, m) == "before"

# A truncated vector lies: dropping B makes m look merely later.
truncated = {"A": 2}
assert compare(truncated, m) == "before" # missing B reads as 0, hiding what was pruned
print("PASS: before/after/equal/concurrent and merge all behave as expected")

Expect before and after on the extended vector, equal on identical histories, and concurrent on the split pair. The merge reads [A:2, B:2] and follows each parent, while truncated [A:2] still reports before, showing how pruning hides causality. The assertions passed with CPython 3.12 on macOS; the sketch omits pruning, dotted versions, and persistence, so treat it as a comparison demo, not a store.

Sources

Dynamo: Amazon’s highly available key-value store, reconciliation with version vectors and sibling retention, checked 2026-09-12.
Riak KV 2.2.3: Causal context (vector clocks and dotted version vectors), vector-clock growth, pruning hazards, and dotted version vectors, checked 2026-09-12.

Choose transaction isolation to preserve the invariant

Sun, 23 Aug 2026 00:00:00 +0000

Choose an isolation level from the invariant an operation must preserve. A stable snapshot is useful, but two transactions can each make a locally reasonable decision that becomes invalid when both commit. PostgreSQL Repeatable Read prevents changes to a transaction’s reading snapshot, yet permits some serialization anomalies. Serializable adds detection that rejects combinations of concurrent work inconsistent with any serial execution, requiring the application to handle retries.

Separate a stable view from a valid decision

Consider two on-call doctors and a rule that at least one must remain available. Each doctor may leave only after observing more than one available doctor. Two Repeatable Read transactions can both observe two, then update different rows to leave. Neither overwrites the other’s row. Both can commit, leaving zero doctors even though each transaction used an internally stable snapshot.

This is write skew, not a lost update to one shared row. Row-level checks on each doctor’s boolean state do not express the global availability rule. Protecting only the row being changed also misses the relationship between the two decisions. A suitable serialization protocol must cover the invariant’s complete scope.

Database implementations matter. PostgreSQL maps Read Uncommitted to Read Committed, and its Repeatable Read prevents phantom reads even though the SQL standard allows that level to permit them. Those properties do not remove write skew. Isolation names should lead to the engine’s documented guarantees, rather than an assumption that every database implements them identically.

When it breaks

Treating Serializable as a mode that physically executes every transaction one at a time is misleading. PostgreSQL runs transactions concurrently and tracks dependencies; a transaction may fail during execution or commit with SQLSTATE 40001. The application must retry the complete decision using fresh reads, not resubmit a commit or reuse values computed from the rejected snapshot.

Serializable also cannot repair a transaction that would violate the business rule even when run alone. Every relevant write path must follow the intended contract. A separate unrestricted update can still remove the last doctor. Higher isolation is a concurrency guarantee, not an automatic implementation of missing application logic.

Check it quickly

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, in the same disposable PostgreSQL 18 database. Use the same role, normal autocommit outside explicit blocks, and an unused schema name. Follow the order exactly; the overlapping reads are what establish the competing decisions.

In A, create the initial roster.

CREATE SCHEMA isolation_demo;
CREATE TABLE isolation_demo.doctors (id integer PRIMARY KEY,on_call boolean NOT NULL);
INSERT INTO isolation_demo.doctors VALUES (1,true),(2,true);

In A, begin and read; then do the same in B. Both counts must be two.

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

Update doctor one in A, then doctor two in B. Each update also checks the availability rule within its own snapshot.

UPDATE isolation_demo.doctors SET on_call=false WHERE id=1
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

Commit A, then B. In A, inspect the broken invariant and reset the roster after both transactions finish.

COMMIT;

\set ON_ERROR_STOP off
COMMIT;
\echo commit_state :SQLSTATE
\set ON_ERROR_STOP on

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

UPDATE isolation_demo.doctors SET on_call=true;

Repeat the overlapping decisions with Serializable: begin A, begin B, update A, update B, commit A, then attempt B’s commit.

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=1
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

COMMIT;

\set ON_ERROR_STOP off
COMMIT;
\echo commit_state :SQLSTATE
\set ON_ERROR_STOP on

In A, check availability. In B, retry the entire decision with a new transaction. Finish by checking and cleaning up in A.

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;
UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;
COMMIT;

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;
DROP SCHEMA isolation_demo CASCADE;

On PostgreSQL 18.6, the Repeatable Read commits succeeded and availability became zero. With the controlled Serializable ordering, A committed and B’s commit reported 40001; one doctor remained. Error stopping is disabled only around the commit being observed so psql can print its SQLSTATE. A different error is not the expected serialization result.

B’s fresh retry observed one available doctor. Its conditional update returned no row, and it committed without removing that last doctor. The final count stayed one. This verifies both anomaly prevention and recomputation after rejection; merely proving that an error occurred would not establish the recovery behavior.

Account for the real cost

Dependency tracking and repeated attempts consume resources, while explicit locking can introduce waiting and coordination overhead. Neither approach is universally faster. Measure the actual transaction mix, keep transactions short, and bound retries by attempts and the request deadline. Choose the mechanism that preserves the invariant under all participating paths, then verify its contention behavior with representative work.

Sources

PostgreSQL 18: Transaction isolation, snapshot isolation and Serializable dependency checks, checked 2026-09-10.
PostgreSQL 18: Serialization failure handling, SQLSTATE classification and full-transaction retry, checked 2026-09-10.

Consistency · XB Field Notes

Vector clocks order concurrent writes

One timestamp is not enough

Compare vectors to see causality

Concurrent means someone decides

When it breaks

Check it quickly

Further reading

Sources

Choose transaction isolation to preserve the invariant

Separate a stable view from a valid decision

When it breaks

Check it quickly

Account for the real cost

Sources