Locks · XB Field Notes

Locks wait politely when a transaction grabs them

Wed, 16 Feb 2028 00:00:00 +0000

A lock wait can be healthy coordination or an expensive queue. PostgreSQL’s lock_timeout lets an operation stop waiting, but it does not diagnose why the holder is slow or decide whether retrying is appropriate. Choose the timeout from the operation’s latency budget, and inspect the blocking transaction before increasing the limit or multiplying attempts.

When it breaks

lock_timeout applies separately to each lock acquisition attempt, including implicit locks. It measures time waiting for that lock, not all time spent by the statement. A query already doing useful work can still hit the limit when it later requests another lock. Several shorter waits can also accumulate beyond one configured interval. Treating this as a whole-query deadline leaves that accumulation uncontrolled.

statement_timeout bounds statement execution, including lock waiting. If it is shorter, it can cancel the statement before the lock timeout fires. idle_in_transaction_session_timeout addresses a different problem: a session waiting for its next client command while keeping a transaction open. A backend blocked inside an update is ordinarily active, with a lock wait event; calling every blocked connection idle in transaction directs diagnosis at the wrong sessions.

Check it quickly

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, against the same disposable PostgreSQL 18 database. Start with the default lock_timeout of zero and no conflicting timeout settings. The short durations below make the demonstration fast; they are not production recommendations. A creates the fixture:

CREATE SCHEMA timeout_demo;
CREATE TABLE timeout_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO timeout_demo.counter VALUES (1,0);

A updates the row inside an open transaction and keeps its lock:

BEGIN;
UPDATE timeout_demo.counter SET n=10 WHERE id=1;

B waits at most one hundred milliseconds for its conflicting update. Temporarily allowing expected errors also exposes the transaction state before rollback:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='1s';
\set ON_ERROR_STOP off
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1;
\echo lock_state :SQLSTATE
SELECT n FROM timeout_demo.counter WHERE id=1;
\echo transaction_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

Expect lock_state 55P03, then transaction_state 25P02. The first error cancels this statement and leaves the explicit transaction failed. B’s rollback clears that failed state; it does not release A’s lock or undo A’s pending update. Repeating work inside B’s failed transaction would only generate another transaction-state error.

Leave A open. B now gives the whole statement a shorter budget than the lock acquisition:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='50ms';
\set ON_ERROR_STOP off
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1;
\echo statement_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

This returns statement_state 57014. The operation is still waiting for the same row, but the statement deadline wins. These error classes are evidence to interpret with context: 55P03 can also arise from NOWAIT, and 57014 also covers other query cancellations. Neither code by itself proves a deadlock or names the blocking application.

In A, discard the pending value ten and release the row:

ROLLBACK;

B starts a fresh, bounded transaction and succeeds:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='1s';
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

The returned value is one. Neither failed B attempt committed an increment, and A rolled back its ten. Next, B demonstrates that ordinary elapsed execution does not consume the lock-wait budget:

BEGIN;
SET LOCAL lock_timeout='50ms';
SET LOCAL statement_timeout='1s';
SELECT pg_sleep(0.2);
\echo sleep_state :SQLSTATE
COMMIT;
SHOW lock_timeout;

The two-hundred-millisecond sleep succeeds with sleep_state 00000 despite a fifty-millisecond lock timeout. After commit, SHOW returns the session’s original zero because SET LOCAL ended with its transaction. A verifies the counter remains one and cleans up:

SELECT n FROM timeout_demo.counter WHERE id=1;
DROP SCHEMA timeout_demo CASCADE;

Recover without amplifying contention

Select retry policy per operation. A lock timeout may justify a new transaction after rollback, provided the request still has time and replay is valid. A small attempt cap, jitter, and an overall deadline bound the extra load; they cannot make a permanently blocked resource available. Keep the original error and report exhausted attempts instead of silently dropping the request.

A true deadlock is a cycle of dependencies. PostgreSQL detects it and reports 40P01; a lock timeout can fire before that detection, so a timed-out run does not establish that no cycle existed. Inspect blocker identities, transaction age, and lock acquisition order. Shortening a transaction that waits for external input often removes the cause more effectively than raising every caller’s timeout. Do not copy these laboratory millisecond limits into an unrelated workload without measuring normal hold times.

Sources

PostgreSQL 18 client connection defaults, lock, statement, and idle transaction timeout scopes, checked 2026-09-10.
PostgreSQL 18 explicit locking, conflicting locks and deadlock detection, checked 2026-09-10.

A fencing token guards against a stale leader

Tue, 30 Nov 2027 00:00:00 +0000

A distributed lock looks simple, but when a client stalls and wakes late its claim to that lock becomes stale; the fencing token turns that stale claim into a verifiable number.

A lock alone is not enough

A distributed lock protects a resource from two writers, but it does nothing about a writer that already lost the lock. Network latency can freeze one node mid-write, the lease expires, another leader takes over, and the old leader resumes from the stale frame it still remembers as its permission. This is the classic condition a lock cannot see: the writer no longer holds the token but still believes it does. The result is corruption that looks like two legitimate owners wrote different data, because from the resource’s view both held valid credentials at different instants. Depending only on a hopeful lease expiry leaves a window where the corruption is baked in before anyone notices.

Issue a number before you act

A fencing token closes the hole with an increasing number. The current leader obtains the token, a monotonically increasing sequence issued by the coordination service, before it writes the shared resource, and every write carries that number. Any request with an older number is refused, because the resource can see that a newer owner is already writing. The token must come from a trusted source, not from the writer’s own clock, because a wall-clock timestamp is forgeable by exactly the process these fences are meant to constrain. The only thing the resource can safely compare is a number it trusts the issuer produced.

Reject anything older

The rejection rule is the point, and it has to be unconditional. A resource that checks token > last_seen rejects the stale writer not because it is malicious but because its number is older, which is the only proof of freshness the resource can rely on. The design gets simpler when issued tokens share one monotonically increasing sequence across all owners, so a single comparison rejects every superseded leader at once. The same principle extends past files: lease fences, epoch numbers and fencing tokens in distributed systems all exist to trade a fragile trust in clocks for a comparison of numbers.

When it breaks

A fence fails when its number can move backward. If the issuer keeps the counter in memory, a crash restarts the sequence at zero: the paused leader still holds 48 while the replacement receives 2, and the resource mistakes the stale write for the newest one. Persist the counter or derive it from a log index, and refuse to issue after an unclean restart until ownership is reconciled.

Timestamp tokens break the same promise. An NTP step backward, a container migration across skewed hosts, or two leaders stamping the same millisecond can produce equal or decreasing values. The resource then has no ordering to enforce, so equal tokens must be rejected as replays and wall-clock time must never substitute for an issuer sequence.

Split brain removes the single comparison point. Each partition elects a leader and mints its own token 7, each replica accepts its local write, and the merged histories both claim to be fenced. Without one issuer and one checking resource, old and new are just labels.

The quietest break is a lost token. A client upgrade drops the field, a proxy strips the header, or a compatibility shim treats a missing value as zero and skips the check, letting a stale write through. The mirror is a resource that forgets last_seen on restart and re-accepts 33 after serving 34. Send the token on every mutation, fail closed when it is absent, and persist the high-water mark with the data.

Check it quickly

Save this as fencing.py and run python3 fencing.py. It uses only the Python standard library and simulates one issuer plus one checking resource, so no server or clock configuration is needed:

class StaleFenceError(RuntimeError):
 pass


class FencingResource:
 """A resource that only accepts strictly increasing fence tokens."""

 def __init__(self):
 self.last_seen = 0
 self.data = None

 def write(self, token, value):
 if token is None:
 raise StaleFenceError("missing fence token")
 if token <= self.last_seen:
 raise StaleFenceError(f"stale token {token} <= {self.last_seen}")
 self.last_seen = token
 self.data = value
 return True


class Issuer:
 """A monotonic fence issuer; in production this counter must be durable."""

 def __init__(self):
 self.counter = 0

 def acquire(self):
 self.counter += 1
 return self.counter


issuer = Issuer()
resource = FencingResource()

old_leader = issuer.acquire() # 1: lease holder before the pause
new_leader = issuer.acquire() # 2: replacement after the lease expires
assert (old_leader, new_leader) == (1, 2)

assert resource.write(new_leader, "from-new-leader") is True
assert resource.data == "from-new-leader"

try:
 resource.write(old_leader, "from-stale-leader")
except StaleFenceError as error:
 print(f"rejected stale write: {error}")
else:
 raise AssertionError("stale token was accepted")

try:
 resource.write(new_leader, "replay")
except StaleFenceError as error:
 print(f"rejected replay: {error}")
else:
 raise AssertionError("equal token was accepted")

try:
 resource.write(None, "no-token")
except StaleFenceError as error:
 print(f"rejected missing token: {error}")
else:
 raise AssertionError("missing token was accepted")

next_token = issuer.acquire() # 3
assert resource.write(next_token, "from-new-leader-again") is True
assert (resource.data, resource.last_seen) == ("from-new-leader-again", 3)
print("PASS: stale, replayed, and missing tokens rejected; newer token accepted")

The script replays the classic pause: the old leader keeps token 1 while the replacement advances to token 2 and writes first. The stale write, an exact replay of token 2, and a missing token are each rejected, while token 3 succeeds. Expect this output:

rejected stale write: stale token 1 <= 2
rejected replay: stale token 2 <= 2
rejected missing token: missing fence token
PASS: stale, replayed, and missing tokens rejected; newer token accepted

Sources

Redis distributed lock fencing tokens, explains why a lock needs a fencing token against stale holders, checked 2026-06-01.
How to do distributed locking (Martin Kleppmann), fencing tokens with monotonically increasing numbers and why Redlock lacks them, checked 2026-06-01.

Postgres advisory locks are named, app-level locks

Wed, 24 Nov 2027 00:00:00 +0000

An advisory lock lets an application coordinate work using an identifier that need not correspond to a table row. For example, workers can agree that only one connection may refresh a particular report at a time. PostgreSQL manages ownership and contention, while the application defines what the identifier means and which work requires it.

When it breaks

The key is either one bigint or two integer values; there is no advisory-lock overload accepting a text name. Those two key spaces are distinct. If an application hashes names, it must define stable encoding and handle possible collisions; different computations can also give the same resource different keys. A documented integer namespace is often easier to reason about than an undocumented hash convention.

The lock is cooperative. A client that ignores it can still update the supposed protected rows. It does not replace a uniqueness constraint, transaction, or authorization check. Session locks survive rollback and require matching unlocks, or session termination. Transaction locks end with their transaction. In autocommit, a transaction lock obtained by one statement has already been released before a later statement starts, leaving that later work unprotected.

Check it quickly

Use two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, connected to one disposable PostgreSQL 18 database. Reserve the example key pairs for this exercise and execute blocks in order. A creates a counter:

CREATE SCHEMA advisory_demo;
CREATE TABLE advisory_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO advisory_demo.counter VALUES (1,0);

A tests an explicitly typed text key. The expected SQLSTATE is 42883, indicating no matching function:

\set ON_ERROR_STOP off
SELECT pg_advisory_lock('job-name'::text);
\echo text_key_state :SQLSTATE
\set ON_ERROR_STOP on

In A, acquire the same session lock twice inside a transaction, then roll back:

BEGIN;
SELECT pg_advisory_lock(8610,2);
SELECT pg_advisory_lock(8610,2);
ROLLBACK;

B cannot acquire that lock, but can update the counter without participating in the advisory protocol:

SELECT pg_try_advisory_lock(8610,2);
UPDATE advisory_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

Expect false, followed by one. Both observations matter: rollback did not release A’s session lock, and the advisory lock did not prevent B’s ordinary write. In A, release one acquisition:

SELECT pg_advisory_unlock(8610,2);

The unlock returns true, but B still cannot acquire the lock:

SELECT pg_try_advisory_lock(8610,2);

A must release its second acquisition too:

SELECT pg_advisory_unlock(8610,2);

B can now obtain the session lock, and explicitly releases its own acquisition:

SELECT pg_try_advisory_lock(8610,2);
SELECT pg_advisory_unlock(8610,2);

Both calls return true. Successful repeated acquisitions are counted, so a single successful unlock does not prove that a session no longer owns the resource. In A, begin a transaction and acquire a different key using the transaction-scoped function:

BEGIN;
SELECT pg_advisory_xact_lock(8610,3);

B’s nonblocking attempt returns false while A’s transaction remains open:

SELECT pg_try_advisory_xact_lock(8610,3);

A rolls back:

ROLLBACK;

B can now obtain the transaction lock in autocommit:

SELECT pg_try_advisory_xact_lock(8610,3);

Immediately afterward, A can obtain it too:

SELECT pg_try_advisory_xact_lock(8610,3);

These two true results are sequential, not simultaneous ownership. Each statement’s implicit transaction ended and released the lock before the next statement began. Neither connection now protects a future critical section. Finally, A checks that the counter is one and removes the fixture:

SELECT n FROM advisory_demo.counter WHERE id=1;
DROP SCHEMA advisory_demo CASCADE;

An unsuccessful try-lock call adds no ownership to release. Only a successful acquisition contributes to the locks held by that session or transaction.

Match ownership to the operation

For database work that fits one transaction, acquire the transaction lock inside an explicit transaction, then read, decide, and write before ending it. A nonblocking acquisition returning false means the client did not obtain ownership; it must skip or reschedule the protected work, not continue as though it succeeded. Blocking variants can wait, so their timeout and retry policies belong to the operation budget.

Use a session lock only when its longer lifetime is required, and keep acquisition, protected work, and release on the same physical connection. Returning that connection to a pool can leave a lock owned by a backend that serves unrelated callers; transaction pooling is especially unsuitable for session ownership assumptions. A terminated backend releases session locks automatically, although detecting a lost network connection can take time. This mechanism also supplies no durable record that a job completed: a subsequent worker must inspect committed state before deciding whether repeating the work is appropriate.

Sources

PostgreSQL 18 advisory lock functions, accepted key types and blocking or nonblocking functions, checked 2026-09-10.
PostgreSQL 18 explicit locking, advisory ownership, repeated acquisition, and transaction semantics, checked 2026-09-10.

Deadlocks happen when two writers wait on each other

Tue, 11 Aug 2026 00:00:00 +0000

A deadlock requires a cycle of incompatible waits. If one transaction holds a row while another waits, the holder can still finish and release it. If both hold a resource the other needs, neither can complete that path. PostgreSQL detects the cycle and aborts one transaction, allowing the other to continue. The application must distinguish that rejection from an ordinary slow wait.

When it breaks

Row updates acquire locks even without explicit locking statements, so two innocent-looking updates in opposite orders can deadlock. More participants, foreign-key checks, triggers, advisory locks, or lock upgrades can make the cycle less obvious. Sorting just one list of row identifiers is not proof that every resource in a larger operation follows one order.

PostgreSQL reports deadlock detection as 40P01. Applications must not rely on the victim being the youngest transaction or the one that did least work. A statement or lock timeout may fire before deadlock detection, producing a different error. Inspect the actual SQLSTATE and server diagnostics instead of classifying every pause as a deadlock or every retry as a successful repair.

Check it quickly

Use two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, on the same disposable PostgreSQL 18 database. A creates two rows and shows the detector interval:

CREATE SCHEMA deadlock_demo;
CREATE TABLE deadlock_demo.items (id integer PRIMARY KEY,n integer);
INSERT INTO deadlock_demo.items VALUES (1,0),(2,0);
SHOW deadlock_timeout;

The tested interval was one second. The statements below have a ten-second budget, leaving room for detection; ensure no shorter lock timeout interferes. If your detector interval is longer, adjust the demonstration’s statement budget accordingly. A records its increment and updates row one, without committing:

\set delta 10
SET statement_timeout='10s';
BEGIN;
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=1;

B records a different increment and updates row two:

\set delta 100
SET statement_timeout='10s';
BEGIN;
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=2;

In A, submit the next block. It waits for B’s row. Do not wait for this block to finish before switching terminals:

\set ON_ERROR_STOP off
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=2;
\echo second_result :SQLSTATE
\set ON_ERROR_STOP on

While A is waiting, submit B’s second update:

\set ON_ERROR_STOP off
UPDATE deadlock_demo.items SET n=n+:delta WHERE id=1;
\echo second_result :SQLSTATE
\set ON_ERROR_STOP on

The cycle now exists. One terminal prints second_result 40P01, while the other eventually prints second_result 00000. Either terminal can lose. Temporarily allowing expected errors makes both outcomes visible; it does not change which transaction the server rejects. In the losing terminal, clear the failed transaction:

ROLLBACK;

In the successful terminal, commit its complete operation:

COMMIT;

In either terminal, inspect both rows:

SELECT id,n FROM deadlock_demo.items ORDER BY id;

If A won, both values are ten; if B won, both are one hundred. Mixed values would contradict the expected atomic result. The losing transaction’s first update was rolled back along with its second attempt, so counting its first statement as completed work would overstate what was committed.

In the losing terminal, retry the whole operation. Its delta variable still identifies the original increment. This time, explicitly lock both rows in identifier order before updating:

BEGIN;
SELECT id FROM deadlock_demo.items ORDER BY id FOR UPDATE;
UPDATE deadlock_demo.items SET n=n+:delta;
COMMIT;

The locking read returns identifiers one and two. With the winner already committed, the retry completes. In either terminal, verify both increments are present exactly once and remove the fixture:

SELECT id,n FROM deadlock_demo.items ORDER BY id;
DO $$ BEGIN
 IF (SELECT array_agg(n ORDER BY id) FROM deadlock_demo.items) IS DISTINCT FROM ARRAY[110,110] THEN
 RAISE EXCEPTION 'deadlock recovery lost or duplicated work';
 END IF;
END $$;
DROP SCHEMA deadlock_demo CASCADE;

Both values must be one hundred ten. The assertion detects lost or duplicated work, not merely absence of an error message. Keep the two original sessions throughout; opening a third terminal for the retry would lose the client variable identifying which operation needs recovery.

Prevent recurrence across the complete path

Consistent ordering removes the particular two-row cycle shown here when all participants follow it and identifiers remain stable. Apply the ordering to every relevant resource and request the needed lock mode deliberately. Audit trigger and constraint paths too; a local ordering convention cannot exclude cycles introduced elsewhere.

Keep transactions short and avoid holding database locks while waiting for a person or remote service. Retry server-declared deadlocks only after rollback, repeating the reads and decisions as well as writes. Use a bounded budget and jitter under contention. Recurring deadlocks deserve investigation of acquisition order and workload, because repeated aborts still consume time and do not guarantee eventual completion.

Sources

PostgreSQL 18 explicit locking, deadlock cycles, unpredictable victims, and lock ordering, checked 2026-09-10.
PostgreSQL 18 serialization failure handling, full transaction retries after concurrency failures, checked 2026-09-10.

Locks · XB Field Notes

Locks wait politely when a transaction grabs them

When it breaks

Check it quickly

Recover without amplifying contention

Sources

A fencing token guards against a stale leader

A lock alone is not enough

Issue a number before you act

Reject anything older

When it breaks

Check it quickly

Further reading

Sources

Postgres advisory locks are named, app-level locks

When it breaks

Check it quickly

Match ownership to the operation

Sources

Deadlocks happen when two writers wait on each other

When it breaks

Check it quickly

Prevent recurrence across the complete path

Sources