Transactions · XB Field Notes

Locks wait politely when a transaction grabs them

Wed, 16 Feb 2028 00:00:00 +0000

A lock wait can be healthy coordination or an expensive queue. PostgreSQL’s lock_timeout lets an operation stop waiting, but it does not diagnose why the holder is slow or decide whether retrying is appropriate. Choose the timeout from the operation’s latency budget, and inspect the blocking transaction before increasing the limit or multiplying attempts.

When it breaks

lock_timeout applies separately to each lock acquisition attempt, including implicit locks. It measures time waiting for that lock, not all time spent by the statement. A query already doing useful work can still hit the limit when it later requests another lock. Several shorter waits can also accumulate beyond one configured interval. Treating this as a whole-query deadline leaves that accumulation uncontrolled.

statement_timeout bounds statement execution, including lock waiting. If it is shorter, it can cancel the statement before the lock timeout fires. idle_in_transaction_session_timeout addresses a different problem: a session waiting for its next client command while keeping a transaction open. A backend blocked inside an update is ordinarily active, with a lock wait event; calling every blocked connection idle in transaction directs diagnosis at the wrong sessions.

Check it quickly

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, against the same disposable PostgreSQL 18 database. Start with the default lock_timeout of zero and no conflicting timeout settings. The short durations below make the demonstration fast; they are not production recommendations. A creates the fixture:

CREATE SCHEMA timeout_demo;
CREATE TABLE timeout_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO timeout_demo.counter VALUES (1,0);

A updates the row inside an open transaction and keeps its lock:

BEGIN;
UPDATE timeout_demo.counter SET n=10 WHERE id=1;

B waits at most one hundred milliseconds for its conflicting update. Temporarily allowing expected errors also exposes the transaction state before rollback:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='1s';
\set ON_ERROR_STOP off
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1;
\echo lock_state :SQLSTATE
SELECT n FROM timeout_demo.counter WHERE id=1;
\echo transaction_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

Expect lock_state 55P03, then transaction_state 25P02. The first error cancels this statement and leaves the explicit transaction failed. B’s rollback clears that failed state; it does not release A’s lock or undo A’s pending update. Repeating work inside B’s failed transaction would only generate another transaction-state error.

Leave A open. B now gives the whole statement a shorter budget than the lock acquisition:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='50ms';
\set ON_ERROR_STOP off
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1;
\echo statement_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

This returns statement_state 57014. The operation is still waiting for the same row, but the statement deadline wins. These error classes are evidence to interpret with context: 55P03 can also arise from NOWAIT, and 57014 also covers other query cancellations. Neither code by itself proves a deadlock or names the blocking application.

In A, discard the pending value ten and release the row:

ROLLBACK;

B starts a fresh, bounded transaction and succeeds:

BEGIN;
SET LOCAL lock_timeout='100ms';
SET LOCAL statement_timeout='1s';
UPDATE timeout_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

The returned value is one. Neither failed B attempt committed an increment, and A rolled back its ten. Next, B demonstrates that ordinary elapsed execution does not consume the lock-wait budget:

BEGIN;
SET LOCAL lock_timeout='50ms';
SET LOCAL statement_timeout='1s';
SELECT pg_sleep(0.2);
\echo sleep_state :SQLSTATE
COMMIT;
SHOW lock_timeout;

The two-hundred-millisecond sleep succeeds with sleep_state 00000 despite a fifty-millisecond lock timeout. After commit, SHOW returns the session’s original zero because SET LOCAL ended with its transaction. A verifies the counter remains one and cleans up:

SELECT n FROM timeout_demo.counter WHERE id=1;
DROP SCHEMA timeout_demo CASCADE;

Recover without amplifying contention

Select retry policy per operation. A lock timeout may justify a new transaction after rollback, provided the request still has time and replay is valid. A small attempt cap, jitter, and an overall deadline bound the extra load; they cannot make a permanently blocked resource available. Keep the original error and report exhausted attempts instead of silently dropping the request.

A true deadlock is a cycle of dependencies. PostgreSQL detects it and reports 40P01; a lock timeout can fire before that detection, so a timed-out run does not establish that no cycle existed. Inspect blocker identities, transaction age, and lock acquisition order. Shortening a transaction that waits for external input often removes the cause more effectively than raising every caller’s timeout. Do not copy these laboratory millisecond limits into an unrelated workload without measuring normal hold times.

Sources

PostgreSQL 18 client connection defaults, lock, statement, and idle transaction timeout scopes, checked 2026-09-10.
PostgreSQL 18 explicit locking, conflicting locks and deadlock detection, checked 2026-09-10.

A Redis transaction runs its commands as one unit

Wed, 31 Mar 2027 00:00:00 +0000

A Redis transaction is a connection-scoped command queue followed by an isolated execution window. MULTI starts the queue; EXEC runs accepted commands in order. The useful guarantee is that another client’s commands cannot run between those operations. It is not a promise to undo successful writes when a later command fails.

Admission and execution are different

A QUEUED reply means the server accepted a command for later execution, not that its intended write succeeded. Redis checks command existence and argument count while accepting the queue. Errors dependent on the current value, such as a list operation against a string, can instead appear during EXEC. Calling both categories syntax errors obscures their different outcomes.

A GET inside the queue also returns QUEUED immediately. Its actual result arrives in the EXEC response array. Application code cannot inspect that result and choose which later command to queue in the same transaction. Use WATCH for a decision made in the client, or a bounded server-side script for branching that must happen with the reads and writes.

Client-side buffering adds another stage. In redis-py, ordinary transactional pipeline methods collect commands locally; execute sends MULTI, the commands, and EXEC. Receiving a pipeline object from set is not a server acknowledgement. The exercise uses a reserved connection and raw command responses to expose the actual protocol stages.

Isolation starts at execution

MULTI does not reserve the keys against competing writers. While one client is still constructing its queue, another can update those keys. A queued GET observes the value when it executes, not a snapshot captured when MULTI arrived. A later queued GET also sees earlier writes from its own transaction.

Keep the entire transaction on one connection and do not share that connection with unrelated work. A pool that independently assigns connections to MULTI and EXEC would not preserve transaction state. Prefer the client’s transaction abstraction in application code; direct connection access here is an observation technique for the pinned client version.

When it breaks

A wrong argument count while queueing marks the transaction for rejection. EXEC then returns EXECABORT and none of the queued writes run. DISCARD explicitly removes an unexecuted queue. Neither action is an undo facility for commands already executed.

A runtime WRONGTYPE response behaves differently: the failing operation reports an error in its position, while earlier successful writes remain and later commands still run. Inspect every response before describing the business action as successful. A client exception does not establish that the server stopped at the offending command or reverted the batch.

With redis-py’s normal execute call, the client can raise a ResponseError after processing the transaction response. Disabling raise_on_error exposes execution errors as response elements, not as rollback. Transport failure is a separate case: losing the EXEC reply can leave the outcome unknown. Blindly retrying a non-idempotent transaction can duplicate effects.

Keep transactions short because their execution delays other clients. This isolation guarantee also does not select persistence, replication, or failover guarantees. A successful response is an execution result, not proof that an acknowledged write will survive every operational failure.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. The setup compiles Redis core with libc and installs the client in a temporary virtual environment; it does not install a system service.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It starts Redis on a private Unix socket under /tmp, with TCP and persistence disabled. Cleanup stops only that process and removes its runtime directory. Downloads need network access; no existing Redis instance is contacted.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


def command(connection, *arguments):
 connection.send_command(*arguments)
 return connection.read_response()


with instance() as client, client.client() as queued:
 connection = queued.connection
 client.set("value", "initial")
 assert command(connection, "MULTI") == b"OK"
 assert command(connection, "GET", "value") == b"QUEUED"
 assert command(connection, "SET", "value", "own-write") == b"QUEUED"
 assert command(connection, "GET", "value") == b"QUEUED"
 assert client.get("value") == b"initial"
 client.set("value", "other-client")
 assert command(connection, "EXEC") == [b"other-client", b"OK", b"own-write"]
 assert client.get("value") == b"own-write"
 print("QUEUED does not execute; EXEC reads current state and then its own writes")

 assert command(connection, "MULTI") == b"OK"
 assert command(connection, "SET", "discarded", "yes") == b"QUEUED"
 assert command(connection, "DISCARD") == b"OK"
 assert client.get("discarded") is None
 print("DISCARD removes the unexecuted queue")

 assert command(connection, "MULTI") == b"OK"
 assert command(connection, "SET", "before-arity", "yes") == b"QUEUED"
 try:
 command(connection, "SET", "missing-value")
 except redis.ResponseError as error:
 assert "wrong number of arguments" in str(error)
 else:
 raise AssertionError("invalid SET was accepted")
 assert command(connection, "SET", "after-arity", "yes") == b"QUEUED"
 try:
 command(connection, "EXEC")
 except redis.exceptions.ExecAbortError:
 pass
 else:
 raise AssertionError("queue error did not abort EXEC")
 assert client.mget("before-arity", "after-arity") == [None, None]
 print("wrong arity fails during queueing; EXECABORT applies none of the queue")

 client.set("wrong-type", "string")
 assert command(connection, "MULTI") == b"OK"
 assert command(connection, "SET", "before-runtime", "yes") == b"QUEUED"
 assert command(connection, "LPUSH", "wrong-type", "item") == b"QUEUED"
 assert command(connection, "SET", "after-runtime", "yes") == b"QUEUED"
 responses = command(connection, "EXEC")
 assert len(responses) == 3
 assert responses[0] == responses[2] == b"OK"
 assert isinstance(responses[1], redis.ResponseError)
 assert "WRONGTYPE" in str(responses[1])
 assert client.mget("before-runtime", "wrong-type", "after-runtime") == [
 b"yes", b"string", b"yes",
 ]
 print("runtime error occupies one reply; both surrounding writes remain applied")

The first assertion sequence queues GET, SET, and GET. A different connection changes the value before EXEC, so the replies contain other-client, OK, and own-write. Before EXEC, the queued SET has changed nothing. Together these observations distinguish deferred execution from a frozen snapshot and demonstrate reads of the transaction’s own writes.

The DISCARD case leaves its marker absent. The malformed SET then fails at queue admission, and EXECABORT leaves both surrounding markers absent even though their commands received QUEUED. Checking actual keys is important: an early error message alone would not prove the rest of the queue was discarded.

Finally, LPUSH targets an existing string. EXEC returns three elements: success, WRONGTYPE, success. Both surrounding markers remain written and the string remains unchanged. Four printed summaries follow assertions on responses and stored values; they are not assumed output.

This local exercise demonstrates the two error phases, explicit cancellation, and execution-time reads. It does not reproduce a lost connection, crash recovery, cluster routing, or durability guarantees. Those boundaries require separate tests before this mechanism is used for an irreversible business operation.

Sources

Redis transactions, queue admission, execution isolation, and error handling, checked 2026-09-08.
Redis MULTI, connection-scoped transaction entry, checked 2026-09-08.
Redis EXEC, ordered replies and watched-key aborts, checked 2026-09-08.
Redis DISCARD, discarding queued commands, checked 2026-09-08.
redis-py 8.1.0 client implementation, pipeline buffering and response processing, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.

A transaction can fail and leave you to retry

Mon, 01 Feb 2027 00:00:00 +0000

A competing update does not automatically make a PostgreSQL transaction fail. The outcome depends on isolation, statement behavior, and the state another transaction commits. Treating every collision as the same retryable event can conceal invalid input or repeat a decision made from stale data. The useful boundary is a complete application operation whose reads and writes can be attempted again.

When it breaks

Under Read Committed, an ordinary query gets a snapshot for that statement. An update can wait for another updater and then recheck its condition against the updated row. Two increments can therefore both succeed. An application that reads a number separately and later writes a previously calculated absolute value has a different contract: Read Committed does not automatically recompute that client calculation.

Repeatable Read retains a transaction snapshot, established by its first nontransaction-control statement. If a row changed after that snapshot, attempting to update it can raise 40001 immediately. Serializable can also reject dependency patterns involving different rows. Neither level promises that failures occur only at commit, nor does stronger isolation imply a universally earlier failure. Check statement execution and commit, using the driver’s structured error information.

Record the first SQLSTATE alongside the isolation level and operation identifier. A later 25P02 is a consequence of an earlier error, so logging only the final failed query can hide the actual reason the transaction stopped.

Check it quickly

Use PostgreSQL 18 and two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, connected to the same disposable database. Run blocks in the stated order; B uses autocommit unless a block explicitly begins a transaction. In A, create the isolated fixture:

CREATE SCHEMA conflict_demo;
CREATE TABLE conflict_demo.counter (id integer PRIMARY KEY,n integer);
INSERT INTO conflict_demo.counter VALUES (1,0);

A begins at Read Committed and reads zero:

BEGIN ISOLATION LEVEL READ COMMITTED;
SELECT n FROM conflict_demo.counter WHERE id=1;

B commits an increment and returns one:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

A now increments the current row and returns two, then commits successfully:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

This is a successful concurrent history. A’s earlier read did not freeze the value used by its later arithmetic update. In B, reset the counter after A commits:

UPDATE conflict_demo.counter SET n=0 WHERE id=1;

In A, begin Repeatable Read and establish the snapshot by reading zero:

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT n FROM conflict_demo.counter WHERE id=1;

B again commits an increment and returns one:

UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;

A attempts the same update. Temporarily allow the expected errors so the script can show both states and perform cleanup:

\set ON_ERROR_STOP off
UPDATE conflict_demo.counter SET n=n+1 WHERE id=1;
\echo update_state :SQLSTATE
SELECT n FROM conflict_demo.counter WHERE id=1;
\echo transaction_state :SQLSTATE
\set ON_ERROR_STOP on
ROLLBACK;

The update reports 40001; the subsequent read reports 25P02, meaning the transaction is already failed. No commit was attempted, so this directly disproves a commit-only error model. Restoring the client’s stop-on-error setting does not repair a failed server transaction; rollback does. In A, retry the complete operation after B has finished:

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT n FROM conflict_demo.counter WHERE id=1;
UPDATE conflict_demo.counter SET n=n+1 WHERE id=1 RETURNING n;
COMMIT;

The fresh read returns one and the update returns two. Finally, B verifies the committed result and removes the fixture:

SELECT n FROM conflict_demo.counter WHERE id=1;
DROP SCHEMA conflict_demo CASCADE;

The final two includes one committed increment from each participant. The failed attempt contributed nothing. If a result differs, check that both terminals selected the same database, used the intended isolation, and followed the reset and commit boundaries before interpreting it as a server anomaly.

Define the retry boundary

A 40001 response calls for a new transaction and new decisions, including reads used to choose keys or amounts. A 40P01 deadlock may also justify retrying the complete operation. Bound attempts and elapsed time; replay can be expensive and may still fail under sustained contention. Roll back before returning a failed connection to a pool, and preserve the final error when the budget expires.

Do not classify every 23505 uniqueness violation as transient. A duplicate permanent identifier remains invalid after waiting. Some application-generated keys depend on earlier reads, so a particular uniqueness failure can justify recomputation, but that decision needs operation-specific evidence. Likewise, losing the network acknowledgement of commit leaves an unknown outcome: it is different from receiving a server-declared serialization failure. Resolve that ambiguity with an operation identifier and durable result before replaying a payment or other externally visible effect.

Sources

PostgreSQL 18 transaction isolation, statement snapshots and concurrent update behavior, checked 2026-09-10.
PostgreSQL 18 serialization failure handling, complete transaction retries and error classification, checked 2026-09-10.

A bounded retry gives a transaction another chance

Mon, 09 Nov 2026 00:00:00 +0000

A retry can recover a transaction rejected because of concurrent work. It cannot guarantee success, and it does not automatically make external effects idempotent. The retry unit should include every database read and application decision that determined the writes. Reusing an amount calculated from the first attempt’s snapshot can defeat the purpose of starting a fresh transaction.

When it breaks

PostgreSQL identifies serialization failures with 40001; deadlocks use 40P01. Either can justify another complete attempt after rollback, within a finite budget. A duplicate identifier producing 23505 may be permanent. Retrying all exceptions indiscriminately wastes capacity and hides bad input. Some uniqueness failures depend on earlier key-selection reads, but their policy requires knowledge of that particular operation.

Errors can appear during statements or commit. A server-declared rejection differs from losing the connection while awaiting commit confirmation: the latter may leave a committed operation whose acknowledgement never reached the caller. Replaying it blindly can duplicate work. A payment or message sent outside the database transaction also survives database rollback; a retry loop alone cannot undo or deduplicate it.

Check it quickly

Use Python 3 on Linux or macOS and PostgreSQL 18’s psql in your executable path. Set PGDATABASE, PGHOST, and any required connection settings to a disposable database where you can create a schema. Arrange authentication beforehand; the example disables interactive password prompts. Save the following program as retry-demo.py and run python3 retry-demo.py. It creates and removes only its own fixture schema; an existing schema with that name makes setup fail.

import os
import random
import re
import selectors
import subprocess
import time

PSQL = ['psql', '-X', '-w', '-qAt', '-v', 'ON_ERROR_STOP=1', '-v', 'VERBOSITY=sqlstate']
ENV = dict(os.environ, PGCONNECT_TIMEOUT='2')


def run(sql):
 return subprocess.run(PSQL, input=sql, text=True, capture_output=True,
 check=True, timeout=5, env=ENV).stdout.strip()


def read_value(worker):
 deadline = time.monotonic() + 5
 data = bytearray()
 with selectors.DefaultSelector() as ready:
 ready.register(worker.stdout, selectors.EVENT_READ)
 while b'\n' not in data:
 remaining = deadline - time.monotonic()
 if remaining <= 0 or not ready.select(remaining):
 raise TimeoutError('Initial read was not confirmed')
 chunk = os.read(worker.stdout.fileno(), 4096)
 if not chunk:
 raise RuntimeError('Connection ended before initial read')
 data.extend(chunk)
 return int(data.strip())


def attempt(mode, number):
 worker = subprocess.Popen(PSQL, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
 stderr=subprocess.PIPE, text=True, env=ENV)
 try:
 worker.stdin.write("SET statement_timeout='1s';\n"
 'BEGIN ISOLATION LEVEL REPEATABLE READ;\n'
 'SELECT n FROM retry_demo.counter WHERE id=1;\n')
 worker.stdin.flush()
 observed = read_value(worker)
 # A real second connection commits after this attempt's snapshot.
 if mode == 'always' or (mode == 'once' and number == 1):
 run('UPDATE retry_demo.counter SET n=n+1 WHERE id=1;')
 statement = ('INSERT INTO retry_demo.counter VALUES (1,99);' if mode == 'permanent'
 else f'UPDATE retry_demo.counter SET n={observed + 1} WHERE id=1;')
 finish = r'''
\set ON_ERROR_STOP off
__STATEMENT__
\if :ERROR
 \echo ATTEMPT :SQLSTATE
 ROLLBACK;
\else
 COMMIT;
 \if :ERROR
 \echo ATTEMPT :SQLSTATE
 \else
 \echo ATTEMPT 00000
 \endif
\endif
'''.replace('__STATEMENT__', statement)
 output, _ = worker.communicate(finish, timeout=5)
 match = re.fullmatch(r'ATTEMPT ([0-9A-Z]{5})', output.strip())
 if worker.returncode != 0 or match is None:
 raise RuntimeError('No confirmed transaction outcome; do not replay blindly')
 return match[1], observed
 finally:
 if worker.poll() is None:
 worker.kill()
 worker.communicate()


def increment(mode):
 observations = []
 for number in range(1, 4):
 state, observed = attempt(mode, number)
 observations.append(observed)
 if state not in {'40001', '40P01'} or number == 3:
 return {'attempts': number, 'state': state, 'reads': observations}
 time.sleep(random.uniform(0, 0.01 * 2 ** (number - 1)))


run('CREATE SCHEMA retry_demo; '
 'CREATE TABLE retry_demo.counter (id integer PRIMARY KEY,n integer); '
 'INSERT INTO retry_demo.counter VALUES (1,0);')
try:
 cases = [
 ('once', {'attempts': 2, 'state': '00000', 'reads': [0, 1]}, 2),
 ('permanent', {'attempts': 1, 'state': '23505', 'reads': [0]}, 0),
 ('always', {'attempts': 3, 'state': '40001', 'reads': [0, 1, 2]}, 3),
 ]
 for mode, expected, expected_value in cases:
 run('UPDATE retry_demo.counter SET n=0 WHERE id=1;')
 result = increment(mode)
 value = int(run('SELECT n FROM retry_demo.counter WHERE id=1;'))
 assert result == expected and value == expected_value, (mode, result, value)
 print(mode, result, 'stored', value)
finally:
 run('DROP SCHEMA retry_demo CASCADE;')

The program uses two actual database connections. Each attempt starts Repeatable Read, reads the counter, and calculates a new absolute value. A competing connection can commit between that read and update, forcing the original snapshot’s update to fail. This deliberately exposes whether the retry reads again rather than merely resending the original calculated value.

For the once case, expect two attempts, final state 00000, observations zero then one, and stored value two. One increment belongs to the competing connection and one to the successful retry. The rejected first attempt contributes nothing. If the second attempt reused the old calculated value, it would not preserve that result.

For permanent, expect one attempt, state 23505, observation zero, and stored value zero. The existing primary key makes the requested insert invalid. The classifier stops immediately instead of spending the whole retry allowance on unchanged input. No successful application increment occurred.

For always, expect three attempts, state 40001, observations zero, one, and two, and stored value three. All three stored increments belong to the competing connection; the application never committed its operation. This is deliberate exhaustion, not eventual success. The assertions check both the control-flow report and durable data, because the database value alone would not identify which participant succeeded.

Keep error handling attached to the decision

The example asks psql for its structured SQLSTATE and emits a dedicated result marker. It does not search localized error-message wording. Expected statement errors are followed by rollback; commit errors are checked separately. Missing confirmation, process failure, and transport timeouts escape the retry classifier instead of being guessed into a concurrency category. The initial read and process completion have separate time bounds.

In an application, use the database driver’s transaction and structured error APIs rather than copying this process harness. Include connection acquisition, every attempt, and backoff in the request’s total deadline. Record attempts and the final error, and stop when the caller cancels. Jitter spreads retries but does not repair permanent contention or guarantee fairness.

Give externally visible operations stable identifiers and a durable way to discover their outcome. Reconcile an uncertain commit before reissuing a non-idempotent action. Keep irreversible effects outside a retryable callback unless their delivery protocol explicitly tolerates repetition. These boundaries are what make recovery safe; a small loop and a sleep only implement the scheduling part.

Sources

PostgreSQL 18 serialization failure handling, retry classifications and recomputing the whole transaction, checked 2026-09-10.
PostgreSQL 18 transaction isolation, snapshot conflicts used by the experiment, checked 2026-09-10.
PostgreSQL 18 psql, SQLSTATE, ERROR, conditional execution, and client error handling, checked 2026-09-10.

Choose transaction isolation to preserve the invariant

Sun, 23 Aug 2026 00:00:00 +0000

Choose an isolation level from the invariant an operation must preserve. A stable snapshot is useful, but two transactions can each make a locally reasonable decision that becomes invalid when both commit. PostgreSQL Repeatable Read prevents changes to a transaction’s reading snapshot, yet permits some serialization anomalies. Serializable adds detection that rejects combinations of concurrent work inconsistent with any serial execution, requiring the application to handle retries.

Separate a stable view from a valid decision

Consider two on-call doctors and a rule that at least one must remain available. Each doctor may leave only after observing more than one available doctor. Two Repeatable Read transactions can both observe two, then update different rows to leave. Neither overwrites the other’s row. Both can commit, leaving zero doctors even though each transaction used an internally stable snapshot.

This is write skew, not a lost update to one shared row. Row-level checks on each doctor’s boolean state do not express the global availability rule. Protecting only the row being changed also misses the relationship between the two decisions. A suitable serialization protocol must cover the invariant’s complete scope.

Database implementations matter. PostgreSQL maps Read Uncommitted to Read Committed, and its Repeatable Read prevents phantom reads even though the SQL standard allows that level to permit them. Those properties do not remove write skew. Isolation names should lead to the engine’s documented guarantees, rather than an assumption that every database implements them identically.

When it breaks

Treating Serializable as a mode that physically executes every transaction one at a time is misleading. PostgreSQL runs transactions concurrently and tracks dependencies; a transaction may fail during execution or commit with SQLSTATE 40001. The application must retry the complete decision using fresh reads, not resubmit a commit or reuse values computed from the rejected snapshot.

Serializable also cannot repair a transaction that would violate the business rule even when run alone. Every relevant write path must follow the intended contract. A separate unrestricted update can still remove the last doctor. Higher isolation is a concurrency guarantee, not an automatic implementation of missing application logic.

Check it quickly

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, in the same disposable PostgreSQL 18 database. Use the same role, normal autocommit outside explicit blocks, and an unused schema name. Follow the order exactly; the overlapping reads are what establish the competing decisions.

In A, create the initial roster.

CREATE SCHEMA isolation_demo;
CREATE TABLE isolation_demo.doctors (id integer PRIMARY KEY,on_call boolean NOT NULL);
INSERT INTO isolation_demo.doctors VALUES (1,true),(2,true);

In A, begin and read; then do the same in B. Both counts must be two.

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

Update doctor one in A, then doctor two in B. Each update also checks the availability rule within its own snapshot.

UPDATE isolation_demo.doctors SET on_call=false WHERE id=1
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

Commit A, then B. In A, inspect the broken invariant and reset the roster after both transactions finish.

COMMIT;

\set ON_ERROR_STOP off
COMMIT;
\echo commit_state :SQLSTATE
\set ON_ERROR_STOP on

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

UPDATE isolation_demo.doctors SET on_call=true;

Repeat the overlapping decisions with Serializable: begin A, begin B, update A, update B, commit A, then attempt B’s commit.

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=1
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;

COMMIT;

\set ON_ERROR_STOP off
COMMIT;
\echo commit_state :SQLSTATE
\set ON_ERROR_STOP on

In A, check availability. In B, retry the entire decision with a new transaction. Finish by checking and cleaning up in A.

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;
UPDATE isolation_demo.doctors SET on_call=false WHERE id=2
 AND (SELECT count(*) FROM isolation_demo.doctors WHERE on_call)>1 RETURNING id;
COMMIT;

SELECT count(*) AS available FROM isolation_demo.doctors WHERE on_call;
DROP SCHEMA isolation_demo CASCADE;

On PostgreSQL 18.6, the Repeatable Read commits succeeded and availability became zero. With the controlled Serializable ordering, A committed and B’s commit reported 40001; one doctor remained. Error stopping is disabled only around the commit being observed so psql can print its SQLSTATE. A different error is not the expected serialization result.

B’s fresh retry observed one available doctor. Its conditional update returned no row, and it committed without removing that last doctor. The final count stayed one. This verifies both anomaly prevention and recomputation after rejection; merely proving that an error occurred would not establish the recovery behavior.

Account for the real cost

Dependency tracking and repeated attempts consume resources, while explicit locking can introduce waiting and coordination overhead. Neither approach is universally faster. Measure the actual transaction mix, keep transactions short, and bound retries by attempts and the request deadline. Choose the mechanism that preserves the invariant under all participating paths, then verify its contention behavior with representative work.

Sources

PostgreSQL 18: Transaction isolation, snapshot isolation and Serializable dependency checks, checked 2026-09-10.
PostgreSQL 18: Serialization failure handling, SQLSTATE classification and full-transaction retry, checked 2026-09-10.

MVCC selects row versions through visibility rules

Sat, 08 Aug 2026 00:00:00 +0000

Multi-version concurrency control lets PostgreSQL keep different row versions and decide which ones a query may see. An ordinary reader can continue seeing an older committed version while another transaction writes a newer one. This reduces conflicts between reading and writing, but does not give every transaction one permanently frozen view. Snapshot lifetime depends on isolation, and a transaction can see its own earlier changes even when other transactions cannot.

Visibility follows the snapshot

Under Read Committed, an ordinary read normally gets a new snapshot for each statement. Under PostgreSQL Repeatable Read, the transaction reuses the snapshot established by its first non-transaction-control statement. Merely sending BEGIN does not establish the useful reading boundary. A transaction that begins earlier but first reads later can therefore see commits that occurred between those events.

Updating a row creates a new row version. Visibility rules determine whether an older snapshot reads the earlier version while a newer snapshot reads the update. The physical presence of a tuple is not sufficient to make it visible, and a SQL query does not simply return the newest bytes that happen to be stored on a page.

The transaction’s own changes are an important exception to a simplistic historical-picture metaphor. It can read records it inserted after its snapshot was established. Consequently, its visible data can combine an earlier view of other transactions with its own new work. That behavior is necessary for useful transactions, but it is not a photograph of one externally committed moment.

When it breaks

A plain read’s ordinary table lock does not conflict with normal row updates, yet locking has not disappeared. Two writers can contend for the same row. SELECT FOR UPDATE explicitly requests row locks, and an ACCESS EXCLUSIVE table lock conflicts with ordinary readers. MVCC does not promise that every command containing the word SELECT will proceed without waiting.

An old snapshot can also keep obsolete row versions needed by a reader. Vacuum cannot simply remove a version that remains visible to a relevant snapshot. Space pressure depends on update volume, retained versions, and cleanup progress, not just the number of open transactions. Starting another vacuum does not make an active snapshot forget its visibility requirements.

Check it quickly

Open two fresh psql -X -v ON_ERROR_STOP=1 sessions, A and B, in the same disposable PostgreSQL 18 database. Use the same role and normal autocommit outside explicit blocks. The schema name must be unused. Keep the following order so B commits its changes while A’s reading transaction remains open.

In B, create one committed original row.

CREATE SCHEMA mvcc_visibility_demo;
CREATE TABLE mvcc_visibility_demo.items (id integer PRIMARY KEY,note text);
INSERT INTO mvcc_visibility_demo.items VALUES (1,'old');

In A, establish a Repeatable Read snapshot by reading it.

BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

In B, update the original row, insert a second row, and inspect the committed result.

UPDATE mvcc_visibility_demo.items SET note='new' WHERE id=1;
INSERT INTO mvcc_visibility_demo.items VALUES (2,'other');
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

In A, read again, insert its own third row, read again, and commit.

SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;
INSERT INTO mvcc_visibility_demo.items VALUES (3,'mine');
SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;
COMMIT;

In A, read with a fresh statement after committing. Then clean up in B.

SELECT id,note FROM mvcc_visibility_demo.items ORDER BY id;

DROP SCHEMA mvcc_visibility_demo CASCADE;

On PostgreSQL 18.6, A’s first read returned (1,old). B completed without waiting for that ordinary reader and saw (1,new) plus (2,other). A still saw only (1,old) when it read again. After A’s own insertion, it saw (1,old) and (3,mine), while B’s second row remained outside A’s snapshot.

Once A committed, its next read returned all three current rows: (1,new), (2,other), and (3,mine). This sequence checks visibility of both another transaction’s writes and one’s own writes. A single reader with no concurrent update would not establish that distinction.

End the snapshot when its purpose ends

Choose a transaction snapshot for work that requires that reading contract, and finish it when the operation is complete. Investigate long-lived snapshots alongside write volume and vacuum progress rather than assuming version storage cleans itself immediately. Repeatable visibility also does not prevent every cross-row business anomaly; an operation may still require Serializable isolation, explicit coordination, or constraints. MVCC supplies row-version visibility, while the application’s complete concurrency contract determines whether its decisions remain correct.

Sources

PostgreSQL 18: MVCC introduction, row versions and reading concurrency, checked 2026-09-10.
PostgreSQL 18: Transaction isolation, snapshot lifetime and visibility of own writes, checked 2026-09-10.
PostgreSQL 18: Explicit locking, reader and writer lock conflicts, checked 2026-09-10.
PostgreSQL 18: Routine vacuuming, retaining visible versions and reclaiming obsolete rows, checked 2026-09-10.