Durability · XB Field Notes

Redis append-only log trades speed for durability

Wed, 17 Nov 2027 00:00:00 +0000

An append-only file records Redis mutations for recovery, but choosing AOF is not the same as promising that no acknowledged write can ever disappear. Durability depends on the fsync policy, effective configuration, storage behavior, and which failure you mean. Start with a recovery objective rather than the reassuring word “log.”

Execution, logging, and acknowledgment

Redis executes commands against its in-memory dataset and then propagates their effects to AOF. This is not a rule that each command must reach disk before memory changes. With appendfsync always, Redis synchronizes appended data before sending replies; commands processed together may share a write and fsync. A lost connection still leaves the client uncertain about an operation whose reply never arrived.

The file is a recovery representation, not a permanent audit trail of every original request. Redis can transform commands for persistence, and rewriting retains the current dataset rather than its entire history. A deleted value is not guaranteed to remain recoverable from a rewritten log.

Choose the synchronization contract

appendfsync everysec schedules synchronization approximately once per second. Its usual tradeoff is roughly one second of exposed writes, not a universal loss bound under arbitrary storage stalls. appendfsync no leaves flushing to the operating system; it means Redis does not request fsync, not that data never reaches disk. Neither policy makes every successful reply evidence of durable storage.

appendfsync always offers a stronger local acknowledgment contract at a latency and throughput cost, but it cannot protect against losing the disk, a faulty storage stack, or deleting the wrong data. Also inspect no-appendfsync-on-rewrite: enabling it suspends normal fsync during background persistence work, weakening durability even when the main policy says always.

Rewriting creates a recoverable file set

Since Redis 7, AOF uses a manifest plus a base file and incremental files. With aof-use-rdb-preamble enabled, the base uses RDB format; later mutations live in incremental AOF files. That base belongs to AOF and is not the independently scheduled dump.rdb snapshot.

BGREWRITEAOF starts or schedules background work; its successful reply does not certify completion. Inspect INFO persistence for the completed rewrite and its status. Copying one conveniently named file, or racing a rewrite while copying the directory, can produce an unusable backup. Preserve a consistent manifest and its referenced active files.

When it breaks

Disk errors, exhausted space, and rewrite failures require monitoring, not merely an appendonly yes setting. Check aof_last_write_status and aof_last_bgrewrite_status alongside disk capacity, rewrite state, and application errors. Background work still consumes I/O and copy-on-write memory; “background” does not mean zero latency impact.

Truncated tails and corruption in the middle have different recovery semantics. Preserve the original artifact before inspection or repair: a repair can discard subsequent commands. AOF also replays application mistakes, so maintain independently retained backups. Do not turn a process-restart demonstration into a claim about power-loss safety, replication acknowledgment, or exactly-once retries.

Check it quickly

Use a POSIX machine with a C compiler, make, curl, tar, and Python 3.10 or newer. This exercise ran on macOS with Redis 8.10.1, redis-py 8.1.0, and Python 3.14.4. The setup builds Redis core with libc and installs the client in a temporary virtual environment.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It creates a private Unix-socket server, without TCP, scheduled snapshots, or automatic AOF rewrites. AOF uses always and keeps fsync enabled during rewriting. Only the subprocess it creates is killed. Persistence files survive that process restart inside the temporary directory, which is removed after the checks.

from contextlib import contextmanager
import os
from pathlib import Path
import signal
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance(directory, aof):
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--dir", directory, "--dbfilename", "dump.rdb", "--save", "",
 "--appendonly", "yes" if aof else "no", "--appendfsync", "always",
 "--appenddirname", "appendonlydir", "--appendfilename", "appendonly.aof",
 "--aof-use-rdb-preamble", "yes", "--no-appendfsync-on-rewrite", "no",
 "--auto-aof-rewrite-percentage", "0",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client, process
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode in (0, -signal.SIGKILL), errors.decode()


def crash(process):
 assert process.poll() is None
 process.kill()
 assert process.wait(timeout=5) == -signal.SIGKILL


with TemporaryDirectory(prefix="redis-aof-", dir="/tmp") as directory:
 with instance(directory, aof=True) as (client, process):
 client.set("counter", 0)
 for expected in range(1, 5):
 assert client.incr("counter") == expected
 assert client.hset("profile", mapping={"name": "Ada", "level": "2"}) == 2
 client.set("deleted", "not retained")
 assert client.delete("deleted") == 1
 rewrites = client.info("persistence")["aof_rewrites"]
 assert client.bgrewriteaof()
 deadline = time.monotonic() + 5
 while True:
 state = client.info("persistence")
 if (state["aof_rewrites"] > rewrites
 and not state["aof_rewrite_in_progress"]
 and not state["aof_rewrite_scheduled"]):
 assert state["aof_last_bgrewrite_status"] == "ok"
 break
 assert time.monotonic() < deadline, "AOF rewrite did not finish"
 time.sleep(0.02)
 assert client.incr("counter") == 5
 assert client.info("persistence")["aof_last_write_status"] == "ok"
 assert not (Path(directory) / "dump.rdb").exists()
 print("rewrite completed; a later INCR returned 5; no standalone RDB")
 crash(process)

 aof_dir = Path(directory) / "appendonlydir"
 entries = []
 for line in (aof_dir / "appendonly.aof.manifest").read_text().splitlines():
 parts = line.split()
 entries.append(dict(zip(parts[::2], parts[1::2], strict=True)))
 active = [entry for entry in entries if entry["type"] in {"b", "i"}]
 assert {entry["type"] for entry in active} == {"b", "i"}
 for entry in active:
 assert (aof_dir / entry["file"]).is_file()
 base = next(aof_dir / entry["file"] for entry in active if entry["type"] == "b")
 assert base.read_bytes().startswith(b"REDIS")
 print("manifest references an RDB-format AOF base and an incremental AOF")

 with instance(directory, aof=True) as (client, process):
 assert client.get("counter") == b"5"
 assert client.hgetall("profile") == {b"name": b"Ada", b"level": b"2"}
 assert client.get("deleted") is None
 assert client.dbsize() == 2
 print("after SIGKILL and restart: counter=5, hash restored, deletion retained")

The test waits for an observed successful rewrite, then adds one more increment. After SIGKILL, the manifest references an RDB-format AOF base and incremental data, while no standalone dump.rdb exists. Restarting restores counter five, both hash fields, and the absence of the deleted key. Checking values and types is stronger than counting keys alone.

The manifest inspection is scoped to this fixture’s simple filenames; it is not a general backup parser. SIGKILL skips graceful shutdown but leaves the operating system running, including its filesystem cache. The result proves this local AOF recovery path, not a measured loss window for everysec or no, nor survival of hardware failure.

Sources

Redis persistence, synchronization, multipart AOF, recovery, and backup boundaries, checked 2026-09-08.
Redis BGREWRITEAOF, scheduled work versus completed rewriting, checked 2026-09-08.
Redis INFO, persistence status fields, checked 2026-09-08.
Redis 8.10.1 configuration, fsync policies, rewrite overrides, and AOF file layout, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only build and allocator selection, checked 2026-09-08.

Checkpoints establish a recovery starting point

Sun, 17 Oct 2027 00:00:00 +0000

A checkpoint records progress that lets a database recover without treating its entire log history as outstanding work. The exact operation depends on the engine. PostgreSQL synchronizes the data changes required for a checkpoint and establishes a redo starting point. SQLite in WAL mode transfers eligible committed page frames from its WAL into the main database file. Neither operation is simply another name for transaction commit.

When it breaks

Checkpoint requests and complete checkpoints are different observations. SQLite readers can hold snapshots that prevent newer WAL frames from being copied into the database file or prevent the log from being reset. A passive checkpoint can return successfully after doing only the work currently possible. Reading only its success flag hides the remaining frame count and the reader that still prevents completion.

Log size also needs context. Reusing an existing file can preserve its allocated size while useful checkpoint progress advances. PostgreSQL can retain older WAL for archives or replication slots even after a checkpoint. Removing log files manually to make a graph smaller can destroy recovery material. Check the engine’s progress markers and retention obligations before equating filesystem size with unfinished recovery work.

An interval is not a hard recovery-time bound. Replay speed, storage throughput, workload shape, and the actual crash point affect recovery duration. More frequent checkpoints may reduce later replay while increasing normal write traffic. Moving WAL onto another device does not move all the data-page writes there or make the checkpoint’s cost disappear.

Check it quickly

Save the following as checkpoint_lab.py and run python3 checkpoint_lab.py with assertions enabled. It uses Python’s standard sqlite3 module and private temporary files. The recorded run used Python 3.14.4 with SQLite 3.53.0. Check your runtime’s SQLite version when comparing results, since the Python and database engine versions are distinct.

import sqlite3
import tempfile
from pathlib import Path

with tempfile.TemporaryDirectory(prefix="xbcatm-checkpoint-") as directory:
 path = Path(directory) / "test.db"
 writer = sqlite3.connect(path, isolation_level=None, timeout=0.1)
 reader = sqlite3.connect(path, isolation_level=None, timeout=0.1)
 try:
 assert writer.execute("PRAGMA journal_mode=WAL").fetchone() == ("wal",)
 writer.execute("PRAGMA synchronous=FULL")
 writer.execute("PRAGMA wal_autocheckpoint=0")
 writer.execute("CREATE TABLE marker (id INTEGER PRIMARY KEY, value INTEGER)")
 writer.execute("INSERT INTO marker VALUES (1,10)")
 assert writer.execute("PRAGMA wal_checkpoint(TRUNCATE)").fetchone() == (0, 0, 0)
 reader.execute("BEGIN")
 assert reader.execute("SELECT value FROM marker").fetchone() == (10,)
 writer.execute("UPDATE marker SET value=20 WHERE id=1")
 assert writer.execute("SELECT value FROM marker").fetchone() == (20,)
 passive = writer.execute("PRAGMA wal_checkpoint(PASSIVE)").fetchone()
 blocked = writer.execute("PRAGMA wal_checkpoint(TRUNCATE)").fetchone()
 print("passive", passive)
 print("truncate_with_reader", blocked)
 assert passive[0] == 0 and passive[1] > passive[2] >= 0
 assert blocked[0] == 1 and blocked[1] > blocked[2]
 assert Path(str(path) + "-wal").stat().st_size > 0
 assert reader.execute("SELECT value FROM marker").fetchone() == (10,)
 reader.execute("COMMIT")
 completed = writer.execute("PRAGMA wal_checkpoint(TRUNCATE)").fetchone()
 print("truncate_after_reader", completed)
 assert completed == (0, 0, 0)
 assert Path(str(path) + "-wal").stat().st_size == 0
 assert reader.execute("SELECT value FROM marker").fetchone() == (20,)
 print("fresh_reader", 20)
 finally:
 reader.close()
 writer.close()

The initial truncate checkpoint moves value ten into the database file and resets the WAL. A reader then opens an explicit transaction and reads ten. Another connection commits value twenty with automatic checkpointing disabled. The writer can read twenty immediately, while the old reader still needs its original snapshot. This schedule makes the constraint visible without relying on concurrent thread timing.

In the recorded run, passive checkpoint output is (0, 1, 0): no blocking failure is reported, one frame exists, and none is checkpointed. The assertions deliberately check the relationship between frame counts instead of requiring that exact count on every engine build. The meaningful result is that a successful passive call left work behind.

The truncate request next returns (1, 1, 0) after the short busy wait. Its first field signals that the requested completion was blocked. The WAL remains nonempty, and the old reader still sees ten. The Python call returning a tuple is therefore not evidence that truncation succeeded; the tuple’s status must be inspected.

After the reader commits, another truncate checkpoint returns (0, 0, 0). The script checks the actual WAL length is zero and a fresh read sees twenty. The update was already committed before this final checkpoint. Both connections close and the temporary directory is removed even if an assertion fails.

Diagnose the work that remains

For SQLite, inspect long-lived read transactions and unfinished cursors when checkpoints cannot finish. Decide whether passive progress is sufficient or a blocking checkpoint belongs in a controlled maintenance window. Increasing the busy timeout merely waits longer for the same constraint; it does not release an application transaction. Forcing truncation after every write can add contention and defeat the concurrency benefit of WAL mode.

For PostgreSQL, use checkpoint and WAL-retention statistics appropriate to the deployed version. A completed checkpoint can coexist with retained log files and ongoing transactions. Compare foreground latency and a measured restart rehearsal when changing checkpoint frequency. This fixture verifies SQLite’s completion boundary; it does not measure a production recovery-time objective or claim identical checkpoint APIs across database engines.

Sources

SQLite write-ahead logging, reader snapshots, checkpoint progress, and WAL reuse, checked 2026-09-12.
SQLite checkpoint API, passive, restart, and truncate completion behavior, checked 2026-09-12.
SQLite checkpoint pragma, returned busy flag and frame counts, checked 2026-09-12.
PostgreSQL 18 WAL and checkpoints, checkpoint costs and independent WAL retention requirements, checked 2026-09-12.

WAL makes commit durable before page flushing

Fri, 14 Aug 2026 00:00:00 +0000

Write-ahead logging lets a database make a transaction durable without immediately flushing every table and index page that transaction changed. In PostgreSQL, the log records needed to recover a page must be durable before that page is written to storage. Normal synchronous commit also waits for the commit record to be flushed. Buffered table changes and durable log records therefore advance through different paths.

When it breaks

WAL is not simply a promise to apply an SQL statement later. Execution already changes in-memory pages, and log records describe recovery actions. Some WAL records concern transactions that never commit. Recovery must combine redo with transaction status so an unfinished transaction does not become visible merely because some of its records or data pages reached disk.

Durability settings change the contract. With synchronous_commit=off, a successful reply can precede durable WAL, allowing recent acknowledged transactions to disappear after a crash. With fsync=off, the risks extend to inconsistent storage after a system failure. Full-page writes address another problem: reconstructing pages after partial page writes. None of these mechanisms compensates for storage that falsely acknowledges synchronization or for losing every durable copy.

A lost client reply creates a different uncertainty. The server can commit successfully before the connection fails. Recovery may preserve that transaction even though the client never saw success. Blindly retrying the business action can then duplicate it; use a stable operation identity and reconciliation where the application requires it.

Check it quickly

Run this script as a non-root Linux account with PostgreSQL 18 server and client programs in PATH. It was executed with PostgreSQL 18.6. It initializes an isolated temporary cluster, uses a private Unix socket, and enables normal durability settings explicitly. The immediate shutdown targets only the cluster created by the script. It intentionally exercises a database process interruption, so use the complete fixture rather than substituting a production data directory.

#!/bin/sh
set -eu
umask 077
lab=$(mktemp -d /tmp/xbcatm-crash.XXXXXX)
mkdir "$lab/socket"
cluster=$lab/data
writer_pid=''
cleanup() {
 if test -f "$cluster/postmaster.pid"; then
 pg_ctl -D "$cluster" -m fast -w stop >/dev/null
 fi
 if test -n "$writer_pid"; then wait "$writer_pid" || :; fi
}
trap cleanup EXIT
initdb -D "$cluster" --no-locale --encoding=UTF8 --auth-local=trust --auth-host=reject >/dev/null
cat >> "$cluster/postgresql.conf" <<CONF
listen_addresses = ''
unix_socket_directories = '$lab/socket'
port = 56432
fsync = on
synchronous_commit = on
full_page_writes = on
checkpoint_timeout = '1h'
CONF
pg_ctl -D "$cluster" -l "$lab/server.log" -w -t 15 start >/dev/null
export PGHOST=$lab/socket PGPORT=56432 PGDATABASE=postgres PGUSER=$(id -un)
export PGCONNECT_TIMEOUT=2
psql -X -v ON_ERROR_STOP=1 -qAt <<'SQL'
CREATE TABLE marker (id integer PRIMARY KEY,value text);
INSERT INTO marker VALUES (1,'base');
CHECKPOINT;
SQL
checkpoint_before=$(psql -X -v ON_ERROR_STOP=1 -qAt -c 'SELECT checkpoint_lsn FROM pg_control_checkpoint()')
psql -X -v ON_ERROR_STOP=1 -qAt -c "INSERT INTO marker VALUES (2,'committed')"
commit_end=$(psql -X -v ON_ERROR_STOP=1 -qAt -c 'SELECT pg_current_wal_lsn()')
test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT pg_current_wal_flush_lsn()>='$commit_end'::pg_lsn")" = t
PGAPPNAME=crash_uncommitted psql -X -v ON_ERROR_STOP=1 -qAt >"$lab/writer.log" 2>&1 <<'SQL' &
BEGIN;
UPDATE marker SET value='uncommitted' WHERE id=1;
SELECT pg_sleep(120);
COMMIT;
SQL
writer_pid=$!
attempt=0
while test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT EXISTS (SELECT FROM pg_stat_activity WHERE application_name='crash_uncommitted' AND wait_event='PgSleep')")" != t; do
 attempt=$((attempt+1))
 test "$attempt" -lt 100 || { echo 'uncommitted writer was not ready' >&2; exit 1; }
 sleep 0.1
done
test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT checkpoint_lsn='$checkpoint_before'::pg_lsn FROM pg_control_checkpoint()")" = t
echo committed_wal_flushed_without_new_checkpoint
pg_ctl -D "$cluster" -m immediate -w stop >/dev/null
if wait "$writer_pid"; then
 writer_pid=''
 echo 'interrupted writer unexpectedly succeeded' >&2
 exit 1
fi
writer_pid=''
pg_ctl -D "$cluster" -l "$lab/server.log" -w -t 15 start >/dev/null
grep -F 'redo starts at' "$lab/server.log"
psql -X -v ON_ERROR_STOP=1 -qAt <<'SQL'
SELECT id,value FROM marker ORDER BY id;
DO $$ BEGIN
 IF (SELECT string_agg(id||':'||value,',' ORDER BY id) FROM marker)
 IS DISTINCT FROM '1:base,2:committed' THEN
 RAISE EXCEPTION 'crash recovery result mismatch';
 END IF;
END $$;
SQL
printf 'lab_directory %s\n' "$lab"

The first row is checkpointed with value base. The script then commits a second row and verifies that the flushed WAL location covers the captured post-commit location. A separate backend changes the first row inside a transaction and enters a sleep before its commit. Observing that backend’s actual sleep state confirms that the unfinished update has executed.

Before interruption, the control-file checkpoint location must still match the baseline. Thus the acknowledged insert did not require another checkpoint. Immediate shutdown interrupts the sleeping backend without a normal shutdown checkpoint. The script requires that client’s operation to fail, restarts the same cluster, and checks the server log for redo starting.

Recovered output must be 1|base and 2|committed; a SQL assertion rejects missing, additional, or incorrectly valued rows. The committed transaction survives and the unfinished update is not visible. This does not prove which individual page was reconstructed by redo rather than already written by background activity. It verifies the final recovery contract across a real interrupted server with pending work.

The script stops its recovered cluster on exit and prints the directory containing the logs. It does not simulate a power cut, dishonest disk cache, lost volume, or remote replication failure. Those scenarios need their own recovery evidence.

Account for the complete write path

Appending WAL can avoid forcing scattered data pages at every commit, but it is not free. WAL still consumes bandwidth, synchronization latency, storage, and replication capacity. Full-page images can enlarge the stream. Dirty pages still need later writeback, and checkpoints establish recovery starting points rather than eliminating that work.

Measure commit latency, WAL generation, checkpoint behavior, and recovery duration together. A lower acknowledgment latency obtained by weakening synchronization is a changed durability promise, not an equivalent optimization. Keep local crash recovery separate from archived recovery and failover guarantees. The useful result is a documented relationship between acknowledged transactions, recoverable state, and the failure model actually rehearsed.

Sources

PostgreSQL 18 WAL introduction, write-ahead ordering and commit without immediate data-file flushing, checked 2026-09-11.
PostgreSQL 18 WAL configuration, synchronous commit, fsync, and full-page-write guarantees, checked 2026-09-10.
PostgreSQL 18 WAL and checkpoint tuning, writeback, checkpoints, and recovery work, checked 2026-09-12.