Continuous WAL archiving preserves completed log files outside PostgreSQL’s active WAL directory. Together with a suitable base backup and an uninterrupted recovery chain, those files let recovery replay logged database changes to a covered target. Archiving is separate from ordinary commit acknowledgment: a transaction can commit while its WAL segment is still open or while the archive destination is unavailable.

When it breaks

PostgreSQL modifies buffered data pages during execution. Before a dependent page reaches durable storage, the necessary WAL must be durable first. This ordering does not mean that data files are never written, or that every commit waits for archiving. Temporary and unlogged table contents also do not gain ordinary WAL recovery guarantees just because archiving is enabled.

The archive command’s return code is a data-integrity contract. Zero means the file has been preserved successfully; PostgreSQL can eventually recycle the source when other retention requirements permit. A command that reports success after skipping a failed copy silently breaks that contract. Nonzero results cause retries, and repeated failure retains unarchived WAL, potentially filling the primary’s filesystem. A checkpoint does not repair the destination or discard that obligation.

Existing names need careful treatment. Repeated archival of identical, fully persisted content should succeed. Different bytes under the same name must fail without overwriting the original. Returning success merely because a destination exists can conceal corruption or archives accidentally mixed from different clusters.

Check it quickly

Run the following script as a non-root account with PostgreSQL 18 server and client binaries in PATH. It requires a Linux shell environment supporting cp -a, mktemp, hard links, and sync FILE. The drill was executed on PostgreSQL 18.6. It creates its own private temporary cluster and Unix socket, disables TCP listening, and uses only test data. The local archive illustrates the command contract; it is not an independent disaster-recovery copy.

#!/bin/sh
set -eu
umask 077
lab=$(mktemp -d /tmp/xbcatm-archive.XXXXXX)
mkdir "$lab/socket" "$lab/archive"
primary=$lab/primary
cleanup() {
    if test -f "$primary/postmaster.pid"; then
        pg_ctl -D "$primary" -m fast -w stop >/dev/null
    fi
}
trap cleanup EXIT
initdb -D "$primary" --no-locale --encoding=UTF8 --auth-local=trust --auth-host=reject >/dev/null
cat > "$lab/archive-copy.sh" <<'ARCHIVE_HELPER'
#!/bin/sh
set -eu
wal_source=$1
wal_name=$2
archive_directory=$3
archive_target=$archive_directory/$wal_name
if test -e "$archive_target"; then
    cmp -s "$wal_source" "$archive_target"
    sync "$archive_target" "$archive_directory"
    exit 0
fi
archive_temporary=$(mktemp "$archive_directory/.incoming.XXXXXX")
trap 'rm -f "$archive_temporary"' EXIT HUP INT TERM
cp "$wal_source" "$archive_temporary"
sync "$archive_temporary"
ln "$archive_temporary" "$archive_target" || cmp -s "$wal_source" "$archive_target"
sync "$archive_target" "$archive_directory"
ARCHIVE_HELPER
chmod 700 "$lab/archive-copy.sh"
cat >> "$primary/postgresql.conf" <<CONF
listen_addresses = ''
unix_socket_directories = '$lab/socket'
port = 56432
wal_level = replica
archive_mode = on
archive_command = '$lab/archive-copy.sh "%p" "%f" "$lab/archive"'
CONF
pg_ctl -D "$primary" -l "$lab/primary.log" -w -t 15 start >/dev/null
export PGHOST=$lab/socket PGPORT=56432 PGDATABASE=postgres PGUSER=$(id -un)
export PGCONNECT_TIMEOUT=2
psql -X -v ON_ERROR_STOP=1 -qAt -c 'CREATE TABLE archive_marker (id integer PRIMARY KEY)'
pg_basebackup -D "$lab/base" -X stream -c fast -w
pg_verifybackup -q "$lab/base"
echo base_backup_verified
failures_before=$(psql -X -v ON_ERROR_STOP=1 -qAt -c 'SELECT failed_count FROM pg_stat_archiver')
psql -X -v ON_ERROR_STOP=1 -qAt <<'SQL'
ALTER SYSTEM SET archive_command='/bin/false';
SELECT pg_reload_conf();
SQL
attempt=0
while test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT current_setting('archive_command')='/bin/false'")" != t; do
    attempt=$((attempt+1))
    test "$attempt" -lt 100 || exit 1
    sleep 0.1
done
psql -X -v ON_ERROR_STOP=1 -qAt -c 'INSERT INTO archive_marker VALUES (1)' -c 'SELECT pg_switch_wal()' >/dev/null
attempt=0
while test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT failed_count>$failures_before FROM pg_stat_archiver")" != t; do
    attempt=$((attempt+1))
    test "$attempt" -lt 700 || { echo 'archive failure was not observed' >&2; exit 1; }
    sleep 0.1
done
failed_segment=$(psql -X -v ON_ERROR_STOP=1 -qAt -c 'SELECT last_failed_wal FROM pg_stat_archiver')
test ! -f "$lab/archive/$failed_segment"
echo archive_failure_observed
psql -X -v ON_ERROR_STOP=1 -qAt -c CHECKPOINT
test -f "$primary/pg_wal/$failed_segment"
echo unarchived_wal_retained_after_checkpoint
archived_before=$(psql -X -v ON_ERROR_STOP=1 -qAt -c 'SELECT archived_count FROM pg_stat_archiver')
psql -X -v ON_ERROR_STOP=1 -qAt <<'SQL'
ALTER SYSTEM RESET archive_command;
SELECT pg_reload_conf();
SQL
attempt=0
while ! test -f "$lab/archive/$failed_segment" || test "$(psql -X -v ON_ERROR_STOP=1 -qAt -c "SELECT archived_count>$archived_before FROM pg_stat_archiver")" != t; do
    attempt=$((attempt+1))
    test "$attempt" -lt 700 || { echo 'archive repair was not observed' >&2; exit 1; }
    sleep 0.1
done
cmp -s "$primary/pg_wal/$failed_segment" "$lab/archive/$failed_segment"
echo archive_repaired
"$lab/archive-copy.sh" "$primary/pg_wal/$failed_segment" "$failed_segment" "$lab/archive"
echo identical_repeat_accepted
printf 'different bytes\n' > "$lab/conflicting-input"
if "$lab/archive-copy.sh" "$lab/conflicting-input" "$failed_segment" "$lab/archive"; then
    echo 'conflicting archive input was accepted' >&2
    exit 1
fi
cmp -s "$primary/pg_wal/$failed_segment" "$lab/archive/$failed_segment"
echo conflicting_repeat_rejected
printf 'lab_directory %s\nfailed_segment %s\n' "$lab" "$failed_segment"

The script prints base_backup_verified, then temporarily replaces the archive command with a real failing executable. It waits for an observed archiver failure, checks that the failed file is absent from the archive, requests a checkpoint, and confirms the original WAL remains. Restoring the command produces archive_repaired only after archival progress and the expected destination file are observed; byte comparison checks the copy itself.

The final controls print identical_repeat_accepted and conflicting_repeat_rejected. The latter supplies deliberately different bytes under the existing name and verifies the genuine archive remains unchanged. The helper stages a copy, synchronizes it, and links the final name without overwriting another file. It returns success for an existing name only after comparing and synchronizing that content. These checks exercise the helper’s behavior, rather than assuming a successful shell invocation proves integrity.

All experimental servers stop on exit, while the printed directory retains logs and test files for inspection. A failed wait or comparison stops the drill. The deliberate archiver failure is an expected observation, not a reason to suppress errors from unrelated steps.

Preserve a recoverable interval

Monitor archive progress alongside backlog and available space. pg_stat_archiver.failed_count captures ordinary command failures such as this test; some shell failures or archiver crashes restart the process without incrementing that counter. Read server logs as well. A growing archived count alone does not prove that every file required by a particular backup and target is present.

WAL normally becomes archivable when a segment completes. On quiet databases, archive_timeout can bound waiting for segment switches, at the cost of more archive storage; streaming replication has different delivery timing. Retention must preserve the base backup, all required WAL, and relevant timeline history for the promised interval. Protect configuration and external dependencies separately. Manifest verification checks backup integrity but does not replace a restore rehearsal. Record the recovered business state and covered endpoint before treating an archive as evidence that a recovery objective is met.

Further reading

Sources