Query timeouts bound a bad query's cost

Wed, 02 Sep 2026 00:00:00 +0000

A query timeout limits one part of database work. It does not automatically bound connection-pool waiting, cancel remote execution, or finish transaction cleanup. Protecting shared capacity requires knowing which component owns the timer, what that component actually interrupts, and what state remains after the error.

Name the clock and the work it covers

PostgreSQL’s statement_timeout measures from a command’s arrival at the server until server completion. It is not the caller’s whole request deadline: earlier pool acquisition and client-side result handling after server completion are outside that measurement. Lock waiting can consume statement time, so “the query never started doing useful work” does not mean its statement budget was unused.

lock_timeout has a narrower meaning. It applies separately to each attempt to acquire a lock, whereas a statement timeout can also stop work that is not waiting for a lock. An idle-in-transaction session timeout is another boundary: it terminates a session waiting for a client query inside an open transaction, not a currently executing statement.

Driver settings need equally careful reading. Python’s sqlite3.connect(timeout=...) configures how long to wait for a database lock, not a universal SQL execution deadline. Copying a setting named “timeout” between drivers without checking its contract can leave the expensive query entirely unbounded.

When it breaks

A caller may stop waiting while the server continues executing. Even successfully dispatching PostgreSQL’s cancellation request does not guarantee that it changed anything; the command might already have completed. Observe the command result and transaction outcome instead of treating a timeout exception or a sent cancel request as proof that all work stopped.

A statement limit also does not refund resources already consumed. Cleanup and transaction recovery can take time, and earlier actions may have their own outcomes. Before returning a connection for reuse, finish the required recovery and verify its state. A partial result is not an acceptable substitute for an entire report unless that is the report’s declared contract.

Choose limits by workload and request budget, leaving room for acquisition, response delivery, and recovery. A fixed multiple of p99 is not a universal safety rule: the percentile describes observed durations, not which requests are legitimate. PostgreSQL warns against setting a global statement timeout indiscriminately because it affects every session; interactive requests and deliberate analytical jobs can need different policies.

Check it quickly

Save this as query_probe.py and run python3 query_probe.py. It uses standard-library SQLite and was checked with Python 3.14.4 and SQLite 3.53.0. Two connections share a disposable local database file; no PostgreSQL server or production credentials are used. The callbacks and short durations are test settings, not recommended deployment defaults.

from contextlib import closing
from pathlib import Path
import sqlite3
from tempfile import TemporaryDirectory
import time


def exercise():
 with TemporaryDirectory(prefix="query-budget-") as directory:
 database = Path(directory) / "probe.db"
 with (
 closing(sqlite3.connect(database, timeout=0.02, isolation_level=None)) as connection,
 closing(sqlite3.connect(database, timeout=0.02, isolation_level=None)) as contender,
 ):
 connection.execute("CREATE TABLE ledger(id INTEGER PRIMARY KEY)")
 connection.execute("INSERT INTO ledger VALUES (1)")
 connection.execute("BEGIN IMMEDIATE")
 try:
 contender.execute("INSERT INTO ledger VALUES (3)")
 except sqlite3.OperationalError as error:
 assert error.sqlite_errorcode == sqlite3.SQLITE_BUSY
 else:
 raise AssertionError("the held writer lock must prevent this insert")
 finally:
 connection.rollback()
 print("busy timeout rejects a competing writer; row 3 was not inserted")

 connection.execute("BEGIN")
 connection.execute("INSERT INTO ledger VALUES (2)")
 callbacks = 0
 started = time.monotonic()
 deadline = started + 0.05

 def expired():
 nonlocal callbacks
 callbacks += 1
 return int(time.monotonic() >= deadline)

 connection.set_progress_handler(expired, 1000)
 try:
 connection.execute("""
 WITH RECURSIVE numbers(n) AS (
 VALUES(1)
 UNION ALL SELECT n + 1 FROM numbers WHERE n < 100000000
 )
 SELECT sum(n) FROM numbers
 """).fetchone()
 except sqlite3.OperationalError as error:
 assert error.sqlite_errorcode == sqlite3.SQLITE_INTERRUPT
 else:
 raise AssertionError("the expensive query must reach its progress deadline")
 finally:
 connection.set_progress_handler(None, 0)
 connection.rollback()
 elapsed = time.monotonic() - started
 assert callbacks > 0 and elapsed >= 0.05
 rows = connection.execute("SELECT id FROM ledger ORDER BY id").fetchall()
 assert rows == [(1,)]
 assert connection.execute("SELECT 42").fetchone() == (42,)
 assert not connection.in_transaction
 print("progress deadline interrupts SQL after", round(elapsed, 3), "seconds")
 print("explicit rollback leaves only committed row 1; the connection remains usable")
 print("SQLite", sqlite3.sqlite_version)


exercise()

The first connection holds a write lock. The second cannot insert row 3 and returns SQLITE_BUSY after its lock-wait allowance. Releasing the first transaction clears the deliberate contention. This checks the meaning of SQLite’s connection timeout using a real lock, not a fabricated driver error.

Next, row 2 is inserted in an explicit transaction but not committed. A large recursive query runs with a progress handler that checks a monotonic deadline every thousand virtual-machine instructions. Returning nonzero interrupts the SQL, and the observed error code must be SQLITE_INTERRUPT. The elapsed query budget is longer than the connection’s lock timeout, demonstrating that these are distinct controls.

The finally block removes the handler before explicit rollback. The final table contains only previously committed row 1; rows 2 and 3 are absent. A subsequent SELECT succeeds, and the connection is outside a transaction. Those state checks establish recovery more clearly than merely catching an exception.

The progress callback runs when SQLite invokes it, so this is not a general wall-clock preemptor for arbitrary blocking operations or application code. It demonstrates local query interruption and cleanup, not PostgreSQL cancellation transport or exact timing on every platform. For the real backend, test the relevant error, transaction state, and pool-return path before claiming that a deadline contains the query’s cost.

Sources

Python sqlite3, lock-wait timeout and progress-handler interruption, checked 2026-09-09.
PostgreSQL client connection defaults, statement, lock and idle-transaction timeout semantics, checked 2026-09-09.
PostgreSQL query cancellation, cancellation dispatch is not confirmation of its effect, checked 2026-09-09.

Timeouts · XB Field Notes

Query timeouts bound a bad query's cost

Name the clock and the work it covers

When it breaks

Check it quickly

Sources