Explain · XB Field Notes

A slower query needs both a plan and a wait check

Mon, 31 Jan 2028 00:00:00 +0000

A formerly fast query can become slow with the same data, statistics, and access plan. Another transaction may hold a lock, the storage cache may be colder, or the application may wait before PostgreSQL even receives the statement. Rechecking the plan is useful, but it belongs inside a broader comparison of where the elapsed time went. Begin by distinguishing time spent doing work from time spent waiting to do it.

Keep a comparable baseline

Record the statement shape, representative parameter values, returned row count, relevant session settings, and whether the application uses a prepared statement. Different parameters can produce very different populations. A manual query with a literal may not reproduce a prepared statement’s generic plan. A different role or search path can also change which objects and permissions apply.

Inspect current activity while the slowdown is happening. pg_stat_activity can show an active query with a lock wait; active does not mean continuously consuming CPU. pg_blocking_pids identifies sessions currently blocking that backend. Visibility into other roles’ activity requires appropriate privileges, so missing details are not evidence that nothing is running. Use fresh autocommit observations rather than repeatedly reading a cached monitoring snapshot inside one long transaction.

When it breaks

Blindly running ANALYZE treats every slowdown as a stale-statistics problem. That can add maintenance work while leaving the actual blocker untouched. Likewise, building an index on a one-row table cannot release another transaction’s row lock. Accurate estimated rows and a tiny sequential scan can coexist with arbitrarily long application waits.

EXPLAIN ANALYZE is an execution, including for writes, and can itself block. A broad update copied from production is not a harmless inspection command. First inspect waiting and lock ownership; measure a safe fixture or a statement whose effects are understood. Do not terminate an unfamiliar transaction merely because its identifier appears in the blocker list.

Check it quickly

Open three fresh psql sessions, A, B, and C, using the same role and disposable PostgreSQL 18 database with normal autocommit enabled. The named schema must not already exist. Keep the session roles separate; pasting every block into one connection cannot demonstrate blocking between transactions.

In A, create the fixture, then begin a transaction that changes its only row without committing.

\set ON_ERROR_STOP on
CREATE SCHEMA query_recheck_demo;
CREATE TABLE query_recheck_demo.items (id integer PRIMARY KEY,value integer);
INSERT INTO query_recheck_demo.items VALUES (1,0);
ANALYZE query_recheck_demo.items;

BEGIN;
UPDATE query_recheck_demo.items SET value=10 WHERE id=1;
SELECT pg_backend_pid() AS blocker_pid;

In B, start the measured update. It waits until A ends its transaction or the sixty-second statement timeout expires.

SET application_name='xbcatm_recheck_waiter';
SET statement_timeout='60s';
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
UPDATE query_recheck_demo.items SET value=value+1 WHERE id=1;

While B is waiting, run this in C, before the timeout. The test application name identifies B without exposing its complete query text.

SELECT pid,state,wait_event_type,wait_event,pg_blocking_pids(pid) AS blockers
FROM pg_stat_activity WHERE application_name='xbcatm_recheck_waiter';

In A, undo the uncommitted change and release its lock. Then collect B’s completed plan.

ROLLBACK;

After B finishes successfully, use C to verify the stored value and remove the fixture.

SELECT value FROM query_recheck_demo.items WHERE id=1;
DROP SCHEMA query_recheck_demo CASCADE;

The PostgreSQL 18.6 observation showed B as active with wait_event_type=Lock, wait_event=transactionid, and A’s process identifier in blockers. The wait-event detail can depend on the exact contention; the blocking relationship is the central evidence. After A rolled back, B incremented the original zero to one. The final query returned one, and cleanup removed the schema.

B’s scan estimated and read one row. Its top Update node returned zero rows because there was no RETURNING clause, not because it changed nothing. Execution time includes the induced wait and depends on how quickly A releases it. If the timeout occurs first, roll back A and use C’s final block to remove the fixture before restarting. The canceled update did not complete its increment.

Follow the observed cause

For a lock wait, investigate transaction lifetime and ownership before changing access paths. For excessive processed rows, compare selectivity and current statistics. For sorts or hashes that spill, inspect their memory and temporary I/O. Finally, compare database execution with application latency to expose connection or client delays. A new plan is meaningful only when it explains the measured regression and preserves the required result.

Sources

PostgreSQL 18: Monitoring database activity, active sessions and wait observations, checked 2026-09-10.
PostgreSQL 18: Using EXPLAIN, row counts and loops and plan interpretation, checked 2026-09-10.

EXPLAIN ANALYZE runs the query to show real cost

Thu, 16 Dec 2027 00:00:00 +0000

EXPLAIN describes a PostgreSQL execution plan; EXPLAIN ANALYZE also executes the statement and measures its work. The useful comparison is between estimated rows and observed rows at the same node. Neither number is a verdict by itself: an estimate is a model, while a measurement describes one execution under particular data, cache, concurrency, and instrumentation conditions.

Read comparable quantities

Planner costs are relative units, not milliseconds. The two cost values describe startup and completion, and the preferred plan can change when LIMIT makes early rows especially valuable. Actual timings use milliseconds. Comparing a cost of 100 with an elapsed time of 100 is therefore meaningless even when the printed values happen to match.

For a node executed repeatedly, actual rows and node timings are averages per loop. Multiplying by loops helps expose repeated work, but summing every node’s time double-counts work included in parent nodes. Parallel execution makes naive totals still less useful. Read the tree, its loop counts, filters, and row flow before assigning blame to the visually largest number.

BUFFERS separates shared, local, and temporary block activity. Shared hits mean PostgreSQL found blocks in its shared buffer cache. A shared read does not prove a physical disk access: the operating system may satisfy it from another cache. Temporary tables use local buffers, while sort or hash spills can use temporary files. Parent counters include children, so adding all rows of counters inflates the apparent I/O.

Investigate the first meaningful discrepancy

A selective predicate estimated at ten rows but returning ten thousand deserves investigation before rewriting a join above it. Stale statistics, skew, correlated columns, parameter values, and unsupported selectivity models can all contribute. ANALYZE refreshes statistics, but it cannot guarantee exact estimates or repair every plan. Inspect representative data and consider targeted statistics changes only after identifying the missing information.

An accurate row estimate does not prove the query is efficient. Missing access paths, expensive expressions, spills, lock waits, and workload contention can remain. Conversely, a sequential scan can be appropriate for a small table or a query returning much of it. Measure the application’s actual query shape and parameters instead of treating every index-free plan as a defect.

When it breaks

The dangerous mistake is treating ANALYZE as a dry run. An UPDATE changes rows, triggers execute, locks are acquired, and a SELECT can call a function with side effects. Wrapping the statement in a transaction and rolling back restores transactional table changes, but sequence advances and external effects are not generally reversible. Use a disposable dataset when the statement’s effects are uncertain.

Timing also misleads when the first execution warms caches or instrumentation dominates a cheap operation. TIMING OFF retains actual row counts and total execution time while avoiding per-node clock reads. EXPLAIN discards ordinary result delivery, so its duration does not include sending the application’s full response over the network. Repeated representative measurements and application latency answer different parts of the performance question.

Check it quickly

Use psql against a disposable PostgreSQL 18 database. Save this block as explain.sql and run psql -X -v ON_ERROR_STOP=1 -f explain.sql. Connection options must name that database. Everything created here is temporary, and the final rollback removes it.

\set ON_ERROR_STOP on
BEGIN;
SET LOCAL statement_timeout = '5s';
CREATE TEMP TABLE orders (id integer, status text);
INSERT INTO orders SELECT g, CASE WHEN g <= 100 THEN 'paid' ELSE 'new' END
FROM generate_series(1, 10000) AS g;
ANALYZE orders;
EXPLAIN (COSTS ON) SELECT * FROM orders WHERE status = 'paid';
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
SELECT * FROM orders WHERE status = 'paid';
CREATE TEMP SEQUENCE measured_calls;
SAVEPOINT before_measurement;
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)
UPDATE orders SET id = id + 10000 WHERE id <= 3;
EXPLAIN (ANALYZE, TIMING OFF) SELECT nextval('measured_calls');
ROLLBACK TO SAVEPOINT before_measurement;
SELECT count(*) AS updated_rows_after_rollback FROM orders WHERE id > 10000;
SELECT nextval('measured_calls') AS sequence_after_rollback;
DO $$ BEGIN
 IF (SELECT count(*) FROM orders WHERE status = 'paid') <> 100 THEN
 RAISE EXCEPTION 'unexpected selected row count';
 END IF;
 IF EXISTS (SELECT FROM orders WHERE id > 10000) THEN
 RAISE EXCEPTION 'UPDATE was not rolled back';
 END IF;
 IF (SELECT last_value FROM measured_calls) <> 2 THEN
 RAISE EXCEPTION 'sequence advance unexpectedly rolled back';
 END IF;
END $$;
ROLLBACK;

The fixture contains exactly 100 paid orders. The measured SELECT reports 100 actual rows and 9,900 removed by its filter. The UPDATE’s scan reports three rows even though its top node returns zero without RETURNING; zero there does not mean nothing changed. After the savepoint rollback, the changed-row count is zero, but the next sequence value is two because the measured SELECT already consumed one.

On PostgreSQL 18.6 these assertions pass. Buffer counts, timings, formatting, and chosen plans can vary; the deterministic results are the row counts, rollback behavior, and sequence advance. The five-second statement timeout bounds individual statements in this exercise. It does not make arbitrary production writes safe to inspect.

Sources

PostgreSQL EXPLAIN, execution effects and instrumentation options, checked 2026-09-10.
PostgreSQL using EXPLAIN, costs, loops and buffer interpretation, checked 2026-09-10.
PostgreSQL planner statistics, estimates and statistics limits, checked 2026-09-10.
PostgreSQL sequence functions, sequence advances outside rollback, checked 2026-09-10.

EXPLAIN shows how Postgres will do the work

Thu, 08 Jul 2027 00:00:00 +0000

EXPLAIN prints the planner’s proposed execution plan; EXPLAIN ANALYZE executes the statement and adds measured rows and times. A slow query needs both a hypothesis and a measurement. Save the query parameters, PostgreSQL version, and original plan before changing an index or rewriting a join. Those details make the next plan comparable and prevent an improvement on one input from hiding a regression on another.

Two habits separate effective plan readers from collectors of EXPLAIN output. First, always compare estimated and actual rows before touching anything; the gap between them is the diagnosis. Second, change one thing at a time and re-check the plan, because query tuning without a before-and-after plan is indistinguishable from luck.

Read the shape top-down or leaves-first

Text output prints a parent before its indented children. Read downwards to understand which inputs a node consumes, then follow rows upwards to understand the result. Execution can be demand-driven: a LIMIT may stop requesting rows early, while a sort often needs its entire input before returning anything. Indentation describes dependencies, not a universal chronological execution order.

Costs use planner units, not milliseconds. A node’s total cost includes its children, so adding every displayed cost double-counts work. Startup cost estimates the work before the first row; total cost estimates completing the node. Actual times and rows are averages per execution when loops exceeds one. Multiply by loops when judging repeated work, and remember that parent timing also includes descendant work.

Learn the small vocabulary of node types and each plan becomes skimmable. Sequential scan, index scan, bitmap heap scan; nested loop, hash join, merge join; sort, aggregate, limit — a dozen shapes cover nearly every plan. For each, know its scaling story: nested loops love small outer inputs and punish large ones, hash joins need memory for the inner table, merge joins demand sorted inputs. The plan names the strategy; your job is knowing what each strategy costs under your data shape.

Focus on the widest node

The largest row count is a clue, not proof of the bottleneck. A cheap sequential scan can outperform many random index reads; a small node repeated thousands of times can dominate a query. Find the first substantial estimated-versus-actual row mismatch, then trace its effect on joins, sorting, and memory use. Check buffers and temporary reads or writes before deciding whether storage access, repeated execution, or a spill explains the delay.

Ask why the planner chose that shape rather than assuming stupidity. It picks the cheapest plan under its statistics; if the statistics lie, the choice looks foolish but the reasoning was sound. Check pg_stats for the misestimated column: stale stats, skewed distributions, or correlated columns the planner treats as independent. Sometimes the right fix is ANALYZE or extended statistics rather than a rewritten query — cheaper, and it fixes every query sharing the misestimate, not just this one.

Bind rows to cost with ANALYZE

EXPLAIN ANALYZE runs the real query. Large row-estimation errors can come from stale statistics, skew, or correlations between columns. Refreshing statistics may help, but does not guarantee a fix. Extended statistics can describe some correlations; a representative test must still confirm the chosen plan and result. An index is justified by the access pattern and measured cost, not just by seeing many filtered rows.

Use an isolated database for write experiments. Rolling back table changes does not undo sequence allocation or external effects from functions and triggers. Even a SELECT can call a function with side effects. Set a statement timeout, understand the statement, and use plain EXPLAIN first when execution cost is uncertain. Buffer reads also do not prove physical disk I/O: the operating system may satisfy them from its cache.

When it breaks

The classic failure is tuning the wrong query: the slow page fires five statements, the engineer optimizes the familiar one, and p99 never moves because the expensive statement was the obscure background refresh. Profile at the workload level first — pg_stat_statements ranked by total time — and let the ranking pick the target. Optimizing without ranking is how teams shave milliseconds off the innocent while the guilty runs unexamined.

Another failure is parameter sensitivity: one tenant is tiny and another holds most rows, while a prepared statement’s generic plan treats both alike. Compare representative parameters through the same prepared-statement path used by the application. PostgreSQL core has no general SQL hint syntax; investigate statistics, query shape, and generic versus custom plans before changing planner settings globally.

Check it quickly

In a writable scratch database, create data owned only by this session:

BEGIN;
SET LOCAL statement_timeout = '10s';
CREATE TEMP TABLE explain_demo AS
SELECT n AS id, n % 100 AS tenant_id FROM generate_series(1, 20000) AS g(n);
ANALYZE explain_demo;
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
CREATE INDEX explain_demo_lookup ON explain_demo (tenant_id, id);
EXPLAIN (ANALYZE, BUFFERS)
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
SELECT id FROM explain_demo WHERE tenant_id = 17 ORDER BY id LIMIT 10;
ROLLBACK;

Both queries must return the same ten IDs. Compare scan type, sort presence, rows filtered, and buffers; the second plan will commonly use the ordered index. Do not require an exact time or node shape across versions. Repeat on representative data before applying the index to a real workload; rollback removes this experiment.

Sources

PostgreSQL EXPLAIN documentation, EXPLAIN options and execution side effects, checked 2026-09-05.
PostgreSQL using EXPLAIN, plan trees, row estimates, loops and measured costs, checked 2026-09-05.