MULTI EXEC runs a batch of commands as one

Thu, 22 Jul 2027 00:00:00 +0000

MULTI/EXEC prevents another client’s commands from interleaving while the queued commands execute. That alone does not protect a value read earlier in the application. If two buyers read the same stock count and later write their own calculated value, wrapping each SET in a transaction can still lose an update.

Watch before reading the dependency

WATCH makes EXEC conditional on specified keys remaining unmodified since they were watched. Start the watch before reading the values used in the decision. If a watched key changes before execution, Redis rejects the whole queue and returns a null reply. Rejection leaves the rival’s change intact.

The watch and transaction belong to one connection. A redis-py pipeline reserves that connection after watch; reads before multi execute immediately, so the application can inspect them. After multi, commands are buffered for execution. A context manager releases the reserved connection and resets pipeline state when the attempt finishes.

Watch every key whose value or absence affects the decision. Watching stock while ignoring a separately stored limit does not protect that limit. Even SET with identical bytes can invalidate a watch. Creating a previously absent watched key also invalidates it. Expiration and eviction count as changes in the tested Redis version.

Retry the decision, not its old output

After a confirmed conflict, start a new attempt: establish watches, reread the dependencies, recheck the business condition, and calculate the new writes. Reusing the old quantity merely repeats the stale decision. EXEC clears watches whether it executes or aborts; UNWATCH can cancel watching when the application decides not to proceed.

Bound retries and expose exhaustion. Frequent conflicts can make a hot key expensive, even when no data is lost. Backoff may reduce contention but does not repair an incorrect dependency set. If a native atomic command expresses the operation, use it; the stock arithmetic here is deliberately split to demonstrate the stale-read problem.

Successful WATCH validation does not turn a transaction into a rollback system. Individual runtime errors inside EXEC can leave other successful commands applied. Validate the relevant data model, inspect responses, and avoid treating a transaction containing several unrelated side effects as an all-or-nothing business unit.

When it breaks

Watching after the read leaves a race before the watch began. Watching one connection and committing on another protects nothing about that second connection’s queue. Sending an email or making a payment during the retry loop can repeat an external effect even when Redis later rejects the attempt.

Distinguish a server-confirmed conflict from a transport problem. In redis-py 8.1.0, WatchError can also be raised when a connection fails while watching. That exception class alone does not prove EXEC never ran. A lost reply after submission requires outcome reconciliation or an idempotent business protocol, not an unconditional retry of an irreversible action.

The example deliberately creates conflicts through a second connection and checks the pinned client’s conflict message. That is a test assertion, not a general production error classifier. Network retries are disabled for this experiment. In Redis Cluster, ordinary multi-key transactions also require the involved keys to share a hash slot; WATCH does not add a cross-shard transaction coordinator.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. The setup compiles Redis core with libc and installs the client in a temporary virtual environment; it does not install a system service.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It starts Redis on a private Unix socket under /tmp, with TCP and persistence disabled. Cleanup stops only that process and removes its runtime directory. Downloads need network access; no existing Redis instance is contacted.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


with instance() as client, client.client() as rival:
 client.set("stock", 2)
 stale = int(client.get("stock"))
 assert rival.decr("stock") == 1
 with client.pipeline(transaction=True) as transaction:
 transaction.set("stock", stale - 1)
 assert transaction.execute() == [True]
 assert client.get("stock") == b"1"
 print("a stale value written inside MULTI/EXEC still loses another decrement")

 client.set("stock", 2)
 conflicts = 0
 for attempt in range(2):
 with client.pipeline(transaction=True) as transaction:
 transaction.watch("stock")
 current = int(transaction.get("stock"))
 assert current > 0
 if attempt == 0:
 assert rival.decr("stock") == 1
 transaction.multi()
 transaction.set("stock", current - 1)
 transaction.set("reservation", "accepted")
 try:
 responses = transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 conflicts += 1
 assert client.get("stock") == b"1"
 assert client.get("reservation") is None
 continue
 assert responses == [True, True]
 break
 else:
 raise AssertionError("both attempts conflicted")
 assert conflicts == 1 and attempt == 1
 assert client.get("stock") == b"0"
 assert client.get("reservation") == b"accepted"
 print("WATCH rejects the stale queue; the second attempt rereads 1 and commits 0")

 client.set("watched", "unchanged")
 with client.pipeline(transaction=True) as transaction:
 transaction.watch("watched")
 assert transaction.get("watched") == b"unchanged"
 rival.set("watched", "unchanged")
 transaction.multi()
 transaction.set("same-value-marker", "yes")
 try:
 transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 else:
 raise AssertionError("same-value SET did not invalidate WATCH")
 assert client.get("same-value-marker") is None
 print("WATCH detects a SET even when the resulting bytes are unchanged")

 with client.pipeline(transaction=True) as transaction:
 transaction.watch("not-yet-created")
 assert transaction.get("not-yet-created") is None
 rival.set("not-yet-created", "now-present")
 transaction.multi()
 transaction.set("absent-marker", "yes")
 try:
 transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 else:
 raise AssertionError("creating a watched key did not invalidate WATCH")
 assert client.get("absent-marker") is None
 print("WATCH also protects a read of an absent key against later creation")

The first case reads stock as two, lets another connection decrement it to one, and writes the stale calculation inside MULTI/EXEC. Stock incorrectly remains one after two intended decrements. This deterministic ordering demonstrates that non-interleaving execution cannot repair a stale read taken outside the transaction.

The second case watches before reading. A deliberate competing decrement rejects the first queue, leaving the reservation marker absent and the rival’s stock change intact. A second attempt watches again, reads one, then commits zero with the marker. Both attempts are bounded and every resulting value is asserted.

The last cases show two less obvious conflicts: SET with identical bytes, and creating a watched key that was absent. Each prevents its queued marker from appearing. Four printed summaries follow checked responses and final state, rather than relying on timing to provoke a race.

This run tests optimistic conflict handling on an isolated server. It does not establish fair scheduling, exactly-once reservations, network-failure recovery, persistence, or cluster failover. A production reservation also needs its own business identifier and a defined response to an unknown commit outcome.

Sources