Atomic · XB Field Notes

Redis Lua scripts run without interleaving

Tue, 11 Jan 2028 00:00:00 +0000

A Redis Lua script can read data, make a decision, and update it without another client’s command interleaving between those steps. It does not mean every statement succeeds, that earlier writes are undone after an error, or that the result is automatically durable.

Keep the decision beside the data

Consider deleting a token only if it still belongs to a particular owner. A client-side GET followed later by DEL has a gap: another owner can replace the value before deletion. A script can compare the current value and conditionally delete it during one execution. A stale owner then leaves the replacement intact.

Use redis.call to access Redis data. The sandbox does not provide general filesystem or network access. Pass every accessed key through KEYS and ordinary values through ARGV, rather than constructing key names from stored data. Normal multi-key use in Redis Cluster needs keys in the same hash slot.

Prefer a native atomic command when it already expresses the condition. Redis 8.4 introduced DELEX with IFEQ for conditional string deletion. The compare-and-delete script below teaches branching and response handling; it is not a reason to replace that command with custom code. Deleting an ownership token neither completes a distributed-lock protocol nor protects external resources from stale workers.

When it breaks

An unhandled redis.call error stops the script. A SET executed before a failing LPUSH remains applied, while statements after that failure are not reached. That differs from MULTI/EXEC, where an individual runtime error normally leaves later queued commands running. Neither mechanism supplies automatic rollback of earlier successful writes.

redis.pcall returns a Redis command error to the script as a table with an err field. The script can inspect that field and continue deliberately. Catching the error does not reverse previous writes or make a failed business operation successful. Validate arguments and expected types before the first write where possible, and design the remaining failure path explicitly.

A caller timeout also is not cancellation or evidence that nothing happened. Retrying a script that increments a counter may increment it twice. Keep operations bounded and use an application-level outcome protocol where repeated effects are unacceptable. SCRIPT KILL can interrupt a running script only if it has not yet modified the dataset; it is not a safe undo button for a stuck writer.

Keep source available

EVALSHA refers to a script by digest, not a permanently installed routine. The script cache is volatile and may disappear on restart, failover, or SCRIPT FLUSH. EVAL-loaded scripts can also be evicted under the policy introduced in Redis 7.4. Keep parameterized source in the application and handle a confirmed NOSCRIPT response with the client’s supported loading mechanism.

Do not generate a different script body for every value; use arguments. A confirmed NOSCRIPT means that invocation did not execute the missing script. Reloading or sending EVAL addresses missing code, not an uncertain earlier business outcome.

Bound both loop counts and the amount of data a command processes. A short script containing an unbounded collection read can still monopolize the server. Moving logic into Redis removes client round trips, not the need for latency and memory budgets.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. The setup compiles Redis core with libc and installs the client in a temporary virtual environment; it does not install a system service.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It starts Redis on a private Unix socket under /tmp, with TCP and persistence disabled. Cleanup stops only that process and removes its runtime directory. Downloads need network access; no existing Redis instance is contacted.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


compare_delete = """
if redis.call('GET', KEYS[1]) == ARGV[1] then
 return redis.call('DEL', KEYS[1])
end
return 0
"""

with instance() as client, client.client() as rival:
 client.set("lease", "owner-a")
 assert client.get("lease") == b"owner-a"
 rival.set("lease", "owner-b")
 assert client.eval(compare_delete, 1, "lease", "owner-a") == 0
 assert client.get("lease") == b"owner-b"
 assert client.eval(compare_delete, 1, "lease", "owner-b") == 1
 assert client.eval(compare_delete, 1, "lease", "owner-b") == 0
 print("compare-and-delete preserves a new owner's token and removes only a match")

 client.set("wrong-type", "string")
 fails_after_write = """
redis.call('SET', KEYS[1], 'before')
redis.call('LPUSH', KEYS[2], 'item')
redis.call('SET', KEYS[3], 'after')
return 1
"""
 try:
 client.eval(fails_after_write, 3, "written", "wrong-type", "not-reached")
 except redis.ResponseError as error:
 assert "WRONGTYPE" in str(error)
 else:
 raise AssertionError("script should fail on the string key")
 assert client.mget("written", "wrong-type", "not-reached") == [
 b"before", b"string", None,
 ]
 print("redis.call error stops the script but keeps the earlier write")

 catches_error = """
local reply = redis.pcall('LPUSH', KEYS[1], 'item')
if type(reply) == 'table' and reply.err then
 redis.call('SET', KEYS[2], 'handled')
 return reply.err
end
return reply
"""
 response = client.eval(catches_error, 2, "wrong-type", "handled-error")
 assert response.startswith(b"WRONGTYPE")
 assert client.get("handled-error") == b"handled"
 assert client.eval(catches_error, 2, "new-list", "unused-marker") == 1
 assert client.lrange("new-list", 0, -1) == [b"item"]
 assert client.get("unused-marker") is None
 print("redis.pcall exposes errors for handling and preserves successful command replies")

 digest = client.script_load(compare_delete)
 assert client.script_exists(digest) == [True]
 client.set("lease", "owner-c")
 assert client.evalsha(digest, 1, "lease", "owner-c") == 1
 client.set("lease", "owner-d")
 assert client.script_flush(sync_type="SYNC") is True
 assert client.script_exists(digest) == [False]
 try:
 client.evalsha(digest, 1, "lease", "owner-d")
 except redis.exceptions.NoScriptError:
 pass
 else:
 raise AssertionError("flushed script remained available")
 assert client.get("lease") == b"owner-d"
 assert client.eval(compare_delete, 1, "lease", "owner-d") == 1
 print("SCRIPT FLUSH causes NOSCRIPT; EVAL with retained source can run it again")

The first case replaces owner-a with owner-b before the stale owner’s script runs. The stale token returns zero and preserves owner-b; the matching token returns one and deletes it. Another call returns zero because the key is absent. The script reads the current value rather than trusting the earlier client-side read.

The next script writes one marker, attempts LPUSH against a string, and never reaches its final marker. Assertions confirm the first write survives the error. The pcall variant handles the same error and writes its handled marker; successful LPUSH instead returns its integer result. Type-checking distinguishes that number from an error table.

Finally, SCRIPT LOAD supplies a digest that EVALSHA can execute. SCRIPT FLUSH removes that code from this private test server. The resulting NOSCRIPT leaves the token intact, and a subsequent EVAL using retained source removes it. Never run this cache-clearing exercise against a shared instance.

Four output summaries follow assertions on results and stored values. The run does not measure throughput, test a long-running script, reproduce a network failure, or establish distributed-lock, cross-shard, or durability guarantees.

Sources

Introduction to Redis Eval scripts, execution isolation and volatile script caching, checked 2026-09-08.
Redis Lua API, sandbox, command calls, and error conversion, checked 2026-09-08.
Redis EVAL, key arguments and eviction of EVAL-loaded scripts, checked 2026-09-08.
Redis SCRIPT KILL, restriction after a script performs writes, checked 2026-09-08.
Redis SCRIPT FLUSH, explicit cache removal, checked 2026-09-08.
Redis DELEX, native conditional deletion since Redis 8.4, checked 2026-09-08.
Redis Cluster specification, hash-slot boundaries for multi-key operations, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.

MULTI EXEC runs a batch of commands as one

Thu, 22 Jul 2027 00:00:00 +0000

MULTI/EXEC prevents another client’s commands from interleaving while the queued commands execute. That alone does not protect a value read earlier in the application. If two buyers read the same stock count and later write their own calculated value, wrapping each SET in a transaction can still lose an update.

Watch before reading the dependency

WATCH makes EXEC conditional on specified keys remaining unmodified since they were watched. Start the watch before reading the values used in the decision. If a watched key changes before execution, Redis rejects the whole queue and returns a null reply. Rejection leaves the rival’s change intact.

The watch and transaction belong to one connection. A redis-py pipeline reserves that connection after watch; reads before multi execute immediately, so the application can inspect them. After multi, commands are buffered for execution. A context manager releases the reserved connection and resets pipeline state when the attempt finishes.

Watch every key whose value or absence affects the decision. Watching stock while ignoring a separately stored limit does not protect that limit. Even SET with identical bytes can invalidate a watch. Creating a previously absent watched key also invalidates it. Expiration and eviction count as changes in the tested Redis version.

Retry the decision, not its old output

After a confirmed conflict, start a new attempt: establish watches, reread the dependencies, recheck the business condition, and calculate the new writes. Reusing the old quantity merely repeats the stale decision. EXEC clears watches whether it executes or aborts; UNWATCH can cancel watching when the application decides not to proceed.

Bound retries and expose exhaustion. Frequent conflicts can make a hot key expensive, even when no data is lost. Backoff may reduce contention but does not repair an incorrect dependency set. If a native atomic command expresses the operation, use it; the stock arithmetic here is deliberately split to demonstrate the stale-read problem.

Successful WATCH validation does not turn a transaction into a rollback system. Individual runtime errors inside EXEC can leave other successful commands applied. Validate the relevant data model, inspect responses, and avoid treating a transaction containing several unrelated side effects as an all-or-nothing business unit.

When it breaks

Watching after the read leaves a race before the watch began. Watching one connection and committing on another protects nothing about that second connection’s queue. Sending an email or making a payment during the retry loop can repeat an external effect even when Redis later rejects the attempt.

Distinguish a server-confirmed conflict from a transport problem. In redis-py 8.1.0, WatchError can also be raised when a connection fails while watching. That exception class alone does not prove EXEC never ran. A lost reply after submission requires outcome reconciliation or an idempotent business protocol, not an unconditional retry of an irreversible action.

The example deliberately creates conflicts through a second connection and checks the pinned client’s conflict message. That is a test assertion, not a general production error classifier. Network retries are disabled for this experiment. In Redis Cluster, ordinary multi-key transactions also require the involved keys to share a hash slot; WATCH does not add a cross-shard transaction coordinator.

Check it quickly

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


with instance() as client, client.client() as rival:
 client.set("stock", 2)
 stale = int(client.get("stock"))
 assert rival.decr("stock") == 1
 with client.pipeline(transaction=True) as transaction:
 transaction.set("stock", stale - 1)
 assert transaction.execute() == [True]
 assert client.get("stock") == b"1"
 print("a stale value written inside MULTI/EXEC still loses another decrement")

 client.set("stock", 2)
 conflicts = 0
 for attempt in range(2):
 with client.pipeline(transaction=True) as transaction:
 transaction.watch("stock")
 current = int(transaction.get("stock"))
 assert current > 0
 if attempt == 0:
 assert rival.decr("stock") == 1
 transaction.multi()
 transaction.set("stock", current - 1)
 transaction.set("reservation", "accepted")
 try:
 responses = transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 conflicts += 1
 assert client.get("stock") == b"1"
 assert client.get("reservation") is None
 continue
 assert responses == [True, True]
 break
 else:
 raise AssertionError("both attempts conflicted")
 assert conflicts == 1 and attempt == 1
 assert client.get("stock") == b"0"
 assert client.get("reservation") == b"accepted"
 print("WATCH rejects the stale queue; the second attempt rereads 1 and commits 0")

 client.set("watched", "unchanged")
 with client.pipeline(transaction=True) as transaction:
 transaction.watch("watched")
 assert transaction.get("watched") == b"unchanged"
 rival.set("watched", "unchanged")
 transaction.multi()
 transaction.set("same-value-marker", "yes")
 try:
 transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 else:
 raise AssertionError("same-value SET did not invalidate WATCH")
 assert client.get("same-value-marker") is None
 print("WATCH detects a SET even when the resulting bytes are unchanged")

 with client.pipeline(transaction=True) as transaction:
 transaction.watch("not-yet-created")
 assert transaction.get("not-yet-created") is None
 rival.set("not-yet-created", "now-present")
 transaction.multi()
 transaction.set("absent-marker", "yes")
 try:
 transaction.execute()
 except redis.WatchError as error:
 assert str(error) == "Watched variable changed."
 else:
 raise AssertionError("creating a watched key did not invalidate WATCH")
 assert client.get("absent-marker") is None
 print("WATCH also protects a read of an absent key against later creation")

The first case reads stock as two, lets another connection decrement it to one, and writes the stale calculation inside MULTI/EXEC. Stock incorrectly remains one after two intended decrements. This deterministic ordering demonstrates that non-interleaving execution cannot repair a stale read taken outside the transaction.

The second case watches before reading. A deliberate competing decrement rejects the first queue, leaving the reservation marker absent and the rival’s stock change intact. A second attempt watches again, reads one, then commits zero with the marker. Both attempts are bounded and every resulting value is asserted.

The last cases show two less obvious conflicts: SET with identical bytes, and creating a watched key that was absent. Each prevents its queued marker from appearing. Four printed summaries follow checked responses and final state, rather than relying on timing to provoke a race.

This run tests optimistic conflict handling on an isolated server. It does not establish fair scheduling, exactly-once reservations, network-failure recovery, persistence, or cluster failover. A production reservation also needs its own business identifier and a defined response to an unknown commit outcome.

Sources

Redis transactions, optimistic concurrency and transaction failure semantics, checked 2026-09-08.
Redis WATCH, conditional execution based on watched keys, checked 2026-09-08.
Redis EXEC, null replies and execution results, checked 2026-09-08.
Redis UNWATCH, clearing connection-scoped watches, checked 2026-09-08.
redis-py 8.1.0 client implementation, immediate reads, connection pinning, and WatchError paths, checked 2026-09-08.
Redis Cluster specification, hash-slot boundaries for multi-key operations, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.