List · XB Field Notes

A Redis list and a sorted set solve different problems

Tue, 04 Jan 2028 00:00:00 +0000

A Redis list stores an ordered sequence that permits repeated values. A sorted set stores unique members ordered by numeric scores. The useful distinction is not that one is universally faster or cheaper: it is whether the application needs positional sequence operations, unique membership, score ranges, or a combination requiring a different data model.

Define what makes an entry distinct

With RPUSH, a list keeps values in append order, including duplicates. RPUSH followed by LPOP removes the oldest value; RPUSH followed by RPOP removes the newest. Other choices of ends change that order, so a list is not automatically FIFO regardless of producer commands. Removal order also does not guarantee completion order among parallel workers.

ZADD associates one score with each member. Adding an existing member again updates that score rather than creating another record. Its default return value counts new members, so zero can be a successful update. Repeated event payloads used directly as member names will collapse into one membership, which is wrong when each occurrence represents separate work.

Equal scores are allowed. Ties sort by member bytes, not arrival time or locale-aware text order; REV reverses the tie order as well. A leaderboard may accept that rule, while a timestamp index may need a distinct event identifier and an explicit tie policy. Scores are floating-point numbers, so choose units that preserve the required numeric precision.

Positions and scores select different things

Lists support more than end access. LINDEX retrieves a position and LRANGE returns an inclusive positional range. Reaching an interior position involves traversal; end access is cheap, but a large response still costs work proportional to its size. A short capped list with head or tail reads can be appropriate.

ZRANGE defaults to zero-based rank positions with an inclusive end. BYSCORE interprets the bounds as numeric scores instead; an opening parenthesis excludes a score boundary. The documented range cost is O(log N + M), with N members and M results. Large result sets and deep offset scans are not free just because the structure is indexed.

Rank-based pagination in either changing structure is not a snapshot. Insertions or score updates can shift later pages, producing repeats or omissions across separate requests. Choose the read contract before selecting a container. There is no general guarantee of similar memory usage: representation, cardinality, payload sizes, and configuration all affect the result. Measure representative data when capacity is the deciding factor.

When it breaks

A timestamp score is not a member expiration deadline. ZREMRANGEBYSCORE explicitly removes a numeric interval, while LTRIM explicitly retains a positional window. Neither cleanup happens because time passed or a read occurred. A separate append and cleanup are separate operations, so retention logic needs its own execution and concurrency policy.

A sorted set also is not a complete delayed-work scheduler. ZPOPMIN immediately removes the minimum member, even if its score encodes a future time. Reading due entries and later removing them in a second command can let multiple workers select the same job. Atomic claiming, processing acknowledgement, and recovery of abandoned work are additional requirements, not properties supplied by a timestamp score.

List pops likewise remove values before business processing finishes. A consumer failure after removal can lose the task unless the application uses a suitable processing protocol. Choose lists for sequence operations and sorted sets for score-indexed membership; do not infer delivery guarantees from either ordering scheme.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. The commands compile Redis core with libc and install its client in a temporary virtual environment, without optional modules or system-wide installation.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It creates a private Unix socket under /tmp, disables TCP, scheduled snapshots, and append-only persistence, then stops only its own process and removes its runtime directory. Downloads require network access. No default local Redis is contacted.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


with instance() as client:
 client.rpush("arrival", b"b", b"a", b"b")
 assert client.lrange("arrival", 0, -1) == [b"b", b"a", b"b"]
 assert client.lindex("arrival", 1) == b"a"
 assert client.llen("arrival") == 3
 client.zadd("ranked", {"b": 10})
 client.zadd("ranked", {"a": 10})
 assert client.zadd("ranked", {"b": 20}) == 0
 assert client.zcard("ranked") == 2
 assert client.zrange("ranked", 0, -1, withscores=True) == [(b"a", 10.0), (b"b", 20.0)]
 client.zadd("ranked", {"b": 10})
 assert client.zrange("ranked", 0, -1) == [b"a", b"b"]
 assert client.zrange("ranked", 0, -1, desc=True) == [b"b", b"a"]
 print("lists retain duplicates and positions; sorted sets update unique members and order ties by bytes")

 assert client.lpop("arrival") == b"b"
 assert client.rpop("arrival") == b"b"
 assert client.lrange("arrival", 0, -1) == [b"a"]
 client.rpush("fifo", b"old", b"new")
 client.rpush("lifo", b"old", b"new")
 assert client.lpop("fifo") == b"old"
 assert client.rpop("lifo") == b"new"
 print("RPUSH with LPOP removes oldest first; RPUSH with RPOP removes newest first")

 client.zadd("times", {"at-10": 10, "at-20": 20, "at-30": 30})
 assert client.zrange("times", 0, 1) == [b"at-10", b"at-20"]
 assert client.zrange("times", 10, 20, byscore=True) == [b"at-10", b"at-20"]
 assert client.zrange("times", "(10", 20, byscore=True) == [b"at-20"]
 assert client.zremrangebyscore("times", "-inf", "(20") == 1
 assert client.zrange("times", 0, -1) == [b"at-20", b"at-30"]
 assert client.ttl("times") == -1
 client.rpush("window", b"old", b"middle", b"new")
 client.ltrim("window", -2, -1)
 assert client.lrange("window", 0, -1) == [b"middle", b"new"]
 print("rank selects positions; score selects values; score cleanup and list trimming are explicit")

 now = 100
 client.zadd("schedule", {"future-job": now + 50})
 assert client.zrange("schedule", "-inf", now, byscore=True) == []
 assert client.zpopmin("schedule") == [(b"future-job", 150.0)]
 assert client.exists("schedule") == 0
 print("ZPOPMIN removes the minimum immediately even when its score represents a future deadline")

The list retains b, a, b as three entries and allows an interior LINDEX read. The sorted set contains only two members; updating b returns zero while changing its score. Equal-score ordering becomes a, b regardless of insertion order, and REV reverses it.

Separate FIFO and LIFO examples expose the choice of pop end. The range checks distinguish ranks from scores and inclusive from exclusive bounds. Score-based deletion removes only values below twenty, while a list trim retains its newest two values; the sorted-set key still has no expiration.

Finally, the numeric test clock is one hundred and the only scheduled score is one hundred fifty. The due-range query is empty, yet ZPOPMIN removes that future member immediately. This verifies command semantics, not a scheduler, memory comparison, crash-recovery protocol, or throughput benchmark.

Sources

Redis RPUSH, appending list values, checked 2026-09-08.
Redis LINDEX, positional access and traversal cost, checked 2026-09-08.
Redis LRANGE, inclusive positional ranges and cost, checked 2026-09-08.
Redis LTRIM, retaining a positional window, checked 2026-09-08.
Redis ZADD, unique members, updates, and numeric scores, checked 2026-09-08.
Redis ZRANGE, rank and score queries, bounds, and tie ordering, checked 2026-09-08.
Redis ZPOPMIN, immediate minimum-score removal, checked 2026-09-08.
Redis ZREMRANGEBYSCORE, explicit score-based cleanup, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.

A Redis list is a queue, a stack, or both

Sun, 18 Apr 2027 00:00:00 +0000

A Redis list is an ordered sequence of byte values with operations at both ends. Choosing the push and pop ends determines whether it behaves as a queue or a stack. Blocking operations wait for data; they are not distributed locks, acknowledgements, or a complete message-processing protocol.

Choose the ends deliberately

RPUSH followed by LPOP provides FIFO removal: older entries leave first. RPUSH followed by RPOP provides LIFO removal. Multiple values in one LPUSH are inserted at the left in argument order, so pushing a, b, c leaves c, b, a from left to right. Test actual producer commands, not just the queue diagram.

Lists allow duplicate values. Two identical messages occupy two entries; list insertion does not provide deduplication. Use distinct job identifiers when separate deliveries must remain distinguishable. LRANGE uses inclusive endpoints, with -1 denoting the final element. Removing the last entry removes the list key, and popping a missing list returns nil rather than creating an empty placeholder.

End operations are inexpensive per element, but returning a large range or removing many entries still requires proportional work. LTRIM can retain a recent window, but truncating pending work deliberately discards entries. A producer push and a separate trim are not one atomic operation. Select retention boundaries according to the data’s meaning.

Wait without monopolizing the server

BLPOP waits on the connection until an item becomes available or the timeout expires. Other connections continue operating, including the producer that supplies the item. With redis-py, success returns the key and value; timeout returns None. A zero timeout waits indefinitely. A client’s socket deadline must allow the intended server wait and response.

Blocking consumers occupy connections even when they do no work. Allocate a bounded connection budget and avoid sharing one connection between a waiting consumer and its producer. A FIFO list also orders removal, not completion: parallel workers can finish later jobs first. Blocking behavior alone does not enforce exclusive ownership of a business resource.

When it breaks

LPOP and BLPOP remove a job before its business work finishes. If the consumer crashes or loses the response, the job is no longer on the ready list. Redis does not automatically restore it or require acknowledgement. Persistence settings do not repair this application-level handoff by themselves.

LMOVE, available since Redis 6.2, atomically transfers an item between a ready list and a processing list. BLMOVE is its blocking variant. After successful processing, LREM can acknowledge one uniquely identified job by removing it from processing. The application still needs a recovery policy for abandoned work; a list does not attach a lease, timestamp, or retry count to each item.

If work succeeds but acknowledgement fails, recovery can repeat the side effect. Unique job identifiers, idempotent processing, and a policy for late workers matter; moving between lists does not establish exactly-once processing. The exercise deliberately requeues one known job to demonstrate the mechanics, not an automatic failure detector or production recovery scheduler.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. These commands build Redis core with libc and install the client in a temporary virtual environment, without optional modules or system-wide installation.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It creates a private Redis Unix socket under /tmp, disables TCP, scheduled snapshots, and append-only persistence, then terminates only its own process and removes its runtime directory. Dependency downloads need network access. No default local Redis is contacted.

from concurrent.futures import ThreadPoolExecutor

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


with instance() as client:
 client.rpush("fifo", b"a", b"b", b"c")
 assert [client.lpop("fifo") for _ in range(3)] == [b"a", b"b", b"c"]
 assert client.exists("fifo") == 0
 assert client.lpop("fifo") is None
 client.rpush("stack", b"a", b"b", b"c")
 assert [client.rpop("stack") for _ in range(3)] == [b"c", b"b", b"a"]
 client.lpush("left", b"a", b"b", b"c")
 assert client.lrange("left", 0, -1) == [b"c", b"b", b"a"]
 client.rpush("duplicates", b"same", b"same")
 assert client.llen("duplicates") == 2
 print("FIFO a,b,c; LIFO c,b,a; multi-value LPUSH reverses argument order; duplicates stay")

 def consume():
 with client.client() as connection:
 connection.client_setname("list-lab-consumer")
 return connection.blpop("channel", timeout=1)

 with ThreadPoolExecutor(max_workers=1) as worker:
 future = worker.submit(consume)
 deadline = time.monotonic() + 1
 while not any(
 entry["name"] == "list-lab-consumer" and "b" in entry["flags"]
 for entry in client.client_list()
 ):
 assert time.monotonic() < deadline, "consumer never blocked"
 time.sleep(0.01)
 client.rpush("channel", b"job:1")
 assert future.result(timeout=2) == (b"channel", b"job:1")
 assert client.blpop("empty", timeout=0.1) is None
 print("observed a blocked connection; RPUSH wakes it; empty BLPOP times out")

 client.rpush("ready", b"job:2", b"job:3")
 job = client.lmove("ready", "processing", "LEFT", "RIGHT")
 assert job == b"job:2"
 assert client.lrange("ready", 0, -1) == [b"job:3"]
 assert client.lrange("processing", 0, -1) == [b"job:2"]
 assert client.lmove("processing", "ready", "LEFT", "LEFT") == job
 assert client.lrange("ready", 0, -1) == [b"job:2", b"job:3"]
 assert client.lmove("ready", "processing", "LEFT", "RIGHT") == job
 assert client.lrem("processing", 1, job) == 1
 assert client.exists("processing") == 0
 print("LMOVE retains an in-flight job; explicit requeue and LREM acknowledgement work")

 client.set("wrong", b"not a list")
 try:
 client.lmove("ready", "wrong", "LEFT", "RIGHT")
 except redis.ResponseError as error:
 assert "WRONGTYPE" in str(error)
 else:
 raise AssertionError("LMOVE must reject a non-list destination")
 assert client.lrange("ready", 0, -1) == [b"job:3"]
 assert client.get("wrong") == b"not a list"
 client.rpush("recent", b"1", b"2", b"3", b"4")
 client.ltrim("recent", -2, -1)
 assert client.lrange("recent", 0, -1) == [b"3", b"4"]
 print("wrong-type move preserves source; LTRIM keeps only the requested window")

The first assertions distinguish FIFO, LIFO, multi-value LPUSH order, and duplicate entries. The producer waits until CLIENT LIST actually reports a blocked consumer before submitting work; a separate empty pop returns None after its finite timeout. This observes real blocking without asserting exact scheduler latency.

LMOVE leaves the claimed job in processing, an explicit reverse move requeues it, and LREM removes the later acknowledgement. A wrong-type destination is rejected without losing the source entry. Finally, LTRIM retains two selected values. These checks verify command behavior, not crash durability, automatic recovery, throughput, or distributed locking.

Sources

Redis LPUSH, multi-value insertion order, checked 2026-09-08.
Redis LRANGE, inclusive range bounds and operation cost, checked 2026-09-08.
Redis BLPOP, connection blocking, timeouts, and destructive delivery, checked 2026-09-08.
Redis LMOVE, atomic transfer and the processing-list pattern, checked 2026-09-08.
Redis LREM, removal by value and count, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.