Hash · XB Field Notes

A Redis hash stores many fields in one key

Thu, 24 Feb 2028 00:00:00 +0000

A Redis hash is a field-addressable record under one key, not a schema-checked object or an automatically nested document. It is useful when callers need a few attributes without replacing the whole record. That advantage depends on access patterns, field sizes, and the meaning of replies, not merely on choosing the hash type.

Read the fields your caller actually needs

HGET retrieves one field. HMGET retrieves selected fields in the requested order, while HGETALL returns the entire mapping. With redis-py’s binary responses, an absent field becomes None and an existing empty value becomes empty bytes. A truthiness check incorrectly treats those two states as equivalent. An absent key also produces missing values through HMGET.

Field names and values are bytes; a name containing a dot is still a literal name. Redis does not interpret name.first as traversal through a nested object. Likewise, a string containing JSON remains a Redis string: HGET does not parse it. Decide serialization and validation in the application and preserve the distinction between an omitted attribute and an intentionally empty one.

Updates have precise return values

HSET changes the specified fields and leaves others untouched. A multi-field call is one command, but its integer reply counts newly added fields, not successful replacements. Updating an existing name can correctly return zero. Treating that zero as failure can cause needless retries or misleading error reports.

HINCRBY changes an integer field in one command and starts from zero when the field is absent. It rejects noninteger values and overflow beyond signed 64-bit range. This avoids a read-modify-write race for that increment; it does not make surrounding application checks atomic. Separate HGET, local arithmetic, and HSET can still overwrite another client’s update.

HDEL counts fields actually removed, and deleting the final field removes the key. Do not use an empty hash as a persistent placeholder for an entity. Whole-key expiration also removes its fields. Current Redis additionally supports field deadlines; the exercise below assigns no expirations and does not test lifetime behavior.

Compact representation is an implementation choice

Small hashes can use listpack encoding instead of a hash table. Redis 8.10.1’s standard thresholds are 512 entries and 64 bytes per field name or value; deployed settings can differ. Crossing an applicable limit changes the representation, not the public hash operations or the stored values.

Compact storage trades internal scanning work for reduced overhead and locality. Raising thresholds substantially requires measurement: a command’s advertised complexity is not a promise that arbitrarily large fields or changed configuration have no cost. Inspect OBJECT ENCODING when diagnosing representation, but do not make application correctness depend on that internal label or an assumed memory-saving ratio.

When it breaks

HGETALL grows with the whole hash and transfers both names and values. Fetching thousands of fields when the caller needs two wastes work, even if they share one key. HMGET’s work grows with the requested field count; it is not free either. Never use returned map iteration order as an application ordering contract.

WRONGTYPE indicates that the key holds another Redis type, not that a field is missing. Noninteger counters require an explicit data decision, not a retry loop. One huge hash also remains one key rather than automatically spreading its fields across cluster nodes. Choose record boundaries around actual access and ownership rather than packing unrelated data together.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Redis 8.10.1, redis-py 8.1.0, and Python 3.14.4. The setup builds only Redis core with libc and installs the client in a temporary virtual environment.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. It starts a private Unix-socket server with TCP and persistence disabled. Only this server receives deliberately small thresholds: four entries and eight bytes. The test closes its client, terminates its own process, and removes the temporary runtime directory; it never connects to a default local Redis.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-hash-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--hash-max-listpack-entries", "4", "--hash-max-listpack-value", "8",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()


with instance() as client:
 assert client.hset("profile", mapping={"name": "Ada", "visits": "0", "empty": b""}) == 3
 assert client.hset("profile", "name", "Ada L") == 0
 assert client.hmget("profile", ["visits", "name", "missing", "empty"]) == [
 b"0", b"Ada L", None, b"",
 ]
 assert client.hmget("absent", ["a", "b"]) == [None, None]
 assert client.hgetall("profile") == {b"name": b"Ada L", b"visits": b"0", b"empty": b""}
 assert client.hincrby("profile", "visits", 1) == 1
 assert client.hget("profile", "name") == b"Ada L"
 print("HSET counts new fields; HMGET preserves order and distinguishes missing from empty")

 client.set("json", '{"name":"Ada"}')
 try:
 client.hget("json", "name")
 except redis.ResponseError as error:
 assert "WRONGTYPE" in str(error)
 else:
 raise AssertionError("a JSON string is not a hash")
 client.hset("literal", "name.first", b"Ada")
 assert client.hget("literal", "name") is None
 assert client.hget("literal", "name.first") == b"Ada"
 client.hset("invalid", "counter", "3.5")
 try:
 client.hincrby("invalid", "counter", 1)
 except redis.ResponseError as error:
 assert "integer" in str(error)
 else:
 raise AssertionError("an integer counter cannot contain 3.5")
 assert client.hget("invalid", "counter") == b"3.5"
 client.hset("overflow", "counter", 2**63 - 1)
 try:
 client.hincrby("overflow", "counter", 1)
 except redis.ResponseError as error:
 assert "overflow" in str(error)
 else:
 raise AssertionError("signed 64-bit integer overflow must fail")
 assert client.hget("overflow", "counter") == str(2**63 - 1).encode()
 print("wrong types and invalid or overflowing counters fail; dotted fields are literal")

 assert client.object("ENCODING", "profile") == b"listpack"
 assert client.hset("profile", "fourth", "ok") == 1
 assert client.object("ENCODING", "profile") == b"listpack"
 assert client.hset("profile", "fifth", "ok") == 1
 assert client.object("ENCODING", "profile") == b"hashtable"
 client.hset("value-limit", "field", b"abcdefgh")
 assert client.object("ENCODING", "value-limit") == b"listpack"
 client.hset("value-limit", "field", b"abcdefghi")
 assert client.object("ENCODING", "value-limit") == b"hashtable"
 assert client.hget("profile", "visits") == b"1"
 assert client.hget("value-limit", "field") == b"abcdefghi"
 print("private 4-field and 8-byte thresholds each trigger listpack to hashtable conversion")

 assert client.hdel("value-limit", "absent") == 0
 assert client.hdel("value-limit", "field") == 1
 assert client.exists("value-limit") == 0
 print("deleting the last field removes the hash key")

The assertions distinguish added fields from replacements, preserve HMGET ordering and missing-versus-empty values, and verify unchanged neighboring fields. A JSON string raises WRONGTYPE, a dotted field stays literal, and invalid or overflowing counters remain unchanged after rejection.

At four short fields the profile remains listpack; the fifth switches it to hashtable. A separate hash switches when its value grows from eight to nine bytes. Reads still return the expected values, and deleting the final field removes that key. These results demonstrate configured encoding transitions and command semantics, not production memory savings, latency, concurrent-load behavior, or cluster distribution.

Sources

Redis hashes, field-oriented operations and current expiration capabilities, checked 2026-09-08.
Redis HSET, partial updates and added-field counts, checked 2026-09-08.
Redis HMGET, requested ordering and missing-field replies, checked 2026-09-08.
Redis HGETALL, full-hash reads and complexity, checked 2026-09-08.
Redis HINCRBY, integer increments and supported range, checked 2026-09-08.
Redis HDEL, removal counts and final-field deletion, checked 2026-09-08.
Redis OBJECT ENCODING, internal representations and conversion, checked 2026-09-08.
Redis memory optimization, compact encoding thresholds and measurement guidance, checked 2026-09-08.
Redis 8.10.1 configuration, default hash thresholds, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only build and allocator selection, checked 2026-09-08.

Hash indexes recheck collisions and maintain changed keys

Mon, 24 Jan 2028 00:00:00 +0000

PostgreSQL hash indexes narrow an equality search using a hash code, then verify the original value in the table. A hash match is a candidate, not proof of equality. Two different values can have the same code, and different codes can also map to the same bucket. Neither collisions nor shared buckets imply incorrect query results when the normal recheck is performed.

What lives in a bucket

An index tuple contains a four-byte hash value and a reference to the table tuple; it does not contain the full indexed value. The bucket mapping directs a lookup to relevant index pages. Buckets are shared storage, not one separately allocated bucket for every distinct key. An overflowing bucket can acquire additional pages that its scans must visit.

PostgreSQL can sort hash codes within an individual index page to accelerate local searching, but that does not provide ordering by the original column. A hash index cannot satisfy a key range or an ordered result. It supports single-column equality access, cannot enforce uniqueness, and does not support IS NULL. A B-tree is still a candidate for equality as well as the broader operations it supports.

The absence of original values makes hash scans lossy: matching index entries must be checked against table values. A collision can therefore appear as Rows Removed by Index Recheck in an execution plan. That is evidence that candidate verification worked, not an indication that the database returned both colliding values as equal.

When it breaks

Describing an indexed update as touching only one bucket hides important work. Changing the indexed key changes the tuple’s indexed value and may send its new entry to another bucket. PostgreSQL’s versioned rows and deferred index cleanup mean obsolete references do not necessarily disappear immediately. Growth can also split a bucket, and overflow pages add I/O.

Consequently, a workload with heavily repeated values or rapid growth can behave differently from a nearly unique, steady equality workload. Hash lookup is not an unconditional constant-time latency guarantee. Modern PostgreSQL hash indexes are crash recoverable, but durability does not eliminate storage maintenance or turn the structure into a uniqueness constraint.

Check it quickly

Run this with psql on a disposable PostgreSQL 18 database. It creates two hundred thousand temporary text keys using C collation, discovers a collision with the server’s own hashtext, and tests lookup and update behavior. It does not assume hash values are portable identifiers. Rollback removes the fixture.

\set ON_ERROR_STOP on
BEGIN;
CREATE TEMP TABLE hash_items (key text COLLATE "C");
INSERT INTO hash_items SELECT 'key-'||g FROM generate_series(1,200000) AS g;
CREATE TEMP TABLE collision_pair AS
SELECT min(key) AS a,max(key) AS b FROM hash_items
GROUP BY hashtext(key) HAVING count(*)>1 ORDER BY hashtext(key) LIMIT 1;
SELECT a AS collision_a,b AS collision_b FROM collision_pair \gset
SELECT a,b,hashtext(a),hashtext(b) FROM collision_pair;
CREATE INDEX hash_items_key ON hash_items USING hash(key);
ANALYZE hash_items;
EXPLAIN (ANALYZE, BUFFERS, TIMING OFF) SELECT key FROM hash_items WHERE key=:'collision_a';
UPDATE hash_items SET key='changed-'||key WHERE key=:'collision_a';
SELECT count(*) AS old_key_rows FROM hash_items WHERE key=:'collision_a';
SELECT count(*) AS new_key_rows FROM hash_items WHERE key='changed-'||:'collision_a';
DO $$ BEGIN
 IF (SELECT hashtext(a)=hashtext(b) AND a<>b FROM collision_pair) IS DISTINCT FROM true THEN
 RAISE EXCEPTION 'fixture did not establish a collision';
 END IF;
 IF (SELECT count(*) FROM hash_items h JOIN collision_pair p ON h.key=p.a) <> 0 OR
 (SELECT count(*) FROM hash_items h JOIN collision_pair p ON h.key='changed-'||p.a) <> 1 OR
 (SELECT count(*) FROM hash_items h JOIN collision_pair p ON h.key=p.b) <> 1 THEN
 RAISE EXCEPTION 'hash lookup or indexed update returned wrong keys';
 END IF;
END $$;
ROLLBACK;

On PostgreSQL 18.6, the discovered keys were key-129539 and key-7295, both with hash code -1960436116. The equality plan used hash_items_key, returned one row, and reported one row removed by index recheck. The two strings remained distinct despite their identical hash code. This is a concrete negative control against treating hashes as collision-free keys.

Updating the first key to its changed- form left zero rows under its old value and one under its new value. The other colliding key still had one row. The final assertions verify all three facts and verify that the original pair really collided. A successful update alone would not establish that unrelated colliding entries remained searchable.

The particular pair and plan are observations from this version. If a different environment does not discover a pair, the script must stop rather than claiming a collision was tested. Even when the pair changes, the required semantics stay the same: exact equality returns only matching original values, and updating one value does not change another merely because their hashes coincide.

Evaluate maintenance with lookup performance

Measure page accesses, equality latency, update behavior, and index size using representative distributions. VACUUM can remove dead references and recycle overflow pages, but it does not shrink the hash index’s bucket count or return its allocation to the operating system. Rebuilding is a separate maintenance operation with its own cost. A smaller hash entry can be useful, yet only a full workload comparison establishes whether that benefit exceeds recheck, overflow, and write overhead.

Sources

PostgreSQL 18: Hash indexes, collision rechecks and bucket maintenance, checked 2026-09-10.
PostgreSQL 18: Index types, supported predicates for B-tree and hash indexes, checked 2026-09-10.

A Redis hash keeps related fields under one key

Sat, 13 Mar 2027 00:00:00 +0000

A Redis hash maps field names to byte values under one key. It fits an entity whose attributes are read or changed independently, such as a profile with a name and score. HGET and HSET each operate in one command; using a hash does not inherently add a second network hop compared with a string.

Read and update the intended fields

HGET fetches one field, HMGET fetches selected fields in the requested order, and HGETALL returns all field-value pairs. Redis does not automatically interpret a value as a nested object, follow a dotted field path, or index arbitrary combinations of attributes. Field naming and serialization still belong to the application.

HSET can write several fields in one command, leaving unspecified fields alone. Its integer reply counts newly added fields, not every changed value: updating an existing field can succeed while returning zero. HDEL reports how many fields it actually removed; deleting the final field removes the hash key itself. An empty hash is not a durable placeholder.

Missing fields and empty values differ. With redis-py binary responses, HMGET returns None for a missing field and empty bytes for an existing empty field. Avoid truthiness checks that collapse these states. HGETALL is convenient for small objects, but its work and response grow with the entire hash. Prefer selected fields when the caller needs only a subset.

Expiration has two levels

A key-level expiration removes the entire hash. HSET preserves that key deadline, but Redis 7.4 and newer also support individual field expiration through HEXPIRE and HPEXPIRE. Plain HSET clears the expiration of a field it overwrites; HINCRBY preserves that field’s existing deadline. Field TTL -1 means no field deadline, not immunity from the parent key’s expiration.

For Redis 8.0 and newer, HSETEX can write field values and their expiration together. Its KEEPTTL option retains field deadlines during replacement. Separate HSET and HPEXPIRE calls have a failure window between updating the value and assigning its lifetime. Choose the operation that matches the intended lifetime, and verify the actual deployed Redis version before relying on field-expiration commands.

When it breaks

Partial updates prevent replacing unrelated attributes, but they do not make HGET, local arithmetic, and HSET one atomic operation. Two clients can read the same score, each add one, and both write the same result. HINCRBY performs an integer change in one command; invalid integer values are rejected. Multi-field business conditions still need an explicitly atomic protocol, not merely a hash-shaped data model.

A serialized JSON string remains a string, so HGET against it returns WRONGTYPE. Meanwhile, large fields or unbounded field counts can make the hash expensive to transfer and operate on. A single hash is still one Redis key, not an automatic partitioning scheme. Choose an opaque string when the application always exchanges one whole payload; choose a hash when independent field access provides real value.

Check it quickly

Use a POSIX environment with a C compiler, make, curl, tar, and Python 3.10 or newer. This ran on macOS with Python 3.14.4, Redis 8.10.1, and redis-py 8.1.0. The setup builds only Redis core with libc and installs the client in a temporary virtual environment, without optional modules or system-wide installation.

redis_lab=$(mktemp -d)
curl --fail --location --output "$redis_lab/redis.tar.gz" https://download.redis.io/releases/redis-8.10.1.tar.gz
tar -xzf "$redis_lab/redis.tar.gz" -C "$redis_lab"
make -C "$redis_lab/redis-8.10.1" build redis -j2 MALLOC=libc
export REDIS_SERVER="$redis_lab/redis-8.10.1/src/redis-server"
python3 -m venv "$redis_lab/venv"
"$redis_lab/venv/bin/python" -m pip install redis==8.1.0

Run the Python block with "$redis_lab/venv/bin/python" in the same shell. Its private Redis uses a Unix socket beneath /tmp, with no TCP listener, scheduled snapshots, or append-only persistence. It closes its own client, terminates its process, and removes the temporary runtime directory. Dependency downloads need network access; the experiment cannot reach a default local Redis.

from contextlib import contextmanager
import os
from pathlib import Path
import subprocess
from tempfile import TemporaryDirectory
import time

import redis
from redis.backoff import NoBackoff
from redis.retry import Retry


@contextmanager
def instance():
 with TemporaryDirectory(prefix="redis-lab-", dir="/tmp") as directory:
 endpoint = str(Path(directory) / "redis.sock")
 process = subprocess.Popen([
 os.environ["REDIS_SERVER"], "--port", "0",
 "--unixsocket", endpoint, "--unixsocketperm", "700",
 "--save", "", "--appendonly", "no", "--dir", directory,
 "--maxmemory-policy", "noeviction",
 ], stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
 client = redis.Redis(unix_socket_path=endpoint, decode_responses=False,
 socket_timeout=2, retry=Retry(NoBackoff(), 0))
 try:
 deadline = time.monotonic() + 5
 while True:
 assert process.poll() is None, "Redis exited during startup"
 try:
 if client.ping():
 break
 except redis.ConnectionError:
 if time.monotonic() >= deadline:
 raise
 time.sleep(0.02)
 yield client
 finally:
 client.close()
 if process.poll() is None:
 process.terminate()
 try:
 _, errors = process.communicate(timeout=5)
 except subprocess.TimeoutExpired:
 process.kill()
 process.communicate()
 raise
 assert process.returncode == 0, errors.decode()



with instance() as client:
 assert client.hset("profile", mapping={
 "name": b"Ada", "score": b"0", "empty": b"",
 }) == 3
 assert client.hset("profile", "name", b"Ada L") == 0
 assert client.hmget("profile", ["name", "empty", "missing"]) == [
 b"Ada L", b"", None,
 ]
 assert client.hlen("profile") == 3
 print("HSET reports added fields, not updates; empty bytes differ from None")

 assert client.pexpire("profile", 60000)
 key_deadline = client.pexpiretime("profile")
 assert client.hpexpire("profile", 30000, "name", "score") == [1, 1]
 field_deadlines = client.hpexpiretime("profile", "name", "score")
 assert all(value > 0 for value in field_deadlines)
 assert client.hset("profile", "name", b"Grace") == 0
 assert client.pexpiretime("profile") == key_deadline
 assert client.hpttl("profile", "name") == [-1]
 assert client.hincrby("profile", "score", 1) == 1
 assert client.hpexpiretime("profile", "score") == [field_deadlines[1]]
 print("HSET keeps key TTL but clears its field TTL; HINCRBY keeps field TTL")

 assert client.hsetex("session", mapping={"token": b"A"}, px=60000) == 1
 token_deadline = client.hpexpiretime("session", "token")
 assert token_deadline[0] > 0
 assert client.hsetex("session", mapping={"token": b"B"}, keepttl=True) == 1
 assert client.hget("session", "token") == b"B"
 assert client.hpexpiretime("session", "token") == token_deadline
 print("HSETEX writes field value and TTL together; KEEPTTL preserves its deadline")

 client.hset("race", "score", 0)
 a = int(client.hget("race", "score"))
 b = int(client.hget("race", "score"))
 client.hset("race", "score", a + 1)
 client.hset("race", "score", b + 1)
 assert client.hget("race", "score") == b"1"
 client.hset("race", "score", 0)
 assert client.hincrby("race", "score", 1) == 1
 assert client.hincrby("race", "score", 1) == 2
 print("controlled stale-read schedule loses an update; two HINCRBY calls yield 2")

 client.hset("bad", "score", b"3.5")
 try:
 client.hincrby("bad", "score", 1)
 except redis.ResponseError:
 pass
 else:
 raise AssertionError("HINCRBY requires an integer")
 assert client.hget("bad", "score") == b"3.5"
 client.set("blob", b'{"name": "Ada"}')
 try:
 client.hget("blob", "name")
 except redis.ResponseError as error:
 assert "WRONGTYPE" in str(error)
 else:
 raise AssertionError("a serialized string is not a Redis hash")
 print("invalid counters and wrong key types are rejected")

 client.hset("single", "field", b"value")
 assert client.hdel("single", "missing") == 0
 assert client.hdel("single", "field") == 1
 assert client.exists("single") == 0
 print("HDEL of the last field removes the key")

HSET first returns three, then zero for a successful replacement. HMGET preserves the distinction between empty and missing. Comparing absolute deadlines shows that a name update preserves the key deadline but removes that field’s deadline, while incrementing score retains its field deadline. HSETEX demonstrates atomic field lifetime assignment and deadline-preserving replacement.

The stale-read sequence deliberately interleaves two hypothetical clients’ operations; it is a deterministic lost-update reproduction, not a concurrent load test. Its final score is one, whereas two HINCRBY calls produce two. Invalid counters and wrong types are rejected, and deleting the last field removes its key. These assertions establish command behavior, not memory efficiency, throughput, cluster scaling, or retry safety.

Sources

Redis hashes, field-oriented data modeling and command behavior, checked 2026-09-08.
Redis HSET, partial updates and newly added field counts, checked 2026-09-08.
Redis HEXPIRE, field deadlines and overwrite behavior, checked 2026-09-08.
Redis HSETEX, atomic field value/expiration updates and KEEPTTL, checked 2026-09-08.
Redis HDEL, removal counts and deletion of the final field, checked 2026-09-08.
Redis 8.10.1 build instructions, core-only compilation and allocator selection, checked 2026-09-08.