Api · XB Field Notes

Accept headers let the server negotiate a reply

Tue, 23 Nov 2027 00:00:00 +0000

Negotiation selects among representations the server can actually produce. Request preferences are inputs to that decision, not evidence that the response will satisfy them. A client still needs to inspect the selected representation. Vary records the request dimensions relevant to reuse; it does not identify the chosen format, language, or compression method.

Keep the dimensions separate

Request preference	Selected representation metadata
Accept	Content-Type
Accept-Language	Content-Language
Accept-Encoding	Content-Encoding, when a content coding is applied

These dimensions are independent. Requesting JSON does not request English or gzip. Content-Language describes the intended audience’s language, not a guarantee that every character belongs to one language. A server offering English HTML and Chinese JSON cannot combine the independently preferred format and language into Chinese HTML unless that complete representation exists. Define the available combinations before deciding how to rank them.

A quality value ranges from zero to one, defaults to one, and permits at most three fractional digits. Zero means unacceptable, not merely less desirable. For media types, the most specific matching range determines a candidate’s quality. An explicit text/html;q=0 is not rescued by */*;q=1. Looking for the largest number anywhere in the field, or choosing the first comma-separated item, loses this relationship between range and candidate.

When it breaks

A missing preference field generally expresses no preference on that dimension. It is not the same input as an explicit exclusion. RFC 9110 permits a server with no acceptable representation to send 406 or disregard the field; a default response might therefore be unusable to the client. An API promising strict negotiation must define that policy itself. An exclusion communicates a constraint but does not force every deployed server to enforce it.

Different fields also have different matching rules. Accept uses media ranges; language negotiation needs a documented language-tag matching scheme; Accept-Encoding has special rules for identity, absence, and an empty value. A generic list parser cannot supply all those policies. Do not silently equate an absent field with an empty parsed collection, then conclude that the client rejected everything.

Equal weights do not establish a universal ordering rule across all servers. Make the application’s tie policy deterministic, and test it. Detailed language preferences can also expose identifying information, so neither logging complete preference profiles nor adding every possible field to Vary is automatically justified. Vary affects matching, while storage eligibility, freshness, and authorization still require their own rules.

Check it quickly

Use Python 3.11 or newer and a POSIX shell. Install the tested parser in a temporary environment, then run the Python block with "$accept_lab/venv/bin/python". Installation needs the network; the experiment itself does not send requests or start a server. It checks selection behavior in Werkzeug 3.1.8 rather than claiming to reproduce an arbitrary browser or CDN.

accept_lab=$(mktemp -d)
python3 -m venv "$accept_lab/venv"
"$accept_lab/venv/bin/python" -m pip install Werkzeug==3.1.8

from werkzeug.datastructures import Accept, LanguageAccept, MIMEAccept
from werkzeug.http import parse_accept_header

offers = ["text/html", "application/json"]
cases = [
 (None, "text/html"),
 ("application/json;q=0.9, text/html;q=0.2", "application/json"),
 ("text/*;q=0.9, text/html;q=0, application/json;q=0.5", "application/json"),
 ("application/json;q=0, text/*;q=0.4, */*;q=1", "text/html"),
 ("application/xml", None),
 ("*/*;q=0", None),
]
for header, expected in cases:
 parsed = parse_accept_header(header, MIMEAccept)
 selected = offers[0] if header is None else parsed.best_match(offers)
 assert selected == expected
 print(repr(header), "->", selected)

denied = parse_accept_header("text/html;q=0, */*;q=1", MIMEAccept)
assert denied.quality("text/html") == 0
assert denied.quality("application/json") == 1
print("specific exclusion:", denied.quality("text/html"), denied.quality("application/json"))

tied = parse_accept_header("application/json, text/html", MIMEAccept)
assert tied.best_match(offers) == "text/html"
assert tied.best_match(list(reversed(offers))) == "application/json"
print("tie: server offer order decides")

languages = parse_accept_header("fr;q=1, zh;q=0.8, en;q=0.5", LanguageAccept)
codings = parse_accept_header("gzip;q=0, br;q=1", Accept)
assert languages.best_match(["en", "zh"]) == "zh"
assert codings.best_match(["gzip", "br"]) == "br"
print("independent axes:", languages.best_match(["en", "zh"]), codings.best_match(["gzip", "br"]))

The six media results are HTML, JSON, JSON, HTML, no match, and no match. The missing-field HTML default is explicitly chosen by the caller. The two exclusion cases prove that a broader range does not override the specific candidate’s quality. The unavailable XML and universal zero cases return None; translating that result into 406 belongs to an HTTP endpoint, not this parser.

The tie test reverses the server’s offered list while leaving the request unchanged. Werkzeug follows that list when quality and specificity tie. The final checks independently select Chinese and Brotli from declared candidates; they neither create a combined representation nor perform compression. These language cases use direct tags and do not validate regional fallback. The coding check deliberately does not implement identity or missing-header semantics.

For a real integration, preserve these negative cases and inspect the actual outgoing request, returned metadata, and body. Test the default, supported alternatives, explicit exclusions, and unavailable combinations through the deployed path. Only then decide which request fields belong in Vary. A passing parser test establishes a useful component contract, not the correctness of the complete negotiation and caching pipeline.

Sources

RFC 9110: content negotiation, absence, quality values, field-specific matching, and Vary, checked 2026-09-07.
Werkzeug Accept data structures, specificity, quality, and tie behavior, checked 2026-09-07.
Werkzeug header parsing, the parser and its type-specific classes, checked 2026-09-07.

Protobuf lets schemas evolve without breaking encoders

Fri, 19 Nov 2027 00:00:00 +0000

Backward compatibility comes from never reusing a field number and from defaulting to new, unknown fields being skipped rather than fatal.

Numbers pin the contract

A protobuf field is committed to a wire identity the moment it is given a number, and that number is the contract, not the name. Renaming a field is safe, changing its wire number is a breaking change. So the mapping of tag numbers to meanings must be adjudicated carefully and never reused for a different type, because old binaries holding the old meaning will decode it wrongly the moment the number is reassigned.

The declaration order in .proto is only a convenience for reading; what travels over the wire is a sequence of (field_number, wire_type, value) pairs. Since both sides agree on those numbers, a client can safely ignore fields it does not know. That is what gives protobuf its forward-compatibility: reserving numbers for fields you plan to grow into keeps a stable namespace even before the field ships.

Forward and backward compatibility come from a few rules. Backward compatible means a newer binary can read data written by an older one, so old fields stay, and when a field goes away you mark it reserved. Forward compatible means an older binary tolerates data written by a newer schema, which works because unknown fields are preserved and re-serialized rather than dropped. Adding a new field with a new number is compatible both ways; changing the wire type of an existing field is not.

The real trap is not the field numbers but reusing them. Deleting and repurposing a number breaks every binary that still has the old definition, because the same tag now means a different thing to older readers. The norms are to reserve numbers, never delete-and-reuse, keep wire types stable for a number, and add validation for values at the boundary where data enters the system.

Unknown fields skip, not fail

The behaviours that make evolution painless are hiding in the encoding. Unknown fields are stored and passed through unchanged by default, so an old process can relay a message it does not fully understand without corrupting the parts it does. That means a gateway or a log shipper is no longer a reason to upgrade every consumer whenever the schema changes; a consumer can ignore what it does not need.

Good hygiene on top of the wire format keeps this true at scale: generate code from the same .proto, keep a schema registry, run cross-version tests between the oldest and newest clients, store field numbers in a table for review, and treat a field whose type must change as a new field plus a deprecated marker, so the incompatible rebuild stays visible instead of silently biting the next deploy.

When it breaks

Reusing a deleted field number is the classic silent corruption: an old writer stores an int32 count under tag 5 while a new reader expects a string name there, so the bytes parse as the wrong type or fail the wire-type check. Adding a required field is a different outage: old messages lack it, and strict parsers reject them, which breaks rolling upgrades. Changing int32 to string, or singular to repeated without a plan, splits the fleet into writers and readers that disagree on the same tag. Implicit-presence defaults add a quieter trap: a missing field and an explicit zero look identical, so a consumer cannot tell “not sent” from “set to false”.

Check it quickly

Save the following as proto_compat_check.py and run python3 proto_compat_check.py. It models an old and a new schema as plain dictionaries, so it needs only the standard library and changes nothing on disk. The bad pair reuses a number with a new type and adds a required field; the good pair adds an optional field with a fresh number.

"""Minimal compatibility checker: pure stdlib, no protoc required."""

def check(old, new):
 errors = []
 for num, field in new.items():
 if num in old and field["type"] != old[num]["type"]:
 errors.append(
 f"wire-type change at field {num}: "
 f"{old[num]['type']} -> {field['type']}"
 )
 if num not in old and field.get("label") == "required":
 errors.append(f"required field added at field {num}: {field['name']}")
 return errors


old = {
 1: {"name": "display_name", "type": "string", "label": "optional"},
 2: {"name": "login_count", "type": "int32", "label": "optional"},
}
bad = {
 1: {"name": "display_name", "type": "string", "label": "optional"},
 2: {"name": "login_count", "type": "string", "label": "optional"},
 3: {"name": "email", "type": "string", "label": "required"},
}
good = {
 1: {"name": "display_name", "type": "string", "label": "optional"},
 2: {"name": "login_count", "type": "int32", "label": "optional"},
 4: {"name": "email", "type": "string", "label": "optional"},
}

bad_errors = check(old, bad)
good_errors = check(old, good)
print(f"REJECT bad: {'; '.join(bad_errors)}")
print(f"ACCEPT good: 1 field added, {len(good_errors)} broken rules")
assert len(bad_errors) == 2, bad_errors
assert good_errors == [], good_errors
print("PASS: bad rejected, good accepted")

The run prints REJECT for the bad pair with two rule names, then ACCEPT for the good pair. If you see that output, the checker distinguishes number reuse and required additions from a safe additive change.

REJECT bad: wire-type change at field 2: int32 -> string; required field added at field 3: email
ACCEPT good: 1 field added, 0 broken rules
PASS: bad rejected, good accepted

The script encodes numbering and presence rules, not a full protoc compile; run protoc --version separately if you need code generation.

Sources

Protocol Buffers language guide, field numbers and evolvability, checked 2026-06-12.
Protocol Buffers overview, updating definitions without rewriting code, checked 2026-06-12.

Idempotency keys are client promises

Wed, 20 Jan 2027 00:00:00 +0000

An idempotency key names an intended action, not a transmission attempt. When a service supports it, the client sends the same key when retrying that action after an uncertain result. Generating a new key for every retry tells the server something different: this is another action, which may legitimately produce another business effect.

Create once, retain across attempts

Generate the key before the first attempt and keep it with the operation’s parameters and target. If a job can resume after a client restart, its operation record needs to retain that identity too. An identifier held only in a temporary network connection cannot reliably connect later attempts to the original intent.

Use a new key for a genuinely new action, even when the parameters happen to match a previous purchase. A body hash alone cannot distinguish “buy the same item again” from “retry the first purchase.” Keys and request fingerprints have different jobs: one identifies the action, while the other detects incompatible reuse.

Follow the provider’s format and scope. Stripe suggests random identifiers such as UUIDv4 and warns against sensitive data in keys. A key is not authentication, and possession of it must not replace authorization to access a stored result. Do not put bearer credentials or personal details into an identifier intended for correlation.

The receiver must implement the promise

Merely adding an Idempotency-Key request header does not implement deduplication. The service must recognize the identifier under its documented contract. That contract should explain which parameters are compared, which results are retained, what happens while execution is in progress, and how long records remain available.

Stripe documents saving the first executed request’s status and body, including a 500 response. It does not save results for validation failures or conflicts with concurrently executing requests when endpoint execution has not begun. Therefore, “every duplicate waits for the first result” is not a universal rule.

Nor does every provider replay identical response bytes. EC2 documents that a successful retry avoids additional action while its result can contain updated information, such as creation status. RFC 9110’s definition concerns intended effect; a provider’s result-replay policy is an additional contract.

Keep meaning stable

Reusing one key with incompatible parameters should follow the service’s explicit error behavior, not silently create a different operation or overwrite the saved result. JSON field ordering, defaults, numerical representation, and omitted fields complicate comparisons. Do not claim that arbitrary serialized bodies are semantically equivalent merely because one small example canonicalizes them.

The example accepts exactly a product string and a positive integer quantity. Sorting these two fields defines its comparison rule; it is not a general JSON equivalence algorithm. Its 409 conflict is a fixture policy, not a status guaranteed by every idempotent API.

When it breaks

A timeout does not prove that execution failed. Retrying with a fresh key can turn a lost successful reply into a duplicate purchase. Similarly, switching keys to escape a remembered error can create new work while the original outcome remains uncertain. Resolve the original action using the provider’s documented recovery path.

Retention also bounds the promise. Stripe permits pruning keys after they are at least twenty-four hours old and treats reuse after pruning as a new request. Do not read that as permanent identity storage or an exact universal expiration timer. Align client recovery windows with the actual service contract.

Check it quickly

Save the block as intent_check.py and run python3 intent_check.py with Python 3.10 or newer. It ran with Python 3.14.4 and requires only the standard library. It writes one intent record inside a temporary directory, reads it back, and removes the directory on exit. The API and response loss are explicit sequential simulations, not network requests.

import json
from pathlib import Path
from tempfile import TemporaryDirectory
from uuid import uuid4


def new_intent(parameters):
 return {"key": str(uuid4()), "parameters": dict(parameters)}


class FixtureAPI:
 # One already authenticated scope, sequential execution, no network service.
 def __init__(self):
 self.saved = {}
 self.orders = []
 self.attempts = []

 def execute(self, intent):
 key, parameters = intent["key"], intent["parameters"]
 self.attempts.append(key)
 if (set(parameters) != {"product", "quantity"}
 or not isinstance(parameters["product"], str) or not parameters["product"]
 or type(parameters["quantity"]) is not int or parameters["quantity"] < 1):
 return 400, '{"error":"invalid parameters"}'
 fingerprint = json.dumps(parameters, sort_keys=True, separators=(",", ":"))
 if key in self.saved:
 original, response = self.saved[key]
 return response if original == fingerprint else (409, '{"error":"key reused with other parameters"}')
 self.orders.append(dict(parameters))
 response = (201, json.dumps({"order_id": len(self.orders)}, separators=(",", ":")))
 self.saved[key] = (fingerprint, response)
 return response


def deliver(api, intent, lose_response=False):
 response = api.execute(intent)
 if lose_response:
 raise TimeoutError("simulated response loss after the fixture saved its result")
 return response


api = FixtureAPI()
parameters = {"product": "notebook", "quantity": 2}
intent = new_intent(parameters)

with TemporaryDirectory(prefix="idempotency-intent-") as directory:
 record = Path(directory) / "intent.json"
 with record.open("x", encoding="utf-8") as stream:
 json.dump(intent, stream)
 try:
 deliver(api, intent, lose_response=True)
 except TimeoutError:
 pass
 else:
 raise AssertionError("expected simulated response loss")
 assert len(api.orders) == 1
 with record.open(encoding="utf-8") as stream:
 recovered = json.load(stream)
 assert recovered == intent
 first_known_result = deliver(api, recovered)
 assert first_known_result == (201, '{"order_id":1}')
 assert deliver(api, recovered) == first_known_result
 assert len(api.orders) == 1 and api.attempts[:3] == [intent["key"]] * 3
 print("one persisted intent survives serialization and three attempts produce one order")

 reordered = {"key": intent["key"], "parameters": {"quantity": 2, "product": "notebook"}}
 assert deliver(api, reordered) == first_known_result
 changed = {"key": intent["key"], "parameters": {**parameters, "quantity": 3}}
 assert deliver(api, changed)[0] == 409 and len(api.orders) == 1
 assert deliver(api, recovered) == first_known_result
 print("equivalent field ordering replays; changed parameters conflict without replacing the saved result")

 mistaken_retry = new_intent(parameters)
 assert mistaken_retry["key"] != intent["key"]
 assert deliver(api, mistaken_retry) == (201, '{"order_id":2}')
 assert len(api.orders) == 2
 print("generating a new key for an uncertain retry creates a second order")

 invalid = new_intent({"product": "notebook", "quantity": 0})
 assert deliver(api, invalid)[0] == 400
 assert invalid["key"] not in api.saved and len(api.orders) == 2
 corrected = {**invalid, "parameters": {"product": "notebook", "quantity": 1}}
 assert deliver(api, corrected) == (201, '{"order_id":3}')
 assert len(api.orders) == 3
 print("this fixture does not save pre-execution validation failures; the corrected request can execute")

The fixture saves the first result before simulating a timeout. Reading the saved intent and reusing its key gives three attempts but one order. Reordered fields replay the same result; changed quantity conflicts without replacing it. A fresh key for the same uncertain operation creates order two.

A validation failure is not retained in this fixture, so correcting its invalid quantity can execute with that still-unused key. That illustrates this stated policy, not permission to change an already executed request.

The file round trip verifies identity preservation through serialization. It does not test an actual process restart, crash-safe client storage, durable server receipts, concurrent execution, real transport loss, or provider APIs. Those boundaries must be validated before treating an application deployment as retry-safe.

Sources

Stripe: Idempotent requests, stable keys, result retention, parameter checks, and pre-execution failures, checked 2026-09-08.
EC2 API idempotency, client-token semantics and results that can reflect updated status, checked 2026-09-08.
RFC 9110, section 9.2.2 defines intended effects and conditions for nonidempotent-method retries, checked 2026-09-08.

Accept plus Vary let one URL answer in several formats

Tue, 19 Jan 2027 00:00:00 +0000

One resource can have an HTML view for people and a JSON representation for programs. Sharing a URL is useful only when the origin and its intermediaries preserve that distinction. The endpoint must select an available format, serialize it honestly, describe its cache variation, and validate the selected representation rather than some other version of the same resource.

Select before you serialize and validate

For a GET response, the request’s Accept field expresses desired response media types. The request’s Content-Type instead describes request content; setting it to JSON does not ask the server to return JSON. Use the framework’s media-range parser for quality and wildcard handling. Define an explicit default for an absent Accept field, and separately define what happens when a supplied field matches nothing.

This example defaults to HTML only when Accept is absent. A supplied preference with no acceptable offer produces 406 and lists the available types. That is a deliberate endpoint policy: HTTP also allows disregarding the preference. Do not substitute 415, which concerns unsupported request content. Distinguishing these outcomes makes an API client diagnose the correct direction of the contract.

After selection, generate Content-Type from the serializer actually used. HTML data needs appropriate escaping; JSON needs JSON serialization, not string interpolation into an HTML template. Set Vary: Accept on the route’s representations, including its default. Vary does not permit storage or protect private data; the public cache directive below is appropriate only because the fixture contains the same harmless status record for every caller.

When it breaks

Testing only the origin can miss a cache serving yesterday’s HTML to today’s JSON request. Check the metadata and parsed body through the delivery path, with both request orders. A correct Vary field on new responses does not retroactively fix an already cached response whose selection metadata was missing. Use a controlled test URL or an approved invalidation procedure rather than indiscriminately flushing a shared cache.

A subtler bug uses the database row’s revision as the same strong ETag for both formats. An ETag identifies the selected representation; distinct HTML and JSON bytes cannot share that strong validator merely because they describe the same record. If conditional processing runs before format selection, the server can incorrectly return 304 for a representation the client does not hold. Select, serialize, assign the representation’s validator, then evaluate the condition.

Check it quickly

Use Python 3.11 or newer. Create the temporary environment below, then run the Python block with "$media_lab/venv/bin/python". It starts a loopback-only Werkzeug development server on an automatically allocated port and closes it afterward. Installation needs network access; the six test requests do not. This is a local protocol fixture, not a production WSGI deployment or a shared-cache test.

media_lab=$(mktemp -d)
python3 -m venv "$media_lab/venv"
"$media_lab/venv/bin/python" -m pip install Werkzeug==3.1.8

import hashlib
import html
import json
from http.client import HTTPConnection
from threading import Thread

from werkzeug.serving import make_server
from werkzeug.wrappers import Request, Response

record = {"message": "ready"}
variants = {
 "text/html": f"<!doctype html><p>{html.escape(record['message'])}</p>\n".encode(),
 "application/json": json.dumps(record).encode("utf-8"),
}


@Request.application
def application(request):
 if request.path != "/status":
 return Response("not found\n", status=404)
 if request.method != "GET":
 return Response("GET only\n", status=405, headers={"Allow": "GET"})
 media = "text/html" if "Accept" not in request.headers else (
 request.accept_mimetypes.best_match(variants)
 )
 if media is None:
 response = Response("available: text/html, application/json\n", status=406)
 response.headers["Cache-Control"] = "no-store"
 else:
 body = variants[media]
 response = Response(body, mimetype=media)
 response.set_etag(hashlib.sha256(body).hexdigest())
 response.headers["Cache-Control"] = "public, max-age=60"
 response.make_conditional(request)
 response.vary.add("Accept")
 return response


server = make_server("127.0.0.1", 0, application)
worker = Thread(target=server.serve_forever)
worker.start()


def get(accept=None, etag=None):
 headers = {}
 if accept is not None:
 headers["Accept"] = accept
 if etag is not None:
 headers["If-None-Match"] = etag
 connection = HTTPConnection("127.0.0.1", server.server_port, timeout=3)
 try:
 connection.request("GET", "/status", headers=headers)
 response = connection.getresponse()
 body = response.read()
 assert response.headers["Vary"] == "Accept"
 print(accept, "->", response.status, "bytes=", len(body))
 return response.status, response.headers, body
 finally:
 connection.close()


try:
 status, html_headers, body = get()
 assert status == 200 and body == variants["text/html"]
 assert html_headers.get_content_type() == "text/html"
 status, json_headers, body = get("application/json")
 assert status == 200 and json.loads(body) == record
 assert json_headers.get_content_type() == "application/json"
 assert html_headers["ETag"] != json_headers["ETag"]
 status, _, body = get("application/json", json_headers["ETag"])
 assert status == 304 and body == b""
 status, _, body = get("text/html", json_headers["ETag"])
 assert status == 200 and body == variants["text/html"]
 for accept in ["application/xml", "*/*;q=0"]:
 status, headers, _ = get(accept)
 assert status == 406 and headers["Cache-Control"] == "no-store"
finally:
 server.shutdown()
 server.server_close()
 worker.join()

Expected statuses are 200, 200, 304, 200, 406, and 406. The first two bodies contain the same status information in different formats, and their strong ETags differ. The third request revalidates JSON and receives no body. The fourth carries that JSON tag while requesting HTML, so it must receive HTML with 200. This is the critical negative check that a success-only format test would miss.

The final requests ask for unavailable XML and explicitly exclude every media type. Both exercise the chosen rejection policy, while no-store keeps those fixture errors out of caches. Vary is asserted on every tested response, including 304. Werkzeug supplies parsing and conditional-response handling; the example does not hand-roll header grammar or entity-tag comparisons.

Before adopting this design, ensure both serializers are part of the same resource contract and can be tested together. Distinct URLs may be operationally simpler when clients, access rules, or release schedules differ substantially. A single URL is not a reason to merge unrelated authorization rules. Whichever addressing scheme you choose, keep body verification and cross-representation validator tests in the integration suite.

Sources

RFC 9110: HTTP semantics, Accept, ETag, If-None-Match, and the distinction between 406 and 415, checked 2026-09-07.
RFC 9111: Vary matching, request matching before cached response reuse, checked 2026-09-07.
Werkzeug conditional responses, response validators and conditional body handling, checked 2026-09-07.

HTTP methods describe the intent

Sat, 24 Oct 2026 00:00:00 +0000

A URL identifies a target resource; the request method describes the operation the client asks the origin to apply. Together they form a contract read by the server, caches, proxies, crawlers, and retrying clients. A method is not a decoration on a path. It is a promise about the intended effect, and infrastructure can only make safe decisions when that promise is honest.

Safe and idempotent are different promises

Safe methods are defined as essentially read-only: GET and HEAD do not ask the origin to change state. A server may still record access logs or charge for a request; safety is about the requested target effect, not every incidental implementation action. Idempotency means that sending the same request more than once has the same intended server effect as sending it once. PUT and DELETE are idempotent by definition, but a response can still differ and a network failure can leave the client unsure whether the first request arrived.

POST usually asks the server to process a subordinate action or create a new resource. It is not idempotent merely because its handler happens to be written without a duplicate today. PATCH communicates partial modification but does not automatically promise idempotency. OPTIONS asks for communication options, and TRACE has a narrow diagnostic purpose. The origin still defines which methods are allowed on a particular target and should report that contract with Allow when returning 405.

Let the verb guide intermediaries

GET and HEAD can be prefetched or cached under the cache rules for the response. A crawler following a link must not trigger a destructive action simply because the action was hidden behind a GET URL. A retry library can consider an idempotent request for replay, but it still needs to account for authentication, deadlines, body replayability, and application preconditions. Idempotency is not a distributed transaction or a guarantee that a client will never see two responses.

PUT replaces the selected target representation, so repeating the same bytes converges on the same intended state. DELETE makes the target absent, although the first and later responses may be 204 and 404. If a service chooses an identifier and creates a new object, POST plus an idempotency key or a separate reservation record may be more accurate than pretending the operation is PUT. Write the method, response status, preconditions, and retry policy in the API contract rather than relying on folklore.

When it breaks

Using GET for a logout, purchase, or delete exposes that action to prefetchers, link scanners, browser history tools, and caches. Using POST for a full replacement hides the fact that a retry may create a second object. Treating every PUT as safe to retry without replaying its body can send an empty request or a different version. A handler that updates state for HEAD violates the method’s read-only intent even if its GET response looks correct.

Method checks also fail at boundaries. An endpoint may return 405 for a known but unsupported method, 501 for an unknown method, or 415 when the representation’s media type is unacceptable. A proxy may preserve the verb while dropping the body, or a redirect may change it; observe the actual wire request and the final response. Validate authorization and conditional headers separately. The method communicates intent, not permission, schema validity, or successful persistence.

Check it quickly

Save this as methods_probe.go and run it with Go 1.23 or newer. The loopback server stores one value, exposes GET/HEAD/PUT/DELETE/OPTIONS, rejects other methods, and enforces a small PUT body limit. It records the write count so repeated idempotent requests can be distinguished from safe reads. The state is in memory and the experiment does not claim a durable transaction.

package main

import (
 "errors"
 "fmt"
 "io"
 "net/http"
 "net/http/httptest"
 "runtime"
 "strconv"
 "strings"
 "sync"
 "sync/atomic"
 "time"
)

func must(err error) {
 if err != nil {
 panic(err)
 }
}

func check(ok bool, message string) {
 if !ok {
 panic(message)
 }
}

func main() {
 const allowed = "GET, HEAD, PUT, DELETE, OPTIONS"
 var mu sync.Mutex
 value, exists := "seed", true
 var handled, writeCalls atomic.Int64
 server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 handled.Add(1)
 defer r.Body.Close()
 if r.URL.Path != "/item" {
 http.NotFound(w, r)
 return
 }
 switch r.Method {
 case http.MethodGet, http.MethodHead:
 mu.Lock()
 body, found := value, exists
 mu.Unlock()
 if !found {
 http.NotFound(w, r)
 return
 }
 w.Header().Set("Content-Type", "text/plain; charset=utf-8")
 w.Header().Set("Content-Length", strconv.Itoa(len(body)))
 if r.Method == http.MethodGet {
 fmt.Fprint(w, body)
 }
 case http.MethodPut:
 body, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64))
 if err != nil {
 status := http.StatusBadRequest
 var oversized *http.MaxBytesError
 if errors.As(err, &oversized) {
 status = http.StatusRequestEntityTooLarge
 }
 http.Error(w, "body rejected", status)
 return
 }
 mu.Lock()
 created := !exists
 value, exists = string(body), true
 writeCalls.Add(1)
 mu.Unlock()
 if created {
 w.WriteHeader(http.StatusCreated)
 } else {
 w.WriteHeader(http.StatusNoContent)
 }
 case http.MethodDelete:
 mu.Lock()
 found := exists
 value, exists = "", false
 writeCalls.Add(1)
 mu.Unlock()
 if found {
 w.WriteHeader(http.StatusNoContent)
 } else {
 http.NotFound(w, r)
 }
 case http.MethodOptions:
 w.Header().Set("Allow", allowed)
 w.WriteHeader(http.StatusNoContent)
 case http.MethodPost, http.MethodPatch, http.MethodTrace, http.MethodConnect:
 w.Header().Set("Allow", allowed)
 http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
 default:
 http.Error(w, "unknown method", http.StatusNotImplemented)
 }
 }))
 server.Config.ReadHeaderTimeout = time.Second
 server.Config.ReadTimeout = 2 * time.Second
 server.Start()
 defer server.Close()
 transport := &http.Transport{}
 defer transport.CloseIdleConnections()
 client := &http.Client{Transport: transport, Timeout: 3 * time.Second}
 probe := func(method, payload string, status int, expected string) {
 request, err := http.NewRequest(method, server.URL+"/item", strings.NewReader(payload))
 must(err)
 response, err := client.Do(request)
 must(err)
 body, readError := io.ReadAll(response.Body)
 closeError := response.Body.Close()
 must(readError)
 must(closeError)
 check(response.Proto == "HTTP/1.1" && response.StatusCode == status, method+": wrong status/protocol")
 check(string(body) == expected, method+": wrong response body")
 if method == http.MethodHead {
 check(response.ContentLength == 4 && len(body) == 0, "HEAD must report the GET length without its body")
 }
 if method == http.MethodOptions || status == 405 {
 check(response.Header.Get("Allow") == allowed, "Allow must describe the target methods")
 }
 fmt.Printf("%s: status=%d body=%q\n", method, status, body)
 }
 probe("GET", "", 200, "seed")
 probe("GET", "", 200, "seed")
 probe("HEAD", "", 200, "")
 check(writeCalls.Load() == 0, "safe requests must not call the write operation")
 probe("PUT", "new", 204, "")
 probe("PUT", "new", 204, "")
 probe("GET", "", 200, "new")
 check(writeCalls.Load() == 2, "both repeated PUT requests must reach the server")
 probe("PUT", strings.Repeat("x", 65), 413, "body rejected\n")
 probe("GET", "", 200, "new")
 check(writeCalls.Load() == 2, "rejected PUT must leave state unchanged")
 probe("DELETE", "", 204, "")
 probe("DELETE", "", 404, "404 page not found\n")
 probe("GET", "", 404, "404 page not found\n")
 probe("OPTIONS", "", 204, "")
 probe("POST", "", 405, "method not allowed\n")
 probe("FROB", "", 501, "unknown method\n")
 check(writeCalls.Load() == 4, "method discovery and rejection must not write")
 probe("PUT", "ready", 201, "")
 probe("GET", "", 200, "ready")
 check(handled.Load() == 16 && writeCalls.Load() == 5, "wrong final request counts")
 fmt.Println("sixteen requests; five PUT/DELETE handler calls; final state ready")
 fmt.Println(runtime.Version())
}

The sixteen requests show two safe GETs and a bodyless HEAD before any write. Two identical PUTs both reach the origin but converge on the same new value; the oversized PUT is rejected with 413 and leaves that value intact. DELETE is idempotent in its state effect even though the first response is 204 and the second is 404. OPTIONS reports Allow, POST receives 405, and an unknown FROB method receives 501. A final PUT creates the missing target and GET confirms the recovery value. These results demonstrate method handling in this fixture; they do not prove cache freshness, authorization, network replay, or durable storage.

Sources

RFC 9110: HTTP Semantics, safe/idempotent methods, Allow, and status semantics, checked 2026-09-09.
MDN: HTTP request methods, method behavior and practical usage, checked 2026-09-09.
Go net/http, MaxBytesReader, request methods, and response handling, checked 2026-09-09.

Idempotency keys scope one logical action

Mon, 12 Oct 2026 00:00:00 +0000

An idempotency key identifies one logical action only within a service’s defined scope and retention window. Reusing the same text is not a universal exactly-once guarantee. The service must associate that identity with the right principal and operation, coordinate competing attempts, and retain enough authoritative information to recognize the action later.

Scope is part of the identity

A key looked up globally can accidentally merge unrelated tenants or expose another caller’s stored result. Authenticate and authorize before consulting or replaying the record. Build the lookup identity from verified scope, a stable operation identifier, and the client’s action key; preserve component boundaries instead of ambiguous string concatenation.

Which dimensions belong in scope is an API decision. EC2 explicitly distinguishes regional and zonal idempotency. The same client token can permit separate actions in different regions or availability zones under those documented rules. Do not assume that moving a retry to another region preserves a global deduplication promise.

Request meaning matters too. The same scoped key with different parameters must follow a declared conflict policy. The example fingerprints one positive integer quantity; it does not compare arbitrary HTTP bodies, perform authentication, or infer business equivalence from a hash.

Couple the effect to its receipt

A lookup followed by an independent write can race: two workers both see no record and both execute. A unique key on the receipt table is insufficient if both business effects have already occurred before one receipt insert fails. The effect and remembered result need an appropriate atomic boundary.

When both live in one transactional database, they can be committed together. This fixture uses SQLite BEGIN IMMEDIATE before lookup and writes its business row and receipt in that transaction. SQLite permits one simultaneous writer, so competing connections serialize; the composite primary key also expresses the lookup identity.

The transaction’s protection ends at that database. An external payment, email, or message publication does not roll back because a local transaction fails. Such effects need their own end-to-end delivery and deduplication contract. Calling the whole workflow exactly once does not supply one.

Retention is a correctness boundary

If a remembered key expires while an old request can still return, that request may execute again. Stripe documents that a key reused after its record is pruned creates a new request; its minimum twenty-four-hour retention statement is provider-specific, not a universal timer.

Keep recovery windows and retention aligned, and distinguish a temporary deduplication receipt from a lasting business uniqueness rule. Expiring a receipt does not undo the original order. If the business forbids that order from ever being recreated, it needs an authoritative invariant that outlives a short retry cache.

Restoring an older receipt database, clearing state, or switching to an independent store can also forget completed work. Preserve the relationship between business data and operation records when planning recovery, not merely their individual file availability.

When it breaks

A lost response after commit leaves the client uncertain but does not remove the recorded result. A failed transaction before commit should leave neither a business effect nor a successful receipt. Those are different cases and require different evidence.

Do not treat database contention, unavailable storage, or unknown outcomes as an empty lookup. Nor should an in-progress attempt automatically permit another effect. Define whether duplicates wait, receive an in-progress response, or use another documented recovery path; successful replay is only one possible state.

Check it quickly

Save the block as scope_check.py and run python3 scope_check.py with Python 3.10 or newer. It ran with Python 3.14.4 and SQLite 3.53.0. The standard-library fixture creates a database in a private temporary directory and removes it afterward. Each attempt opens its own connection; tenant and operation inputs represent already verified fixture identities.

from concurrent.futures import ThreadPoolExecutor
from contextlib import closing
import json
from pathlib import Path
import sqlite3
from tempfile import TemporaryDirectory
from threading import Barrier


with TemporaryDirectory(prefix="idempotency-scope-") as directory:
 database = Path(directory) / "orders.sqlite"
 with closing(sqlite3.connect(database)) as db:
 db.executescript("""
 CREATE TABLE effects (
 id INTEGER PRIMARY KEY, tenant TEXT NOT NULL,
 operation TEXT NOT NULL, quantity INTEGER NOT NULL CHECK (quantity > 0)
 );
 CREATE TABLE receipts (
 tenant TEXT NOT NULL, operation TEXT NOT NULL, request_key TEXT NOT NULL,
 fingerprint TEXT NOT NULL, result TEXT NOT NULL, expires_at INTEGER NOT NULL,
 PRIMARY KEY (tenant, operation, request_key)
 );
 """)

 def execute(tenant, operation, key, quantity, now, fail_before_receipt=False):
 # Tenant/operation are already authenticated/routed fixture identifiers.
 assert type(quantity) is int and quantity > 0
 fingerprint = json.dumps({"quantity": quantity}, separators=(",", ":"))
 scope = (tenant, operation, key)
 with closing(sqlite3.connect(database, timeout=5)) as db:
 with db:
 db.execute("BEGIN IMMEDIATE")
 previous = db.execute(
 "SELECT fingerprint, result, expires_at FROM receipts "
 "WHERE tenant=? AND operation=? AND request_key=?", scope,
 ).fetchone()
 if previous is not None and previous[2] > now:
 if previous[0] != fingerprint:
 return "conflict", None
 return "replay", json.loads(previous[1])
 if previous is not None:
 db.execute("DELETE FROM receipts WHERE tenant=? AND operation=? AND request_key=?", scope)
 cursor = db.execute(
 "INSERT INTO effects (tenant, operation, quantity) VALUES (?, ?, ?)",
 (tenant, operation, quantity),
 )
 result = {"effect_id": cursor.lastrowid, "quantity": quantity}
 if fail_before_receipt:
 raise RuntimeError("injected failure after effect insertion, before receipt insertion")
 db.execute("INSERT INTO receipts VALUES (?, ?, ?, ?, ?, ?)",
 (*scope, fingerprint, json.dumps(result), now + 60))
 return "created", result

 def counts():
 with closing(sqlite3.connect(database)) as db:
 return db.execute("SELECT (SELECT count(*) FROM effects), (SELECT count(*) FROM receipts)").fetchone()

 gate = Barrier(8)

 def attempt(_):
 gate.wait(timeout=5)
 return execute("north", "web-order", "intent-a", 2, 0)

 with ThreadPoolExecutor(max_workers=8) as pool:
 outcomes = list(pool.map(attempt, range(8)))
 assert [kind for kind, _ in outcomes].count("created") == 1
 assert [kind for kind, _ in outcomes].count("replay") == 7
 original = outcomes[0][1]
 assert all(result == original for _, result in outcomes) and counts() == (1, 1)
 print("eight independent connections produce one effect, one receipt, and seven replays")

 assert execute("north", "web-order", "intent-a", 3, 1) == ("conflict", None)
 assert counts() == (1, 1)
 assert execute("south", "web-order", "intent-a", 2, 1)[0] == "created"
 assert execute("north", "batch-order", "intent-a", 2, 1)[0] == "created"
 assert execute("north", "web-order", "intent-b", 2, 1)[0] == "created"
 assert counts() == (4, 4)
 print("changed parameters conflict; a different tenant, operation, or intent key has separate scope")

 try:
 execute("north", "web-order", "rolled-back", 1, 2, fail_before_receipt=True)
 except RuntimeError as error:
 assert str(error).startswith("injected failure")
 else:
 raise AssertionError("expected the transaction to fail")
 assert counts() == (4, 4)
 assert execute("north", "web-order", "rolled-back", 1, 2)[0] == "created"
 assert counts() == (5, 5)
 print("a raised exception rolls back the effect and receipt together; retry then creates one effect")

 _, committed = execute("north", "web-order", "committed-no-reply", 1, 3)
 # Discard the reply, then query through a fresh connection; no network loss is simulated here.
 assert execute("north", "web-order", "committed-no-reply", 1, 3) == ("replay", committed)
 assert counts() == (6, 6)
 print("a fresh connection replays a committed result without adding another effect")

 assert execute("north", "web-order", "intent-a", 2, 59) == ("replay", original)
 kind, after_expiry = execute("north", "web-order", "intent-a", 2, 60)
 assert kind == "created" and after_expiry["effect_id"] != original["effect_id"]
 assert counts() == (7, 6)
 print("at the explicit 60-second retention boundary, the same key creates a new effect")
 print("SQLite", sqlite3.sqlite_version, "verified counts: 7 effects, 6 retained receipts")

Eight competing connections produce one effect, one receipt, and seven replays with identical results. Changed quantity conflicts. Changing tenant, operation, or action key creates separate work under the declared scope, reaching four effects and four receipts.

An exception after inserting an effect but before its receipt rolls back both. Retrying then creates one effect. A committed result is available through a fresh connection without adding work; this discards an application reply but does not simulate network loss.

Injected times verify replay at second fifty-nine and a new effect at the exact sixty-second expiration boundary. Final counts are seven effects and six retained receipts: the expired record disappears, not its original business row.

The sixty-second window is an explicit test policy, not a recommendation. This experiment does not test process crashes, power loss, distributed databases, external side effects, authentication, or production contention. BEGIN IMMEDIATE can still fail with database-busy errors outside the small successful workload.

Sources

EC2 API idempotency, regional and zonal scope plus parameter mismatch behavior, checked 2026-09-08.
Stripe: Idempotent requests, retention, pruning, parameter checks, and concurrent-request behavior, checked 2026-09-08.
SQLite transactions, one simultaneous writer, BEGIN IMMEDIATE, and SQLITE_BUSY, checked 2026-09-08.
SQLite atomic commit, the atomic boundary around database changes and its storage assumptions, checked 2026-09-08.
Python sqlite3, connection context-manager commit/rollback and explicit connection closing, checked 2026-09-08.

HTTP methods express a contract you must implement

Thu, 08 Oct 2026 00:00:00 +0000

An HTTP method describes the action requested on a target resource. Clients and intermediaries use those semantics when deciding whether to fetch, cache, or retry. Choosing a familiar verb is only the beginning: the application must implement the behavior that the verb promises. Calling a destructive operation GET does not make it safe. A useful API documents its resource model, representation, authorization, and error conditions alongside the method.

Separate safety from idempotency

Safe methods have essentially read-only requested semantics. GET retrieves a representation, while HEAD asks for the corresponding response metadata without response content. Incidental effects such as access logging do not make a retrieval unsafe in the protocol sense. Charging an account because a crawler followed a GET link is different: the requested operation itself changes business state and violates the safety expectation.

Idempotency concerns the intended server effect of repeating the same request. PUT replaces the target resource’s state according to the supplied representation, and DELETE removes the association between the target and its current functionality. Repeating either should have the same intended effect as one request. The response code, log entries, or timing can differ. Idempotency does not promise immunity to concurrent writers or make an unauthorized request acceptable.

POST asks the resource to process the supplied content according to its own semantics; creation is one common use, not the entire definition. POST is not generally idempotent, though an application can provide a documented deduplication contract. PATCH describes a partial modification whose properties depend on the patch format and operation. Repeating “set this value” differs from repeating “increment this value.”

When it breaks

A retry layer that treats every timeout as permission to repeat a POST can duplicate an event or charge. Even with an idempotent method, the caller must understand concurrency and the resource’s current state. Conditional requests can guard against overwriting a representation changed by someone else. Method semantics help define the action; they do not replace application-level preconditions or transaction rules.

Likewise, GET does not mean every response is automatically reusable by every cache. Status, freshness directives, authorization, and representation selection still matter. Content in GET or HEAD requests has no generally defined semantics and may be rejected by implementations or intermediaries. Avoid treating a locally accepted request body as proof of an interoperable query interface.

A known method that is not allowed for a target can receive 405 with an Allow field identifying supported methods. That capability list does not grant authorization and is distinct from CORS permission headers. OPTIONS can describe available communication options without making an unrelated browser origin eligible to read the response.

Check it quickly

Save this as methods.py and run python3 methods.py. The standard-library fixture runs a temporary loopback server with an item resource and an event-creation endpoint. Requests are sent sequentially through actual HTTP connections, and the script checks both responses and the resulting application state. It is a small method-contract exercise, not a general production API.

from http.client import HTTPConnection
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
import json

state = {'item': None, 'events': []}
allowed = 'GET, HEAD, PUT, DELETE, OPTIONS'


class Handler(BaseHTTPRequestHandler):
 def reply(self, status, data=b'', extra=None, head=False):
 self.send_response(status)
 if status != 204:
 self.send_header('Content-Length', str(len(data)))
 for key, value in (extra or {}).items():
 self.send_header(key, value)
 self.end_headers()
 if not head and status != 204:
 self.wfile.write(data)

 def do_GET(self):
 if self.path == '/item' and state['item'] is not None:
 self.reply(200, state['item'])
 else:
 self.reply(404, b'missing')

 def do_HEAD(self):
 data = state['item'] if self.path == '/item' else None
 self.reply(200 if data is not None else 404, data if data is not None else b'missing', head=True)

 def do_PUT(self):
 if self.path != '/item':
 self.reply(404, b'missing')
 return
 existed = state['item'] is not None
 state['item'] = self.rfile.read(int(self.headers['Content-Length']))
 self.reply(204 if existed else 201, extra={'Location': '/item'} if not existed else None)

 def do_POST(self):
 if self.path != '/events':
 self.reply(405, extra={'Allow': allowed})
 return
 state['events'].append(self.rfile.read(int(self.headers['Content-Length'])))
 self.reply(201, extra={'Location': f'/events/{len(state["events"])}'})

 def do_DELETE(self):
 existed = self.path == '/item' and state['item'] is not None
 if existed:
 state['item'] = None
 self.reply(204 if existed else 404)

 def do_OPTIONS(self):
 self.reply(204, extra={'Allow': allowed}) if self.path == '/item' else self.reply(404)

 def log_message(self, *args):
 pass


server = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
thread = Thread(target=server.serve_forever)
thread.start()
try:
 observed = []

 def request(method, path, body=None):
 connection = HTTPConnection('127.0.0.1', server.server_port, timeout=5)
 try:
 connection.request(method, path, body)
 response = connection.getresponse()
 item = {'method': method, 'status': response.status, 'body': response.read().decode(),
 'length': response.getheader('Content-Length'),
 'location': response.getheader('Location'), 'allow': response.getheader('Allow')}
 observed.append(item)
 return item
 finally:
 connection.close()

 assert request('PUT', '/item', b'first')['status'] == 201
 assert request('PUT', '/item', b'first')['status'] == 204
 assert state['item'] == b'first'
 assert request('GET', '/item')['body'] == 'first'
 head = request('HEAD', '/item')
 assert head['body'] == '' and head['length'] == '5'
 assert request('POST', '/events', b'same')['location'] == '/events/1'
 assert request('POST', '/events', b'same')['location'] == '/events/2'
 assert len(state['events']) == 2
 assert request('DELETE', '/item')['status'] == 204
 assert request('DELETE', '/item')['status'] == 404 and state['item'] is None
 assert request('OPTIONS', '/item')['allow'] == allowed
 assert request('POST', '/item', b'wrong')['status'] == 405
 print(json.dumps({'responses': observed, 'final_item': state['item'], 'event_count': len(state['events'])}))
finally:
 server.shutdown()
 thread.join()
 server.server_close()

The first PUT creates the item and returns 201. Repeating the same PUT returns 204, while the stored value remains first. GET returns those five bytes. HEAD returns no content but reports Content-Length five, showing that the field describes the corresponding GET representation rather than a HEAD message body.

Two identical POSTs create two event entries with different Location values. DELETE then returns 204, and a second DELETE returns 404 while the item stays absent. The differing responses do not contradict the idempotent deletion effect. OPTIONS reports the item’s methods, and POST to that item returns 405 with Allow.

The result demonstrates the distinctions the client must preserve. It does not exercise PATCH, caches, conditional updates, or automatic retries. Before introducing retry behavior, specify the intended repeated effect and check the server’s implementation. A standard method communicates a promise to other components; those components cannot make a dishonest handler satisfy it.

Sources

RFC 9110: HTTP Semantics, safety, idempotency, method definitions, and Allow, checked 2026-09-13.
MDN: HTTP request methods, the purposes and properties of standard methods, checked 2026-09-13.

HTTP idempotency makes retries safe

Wed, 26 Aug 2026 00:00:00 +0000

HTTP idempotency concerns the intended effect of repeating an identical request, not whether every response looks the same. This distinction matters when a connection closes after the server applies a change but before the client receives the answer. Repetition can be appropriate, but only within the operation’s actual semantics and the client’s retry policy.

Same effect, not identical responses

RFC 9110 defines an idempotent method by comparing the intended server-side effect of multiple identical requests with one request. PUT, DELETE, and the safe methods are idempotent. A successful first PUT might create a resource and return 201; repeating it might replace that resource and return 204. Deleting an existing resource and then an absent one can likewise produce different status codes while leaving it absent.

“Safe” is a separate property: the client does not request a state change. GET, HEAD, OPTIONS, and TRACE are defined as safe. Incidental access logging is allowed, so safe does not mean literally no writes anywhere. PUT and DELETE are idempotent but not safe.

Method semantics require an implementation

Calling an incrementing endpoint PUT does not magically make repeated increments idempotent. The server must implement the advertised meaning. Likewise, a GET route that deletes a requested object contradicts safe-method semantics; prefetchers and crawlers can trigger it without intending a destructive action.

POST is not generally idempotent, but a particular POST operation can have an application-level deduplication contract. It is also not restricted to resource creation. Understand that contract before automatic replay rather than assuming every POST is impossible to retry or every method name guarantees a correct implementation.

Naturally repeatable writes and remembered operation keys solve related but distinct problems. Replacing a resource with a specified representation expresses desired state. Replaying a keyed order creation recognizes a particular action. Neither means every internal function executes exactly once.

Preconditions address competing changes

Idempotency alone does not prevent overwriting another client’s intervening update. Repeating an unconditional replacement can restore your old value over someone else’s newer value. A strong entity tag with If-Match can condition a write on the representation you actually observed.

A conditional retry can also fail after your original write succeeded but its reply was lost: the original entity tag no longer matches. The fixture returns 412 in that case. RFC 9110 also permits a success response when the requested change appears already applied; do not assume every implementation makes the same choice.

Reconcile the current resource or consult an operation record when an outcome is ambiguous. A status code alone does not reconstruct the full history of concurrent writers.

When it breaks

Blind retries can amplify load, exceed deadlines, repeat billing side effects, or overwrite intervening work. Bound attempts and elapsed time, preserve the intended target and request, and respect applicable server guidance. Idempotency is neither a concurrency-control strategy nor permission to retry indefinitely.

Also distinguish intended resource changes from incidental logging or revision history. Two diagnostic entries do not by themselves prove duplicate business effects. Conversely, two identical successful responses do not prove that only one business action occurred; inspect authoritative state.

Check it quickly

Save the block as methods_check.py and run python3 methods_check.py with Python 3.10 or newer. It ran with Python 3.14.4 and uses only the standard library. It sends real HTTP requests to its own loopback server on an assigned port, then stops it. All business state remains in fixture memory.

from http.client import HTTPConnection, RemoteDisconnected
from http.server import BaseHTTPRequestHandler, HTTPServer
from threading import Thread


state = {"body": None, "version": 0, "orders": [], "requests": []}


def etag():
 return '"v%d"' % state["version"]


class Handler(BaseHTTPRequestHandler):
 def handle_request(self):
 state["requests"].append((self.command, self.path))
 body = self.rfile.read(int(self.headers.get("Content-Length", "0")))
 status, answer = 404, b""
 if self.path == "/item":
 if self.command in ("GET", "HEAD"):
 if state["body"] is not None:
 status, answer = 200, state["body"]
 elif self.command == "PUT":
 condition = self.headers.get("If-Match")
 # The fixture accepts one exact strong entity tag, not the full grammar.
 if condition is not None and (state["body"] is None or condition != etag()):
 status = 412
 else:
 status = 201 if state["body"] is None else 204
 if body != state["body"]:
 state["version"] += 1
 state["body"] = body
 elif self.command == "DELETE":
 status = 404 if state["body"] is None else 204
 state["body"] = None
 elif self.path == "/orders" and self.command == "POST":
 state["orders"].append(body)
 status, answer = 201, str(len(state["orders"])).encode()
 if self.headers.get("X-Fixture-Drop-Reply") == "yes" and 200 <= status < 300:
 self.close_connection = True # Apply the change, then close without a response.
 return
 self.send_response(status)
 if self.path == "/item" and state["body"] is not None:
 self.send_header("ETag", etag())
 if status != 204:
 self.send_header("Content-Length", str(len(answer)))
 self.end_headers()
 if self.command != "HEAD":
 self.wfile.write(answer)

 do_GET = do_HEAD = do_PUT = do_DELETE = do_POST = handle_request

 def log_message(self, *_):
 pass


with HTTPServer(("127.0.0.1", 0), Handler) as server:
 thread = Thread(target=server.serve_forever, kwargs={"poll_interval": 0.05})
 thread.start()

 def request(method, route="/item", body=b"", headers=None):
 client = HTTPConnection(*server.server_address, timeout=2)
 try:
 client.request(method, route, body=body, headers=headers or {})
 response = client.getresponse()
 return response.status, response.read(), response.getheader("ETag")
 finally:
 client.close()

 def lost_put(body, headers=None):
 try:
 request("PUT", body=body, headers={**(headers or {}), "X-Fixture-Drop-Reply": "yes"})
 except RemoteDisconnected:
 return
 raise AssertionError("the fixture should close before sending a response")

 try:
 assert request("PUT", body=b"blue")[0] == 201
 assert request("PUT", body=b"blue")[0] == 204
 before = len(state["requests"])
 assert request("GET")[1] == request("GET")[1] == b"blue"
 assert len(state["requests"]) == before + 2 and state["body"] == b"blue"
 assert request("HEAD")[1] == b""
 assert [request("DELETE")[0], request("DELETE")[0]] == [204, 404]
 print("PUT returns 201 then 204; DELETE returns 204 then 404; intended state is unchanged by repeats")
 print("GET appends separate diagnostic records without changing the requested resource")

 lost_put(b"blue")
 assert request("GET")[1] == b"blue"
 assert request("PUT", body=b"blue")[0] == 204
 old_tag = request("GET")[2]
 lost_put(b"green", {"If-Match": old_tag})
 assert request("PUT", body=b"green", headers={"If-Match": old_tag})[0] == 412
 assert request("GET")[1] == b"green"
 print("a lost reply can follow a committed PUT; replaying its old precondition can return 412")

 tag = request("GET")[2]
 assert request("PUT", body=b"other-writer")[0] == 204
 assert request("PUT", body=b"green", headers={"If-Match": tag})[0] == 412
 assert request("GET")[1] == b"other-writer"
 assert request("PUT", body=b"green")[0] == 204
 assert request("GET")[1] == b"green"
 print("a stale conditional PUT is refused; an unconditional repeat overwrites the intervening writer")

 first = request("POST", "/orders", b"same order")
 second = request("POST", "/orders", b"same order")
 assert first[:2] == (201, b"1") and second[:2] == (201, b"2")
 assert state["orders"] == [b"same order", b"same order"]
 print("two identical POST bodies create two orders in this endpoint's contract")
 finally:
 server.shutdown()
 thread.join(timeout=2)
 assert not thread.is_alive()

Repeated PUT produces 201 then 204; repeated DELETE produces 204 then 404. GET adds two diagnostic records without changing the requested resource, and HEAD returns no body. The test compares state and responses separately.

The server then deliberately applies PUT and closes without replying. An unconditional retry preserves the requested value; replaying an old If-Match after a successful conditional write returns 412. A separate writer’s update survives a stale conditional request but is overwritten by an unconditional replacement.

Finally, identical POST bodies create two orders under this fixture’s unkeyed creation contract. These assertions exercise transport ambiguity and state transitions, not production durability, distributed concurrency, complete HTTP parsing, or business deduplication. If-Match accepts one exact strong tag here, not its full grammar. Python’s basic HTTP server is only the isolated test harness.

Sources

RFC 9110, sections 9.2.1–9.2.2 and 13.1.1 distinguish safe methods, idempotency, retries, and If-Match, checked 2026-09-08.
MDN: Idempotent, intended effects, implementation responsibility, and differing responses, checked 2026-09-08.
Python HTTP server, the local fixture’s server API and production-use warning, checked 2026-09-08.

Cursor pagination continues from a specific boundary

Wed, 19 Aug 2026 00:00:00 +0000

Keyset pagination continues after the last ordering key returned to a client. Instead of skipping a number of rows from the current beginning, it compares each candidate with a concrete boundary. This prevents a new item at the top of a feed from shifting every later page by one position. It does not freeze the dataset or promise that every future request sees the same records.

When it breaks

A timestamp alone usually fails to define a unique boundary. Several rows can share the same creation time, even when timestamps have fine precision. A strict comparison skips the remaining ties; an inclusive comparison can return already displayed rows again. Add a unique tiebreaker and use the complete ordering key consistently in both the sort and the continuation predicate.

Mutable ordering keys create a different failure. A row already displayed can move behind the cursor and appear again. An unseen row can move ahead of the cursor and disappear from the remaining traversal. Deletions and changes to filter membership also affect what a later request can return. A stable comparison boundary cannot make changing data behave like an immutable snapshot.

Offset pagination has its own useful contract: numbered positions in the current query result. It supports page numbers conveniently, but an insertion before the next offset can repeat a row and a deletion can skip one. Neither approach supplies a cross-request snapshot merely because an API calls its response a page.

Check it quickly

Use one psql session connected to a disposable PostgreSQL database. Ensure article_cursor is unused and keep autocommit enabled. The recorded run used PostgreSQL 18.6. This exercise deliberately places two rows at each timestamp, saves the first returned page, inserts another row, and finally changes two ordering keys:

CREATE SCHEMA article_cursor;
SET search_path=article_cursor,public;
SET TIME ZONE 'UTC';
CREATE TABLE feed (id bigint PRIMARY KEY,published_at timestamptz NOT NULL);
INSERT INTO feed VALUES
 (1,'2026-01-01 10:00:00+00'),(2,'2026-01-01 10:00:00+00'),
 (3,'2026-01-01 11:00:00+00'),(4,'2026-01-01 11:00:00+00'),
 (5,'2026-01-01 12:00:00+00'),(6,'2026-01-01 12:00:00+00');
CREATE INDEX feed_order ON feed (published_at DESC,id DESC);
CREATE TEMP TABLE first_page AS
 SELECT id,published_at FROM feed ORDER BY published_at DESC,id DESC LIMIT 3;
SELECT array_agg(id ORDER BY published_at DESC,id DESC) FROM first_page;
SELECT published_at AS cursor_time,id AS cursor_id FROM first_page
ORDER BY published_at,id LIMIT 1 \gset

SELECT array_agg(id ORDER BY published_at DESC,id DESC)
FROM feed WHERE published_at<:'cursor_time'::timestamptz;

PREPARE after_cursor(timestamptz,bigint) AS
SELECT array_agg(id ORDER BY published_at DESC,id DESC) FROM (
 SELECT id,published_at FROM feed
 WHERE (published_at,id)<($1,$2)
 ORDER BY published_at DESC,id DESC LIMIT 3
) AS page;
INSERT INTO feed VALUES (7,'2026-01-01 13:00:00+00');
SELECT array_agg(id ORDER BY published_at DESC,id DESC) FROM (
 SELECT id,published_at FROM feed
 ORDER BY published_at DESC,id DESC LIMIT 3 OFFSET 3
) AS page;
EXECUTE after_cursor(:'cursor_time'::timestamptz,:cursor_id);

UPDATE feed SET published_at='2026-01-01 09:00:00+00' WHERE id=6;
UPDATE feed SET published_at='2026-01-01 14:00:00+00' WHERE id=3;
EXECUTE after_cursor(:'cursor_time'::timestamptz,:cursor_id);
DEALLOCATE after_cursor;
DROP TABLE first_page;
DROP SCHEMA article_cursor CASCADE;

The first array is {6,5,4}. The cursor therefore contains row four’s timestamp and identifier. It is extracted from the materialized first page, not from a second lookup against an independently changing feed. That distinction matters when requests overlap with writes.

The timestamp-only comparison returns {2,1} and incorrectly omits row three, which shares the boundary timestamp. After inserting row seven at the top, the offset query returns {4,3,2}: row four has appeared on both pages. Using the full tuple instead returns {3,2,1}, the correct continuation for the unchanged original ordering keys.

The final updates move previously displayed row six behind the boundary and unseen row three ahead of it. Reusing the same cursor now returns {2,1,6}. Row six repeats, and row three is absent from the remaining traversal. These exact arrays demonstrate both the benefit and the limitation; they do not justify claiming that cursor pagination never repeats or misses an item.

Carry the whole query contract

For descending timestamp and descending identifier order, the continuation predicate uses a strict tuple comparison with <. An ascending traversal needs the corresponding opposite direction. Mixed sort directions require a predicate that follows that actual order; copying one tuple comparison without checking directions is unsafe. The example excludes null timestamps, because null comparison behavior needs an explicit pagination policy.

A cursor should preserve the values exactly, including timestamp precision. Associate it with the relevant filter and sorting contract so a client cannot silently apply a boundary from one search to a different search. Encoding values into an opaque string helps API evolution, but encoding alone provides neither tamper protection nor authorization. Validate input and still enforce resource access on every page.

A matching index can support locating the boundary and reading nearby entries without discarding a deep offset. The tiny fixture does not establish that the optimizer chooses this index or measure a speedup. Examine representative plans, filtering selectivity, and page depth on realistic data before making a performance claim.

If the product requires a fixed export, specify that stronger guarantee separately: a bounded database snapshot or a materialized result set has retention and resource costs. A live feed can instead document that records may change between requests. Choose the contract deliberately so users know whether they are traversing current data or a preserved result.

Sources

PostgreSQL 18 LIMIT and OFFSET, deterministic ordering and skipped-row work, checked 2026-09-12.
PostgreSQL 18 row comparisons, tuple ordering and null semantics, checked 2026-09-12.
Relay Cursor Connections, cursor boundaries and consistent edge order, checked 2026-09-12.

Api · XB Field Notes

Accept headers let the server negotiate a reply

Keep the dimensions separate

When it breaks

Check it quickly

Sources

Protobuf lets schemas evolve without breaking encoders

Numbers pin the contract

Old and new share the wire

Unknown fields skip, not fail

When it breaks

Check it quickly

Sources

Idempotency keys are client promises

Create once, retain across attempts

The receiver must implement the promise

Keep meaning stable

When it breaks

Check it quickly

Further reading

Sources

Accept plus Vary let one URL answer in several formats

Select before you serialize and validate

When it breaks

Check it quickly

Sources

HTTP methods describe the intent

Safe and idempotent are different promises

Let the verb guide intermediaries

When it breaks

Check it quickly

Further reading

Sources

Idempotency keys scope one logical action

Scope is part of the identity

Couple the effect to its receipt

Retention is a correctness boundary

When it breaks

Check it quickly

Further reading

Sources

HTTP methods express a contract you must implement

Separate safety from idempotency

When it breaks

Check it quickly

Further reading

Sources

HTTP idempotency makes retries safe

Same effect, not identical responses

Method semantics require an implementation

Preconditions address competing changes

When it breaks

Check it quickly

Further reading

Sources

Cursor pagination continues from a specific boundary

When it breaks

Check it quickly

Carry the whole query contract

Sources