Build · XB Field Notes

CI artifacts need an explicit handoff contract

Fri, 03 Sep 2027 00:00:00 +0000

A CI job can finish successfully while the next job receives the wrong files. An artifact is a stored output handed to another step, job, person, or release process. Its name alone rarely identifies everything the consumer needs to know. Treat the handoff as a contract covering the producing run, source revision, payload, integrity evidence, access policy, and availability window. Those details turn “download the build” into a checkable operation.

Distinguish transfer from retention

An artifact can contain a binary, test report, coverage data, or a package that later becomes a release input. It is not inherently excluded from the release path. A dependency cache serves a different purpose: speeding up work that can be recreated. Decide what must remain available for approval, diagnosis, or rollback before selecting storage and retention. Rebuilding an expired output may create different bytes and requires its own validation.

GitHub Actions allows custom artifact retention subject to repository, organization, or enterprise limits. Plan around the actual configured window, including weekends and delayed approvals. A requested retention value cannot override the platform’s upper limit. If an artifact must outlive that window, use the project’s durable release or object storage with an explicit lifecycle. Do not describe a short-lived job store as an indefinite release archive.

When it breaks

Downloading a convenient “latest successful” run can pick a different commit, branch, or workflow. Resolve the intended producer explicitly and check the relationship between its source and the release being approved. The consumer also needs a dependency on successful production of the required files. An empty upload or missing artifact must not silently trigger deployment of whatever happens to be present in a workspace.

Integrity and provenance answer different questions. A checksum can detect a changed payload, but an attacker able to replace both the file and its checksum can create a consistent pair. An attestation needs verification of its subject, signer or workflow identity, and the relevant policy. Merely finding an attestation beside a binary is not evidence that the consumer validated it.

Access depends on the platform, repository visibility, and granted permissions; it is not universally equivalent to seeing a workflow. Still, package only the files needed for the handoff. Credentials, environment dumps, private headers, and unfiltered logs can become lasting copies when uploaded. Review both the member list and content before storing an artifact, particularly when build input comes from an untrusted contribution.

Check it quickly

Save the script as artifacts.py and run python3 artifacts.py. It uses only the Python standard library. The producer creates a temporary ZIP containing app.txt and a manifest with a local run identifier, a synthetic source digest, and the payload digest. The consumer reads only those named members and compares the expected producer before returning the payload.

from hashlib import sha256
from pathlib import Path
from tempfile import TemporaryDirectory
from zipfile import ZipFile
import json

payload = b'local build output\n'
source_digest = sha256(b'example source revision\n').hexdigest()
manifest = {'producer_run': 'local-run-41', 'source_sha256': source_digest,
 'files': {'app.txt': sha256(payload).hexdigest()}}


def produce(path, data):
 with ZipFile(path, 'w') as archive:
 archive.writestr('manifest.json', json.dumps(manifest, sort_keys=True))
 archive.writestr('app.txt', data)


def consume(path, expected_run):
 with ZipFile(path) as archive:
 if sorted(archive.namelist()) != ['app.txt', 'manifest.json']:
 raise ValueError('unexpected member set')
 received = json.loads(archive.read('manifest.json'))
 if (received['producer_run'] != expected_run
 or received['source_sha256'] != source_digest):
 raise ValueError('producer identity mismatch')
 data = archive.read('app.txt')
 if sha256(data).hexdigest() != received['files']['app.txt']:
 raise ValueError('payload digest mismatch')
 return data


with TemporaryDirectory() as directory:
 archive = Path(directory) / 'handoff.zip'
 produce(archive, payload)
 assert consume(archive, 'local-run-41') == payload
 rejected = []
 try:
 consume(archive, 'local-run-42')
 except ValueError as error:
 rejected.append(str(error))
 produce(archive, payload + b'changed\n')
 try:
 consume(archive, 'local-run-41')
 except ValueError as error:
 rejected.append(str(error))
 archive.unlink()
 try:
 consume(archive, 'local-run-41')
 except FileNotFoundError:
 rejected.append('artifact missing')
 assert rejected == ['producer identity mismatch', 'payload digest mismatch', 'artifact missing']
 print(json.dumps({'valid_payload_bytes': len(payload), 'rejected': rejected,
 'platform_retention_tested': False}))

The valid handoff returns nineteen bytes. Requesting local-run-42 from the archive produced by local-run-41 raises an identity mismatch. Replacing the payload while keeping its old manifest raises a digest mismatch. Deleting the archive produces a missing-artifact failure. The final result lists all three rejected cases; no failure substitutes a different build.

This is a real local archive exercise, not an execution of GitHub’s retention service or a provenance attestation. The source digest is an illustrative identifier, not a verified Git commit. A modified manifest can still tell a consistent lie. The example proves the stated checks and their limits, including that local deletion models absence without proving a platform expiration schedule.

Make the consumer’s gate explicit

Review the actual download action’s failure behavior. GitHub’s documented digest mismatch behavior includes a warning; do not assume that every integrity warning automatically fails a deployment job. If release policy requires rejection, enforce that gate and retain its result alongside the producer identity. Record where durable release bytes are stored, who can retrieve them, and when transient handoff copies may safely expire.

Sources

GitHub: store and share workflow data, artifact transfer, retention limits, and digest validation, checked 2026-09-13.
GitHub: artifact attestations, provenance evidence and its verification boundary, checked 2026-09-13.

Copy dependency inputs before application code

Tue, 17 Aug 2027 00:00:00 +0000

Dockerfile order determines which earlier inputs an expensive build operation depends on. If application source is copied before dependency installation, changing that source can make the installation run again even when the dependency declaration is unchanged. Copying the dependency inputs first, installing them, and copying application code afterward creates a smaller dependency boundary that matches the work the installer actually needs.

When it breaks

A broad COPY . . before package installation makes documentation, generated output, or unrelated source edits potential reasons to repeat expensive work. The problem is not that COPY is inherently slow. It is that the later operation depends on filesystem state containing changes that were unnecessary for resolving and installing the selected dependencies.

Copying only a manifest is insufficient when the install also consumes a lockfile, local package source, patches, workspace metadata, or a package-manager configuration file. Omitting a real dependency can preserve a cached result that no longer matches the intended build. The correct boundary includes every actual installer input, not simply whichever single filename appears in a common Dockerfile example.

Cache ordering also does not fix mutable dependencies. An unchanged install command may retain an old result while a clean build resolves different remote content. Pinning versions and verifying hashes addresses artifact selection; controlling COPY order addresses reuse. Both concerns matter, and making a build faster does not establish that its package graph is reproducible or current.

Check it quickly

Run this host Python script with a Docker builder and the pinned base available. It downloads two public idna wheels over HTTPS and verifies their fixed SHA-256 values before any build. Container builds use no network, and pip installs with required hashes from the local wheel directory. The recorded environment used Docker 29.3.1 and Python 3.11.16 inside the Linux arm64 base:

from pathlib import Path
from tempfile import TemporaryDirectory
from urllib.request import urlopen
import hashlib,json,subprocess,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
WHEELS={
 '3.10':('https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl','946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3'),
 '3.11':('https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl','771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea')}
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result

name='xbcatm-order-'+uuid.uuid4().hex[:10];tags=[name+':early',name+':late']
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'wheels').mkdir()
 for version,(url,digest) in WHEELS.items():
 with urlopen(url,timeout=40) as response:data=response.read()
 assert hashlib.sha256(data).hexdigest()==digest
 (root/'wheels'/url.rsplit('/',1)[1]).write_bytes(data)
 install='RUN pip install --no-index --no-deps --require-hashes --find-links=/wheels -r /app/requirements.txt && python -c "import time;open(\'/installed\',\'w\').write(str(time.time_ns()))"\n'
 common=f'FROM {BASE}\nLABEL example.run="{name}"\nCOPY wheels /wheels\n'
 (root/'early.Dockerfile').write_text(common+'COPY requirements.txt app.py /app/\n'+install+'CMD ["python", "/app/app.py"]\n')
 (root/'late.Dockerfile').write_text(common+'COPY requirements.txt /app/\n'+install+'COPY app.py /app/\nCMD ["python", "/app/app.py"]\n')
 def requirement(version):
 (root/'requirements.txt').write_text(f'idna=={version} --hash=sha256:{WHEELS[version][1]}\n')
 def application(message):
 (root/'app.py').write_text(f'import idna\nprint({message!r},idna.__version__,idna.encode("bücher.example").decode())\n')
 def build():
 for tag,kind in zip(tags,['early','late']):docker('build','--network','none','-f',str(root/(kind+'.Dockerfile')),'-t',tag,str(root))
 def marks():
 return [docker('run','--rm','--network','none',tag,'cat','/installed').stdout for tag in tags]
 requirement('3.10');application('first');build();first=marks()
 application('second');build();second=marks()
 assert first[0]!=second[0] and first[1]==second[1]
 for tag in tags:assert docker('run','--rm','--network','none',tag).stdout.strip()=='second 3.10 xn--bcher-kva.example'
 print('editing only application code reruns the early-copy dependency install but reuses the late-copy install')
 requirement('3.11');build();third=marks()
 assert all(a!=b for a,b in zip(second,third))
 for tag in tags:assert docker('run','--rm','--network','none',tag).stdout.strip()=='second 3.11 xn--bcher-kva.example'
 print('changing the hashed requirement reruns both installs and actually changes idna from 3.10 to 3.11')
finally:docker('image','rm',*tags,check=False)

The early-copy Dockerfile copies both requirements.txt and app.py before pip runs. The late-copy version copies the requirement first, runs the same installation command, and copies the application afterward. Both include the verified wheel directory before installation because those wheel files are genuine inputs. A marker written immediately after pip records whether that installation operation executed.

The initial application uses idna 3.10 to encode bücher.example. Both images run the actual installed library and produce the expected ASCII domain xn--bcher-kva.example. This exercises a real third-party dependency, rather than replacing dependency installation with a sleep or treating a successful image build as proof that the application can import its package.

After only the application message changes from first to second, both images print the new message with version 3.10. The early-copy image gets a different installation marker, while the late-copy image keeps its original marker. The output proves the code update reached both images; the marker comparison identifies which dependency work was unnecessarily repeated.

Next, the requirement changes to idna 3.11 with its matching hash. Both installation markers change, and both applications report version 3.11 with the same expected domain conversion. This is the necessary opposite case: a genuine dependency declaration change must invalidate the install in both layouts. A layout that ignored this input would be fast but incorrect.

The experiment supplies both wheels from the beginning so changing the requirement does not simultaneously alter the wheel directory. That keeps the cause of the second invalidation explicit. Temporary images and files are cleaned up without pruning unrelated builder state. Installation timestamps are test instrumentation, not a recommended input for reproducible production artifacts.

Match the cache boundary to the package boundary

Review what the installer reads, then group those inputs before the install operation. Keep unrelated files outside that COPY scope and use .dockerignore to exclude material the build never needs. An ignored file cannot later satisfy a required COPY, so check actual build-context contents when a supposedly present dependency disappears.

Package-manager cache mounts can reduce downloads when an install genuinely must rerun, but they do not eliminate the need for correct invalidation or verified outputs. Dependency installation and download-cache reuse are different layers of optimization. Test the resulting application after changes to either one.

Measure improvement on representative builds: unchanged inputs, source-only edits, and dependency upgrades. Retain correctness assertions for all three cases. A useful Dockerfile makes the necessary work repeat and the unnecessary work reusable, with enough evidence to tell those cases apart.

Sources

Optimize Docker cache usage, stable dependency layers and narrow COPY operations, checked 2026-09-12.
Docker cache invalidation, how changed inputs invalidate dependent work, checked 2026-09-12.
pip install reference, offline wheel installation and required hashes, checked 2026-09-12.

Reproducible builds connect source to artifacts

Sun, 15 Aug 2027 00:00:00 +0000

A reproducible build lets another party recreate specified artifacts byte for byte from the stated source, build instructions, and relevant environment. The promise must name its outputs: matching object files is insufficient when the shipped executable differs. Reproduction provides evidence connecting source and distribution, while source review, provenance, and signatures address other questions. Two identical malicious builds are still malicious, and a compromised toolchain can reproduce its own unwanted behavior.

Record the inputs that affect bytes

Source revision and a compiler version are only part of the input. Dependencies, target architecture, SDK, linker, flags, file order, build paths, timestamps, locale, and packaging tools may influence the result. Pin the inputs that matter and describe the comparison boundary. A container image can help specify an environment, but using a container does not automatically remove network downloads, host mounts, or variable timestamps.

Time macros and debug information are common sources of differences. SOURCE_DATE_EPOCH supplies a stable timestamp convention to tools that support it; it does not rewrite every program’s notion of time. Compiler prefix mapping can normalize embedded source paths. A linker may independently record object paths or metadata, so compile-stage normalization must be checked against the final linked output.

When it breaks

Comparing only a selected section can conceal differences in the actual release. State whether the reproducibility claim covers an executable, its debug symbols, an archive, or a signed installer. If signing adds variable data, define and justify the boundary around the reproducible payload, and verify the signed distribution separately. Do not silently strip inconvenient bytes and continue claiming equality of the original files.

A matching hash from two runs on one host is useful evidence of local repeatability under those inputs. It is not independent verification across builders, and it does not establish the correctness of either source or tools. Conversely, a mismatch is a reason to investigate rather than automatic proof of compromise. Identify the differing component before deciding whether an input was missing from the declared environment.

Check it quickly

This fixture was tested on arm64 macOS with Apple clang 21.0.0 and its installed linker. Save it as repro-build.py and run python3 repro-build.py with the command-line developer tools installed. The linker flags are Apple-specific; do not assume a Linux linker accepts them. The script creates two temporary build directories and compiles the same small C program under three treatments.

from hashlib import sha256
from pathlib import Path
from tempfile import TemporaryDirectory
import json
import os
import subprocess

source = '#include <stdio.h>\nint main(void) { puts(__FILE__); puts(__DATE__ " " __TIME__); }\n'
with TemporaryDirectory() as directory:
 base = Path(directory)
 hashes = {mode: {'objects': [], 'executables': []}
 for mode in ('unmapped', 'compiler-only', 'complete')}
 output = []
 for name in ('one', 'two'):
 root = base / name
 root.mkdir()
 src = root / 'main.c'
 src.write_text(source)
 env = {'PATH': os.environ['PATH'], 'LC_ALL': 'C', 'SOURCE_DATE_EPOCH': '1700000000'}
 for mode in hashes:
 obj = root / (mode + '.o')
 binary = root / mode
 flags = [] if mode == 'unmapped' else [f'-ffile-prefix-map={root}=/src']
 subprocess.run(['clang', '-g', '-O0', *flags, '-c', str(src), '-o', str(obj)],
 cwd=root, env=env, check=True, capture_output=True)
 os.utime(obj, (1700000000, 1700000000))
 linker = ['-Wl,-reproducible']
 if mode == 'complete':
 linker.append(f'-Wl,-oso_prefix,{root}/')
 subprocess.run(['clang', *linker, str(obj), '-o', str(binary)],
 cwd=root, env=env, check=True, capture_output=True)
 hashes[mode]['objects'].append(sha256(obj.read_bytes()).hexdigest())
 hashes[mode]['executables'].append(sha256(binary.read_bytes()).hexdigest())
 if mode == 'complete':
 output.append(subprocess.check_output([str(binary)], text=True).splitlines())
 equality = {mode: {kind: pair[0] == pair[1] for kind, pair in records.items()}
 for mode, records in hashes.items()}
 assert equality == {'unmapped': {'objects': False, 'executables': False},
 'compiler-only': {'objects': True, 'executables': False},
 'complete': {'objects': True, 'executables': True}}
 assert output == [['/src/main.c', 'Nov 14 2023 22:13:20']] * 2
 print(json.dumps({'sha256': hashes, 'equal_between_directories': equality,
 'complete_program_output': output,
 'same_host_only': True}))

All treatments use epoch 1700000000 and fixed object modification times. Without path mapping, the object files and executables differ. With compiler mapping alone, the object hashes match but the executables still differ: the linker debug map retains the different object paths. This intermediate failure is why an object-level check cannot certify a release binary.

The complete treatment also enables the linker’s reproducible mode and removes the build-directory prefix from its object debug map. Both object and executable hashes then match between the two directories. Running the completed binaries prints /src/main.c and Nov 14 2023 22:13:20. The assertions compare whole files, including the resulting Mach-O metadata, rather than deleting differences after building.

These results cover one compiler, SDK, architecture, and simple program on one host. They do not establish that a larger package, another toolchain, or an independently signed release will reproduce. The script prints every digest and the equality decisions so the observed boundary remains explicit.

Investigate before broadening the claim

When a production comparison fails, retain both artifacts and their build manifests. Locate differences in archives, sections, and metadata, then remove the responsible uncontrolled input and rebuild. Keep the expected output set stable throughout the investigation. Once local repeatability is established, ask an independent environment to follow the published instructions and compare the declared release artifacts. That adds evidence that another party can reproduce the result, without turning reproducibility into a substitute for security review.

Sources

Reproducible Builds: definitions, specified artifacts and bit-for-bit reproduction, checked 2026-09-13.
Reproducible Builds: build path, paths embedded by compilers and debug information, checked 2026-09-13.
Reproducible Builds: SOURCE_DATE_EPOCH, a stable timestamp convention for participating tools, checked 2026-09-13.
Clang command-line reference, compiler path mapping options, checked 2026-09-13.

Related images share layers; running containers keep separate changes

Mon, 22 Mar 2027 00:00:00 +0000

An image describes an ordered filesystem assembled from immutable layer changesets, together with configuration such as its default command. Related images can reference the same base layers while adding different application files. A running container gets its own writable layer above that image. Distinguishing those three things explains how image reuse saves work without making every container’s runtime edits visible to its neighbors.

When it breaks

Summing the displayed sizes of several related images can overstate their combined local storage use because shared content may be counted repeatedly. Transfer size, unpacked filesystem size, and actual storage allocated by a snapshotter are also different measurements. Layer sharing supports reuse, but it does not justify claiming an exact disk saving without measuring the relevant storage implementation and accounting method.

The opposite mistake is treating container changes as edits to the image. Installing a package or changing a file inside one ordinary container modifies that container’s writable filesystem state. A new container from the original image does not automatically inherit the repair. An operator who patches a running service and then replaces it can therefore see the original failure return.

Sharing does not make an image layer a universal application compatibility boundary. Two applications might use equal base layers yet depend on different configuration, mounted data, architectures, or library versions introduced later. Compare the ordered filesystem identifiers and the actual behavior under test; a matching readable image tag alone says little about which bytes or settings are in use.

Check it quickly

Run this script with Python and a Docker builder after obtaining the pinned Python base digest. The recorded environment used Docker 29.3.1 on Linux arm64, with Python 3.11.16 inside the image. It creates two related images and two temporary containers, uses no container networking, and removes only its own named containers and tags when finished:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(["docker",*args],capture_output=True,text=True,timeout=120)
 if check:assert result.returncode==0,result.stderr
 return result
def inspect(image):return json.loads(docker("image","inspect",image).stdout)[0]

name='xbcatm-layers-'+uuid.uuid4().hex[:10];tags=[name+':left',name+':right'];containers=[name+'-one',name+'-two']
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory);(root/'shared.py').write_text('value=10\n# '+name+'\n')
 (root/'Dockerfile').write_text(f'FROM {BASE}\nCOPY shared.py /app/shared.py\nCOPY main.py /app/main.py\nCMD ["python", "/app/main.py"]\n')
 for tag,increment in zip(tags,[1,2]):
 (root/'main.py').write_text(f'from shared import value\nprint(value+{increment})\n')
 docker('build','--network','none','-t',tag,str(root))
 left=inspect(tags[0])['RootFS']['Layers'];right=inspect(tags[1])['RootFS']['Layers']
 assert left[:-1]==right[:-1] and left[-1]!=right[-1]
 assert docker('run','--rm','--network','none',tags[0]).stdout.strip()=='11'
 assert docker('run','--rm','--network','none',tags[1]).stdout.strip()=='12'
 print('two images share',len(left)-1,'filesystem layer identities and differ in the final application layer')
 for container in containers:
 docker('run','-d','--name',container,'--network','none',tags[0],'python','-c','import time;time.sleep(60)')
 docker('exec',containers[0],'python','-c',"from pathlib import Path;Path('/app/shared.py').write_text('value=100\\n')")
 assert docker('exec',containers[0],'python','/app/main.py').stdout.strip()=='101'
 assert docker('exec',containers[1],'python','/app/main.py').stdout.strip()=='11'
 assert '/app/shared.py' in docker('diff',containers[0]).stdout
 assert inspect(tags[0])['RootFS']['Layers']==left
 assert docker('run','--rm','--network','none',tags[0]).stdout.strip()=='11'
 print('a write in one container changes its writable layer while the sibling container and original image retain the old code')
finally:
 docker('rm','-f',*containers,check=False);docker('image','rm',*tags,check=False)

Both builds copy the same shared.py, which defines a value of ten. Their final main.py differs: one adds one and the other adds two. The resulting programs print 11 and 12. Image inspection shows an identical ordered prefix of five filesystem layers and a different final application layer in this recorded base and build arrangement.

The five-layer count is an observation, not a requirement for every Python image. The meaningful assertion compares all layers before the final COPY and confirms that the last identifiers differ. If the publisher changes the base structure, a new digest may have another number of layers while the same sharing principle continues to apply.

Next, two long-running containers start from the first image. A command changes /app/shared.py in only the first container to define one hundred. Executing its application now prints 101. Executing the sibling’s application still prints 11, and docker diff reports the changed path in the edited container’s writable state.

The original image retains its layer identifiers. A fresh container created from that image also prints 11, proving that the runtime edit was not written back into the image. The experiment uses ordinary image-backed paths with no shared volume or host bind mount. A shared writable mount would have a different visibility contract and must be examined separately.

The inspected RootFS.Layers list identifies uncompressed filesystem changesets through DiffIDs. Those values should not be casually substituted for registry manifest digests, compressed layer-blob digests, or image configuration digests. All are content identifiers, but they identify different serialized objects and serve different purposes in storage, retrieval, and artifact selection.

Build and replace the intended artifact

Keep common dependency inputs stable when related applications can actually share them, then copy application-specific code later. This improves the opportunity for cache and content reuse without requiring that unrelated applications use an artificial common dependency set. Correct dependency boundaries come from the software, not from maximizing the number of matching hash strings.

Turn a necessary runtime repair into a reviewed image change and test a replacement container. Persist durable application data through an explicit storage contract; do not rely on the writable layer surviving replacement. Conversely, do not assume deleting one container removes data held in an independently managed volume.

For operational accounting, measure shared and unique storage with tools appropriate to the builder and image store. For release identity, record the artifact reference and runtime configuration. For debugging, inspect both the image and container differences. These observations answer related questions, but no single image-size column or tag answers all of them.

Sources

Docker image layers, shared read-only layers and container writable layers, checked 2026-09-12.
OCI image configuration 1.1.1, ordered filesystem DiffIDs and their distinction from other digests, checked 2026-09-12.
Optimize Docker cache usage, dependency ordering and narrow build inputs, checked 2026-09-12.

Multi-stage builds need a tested runtime boundary

Fri, 25 Sep 2026 00:00:00 +0000

A multistage Dockerfile can compile software in one stage and copy selected outputs into another stage that becomes the runtime image. The final stage inherits its own base and the files explicitly copied into it, rather than automatically inheriting the builder’s compiler, source tree, and package caches. This creates a useful boundary, but the author must still identify everything the program needs when it executes.

When it breaks

COPY can successfully place a dynamically linked executable into a scratch image even when its ELF interpreter and shared libraries are missing. The image build may finish normally because copying does not execute the program or validate its entire runtime dependency graph. Startup can then report no such file or directory despite the executable itself being present at the configured path.

Scripts need their interpreters, applications may need certificate roots or timezone data, and programs can load plugins or assets only on particular request paths. A simple launch test might miss those later requirements. Selecting a small runtime base is useful only when it provides the actual contract the application relies on, including supported architecture and necessary filesystem paths.

The earlier stage is also not securely erased just because it is absent from the final image’s ancestry. Builder caches and explicitly exported intermediate images can retain its contents. Sensitive build inputs require appropriate secret handling, and copied outputs must be checked for accidental inclusion. Multistage structure controls final filesystem composition; it is not an automatic confidentiality guarantee.

Check it quickly

Run this script with Python, Docker, the pinned Python base, and network access for the builder’s Debian package installation. Runtime containers disable networking. The recorded run used Docker 29.3.1 on Linux arm64 and GCC 12.2.0 from Debian’s 12.2.0-14+deb12u1 package. The package repository is live, so the exercise demonstrates runtime composition rather than promising identical compiler bytes forever:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,tarfile,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
 result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=360)
 if check:assert result.returncode==0,result.stderr
 return result
def inspect(image):return json.loads(docker('image','inspect',image).stdout)[0]

name='xbcatm-stages-'+uuid.uuid4().hex[:10];tags=[name+':builder',name+':good',name+':bad'];container=name+'-inspect'
try:
 with TemporaryDirectory(prefix=name+'-') as directory:
 root=Path(directory)
 (root/'square.c').write_text('#include <stdio.h>\n#include <stdlib.h>\n#include <errno.h>\nint main(int argc,char **argv){\n char *end;long value;\n if(argc!=2){fprintf(stderr,"one integer required\\n");return 2;}\n errno=0;value=strtol(argv[1],&end,10);\n if(errno||end==argv[1]||*end||value<0||value>100){fprintf(stderr,"range 0..100\\n");return 2;}\n printf("%ld\\n",value*value);return 0;\n}\n')
 (root/'Dockerfile').write_text(f'''FROM {BASE} AS builder
RUN apt-get update && apt-get install -y --no-install-recommends gcc libc6-dev && rm -rf /var/lib/apt/lists/*
WORKDIR /source
COPY square.c .
RUN gcc -O2 -Wall -Wextra -Werror -static -o /square-static square.c && gcc -O2 -Wall -Wextra -Werror -o /square-dynamic square.c
FROM scratch AS good
COPY --from=builder /square-static /square
USER 65532:65532
ENTRYPOINT ["/square"]
FROM scratch AS bad
COPY --from=builder /square-dynamic /square
USER 65532:65532
ENTRYPOINT ["/square"]
''')
 for tag,target in zip(tags,['builder','good','bad']):docker('build','--target',target,'-t',tag,str(root))
 print('compiler:',docker('run','--rm','--network','none',tags[0],'gcc','--version').stdout.splitlines()[0])
 for argument,expected in [('0','0'),('7','49'),('100','10000')]:
 assert docker('run','--rm','--network','none',tags[1],argument).stdout.strip()==expected
 for argument in ['7x','-1','101','','999999999999999999999999999']:
 invalid=docker('run','--rm','--network','none',tags[1],argument,check=False)
 assert invalid.returncode==2 and 'range 0..100' in invalid.stderr
 invalid=docker('run','--rm','--network','none',tags[1],check=False)
 assert invalid.returncode==2 and 'one integer required' in invalid.stderr
 invalid=docker('run','--rm','--network','none',tags[1],'7','8',check=False)
 assert invalid.returncode==2 and 'one integer required' in invalid.stderr
 good=inspect(tags[1]);builder=inspect(tags[0]);assert good['Size']<builder['Size'] and len(good['RootFS']['Layers'])==1
 assert good['Config']['User']=='65532:65532'
 docker('create','--name',container,tags[1],'7');archive=root/'rootfs.tar';docker('export','-o',str(archive),container)
 with tarfile.open(archive) as contents:
 names={entry.name.removeprefix('./') for entry in contents}
 assert 'square' in names and 'usr/bin/gcc' not in names and 'source/square.c' not in names
 bad=docker('run','--rm','--network','none',tags[2],'7',check=False)
 assert bad.returncode!=0 and 'no such file or directory' in bad.stderr
 print('the static final image runs valid input, rejects invalid input, and contains one application layer without compiler or source')
 print('recorded image sizes:',builder['Size'],'builder bytes;',good['Size'],'final bytes')
 print('the dynamic scratch image builds successfully but cannot start because its interpreter/runtime was not copied')
finally:
 docker('rm','-f',container,check=False);docker('image','rm',*tags,check=False)

The builder compiles the same small C program twice, once statically and once dynamically. The program accepts one integer in the range zero through one hundred and prints its square. Compilation enables warnings and treats them as errors. Named final targets copy only the corresponding executable into scratch and select numeric user and group 65532.

The static final image prints 0, 49, and 10000 for inputs zero, seven, and one hundred. It returns status two for trailing text, negative or out-of-range values, an empty argument, and an integer too large for strtol. Missing or extra arguments also return status two. These checks exercise valid boundaries and rejected inputs in the actual final image.

Inspection confirms one filesystem layer in that final image and the intended nonroot numeric user. Exporting a created container shows /square, with no compiler or source file at the builder’s paths. The export is inspected as an archive; it is not extracted over the host filesystem. This verifies final composition independently of whether the Dockerfile merely looks minimal.

Docker reported 109,490,331 bytes for the builder image and 312,508 bytes for the static final image in the recorded run. These are the engine’s image-size fields, not a measurement of unique physical disk allocation or network transfer time. The useful invariant is that the final image excludes the toolchain and is smaller under the same reported accounting method.

The dynamic final target also builds successfully, but starting it fails with a missing-file error because its interpreter/runtime was not copied. This is the critical negative test. The failure is expected and asserted; it disproves the idea that a successful final COPY automatically tells the builder which runtime libraries should be added.

Validate what the last stage actually runs

Choose an appropriate runtime image for the application. Static linking fits this small calculation, but does not prove that every statically linked service needs no external files or runtime-loaded components. For a dynamic application, supply compatible runtime libraries or a suitable runtime base and execute representative functionality inside the final artifact.

Use named stages and narrow COPY paths so reviewers can see the handoff. Avoid copying a broad builder directory that silently includes tests, credentials, caches, or unused tools. Set working directories, writable paths, user identity, and startup arguments according to the runtime’s real requirements.

Test the final stage after dependency or compiler updates, including errors and less common paths that load extra resources. Keep the selected artifact identity with the results. A smaller image is a useful outcome when it still fulfills the service contract; build success alone cannot establish that contract.

Sources

Docker multi-stage builds, named stages, target selection, and copying selected artifacts, checked 2026-09-12.
Dockerfile reference, COPY –from, scratch, USER, and exec-form entrypoints, checked 2026-09-12.
execve(2), missing executable interpreters and execution errors, checked 2026-09-12.