Pinning a container base by digest fixes the referenced image object instead of relying on a movable tag. That removes one source of unexpected build changes, but it does not make every derived image or running instance identical. Application source, build arguments, fetched dependencies, target platform, runtime configuration, and persistent data can still change what gets built and how it behaves.

When it breaks

Two Dockerfiles can use the same FROM digest and produce different applications. Even the same Dockerfile can produce different output when a build argument changes or an unpinned download resolves new content. Declaring the base immutable therefore does not prove that the complete build is deterministic. The full set of inputs and the transformations applied to them determine that property.

Running the exact same application image with different environment values can select different endpoints, feature behavior, or storage. A writable bind mount can also replace files that were tested inside the image. Keeping configuration outside the image is often useful, but the configuration still needs versioning and review when it contributes to the release’s intended behavior.

An ordinary container also has a writable filesystem layer. An emergency edit can change one instance while leaving the image and other instances untouched. Replacement from the unchanged image then loses the edit. This is why immutable deployment practice combines artifact replacement with controlled runtime state; naming a base digest alone does not enforce that operational discipline.

Check it quickly

Run the script using host Python and a Docker builder with the pinned base available. It was executed with Docker 29.3.1 and Linux arm64 Python 3.11.16. Builds and containers disable networking, so remote package changes cannot explain the results. The fixture removes its temporary containers and tags and never edits an existing service:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(['docker',*args],capture_output=True,text=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def inspect(image):return json.loads(docker('image','inspect',image).stdout)[0]

name='xbcatm-pinning-'+uuid.uuid4().hex[:10];tags=[name+':one',name+':two'];container=name+'-running'
try:
    with TemporaryDirectory(prefix=name+'-') as directory:
        root=Path(directory)
        (root/'Dockerfile').write_text(f'FROM {BASE}\nARG MESSAGE\nRUN printf "%s" "$MESSAGE" > /built-message\nENV MODE=default\nCOPY app.py /app.py\nCMD ["python", "/app.py"]\n')
        (root/'app.py').write_text('from pathlib import Path\nimport os\nprint(Path("/built-message").read_text(),os.environ["MODE"])\n')
        layers=inspect(BASE)['RootFS']['Layers']
        for tag,message in zip(tags,['one','two']):
            docker('build','--network','none','--build-arg','MESSAGE='+message,'-t',tag,str(root))
            assert inspect(tag)['RootFS']['Layers'][:len(layers)]==layers
        assert inspect(tags[0])['Id']!=inspect(tags[1])['Id']
        assert docker('run','--rm','--network','none',tags[0]).stdout.strip()=='one default'
        assert docker('run','--rm','--network','none',tags[1]).stdout.strip()=='two default'
        assert docker('run','--rm','--network','none','-e','MODE=changed',tags[0]).stdout.strip()=='one changed'
        print('both builds use the same pinned base layers; different build arguments produce different artifacts')
        print('the identical application image behaves differently when runtime configuration changes')
        docker('run','-d','--name',container,'--network','none',tags[0],'python','-c','import time;time.sleep(60)')
        docker('exec',container,'python','-c','from pathlib import Path;Path("/built-message").write_text("edited")')
        assert docker('exec',container,'python','/app.py').stdout.strip()=='edited default'
        assert docker('run','--rm','--network','none',tags[0]).stdout.strip()=='one default'
        failed=docker('run','--rm','--network','none','--read-only',tags[0],'python','-c','from pathlib import Path;Path("/built-message").write_text("edited")',check=False)
        assert failed.returncode!=0 and 'Read-only file system' in failed.stderr
        print('a writable container can drift while its image stays unchanged; a read-only root rejects that particular write')
finally:
    docker('rm','-f',container,check=False);docker('image','rm',*tags,check=False)

Both builds use the same base reference and application source. The only selected build difference is MESSAGE, which the Dockerfile writes to /built-message. One image prints one default; the other prints two default. Their complete image identities differ, while inspection confirms that their ordered base filesystem-layer prefix is identical.

That pair establishes a bounded claim: the base remained fixed, but the derived artifact changed because another explicit input changed. It does not attempt to prove bit-for-bit reproducibility of repeated builds with every input held constant. Such a test would also need to control timestamps, dependency retrieval, toolchain behavior, and metadata emitted by the builder.

Next, the first application image runs with MODE overridden to changed. It prints one changed without rebuilding the image. This separates artifact identity from release configuration. A registry digest can identify application bytes, but it cannot by itself tell an incident investigator which environment values, credentials, mounted configuration, or external services the process actually used.

The script then starts a long-running container from the first image and edits its message file. Running the application inside that container prints edited default, while a fresh container from the unchanged image still prints one default. The repair exists only in the modified container’s writable state and will not automatically propagate into replacements.

A final run enables a read-only root filesystem and attempts the same file edit. It fails with a read-only filesystem error. This demonstrates a useful enforcement mechanism for that image-backed path, not a guarantee that every possible runtime state is frozen. Explicit writable mounts, external databases, environment settings, and the host kernel remain separate parts of the execution environment.

Record a release, then replace its instances

Keep a release record connecting the tested application digest, target platform, configuration revision, and required data compatibility. An OCI index can contain manifests for several platforms, so pinning the index still allows the runtime to select a platform-specific image. Record the selection when architecture affects reproducibility or diagnosis.

Build required changes into a new artifact, validate it, and replace instances through the deployment mechanism. Put durable data in storage with its own lifecycle and recovery plan. Avoid treating the container’s writable layer as the only durable copy, and avoid silently changing shared configuration underneath instances that supposedly belong to one fixed release.

Digest pinning also means new base fixes do not arrive automatically. Track available updates, review a changed digest, rebuild dependent applications, and test before promotion. Retaining a known input is valuable, but retaining an obsolete input forever is not the same as maintaining a secure release.

When investigating drift, compare artifact, configuration, mounts, and runtime changes separately. Each has a different owner and evidence trail. Replacing ad hoc instance edits with reviewed releases makes those differences explainable, while acknowledging that identical image bytes do not imply identical external state or identical behavior under every workload.

Further reading

Sources