Container image layers describe filesystem changes applied in order. A layer can add or replace a file, or record that an earlier path should disappear from the merged filesystem. That final view is what a normal container process sees, but it is not the complete history of bytes distributed with the image. Inspecting only the running container can therefore miss data retained in earlier layers.

When it breaks

Copying a credential into an image and deleting it in a later RUN does not remove the credential from the earlier immutable layer. Anyone who can obtain the relevant layer may recover the original bytes. The same mechanism explains why deleting a large build archive in a later instruction can leave the image’s distributed content unnecessarily large even though the final path is absent.

Layer changes are filesystem changesets, not necessarily compact binary differences. Replacing a large file can place its replacement content in another layer. Deletion uses a whiteout marker to suppress an earlier path when layers are applied. OCI also defines opaque-directory behavior for hiding lower directory contents, so implementing layer application as ordinary repeated tar extraction is insufficient.

Not every Dockerfile instruction creates a filesystem layer. ENV, LABEL, and CMD change image configuration and can add history entries marked empty_layer. Counting Dockerfile lines or history records therefore does not reliably count root filesystem layers. Configuration changes still matter to execution and artifact identity even when they add no application file bytes.

Check it quickly

Use Python with access to a Docker builder and the pinned base digest. The recorded run used Docker 29.3.1 with its containerd image store and a Linux arm64 Python 3.11.16 base. The marker is deliberately public. The script builds and saves one temporary image, inspects archive members without extracting them into the host filesystem, and removes its own tag:

from pathlib import Path
from tempfile import TemporaryDirectory
import json,subprocess,tarfile,uuid

BASE='python@sha256:528257d48c1da0dcecc2e725d1ae34498d60c965f1241e39cd6a85a8859bdf84'
def docker(*args,check=True):
    result=subprocess.run(["docker",*args],capture_output=True,text=True,timeout=120)
    if check:assert result.returncode==0,result.stderr
    return result
def inspect(image):return json.loads(docker("image","inspect",image).stdout)[0]

name='xbcatm-whiteout-'+uuid.uuid4().hex[:10];tag=name+':probe';marker=b'PUBLIC_DEMONSTRATION_BYTES\n'
try:
    with TemporaryDirectory(prefix=name+'-') as directory:
        root=Path(directory);(root/'original.txt').write_bytes(marker)
        (root/'Dockerfile').write_text(f'FROM {BASE}\nCOPY original.txt /demo/original.txt\nRUN rm /demo/original.txt\nENV EXAMPLE_MODE=demo\nLABEL example.purpose="public fixture"\nCMD ["python", "-c", "from pathlib import Path;assert not Path(\'/demo/original.txt\').exists();print(\'absent\')"]\n')
        docker('build','--network','none','-t',tag,str(root))
        assert docker('run','--rm','--network','none',tag).stdout.strip()=='absent'
        archive=root/'image.tar';docker('image','save','-o',str(archive),tag)
        with tarfile.open(archive) as outer:
            manifests=json.load(outer.extractfile('manifest.json'));assert len(manifests)==1
            manifest=manifests[0];config=json.load(outer.extractfile(manifest['Config']))
            assert len(manifest['Layers'])==len(config['rootfs']['diff_ids'])
            assert len(manifest['Layers'])==len(inspect(BASE)['RootFS']['Layers'])+2
            found_data=False;found_whiteout=False
            for layer in manifest['Layers'][-2:]:
                with tarfile.open(fileobj=outer.extractfile(layer),mode='r:*') as contents:
                    for entry in contents:
                        path=entry.name.removeprefix('./')
                        if path=='demo/original.txt':found_data=contents.extractfile(entry).read()==marker
                        if path=='demo/.wh.original.txt':found_whiteout=True
            assert found_data and found_whiteout
            assert all(item.get('empty_layer') for item in config['history'][-3:])
        print('the final container hides the removed file, but the older layer still contains the exact public marker')
        print('the deletion layer contains a whiteout; ENV, LABEL and CMD add metadata history without filesystem layers')
finally:docker('image','rm',tag,check=False)

The Dockerfile copies original.txt into /demo, then removes it in a later RUN. The configured container command asserts the path is absent and prints absent. That confirms the merged runtime view. It does not establish that the bytes are absent from the image archive, so the next part examines the saved representation separately.

The archive’s manifest.json identifies its configuration and ordered layer files. The script reads the configuration’s rootfs.diff_ids and checks that the number of filesystem layers matches the manifest. It then inspects the final two layer archives, which correspond to the COPY addition and the subsequent deletion in this particular build.

The addition layer still contains the exact PUBLIC_DEMONSTRATION_BYTES marker. The deletion layer contains demo/.wh.original.txt. That whiteout instructs layer application to hide the lower file; it does not edit the original layer in place. Both facts are asserted directly, so an absent file in the container cannot be mistaken for evidence of secure erasure.

The final three configuration-history entries describe ENV, LABEL, and CMD and are marked as having no filesystem layer. The image has two additional filesystem layers above its base, not five. This is a concrete counterexample to the claim that each Dockerfile instruction necessarily appends another filesystem diff.

Archive representation and runtime storage are different interfaces. The experiment inspects the image-save format produced by the recorded Docker version and OCI changeset semantics. It does not assume that a snapshotter must store its internal deletion markers as the same visible .wh. filenames, nor does it measure how a particular host allocates physical blocks.

Exclude unnecessary bytes before they enter an image

Use an appropriate build context and ignore rules to keep unrelated files out of COPY inputs. Pass build credentials through the builder’s secret mechanism, and ensure commands do not write those credentials into retained outputs. If a real credential entered a published layer, rotate it and handle retained copies; a later deletion layer is not remediation.

Create and remove disposable package indexes or temporary archives within the same build operation when they are needed only during that operation. A separate final stage can copy the required runtime artifacts without inheriting the builder’s filesystem layers. Still inspect exported caches and copied outputs, because earlier build material can remain in the builder or escape through another export.

Judge the result by the actual final artifact and its intended runtime dependencies. Fewer layers alone does not guarantee smaller downloads, faster starts, or safer content. The useful question is which bytes are distributed, which paths become visible, and whether every retained file has a justified role.

Sources