Log rotation divides a continuing stream into files that can be retained, searched, compressed, and eventually removed. A useful policy defines both when to rotate and which history to keep. It must also account for how the application holds its output file open. Renaming a pathname and successfully redirecting future log records are separate steps in that operational contract.

When it breaks

A writer usually keeps an open descriptor rather than resolving the log pathname for every record. Renaming the file and creating a new file at the old name does not move that descriptor. The active pathname can look empty while the archive keeps growing. Monitoring only the new name may falsely suggest that the application stopped logging.

Removing an archive name also does not close a writer that still references the inode. The file can remain open, consume resources, and accept further writes without a normal directory entry. A cleanup job that reports successful deletion therefore has not necessarily recovered the expected space or redirected the application’s logging.

Copy-and-truncate avoids requiring the writer to reopen, but copying and truncating are separate operations. A record written after the copy and before truncation can disappear from both the archive and active file. A quiet period can reduce the chance of that interleaving; it does not turn the method into an atomic, lossless handoff.

Check it quickly

Save this as rotation.py and run python3 rotation.py in a disposable Linux environment. The recorded run used Linux 6.12.76, Python 3.11.2, and ext4. The fixture uses only temporary files and explicitly controls the order of writes. Its copytruncate case is a deterministic interleaving of the underlying operations, not a claim that a race happened accidentally:

from pathlib import Path
from tempfile import TemporaryDirectory
import os,shutil

with TemporaryDirectory(prefix='xbcatm-rotation-') as directory:
    root=Path(directory);active=root/'app.log';archive=root/'app.log.1'
    with active.open('a+b',buffering=0) as writer:
        writer.write(b'before\n');identity=os.fstat(writer.fileno()).st_ino
        active.rename(archive);active.touch();writer.write(b'after-rename\n')
        assert active.read_bytes()==b'' and archive.read_bytes()==b'before\nafter-rename\n'
        assert archive.stat().st_ino==identity and active.stat().st_ino!=identity
        print('renaming and creating a replacement path leaves an existing writer appending to the archive inode')
        archive.unlink();assert os.fstat(writer.fileno()).st_nlink==0
        writer.write(b'after-unlink\n');writer.seek(0)
        assert b'after-unlink\n' in writer.read() and not archive.exists()
        print('deleting the archive name does not close the writer; its unlinked file remains open and writable')
    with active.open('ab',buffering=0) as reopened:reopened.write(b'new-active\n')
    assert active.read_bytes()==b'new-active\n'
    print('closing and reopening the active path switches subsequent writes to the replacement inode')
    source=root/'copytruncate.log';copied=root/'copytruncate.log.1'
    with source.open('a+b',buffering=0) as writer:
        writer.write(b'before-copy\n');shutil.copyfile(source,copied)
        writer.write(b'between-copy-and-truncate\n')
        os.ftruncate(writer.fileno(),0);writer.write(b'after-truncate\n')
    assert copied.read_bytes()==b'before-copy\n' and source.read_bytes()==b'after-truncate\n'
    assert b'between-copy-and-truncate' not in copied.read_bytes()+source.read_bytes()
    print('deterministic copy-then-truncate interleaving loses the write placed in that gap')

The first writer appends before, then the controller renames the log and creates a replacement active pathname. The same open writer appends after-rename. Both records appear in the archive, and the replacement remains empty. Inode checks confirm that the writer retained the original object while the active pathname now identifies a different file.

Next, the archive is unlinked while that writer remains open. Its link count becomes zero, and another appended record is still readable through the retained reference. The archive pathname no longer exists. These observations establish why pathname deletion and the end of a writer’s file lifetime must be investigated separately during a disk-space incident.

Closing the old writer and reopening the active pathname makes a subsequent record land in the replacement file. This is the essential handoff a service’s documented reopen mechanism must perform. Sending a conventionally named signal is useful only if that application actually implements the required behavior and its result is observed.

The second case copies a log containing before-copy, writes another record into the original, then truncates the original and appends after-truncate. The archive contains only the first record; the active file contains only the last. The middle record is absent from both. This demonstrates the precise loss window described for copytruncate.

The script closes all references and removes its temporary files. It checks runtime visibility and data placement, not crash durability or a remote backup system. Those are additional properties that cannot be inferred from a successful rename, a completed compression, or an archive directory containing the expected number of names.

Budget time, growth, and retrieval

A size threshold is evaluated when the rotation tool runs; it is not a continuously enforced maximum file size. For example, a ten-MiB threshold checked every sixty seconds can permit roughly seventy MiB before the next check if writing is bounded at one MiB per second. Delayed checks or an unbounded producer invalidate that illustrative growth bound.

Account for the active file, retained generations, temporary compression or copying space, and still-open old files. Keeping four generations does not automatically preserve four days: frequent rotations can consume those generations much sooner. Choose both a capacity budget and a history window that supports the incidents the team actually needs to investigate.

Verify retrieval as well as deletion. Confirm that representative records can be found across rotated and compressed files and that the oldest required interval remains available. If history must survive host or disk loss, maintain an independently stored copy and test recovery. Local rotation organizes and limits local history; its success alone does not prove that the history remains recoverable after the machine is gone.

Further reading

Sources

  • rename(2), name replacement without redirecting existing open descriptions, checked 2026-09-12.
  • unlink(2), unlinked files retained by open references, checked 2026-09-12.
  • logrotate 3.21.0 manual, retention directives, size evaluation, and copytruncate loss window, checked 2026-09-12.