Skip to content

Add chain-aware native rotation, compression, and retention for audit/attestation logs #123

Description

@se-jo-ma

Context

The append-only audit log is durably fsync'd per write (nautilus/audit/logger.py _flush_sink) but grows without bound. Rotation is delegated entirely to external tooling: docs/how-to/operator-guide.md:174-181 tells operators to "rotate and archive it like any log (e.g. logrotate with copytruncate disabled)". There are no rotation, retention, compression, or size-cap fields anywhere in code (grep for rotat|retention|max_size|maxBytes|RotatingFile|backupCount|gzip over nautilus/audit/ and config/models.py returns nothing).

Unbounded growth is observable in-repo today: audit.jsonl is 1.3MB and nautilus-audit.jsonl is 612KB with no management.

External logrotate is also unsafe for the chained attestation sink. FileSinkSpec (config/models.py:155-170) supports chained: true (hash-linked prev_sha256 + EdDSA JWS via fathom.chained_log.ChainedAttestationLog) and checkpoint_interval, verified offline by nautilus attestation verify (nautilus/cli/attestation.py). A naive external move/truncate breaks the hash chain so verify_chain (used at nautilus/ui/router.py:106) can no longer chain across the cut.

Note: two distinct files are involved.

  • The plain audit JSONL — AuditConfig.path (config/models.py:233-236), built as AuditLogger(sink=FileSink(path=audit_path)) at core/broker.py:577. No chain; rotation here is "simple".
  • The chained attestation log — FileSinkSpec (config/models.py:155-170). Rotation here MUST be chain-aware.

Task

Add opt-in native rotation (off by default; default behavior unchanged when unconfigured).

  • Add a rotation spec (e.g. max_size_mb and/or max_age + compress: gzip + keep: N) to AuditConfig for the plain audit log, and to FileSinkSpec for the attestation log.
  • Roll the active file when the size/age threshold is hit; optionally gzip the rotated segment; prune retained segments to keep. Account for Fathom FileSink opening/closing per write — rotation must check the threshold and re-open the active path consistently.
  • For chained: true sinks, write a closing checkpoint/anchor record at the rotation boundary so offline nautilus attestation verify can chain across rotated segments.
  • Update the operator guide to document the supported rotation modes and the chained-log interaction, replacing the bare logrotate advice.

Where

  • nautilus/audit/logger.py (rotation hook around _flush_sink / emit)
  • nautilus/core/broker.py (AuditLogger/sink construction, ~line 577; chained-sink wiring ~line 737)
  • nautilus/config/models.py (AuditConfig line 233; FileSinkSpec line 155)
  • nautilus/cli/attestation.py (ensure verify chains across segments)
  • docs/how-to/operator-guide.md:174-181

Acceptance criteria

  • With rotation configured, the active audit file rolls at the size/age threshold.
  • Rotated segments are optionally gzipped and pruned down to keep.
  • For a chained attestation sink, nautilus attestation verify passes across a rotation boundary (closing anchor/checkpoint written at the cut).
  • Rotation off (unconfigured) leaves current single-file behavior byte-for-byte unchanged.
  • Unit tests cover size/age trigger, gzip, and pruning; an integration test covers chained verification across a rotation boundary.
  • Operator guide documents rotation modes and chained-log interaction; bare logrotate advice removed/updated.

Size: M

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priority: v2 epic or non-blocking gaparea/auditAudit log, attestations, JWSarea/coreBroker core, CLIPS routing, session statearea/docsDocumentation siteenhancementNew feature or requestsize/M<2 days: multi-file feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions