Skip to content

Bug: _save_narrative does not fsync before rename β€” narrative loss on crashΒ #140

@tcconnally

Description

@tcconnally

Severity: 🟠 High (durability gap; narrative loss on crash)

_save_narrative in src/perseus/mneme_narrative.py:60–67:

def _save_narrative(path, frontmatter, body):
    path.parent.mkdir(parents=True, exist_ok=True)
    fm_yaml = yaml.safe_dump(frontmatter, ...).strip()
    payload = f"---\n{fm_yaml}\n---\n\n{body.rstrip()}\n"
    tmp = path.with_suffix(path.suffix + ".tmp")
    tmp.write_text(payload, encoding="utf-8")
    os.replace(tmp, path)

tmp.write_text returns after the write enters the page cache but BEFORE fsync. On macOS specifically, os.replace immediately afterwards can leave the new inode with zero bytes if the kernel crashes or is force-rebooted before flush.

cache_set in renderer.py:178–187 does this correctly with tmp.flush() + os.fsync(tmp.fileno()). The pattern is inconsistent across the codebase.

Suggested fix

def _save_narrative(path, frontmatter, body):
    path.parent.mkdir(parents=True, exist_ok=True)
    fm_yaml = yaml.safe_dump(frontmatter, default_flow_style=False,
                             allow_unicode=True, sort_keys=False).strip()
    payload = f"---\n{fm_yaml}\n---\n\n{body.rstrip()}\n"
    tmp = path.with_suffix(path.suffix + ".tmp")
    try:
        with open(tmp, "w", encoding="utf-8") as f:
            f.write(payload)
            f.flush()
            os.fsync(f.fileno())
        os.replace(tmp, path)
    finally:
        try:
            tmp.unlink(missing_ok=True)
        except OSError:
            pass

Extract a helper safe_atomic_write(path, payload) and use it consistently across the codebase (audit _save_task_file, frontmatter writes, etc.).

Acceptance criteria

  • Audit all "tmp + rename" sites for fsync consistency.
  • Mock os.fsync and assert it is called for _save_narrative.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions