Skip to content

Bug: _read_all_pythia_entries silently drops malformed JSONL lines #148

@tcconnally

Description

@tcconnally

Severity: 🟡 Medium (observability)

mneme_narrative.py:96–114 swallows json.loads errors per line with continue, and swallows the outer file-read error returning []. The operator has no signal that entries are being dropped.

Suggested fix

def _read_all_pythia_entries() -> list[dict]:
    log_path = _pythia_log_path()
    if not log_path.exists():
        return []
    entries: list[dict] = []
    dropped = 0
    try:
        with log_path.open("r", encoding="utf-8") as f:
            for raw in f:
                line = raw.strip()
                if not line:
                    continue
                try:
                    entries.append(json.loads(line))
                except Exception:
                    dropped += 1
    except Exception as exc:
        print(f"⚠ Could not read Pythia log {log_path}: {exc}", file=sys.stderr)
        return []
    if dropped:
        print(f"⚠ Pythia log: skipped {dropped} malformed entries in {log_path}",
              file=sys.stderr)
    return entries

Acceptance criteria

  • Test: JSONL log with one good + one truncated line. Good entry returned; stderr contains skip warning.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions