Skip to content

fix: test suite clobbers live ~/.pyauto-heart state #78

Description

@Jammy2211

Overview

Running PyAutoHeart's own unit tests overwrites the developer's live ~/.pyauto-heart state: tests/test_test_run.py isolates the input (results_dir=tmp_path) but heart/checks/test_run.py resolves HEART_STATE_DIR at import and writes test_run.json unconditionally inside run(). Measured 2026-07-15: a 10,234-byte real state file was replaced by a 251-byte fixture, wiping the recorded failures and the 58-entry parked_stale list — recoverable only by luck. Heart's state dir is the input to the release gate, so this is a release-safety issue. Phase 0 of the build-chain integrity campaign (epic: see PyAutoBuild umbrella issue).

Plan

  • Remove the hidden global write: run(results_dir=...) returns the summary; only the CLI/tick entrypoint persists it (delete the trap, not just the one test).
  • Sweep heart/checks/ empirically for sibling module-level-resolved writes reachable from tests.
  • Add an autouse conftest.py fixture sandboxing HEART_STATE_DIR for the whole suite (belt-and-braces; deliberate decision per the prompt).
  • Verify: full suite with HEART_STATE_DIR unset against a throwaway HOME; diff the state dir before/after — measured, not argued.
Detailed implementation plan

Work Classification

Library (organism repo: PyAutoHeart)

Affected Repositories

  • PyAutoHeart (primary, only)

Branch Survey (2026-07-16)

Repository Current Branch Dirty?
./PyAutoHeart main clean

Suggested branch: feature/heart-state-clobber
Worktree root: ~/Code/PyAutoLabs-wt/heart-state-clobber/ (created by /start_library)

Implementation Steps

  1. heart/checks/test_run.py — move the state.atomic_write_json(HEART_STATE_DIR / "test_run.json", summary) call (currently ~line 231, unconditional inside run()) out of run(); run() returns the summary dict only. The CLI/tick entrypoint that invokes the check performs the persist.
  2. Audit every module in heart/checks/ for the same shape: HEART_STATE_DIR (or any global path) resolved at import + written from library code that tests call directly. Empirical proof: run each test file with HEART_STATE_DIR pointed at a sandbox and list what appears there.
  3. tests/conftest.py — autouse fixture setting HEART_STATE_DIR to tmp_path-derived sandbox for the whole suite (note: env var is read at import in test_run.py:36, so the fixture must account for import-time resolution — monkeypatching the module attribute, not just the env var, or moving resolution to call time as part of step 1).
  4. Update tests/test_test_run.py for the new run() contract; add a regression test asserting no test writes outside the sandbox.
  5. Verification (from the prompt's constraints): run the full suite with HEART_STATE_DIR unset against a throwaway HOME, diff the state dir before/after — must be byte-identical.

Key Files

  • heart/checks/test_run.pyHEART_STATE_DIR resolved at import (:36); unconditional write in run() (:231)
  • tests/test_test_run.py — calls tr.run(results_dir=tmp_path) at lines 48, 61, 69, 101, 110
  • tests/conftest.py — autouse sandbox fixture (new)
  • heart/state.pyatomic_write_json helper; reads of test_run.json

Constraints

  • Heart state dir is release-gate input: verify by measurement, not argument.
  • Do not run the suite against live ~/.pyauto-heart until the fix is in (that is how the bug was found).

Original Prompt

Click to expand starting prompt

PyAutoHeart's test suite writes into the user's live ~/.pyauto-heart state

Type: bug
Target: PyAutoHeart
Repos:

  • PyAutoHeart
    Difficulty: small
    Autonomy: supervised
    Priority: high
    Status: formalised

Running PyAutoHeart's own unit tests overwrites the developer's live Heart state.
tests/test_test_run.py calls tr.run(results_dir=tmp_path) (lines 48, 61, 69, 101, 110) and
never isolates HEART_STATE_DIR. heart/checks/test_run.py:36 resolves HEART_STATE_DIR at
import time (env or ~/.pyauto-heart default), and :231 unconditionally does
state.atomic_write_json(HEART_STATE_DIR / "test_run.json", summary). So tmp_path isolates the
input (results_dir) but not the output: the summary lands in the real state dir.

Measured 2026-07-15 (on main, not a feature branch). Running pytest tests/test_test_run.py
replaced a 10,234-byte real test_run.json with this 251-byte fixture — the last test to run wins:

{"cloud_url": "U", "failed": 0, "parked_stale": [], "parked_stale_count": 0,
 "passed": 0, "per_project": {}, "ready": true, "run_label": "cloud#9",
 "skipped": 0, "source": "cloud", "timeout": 0, "ts": "2026-06-25T00:00:00Z"}

Proven by pointing HEART_STATE_DIR at a sandbox and re-running that file alone: the same stub
appears there. With the variable unset — the normal case for anyone running pytest in this repo —
the target is ~/.pyauto-heart/.

Impact, measured not reasoned. Two of the three legs currently holding Heart at YELLOW read
from test_run. Re-aggregating with the stub in place:

before:  yellow score 60  — "workspace validation not passing (3 failed, 2026-07-09T09-48-30Z)"
                          — "58 stale parked script(s)"
after:   stale  score 70  — "test run stale (20d old)"

Both real reasons disappear, replaced by a bogus one. It is not a fake-GREEN today — the
fixture's hardcoded ts (2026-06-25) reads as 20d old, past TEST_STALE_DAYS=10, so the verdict
degrades to STALE and still blocks the GREEN-gated nightly. That is luck, not design: a fixture
with a fresh ts would clear the leg outright. Do not treat "it only goes STALE" as the safety
property.

The real damage is evidence destruction. The stub wipes the 3 recorded failures and the entire
58-entry parked_stale list — which is exactly the hygiene-triage input for the other open YELLOW
leg. On 2026-07-15 this was recoverable only because ~/.pyauto-heart/state.json still held a
pre-clobber copy of the block (restored from it, back to a byte-identical 10,234 bytes). Had a
tick re-aggregated between the pytest run and the restore, state.json would have been
overwritten too and the data would be gone permanently.

Fix direction (design first — do not assume)

The obvious monkeypatch.setenv("HEART_STATE_DIR", tmp_path) in the test only fixes this test
file. Prefer removing the trap over documenting it:

  • Is the write in tr.run() the actual defect? A run(results_dir=...) that also performs a
    hidden write to a global path is doing two jobs. Consider having run() return the summary and
    letting the CLI entrypoint persist it — then the tests cannot pollute anything because there is
    nothing to pollute.
  • Sweep for siblings. heart/checks/ resolves HEART_STATE_DIR at import in several modules;
    find every check whose test path can reach a module-level-resolved write. test_run was found by
    accident, so assume it is not alone. git status-style proof: run each test file with
    HEART_STATE_DIR sandboxed and list what appears.
  • An autouse fixture in tests/conftest.py pointing HEART_STATE_DIR at tmp_path for the
    whole suite is a cheap belt-and-braces backstop, but it hides the design smell rather than fixing
    it — decide deliberately whether to do both.

Constraints

  • Heart's state dir is the input to the release gate. Anything that can silently rewrite it
    with passing values is a release-safety issue, not a tidiness issue — hence Priority: high.
  • Verify any fix by running the full suite with HEART_STATE_DIR unset against a throwaway
    HOME, and diffing the state dir before/after. "No test writes it" must be measured, not argued.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions