validwhile

A stored fact carries the condition that would make it false, as code you can re-run.

Your CLAUDE.md says "authentication uses JWT bearer tokens." Forty commits ago someone replaced JWTs with session cookies. Nobody edited the CLAUDE.md, so it has been lying to your coding agent ever since, confidently, in every prompt.

validwhile stores that sentence next to a machine-checkable condition, valid while src/auth/** is unchanged since a0eae80, and re-evaluates it on demand, offline, in about the time a git diff takes. No contradicting fact has to arrive and no model is asked. When the condition breaks the claim is marked stale and validwhile check exits 1.

git clone https://github.com/jimmyjames177414/validwhile && cd validwhile
uv run examples/stale_claude_md/demo.py

That demo builds a real git repository in a temp directory, commits a fictional CLAUDE.md, then commits a change under src/auth/ and never touches the CLAUDE.md again. No API key, no network, no model, ever.

The same output as text, so you can copy it, grep it, and read it with a screen reader. Commit shas, ids and dates differ on every run; nothing else does.

==============================================================================
2. Record the claim, bound to the code it rests on
==============================================================================
$ validwhile init
initialised validwhile store at /tmp/validwhile-demo-b3j4r8m_/.validwhile
next: validwhile add "<claim>" --evidence '<path glob>'
$ validwhile add 'Authentication uses JWT bearer tokens' --evidence 'src/auth/**' --while git_paths_unchanged --source CLAUDE.md:8 --confidence 0.95
01M1J0DP5FE142XE4TMQD5AKRW  verified  Authentication uses JWT bearer tokens
    why      src/auth/** unchanged since 80512d5

==============================================================================
3. Nothing has changed yet, so the claim still holds
==============================================================================
$ validwhile check
VALID (1)

1 memory checked, none need attention.
$ echo $?
0

==============================================================================
4. Someone rips out JWTs and ships session cookies
==============================================================================
9199d65 Replace JWTs with server-side sessions
80512d5 Larkspur: JWT auth, plus notes for the agent

==============================================================================
5. Nobody edited CLAUDE.md. validwhile notices anyway
==============================================================================
$ validwhile check
STALE   (1)
  "Authentication uses JWT bearer tokens"
    why      src/auth/** changed after this was verified
    changed  src/auth/middleware.js, src/auth/session_store.js
    since    80512d5 (2026-09-02) -> HEAD 9199d65
    source   CLAUDE.md:8
    id       01M1J0DP5FE142XE4TMQD5AKRW
    action   validwhile revalidate 01M1J0DP   # after re-reading it

1 of 1 memory needs attention.
$ echo $?
1

Nobody told the tool anything. No contradicting fact arrived, no episode was narrated, no model was asked whether the sentence was still true. The world changed and the check noticed.

This is not a vector store and it is not a Mem0 replacement. It stores no embeddings, does no retrieval, and calls no model. It sits beside whatever memory you already have and answers one question: is this particular fact still safe to put in a prompt?

The eleven predicates

All deterministic, none of which calls a model.

git_paths_unchanged	paths unchanged in git since a pinned commit
git_commit_reachable	a commit is still an ancestor of a ref, so it survives a force-push
file_checksum	a file still has the recorded sha256
file_exists / file_absent	paths and globs still match, or still do not
url_etag / url_last_modified	conditional GET, RFC 9110 §13.1.1
dependency_version	a package is still pinned in uv.lock, poetry.lock, package-lock.json or requirements.txt
expires_at	an explicit deadline
superseded_by	valid until a named replacement exists
command_exit_code	an escape hatch, off by default, see SECURITY.md

Full reference: docs/predicates.md.

A check that cannot run is never a pass

git missing, network down, file unreadable, lockfile half-written, commands disabled by policy: every one of those returns UNKNOWN and surfaces as suspect. Never valid.

SUSPECT (1)
  "The pricing page lists three tiers"
    why      http://127.0.0.1:1/pricing unreachable (URLError: <urlopen error [Errno 111] Connection refused>) - could not verify
    id       01M1J0D70VP22QDRHDW8902CA9
    action   could not verify; treat as unverified until it can be

It also never manufactures stale: "I could not look" is not evidence that anything changed, and false alarms are how a check acquires a || true. And an unevaluable check never downgrades an already-stale record to a mere suspicion.

stateDiagram-v2
    direction LR
    proposed --> verified: first check PASS
    verified --> valid: a later check PASS
    valid --> valid: PASS
    valid --> stale: FAIL
    valid --> suspect: UNKNOWN
    suspect --> valid: PASS
    suspect --> suspect: UNKNOWN
    suspect --> stale: FAIL
    stale --> stale: PASS or UNKNOWN
    stale --> verified: revalidate

Loading

The two edges worth staring at are stale --> stale: PASS and valid --> suspect: UNKNOWN. A predicate that happens to pass again on a stale record has not re-read the claim, so only a deliberate validwhile revalidate clears it. An unreachable check downgrades a record rather than confirming it. invalidated and superseded are terminal and never re-checked. The full verdict-to-state table is in docs/states.md.

That rule is the difference between a useful tool and a dangerous one, so it is enforced by tests/test_predicate_unknown.py. Break it on purpose, make UNKNOWN return valid, and 33 tests fail. That was verified by doing it, not assumed.

Install

Nothing here is published to PyPI. Run it straight from git:

uvx --from git+https://github.com/jimmyjames177414/validwhile validwhile --help
uvx --from git+https://github.com/jimmyjames177414/validwhile validwhile predicates

Python 3.10 to 3.13. One runtime dependency, tomli, and only on 3.10. Everything else is the standard library and the git CLI you already have.

git clone https://github.com/jimmyjames177414/validwhile && cd validwhile
uv venv && uv pip install -e ".[dev]"

Where a command below is written as a bare validwhile ..., use whichever form you installed: uv run validwhile ... inside a clone, or the uvx --from git+... prefix above.

Commands

validwhile init

validwhile add "Authentication uses JWT bearer tokens" \
    --evidence 'src/auth/**' \
    --source CLAUDE.md:42

validwhile check          # exit 1 if anything is stale, 0 if not
validwhile check --json   # same, machine readable
validwhile list --state stale
validwhile revalidate <id>   # after you have re-read the claim

Flags, exit codes and the CI recipe: docs/cli.md.

Gate a commit on it

Exiting non-zero is how this actually gets adopted.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/jimmyjames177414/validwhile
    rev: main          # no release is tagged yet; pin a commit sha for reproducibility
    hooks:
      - id: validwhile

The shipped hook uses --include-uncommitted (at pre-commit time your change is in the index, not in HEAD), --offline (a hook must not be able to hang on someone else's server) and --no-write (a gate reports; it does not mutate). validwhile-online and validwhile-strict are the other two hook ids.

Prior art, and what is actually new here

The contribution is synthesis and packaging, not a new concept. Every mechanism below has mature ancestors, and pretending otherwise would be the fastest way to make this repository not worth reading.

Prior art	What it already does
Doyle, A Truth Maintenance System (1979); de Kleer, An Assumption-Based TMS (1986)	Justification plus automatic retraction. Our state machine is a stripped-down JTMS. Ships in Drools today.
EvidenceSpine	Already re-reads live files, recomputes sha256 over grounded excerpts, flags evidence_stale, and ships git post-commit/post-merge hooks. The closest existing work.
agents-md-drift	Already verifies that paths and package scripts referenced by an AGENTS.md still exist.
Bazel Skyframe action-cache keys; Nix; Rails cache digests	Digest-over-declared-inputs invalidation, at scale, for a decade.
RFC 9110 §13.1.1	Conditional revalidation. Our URL predicates are ordinary If-None-Match.
Zep / Graphiti (arXiv:2501.13956)	Bi-temporal fact validity and LLM contradiction detection when new information arrives.
MemGuard	Externally computed staleness heuristics, including source-URL re-fetching.
W3C PROV-O	Evidence-linked claims, though it records that a derivation happened, not that its result still holds.

So what is left? Three qualifiers deep, and no deeper:

1. Pull-based re-evaluation as the primary model. Mem0 needs a contradicting fact to arrive; Graphiti needs a new episode narrated; most systems must be told. Re-evaluating a stored belief on demand, against the world, unprompted, is unusual, though EvidenceSpine does exactly this for one predicate type.
2. A pluggable predicate taxonomy under one interface. EvidenceSpine ships checksums, agents-md-drift ships path existence, HTTP ships ETag, Bazel ships input hashing. Assembling them behind one valid_while field with a uniform state machine is the actual work here.
3. Gating the prompt-assembly step specifically. Others gate actions, durable effects, or plan execution. Deciding what a model is allowed to be told it knows is a less-occupied position.

Plus one packaging point: the predicate is a declared, portable, serialisable artifact. In a TMS the justification is an internal graph; in Bazel it is implicit in the action graph. Here it is a JSON object that travels with the fact and can be re-executed later by a different process.

The full assessment, including the claims this project is forbidden from making, is in NOVELTY.md. It was written before any code was. It is also why the project is not called TruthLease: that name was taken by a serious adjacent project, and "lease" was the wrong metaphor anyway, because a lease is defined by time-bounded expiry, which is precisely what this design rejects.

What it will not do

Read this section. It is the one that will save you time.

• "Evidence changed" is not "the claim is false." src/auth/** changing might mean JWTs were removed, or it might mean somebody fixed a typo in a comment. validwhile detects suspicion, not falsity. Never blur this; the tool would become either noise or a liar.
• Binding is semi-automatic and easy to get wrong. scan-rules proposes, a human confirms. A claim bound to the wrong paths produces confident silence, a check that passes forever while the thing it was meant to watch drifts. That is worse than no claim.
• scan-rules misses most claims. It only finds sentences that name a path or a URL. "Session cookies are not used anywhere" is a real, important claim and it finds nothing to bind it to. docs/scan-rules.md lists the gaps.
• No sub-file granularity. A claim about one function binds to the whole file.
• No cascade. Invalidating B does not flag the A that depended on it. There is no dependency graph in v0.1.
• No adapters. Memories already in Mem0, Zep or Letta are not covered. You would be keeping a second store.
• URL predicates are only as good as the server. Plenty of sites send no ETag (permanently suspect) or a fresh one on every request because the footer has a timestamp (permanently stale). Neither is fixable from this side.
• confidence is whatever the author typed. validwhile never computes, adjusts or infers it. There is no model scoring your claims. A computed-looking confidence number would be a fake metric, so there isn't one.
• The question that would justify the whole project is unanswered. Does gating stale memories out of an agent's context measurably improve task success? Nobody knows, including us. validwhile export --format cxs emits the intervention list so someone can run that experiment; see docs/cxs-interop.md.
• No batching, so it does not scale to thousands of claims. Each git predicate spawns its own git subprocesses, roughly five per record, so a store with hundreds of records takes seconds rather than milliseconds. --workers barely helps, because process spawning dominates rather than I/O. Measure your own with time validwhile check. One shared diff per commit range would fix this and has not been written.
• Alpha. The record format and CLI may change before 1.0.

Not built yet

Named here rather than stubbed, because a stub that looks implemented is worse than an admitted gap:

• Mem0 / Zep / Letta adapters
• An MCP server, so an agent can ask "is this still valid?" before injecting a memory
• LLM claim extraction (scan-rules is deliberately regex-only today)
• Database-query and API-schema predicates
• Cascade invalidation across a dependency graph
• Function-level evidence binding
• A web dashboard

Docs

docs/predicates.md	Every predicate, what it proves, and when it returns UNKNOWN
docs/states.md	The state machine and the safety rule
docs/cli.md	Commands, flags, exit codes, CI and pre-commit
docs/records.md	The record format, and why confidence is not computed
docs/scan-rules.md	Claim extraction, and what it misses
docs/cxs-interop.md	Emitting stale memories as CXS v0.1 Interventions
SECURITY.md	Why importing a store is not code execution
NOVELTY.md	The prior-art assessment, written before the code

Development

git clone https://github.com/jimmyjames177414/validwhile && cd validwhile
uv venv && uv pip install -e ".[dev]"

uv run pytest -m "not live"    # 357 tests
uv run ruff check .
uv run mypy --strict src/

No API key needed to contribute, because there is nothing to authenticate against. Git is never mocked in the tests: they build real throwaway repositories and make real commits, because the realness of the check is the entire point. See CONTRIBUTING.md.

Licence

Apache-2.0.

---

@jimmyjames177414 · Apache-2.0

_{One of nine open-source tools for measuring what context and tools actually do to AI systems:
stopless · stopbench · mincontext · validwhile · errorbars
assumptionledger · toolsweep · knowwhen · inconclusive}