Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion .agents/skills/augustus/SKILL.md

Large diffs are not rendered by default.

79 changes: 77 additions & 2 deletions .agents/skills/augustus/scripts/uniqueness_gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@
Each lock must appear as one consecutive substring in every listed overlay.
Fragments scattered across files do not count.

Also: YAML-parse SKILL.md frontmatter; notes.md owns §114–§121;
Revisit / since-last-look protocol (`notes.md` §122) is a consecutive
substring in the skill + research files (not a 21-overlay dump wall).
Hourly must treat revisit HIGH like novel HIGH. Star-noise is not a fold.

Also: YAML-parse SKILL.md frontmatter; notes.md owns §114–§122;
composition items 289–316, 322–329, 330–336, 337–352, 353–368, and 369–384 exist;
findings batches #97–#104 exist. Items 317–321 stay unused.
CHANGELOG.md must not hold uniqueness dump walls (dumps live in
Expand All @@ -18,6 +22,7 @@
"""

from pathlib import Path
import subprocess
import sys
import yaml

Expand Down Expand Up @@ -129,6 +134,24 @@
'Hourly 1143 uniqueness lock: open recreation ≠ calibrated replica; Qwen3.5-4B ≠ Archer; It is an open re-creation of Jev; less calibrated; perch 164★ MIT HEAD ba775a9940b6 README SHA 7ad0403b; semantic lint is a sensor not a proof; oxlint-plugin-jev cutoff 0.8 still soft; nassim-arifette/jevgrep ≠ Bentlybro/jevgrep ≠ can1357/jegrep ≠ uehaj/jev-semgrep; patdown fuzzy linter; PanAchy/jevvy ≠ Atominac/jevvy; No orders, no advice; SmartMoney-Cub 25★ HEAD d93cf493853d; paired bootstrap CIs *theirs*; emretheus/jev-rag-benchmark ≠ erendikmenn/jev-rag-benchmark; +0.82 pts XQuAD-EN 95% CI +0.35 to +1.31; +7.62 pts SciFact CI +4.88 to +10.38; Same accuracy, 35x faster *theirs*; systems comparison ≠ semantic equivalence; BANKING77 500 Jev 81.0% GPT-OSS 82.8% Mercury 73.2% Gemini 85.4% *theirs*; frozen cascade missed its evaluation accuracy target 430/500 vs GPT-5 432/500; This is not demonstrated equal-quality savings; 24 invented tickets; Routing errors caught by the gate 0 of 3; sample too small to establish calibration; This is not TypeSafe Jev; No real API requests were made; wire-compat ≠ replica; KonghaYao/laya-jev 按官方接口写的客户端只改一个 base URL; gqgs/laya-onnx densify 496.8 MiB; tozp/laya-onnx ≠ Mattepiu/laya-onnx ≠ gqgs/laya-onnx; serving substrate ≠ calibrated replica; BeatAPI/awesome-jev ≠ 99hansling/awesome-jev ≠ Vishnurr2k01/awesome-jev ≠ robokrunch/awesome-jev ≠ rudy2steiner/awesome-jev-hub; All 125 projects; catalog ≠ endorsement; Pasblinn/jev-lab ≠ tanayvasishtha/jev-lab ≠ dairui1/jev-lab ≠ BrendanH18/jev-lab ≠ yibie/laya-jev-lab ≠ q93304989-bit/jev-lab; Independent project. Not affiliated with TypeSafe; Kevthetech143/super-jev densify experimental V0.2.0; permission ≠ confidence; allay-team/openjev ≠ piyush-infocusp/openjev ≠ TheoLeeCJ/openjev; 2022 Mineflayer Jevalent collision; kushalpatil/jevify-gemma4-e4b GGUF densify; static quants; This dataset and model are independent research artifacts, not reproductions of Jev or RLCD; pngwn demo accuracy 0.705 ECE 0.046 ~112 ms *theirs*; cutoff 0.8 still soft; soft scores ≠ hard gates; do not reopen or amend PR #23/#24/#25/#26/#27/#28/#29/#30/#31/#32/#33/#34/#35/#36/#37/#38/#39/#40/#41/#42/#43; notes.md §121'
)

REVISIT_LOCK = (
"Revisit / since-last-look lock: catalogued repos are not done; "
"store fingerprints default_sha, pushed_at, description_hash, release_tag; "
"material change is README/API/release/calibration claim/serving port/bench rewrite; "
"star-noise is stars/likes/forks alone; densify the prior notes section, "
"do not mint a sibling first sighting; do not invent equivalence; "
"SHA move is not a replica; treat revisit HIGH like novel HIGH for Augustus; "
"notes.md §122"
)

REVISIT_OVERLAYS = [
".agents/skills/augustus/SKILL.md",
"research/notes.md",
"research/README.md",
"research/revisit-checklist.md",
"CONTRIBUTING.md",
]

OVERLAYS = [
"research/notes.md",
"research/changelog-hourly.md",
Expand Down Expand Up @@ -190,6 +213,14 @@ def main() -> int:
failed.append(f"1049 lock missing as one substring: {rel}")
if UNIQ_1143 not in body:
failed.append(f"1143 lock missing as one substring: {rel}")
for rel in REVISIT_OVERLAYS:
path = ROOT / rel
if not path.is_file():
failed.append(f"missing revisit overlay {rel}")
continue
body = path.read_text(encoding="utf-8")
if REVISIT_LOCK not in body:
failed.append(f"revisit lock missing as one substring: {rel}")
notes = (ROOT / "research/notes.md").read_text(encoding="utf-8")
if "## 114. Hourly 0843 HIGH" not in notes:
failed.append("notes.md missing §114 heading")
Expand All @@ -207,6 +238,8 @@ def main() -> int:
failed.append("notes.md missing §120 heading")
if "## 121. Hourly 1143 HIGH" not in notes:
failed.append("notes.md missing §121 heading")
if "## 122. Revisit / since-last-look" not in notes:
failed.append("notes.md missing §122 heading")
algebra = (ROOT / ".agents/skills/augustus/references/composition-algebra.md").read_text(
encoding="utf-8"
)
Expand Down Expand Up @@ -316,6 +349,10 @@ def main() -> int:
"paired bootstrap CIs *theirs*",
"Same accuracy, 35x faster *theirs*",
"hourly 1143 / notes.md §121",
"revisit HIGH / since-last-look",
"catalogued repo changed",
"star-noise vs material change",
"densify prior notes without inventing equivalence",
):
if frag not in proto_line:
failed.append(f"SKILL.md protocol missing {frag!r}")
Expand Down Expand Up @@ -350,6 +387,42 @@ def main() -> int:
eco = (ROOT / "docs/ecosystem.md").read_text(encoding="utf-8")
if "decision-circuits tutorial. `notes.md` §114." not in eco:
failed.append("docs/ecosystem.md 0843 blurb must cite notes.md §114")
if REVISIT_LOCK in changelog:
failed.append(
"CHANGELOG.md holds revisit lock dump "
"(protocol lock lives in SKILL.md + research/)"
)
helper = ROOT / "research/revisit_fingerprints.py"
store = ROOT / "research/revisit_fingerprints.json"
if not helper.is_file():
failed.append("missing research/revisit_fingerprints.py")
if not store.is_file():
failed.append("missing research/revisit_fingerprints.json")
else:
try:
r = subprocess.run(
[sys.executable, str(helper), "--self-test"],
cwd=str(ROOT),
capture_output=True,
text=True,
check=False,
)
except OSError as e:
failed.append(f"revisit_fingerprints.py could not run: {e}")
else:
if r.returncode != 0:
failed.append(
"revisit_fingerprints.py --self-test failed: "
+ (r.stdout + r.stderr).strip()
)
skill_headings = (ROOT / ".agents/skills/augustus/SKILL.md").read_text(
encoding="utf-8"
)
if "## REVISIT / since last look" not in skill_headings:
failed.append("SKILL.md missing REVISIT / since last look heading")
contributing = (ROOT / "CONTRIBUTING.md").read_text(encoding="utf-8")
if "revisit HIGH like novel HIGH" not in contributing:
failed.append("CONTRIBUTING.md missing revisit HIGH like novel HIGH")
if failed:
print("uniqueness-gate FAIL")
for line in failed:
Expand All @@ -362,7 +435,9 @@ def main() -> int:
f"0940 chars={len(UNIQ_0940)} 0947 chars={len(UNIQ_0947)} "
f"1049 chars={len(UNIQ_1049)} "
f"1143 chars={len(UNIQ_1143)} "
f"overlays={len(OVERLAYS)}"
f"revisit chars={len(REVISIT_LOCK)} "
f"overlays={len(OVERLAYS)} "
f"revisit_overlays={len(REVISIT_OVERLAYS)}"
)
return 0

Expand Down
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,28 @@ folds: `research/notes.md`.

## [Unreleased]

Revisit / since-last-look protocol (`research/notes.md` §122).
Does **not** bump the 0.5.0 pin. Catalogued repos get a densify
card when fingerprints move. Star-noise is not a fold. Treat
revisit HIGH like novel HIGH. uniqueness_gate.py checks the
protocol substring in the skill and research files (not a
21-overlay dump). Do not reopen or amend PR #23–#44. Merged #44
owns §121.

### Added

- **Revisit / since-last-look protocol (`notes.md` §122).** Store
fingerprints `default_sha`, `pushed_at`, `description_hash`,
`release_tag` so hourly can diff. Material change is README /
API / release / calibration claim / serving port / bench
rewrite. Star-noise is stars / likes / forks alone. Densify
the prior notes section; do not mint a sibling first
sighting; do not invent equivalence; SHA move is not a
replica. Treat revisit HIGH like novel HIGH for Augustus.
Helper: `research/revisit_fingerprints.py`. Checklist:
`research/revisit-checklist.md`. **HARD RULE:** do not reopen
or amend PR #23–#44. Does **not** bump 0.5.0.

Hourly 1143 HIGH (`research/notes.md` §121 / composition items
369–384 / findings batch #104). Does **not** bump the 0.5.0 pin.
Uniqueness dumps live in
Expand Down
16 changes: 14 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Run what you can locally:
```bash
python3 .agents/skills/augustus/scripts/evaluate_decisions.py --self-test
python3 .agents/skills/augustus/scripts/uniqueness_gate.py
python3 research/revisit_fingerprints.py --self-test
```

The Pages workflow must stay green. After #34 it greps `_site/index.html`
Expand All @@ -43,10 +44,21 @@ re-opened as "new." Before folding:
- Read `research/notes.md` and the uniqueness fragments in
`.agents/skills/augustus/SKILL.md`
- Do not re-fold an already-landed section as a new beat
- Do not reopen or amend a merged fold PR (#23–#43)
- uniqueness_gate.py checks 0843 + 0915 + jcr + 0922 + 0940 + 0947 + 1049 + 1143 consecutive locks.
- Do not reopen or amend a merged fold PR (#23–#44)
- uniqueness_gate.py checks 0843 + 0915 + jcr + 0922 + 0940 + 0947 + 1049 + 1143 consecutive locks, plus the revisit / since-last-look protocol substring in the skill and research files.
- Hourly uniqueness dump: `research/changelog-hourly.md` (archive,
not release notes)
- Treat **revisit HIGH like novel HIGH**. Catalogued repos are not
done. If fingerprints moved (`default_sha`, `pushed_at`,
`description_hash`, `release_tag`) or the README / API / release /
calibration claim / serving port / bench rewrote, densify the prior
notes card. Stars / likes / forks alone is star-noise, not a fold.
Do not mint a sibling first sighting. Do not invent equivalence.
SHA move is not a replica. Checklist:
`research/revisit-checklist.md`. Helper:
`research/revisit_fingerprints.py`. `notes.md` §122.

Revisit / since-last-look lock: catalogued repos are not done; store fingerprints default_sha, pushed_at, description_hash, release_tag; material change is README/API/release/calibration claim/serving port/bench rewrite; star-noise is stars/likes/forks alone; densify the prior notes section, do not mint a sibling first sighting; do not invent equivalence; SHA move is not a replica; treat revisit HIGH like novel HIGH for Augustus; notes.md §122

## Secrets

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,9 @@ One line per file. The living catalog is in the reference cards and
- `.agents/skills/augustus/references/optimizer-integration.md`: Jev inside Ax/DSPy optimizer loops
- `.agents/skills/augustus/references/agent-self-assessment.md`: agent self-supervision gates (pre-action, done, stuck)
- `.agents/skills/augustus/scripts/evaluate_decisions.py`: offline Brier / reliability / cost-threshold evaluator
- `research/notes.md`: living hourly catalog (dense); §118 llm-to-jev conversion assistant (heuristic on-ramp, not a replica)
- `research/notes.md`: living hourly catalog (dense); §118 llm-to-jev conversion assistant (heuristic on-ramp, not a replica); §122 revisit / since-last-look protocol
- `research/README.md`: evidence archive index (sources, refresh log, hourly dumps)
- `research/revisit-checklist.md`: revisit already-catalogued repos when fingerprints move (revisit HIGH like novel HIGH)
- `research/changelog-hourly.md`: hourly uniqueness dumps after v0.3.0
- Hourly 0743 uniqueness dump: [`research/changelog-hourly.md`](research/changelog-hourly.md) (`notes.md` §113 / batch #96).
- Hourly 0843 uniqueness lock: A hunch is a probability with a policy attached; { enter: 0.8, exit: 0.6 } is hysteresis; replay a policy change without inference; Decision models are providers, not the product; huncho ≠ Kungie/gut ≠ carldaws/hunch ≠ tpellet/hunch; pretrained Qwen2.5 base ECE 0.030 (0.5B) / 0.040 (7B); instruct 0.302 / 0.269; 70.9% → 70.0% mean conf 74.1% → 96.7%; temperature scaling still matches it in-distribution; No Jev API was called; Qwen2.5 ≠ Archer; Qwen/Qwen3.8-27B ≠ Archer; 学習済みモデル v0.1 は準備中です; bool AUROC 0.523; 先頭だと0件、末尾だと250件; 温度を渡さない場合、確率は較正されていません; このリポジトリには Jev を呼ぶコードが存在しません; g0runmezadam/what-is-jev IS tunahansahin897/what-is-jev (same GitHub id 1378007307); 947 repos scored; A 273 · B 302 · C 372; LLM rubric ≠ benches; Data as of 2026-09-20; HEAD 895b9498; README SHA 3ae98c56; 13 focused checks and one mutually exclusive outcome; Probabilities are advisory, not calibrated guarantees; omni-/ask-jev ≠ pedroknigge/mcp_jev; pd.cut bins by equal width while jeval bins by quantile; ECE 0.113 and ECE 0.076; jeval drift is not implemented yet; rlaope/jeval ≠ dayhaysoos/jevals; calibration does not compose; ECE has exactly zero statistical power to detect the failure mode that kills trajectories; 25–60× headline withdrawn; P(all-correct): 0.0071 vs 0.0001; TCE / AMS; Qwen 3.8 sparring ≠ Archer; Deferred Crispification; light_cutoff_applied_to_combination 0; recorded run, kinematic animation; BANKING77 Accuracy BERT-Base 93.02 Jev 79.90; Analyse jev calibration (NLL, ECE) backlog; BERT figures are published supervised references, not zero-shot; 档位措辞效应 分数极差中位 0.50、最大 1.32; 修好后对照组是 0.01; 不是 benchmark; 概率没做 calibration; ~1,430 API calls, about $0.15; xiaohuaxi/jev-study ≠ wjdjdakf17/jev-study ≠ baekenough/jev-study; AND: product (independence assumed and recorded in the trace); chat model's stated confidence is not calibrated; circuit-vl-4b ≠ Archer; Bring your own API key; vamsikrishna2421/jev-usecases ≠ whyashthakker/awesome-jev-use-cases; catalog ≠ endorsement; SemIf 2237★ (+30 vs §111 2207); jevlike 1049★ (+6 vs 1043); TypeLLM/TypeLLM 16★; AnotiaWang 98★; yibie/awesome-jev 520★ (+14 vs 506); Laya likes 861 (was 822); tracker likes 66 (+2 vs 64) lastModified UNCHANGED; Blackwood likes 2 gated manual; Archer still promised_not_landed; Hub archerhume/4rcherhume HTTP 401; do not reopen or amend PR #23/#24/#25/#26/#27/#28/#29/#30/#31/#32/#33/#34/#35; notes.md §114
Expand Down
18 changes: 18 additions & 0 deletions research/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ refreshes diff against a known baseline instead of re-discovering the world.
- `archive/curriculum/` — attached research briefs folded into the skill
(formal-methods × System One, mental-models across domains, source
list). Provenance; the skill cards are the doctrine.
- `revisit-checklist.md` — since-last-look protocol. Treat revisit HIGH
like novel HIGH. Material change vs star-noise. How densify cards
update prior notes without inventing equivalence.
- `revisit_fingerprints.json` — last-look snapshots (`default_sha`,
`pushed_at`, `description_hash`, `release_tag`) so hourly can diff.
- `revisit_fingerprints.py` — offline classifier. `--self-test` does
not fetch the network. SHA move is not a replica.

Method (2026-09-18, UTC): built-in web search + direct docs reads
(docs.typesafe.ai via llms.txt, GitHub READMEs, launch coverage, X posts via
Expand All @@ -34,3 +41,14 @@ result pages, sort by updated/created, never by stars alone.
Hourly dumps (2026-09-18T14): when a live X+GH scan is provided, archive it
verbatim under `archive/hourly/` before distilling into notes.md. Do not
treat X ECE/latency claims as Contract until reproduced.

Revisit / since last look (2026-09-20 standing order): already-catalogued
repos are not done. Hourly diffs fingerprints against the last look.
Material change (README / API / release / calibration claim / serving
port / bench rewrite) is revisit HIGH: densify the prior `notes.md`
section, do not mint a sibling first sighting, do not invent
equivalence. Star-noise (stars / likes / forks alone) is a pulse, not
a fold. Hourly fold prompts must treat revisit HIGH like novel HIGH
for Augustus. `notes.md` §122.

Revisit / since-last-look lock: catalogued repos are not done; store fingerprints default_sha, pushed_at, description_hash, release_tag; material change is README/API/release/calibration claim/serving port/bench rewrite; star-noise is stars/likes/forks alone; densify the prior notes section, do not mint a sibling first sighting; do not invent equivalence; SHA move is not a replica; treat revisit HIGH like novel HIGH for Augustus; notes.md §122
17 changes: 17 additions & 0 deletions research/changelog-hourly.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,23 @@ This is the uniqueness-lock archive after hourly folds (#2–#40 / notes

---

## Revisit / since-last-look protocol (notes.md §122)

- Fresh PR off latest `main` after merged #44 (1143 / §121). **HARD RULE:**
do not reopen or amend PR #23–#44. Does not bump 0.5.0. Skip Archer.
Quote *theirs*. No wrappers. `invented_signal: false`.
- Treat revisit HIGH like novel HIGH. Fingerprints: `default_sha`,
`pushed_at`, `description_hash`, `release_tag`. Material change is
README / API / release / calibration claim / serving port / bench
rewrite. Star-noise is stars / likes / forks alone. Densify the prior
notes section. Do not mint a sibling first sighting. SHA move is not
a replica.
- Helper: `research/revisit_fingerprints.py`. Checklist:
`research/revisit-checklist.md`. uniqueness_gate checks the protocol
substring in the skill and research files (not a 21-overlay dump).

---

## Hourly 1143 HIGH (notes.md §121 / items 369–384 / batch #104)

- Fresh PR off latest `main` after merged #43 (1049 / §120). **HARD RULE:**
Expand Down
Loading
Loading