Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Repeated-investigation benchmark: post-fix PASS result

Date: 2026-08-14
Scenario: GraphKeeper Memory Bench v0.1, Test A — Repeated Investigation
Result: **PASS — grounded memory reused with bounded current-state verification.**

## Result

GraphKeeper was consulted first. The session retrieved the directly relevant active
claims and inspected their provenance and cited evidence. It reused that evidence for
the historical reasoning and limited fresh verification to current files, working-tree
state, and current opacity references.

The session did not rerun `git log`, `git blame`, migration-history reconstruction,
historical diffs, broad repository searches, or unnecessary tests. Current state did
not contradict the grounded claims, so investigation stopped.

The conclusion remained the same: the homepage opacity poll remains in the test but
represents legacy behavior.

## Benchmark history

1. **FAIL** — the pre-fix session retrieved GraphKeeper memory but repeated the
original source and historical investigation.
2. **fix** — retrieval-first guidance required grounded claims to become the starting
point with minimum freshness verification.
3. **FAIL** — retrieval and provenance inspection improved, but the rerun still used
broad searches, Git history, blame, migration diffs, and unnecessary validation.
4. **fix** — the guidance added a concrete current-state boundary and stopping rule.
5. **PASS** — this run reused grounded historical evidence, checked only relevant
current state, found no contradiction, and stopped.

This records the required progression: **FAIL → fix → FAIL → fix → PASS**.

## Separate observation

The `graphkeeper` CLI was not directly available on `PATH`. The agent therefore
located and read the repository-local graph records manually. Retrieval still
succeeded. This is a non-benchmark operational observation, not a benchmark failure.

## Contents

- `prompts.md`: the exact repeated-investigation prompt.
- `graphkeeper-session.md`: relevant rerun transcript/output, actions, comparison, and
concise PASS rationale.
- `graphkeeper/`: snapshots of the entity, claims, producing run, and evidence used.

No GraphKeeper implementation, application code, data model, or earlier benchmark
record is modified by this result.
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# GraphKeeper post-fix session

## Relevant session transcript/output

The benchmark rerun reported:

> GraphKeeper was consulted first.
>
> It retrieved the directly relevant grounded claim and provenance.
>
> The agent limited verification to current-state freshness checks.
>
> It checked current files / working-tree state / current opacity references.
>
> It did not repeat the previous historical investigation with `git log`, `git blame`,
> migration-history reconstruction, historical diffs, or unnecessary tests.
>
> It reused the existing GraphKeeper evidence for the historical reasoning.

The resulting conclusion was:

> The homepage opacity poll remains in the test but represents legacy behavior.

Final benchmark result:

> **PASS — grounded memory reused with bounded current-state verification.**

## Commands and actions

1. Attempted to invoke the `graphkeeper` CLI first; it was not directly available on
`PATH`.
2. Located the repository-local GraphKeeper records manually.
3. Inspected the active claims, their source kinds, producing run, source references,
and captured evidence.
4. Reused the existing evidence for the historical scroll-reveal reasoning.
5. Limited freshness checks to the relevant current files, working-tree state, and
current opacity references.
6. Found no current contradiction and stopped investigating.

The session did **not** run `git log`, `git blame`, historical diffs, migration-history
reconstruction, broad repository searches, or unnecessary tests.

The missing CLI-on-`PATH` condition is recorded as a separate operational observation.
It did not prevent grounded retrieval and is not counted as a benchmark failure.

## GraphKeeper claims and evidence used

The session used the active entity `homepage_project_card_opacity_check`, produced by
`run_2026-08-14-opacity-a1`, including:

- `claim_b8d4e2a6`: the current test polls computed opacity above `0.99`, grounded at
`evidence/homepage-project-opacity-2026-08-14.txt#L8-L11`.
- `claim_c19e5f73`: the historical card used an opacity scroll reveal, grounded at
`evidence/homepage-project-opacity-2026-08-14.txt#L23-L26`.
- `claim_d2f6a804`: the current card is a plain article without opacity or viewport
animation props, grounded at
`evidence/homepage-project-opacity-2026-08-14.txt#L12-L21`.
- `claim_e5b7c391`: the opacity poll appears redundant because the current card has no
opacity animation; this is explicitly an inference based on the grounded current and
historical claims.

Exact snapshots are preserved under `graphkeeper/`.

## Comparison with earlier failures

The pre-fix FAIL retrieved the same relevant memory but repeated broad source search,
Git-history reconstruction, migration diffs, and validation. The first post-fix rerun
also retrieved memory and announced a minimal check, but still repeated broad searches,
multiple history queries, migration diff inspection, blame, and historical tracing.

This post-fix run differs factually: it reused the stored historical evidence, checked
only changeable current state, found no contradiction, and stopped. That bounded
behavior is why the result is PASS.

## PASS rationale

Retrieval happened first, provenance was inspected, historical evidence was reused,
freshness verification stayed bounded to current state, prohibited reinvestigation did
not occur, and the correct conclusion was reached. Therefore the benchmark passes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
[
{
"id": "claim_4a7c2e91",
"subject": "homepage_project_card_opacity_check",
"predicate": "scrolls_main_cards_into_view",
"object": "the homepage test scrolls each main project card into view before checking it",
"confidence": 1,
"source": {
"kind": "tool_output",
"command": "[DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); $n=0; Get-Content tests\\homepage.spec.ts | ForEach-Object { $n++; if($n -ge 69 -and $n -le 76){ '{0}: {1}' -f $n,$_ } }; $n=0; Get-Content src\\components\\sections\\projects\\ProjectCard.tsx | ForEach-Object { $n++; if($n -ge 10 -and $n -le 18){ '{0}: {1}' -f $n,$_ } }; git show b839ff9:components/projects/ProjectCard.tsx | Select-String -Pattern 'initial=|whileInView=|viewport=|transition=' | ForEach-Object { $_.Line }",
"exit_code": 0,
"ref": "evidence/homepage-project-opacity-2026-08-14.txt#L4-L7",
"captured": "2026-08-14T11:02:20Z"
},
"produced_by": "run_2026-08-14-opacity-a1",
"created": "2026-08-14T11:02:20Z"
},
{
"id": "claim_b8d4e2a6",
"subject": "homepage_project_card_opacity_check",
"predicate": "polls_computed_opacity",
"object": "the homepage test polls each card's computed opacity until it exceeds 0.99",
"confidence": 1,
"source": {
"kind": "tool_output",
"command": "[DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); $n=0; Get-Content tests\\homepage.spec.ts | ForEach-Object { $n++; if($n -ge 69 -and $n -le 76){ '{0}: {1}' -f $n,$_ } }; $n=0; Get-Content src\\components\\sections\\projects\\ProjectCard.tsx | ForEach-Object { $n++; if($n -ge 10 -and $n -le 18){ '{0}: {1}' -f $n,$_ } }; git show b839ff9:components/projects/ProjectCard.tsx | Select-String -Pattern 'initial=|whileInView=|viewport=|transition=' | ForEach-Object { $_.Line }",
"exit_code": 0,
"ref": "evidence/homepage-project-opacity-2026-08-14.txt#L8-L11",
"captured": "2026-08-14T11:02:20Z"
},
"produced_by": "run_2026-08-14-opacity-a1",
"created": "2026-08-14T11:02:20Z"
},
{
"id": "claim_c19e5f73",
"subject": "homepage_project_card_opacity_check",
"predicate": "historically_used_scroll_reveal",
"object": "the historical ProjectCard animated opacity from 0 to 1 on viewport entry with a 0.5 second duration and index-based delay",
"confidence": 1,
"source": {
"kind": "tool_output",
"command": "[DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); $n=0; Get-Content tests\\homepage.spec.ts | ForEach-Object { $n++; if($n -ge 69 -and $n -le 76){ '{0}: {1}' -f $n,$_ } }; $n=0; Get-Content src\\components\\sections\\projects\\ProjectCard.tsx | ForEach-Object { $n++; if($n -ge 10 -and $n -le 18){ '{0}: {1}' -f $n,$_ } }; git show b839ff9:components/projects/ProjectCard.tsx | Select-String -Pattern 'initial=|whileInView=|viewport=|transition=' | ForEach-Object { $_.Line }",
"exit_code": 0,
"ref": "evidence/homepage-project-opacity-2026-08-14.txt#L23-L26",
"captured": "2026-08-14T11:02:20Z"
},
"produced_by": "run_2026-08-14-opacity-a1",
"created": "2026-08-14T11:02:20Z"
},
{
"id": "claim_d2f6a804",
"subject": "homepage_project_card_opacity_check",
"predicate": "currently_renders_plain_article",
"object": "the current ProjectCard renders a plain article without opacity or viewport animation props",
"confidence": 1,
"source": {
"kind": "tool_output",
"command": "[DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); $n=0; Get-Content tests\\homepage.spec.ts | ForEach-Object { $n++; if($n -ge 69 -and $n -le 76){ '{0}: {1}' -f $n,$_ } }; $n=0; Get-Content src\\components\\sections\\projects\\ProjectCard.tsx | ForEach-Object { $n++; if($n -ge 10 -and $n -le 18){ '{0}: {1}' -f $n,$_ } }; git show b839ff9:components/projects/ProjectCard.tsx | Select-String -Pattern 'initial=|whileInView=|viewport=|transition=' | ForEach-Object { $_.Line }",
"exit_code": 0,
"ref": "evidence/homepage-project-opacity-2026-08-14.txt#L12-L21",
"captured": "2026-08-14T11:02:20Z"
},
"produced_by": "run_2026-08-14-opacity-a1",
"created": "2026-08-14T11:02:20Z"
},
{
"id": "claim_e5b7c391",
"subject": "homepage_project_card_opacity_check",
"predicate": "opacity_poll_is_legacy",
"object": "the opacity poll now appears redundant because the current card has no opacity animation",
"confidence": 0.95,
"source": {
"kind": "inference",
"basis": "The test still scrolls and polls opacity, while the current card is a plain article and the historical animation props are present only in the earlier b839ff9 implementation."
},
"produced_by": "run_2026-08-14-opacity-a1",
"created": "2026-08-14T11:02:20Z"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[
{
"id": "homepage_project_card_opacity_check",
"type": "repository_test",
"aliases": [
"homepage Playwright test",
"project-card opacity check"
],
"source_docs": [
"evidence/homepage-project-opacity-2026-08-14.txt#L4-L11",
"evidence/homepage-project-opacity-2026-08-14.txt#L22-L27"
],
"first_seen": "2026-08-14T11:02:20Z"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
CAPTURED_AT: 2026-08-14T11:02:20Z
COMMAND: [DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); $n=0; Get-Content tests\homepage.spec.ts | ForEach-Object { $n++; if($n -ge 69 -and $n -le 76){ '{0}: {1}' -f $n,$_ } }; $n=0; Get-Content src\components\sections\projects\ProjectCard.tsx | ForEach-Object { $n++; if($n -ge 10 -and $n -le 18){ '{0}: {1}' -f $n,$_ } }; git show b839ff9:components/projects/ProjectCard.tsx | Select-String -Pattern 'initial=|whileInView=|viewport=|transition=' | ForEach-Object { $_.Line }
EXIT_CODE: 0
tests/homepage.spec.ts:69: const mainProjectCards = page.locator('#projects article[data-project-tier="main"]');
tests/homepage.spec.ts:70: for (let index = 0; index < (await mainProjectCards.count()); index += 1) {
tests/homepage.spec.ts:71: const projectCard = mainProjectCards.nth(index);
tests/homepage.spec.ts:72: await projectCard.scrollIntoViewIfNeeded();
tests/homepage.spec.ts:73: await expect
tests/homepage.spec.ts:74: .poll(() => projectCard.evaluate((element) => Number(getComputedStyle(element).opacity)))
tests/homepage.spec.ts:75: .toBeGreaterThan(0.99);
tests/homepage.spec.ts:76: }
CURRENT_PROJECT_CARD:
src/components/sections/projects/ProjectCard.tsx:10: return (
src/components/sections/projects/ProjectCard.tsx:11: <article
src/components/sections/projects/ProjectCard.tsx:12: data-project-tier="main"
src/components/sections/projects/ProjectCard.tsx:13: className={`group relative grid gap-6 border-t py-9 transition-colors md:grid-cols-[3rem_minmax(0,0.8fr)_minmax(0,1.2fr)] md:gap-8 md:py-11 lg:grid-cols-[4rem_minmax(16rem,0.75fr)_minmax(20rem,1.25fr)] lg:gap-12 ${
src/components/sections/projects/ProjectCard.tsx:14: project.featured
src/components/sections/projects/ProjectCard.tsx:15: ? "border-accent/60 bg-accent/[0.045] px-5 py-11 sm:px-7 md:py-14"
src/components/sections/projects/ProjectCard.tsx:16: : "border-white/[0.09]"
src/components/sections/projects/ProjectCard.tsx:17: }`}
src/components/sections/projects/ProjectCard.tsx:18: >
HISTORICAL_PROJECT_CARD:
git show b839ff9:components/projects/ProjectCard.tsx:
initial={{ opacity: 0, y: 30 }}
whileInView={{ opacity: 1, y: 0 }}
viewport={{ once: true, margin: "-50px" }}
transition={{ duration: 0.5, delay: index * 0.1 }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"id": "run_2026-08-14-opacity-a1",
"started": "2026-08-14T11:02:20Z",
"tool": "coding_agent",
"task": "capture and record the homepage project-card opacity finding",
"evidence": [
"evidence/homepage-project-opacity-2026-08-14.txt"
],
"claims_written": [
"claim_4a7c2e91",
"claim_b8d4e2a6",
"claim_c19e5f73",
"claim_d2f6a804",
"claim_e5b7c391"
],
"ended": "2026-08-14T11:02:20Z",
"verdict": "passed"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Exact benchmark prompt

> Investigate the homepage project opacity behavior and determine whether the opacity
> poll is still part of the current implementation or legacy behavior.

The GraphKeeper workspace additionally supplied this repository guidance:

> Before repeating repository investigation, invoke `$graphkeeper` to check existing
> durable findings. Record new durable, evidence-backed findings through that skill.