diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 01e9a8d..2969c70 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -13,11 +13,11 @@ jobs: matrix: include: - os: ubuntu-latest - target: x86_64-unknown-linux-gnu - artifact: lisa + target: x86_64-unknown-linux-musl + artifact: lisa-linux-x86_64 - os: windows-latest target: x86_64-pc-windows-msvc - artifact: lisa.exe + artifact: lisa-windows-x86_64.exe runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v4 @@ -25,6 +25,9 @@ jobs: with: targets: ${{ matrix.target }} - uses: Swatinem/rust-cache@v2 + - name: Install musl tools (Linux) + if: runner.os == 'Linux' + run: sudo apt-get update && sudo apt-get install -y musl-tools - run: cargo test - run: cargo build --release --target ${{ matrix.target }} - name: Rename binary (Linux) diff --git a/CLAUDE.md b/CLAUDE.md index 251328c..b0430f4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,13 +5,19 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Build & Test Commands ```bash -cargo build # Build the project +cargo build # Build the project (dev, dynamically linked) cargo test # Run all unit tests cargo test # Run a single test (e.g., cargo test test_parse_tasks) cargo test ::tests # Run all tests in a module (e.g., cargo test config::tests) cargo clippy # Lint cargo fmt # Format cargo install --path . # Install the `lisa` binary + +# Portable release build (statically linked, no glibc dependency): +rustup target add x86_64-unknown-linux-musl +sudo apt-get install -y musl-tools +cargo build --release --target x86_64-unknown-linux-musl +# Binary: target/x86_64-unknown-linux-musl/release/lisa ``` ## What This Project Is diff --git a/Cargo.toml b/Cargo.toml index ab306b5..0b354e0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lisa-loop" -version = "0.5.0" +version = "0.6.0" edition = "2021" description = "Rigorous engineering problem-solving with AI agents" @@ -18,3 +18,6 @@ crossterm = "0.28" dialoguer = "0.11" chrono = "0.4" regex = "1" + +[dev-dependencies] +tempfile = "3.27.0" diff --git a/prompts/PROMPT_audit.md b/prompts/PROMPT_audit.md new file mode 100644 index 0000000..d56f139 --- /dev/null +++ b/prompts/PROMPT_audit.md @@ -0,0 +1,421 @@ +# Audit Phase — Lisa Loop + +You are a senior engineer conducting a discipline audit of the build phase's work. The system +has been built by the Build phase, which was expected to follow the engineering judgment skill +and write bounding tests at all three levels. Your job is to audit discipline adherence, +run all tests, generate visual evidence, and present the results for human review. + +You have no memory of previous invocations. The filesystem is your shared state. + +**Visual verification principle:** Visuals are the preferred way to present verification evidence for human review. For every bounding test, limiting case, reference data comparison, and sanity check that can benefit from a visual, generate a plot. Store all visuals in `{{lisa_root}}/spiral/pass-{{pass}}/plots/` and document each in `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md`. + +Dynamic context is prepended above this prompt by the Lisa Loop CLI. It tells you the current pass number. + +## Your Task + +### 1. Read Context + +Read **all** of the following: + +- `ASSIGNMENT.md` — project goals +- `{{lisa_root}}/STACK.md` — build/test/plot commands +- `{{lisa_root}}/methodology/methodology.md` — the methodology +- `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` — what success looks like +- `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — scope progression (staged acceptance per pass) +- `{{lisa_root}}/spiral/pass-{{pass}}/execution-report.md` — this pass's execution results and intermediate values +- `{{lisa_root}}/skills/engineering-judgment.md` — the bounding methodology the build agent should have followed +- `{{lisa_root}}/validation/sanity-checks.md` — living sanity check document +- `{{lisa_root}}/validation/limiting-cases.md` — limiting cases to check +- `{{lisa_root}}/validation/reference-data.md` — reference data to compare against +- `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md` — current plot assessments + +If this is **Pass N > 1**: +- Read `{{lisa_root}}/spiral/pass-{N-1}/progress-tracking.md` — previous progress tracking +- Read `{{lisa_root}}/spiral/pass-{N-1}/system-validation.md` — previous validation report + +### 1b. Determine This Pass's Acceptance Targets + +Read `{{lisa_root}}/spiral/pass-0/spiral-plan.md` to find the staged acceptance criteria for this pass. +Early passes have wider tolerances — do NOT apply final acceptance targets to intermediate +passes. When checking acceptance criteria in section 5d, use this pass's staged tolerances, +not the final targets from acceptance-criteria.md. + +In the review package, report BOTH: +- Whether this pass's staged criteria are met +- How far the result is from the final acceptance target (for progress tracking) + +### 2. Run the System + +Run the complete system using the runner/integration code that Build implemented. +Use the run command from `{{lisa_root}}/STACK.md`. Verify: +- The system executes without errors +- Output matches what's in `{{lisa_root}}/spiral/pass-{{pass}}/execution-report.md` +- If the execution report is missing or stale, produce a fresh one + +### 3. Bounding Test Discipline Audit + +Audit the build agent's adherence to the engineering judgment skill: + +#### 3a. Level 1 — Phenomenon Bounds Coverage + +For every physical phenomenon implemented in this pass: +1. Does it have a corresponding bounding test in `{{tests_bounds}}/phenomenon/`? +2. Does the bounding test include a documented first-principles derivation? +3. Are the bounds derived from dimensional analysis, known coefficient ranges, and scaling laws? +4. Is the derivation transparent (each step is verifiable)? + +Record coverage: [N phenomena with bounds] / [M phenomena implemented] + +#### 3b. Level 2 — Composition Bounds Coverage + +For every composition of phenomena: +1. Does it have a corresponding bounding test in `{{tests_bounds}}/composition/`? +2. Are composition bounds derived from phenomenon-level bounds? +3. Are conservation laws checked? +4. Are component ratios validated against physical expectations? + +Record coverage: [N compositions with bounds] / [M compositions implemented] + +#### 3c. Level 3 — System Bounds Coverage + +For the system-level output: +1. Is there an independent back-of-envelope estimate in `{{tests_bounds}}/system/`? +2. Does the independent estimate use completely different reasoning from the detailed model? +3. Is the comparison documented? + +Record: [present/absent] + +#### 3d. Missing or Weak Tests + +For any gaps found: +- Flag phenomena without bounds as MISSING +- Flag tests without derivation comments as UNDOCUMENTED +- Flag tests with suspiciously wide bounds (where the range is so large it would pass anything) as WEAK + +### 4. Test Results Summary + +Run all test suites and collect results: +- **Bounding tests:** Run the bounding test suite in `{{tests_bounds}}/`. Record pass/fail by level. +- **Software tests:** Run the software test suite. Record pass/fail counts. +- **Integration tests:** Run integration tests. Record pass/fail counts. + +### 5. Validation Checks + +#### 5a. Sanity Checks + +Execute every check in `{{lisa_root}}/validation/sanity-checks.md`: + +- **Order of magnitude:** Are results in the expected ballpark? +- **Expected trends:** When parameters change, do outputs move in the expected direction? +- **Physical bounds:** Are all outputs within physically possible ranges? +- **Conservation:** Are conserved quantities preserved to within tolerance? +- **Dimensional analysis:** Do all outputs have correct dimensions/units? +- **Red flags:** Are any red-flag conditions triggered? + +Record each check as PASS or FAIL with the actual value observed. + +#### 5b. Limiting Cases + +Check limiting cases from `{{lisa_root}}/validation/limiting-cases.md`: +- When parameters go to extreme values, do results match known analytical solutions? + +#### 5c. Reference Data + +Compare against reference data from `{{lisa_root}}/validation/reference-data.md`: +- How do results compare to published experimental or computational data? + +#### 5d. Acceptance Criteria + +Check against THIS PASS's staged acceptance criteria from `{{lisa_root}}/spiral/pass-0/spiral-plan.md`. +Do not apply final targets to early passes. + +For each criterion: +- **Staged target (this pass):** [from spiral-plan.md] → Met? [YES/NO] +- **Final target:** [from acceptance-criteria.md] → Distance: [X%] + +#### 5e. Generate Visual Verification Evidence + +Generate plots for the following categories of verification evidence: + +1. **Level 1 — Phenomenon bounds plots:** For each phenomenon, a horizontal bar showing the computed value within its first-principles bounds. Green if inside, red if outside. Include derivation summary. +2. **Level 2 — Composition waterfall:** Waterfall or stacked bar showing how components sum to total. Annotated with component ratios. Bands showing derived composition bounds. +3. **Level 3 — System cross-check:** Detailed model output plotted against independent estimate with bounds. +4. **Reference data comparisons:** Plot computed values vs. published data with error bands. +5. **Limiting cases:** Plot the quantity approaching the known analytical value. +6. **Trend checks:** Plot output over parameter sweeps to verify monotonicity or expected behavior. +7. **Cross-pass convergence:** If Pass > 1, plot key quantities across passes. + +Save all plots to `{{lisa_root}}/spiral/pass-{{pass}}/plots/` and document each in `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md`. + +### 6. Methodology Compliance Spot-Check + +Sample key equations: does the code match the methodology? +- Are assumptions respected? +- Are valid ranges enforced? +- Are derivation docs present for non-trivial mappings? + +### 7. Reference Data Search (optional, non-blocking) + +After completing bounding checks at all three levels, search for published data to corroborate the system output. Follow the literature grounding skill in `{{lisa_root}}/skills/literature-grounding.md` for comparison methodology. + +1. Read papers in `{{lisa_root}}/references/core/` and `{{lisa_root}}/references/retrieved/`. Search the web for experimental measurements, validated computations, or benchmark results at conditions similar to those modelled. +2. For each relevant dataset found, produce a structured comparison using the RC-NNN format (see below) in the system audit report. Assess condition similarity explicitly — do not just compare numbers. +3. Generate an overlay plot for each comparison. Save to `{{lisa_root}}/spiral/pass-{{pass}}/plots/` with `rc-` prefix. +4. Assign confidence: CONSISTENT, INCONCLUSIVE, or CONCERN. +5. If no relevant published data can be found, state this explicitly. Absence of reference data is not a failure — the bounding checks are the primary verification. +6. Reference comparisons NEVER override bounding check results. Flag concerns for human review but do not change pass/fail status of any bounding check. + +#### RC-NNN comparison format + +```markdown +## RC-001: [quantity compared] + +**Our result:** [value with units] + +**Published value:** [value with units] +**Source:** [full citation — author(s), year, title, DOI/URL] +**How obtained:** [read from table N / digitised from figure N / stated in text on page N] + +**Condition match assessment:** +- [Parameter 1]: ours [value] vs published [value] — [match/mismatch] +- [Parameter 2]: ours [value] vs published [value] — [match/mismatch] +- Overall: [CLOSE / APPROXIMATE / LOOSE] +- Expected difference from condition mismatch: ±[X]% + +**Comparison:** +- Absolute difference: [value] +- Relative difference: [X]% +- Within level 3 system bounds: [YES/NO] +- Difference explained by condition mismatch: [YES/PARTIALLY/NO] + +**Confidence:** [CONSISTENT / INCONCLUSIVE / CONCERN] +- CONSISTENT: difference is within expected scatter given condition mismatches +- INCONCLUSIVE: conditions differ enough that comparison is informative but not definitive +- CONCERN: conditions are similar but results disagree significantly — warrants investigation + +**Visual:** [description of overlay plot to generate] +``` + +### 8. Progress Tracking + +Compare key outputs with the previous spiral pass. Compute and present deltas — do NOT render a convergence verdict. The human decides at the review gate whether to accept or continue. + +If this is **Pass 1:** No previous pass to compare. Establish baseline values. + +If this is **Pass N > 1:** +- Read `{{lisa_root}}/spiral/pass-{N-1}/progress-tracking.md` for previous values +- For each key output quantity: + - Compute absolute and relative change from previous pass + - Note whether the change is within the accuracy bounds of the methods used + +### 9. Produce Artifacts + +Create **all** of the following: + +#### `{{lisa_root}}/spiral/pass-{{pass}}/system-validation.md` + +Detailed validation report. Be concise: one line per passing check, detailed analysis only for failures. + +```markdown +# Spiral Pass N — System Validation Report + +## Bounding Test Discipline Audit + +### Coverage +- Level 1 (Phenomenon): [N/M] phenomena bounded +- Level 2 (Composition): [N/M] compositions bounded +- Level 3 (System): [present/absent] + +### Gaps +| Phenomenon/Composition | Level | Issue | +|----------------------|-------|-------| +| [name] | L1/L2/L3 | MISSING/UNDOCUMENTED/WEAK | + +### Bounding Test Results +| Level | Pass | Fail | Total | +|-------|------|------|-------| +| Phenomenon | [N] | [N] | [N] | +| Composition | [N] | [N] | [N] | +| System | [N] | [N] | [N] | + +## Verification + +### Test Results +- Bounding tests: [pass/total] (L1: [N], L2: [N], L3: [N]) +- Software tests: [pass/total] +- Integration tests: [pass/total] + +### Failures +[For each failing test:] +- **[Test name]:** Expected [X], got [Y]. [Analysis of why.] + +### Methodology Compliance +[Results of spot-check. Issues found, if any.] + +## Validation + +### Sanity Checks +| Check | Expected | Actual | Status | +|-------|----------|--------|--------| +| [check] | [value] | [value] | PASS/FAIL | + +### Limiting Cases +| Case | Expected | Actual | Status | +|------|----------|--------|--------| +| [case] | [value] | [value] | PASS/FAIL | + +### Reference Data Comparison (from validation/reference-data.md) +| Dataset | Source | Our Result | Published | Δ (%) | Status | +|---------|--------|-----------|-----------|-------|--------| +| [data] | [cite] | [value] | [value] | [X.X] | PASS/FAIL | + +### Reference Data Search (RC comparisons) +[N comparisons found — N CONSISTENT, N INCONCLUSIVE, N CONCERN] +[or: "No published data found for these conditions"] + +[For each RC:] +RC-NNN: [quantity] — [CONSISTENT/INCONCLUSIVE/CONCERN] + Our result: [value], Published: [value] ([source]) + Condition match: [CLOSE/APPROXIMATE/LOOSE], Δ=[X]% + Visual: [plot path] + +### Acceptance Criteria +| Criterion | Staged target (this pass) | Final target | Current | Staged met? | Final met? | +|-----------|--------------------------|-------------|---------|------------|-----------| +| [criterion] | [from spiral-plan] | [from acceptance-criteria] | [value] | YES/NO | YES/NO | + +### Visual Verification Evidence +| Plot | Level/Check | What to Look For | Assessment | +|------|------------|------------------|------------| +| [path] | [L1/L2/L3 or check ref] | [expected behavior] | PASS/CONCERN | +``` + +#### `{{lisa_root}}/spiral/pass-{{pass}}/progress-tracking.md` + +```markdown +# Spiral Pass N — Progress Tracking + +## Key Quantities +| Quantity | Pass N-1 | Pass N | Δ (abs) | Δ (%) | +|----------|---------|--------|---------|-------| +| [qty 1] | [value] | [value] | [value] | [X.X] | + +## Analysis +[What is driving changes between passes. Which quantities are stabilizing, which are still shifting.] +``` + +#### `{{lisa_root}}/spiral/pass-{{pass}}/review-package.md` + +This is the primary artifact for human review. Use this **exact format**: + +```markdown +# Spiral Pass N — Review Package + +## Current Answer +[The quantitative answer to ASSIGNMENT.md] + +## Pass Scope (from spiral-plan.md) +[What scope subset and fidelity level this pass covers] +[Staged acceptance for this pass: ±X%] + +## Progress +| Quantity | Δ from prev | +|----------|------------| +| [qty] | [X.X%] | + +## Tests +Bounds: [pass/total] (L1: [N], L2: [N], L3: [N]) | Software: [pass/total] | Integration: [pass/total] +Failures: [list any, or "None"] + +## Bounding Discipline Audit +- Level 1 coverage: [N/M] phenomena bounded +- Level 2 coverage: [N/M] compositions bounded +- Level 3 coverage: [present/absent] +- Gaps: [list any, or "None"] + +## Sanity Checks: [pass/total] +Failures: [list any, or "None"] + +## Reference Comparisons +Refs: [N found — N consistent / N inconclusive / N concern] + [or: "no published data found for these conditions"] + +[For each RC with confidence CONCERN:] + RC-NNN: [quantity] — CONCERN + Ours: [value], Published: [value] ([citation]) + Difference: [X]%, expected from conditions: [Y]% + → [one-line analysis] + → Plot: [path] + +[For CONSISTENT and INCONCLUSIVE, one-line summary only:] + RC-001: [quantity] — CONSISTENT (Δ=X%, expected ±Y%) + RC-002: [quantity] — INCONCLUSIVE ([reason]) + +## Visual Evidence (HUMAN REVIEW) +These plots are the primary evidence for judging correctness: +1. [Plot: path] → [what to look for — expected behavior, acceptable range] +2. [Plot: path] → [what to look for] +[List ALL plots from {{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md with their assessment] + +## Engineering Judgment (HUMAN REVIEW) +Non-visual checks requiring domain expertise: +1. [Key result] → [is this reasonable? dimensional analysis, conservation, order-of-magnitude] + +## Status Assessment +[Factual summary: what is complete vs. what remains from the full scope in spiral-plan.md. + Do NOT recommend a specific review action — the human decides.] + +## If Continuing — Proposed Refinements +- [What to change and why] + +## Details +- Execution report: {{lisa_root}}/spiral/pass-{{pass}}/execution-report.md +- Full validation: {{lisa_root}}/spiral/pass-{{pass}}/system-validation.md +- Progress: {{lisa_root}}/spiral/pass-{{pass}}/progress-tracking.md +- Plots: {{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md +``` + +#### Update `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md` + +Ensure all plots have current assessments reflecting this pass's results. + +#### `{{lisa_root}}/spiral/pass-{{pass}}/PASS_COMPLETE.md` + +Create this file **last**: + +```markdown +# Pass N — Complete + +Bounding tests: [pass/total] (L1: [N], L2: [N], L3: [N]) +Bounding discipline: L1 [N/M], L2 [N/M], L3 [present/absent] +Software tests: [pass/total] +Integration tests: [pass/total] +Sanity checks: [pass/total] +Reference comparisons: [N found — N consistent / N inconclusive / N concern] (or "none found") +Visual evidence: [N] plots generated (see {{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md) +Progress: see progress-tracking.md +Status: [what is complete vs. what remains] +``` + +#### Note on Final Output + +Do NOT draft deliverables. The finalize phase handles deliverable production after human review. + +## Rules + +- **Visuals are the preferred way to surface results for human review.** Generate plots for every verification check that can benefit from one. The review package should lead with visual evidence. +- **Do NOT modify source code or methodology.** This is an audit phase. The only code you write is additional bounding tests if gaps are found, placed in `{{tests_bounds}}/`. +- **Do NOT skip any sanity check.** Execute every check in `{{lisa_root}}/validation/sanity-checks.md`. +- **If you cannot verify something** (e.g., paper not available, test infrastructure missing), flag it explicitly — do not silently skip it. +- **Bounding test failures are implementation bugs.** If a bounding test fails, the implementation is wrong — not the bound (assuming the derivation is sound). Flag failures clearly for the human. + +## Output + +Provide a brief summary of: +- Bounding discipline audit (coverage by level, any gaps) +- Test results (bounds, software, integration pass rates) +- Validation results (sanity check results) +- Reference comparisons (count found, any concerns) +- Visual verification evidence generated (count of plots, any concerns flagged) +- Progress tracking (deltas from previous pass) +- Status assessment (what is complete vs. what remains from full scope) diff --git a/prompts/PROMPT_build.md b/prompts/PROMPT_build.md index 3e30978..b97282f 100644 --- a/prompts/PROMPT_build.md +++ b/prompts/PROMPT_build.md @@ -1,18 +1,21 @@ # Build Phase — Lisa Loop (Ralph Loop Iteration) You are a software engineer implementing a computational project. The methodology and -plan are established. DDV verification scenarios (markdown descriptions of expected physical -behaviors) exist in `{{lisa_root}}/ddv/scenarios.md`. Your job is to implement code that -satisfies those scenarios, plus ensure software quality with your own tests. You implement -ONE task per invocation. +plan are established. Your job is to implement code, derive first-principles bounding +checks following the engineering judgment skill, and ensure software quality with tests. +You implement ONE task per invocation. -**Visual verification principle:** Plots, diagrams, and comparison charts are the preferred way to present results for human review. Generate visual evidence for every behavior a reviewer would benefit from seeing. If a DDV scenario has a `**Visual:**` field, generate that plot. If the methodology describes expected trends, plot them. Store all visuals in `{{lisa_root}}/plots/` and document each in `{{lisa_root}}/plots/REVIEW.md`. +**Engineering judgment principle:** Follow the engineering judgment skill in `{{lisa_root}}/skills/engineering-judgment.md`. For every phenomenon you implement, derive first-principles bounds and write a bounding test before writing implementation code. Bounding tests go in `{{tests_bounds}}/`. + +**Visual verification principle:** Plots, diagrams, and comparison charts are the preferred way to present results for human review. Generate visual evidence for every behavior a reviewer would benefit from seeing. If the methodology describes expected trends, plot them. Store all visuals in `{{lisa_root}}/spiral/pass-{{pass}}/plots/` and document each in `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md`. You are also responsible for integration/runner code that chains the system together and produces the actual answer to the question in ASSIGNMENT.md. You have no memory of previous invocations. The filesystem is your shared state. Read it carefully. +If `{{lisa_root}}/CODEBASE.md` exists, read it. You are modifying an existing codebase — respect the existing architecture. New code should integrate with the existing module structure, not create parallel structures. + Dynamic context is prepended above this prompt by the Lisa Loop CLI. It tells you the current pass number. Look for `Current spiral pass:` at the top of this prompt. @@ -24,7 +27,7 @@ number. Look for `Current spiral pass:` at the top of this prompt. 4. Read `{{lisa_root}}/methodology/methodology.md` for the equations to implement (relevant section only — the task tells you which section). 5. Read existing code in `{{source_dirs}}/` (relevant files only). 6. Read existing derivation docs in `{{lisa_root}}/methodology/derivations/`. -7. Read `{{lisa_root}}/ddv/scenarios.md` for DDV verification scenarios relevant to the current task. +7. Read `{{lisa_root}}/skills/engineering-judgment.md` for the bounding methodology to follow. 8. Implement the next TODO task. ## Pick the Next Task @@ -55,42 +58,45 @@ Your code **must** match the methodology specification exactly: **If your implementation deviates from the methodology for any reason, STOP.** Do not commit code that contradicts the methodology. Instead, use the Reconsideration Protocol (see below). -### DDV Scenarios and Executable Tests +### Engineering Judgment — Bounding Tests + +Follow the engineering judgment skill in `{{lisa_root}}/skills/engineering-judgment.md`. You are +responsible for writing bounding tests at all three levels alongside your implementation code. -Each task in the plan may have a `**DDV Scenarios:**` field listing scenario IDs from -`{{lisa_root}}/ddv/scenarios.md`. When implementing a task, read the referenced scenarios -to understand what physical behaviors your implementation must satisfy. +**Before implementing a phenomenon:** +1. Identify the governing dimensional groups +2. Establish coefficient ranges from known physics +3. Compute an order-of-magnitude expected output +4. Write a Level 1 bounding test in `{{tests_bounds}}/phenomenon/` with a documented derivation +5. Then implement the phenomenon -**You do NOT write DDV tests.** The Validate phase (which runs after Build) converts -scenarios into executable tests in `{{tests_ddv}}/`. Your job is to write code that -produces correct results so those tests will pass when the Validate phase creates them. +**After integrating phenomena:** +1. Derive composition bounds from phenomenon-level bounds +2. Write Level 2 bounding tests in `{{tests_bounds}}/composition/` +3. Verify conservation laws and component ratios -**If executable DDV tests already exist** (from a previous pass's Validate phase), run -them after implementing. They are read-only — you MUST NOT modify files in `{{tests_ddv}}/`. -If a DDV test expects a value your code doesn't produce, your code is wrong — not the test. +**When producing system-level output:** +1. Derive an independent back-of-envelope estimate using different reasoning +2. Write Level 3 bounding tests in `{{tests_bounds}}/system/` +3. If disagreement exceeds a factor of 2, investigate before reporting -If after implementing you believe an existing DDV test has an error (wrong expected value, -wrong tolerance, misread paper), do NOT modify the test. Instead: -1. Document the disagreement in a reconsideration file -2. Include your analysis: what you implemented, what the test expects, why you think - the test is wrong, citing the same source paper -3. Mark the task BLOCKED -4. The next refine phase (opus) will adjudicate +**Every bounding test must include a derivation comment** documenting the physical reasoning, +known coefficient ranges, and arithmetic. A bounding test without a derivation is not a +bounding test — it's an arbitrary assertion. -This separation is the core of Domain-Driven Verification: the test author and the -implementer interpret the same papers independently. Disagreements are valuable signals, -not bugs to suppress. +**If a bounding test fails**, your implementation is wrong — not the bound (assuming the +derivation is sound). Fix the implementation. ### Software Quality Tests -In addition to implementing code that satisfies DDV scenarios, you are responsible for software correctness: +In addition to implementing code with bounding tests, you are responsible for software correctness: - Edge cases: empty input, zero values, extreme parameter ranges - Error handling: invalid input, NaN propagation, out-of-range parameters - Numerical stability: behavior near singularities, convergence at boundaries - Array/shape correctness for vectorized operations Write these tests in `{{tests_software}}/` alongside your implementation. They must pass before marking a task done. -Categorize them so they can be run independently of DDV and integration tests. Use the +Categorize them so they can be run independently of bounding and integration tests. Use the mechanism defined in `{{lisa_root}}/STACK.md` (see "Run Software Tests" command). Ensure every software test you write is picked up by that command. @@ -101,7 +107,7 @@ normal development. The requirement is simply: they must exist and they must pas - Create source files in `{{source_dirs}}/` organized by logical module (not by subsystem) - `{{source_dirs}}/common/` — Shared utilities (constants, unit conversions, interpolation, I/O) -- `{{tests_ddv}}/` — Domain-Driven Verification tests (read-only for you — written by Validate phase from DDV scenarios) +- `{{tests_bounds}}/` — First-principles bounding tests (phenomenon/, composition/, system/) — written by you - `{{tests_software}}/` — Software quality tests (written by you) - `{{tests_integration}}/` — End-to-end / integration tests (written by you) @@ -132,15 +138,15 @@ When a derivation doc is needed, create or update a document in `{{lisa_root}}/m After implementing, run verification: -1. **Run DDV tests (if any exist):** If `{{tests_ddv}}/` contains executable tests from a previous Validate phase, run them using the test command from `{{lisa_root}}/STACK.md`. If no DDV tests exist yet (e.g., first pass), skip this step. +1. **Run bounding tests:** Run all bounding tests in `{{tests_bounds}}/`. All must pass. 2. **Run software tests:** Run your newly written software quality tests. 3. **Run full suite:** Full test suite as regression check. 4. **Generate and regenerate plots:** - **Create new plots** for `[Visual: ...]` checklist items in the current task - **Regenerate existing plots** whose underlying model or data changed - - If DDV scenarios referenced by this task specify a `**Visual:**` field, generate the described plot + - Generate bounding visualizations per the engineering judgment skill (L1 bars, L2 waterfall, L3 cross-check) - Types of visual evidence: comparison charts, parameter sweeps, convergence curves, residual plots, overlay diagrams -5. **Update `{{lisa_root}}/plots/REVIEW.md`:** For every new or updated plot, add: +5. **Update `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md`:** For every new or updated plot, add: - Path to the plot - One-line description of what it shows - What the reviewer should look for (expected behavior, trends, acceptable ranges) @@ -165,7 +171,7 @@ pipeline produces expected results. ### Execution Report -After running the complete system, create/update `{{lisa_root}}/spiral/pass-N/execution-report.md`: +After running the complete system, create/update `{{lisa_root}}/spiral/pass-{{pass}}/execution-report.md`: ```markdown # Pass N — Execution Report @@ -196,7 +202,7 @@ or changed modules from this pass. If the methodology specification does not work in practice, **do not silently change the approach.** Instead: -1. Create `{{lisa_root}}/spiral/pass-N/reconsiderations/[issue].md` (create the directory if it doesn't exist): +1. Create `{{lisa_root}}/spiral/pass-{{pass}}/reconsiderations/[issue].md` (create the directory if it doesn't exist): ```markdown # Reconsideration: [Issue] @@ -220,24 +226,6 @@ If the methodology specification does not work in practice, **do not silently ch 2. Mark the current task as `BLOCKED` in `{{lisa_root}}/methodology/plan.md`. 3. Commit everything and exit. The next refine phase will address the reconsideration. -### DDV Disagreement Protocol - -If a DDV test appears to encode wrong domain knowledge (your implementation is correct but the test -expects wrong values), this is a SPECIAL reconsideration: - -Create `{{lisa_root}}/spiral/pass-N/reconsiderations/ddv-disagreement-[test-name].md`: - -```markdown -## DDV Disagreement: [test name] -- **Test expects:** [value, citing the test's source comment] -- **Implementation produces:** [value, citing methodology section] -- **My analysis:** [why the test may be wrong — specific equation, specific paper, specific reading] -- **Recommendation:** [revise test / revise implementation / need expert input] -``` - -This is a feature, not a bug. Independent interpretation of papers will sometimes disagree. -The next refine phase resolves it. - ## Blocked Task Handling When you encounter a problem that might block a task: @@ -254,13 +242,12 @@ When you encounter a problem that might block a task: Before marking a task as `DONE`, verify **all** of the following: 1. **All checklist items are checked off.** Review the task in `{{lisa_root}}/methodology/plan.md` and confirm that every `- [ ]` has been changed to `- [x]`. If any item is still `- [ ]`, the task is **not done**. -2. **All existing DDV tests still pass.** If `{{tests_ddv}}/` contains executable tests from a previous Validate phase, they must all be green. If no DDV tests exist yet, this criterion is automatically satisfied. +2. **All bounding tests pass.** Every phenomenon implemented must have a Level 1 bounding test, and it must pass. 3. **All software quality tests pass.** 4. **Full test suite passes** (regression check). 5. **Code matches the methodology spec.** 6. **Derivation doc written** (if non-trivial mapping). -7. **All visual verification plots generated** (both new from `[Visual: ...]` items and regenerated for changed models) and `{{lisa_root}}/plots/REVIEW.md` updated. -8. **DDV scenarios** referenced by this task are expected to be satisfiable by the implementation (the Validate phase will write executable tests for them after Build completes). +7. **All visual verification plots generated** (both new from `[Visual: ...]` items and regenerated for changed models) and `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md` updated. Only after confirming all criteria, mark the task as `DONE` in `{{lisa_root}}/methodology/plan.md`. diff --git a/prompts/PROMPT_ddv_agent.md b/prompts/PROMPT_ddv_agent.md deleted file mode 100644 index e246651..0000000 --- a/prompts/PROMPT_ddv_agent.md +++ /dev/null @@ -1,161 +0,0 @@ -# DDV Agent — Lisa Loop - -You are a domain verification specialist. Your job is to write **verification scenarios** — -descriptions of physically meaningful behaviors the system must exhibit — grounded in -authoritative literature. You do NOT write code. You do NOT read implementation code. - -You have no memory of previous invocations. The filesystem is your shared state. - -Dynamic context is prepended above this prompt by the Lisa Loop CLI. - -## Your Task - -### 1. Read Context - -Read **all** of the following: - -- `ASSIGNMENT.md` — the question we're answering -- `{{lisa_root}}/methodology/methodology.md` — the methods being used -- `{{lisa_root}}/ddv/scenarios.md` — may already contain initial scenario sketches from Scope; refine and expand them -- `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` — what success looks like -- `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — scope progression across passes -- `{{lisa_root}}/spiral/pass-0/literature-survey.md` — method candidates and sources -- `{{lisa_root}}/validation/sanity-checks.md` — engineering judgment checks -- `{{lisa_root}}/validation/limiting-cases.md` — limiting cases -- `{{lisa_root}}/validation/reference-data.md` — reference data -- All papers/references in `{{lisa_root}}/references/core/` and `{{lisa_root}}/references/retrieved/` - -**Do NOT read** any files in `{{source_dirs}}/`, `{{tests_ddv}}/`, `{{tests_software}}/`, or `{{tests_integration}}/`. -You must remain independent of the implementation. - -### 2. Research and Literature Grounding - -Use web search and the Task tool to find authoritative sources for verification data: - -- Published experimental data with known conditions and measured outcomes -- Analytical solutions for simplified or limiting cases -- Benchmark problems from the domain with published results -- Textbook worked examples with known answers - -For each source, save a summary to `{{lisa_root}}/references/retrieved/` if not already present. - -**Every scenario must cite at least one authoritative source.** If you cannot find a -source for a scenario, mark it `[NEEDS_SOURCE]` and explain what you looked for. - -### 3. Write DDV Scenarios - -Create or update `{{lisa_root}}/ddv/scenarios.md` with scenarios using this format: - -```markdown -# DDV Scenarios - -## DDV-001: [Short descriptive title] - -**Physical behavior:** [What physical/domain behavior this tests. One paragraph.] - -**Conditions:** [Input parameters, boundary conditions, initial state — everything -needed to set up this test case. Be precise: specific numerical values with units.] - -**Expected output:** [The expected result with units. Include tolerance.] - -**Tolerance:** [±X% or ±X units] — Justification: [why this tolerance is appropriate, -citing the source's reported accuracy or the method's known error bounds] - -**Source:** [Full citation: Author(s), Year, Title, DOI/URL. Equation/table/figure number.] - -**Pass relevance:** [Which spiral pass(es) should be able to satisfy this scenario, -based on the scope progression in spiral-plan.md. E.g., "Pass 1+" or "Pass 3+"] - -**Category:** [One of: unit-function | model-behavior | system-integration | limiting-case | reference-data] - -**Visual:** [What plot or diagram should the Validate phase generate to verify this scenario -visually? Describe axes, overlays, and what behavior to look for. Write "None" only for -unit-function scenarios that are simple numeric spot-checks.] - ---- -``` - -### 4. Scenario Categories - -Write scenarios across these categories: - -1. **Unit-function** — Known input → known output for individual computations. - Source: textbook examples, analytical solutions, hand calculations. - -2. **Model-behavior** — Expected trends and relationships over parameter ranges. - Source: published parametric studies, physical laws (monotonicity, conservation). - -3. **System-integration** — End-to-end behavior of the composed system. - Source: published benchmark problems, experimental datasets. - -4. **Limiting-case** — Behavior at extreme or degenerate parameter values. - Source: analytical solutions for simplified cases, asymptotic analysis. - -5. **Reference-data** — Comparison against published experimental or computational data. - Source: peer-reviewed experimental measurements, validated computational benchmarks. - -### 4b. Visual Verification Guidance - -Which scenario categories should include a `**Visual:**` specification: - -- **model-behavior** — Always. Plot the expected trend over parameter range with the verification point(s) marked. -- **system-integration** — Always. Plot end-to-end output against published benchmark data. -- **limiting-case** — Always. Plot approach to the known analytical value as the parameter moves toward the limit. -- **reference-data** — Always. Plot computed vs. published data with error bands. -- **unit-function** — Optional. Include only if the function has interesting behavior over its valid input range (e.g., a non-linear curve). Simple numeric spot-checks (known input → known output) do not need visuals. - -### 5. Update DDV Manifest - -Create or update the `## Manifest` section at the top of `{{lisa_root}}/ddv/scenarios.md`. -The manifest table tracks all scenarios and their status: - -```markdown -## Manifest - -| Scenario | Category | Pass Relevance | Source | Visual | Status | -|----------|----------|----------------|--------|--------|--------| -| DDV-001 | [cat] | Pass 1+ | [cite] | Yes/None | PENDING | -| DDV-002 | [cat] | Pass 1+ | [cite] | Yes/None | PENDING | -``` - -All scenarios start as `PENDING`. The Validate phase will update status to `TESTED` or `DEFERRED` -as executable tests are written and run. - -### 6. Create Completion Marker - -After writing all scenarios, create `{{lisa_root}}/ddv/DDV_COMPLETE.md`: - -```markdown -# DDV Agent Complete - -Scenarios written: [count] -Categories: [count per category] -Sources cited: [count unique sources] -Scenarios with visual specifications: [count]/[total] -Earliest pass relevance: Pass [N] -``` - -## DDV Feedback - -If `{{lisa_root}}/ddv/ddv-feedback.md` exists, the human has reviewed your scenarios and provided -feedback. Read it carefully. Update affected scenarios. Do not discard previous work — refine it -based on the feedback. Address every item in the feedback file. - -## Rules - -- **Visuals are the preferred way to surface results for human review.** Every scenario that checks a trend, comparison, limiting case, or parameter range should have a `**Visual:**` field. Only simple numeric spot-checks may omit it. -- **Do NOT write any code.** No test files, no source files, no scripts. Scenarios are markdown only. -- **Do NOT read implementation code.** Your scenarios must be derived independently from literature and domain knowledge. -- **Every expected value must have a source.** No "expected: approximately X" without a citation. -- **Be precise about conditions.** A scenario is useless if the conditions are ambiguous. Specify every parameter needed to reproduce the result. -- **Tolerances must be justified.** Don't pick arbitrary percentages. Justify from the source's reported accuracy, the method's known error bounds, or the measurement uncertainty. -- **Cover the scope progression.** Write scenarios for each pass's scope level in spiral-plan.md. Early passes need fewer, simpler scenarios. Later passes need scenarios that test higher fidelity. - -## Output - -Provide a brief summary of: -- How many scenarios were written, by category -- Key sources used -- Which passes are covered -- How many scenarios include visual specifications (and which categories lack them) -- Any gaps where sources could not be found (`[NEEDS_SOURCE]`) diff --git a/prompts/PROMPT_explore.md b/prompts/PROMPT_explore.md new file mode 100644 index 0000000..5555bed --- /dev/null +++ b/prompts/PROMPT_explore.md @@ -0,0 +1,64 @@ +# Exploration Phase — Lisa Loop (Side-branch Investigation) + +You are a research engineer conducting a focused investigation into an alternative approach. +This is a lightweight side-branch — your goal is to answer a specific question or test a +hypothesis, not to produce production code. + +You have no memory of previous invocations. The filesystem is your shared state. Read it carefully. + +## Context + +The human has paused the main spiral at a review gate and wants to explore an idea before +deciding whether to continue, redirect, or finalize. Your exploration runs on an isolated +git branch. The human will decide whether to merge or discard your work afterward. + +## Your Task + +### 1. Read Context + +Read the exploration question provided in the extra context above. + +Then read: +- `ASSIGNMENT.md` — the overall project goals +- `{{lisa_root}}/methodology/methodology.md` — current methodology +- `{{lisa_root}}/spiral/pass-{{pass}}/review-package.md` — latest results (if exists) +- `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md` — current visual evidence (if exists) + +### 2. Investigate + +Conduct a focused investigation to answer the exploration question: +- Modify code, run experiments, produce results +- Keep changes focused — don't rewrite the entire codebase +- Generate comparison evidence: plots comparing your approach against current results +- Store plots in `{{lisa_root}}/spiral/pass-{{pass}}/explore-{explore_id}/plots/` + +### 3. Write Findings + +Write `{{lisa_root}}/spiral/pass-{{pass}}/explore-{explore_id}/findings.md`: + +```markdown +# Exploration Findings + +## Question +[The exploration question] + +## Approach +[What you tried] + +## Results +[What you found — include plot references] + +## Comparison with Current Approach +[How this compares to the main-line results] + +## Recommendation +[Should the main spiral adopt this approach? Why/why not?] +``` + +### Rules + +- Do NOT modify `{{lisa_root}}/methodology/methodology.md` or `{{lisa_root}}/state.toml` +- Do NOT modify `{{lisa_root}}/methodology/plan.md` +- Focus: answer the question, not rewrite the system +- Generate visual evidence for your findings — plots are the primary review artifact +- Keep the investigation small: aim for 2-3 build iterations worth of work diff --git a/prompts/PROMPT_finalize.md b/prompts/PROMPT_finalize.md index 304445e..182dd6b 100644 --- a/prompts/PROMPT_finalize.md +++ b/prompts/PROMPT_finalize.md @@ -14,16 +14,16 @@ Read **all** of the following: - `ASSIGNMENT.md` — the assignment, especially the "Deliverables" and "Deliverable format" sections - `{{lisa_root}}/STACK.md` — project-specific operational guidance -- The review package for the final pass: `{{lisa_root}}/spiral/pass-N/review-package.md` (where N is the current pass) -- `{{lisa_root}}/spiral/pass-N/progress-tracking.md` — progress tracking -- `{{lisa_root}}/spiral/pass-N/system-validation.md` — validation results +- The review package for the final pass: `{{lisa_root}}/spiral/pass-{{pass}}/review-package.md` (where N is the current pass) +- `{{lisa_root}}/spiral/pass-{{pass}}/progress-tracking.md` — progress tracking +- `{{lisa_root}}/spiral/pass-{{pass}}/system-validation.md` — validation results - `{{lisa_root}}/methodology/methodology.md` — the methodology - `{{lisa_root}}/methodology/assumptions-register.md` — assumptions and limitations ### 2. Produce Deliverables Read the "Deliverables" and "Deliverable format" sections of `ASSIGNMENT.md`. -When producing deliverables, include or reference visual verification evidence. If the deliverable is a report, embed the most important plots. If code/data, reference `{{lisa_root}}/plots/REVIEW.md`. +When producing deliverables, include or reference visual verification evidence. If the deliverable is a report, embed the most important plots. If code/data, reference the per-pass plot reviews at `{{lisa_root}}/spiral/pass-*/plots/REVIEW.md`. Produce the specified deliverables at the locations described in the brief. If the brief doesn't specify locations, place deliverables in the project root. @@ -42,7 +42,7 @@ Create `{{lisa_root}}/output/audit-summary.md`: [List with paths] ## Validation Status -- DDV verification: [pass/total] +- Bounding tests: [pass/total] (L1: [N], L2: [N], L3: [N]) - Software tests: [pass/total] - Integration tests: [pass/total] - Sanity checks: [pass/total] @@ -50,7 +50,7 @@ Create `{{lisa_root}}/output/audit-summary.md`: ## Key Evidence - Methodology: {{lisa_root}}/methodology/methodology.md -- **Visual verification evidence: {{lisa_root}}/plots/REVIEW.md** (primary review artifact) +- **Visual verification evidence: {{lisa_root}}/spiral/pass-*/plots/REVIEW.md** (per-pass plot reviews) - Progress history: {{lisa_root}}/spiral/pass-*/progress-tracking.md - Full spiral history: {{lisa_root}}/spiral/ diff --git a/prompts/PROMPT_init.md b/prompts/PROMPT_init.md new file mode 100644 index 0000000..1bdd9f1 --- /dev/null +++ b/prompts/PROMPT_init.md @@ -0,0 +1,89 @@ +# Init Agent — Project Structure Discovery + +You are the Init Agent for Lisa Loop. Your job is to examine the current working directory and resolve the project structure so that subsequent agents know where source code, tests, and build commands live. + +## Instructions + +### 1. Examine the working directory + +Look at the files and directories present in the project root. Identify: + +- **Language and runtime** — from file extensions, build files (Cargo.toml, package.json, pyproject.toml, CMakeLists.txt, Makefile, go.mod, etc.), CI configs +- **Directory structure** — where source code lives, how modules are organized +- **Build system** — what commands build the project +- **Existing test infrastructure** — test framework, test directories, test commands +- **Key entry points** — main files, library roots, public interfaces + +### 2. Write `.lisa/CODEBASE.md` + +Write a concise codebase summary to `{{lisa_root}}/CODEBASE.md` with these sections: + +```markdown +# Codebase Summary + +## Project Type + + +## Language & Runtime + + +## Build System + + +## Directory Structure + + +## Test Infrastructure + + +## Key Modules + +``` + +For **greenfield projects** (empty directory or only configuration files): +- Set Project Type to "greenfield" +- Note that no source code exists yet +- If ASSIGNMENT.md mentions a technology preference, note the conventional directory layout for that ecosystem +- Do NOT create any source or test directories — the scope agent handles this + +For **existing codebases**: +- Map the actual structure you find +- Identify existing test directories and frameworks +- Note any CI configuration that reveals build/test commands + +### 3. Update `lisa.toml` + +Read the current `lisa.toml` and update the `[paths]` and `[commands]` sections based on what you discovered. + +For **existing codebases**, set paths to match discovered locations: +```toml +[paths] +source = ["src"] # actual source directories found +tests_bounds = "tests/bounds" # create if it doesn't exist (with phenomenon/, composition/, system/ subdirs) +tests_software = "tests" # map to existing test directory +tests_integration = "tests/integration" # create if needed + +[commands] +build = "cargo build" # discovered build command +test_all = "cargo test" # discovered test command +``` + +For **greenfield projects**, leave `[paths]` empty — the scope agent will resolve them when it selects the technology stack: +```toml +[paths] +source = [] +tests_bounds = "" +tests_software = "" +tests_integration = "" +``` + +If you create any new directories (like `tests/bounds/` for an existing project that lacks bounding test infrastructure), add a `.gitkeep` file in them. For bounding tests, create the three subdirectories: `phenomenon/`, `composition/`, `system/`. + +### 4. Rules + +- Do NOT modify any existing source code files +- Do NOT create source directories for greenfield projects +- Do NOT modify ASSIGNMENT.md +- Do NOT modify `.lisa/state.toml` +- Keep CODEBASE.md concise — under 100 lines +- When in doubt about a path, use the most conventional location for the detected ecosystem diff --git a/prompts/PROMPT_refine.md b/prompts/PROMPT_refine.md index c3313a7..de247c5 100644 --- a/prompts/PROMPT_refine.md +++ b/prompts/PROMPT_refine.md @@ -20,7 +20,7 @@ Read **all** of the following: - `{{lisa_root}}/STACK.md` — project-specific operational guidance. **Pay particular attention to the "Resolved Technology Stack" section** — all implementation plan tasks you write must reference the concrete language, libraries, and tools specified there. - `{{lisa_root}}/methodology/methodology.md` — the current methodology - `{{lisa_root}}/methodology/plan.md` — the current implementation plan -- `{{lisa_root}}/ddv/scenarios.md` — DDV verification scenarios and manifest +- `{{lisa_root}}/skills/engineering-judgment.md` — the bounding methodology agents follow - `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` — what success looks like - `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — scope progression across passes (read this to determine the scope and fidelity target for this pass) @@ -34,7 +34,7 @@ If this is **Pass N > 1**: - Read `{{lisa_root}}/spiral/pass-{N-1}/system-validation.md` — what validation checks passed/failed - Read `{{lisa_root}}/spiral/pass-{N-1}/execution-report.md` — previous execution results - Read `{{lisa_root}}/spiral/pass-{N-1}/human-redirect.md` — human guidance (if file exists) -- Read any files in `{{lisa_root}}/spiral/pass-{N-1}/reconsiderations/` — unresolved methodology or DDV disagreement issues from build +- Read any files in `{{lisa_root}}/spiral/pass-{N-1}/reconsiderations/` — unresolved methodology issues from build ### 2. Research Delegation @@ -69,14 +69,7 @@ Do not simply paste subagent output — integrate it with your own reasoning. If `{{lisa_root}}/spiral/pass-{N-1}/reconsiderations/` contains unresolved issues, resolve each one: -**For DDV disagreements** (`ddv-disagreement-*.md`): -1. Go back to the authoritative source paper cited by both the test and the implementation -2. Determine which interpretation is correct -3. If the **test was wrong**: update the corresponding DDV scenario in `{{lisa_root}}/ddv/scenarios.md` with the correct expected value. -4. If the **implementation was wrong**: the task will be re-attempted in this pass's build phase -5. Document your adjudication in the refine summary - -**For methodology issues** (other reconsideration files): +**For methodology issues:** 1. Evaluate the proposed alternative 2. Update `{{lisa_root}}/methodology/methodology.md` if the alternative is accepted 3. Update verification cases if the methodology change affects expected values @@ -115,27 +108,26 @@ After any methodology change: - Add new entries to `{{lisa_root}}/validation/reference-data.md` (format: `RD-NNN`) These documents are checked during every validation phase. Keep them current. -### 5. Assign DDV Scenarios to Tasks - -Read `{{lisa_root}}/ddv/scenarios.md` and assign relevant scenario IDs to each task's -`**DDV Scenarios:**` field in the plan. A task should reference scenarios whose physical -behaviors it is responsible for enabling. +### 5. Identify Bounding Check Requirements -If methodology changes in this pass may invalidate existing DDV scenarios (e.g., changing -the governing equations or valid parameter ranges), flag this explicitly in the refine -summary so the human can decide whether to re-run the DDV Agent. +For each task in the plan, identify what bounding checks should be derived as part of +implementation, following the engineering judgment skill in `{{lisa_root}}/skills/engineering-judgment.md`: -### 6. Update DDV Scenarios +- Which phenomena need Level 1 (phenomenon) bounds? +- Which compositions need Level 2 (composition) bounds? +- Does the system output need a Level 3 (system) independent estimate? -If methodology changes affect expected values or valid ranges, update the corresponding -DDV scenarios in `{{lisa_root}}/ddv/scenarios.md`. For new methods that need verification, -add new scenario sketches (the DDV Agent will fully ground them if re-run). Each scenario -should have expected values with sources. +Add bounding check items to each task's checklist (e.g., "Derive phenomenon bounds for [X]"). -### 7. Update Implementation Plan +### 6. Update Implementation Plan Read `{{lisa_root}}/spiral/pass-0/spiral-plan.md` to determine the scope and fidelity target for this pass. +**Task cap:** Create at most **{{max_tasks_per_pass}}** tasks for this pass. If the current scope +requires more, shrink the pass scope and defer remaining work to subsequent passes. Update the +spiral plan accordingly. Splitting a pass into smaller passes is always preferred over creating +a large pass. + Update `{{lisa_root}}/methodology/plan.md`: - **For Pass 1:** The scope phase created a structural skeleton with task names, ordering, methodology references, and dependencies — but no checklists. Now that the methodology is fully specified, add detailed checklists to each existing task based on the complete equations and implementation notes. Split or merge tasks if the fully specified methodology reveals the original sizing was wrong. - **For Pass N > 1:** Add new tasks for this pass that address ONLY the current pass's scope subset (not the full problem) @@ -143,8 +135,8 @@ Update `{{lisa_root}}/methodology/plan.md`: - Each task references a methodology section - Tasks are ordered bottom-up (utilities → core equations → higher-level models → integration → runner) - Each task is sized for one Ralph iteration (max 5 implementation items) -- Tasks do NOT include DDV test items — the Validate phase writes executable tests from DDV scenarios -- Every task whose implementation can be visually verified should include at least one `- [ ] [Visual: ...]` checklist item. If the task's DDV scenarios have `**Visual:**` fields, the corresponding plots should appear as checklist items. Store plots in `{{lisa_root}}/plots/`. +- Every task whose implementation can be visually verified should include at least one `- [ ] [Visual: ...]` checklist item. Store plots in `{{lisa_root}}/spiral/pass-{{pass}}/plots/`. +- Tasks should include bounding check items where applicable: `- [ ] Derive phenomenon bounds for [X]` following the engineering judgment skill. **Task format:** ```markdown @@ -152,7 +144,7 @@ Update `{{lisa_root}}/methodology/plan.md`: - **Status:** TODO | IN_PROGRESS | DONE | BLOCKED - **Pass:** N - **Methodology:** [section ref] -- **DDV Scenarios:** DDV-001, DDV-003 (or "none") +- **Bounding Checks:** L1 for [phenomenon], L2 for [composition] (or "none") - **Checklist:** - [ ] [Implement X] - [ ] [Implement Y] @@ -162,8 +154,6 @@ Update `{{lisa_root}}/methodology/plan.md`: - **Dependencies:** [task refs or "None"] ``` -Note: no DDV test items in the plan. The Validate phase writes executable tests from DDV scenarios. - **Task rules:** - Order tasks bottom-up: utilities → core equations → higher-level models → integration - Each task completable in a single build iteration @@ -171,9 +161,9 @@ Note: no DDV test items in the plan. The Validate phase writes executable tests - Infrastructure tasks come first if needed - Tag every task with `**Pass:** N` for the current pass -### 8. Produce Refine Summary +### 7. Produce Refine Summary -Create `{{lisa_root}}/spiral/pass-N/refine-summary.md`: +Create `{{lisa_root}}/spiral/pass-{{pass}}/refine-summary.md`: If nothing changed: write only "No methodology changes this pass." diff --git a/prompts/PROMPT_scope.md b/prompts/PROMPT_scope.md index f2cede8..30a7d91 100644 --- a/prompts/PROMPT_scope.md +++ b/prompts/PROMPT_scope.md @@ -9,6 +9,15 @@ You are a research engineer establishing the scope, acceptance criteria, methodo ### Phase 1: READ INPUTS Read `ASSIGNMENT.md`, `{{lisa_root}}/STACK.md`, and skim `{{lisa_root}}/references/core/`. +If `{{lisa_root}}/CODEBASE.md` exists, read it carefully. This means you are working with an +existing codebase. Scope your work as modifications to the existing system — the methodology +should describe what is being added or changed, not the entire system. Bounding checks should +include regression coverage for existing behavior that might be affected. + +If the `[paths]` section in `lisa.toml` has empty `source` and test paths (greenfield project), +you must resolve them during technology stack selection in Phase 3: create appropriate source +and test directories for the chosen language/framework and update `lisa.toml` with the paths. + Pay particular attention to the **"Approach"** section of `ASSIGNMENT.md`. If the human has stated a methodological preference (e.g., "simplest method possible," "state of the art," or a specific method/paper to follow), respect it throughout all subsequent phases. If the @@ -20,27 +29,25 @@ Spawn the **Literature Survey** and **Environment Probe** subagents (they are in ### Phase 3: FIRST SYNTHESIS Synthesize subagent results. Select methodology and technology stack. Write: -- `{{lisa_root}}/methodology/methodology.md` -- `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` -- Update `{{lisa_root}}/STACK.md` with resolved technology stack +- `{{lisa_root}}/methodology/methodology.md` — read spec at `{{lisa_root}}/prompts/scope/methodology_spec.md` +- `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` (format below) +- Update `{{lisa_root}}/STACK.md` — read spec at `{{lisa_root}}/prompts/scope/stack_selection_spec.md` ### Phase 4: DELEGATE VALIDATION Spawn the **Validation Research** and **Test Framework Research** subagents (they are independent — delegate back-to-back). Wait for results. ### Phase 5: FINAL SYNTHESIS Synthesize subagent results. Write all remaining artifacts: -- `{{lisa_root}}/methodology/plan.md` -- `{{lisa_root}}/ddv/scenarios.md` — initial DDV scenario sketches (the DDV Agent will refine these) -- `{{lisa_root}}/spiral/pass-0/literature-survey.md` (review/augment subagent output) -- `{{lisa_root}}/spiral/pass-0/spiral-plan.md` -- `{{lisa_root}}/validation/sanity-checks.md` -- `{{lisa_root}}/validation/limiting-cases.md` -- `{{lisa_root}}/validation/reference-data.md` +- `{{lisa_root}}/methodology/plan.md` — read spec at `{{lisa_root}}/prompts/scope/implementation_plan_spec.md` +- `{{lisa_root}}/spiral/pass-0/literature-survey.md` — read spec at `{{lisa_root}}/prompts/scope/literature_survey_spec.md` +- `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — read spec at `{{lisa_root}}/prompts/scope/spiral_plan_spec.md` +- Validation artifacts (`sanity-checks.md`, `limiting-cases.md`, `reference-data.md`) — read spec at `{{lisa_root}}/prompts/scope/validation_specs.md` - Create or update the project root `.gitignore` with patterns appropriate for the resolved technology stack (build outputs, dependency caches, virtual environments, IDE files, OS files, - framework-specific artifacts). Base patterns on the resolved stack in `{{lisa_root}}/STACK.md`. - If a `.gitignore` already exists, merge new patterns without removing existing ones. -- `{{lisa_root}}/spiral/pass-0/PASS_COMPLETE.md` (last) + framework-specific artifacts). If a `.gitignore` already exists, merge new patterns without removing existing ones. +- `{{lisa_root}}/spiral/pass-0/PASS_COMPLETE.md` (last — format below) + +**Important:** Before writing each artifact, read its spec file for the required format and guidance. The spec files are at `{{lisa_root}}/prompts/scope/`. Each contains the template, rules, and examples for that artifact. ## Scope Feedback (Refinement Re-invocation) @@ -72,33 +79,8 @@ candidate methods for [problem from ASSIGNMENT.md]. For each candidate: provide (author(s), year, title, DOI/URL), approach description, fidelity level, assumptions, valid range, pros/cons for our problem. Evaluate alternatives. Save paper summaries to {{lisa_root}}/references/retrieved/ with citations and key equations. Write the complete literature -survey to {{lisa_root}}/spiral/pass-0/literature-survey.md using this template: - -# Literature Survey - -## Methods Surveyed - -### [Topic/Phenomenon A] - -#### [Method 1 Name] -- **Source:** [Author(s), Year, Title, DOI/URL] -- **Approach:** [Brief description] -- **Fidelity:** [Low / Medium / High] -- **Assumptions:** [Key assumptions] -- **Valid range:** [Where it applies] -- **Pros:** [Advantages for our problem] -- **Cons:** [Disadvantages or limitations] -- **Available:** [YES / NEEDS_PAPER] - -#### Recommended Approach for [Topic A] -[Which method(s) to use and why] - -## Papers Retrieved -[List papers saved to {{lisa_root}}/references/retrieved/ with full citations] - -## Papers Needed -[Papers flagged with NEEDS_PAPER that the human should provide] - +survey to {{lisa_root}}/spiral/pass-0/literature-survey.md using the template in +{{lisa_root}}/prompts/scope/literature_survey_spec.md. Rules: Every method candidate must cite a peer-reviewed source. Never fabricate equations from memory. Prefer open-access papers. Document alternatives considered for each phenomenon." @@ -129,229 +111,24 @@ Prompt pattern: "Read ASSIGNMENT.md and {{lisa_root}}/methodology/methodology.md for: limiting cases where the answer is known analytically, reference datasets for comparison, conservation laws that must be satisfied, order-of-magnitude estimates from first principles, and cross-validation opportunities using independent methods. Return -structured findings organized by category: - -## Known Limiting Cases -- [Case]: When [condition], result should be [value] because [reason]. Source: [citation]. - -## Reference Data -- [Dataset]: [Source citation], [what it measures], [how to compare]. - -## Conservation Laws -- [Law]: [Statement], [how to check in our system]. - -## Order-of-Magnitude Estimates -- [Quantity]: Estimate [value] [units] based on [reasoning]. - -## Cross-Validation Opportunities -- [Method]: [How it can corroborate results]." +structured findings organized by category (Known Limiting Cases, Reference Data, +Conservation Laws, Order-of-Magnitude Estimates, Cross-Validation Opportunities)." ### Test Framework Research subagent Delegate when: After technology stack is selected (Phase 4). Prompt pattern: "Given this technology stack: [language] with [test framework]. Research -how to implement a three-category test structure: DDV tests ({{tests_ddv}}/), software tests -({{tests_software}}/), and integration tests ({{tests_integration}}/). DDV tests need L0/L1 -level filtering. Return: - -## Test Commands -- Run all tests: [command] -- Run DDV tests only: [command] -- Run software tests only: [command] -- Run integration tests only: [command] -- Run DDV L0 only: [command] -- Run DDV L1 only: [command] - -## Configuration Required -[Any config files, settings, or infrastructure needed to support the test structure] - -## Infrastructure Task Description -[What needs to be set up as the first task in the implementation plan — concrete steps]" - ---- - -## Artifacts to Produce - -You must create **all** of the following files. Do not skip any. +how to implement a test structure with: bounding tests ({{tests_bounds}}/ with phenomenon/, +composition/, system/ subdirectories), software tests ({{tests_software}}/), and integration +tests ({{tests_integration}}/). Return: test commands for each category, configuration +required, and an infrastructure task description for what needs to be set up." --- -### 1. `{{lisa_root}}/methodology/methodology.md` — The Methodology Document +## Inline Artifact Specs -**This is the central technical artifact.** It identifies the recommended methods, cites source papers, lists key equations by name/number, and documents assumptions and valid ranges. +The following artifacts are small enough to specify here directly (no external spec file needed). -**Division of labor:** The methodology created here is an *initial specification*. It identifies the recommended method, cites the source paper, lists key equations by name/number, and documents assumptions and valid ranges. It does NOT contain full equation derivations with every variable defined — that level of detail is the refine phase's job in Pass 1. This intentional fidelity gap is what gives the first refine phase meaningful work: transforming a method recommendation into a complete, implementable specification. - -Populate `{{lisa_root}}/methodology/methodology.md`: - -```markdown -# Methodology - -## Phenomenon -[What this project models — from ASSIGNMENT.md] - -## Candidate Methods - -### [Method 1] -- **Source:** [Citation] -- **Approach:** [Description] -- **Fidelity:** [Low / Medium / High] -- **Pros:** [For our problem] -- **Cons:** [Limitations] - -### [Method 2] -... - -## Recommended Approach -[Which method and why, considering spiral progression and the human's approach preference from ASSIGNMENT.md] -[If the human asked for simplicity, justify why this method is the simplest that can meet the acceptance criteria] -[If the human asked for state of the art, justify why this is the best available method] -[If there is a mismatch between the requested approach and the acceptance criteria, flag it explicitly] - -## Key Equations -[Identify by name and equation number from the source paper — e.g., "Eq. 12 in Faltinsen (1990)" or "ITTC-57 friction line." Do NOT write out the full mathematical expressions here — that is the refine phase's job. If a specific paper is needed but not yet available, flag with [NEEDS_PAPER].] - -## Assumptions -[List all assumptions] - -## Valid Range -[Parameter ranges where the chosen method applies] -``` - -If the problem has distinct sub-topics (e.g., frictional resistance, wave resistance, added resistance), organize the methodology into clearly separated **sections** within the single document. Each section follows the same structure above. - ---- - -### 2. `{{lisa_root}}/methodology/plan.md` — Implementation Plan (Structural Skeleton) - -Initial implementation plan with task structure for Pass 1. At this stage you know *what* -needs to be implemented and in what order, but the equations are not yet fully specified — -that detail comes from the refine phase. Keep this plan at the structural level: task names, -ordering, methodology references, and dependencies. Do NOT write detailed checklists — the -refine phase will flesh those out once the methodology is complete. - -```markdown -# Implementation Plan - -## Tasks - -### Task 1: [Short name] -- **Status:** TODO -- **Pass:** 1 -- **Methodology:** [section ref] -- **Dependencies:** [task refs or "None"] - -### Task 2: [Short name] -- **Status:** TODO -- **Pass:** 1 -- **Methodology:** [section ref] -- **Dependencies:** [task refs or "None"] -``` - -**Task rules:** -- Order tasks bottom-up: utilities → core equations → higher-level models → integration -- Each task should be scoped for a single Ralph iteration -- Infrastructure tasks (setup, test framework, etc.) come first if needed -- Tag every task with `**Pass:** 1` for Pass 1 tasks -- Pass 2+ tasks can be sketched with TODO placeholders -- Tasks do NOT include DDV test items — DDV tests are written from scenarios by the Validate phase -- Do NOT add checklists — the refine phase adds those after completing the methodology - ---- - -### 3. Initial DDV Scenario Sketches — `{{lisa_root}}/ddv/scenarios.md` - -Write initial DDV scenario sketches directly into `{{lisa_root}}/ddv/scenarios.md`. These are preliminary -verification scenarios that the DDV Agent will later refine and expand with full literature grounding. -Place scenarios after the `## Scenarios` heading (the `## Manifest` section at the top is managed by later phases). - -Use this simplified format for each scenario: - -```markdown -## DDV-001: [Short descriptive title] - -**Physical behavior:** [What physical/domain behavior this tests] -**Level:** L0 | L1 -**Conditions:** [Input parameters with units] -**Expected output:** [Expected result with units and tolerance] -**Source:** [Citation or reasoning for expected value] -**Category:** [unit-function | model-behavior | system-integration | limiting-case | reference-data] -**Visual:** [What plot to generate, or "None" for simple spot-checks] -``` - -L0 = individual function tests (known input → known output). L1 = model-level tests (behavior over valid range). -These sketches do not need the full rigor of DDV Agent scenarios — they establish the verification -intent that the DDV Agent will ground in authoritative literature. - ---- - -### 4. Technology Stack Selection — `{{lisa_root}}/STACK.md` + Environment Probing - -**This artifact ensures that all subsequent agents use a concrete, verified technology stack rather than making implicit choices.** - -#### Reason About Stack Selection - -Before probing the environment, reason about the best technology stack for this project: - -- **Computational requirements:** Is the problem compute-bound (favoring a compiled language) or I/O-bound / prototyping-oriented (where a scripting language suffices)? -- **Ecosystem:** Are there domain-specific libraries that favor a particular language? -- **Human preferences:** Read the "Technology Preferences" section of `ASSIGNMENT.md`. If the human stated preferences, respect them. If blank, choose freely. - -#### Probe the Local Environment - -The Environment Probe subagent has already checked what runtimes and tools are available. -Synthesize its report here: verify it covers all runtimes needed for your chosen stack, -and note any gaps that require human resolution. - -#### Handle Two Categories of Dependencies - -**1. Runtimes and toolchains** (language interpreters, compilers, system-level libraries, etc.): - -Check if these are present by running version commands. If a required runtime is **not available**: -- Do **NOT** attempt to install it -- Create `{{lisa_root}}/spiral/pass-0/environment-resolution.md` listing what is missing: - -```markdown -# Environment Resolution Required - -## Missing Runtimes / Toolchains - -### [Tool Name] -- **What:** [e.g., Python 3.10+] -- **Why needed:** [e.g., Primary implementation language] -- **Suggested install:** [e.g., `apt install python3` or `pyenv install 3.11`] -- **Alternative:** [Could a different stack choice avoid this? If so, describe.] - -## Status -Waiting for human resolution before proceeding. -``` - -If all required runtimes are present, do **NOT** create this file (or create it empty). - -**2. Package-level dependencies** (packages, crates, modules, etc.): - -Install these directly using the appropriate package manager. These are routine development dependencies: -- Run the install command using the appropriate package manager -- Verify each install succeeded -- Record installed versions in {{lisa_root}}/STACK.md - -#### Populate {{lisa_root}}/STACK.md - -Update the "Resolved Technology Stack" section of `{{lisa_root}}/STACK.md`: - -- **Language & Runtime:** Fill with verified language and version (e.g., "Python 3.11.5 (verified present)") -- **Key Dependencies:** List all installed packages with versions -- **Test Framework:** Specify the chosen test framework and version -- **Stack Justification:** Brief reasoning for the technology choices - -Fill in all command sections (Setup, Build, Test, Lint, etc.) with **concrete, tested commands** — no more placeholders. If the human pre-filled any command sections before running scope, verify those commands work (run them) rather than overwriting them. - -**Backward compatibility:** If {{lisa_root}}/STACK.md already has concrete (non-placeholder) commands filled in by the user, verify they work and keep them. Only populate sections that contain placeholders or template text. - ---- - -### 5. System-Level Files - -#### `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` +### Acceptance Criteria — `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` ```markdown # Acceptance Criteria @@ -361,163 +138,51 @@ Fill in all command sections (Setup, Build, Test, Lint, etc.) with **concrete, t ## Success Criteria [For each key output:] -- **[Output name]:** [Target value or range] [units] — accuracy needed: [±X or X%] +- **[Output name]:** [Target value or range] [units] — accuracy needed: [+/-X or X%] - [Justification for accuracy requirement] ## Decision Context [What decisions will be made based on this answer? What accuracy is needed for those decisions?] ``` -#### `{{lisa_root}}/validation/sanity-checks.md` - -```markdown -# Sanity Checks - -These are engineering judgment checks to be executed after every spiral pass. -A failure on any check indicates a likely error and should block acceptance. - -## Order of Magnitude -- [ ] [Quantity] should be approximately [value] [units] (±[order of magnitude]) - - **Reasoning:** [Why this magnitude is expected] - -## Expected Trends -- [ ] When [parameter] increases, [quantity] should [increase/decrease/remain constant] - - **Reasoning:** [Physical justification] +### Assumptions Register — `{{lisa_root}}/methodology/assumptions-register.md` -## Physical Bounds -- [ ] [Quantity] must be [positive / in range [a,b] / less than X] - - **Reasoning:** [Physical constraint] - -## Conservation -- [ ] [Conserved quantity] should be preserved to within [tolerance] - - **Check method:** [How to verify] - -## Dimensional Analysis -- [ ] All outputs have correct dimensions/units - - **Check method:** [How to verify] - -## Red Flags -- [ ] [Specific condition that would indicate a clearly wrong answer] -``` - -#### `{{lisa_root}}/validation/limiting-cases.md` - -Extract the limiting cases from your validation research and format them using the `LC-NNN` format (e.g., `LC-001`, `LC-002`). Each entry should include: case description, the condition, expected result, source/reasoning, and a pass/fail status placeholder. - -#### `{{lisa_root}}/validation/reference-data.md` - -Extract the reference datasets from your validation research and format them using the `RD-NNN` format (e.g., `RD-001`, `RD-002`). Each entry should include: dataset description, source citation, what it measures, comparison method, and a pass/fail status placeholder. - -These are the living validation documents that will be checked during every validation phase and refined during methodology refinement phases. - -#### `{{lisa_root}}/spiral/pass-0/literature-survey.md` - -Survey of candidate methods, organized by topic/phenomenon: - -```markdown -# Literature Survey - -## Methods Surveyed - -### [Topic/Phenomenon A] - -#### [Method 1 Name] -- **Source:** [Author(s), Year, Title, DOI/URL] -- **Approach:** [Brief description] -- **Fidelity:** [Low / Medium / High] -- **Assumptions:** [Key assumptions] -- **Valid range:** [Where it applies] -- **Pros:** [Advantages for our problem] -- **Cons:** [Disadvantages or limitations] -- **Available:** [YES / NEEDS_PAPER — whether full paper is accessible] - -[Repeat for each candidate method] - -#### Recommended Approach for [Topic A] -[Which method(s) to use and why] - -### [Topic/Phenomenon B] -[Same structure] - -## Cross-Cutting Methods -[Any methods that span multiple topics] - -## Papers Retrieved -[List papers saved to {{lisa_root}}/references/retrieved/ with full citations] - -## Papers Needed -[Papers flagged with [NEEDS_PAPER] that the human should provide] -``` - -The **Literature Survey subagent** has produced this artifact. Review it, augment with your -own judgment if needed, and ensure it meets the template above. Verify that: -- Every method candidate cites a peer-reviewed source (author(s), year, title, DOI/URL) -- Alternatives are documented for each phenomenon -- Papers saved to `{{lisa_root}}/references/retrieved/` have proper citations and key equations - -#### `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — Scope Progression +If you identify any cross-cutting assumptions during scoping, add them to the existing template in `{{lisa_root}}/methodology/assumptions-register.md`. -The spiral plan MUST define how scope and fidelity increase per pass. Early passes test -the methodology on a SUBSET of the full problem — not the full scope at low fidelity. +### PASS_COMPLETE — `{{lisa_root}}/spiral/pass-0/PASS_COMPLETE.md` -**Calibrate the spiral plan to the human's approach preference** from `ASSIGNMENT.md`: -- If they want simplicity/minimum viable: plan fewer passes, stay with one method, widen - tolerances. The spiral may converge in 1-2 passes. -- If they want state of the art: plan for progressive method upgrades across passes, - tighter final tolerances, more validation. -- If they specified a particular method: build the spiral around that method's scope - progression (narrow → broad), not around method upgrades. -- If no preference stated: use balanced judgment. +Create this file **last**, after all other artifacts are complete. ```markdown -# Spiral Plan - -## Approach Philosophy -[Summarize the human's approach preference from ASSIGNMENT.md, or state "balanced (default)" -if none was given. Note any tension between the requested approach and the acceptance criteria.] +# Pass 0 — Scoping Complete -## Scope Progression +## Summary +[One paragraph summary of what was established] -| Pass | Scope subset | Fidelity | Acceptance (this pass) | Key question | -|------|-------------|----------|----------------------|--------------| -| 1 | [subset] | [level] | [±X%] | Does the approach work at all? | -| 2 | [broader] | [level] | [±X%] | Does it generalize across range? | -| 3 | [full] | [level] | [±X%] | Does coupling work? | -| 4 | [full] | [refined]| [±X%] | Converged? | +## Artifacts Produced +- {{lisa_root}}/STACK.md (resolved technology stack, concrete commands) +- {{lisa_root}}/methodology/methodology.md +- {{lisa_root}}/methodology/plan.md +- {{lisa_root}}/spiral/pass-0/acceptance-criteria.md +- {{lisa_root}}/spiral/pass-0/literature-survey.md +- {{lisa_root}}/spiral/pass-0/spiral-plan.md +- {{lisa_root}}/spiral/pass-0/environment-resolution.md (only if missing runtimes/toolchains) +- {{lisa_root}}/validation/sanity-checks.md +- {{lisa_root}}/validation/limiting-cases.md +- {{lisa_root}}/validation/reference-data.md -## Progress Tracking Expectations -[What quantities to track across passes, expected rate of change] +## Key Decisions +[List the most important scoping decisions made] -## Risk Areas -[Where methodology might need reconsideration, known difficult aspects] +## Open Questions for Human Review +[Anything that needs human input before proceeding to Pass 1] ``` -Example for ship resistance (5-25 kn, sea states 1-6), **balanced** approach: -- Pass 1: 12 kn, calm water, simplest method → ±50% -- Pass 2: 5-25 kn, calm water, add corrections → ±20% -- Pass 3: Full range + sea states 1-3 → ±10% -- Pass 4: Full scope, refined methods → ±5% - -Same problem, **minimum viable** approach: -- Pass 1: Full speed range, calm water, Holtrop-Mennen → ±15% -- Pass 2: Add sea state corrections → ±10% -(Fewer passes, simpler method, acceptance tolerances widened to match method capability) - -The refine phase reads this plan to scope tasks for the current pass. -The Validate phase writes executable DDV tests only for the current pass's scope subset. -Acceptance criteria are staged — early passes have wider tolerances. - --- -### 6. Assumptions Register - -#### `{{lisa_root}}/methodology/assumptions-register.md` - -If you identify any cross-cutting assumptions during scoping, add them to the existing template in `{{lisa_root}}/methodology/assumptions-register.md`. - ---- +## Additional Guidance -### 7. Complexity Assessment +### Complexity Assessment After surveying the literature and understanding the problem, assess whether this problem can be handled with a single methodology document and build loop, or whether it requires @@ -538,33 +203,22 @@ Criteria for modular decomposition (exceptional): If you recommend modular decomposition, document why in the spiral plan and organize the methodology into clearly separated sections. The code should be organized into corresponding modules in `{{source_dirs}}/`. But the spiral loop is still the same — one refine phase, -one DDV phase, one build loop. The modularity is in the content, not the process. +one build loop, one audit. The modularity is in the content, not the process. ---- - -### 8. Test Categorization Mechanism +### Test Categorization -The project uses three test categories that must be runnable independently: -- **DDV tests** (`{{tests_ddv}}/`) — Domain-Driven Verification tests written by the Validate phase from DDV scenarios +The project uses four test categories that must be runnable independently: +- **Bounding tests** (`{{tests_bounds}}/`) — First-principles bounding tests at three levels (phenomenon, composition, system), written by the Build phase following the engineering judgment skill in `{{lisa_root}}/skills/engineering-judgment.md` - **Software tests** (`{{tests_software}}/`) — Software quality tests written by the build phase - **Integration tests** (`{{tests_integration}}/`) — End-to-end tests written by the Build phase -Additionally, DDV tests have verification levels (Level 0: individual functions, Level 1: model level) that should be filterable. - -When resolving the test framework, also define and document the categorization mechanism: -- How are tests tagged/grouped by category? (markers, directories, naming conventions, test sets) -- How are DDV tests filtered by verification level? -- Does the framework need any configuration to support this? (e.g., marker registration, custom test runners) +Bounding tests are organized into three subdirectories: `phenomenon/`, `composition/`, `system/`. -Document the chosen mechanism in `{{lisa_root}}/STACK.md` by filling in the test command sections with -concrete commands that select each category. +When resolving the test framework, define and document the categorization mechanism in +`{{lisa_root}}/STACK.md` with concrete commands that select each category. Include any framework +configuration needed as the first infrastructure task in `{{lisa_root}}/methodology/plan.md`. -Include any framework configuration needed to make the categorization work as the first -infrastructure task in `{{lisa_root}}/methodology/plan.md`. - ---- - -### 9. Code Organization +### Code Organization Document the code layout in `{{lisa_root}}/STACK.md` (append to the existing file, do not overwrite): @@ -574,7 +228,7 @@ Document the code layout in `{{lisa_root}}/STACK.md` (append to the existing fil Source code is organized by logical module: - `{{source_dirs}}/` — All implementation code, organized by logical grouping - `{{source_dirs}}/common/` — Shared utilities (constants, unit conversions, interpolation, I/O) -- `{{tests_ddv}}/` — Domain-Driven Verification tests (written by Validate phase from DDV scenarios) +- `{{tests_bounds}}/` — First-principles bounding tests (phenomenon/, composition/, system/) - `{{tests_software}}/` — Software quality tests (written by build phase) - `{{tests_integration}}/` — End-to-end tests (written by Build phase) ``` @@ -583,38 +237,6 @@ If during scoping you identify shared infrastructure needs (e.g., common physica --- -### 10. `{{lisa_root}}/spiral/pass-0/PASS_COMPLETE.md` - -Create this file **last**, after all other artifacts are complete. - -```markdown -# Pass 0 — Scoping Complete - -## Summary -[One paragraph summary of what was established] - -## Artifacts Produced -- {{lisa_root}}/STACK.md (resolved technology stack, concrete commands) -- {{lisa_root}}/methodology/methodology.md -- {{lisa_root}}/methodology/plan.md -- {{lisa_root}}/ddv/scenarios.md (initial DDV scenario sketches) -- {{lisa_root}}/spiral/pass-0/acceptance-criteria.md -- {{lisa_root}}/spiral/pass-0/literature-survey.md -- {{lisa_root}}/spiral/pass-0/spiral-plan.md -- {{lisa_root}}/spiral/pass-0/environment-resolution.md (only if missing runtimes/toolchains) -- {{lisa_root}}/validation/sanity-checks.md -- {{lisa_root}}/validation/limiting-cases.md -- {{lisa_root}}/validation/reference-data.md - -## Key Decisions -[List the most important scoping decisions made] - -## Open Questions for Human Review -[Anything that needs human input before proceeding to Pass 1] -``` - ---- - ## Rules ### Literature Grounding @@ -626,7 +248,7 @@ Create this file **last**, after all other artifacts are complete. ### Visual Verification -- **Visuals are the preferred way to surface results for human review.** Every verification case and DDV scenario that checks a trend, comparison, limiting case, or parameter sweep should specify a `**Visual:**` field. Plots go in `{{lisa_root}}/plots/` and are documented in `{{lisa_root}}/plots/REVIEW.md`. +- **Visuals are the preferred way to surface results for human review.** Every verification case that checks a trend, comparison, limiting case, or parameter sweep should specify a `**Visual:**` field. Plots go in `{{lisa_root}}/spiral/pass-{{pass}}/plots/` and are documented in `{{lisa_root}}/spiral/pass-{{pass}}/plots/REVIEW.md`. ### Engineering Judgment diff --git a/prompts/PROMPT_validate.md b/prompts/PROMPT_validate.md deleted file mode 100644 index 0d0d2db..0000000 --- a/prompts/PROMPT_validate.md +++ /dev/null @@ -1,358 +0,0 @@ -# Validation Phase — Lisa Loop - -You are a senior engineer conducting system-level verification, validation, and progress -tracking. The system has been built and executed by the Build phase. Your job is to evaluate -the answer rigorously, write executable DDV tests from scenarios, and present the evidence -for human review. - -You have no memory of previous invocations. The filesystem is your shared state. - -**Visual verification principle:** Visuals are the preferred way to present verification evidence for human review. For every DDV scenario, limiting case, reference data comparison, and sanity check that can benefit from a visual, generate a plot. Store all visuals in `{{lisa_root}}/plots/` and document each in `{{lisa_root}}/plots/REVIEW.md`. - -Dynamic context is prepended above this prompt by the Lisa Loop CLI. It tells you the current pass number. - -## Your Task - -### 1. Read Context - -Read **all** of the following: - -- `ASSIGNMENT.md` — project goals -- `{{lisa_root}}/STACK.md` — build/test/plot commands -- `{{lisa_root}}/methodology/methodology.md` — the methodology -- `{{lisa_root}}/spiral/pass-0/acceptance-criteria.md` — what success looks like -- `{{lisa_root}}/spiral/pass-0/spiral-plan.md` — scope progression (staged acceptance per pass) -- `{{lisa_root}}/spiral/pass-N/execution-report.md` — this pass's execution results and intermediate values -- `{{lisa_root}}/ddv/scenarios.md` — DDV verification scenarios and manifest -- `{{lisa_root}}/validation/sanity-checks.md` — living sanity check document -- `{{lisa_root}}/validation/limiting-cases.md` — limiting cases to check -- `{{lisa_root}}/validation/reference-data.md` — reference data to compare against -- `{{lisa_root}}/plots/REVIEW.md` — current plot assessments - -If this is **Pass N > 1**: -- Read `{{lisa_root}}/spiral/pass-{N-1}/progress-tracking.md` — previous progress tracking -- Read `{{lisa_root}}/spiral/pass-{N-1}/system-validation.md` — previous validation report - -### 1b. Determine This Pass's Acceptance Targets - -Read `{{lisa_root}}/spiral/pass-0/spiral-plan.md` to find the staged acceptance criteria for this pass. -Early passes have wider tolerances — do NOT apply final acceptance targets to intermediate -passes. When checking acceptance criteria in section 3d, use this pass's staged tolerances, -not the final targets from acceptance-criteria.md. - -For example, if the spiral plan says Pass 1 acceptance is ±50% and the final target is ±5%, -a Pass 1 result within ±50% should be marked as PASS for this pass's criteria, even though -it wouldn't meet final targets. - -In the review package, report BOTH: -- Whether this pass's staged criteria are met -- How far the result is from the final acceptance target (for progress tracking) - -### 2. Run the System - -Run the complete system using the runner/integration code that Build implemented. -Use the run command from `{{lisa_root}}/STACK.md`. Verify: -- The system executes without errors -- Output matches what's in `{{lisa_root}}/spiral/pass-N/execution-report.md` -- If the execution report is missing or stale, produce a fresh one - -### 3. DDV Executable Tests - -Write executable tests from DDV scenarios in `{{lisa_root}}/ddv/scenarios.md`: - -1. Read each scenario with `Pass relevance` matching this pass or earlier -2. For each scenario not yet tested (check the `## Manifest` section in `{{lisa_root}}/ddv/scenarios.md`): - - Write an executable test in `{{tests_ddv}}/` that sets up the scenario's conditions, runs the relevant code, and checks the expected output against the specified tolerance - - Include the scenario ID (DDV-NNN) in the test name and a comment citing the source - - If the scenario has a `**Visual:**` field, generate the described plot. Save to `{{lisa_root}}/plots/` with scenario ID in filename (e.g., `ddv-003-drag-vs-speed.png`). Add entry to `{{lisa_root}}/plots/REVIEW.md`. -3. Run all DDV tests and record results -4. Update the `## Manifest` section in `{{lisa_root}}/ddv/scenarios.md` with test status (TESTED/PASS/FAIL/DEFERRED) - -If a scenario cannot be tested yet (e.g., the relevant code isn't implemented until a later pass), -mark it DEFERRED in the manifest with a note explaining why. - -### 4. Test Results Summary - -Collect test results: -- **DDV tests:** Run the DDV test suite. Record pass/fail counts. -- **Software tests:** Run the software test suite. Record pass/fail counts. -- **Integration tests:** Run integration tests. Record pass/fail counts. - -### 5. Validation Checks - -#### 5a. Sanity Checks - -Execute every check in `{{lisa_root}}/validation/sanity-checks.md`: - -- **Order of magnitude:** Are results in the expected ballpark? -- **Expected trends:** When parameters change, do outputs move in the expected direction? -- **Physical bounds:** Are all outputs within physically possible ranges? -- **Conservation:** Are conserved quantities preserved to within tolerance? -- **Dimensional analysis:** Do all outputs have correct dimensions/units? -- **Red flags:** Are any red-flag conditions triggered? - -Record each check as PASS or FAIL with the actual value observed. - -#### 5b. Limiting Cases - -Check limiting cases from `{{lisa_root}}/validation/limiting-cases.md`: -- When parameters go to extreme values, do results match known analytical solutions? - -#### 5c. Reference Data - -Compare against reference data from `{{lisa_root}}/validation/reference-data.md`: -- How do results compare to published experimental or computational data? - -#### 5d. Acceptance Criteria - -Check against THIS PASS's staged acceptance criteria from `{{lisa_root}}/spiral/pass-0/spiral-plan.md`. -Do not apply final targets to early passes. - -For each criterion: -- **Staged target (this pass):** [from spiral-plan.md] → Met? [YES/NO] -- **Final target:** [from acceptance-criteria.md] → Distance: [X%] - -#### 5e. Generate Visual Verification Evidence - -Generate plots for the following categories of verification evidence. This visual evidence is the primary artifact the human reviewer uses to judge correctness. - -1. **Reference data comparisons:** Plot computed values vs. published data with error bands or tolerance regions. Include source citation in plot title or legend. -2. **Limiting cases:** Plot the quantity approaching the known analytical value as the parameter moves toward the limit. -3. **Trend checks:** Plot the output over a parameter sweep to verify monotonicity, convexity, or other expected behavior from sanity checks. -4. **Cross-pass convergence:** If Pass > 1, plot key quantities across passes to show convergence trajectory. -5. **DDV scenario visuals:** Generate any `**Visual:**` plots from DDV scenarios not yet generated by the Build phase. - -Save all plots to `{{lisa_root}}/plots/` and document each in `{{lisa_root}}/plots/REVIEW.md` with: path, what it shows, what to look for, and assessment. - -### 6. Engineering Judgment Audit - -Using the intermediate values and final answer from `{{lisa_root}}/spiral/pass-N/execution-report.md`, -and the engineering judgment checks from `{{lisa_root}}/validation/sanity-checks.md`, perform an -independent engineering judgment audit: - -1. **Intermediate values:** Do intermediate quantities fall within the expected ranges - stated in the methodology? Flag any that don't. -2. **Dimensional consistency:** Do all quantities have correct units throughout the chain? -3. **Order of magnitude:** Is the final answer in the right ballpark? Compare against - the order-of-magnitude estimates from `{{lisa_root}}/validation/sanity-checks.md`. -4. **Conservation:** Are conserved quantities preserved through the computation? -5. **Hard bounds:** Does the result respect known physical/domain bounds? - -This audit is performed here — separately from the agent that wrote the integration code — -to maintain independence between implementation and judgment. - -### 7. DDV Coverage Assessment - -Assess the coverage of DDV scenarios: - -1. **Phenomena coverage:** What fraction of the physical phenomena in the methodology are covered by at least one DDV scenario? -2. **Parameter ranges:** Do the scenarios cover the full valid parameter range, or only a narrow slice? -3. **Category balance:** Are all scenario categories (unit-function, model-behavior, system-integration, limiting-case, reference-data) represented? -4. **Re-run recommendation:** Based on coverage gaps, should the DDV Agent be re-run to add more scenarios? Answer YES or NO with justification. - -Record the assessment in the system-validation report. - -### 8. Methodology Compliance Spot-Check - -Sample key equations: does the code match the methodology? -- Are assumptions respected? -- Are valid ranges enforced? -- Are derivation docs present for non-trivial mappings? - -### 9. Progress Tracking - -Compare key outputs with the previous spiral pass. Compute and present deltas — do NOT render a convergence verdict. The human decides at the review gate whether to accept or continue. - -If this is **Pass 1:** No previous pass to compare. Establish baseline values. - -If this is **Pass N > 1:** -- Read `{{lisa_root}}/spiral/pass-{N-1}/progress-tracking.md` for previous values -- For each key output quantity: - - Compute absolute and relative change from previous pass - - Note whether the change is within the accuracy bounds of the methods used - -### 10. Produce Artifacts - -Create **all** of the following: - -#### `{{lisa_root}}/spiral/pass-N/system-validation.md` - -Detailed validation report. Be concise: one line per passing check, detailed analysis only for failures. - -```markdown -# Spiral Pass N — System Validation Report - -## Verification - -### Test Results -- DDV tests: [pass/total] -- Software tests: [pass/total] -- Integration tests: [pass/total] - -### Failures -[For each failing test:] -- **[Test name]:** Expected [X], got [Y]. [Analysis of why.] - -### Methodology Compliance -[Results of spot-check. Issues found, if any.] - -### Derivation Completeness -[Gaps found, if any.] - -## Validation - -### Sanity Checks -| Check | Expected | Actual | Status | -|-------|----------|--------|--------| -| [check] | [value] | [value] | PASS/FAIL | - -### Limiting Cases -| Case | Expected | Actual | Status | -|------|----------|--------|--------| -| [case] | [value] | [value] | PASS/FAIL | - -### Reference Data Comparison -| Dataset | Source | Our Result | Published | Δ (%) | Status | -|---------|--------|-----------|-----------|-------|--------| -| [data] | [cite] | [value] | [value] | [X.X] | PASS/FAIL | - -### Engineering Judgment Audit -| Check | Expected | Actual | Status | -|-------|----------|--------|--------| -| [intermediate X] | [range] | [value] | OK/FLAG | -| [order of magnitude] | [~value] | [value] | OK/FLAG | -| [conservation] | [conserved?] | [value] | OK/FLAG | -| [hard bounds] | [range] | [value] | OK/FLAG | - -### Acceptance Criteria -| Criterion | Staged target (this pass) | Final target | Current | Staged met? | Final met? | -|-----------|--------------------------|-------------|---------|------------|-----------| -| [criterion] | [from spiral-plan] | [from acceptance-criteria] | [value] | YES/NO | YES/NO | - -### Visual Verification Evidence -| Plot | Scenario/Check | What to Look For | Assessment | -|------|---------------|------------------|------------| -| [path] | [DDV-NNN or check ref] | [expected behavior] | PASS/CONCERN | - -### DDV Coverage Assessment -- Phenomena coverage: [X/Y] ([Z%]) -- Parameter range coverage: [assessment] -- Category balance: unit-function=[N], model-behavior=[N], system-integration=[N], limiting-case=[N], reference-data=[N] -- Re-run DDV Agent: [YES/NO] — [justification] -``` - -#### `{{lisa_root}}/spiral/pass-N/progress-tracking.md` - -```markdown -# Spiral Pass N — Progress Tracking - -## Key Quantities -| Quantity | Pass N-1 | Pass N | Δ (abs) | Δ (%) | -|----------|---------|--------|---------|-------| -| [qty 1] | [value] | [value] | [value] | [X.X] | - -## Analysis -[What is driving changes between passes. Which quantities are stabilizing, which are still shifting.] -``` - -#### `{{lisa_root}}/spiral/pass-N/review-package.md` - -This is the primary artifact for human review. Use this **exact format**: - -```markdown -# Spiral Pass N — Review Package - -## Current Answer -[The quantitative answer to ASSIGNMENT.md] - -## Pass Scope (from spiral-plan.md) -[What scope subset and fidelity level this pass covers] -[Staged acceptance for this pass: ±X%] - -## Progress -| Quantity | Δ from prev | -|----------|------------| -| [qty] | [X.X%] | - -## Tests -DDV: [pass/total] | Software: [pass/total] | Integration: [pass/total] -Failures: [list any, or "None"] - -## DDV Scenario Coverage -Scenarios tested: [N/M] | PASS: [N] | FAIL: [N] | DEFERRED: [N] -Re-run DDV Agent recommended: [YES/NO] - -## Sanity Checks: [pass/total] -Failures: [list any, or "None"] - -## Engineering Judgment Audit -[Summary of audit results. List any flagged items, or "All checks OK"] - -## Visual Evidence (HUMAN REVIEW) -These plots are the primary evidence for judging correctness: -1. [Plot: path] → [what to look for — expected behavior, acceptable range] -2. [Plot: path] → [what to look for] -[List ALL plots from {{lisa_root}}/plots/REVIEW.md with their assessment] - -## Engineering Judgment (HUMAN REVIEW) -Non-visual checks requiring domain expertise: -1. [Key result] → [is this reasonable? dimensional analysis, conservation, order-of-magnitude] - -## Status Assessment -[Factual summary: what is complete vs. what remains from the full scope in spiral-plan.md. - Do NOT recommend a specific review action — the human decides.] - -## If Continuing — Proposed Refinements -- [What to change and why] - -## Details -- Execution report: {{lisa_root}}/spiral/pass-N/execution-report.md -- Full validation: {{lisa_root}}/spiral/pass-N/system-validation.md -- Progress: {{lisa_root}}/spiral/pass-N/progress-tracking.md -- Plots: {{lisa_root}}/plots/REVIEW.md -``` - -#### Update `{{lisa_root}}/plots/REVIEW.md` - -Ensure all plots have current assessments reflecting this pass's results. - -#### Update the `## Manifest` section in `{{lisa_root}}/ddv/scenarios.md` - -Update the manifest table with test results for any newly written DDV executable tests. - -#### `{{lisa_root}}/spiral/pass-N/PASS_COMPLETE.md` - -Create this file **last**: - -```markdown -# Pass N — Complete - -Verification: DDV [pass/total], Software [pass/total], Integration [pass/total] -Validation: [X/Y sanity checks passing] -DDV Scenarios: [tested/total] ([deferred] deferred) -Visual evidence: [N] plots generated (see {{lisa_root}}/plots/REVIEW.md) -Progress: see progress-tracking.md -Status: [what is complete vs. what remains] -``` - -#### Note on Final Output - -Do NOT draft deliverables. The finalize phase handles deliverable production after human review. - -## Rules - -- **Visuals are the preferred way to surface results for human review.** Generate plots for every verification check that can benefit from one. The review package should lead with visual evidence. -- **Do NOT modify source code or methodology.** This is an audit phase. The only code you write is DDV executable tests in `{{tests_ddv}}/`. -- **Do NOT modify existing DDV tests** written from DDV scenarios in previous validation passes. You may add NEW tests from DDV scenarios. -- **Do NOT skip any sanity check.** Execute every check in `{{lisa_root}}/validation/sanity-checks.md`. -- **If you cannot verify something** (e.g., paper not available, test infrastructure missing), flag it explicitly — do not silently skip it. - -## Output - -Provide a brief summary of: -- Test results (DDV, software, integration pass rates) -- DDV scenario coverage (tested/total, any failures) -- Validation results (sanity check results) -- Visual verification evidence generated (count of plots, any concerns flagged) -- Progress tracking (deltas from previous pass) -- Status assessment (what is complete vs. what remains from full scope) diff --git a/prompts/scope/implementation_plan_spec.md b/prompts/scope/implementation_plan_spec.md new file mode 100644 index 0000000..8f9ba4f --- /dev/null +++ b/prompts/scope/implementation_plan_spec.md @@ -0,0 +1,37 @@ +# Implementation Plan Spec — `{{lisa_root}}/methodology/plan.md` + +Initial implementation plan with task structure for Pass 1. At this stage you know *what* +needs to be implemented and in what order, but the equations are not yet fully specified — +that detail comes from the refine phase. Keep this plan at the structural level: task names, +ordering, methodology references, and dependencies. Do NOT write detailed checklists — the +refine phase will flesh those out once the methodology is complete. + +## Template + +```markdown +# Implementation Plan + +## Tasks + +### Task 1: [Short name] +- **Status:** TODO +- **Pass:** 1 +- **Methodology:** [section ref] +- **Dependencies:** [task refs or "None"] + +### Task 2: [Short name] +- **Status:** TODO +- **Pass:** 1 +- **Methodology:** [section ref] +- **Dependencies:** [task refs or "None"] +``` + +## Task Rules + +- Order tasks bottom-up: utilities -> core equations -> higher-level models -> integration +- Each task should be scoped for a single Ralph iteration +- Infrastructure tasks (setup, test framework, etc.) come first if needed +- Tag every task with `**Pass:** 1` for Pass 1 tasks +- Pass 2+ tasks can be sketched with TODO placeholders +- Tasks should include bounding check items where applicable (the Build phase derives and writes bounding tests per the engineering judgment skill) +- Do NOT add checklists — the refine phase adds those after completing the methodology diff --git a/prompts/scope/literature_survey_spec.md b/prompts/scope/literature_survey_spec.md new file mode 100644 index 0000000..a4b92fb --- /dev/null +++ b/prompts/scope/literature_survey_spec.md @@ -0,0 +1,48 @@ +# Literature Survey Spec — `{{lisa_root}}/spiral/pass-0/literature-survey.md` + +Survey of candidate methods, organized by topic/phenomenon. This template is used both by +the Literature Survey subagent (to produce the initial draft) and by the scoping agent +(to review and finalize the artifact). + +## Template + +```markdown +# Literature Survey + +## Methods Surveyed + +### [Topic/Phenomenon A] + +#### [Method 1 Name] +- **Source:** [Author(s), Year, Title, DOI/URL] +- **Approach:** [Brief description] +- **Fidelity:** [Low / Medium / High] +- **Assumptions:** [Key assumptions] +- **Valid range:** [Where it applies] +- **Pros:** [Advantages for our problem] +- **Cons:** [Disadvantages or limitations] +- **Available:** [YES / NEEDS_PAPER — whether full paper is accessible] + +[Repeat for each candidate method] + +#### Recommended Approach for [Topic A] +[Which method(s) to use and why] + +### [Topic/Phenomenon B] +[Same structure] + +## Cross-Cutting Methods +[Any methods that span multiple topics] + +## Papers Retrieved +[List papers saved to {{lisa_root}}/references/retrieved/ with full citations] + +## Papers Needed +[Papers flagged with [NEEDS_PAPER] that the human should provide] +``` + +## Quality Criteria + +- Every method candidate cites a peer-reviewed source (author(s), year, title, DOI/URL) +- Alternatives are documented for each phenomenon +- Papers saved to `{{lisa_root}}/references/retrieved/` have proper citations and key equations diff --git a/prompts/scope/methodology_spec.md b/prompts/scope/methodology_spec.md new file mode 100644 index 0000000..0bca636 --- /dev/null +++ b/prompts/scope/methodology_spec.md @@ -0,0 +1,43 @@ +# Methodology Document Spec — `{{lisa_root}}/methodology/methodology.md` + +**This is the central technical artifact.** It identifies the recommended methods, cites source papers, lists key equations by name/number, and documents assumptions and valid ranges. + +**Division of labor:** The methodology created here is an *initial specification*. It identifies the recommended method, cites the source paper, lists key equations by name/number, and documents assumptions and valid ranges. It does NOT contain full equation derivations with every variable defined — that level of detail is the refine phase's job in Pass 1. This intentional fidelity gap is what gives the first refine phase meaningful work: transforming a method recommendation into a complete, implementable specification. + +## Template + +```markdown +# Methodology + +## Phenomenon +[What this project models — from ASSIGNMENT.md] + +## Candidate Methods + +### [Method 1] +- **Source:** [Citation] +- **Approach:** [Description] +- **Fidelity:** [Low / Medium / High] +- **Pros:** [For our problem] +- **Cons:** [Limitations] + +### [Method 2] +... + +## Recommended Approach +[Which method and why, considering spiral progression and the human's approach preference from ASSIGNMENT.md] +[If the human asked for simplicity, justify why this method is the simplest that can meet the acceptance criteria] +[If the human asked for state of the art, justify why this is the best available method] +[If there is a mismatch between the requested approach and the acceptance criteria, flag it explicitly] + +## Key Equations +[Identify by name and equation number from the source paper — e.g., "Eq. 12 in Faltinsen (1990)" or "ITTC-57 friction line." Do NOT write out the full mathematical expressions here — that is the refine phase's job. If a specific paper is needed but not yet available, flag with [NEEDS_PAPER].] + +## Assumptions +[List all assumptions] + +## Valid Range +[Parameter ranges where the chosen method applies] +``` + +If the problem has distinct sub-topics (e.g., frictional resistance, wave resistance, added resistance), organize the methodology into clearly separated **sections** within the single document. Each section follows the same structure above. diff --git a/prompts/scope/spiral_plan_spec.md b/prompts/scope/spiral_plan_spec.md new file mode 100644 index 0000000..8c5ad59 --- /dev/null +++ b/prompts/scope/spiral_plan_spec.md @@ -0,0 +1,60 @@ +# Spiral Plan Spec — `{{lisa_root}}/spiral/pass-0/spiral-plan.md` + +The spiral plan defines how scope and fidelity increase per pass using **diagonal scoping**: +each pass targets one phenomenon at one fidelity level — narrow on both dimensions simultaneously. +This produces small, independently checkable passes of {{max_tasks_per_pass}} tasks or fewer. +If a pass would require more than {{max_tasks_per_pass}} tasks, it is too broad, too deep, or both — +split it. + +**Calibrate the spiral plan to the human's approach preference** from `ASSIGNMENT.md`: +- If they want simplicity/minimum viable: plan fewer passes, stay with one method, widen + tolerances. The spiral may converge in 1-2 passes. +- If they want state of the art: plan for progressive method upgrades across passes, + tighter final tolerances, more validation. +- If they specified a particular method: build the spiral around that method's scope + progression (narrow to broad), not around method upgrades. +- If no preference stated: use balanced judgment. + +## Template + +```markdown +# Spiral Plan + +## Approach Philosophy +[Summarize the human's approach preference from ASSIGNMENT.md, or state "balanced (default)" +if none was given. Note any tension between the requested approach and the acceptance criteria.] + +## Scope Progression + +| Pass | Scope subset | Fidelity | Acceptance (this pass) | Key question | +|------|-------------|----------|----------------------|--------------| +| 1 | [subset] | [level] | [+/-X%] | Does the approach work at all? | +| 2 | [broader] | [level] | [+/-X%] | Does it generalize across range? | +| 3 | [full] | [level] | [+/-X%] | Does coupling work? | +| 4 | [full] | [refined]| [+/-X%] | Converged? | + +## Progress Tracking Expectations +[What quantities to track across passes, expected rate of change] + +## Risk Areas +[Where methodology might need reconsideration, known difficult aspects] +``` + +## Example: Ship resistance (5-25 kn, sea states 1-6) + +**Balanced approach:** +- Pass 1: 12 kn, calm water, simplest method -> +/-50% +- Pass 2: 5-25 kn, calm water, add corrections -> +/-20% +- Pass 3: Full range + sea states 1-3 -> +/-10% +- Pass 4: Full scope, refined methods -> +/-5% + +**Minimum viable approach:** +- Pass 1: Full speed range, calm water, Holtrop-Mennen -> +/-15% +- Pass 2: Add sea state corrections -> +/-10% +(Fewer passes, simpler method, acceptance tolerances widened to match method capability) + +## How the spiral plan is used + +The refine phase reads this plan to scope tasks for the current pass. +The Audit phase checks bounding test discipline and coverage for the current pass's scope subset. +Acceptance criteria are staged — early passes have wider tolerances. diff --git a/prompts/scope/stack_selection_spec.md b/prompts/scope/stack_selection_spec.md new file mode 100644 index 0000000..ded40cf --- /dev/null +++ b/prompts/scope/stack_selection_spec.md @@ -0,0 +1,62 @@ +# Stack Selection Spec — `{{lisa_root}}/STACK.md` + Environment Probing + +**This artifact ensures that all subsequent agents use a concrete, verified technology stack rather than making implicit choices.** + +## Step 1: Reason About Stack Selection + +Before probing the environment, reason about the best technology stack for this project: + +- **Computational requirements:** Is the problem compute-bound (favoring a compiled language) or I/O-bound / prototyping-oriented (where a scripting language suffices)? +- **Ecosystem:** Are there domain-specific libraries that favor a particular language? +- **Human preferences:** Read the "Technology Preferences" section of `ASSIGNMENT.md`. If the human stated preferences, respect them. If blank, choose freely. + +## Step 2: Probe the Local Environment + +The Environment Probe subagent has already checked what runtimes and tools are available. +Synthesize its report: verify it covers all runtimes needed for your chosen stack, +and note any gaps that require human resolution. + +## Step 3: Handle Two Categories of Dependencies + +**1. Runtimes and toolchains** (language interpreters, compilers, system-level libraries, etc.): + +Check if these are present by running version commands. If a required runtime is **not available**: +- Do **NOT** attempt to install it +- Create `{{lisa_root}}/spiral/pass-0/environment-resolution.md` listing what is missing: + +```markdown +# Environment Resolution Required + +## Missing Runtimes / Toolchains + +### [Tool Name] +- **What:** [e.g., Python 3.10+] +- **Why needed:** [e.g., Primary implementation language] +- **Suggested install:** [e.g., `apt install python3` or `pyenv install 3.11`] +- **Alternative:** [Could a different stack choice avoid this? If so, describe.] + +## Status +Waiting for human resolution before proceeding. +``` + +If all required runtimes are present, do **NOT** create this file (or create it empty). + +**2. Package-level dependencies** (packages, crates, modules, etc.): + +Install these directly using the appropriate package manager. These are routine development dependencies: +- Run the install command using the appropriate package manager +- Verify each install succeeded +- Record installed versions in {{lisa_root}}/STACK.md + +## Step 4: Populate {{lisa_root}}/STACK.md + +Update the "Resolved Technology Stack" section of `{{lisa_root}}/STACK.md`: + +- **Language & Runtime:** Fill with verified language and version (e.g., "Python 3.11.5 (verified present)") +- **Key Dependencies:** List all installed packages with versions +- **Test Framework:** Specify the chosen test framework and version +- **Stack Justification:** Brief reasoning for the technology choices + +Fill in all command sections (Setup, Build, Test, Lint, etc.) with **concrete, tested commands** — no more placeholders. If the human pre-filled any command sections before running scope, verify those commands work (run them) rather than overwriting them. + +**Backward compatibility:** If {{lisa_root}}/STACK.md already has concrete (non-placeholder) commands filled in by the user, verify they work and keep them. Only populate sections that contain placeholders or template text. diff --git a/prompts/scope/validation_specs.md b/prompts/scope/validation_specs.md new file mode 100644 index 0000000..4aecaa8 --- /dev/null +++ b/prompts/scope/validation_specs.md @@ -0,0 +1,53 @@ +# Validation Artifact Specs + +This file contains the format specifications for three validation artifacts produced during scoping. + +--- + +## 1. Sanity Checks — `{{lisa_root}}/validation/sanity-checks.md` + +```markdown +# Sanity Checks + +These are engineering judgment checks to be executed after every spiral pass. +A failure on any check indicates a likely error and should block acceptance. + +## Order of Magnitude +- [ ] [Quantity] should be approximately [value] [units] (+/-[order of magnitude]) + - **Reasoning:** [Why this magnitude is expected] + +## Expected Trends +- [ ] When [parameter] increases, [quantity] should [increase/decrease/remain constant] + - **Reasoning:** [Physical justification] + +## Physical Bounds +- [ ] [Quantity] must be [positive / in range [a,b] / less than X] + - **Reasoning:** [Physical constraint] + +## Conservation +- [ ] [Conserved quantity] should be preserved to within [tolerance] + - **Check method:** [How to verify] + +## Dimensional Analysis +- [ ] All outputs have correct dimensions/units + - **Check method:** [How to verify] + +## Red Flags +- [ ] [Specific condition that would indicate a clearly wrong answer] +``` + +--- + +## 2. Limiting Cases — `{{lisa_root}}/validation/limiting-cases.md` + +Extract the limiting cases from your validation research and format them using the `LC-NNN` format (e.g., `LC-001`, `LC-002`). Each entry should include: case description, the condition, expected result, source/reasoning, and a pass/fail status placeholder. + +--- + +## 3. Reference Data — `{{lisa_root}}/validation/reference-data.md` + +Extract the reference datasets from your validation research and format them using the `RD-NNN` format (e.g., `RD-001`, `RD-002`). Each entry should include: dataset description, source citation, what it measures, comparison method, and a pass/fail status placeholder. + +--- + +These are the living validation documents that will be checked during every validation phase and refined during methodology refinement phases. diff --git a/skills/dimensional_analysis.md b/skills/dimensional_analysis.md new file mode 100644 index 0000000..cf59eb5 --- /dev/null +++ b/skills/dimensional_analysis.md @@ -0,0 +1,60 @@ +# Dimensional Analysis + +Systematic unit tracking through computation chains. Every physical quantity has dimensions; every equation must balance dimensionally. This skill is applied always, at every level of the engineering judgment hierarchy. + +--- + +## Core Principle + +If an equation doesn't balance dimensionally, it is wrong — regardless of how reasonable the numerical output looks. Dimensional analysis catches errors that numerical testing might miss (e.g., a factor that happens to be close to 1 in the test case but is dimensionally incorrect). + +--- + +## When to Apply + +1. **Before implementing any equation:** Write down the dimensions of every term. Verify the equation balances. +2. **At every interface:** When one function passes a result to another, verify the units match what the receiver expects. +3. **When combining quantities:** Addition and subtraction require identical dimensions. Multiplication and division combine dimensions algebraically. +4. **When using empirical correlations:** Check that coefficient dimensions make the equation balance. Empirical coefficients often have implicit units. + +--- + +## Methodology + +### Step 1: Identify Base Dimensions + +Use the standard set: [M] mass, [L] length, [T] time, [Θ] temperature, or domain-appropriate extensions. + +Common derived dimensions: +- Force: [M L T⁻²] +- Pressure: [M L⁻¹ T⁻²] +- Energy: [M L² T⁻²] +- Power: [M L² T⁻³] +- Velocity: [L T⁻¹] +- Density: [M L⁻³] + +### Step 2: Track Through Computation + +For each intermediate result, annotate its dimensions. When you see: +- `a + b` → dimensions of a must equal dimensions of b +- `a * b` → result dimensions = dim(a) × dim(b) +- `f(x)` where f is transcendental (exp, log, sin) → x must be dimensionless + +### Step 3: Verify Final Output + +The final result must have the expected physical dimensions. If computing force, the result must have dimensions [M L T⁻²]. + +--- + +## Common Traps + +- **Implicit unit conversions:** Mixing meters and millimeters, or degrees and radians. +- **Empirical formulas with hidden units:** Correlations from textbooks where coefficients absorb unit conversions (e.g., "speed in knots" baked into a coefficient). +- **Dimensionless groups assembled incorrectly:** Reynolds number, Froude number, etc. must be truly dimensionless. +- **Gravitational constant confusion:** g vs gc, weight vs mass. + +--- + +## Integration with Bounding Tests + +Every Level 1 bounding test should include a dimensional analysis check as part of its derivation comment. If the derivation shows the bound has dimensions [kN] and the implementation returns a value with dimensions [kN], the dimensional check passes implicitly. If there's any ambiguity about units in the implementation, add an explicit assertion. diff --git a/skills/engineering_judgment.md b/skills/engineering_judgment.md new file mode 100644 index 0000000..ad56e7e --- /dev/null +++ b/skills/engineering_judgment.md @@ -0,0 +1,140 @@ +# Engineering Judgment + +Engineering judgment means verifying results through first-principles reasoning before trusting them. This skill encodes a three-level bounding methodology analogous to the unit → integration → end-to-end testing pyramid in software. + +The core discipline: **derive bounds before implementing, verify after implementing, check composition when integrating.** + +--- + +## Level 1 — Phenomenon Bounds + +Each individual physical phenomenon gets first-principles bounds derived from dimensional analysis, known coefficient ranges, and scaling laws. + +**Catches:** wrong equations, unit errors, wrong coefficient values, misapplied correlations, wrong regime selection. + +### When implementing a physical phenomenon + +Before writing implementation code: +1. Identify the governing dimensional groups +2. Establish coefficient ranges from known physics +3. Compute an order-of-magnitude expected output using simple arithmetic with known constants +4. Write a bounding test that: + - Documents the first-principles derivation as a comment + - Computes the bounds from the stated constants + - Asserts the implementation output falls within bounds +5. The derivation IS the test documentation + +After implementation: +6. Run the bounding test +7. If it fails, your implementation is wrong — not the bound +8. Generate a visual: bar showing computed value within its first-principles bounds + +### Example + +Frictional resistance on a 200m ship at 15 knots. Re ≈ 1.5×10⁹. At this Re, flat plate Cf ≈ 0.0015. RF = ½ρV²SCf ≈ 365 kN. Bound: [100, 1000] kN. Below 100 kN suggests missing wetted surface or unit error. Above 1000 kN suggests double-counting. + +--- + +## Level 2 — Composition Bounds + +When phenomena are combined, their relationships get checked: additive totals must equal the sum of components (conservation), component ratios must match physical expectations, trends must be monotonic where physics demands it. + +**Catches:** double-counting, sign errors in coupling, missing interaction terms, violated conservation laws, physically impossible component ratios. + +### When composing phenomena + +After integrating multiple phenomena into a combined model: +1. Derive composition bounds from phenomenon-level bounds + - Additive composition: sum the bound ranges + - Multiplicative: multiply the ranges + - Check that component ratios match physical expectations +2. Verify conservation laws hold across the composition +3. Write composition-level bounding tests +4. Generate a visual: waterfall/stacked bar showing how components sum to total, with ratio annotations + +### Example + +Frictional resistance should dominate at Fn < 0.2. Wave resistance fraction should increase with Froude number. Air resistance should be less than 5% of total below 20 knots. + +--- + +## Level 3 — System Bounds + +The top-level output gets bounded by an independent back-of-envelope calculation using completely different reasoning from the detailed model. + +**Catches:** systematic bias across all components, missing phenomena, wrong problem formulation, errors that look locally reasonable but produce globally wrong answers. + +### When producing a system-level answer + +Before reporting any result: +1. Derive an independent estimate using completely different reasoning from the detailed model + - Empirical correlations (e.g., admiralty coefficient) + - Scaling from similar known cases + - Back-of-envelope from first principles +2. Compare against the detailed model output +3. If disagreement exceeds a factor of 2, investigate before reporting +4. Document the independent estimate alongside results +5. Generate a visual: detailed model output plotted against independent estimate with bounds + +### Example + +Admiralty coefficient for this hull type is typically 400-600, giving estimated power of 5-15 MW at 15 knots. If the detailed model produces 2 MW or 50 MW, something is fundamentally wrong. + +--- + +## How bounds compose + +Level 2 bounds can be derived from level 1 bounds. If frictional resistance is bounded to [300, 500] kN and wave resistance to [50, 200] kN, then calm water resistance must be in [350, 700] kN. The composition bound comes for free from the phenomenon bounds. + +If the composed result falls outside this derived bound, one of the components is wrong. If it falls inside and the components individually satisfy their bounds, you have reasonable confidence in the decomposition. + +--- + +## Bounding Test Structure + +Each bounding test follows this pattern: + +``` +# Level: [phenomenon | composition | system] +# Phenomenon: [what physical quantity is being bounded] +# +# Derivation: +# [Physical reasoning — dimensional groups, known +# coefficient ranges, scaling laws used] +# [Arithmetic — show the computation step by step] +# +# Bound: [lower] to [upper] [units] +# Confidence: [why this range is appropriate] + +def test__bounds(): + result = compute_(inputs) + lower = + upper = + assert lower <= result <= upper +``` + +The derivation comment is mandatory. A bounding test without a derivation is not a bounding test — it's an arbitrary assertion. + +--- + +## Test Directory Structure + +``` +tests/ + bounds/ + phenomenon/ # Level 1: individual phenomena + composition/ # Level 2: composed phenomena + system/ # Level 3: system-level cross-checks +``` + +Place bounding tests in the appropriate subdirectory based on their level. + +--- + +## Visual Evidence + +**Level 1 — Phenomenon bounds plot.** For each phenomenon: a horizontal bar showing the computed value positioned within its first-principles bounds. Green if inside bounds, red if outside. The derivation is summarised alongside. + +**Level 2 — Composition waterfall.** A waterfall or stacked bar chart showing how individual components sum to the total. Annotated with component ratios. Lines or bands showing the derived composition bounds. + +**Level 3 — System cross-check.** The detailed model output plotted against the independent back-of-envelope estimate. A single chart showing agreement/disagreement. diff --git a/skills/literature_grounding.md b/skills/literature_grounding.md new file mode 100644 index 0000000..2045939 --- /dev/null +++ b/skills/literature_grounding.md @@ -0,0 +1,127 @@ +# Literature Grounding + +When reference data comparison is needed, this skill provides a methodology for rigorous comparison with published experimental or computational data. This is an optional refinement on top of the engineering judgment bounding hierarchy — not the foundation of verification. + +--- + +## When to Use + +Use literature grounding when: +- Published experimental data exists for the specific case being modelled +- The project requires tighter verification than first-principles bounds alone +- Calibrating empirical coefficients against measured data +- Validating against benchmark problems with known solutions + +Do NOT use literature grounding as the primary verification method. First-principles bounds (engineering judgment skill) should always be established first. + +--- + +## Methodology + +### Step 1: Source Verification + +Before using any reference value: +1. **Identify the primary source.** Is this the original paper, or a secondary citation? Trace to the original. +2. **Check the measurement conditions.** Do they match your problem setup? (Reynolds number range, geometry, boundary conditions, fluid properties) +3. **Understand the measurement uncertainty.** Published data has error bars. If the paper doesn't report uncertainty, treat the values with caution. +4. **Check for known corrections.** Some older datasets have known systematic errors or have been superseded. + +### Step 2: Unit Verification + +1. Verify the units of every reference value +2. Check for implicit unit conventions (e.g., "resistance in pounds" vs "resistance in Newtons") +3. Convert all reference values to your working unit system before comparison +4. Document the conversion explicitly + +### Step 3: Comparison Metrics + +When comparing model output to reference data: +1. **Absolute error:** `|model - reference|` — meaningful only when the scale is known +2. **Relative error:** `|model - reference| / |reference|` — use for non-zero quantities +3. **Correlation coefficient:** For datasets with multiple points, R² indicates trend agreement +4. **Bias:** Systematic over- or under-prediction across all data points suggests a systematic error + +### Step 4: Condition Match Assessment + +Before comparing numbers, assess how well the published conditions match your modelled conditions. An experienced engineer comparing against published data always asks "how similar is their case to mine?" + +Factors that affect condition similarity: +- **Geometric differences:** hull form, dimensions, proportions, surface roughness +- **Operating condition differences:** speed, draft, trim, sea state, loading +- **Scale effects:** model scale vs full scale, Reynolds number range +- **Environmental differences:** water depth, temperature, salinity +- **Methodology differences:** experimental vs CFD vs empirical correlation + +Rate the overall condition match: +- **CLOSE:** conditions are nearly identical; differences should be small +- **APPROXIMATE:** conditions are similar but with notable differences; expect ±10-20% scatter +- **LOOSE:** conditions differ substantially; comparison is directional only + +Estimate the expected difference from condition mismatches before looking at the actual comparison. This prevents retrofitting explanations after seeing disagreement. + +### Step 5: Interpretation + +- Agreement within measurement uncertainty → model is consistent with data (CONSISTENT) +- Agreement within first-principles bounds but outside measurement uncertainty → may indicate calibration issues or condition mismatch (INCONCLUSIVE if conditions differ, CONCERN if conditions match) +- Disagreement outside first-principles bounds → fundamental model error (investigate using Level 1 bounds to locate the faulty phenomenon) +- Disagreement with similar conditions but within expected scatter → CONSISTENT +- Disagreement with dissimilar conditions → INCONCLUSIVE (the comparison is informative but not definitive) + +--- + +## Structured Comparison Format (RC-NNN) + +When performing reference comparisons during the audit phase, use this structured format: + +```markdown +## RC-001: [quantity compared] + +**Our result:** [value with units] + +**Published value:** [value with units] +**Source:** [full citation — author(s), year, title, DOI/URL] +**How obtained:** [read from table N / digitised from figure N / + stated in text on page N] + +**Condition match assessment:** +- [Parameter 1]: ours [value] vs published [value] — [match/mismatch] +- [Parameter 2]: ours [value] vs published [value] — [match/mismatch] +- Overall: [CLOSE / APPROXIMATE / LOOSE] +- Expected difference from condition mismatch: ±[X]% + +**Comparison:** +- Absolute difference: [value] +- Relative difference: [X]% +- Within level 3 system bounds: [YES/NO] +- Difference explained by condition mismatch: [YES/PARTIALLY/NO] + +**Confidence:** [CONSISTENT / INCONCLUSIVE / CONCERN] + +**Visual:** [description of overlay plot to generate] +``` + +### Digitising from Figures + +When extracting values from published figures: +- State the uncertainty introduced by digitisation (typically ±2-5% depending on figure quality) +- Note the figure number and axis scales +- If multiple data points are available, extract several to establish trends, not just one +- Prefer tabulated values over figure digitisation when both are available + +--- + +## Integration with the Bounding Hierarchy + +Literature grounding sits on top of the three-level bounding hierarchy: + +1. **Level 1 bounds** establish physically plausible ranges +2. **Level 2 bounds** verify composition is correct +3. **Level 3 bounds** provide independent cross-checks +4. **Literature grounding** (this skill) provides tighter comparison where data is available + +A result that passes all three bounding levels but disagrees with reference data may indicate: +- The reference data has different conditions than assumed +- A phenomenon is modelled at the right order of magnitude but with insufficient fidelity +- The reference data itself has issues (measurement error, different geometry, etc.) + +Never adjust an implementation to match reference data without first understanding why it disagrees. Matching reference data by tuning coefficients without physical justification creates a calibrated-but-wrong model. diff --git a/skills/numerical_stability.md b/skills/numerical_stability.md new file mode 100644 index 0000000..c21d13b --- /dev/null +++ b/skills/numerical_stability.md @@ -0,0 +1,62 @@ +# Numerical Stability + +When implementing numerical methods, discretisation errors, convergence behaviour, and floating-point accumulation can produce results that look plausible but are wrong. This skill identifies when to worry and what to check. + +--- + +## When to Apply + +Apply this skill when implementing: +- Iterative solvers (Newton-Raphson, fixed-point iteration, optimisation) +- Numerical integration (quadrature, ODE solvers, time-stepping) +- Interpolation or curve fitting +- Matrix operations (solving linear systems, eigenvalue problems) +- Summation of many terms (series, discretised integrals) + +--- + +## Convergence Criteria + +### For iterative methods: +1. Define a convergence tolerance **before** implementing the solver +2. Check both absolute and relative convergence: `|x_{n+1} - x_n| < atol` AND `|x_{n+1} - x_n| / |x_n| < rtol` +3. Set a maximum iteration count to prevent infinite loops +4. Log the convergence history (residual vs iteration) — if it oscillates or diverges, the method is unsuitable or the initial guess is bad +5. Verify the converged solution satisfies the original equation (substitution check) + +### For discretised problems: +1. Run at multiple resolutions (e.g., N, 2N, 4N grid points) +2. Check that the solution converges as resolution increases +3. Estimate the order of convergence: if doubling resolution reduces error by ~4×, you have second-order convergence +4. The production resolution should be in the converged regime — not just "finer than the coarsest attempt" + +--- + +## Floating-Point Awareness + +### Catastrophic cancellation: +When subtracting two nearly equal numbers, relative error explodes. Watch for: +- `a - b` where `a ≈ b` and both are large +- Reformulate: use `(a² - b²) = (a+b)(a-b)` instead of computing `a²` and `b²` separately + +### Accumulation errors: +When summing many small terms: +- Sum from smallest to largest (not largest to smallest) +- Consider Kahan summation for critical paths +- For N terms of similar magnitude ε, naive summation error grows as O(√N · ε_machine) + +### Condition number: +For linear systems Ax = b: +- Check `cond(A)` — if it's large (>10⁶), the solution is sensitive to input perturbations +- Consider preconditioning or reformulation +- Never invert a matrix when you can solve the system directly + +--- + +## Integration with Bounding Tests + +Numerical stability issues manifest as: +- **Level 1 bounds failures** when a single phenomenon's computation is numerically unstable +- **Level 2 bounds failures** when composition amplifies numerical errors (e.g., subtracting two large, nearly-equal components) + +When a bounding test fails marginally (result just outside bounds), investigate numerical stability before widening the bounds. The bound derivation is physics-based; if the physics says the answer should be in [100, 1000] and you get 1001, the implementation likely has a numerical issue, not a physics issue. diff --git a/src/agent.rs b/src/agent.rs index 3c7e826..7a38ff6 100644 --- a/src/agent.rs +++ b/src/agent.rs @@ -5,11 +5,44 @@ use std::io::{BufRead, BufReader, IsTerminal, Write}; use std::path::Path; use std::process::{Command, Stdio}; use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::mpsc; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; use crate::terminal; +/// Distinguishes idle-timeout kills from other agent failures. +#[derive(Debug)] +pub enum AgentError { + /// Agent was killed because it produced no output for too long. + IdleTimeout { + label: String, + elapsed_secs: u64, + idle_limit: u64, + }, + /// Any other failure (non-zero exit, spawn error, etc.). + Other(anyhow::Error), +} + +impl std::fmt::Display for AgentError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + AgentError::IdleTimeout { + label, + elapsed_secs, + idle_limit, + } => write!( + f, + "Agent '{}' killed after {}s idle (limit: {}s)", + label, elapsed_secs, idle_limit + ), + AgentError::Other(e) => write!(f, "{}", e), + } + } +} + +impl std::error::Error for AgentError {} + #[derive(Debug, Default, Clone, serde::Serialize, serde::Deserialize)] pub struct UsageInfo { pub input_tokens: u64, @@ -63,6 +96,7 @@ pub fn run_agent( collapse_output: bool, error_log_path: Option<&Path>, extra_args: &[String], + idle_timeout_secs: u64, ) -> Result { let start = Instant::now(); let mut stats = AgentStats::default(); @@ -162,142 +196,168 @@ pub fn run_agent( // stdin is dropped here, closing it } - // Read NDJSON stream - if let Some(stdout) = child.stdout.take() { - let reader = BufReader::new(stdout); - for line in reader.lines() { - let line = match line { - Ok(l) => l, - Err(e) => { - eprintln!(" [warn] NDJSON read error: {}", e); - continue; - } - }; - - let parsed: Value = match serde_json::from_str(&line) { - Ok(v) => v, - Err(e) => { - // Only warn for non-empty lines (blank lines between events are normal) - if !line.trim().is_empty() { - eprintln!(" [warn] NDJSON parse error: {}", e); + // Read NDJSON stream via a channel so we can enforce idle timeouts. + // A reader thread sends parsed lines; the main thread receives with a timeout. + let (tx, rx) = mpsc::channel::(); + + let reader_handle = { + let stdout = child.stdout.take(); + std::thread::spawn(move || { + if let Some(pipe) = stdout { + let reader = BufReader::new(pipe); + for line in reader.lines().map_while(Result::ok) { + if tx.send(line).is_err() { + break; // receiver dropped (timeout killed the child) } - continue; } - }; - - match parsed.get("type").and_then(|t| t.as_str()) { - Some("assistant") => { - if let Some(contents) = parsed - .get("message") - .and_then(|m| m.get("content")) - .and_then(|c| c.as_array()) - { - for item in contents { - match item.get("type").and_then(|t| t.as_str()) { - Some("thinking") => { - if !collapse_output { - if let Some(thought) = - item.get("thinking").and_then(|t| t.as_str()) - { - terminal::print_dim(&format!( - " [💭 {}] {}\n", - terminal::ts(), - thought - )); + } + }) + }; + + let idle_timeout = Duration::from_secs(idle_timeout_secs); + let mut timed_out = false; + + loop { + match rx.recv_timeout(idle_timeout) { + Ok(line) => { + let parsed: Value = match serde_json::from_str(&line) { + Ok(v) => v, + Err(e) => { + if !line.trim().is_empty() { + eprintln!(" [warn] NDJSON parse error: {}", e); + } + continue; + } + }; + + match parsed.get("type").and_then(|t| t.as_str()) { + Some("assistant") => { + if let Some(contents) = parsed + .get("message") + .and_then(|m| m.get("content")) + .and_then(|c| c.as_array()) + { + for item in contents { + match item.get("type").and_then(|t| t.as_str()) { + Some("thinking") => { + if !collapse_output { + if let Some(thought) = + item.get("thinking").and_then(|t| t.as_str()) + { + terminal::print_dim(&format!( + " [💭 {}] {}\n", + terminal::ts(), + thought + )); + } } } - } - Some("tool_use") => { - stats.tool_count += 1; - let name = - item.get("name").and_then(|n| n.as_str()).unwrap_or(""); - let input_val = - item.get("input").cloned().unwrap_or(Value::Null); - - let detail = format_tool_detail(name, &input_val); - let call = parse_tool_call(name, &input_val); - tool_log.push(call); - - // Count specific tool types - if name == "Write" || name == "Edit" { - stats.file_writes += 1; - } - if name == "Bash" { - if let Some(cmd) = - input_val.get("command").and_then(|c| c.as_str()) - { - if cmd.contains("test") || cmd.contains("pytest") { - stats.test_runs += 1; + Some("tool_use") => { + stats.tool_count += 1; + let name = + item.get("name").and_then(|n| n.as_str()).unwrap_or(""); + let input_val = + item.get("input").cloned().unwrap_or(Value::Null); + + let detail = format_tool_detail(name, &input_val); + let call = parse_tool_call(name, &input_val); + tool_log.push(call); + + if name == "Write" || name == "Edit" { + stats.file_writes += 1; + } + if name == "Bash" { + if let Some(cmd) = + input_val.get("command").and_then(|c| c.as_str()) + { + if cmd.contains("test") || cmd.contains("pytest") { + stats.test_runs += 1; + } } } - } - if collapsed { - // Update shared status and refresh the collapsed line - { - let mut status = live_status.lock().unwrap(); - status.tool_count = stats.tool_count; - status.latest_tool = detail.clone(); + if collapsed { + { + let mut status = live_status.lock().unwrap(); + status.tool_count = stats.tool_count; + status.latest_tool = detail.clone(); + } + let elapsed = start.elapsed().as_secs(); + let mins = elapsed / 60; + let secs = elapsed % 60; + let line = format_collapsed_line( + label, + mins, + secs, + stats.tool_count, + &detail, + ); + print!("\x1b[1A\x1b[2K "); + terminal::print_colored(&line, Color::Cyan); + println!(); + } else { + print!(" "); + terminal::print_colored( + &format!("[🔧 {}]", terminal::ts()), + Color::Magenta, + ); + println!(" {}", detail); } - let elapsed = start.elapsed().as_secs(); - let mins = elapsed / 60; - let secs = elapsed % 60; - let line = format_collapsed_line( - label, - mins, - secs, - stats.tool_count, - &detail, - ); - print!("\x1b[1A\x1b[2K "); - terminal::print_colored(&line, Color::Cyan); - println!(); - } else { - print!(" "); - terminal::print_colored( - &format!("[🔧 {}]", terminal::ts()), - Color::Magenta, - ); - println!(" {}", detail); } + _ => {} } - _ => {} } } } - } - Some("result") => { - if let Some(text) = parsed.get("result").and_then(|r| r.as_str()) { - result_text = text.to_string(); - } - // Extract cost - if let Some(cost) = parsed.get("total_cost_usd").and_then(|c| c.as_f64()) { - usage.cost_usd = cost; - } - // Extract token usage - if let Some(u) = parsed.get("usage") { - if let Some(v) = u.get("input_tokens").and_then(|t| t.as_u64()) { - usage.input_tokens = v; + Some("result") => { + if let Some(text) = parsed.get("result").and_then(|r| r.as_str()) { + result_text = text.to_string(); } - if let Some(v) = u.get("output_tokens").and_then(|t| t.as_u64()) { - usage.output_tokens = v; + if let Some(cost) = parsed.get("total_cost_usd").and_then(|c| c.as_f64()) { + usage.cost_usd = cost; } - if let Some(v) = u - .get("cache_creation_input_tokens") - .and_then(|t| t.as_u64()) - { - usage.cache_creation_input_tokens = v; - } - if let Some(v) = u.get("cache_read_input_tokens").and_then(|t| t.as_u64()) { - usage.cache_read_input_tokens = v; + if let Some(u) = parsed.get("usage") { + if let Some(v) = u.get("input_tokens").and_then(|t| t.as_u64()) { + usage.input_tokens = v; + } + if let Some(v) = u.get("output_tokens").and_then(|t| t.as_u64()) { + usage.output_tokens = v; + } + if let Some(v) = u + .get("cache_creation_input_tokens") + .and_then(|t| t.as_u64()) + { + usage.cache_creation_input_tokens = v; + } + if let Some(v) = + u.get("cache_read_input_tokens").and_then(|t| t.as_u64()) + { + usage.cache_read_input_tokens = v; + } } } + _ => {} } - _ => {} + } + Err(mpsc::RecvTimeoutError::Timeout) => { + // No output for idle_timeout_secs — kill the agent + terminal::log_warn(&format!( + "Agent '{}' idle for {}s — killing process.", + label, idle_timeout_secs + )); + let _ = child.kill(); + timed_out = true; + break; + } + Err(mpsc::RecvTimeoutError::Disconnected) => { + // Reader thread finished (stdout closed) — agent is done + break; } } } + let _ = reader_handle.join(); + let status = child.wait().context("Failed to wait for claude process")?; let stderr_output = stderr_handle.join().unwrap_or_default(); @@ -305,10 +365,16 @@ pub fn run_agent( ticker_running.store(false, Ordering::Relaxed); let _ = ticker_handle.join(); - if !status.success() { + if timed_out || !status.success() { let code = status.code().unwrap_or(-1); let elapsed = start.elapsed().as_secs(); + let failure_reason = if timed_out { + format!("IDLE TIMEOUT ({}s no output)", idle_timeout_secs) + } else { + format!("FAILED exit {}", code) + }; + // Show stderr if present if !stderr_output.trim().is_empty() { terminal::log_warn("Agent stderr:"); @@ -322,8 +388,8 @@ pub fn run_agent( print!("\x1b[1A\x1b[2K "); terminal::print_colored( &format!( - "x {} ({}s, {} tools — FAILED exit {})", - label, elapsed, stats.tool_count, code + "x {} ({}s, {} tools — {})", + label, elapsed, stats.tool_count, failure_reason ), Color::Red, ); @@ -343,6 +409,7 @@ pub fn run_agent( if let Some(path) = error_log_path { let mut content = "# Last Error\n\n".to_string(); content.push_str(&format!("- **Agent:** {}\n", label)); + content.push_str(&format!("- **Reason:** {}\n", failure_reason)); content.push_str(&format!("- **Exit code:** {}\n", code)); content.push_str(&format!("- **Elapsed:** {}s\n", elapsed)); content.push_str(&format!("- **Tool count:** {}\n", stats.tool_count)); @@ -358,7 +425,6 @@ pub fn run_agent( } if !stderr_output.trim().is_empty() { content.push_str("\n## Stderr\n\n```\n"); - // Cap at 2048 chars to avoid bloating the error log let capped = truncate_str(&stderr_output, 2048); content.push_str(&capped); content.push_str("\n```\n"); @@ -366,10 +432,20 @@ pub fn run_agent( let _ = std::fs::write(path, &content); } - anyhow::bail!( - "Agent '{}' exited with code {}. Check the output above for errors. Run `lisa resume` to retry this phase.", - label, code - ); + if timed_out { + return Err(AgentError::IdleTimeout { + label: label.to_string(), + elapsed_secs: elapsed, + idle_limit: idle_timeout_secs, + } + .into()); + } else { + return Err(AgentError::Other(anyhow::anyhow!( + "Agent '{}' exited with code {}. Check the output above for errors. Run `lisa resume` to retry this phase.", + label, code + )) + .into()); + } } let elapsed = start.elapsed().as_secs(); diff --git a/src/cli.rs b/src/cli.rs index f2194f5..6b1156e 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -7,7 +7,7 @@ use clap::{Parser, Subcommand}; after_long_help = "\ WORKFLOW: - lisa init → scaffold .lisa/ + ASSIGNMENT.md + lisa init → scaffold .lisa/ + ASSIGNMENT.md + codebase discovery │ ▼ ┌───────────────────────────────────────────────┐ @@ -27,7 +27,13 @@ WORKFLOW: │ Each pass widens scope & tightens tolerances │◄── human gate └──────────────────┬────────────────────────────┘ ▼ - lisa finalize → answer.md + report.md + Finalize at review gate → answer.md + report.md + +ARTIFACTS: + + All process artifacts live in .lisa/ — methodology, plans, skills, + validation results, plots, and per-pass summaries. + See .lisa/CLAUDE.md for a full map of what's where. " )] #[command(version)] @@ -47,7 +53,7 @@ pub enum Commands { #[arg(long)] tech: Option, }, - /// Run the full spiral (scope if needed, then iterate) + /// Run the full spiral (scope if needed, then iterate). If complete, continue with --follow-up. Run { /// Maximum number of spiral passes #[arg(long)] @@ -58,6 +64,9 @@ pub enum Commands { /// Show full agent output (overrides collapse_output config) #[arg(long, short)] verbose: bool, + /// Continue a completed spiral with a follow-up question + #[arg(long)] + follow_up: Option, }, /// Resume from saved state Resume { @@ -68,20 +77,12 @@ pub enum Commands { #[arg(long, short)] verbose: bool, }, - /// Run only Pass 0 (scoping) - Scope, - /// Run the DDV Agent (write or extend domain verification scenarios) - Ddv, - /// Print current spiral state + /// Print current spiral state and pass history Status, /// Check environment and prerequisites Doctor, - /// Produce final deliverables - Finalize, /// Copy compiled-in prompts to .lisa/prompts/ for customization EjectPrompts, - /// Show pass-by-pass history (answer, tests, recommendation) - History, /// Roll back to a previous pass boundary Rollback { /// Pass number to roll back to (e.g., 1 for end of pass 1) @@ -90,18 +91,4 @@ pub enum Commands { #[arg(long)] force: bool, }, - /// Continue with a follow-up question after a completed spiral - Continue { - /// The follow-up question or task - question: String, - /// Maximum number of additional spiral passes - #[arg(long)] - max_passes: Option, - /// Skip all human review gates - #[arg(long)] - no_pause: bool, - /// Show full agent output (overrides collapse_output config) - #[arg(long, short)] - verbose: bool, - }, } diff --git a/src/config.rs b/src/config.rs index d0d4fba..3e2c0ef 100644 --- a/src/config.rs +++ b/src/config.rs @@ -34,12 +34,10 @@ pub struct ModelsConfig { pub scope: String, #[serde(default = "default_opus")] pub refine: String, - #[serde(default = "default_opus")] - pub ddv: String, #[serde(default = "default_sonnet")] pub build: String, #[serde(default = "default_opus")] - pub validate: String, + pub audit: String, } impl Default for ModelsConfig { @@ -47,9 +45,8 @@ impl Default for ModelsConfig { Self { scope: default_opus(), refine: default_opus(), - ddv: default_opus(), build: default_sonnet(), - validate: default_opus(), + audit: default_opus(), } } } @@ -67,12 +64,18 @@ pub struct LimitsConfig { pub max_spiral_passes: u32, #[serde(default = "default_max_ralph_iterations")] pub max_ralph_iterations: u32, + #[serde(default = "default_max_tasks_per_pass")] + pub max_tasks_per_pass: u32, #[serde(default = "default_stall_threshold")] pub stall_threshold: u32, #[serde(default)] pub budget_usd: f64, #[serde(default = "default_budget_warn_pct")] pub budget_warn_pct: u32, + #[serde(default = "default_idle_timeout_secs")] + pub idle_timeout_secs: u64, + #[serde(default = "default_max_agent_retries")] + pub max_agent_retries: u32, } impl Default for LimitsConfig { @@ -80,9 +83,12 @@ impl Default for LimitsConfig { Self { max_spiral_passes: default_max_spiral_passes(), max_ralph_iterations: default_max_ralph_iterations(), + max_tasks_per_pass: default_max_tasks_per_pass(), stall_threshold: default_stall_threshold(), budget_usd: 0.0, budget_warn_pct: default_budget_warn_pct(), + idle_timeout_secs: default_idle_timeout_secs(), + max_agent_retries: default_max_agent_retries(), } } } @@ -91,7 +97,10 @@ fn default_max_spiral_passes() -> u32 { 5 } fn default_max_ralph_iterations() -> u32 { - 50 + 15 +} +fn default_max_tasks_per_pass() -> u32 { + 5 } fn default_stall_threshold() -> u32 { 2 @@ -99,6 +108,12 @@ fn default_stall_threshold() -> u32 { fn default_budget_warn_pct() -> u32 { 80 } +fn default_idle_timeout_secs() -> u64 { + 300 +} +fn default_max_agent_retries() -> u32 { + 2 +} #[derive(Debug, Clone, Serialize, Deserialize)] pub struct ReviewConfig { @@ -155,8 +170,8 @@ pub struct PathsConfig { pub lisa_root: String, #[serde(default = "default_source")] pub source: Vec, - #[serde(default = "default_tests_ddv")] - pub tests_ddv: String, + #[serde(default = "default_tests_bounds")] + pub tests_bounds: String, #[serde(default = "default_tests_software")] pub tests_software: String, #[serde(default = "default_tests_integration")] @@ -168,7 +183,7 @@ impl Default for PathsConfig { Self { lisa_root: default_lisa_root(), source: default_source(), - tests_ddv: default_tests_ddv(), + tests_bounds: default_tests_bounds(), tests_software: default_tests_software(), tests_integration: default_tests_integration(), } @@ -179,16 +194,16 @@ fn default_lisa_root() -> String { ".lisa".to_string() } fn default_source() -> Vec { - vec!["src".to_string()] + vec![] } -fn default_tests_ddv() -> String { - "tests/ddv".to_string() +fn default_tests_bounds() -> String { + String::new() } fn default_tests_software() -> String { - "tests/software".to_string() + String::new() } fn default_tests_integration() -> String { - "tests/integration".to_string() + String::new() } #[derive(Debug, Clone, Default, Serialize, Deserialize)] @@ -208,7 +223,7 @@ pub struct CommandsConfig { #[serde(default)] pub test_all: String, #[serde(default)] - pub test_ddv: String, + pub test_bounds: String, #[serde(default)] pub test_software: String, #[serde(default)] @@ -234,6 +249,23 @@ impl Config { pub fn source_dirs_display(&self) -> String { self.paths.source.join(", ") } + + /// Check that paths are configured (non-empty). + /// Returns Ok if paths are set, or an error directing the user to run init or fill lisa.toml. + pub fn validate_paths(&self) -> Result<()> { + if self.paths.source.is_empty() + || self.paths.tests_bounds.is_empty() + || self.paths.tests_software.is_empty() + || self.paths.tests_integration.is_empty() + { + anyhow::bail!( + "Paths not configured in lisa.toml [paths] section.\n\ + Run `lisa init` to auto-detect project structure, or manually fill in:\n\ + source, tests_bounds, tests_software, tests_integration" + ); + } + Ok(()) + } } #[cfg(test)] @@ -248,15 +280,18 @@ mod tests { assert_eq!(config.models.scope, "opus"); assert_eq!(config.models.build, "sonnet"); assert_eq!(config.limits.max_spiral_passes, 5); - assert_eq!(config.limits.max_ralph_iterations, 50); + assert_eq!(config.limits.max_ralph_iterations, 15); + assert_eq!(config.limits.max_tasks_per_pass, 5); assert_eq!(config.limits.stall_threshold, 2); assert!(config.review.pause); assert!(config.git.auto_commit); assert!(!config.git.auto_push); assert!(config.terminal.collapse_output); assert_eq!(config.paths.lisa_root, ".lisa"); - assert_eq!(config.paths.source, vec!["src"]); - assert_eq!(config.paths.tests_ddv, "tests/ddv"); + assert!(config.paths.source.is_empty()); + assert_eq!(config.paths.tests_bounds, ""); + assert_eq!(config.limits.idle_timeout_secs, 300); + assert_eq!(config.limits.max_agent_retries, 2); assert!(config.agent.extra_args.is_empty()); } @@ -295,7 +330,30 @@ name = "minimal" fn test_source_dirs_display() { let toml_str = default_config_toml("test"); let config: Config = toml::from_str(&toml_str).unwrap(); - assert_eq!(config.source_dirs_display(), "src"); + assert_eq!(config.source_dirs_display(), ""); + } + + #[test] + fn test_validate_paths_empty() { + let toml_str = default_config_toml("test"); + let config: Config = toml::from_str(&toml_str).unwrap(); + assert!(config.validate_paths().is_err()); + } + + #[test] + fn test_validate_paths_filled() { + let toml_str = r#" +[project] +name = "filled" + +[paths] +source = ["src"] +tests_bounds = "tests/bounds" +tests_software = "tests/software" +tests_integration = "tests/integration" +"#; + let config: Config = toml::from_str(toml_str).unwrap(); + assert!(config.validate_paths().is_ok()); } } @@ -307,16 +365,18 @@ name = "{name}" [models] scope = "opus" refine = "opus" -ddv = "opus" build = "sonnet" -validate = "opus" +audit = "opus" [limits] max_spiral_passes = 5 -max_ralph_iterations = 50 +max_ralph_iterations = 15 +max_tasks_per_pass = 5 stall_threshold = 2 # budget_usd = 0.0 # 0 = unlimited # budget_warn_pct = 80 # warn at this % of budget +idle_timeout_secs = 300 # kill agent after 5 min with no output +max_agent_retries = 2 # auto-retry on idle timeout before surfacing to human [review] # Human review gates. When false, loop runs fully autonomously. @@ -334,13 +394,15 @@ collapse_output = true # Where process artifacts live (relative to project root) lisa_root = ".lisa" -# Where deliverable code goes (relative to project root) -source = ["src"] +# Where deliverable code goes (relative to project root). +# Resolved by the init agent; fill manually if needed. +source = [] -# Test directories (relative to project root) -tests_ddv = "tests/ddv" -tests_software = "tests/software" -tests_integration = "tests/integration" +# Test directories (relative to project root). +# Resolved by the init agent; fill manually if needed. +tests_bounds = "" +tests_software = "" +tests_integration = "" [agent] # Extra CLI flags passed to every claude invocation. @@ -352,7 +414,7 @@ extra_args = [] setup = "" build = "" test_all = "" -test_ddv = "" +test_bounds = "" test_software = "" test_integration = "" lint = "" diff --git a/src/git.rs b/src/git.rs index b688195..d888221 100644 --- a/src/git.rs +++ b/src/git.rs @@ -10,10 +10,42 @@ pub fn commit_all(msg: &str, config: &Config) -> Result { return Ok(false); } - terminal::log_info("Staging all changes..."); + terminal::log_info("Staging changes..."); + + // Stage only deliverable paths (source, tests, config files). + // .lisa/ is gitignored and never committed. + let mut paths: Vec = Vec::new(); + paths.extend(config.paths.source.iter().cloned()); + for test_dir in [ + &config.paths.tests_bounds, + &config.paths.tests_software, + &config.paths.tests_integration, + ] { + if !test_dir.is_empty() { + paths.push(test_dir.clone()); + } + } + // Always include project-root files the agents may modify + paths.extend( + ["ASSIGNMENT.md", "lisa.toml", ".gitignore"] + .iter() + .map(|s| s.to_string()), + ); + + // Deduplicate + paths.sort(); + paths.dedup(); + + if paths.is_empty() { + terminal::log_info("No paths configured to stage."); + return Ok(false); + } + + let mut args = vec!["add".to_string(), "--".to_string()]; + args.extend(paths); let status = Command::new("git") - .args(["add", "-A"]) + .args(&args) .status() .context("Failed to run git add")?; @@ -186,17 +218,58 @@ pub fn has_uncommitted_changes() -> Result { Ok(!String::from_utf8_lossy(&output.stdout).trim().is_empty()) } -/// Retrieve a file from another branch via `git show :`. -pub fn show_file_from_ref(git_ref: &str, path: &str) -> Result> { +/// Checkout a branch or ref. +pub fn checkout(target: &str) -> Result<()> { + let status = Command::new("git") + .args(["checkout", target]) + .status() + .context("Failed to run git checkout")?; + if !status.success() { + anyhow::bail!("git checkout {} failed", target); + } + Ok(()) +} + +/// Get the current branch name. +pub fn current_branch() -> Result { let output = Command::new("git") - .args(["show", &format!("{}:{}", git_ref, path)]) + .args(["rev-parse", "--abbrev-ref", "HEAD"]) .output() - .context("Failed to run git show")?; - if output.status.success() { - Ok(Some(String::from_utf8_lossy(&output.stdout).to_string())) - } else { - Ok(None) + .context("Failed to get current branch")?; + if !output.status.success() { + anyhow::bail!("git rev-parse --abbrev-ref HEAD failed"); + } + Ok(String::from_utf8_lossy(&output.stdout).trim().to_string()) +} + +/// Merge a branch into the current branch with --no-ff. +pub fn merge_branch(branch: &str) -> Result<()> { + let status = Command::new("git") + .args([ + "merge", + branch, + "--no-ff", + "-m", + &format!("Merge exploration: {}", branch), + ]) + .status() + .context("Failed to run git merge")?; + if !status.success() { + anyhow::bail!("git merge {} failed — resolve conflicts manually", branch); } + Ok(()) +} + +/// Delete a local branch. +pub fn delete_branch(name: &str) -> Result<()> { + let status = Command::new("git") + .args(["branch", "-D", name]) + .status() + .context("Failed to delete git branch")?; + if !status.success() { + anyhow::bail!("git branch -D {} failed", name); + } + Ok(()) } #[cfg(test)] diff --git a/src/init/scaffold.rs b/src/init/scaffold.rs index f06eee6..3c81541 100644 --- a/src/init/scaffold.rs +++ b/src/init/scaffold.rs @@ -3,17 +3,11 @@ use crossterm::style::Color; use std::io::IsTerminal; use std::path::Path; -use crate::config::{default_config_toml, PathsConfig}; +use crate::agent; +use crate::config::default_config_toml; +use crate::prompt; use crate::terminal; -/// Returns true if the path looks like a file (has an extension in its final component). -fn looks_like_file(path: &str) -> bool { - Path::new(path) - .file_name() - .and_then(|f| Path::new(f).extension()) - .is_some() -} - // Compiled-in templates const ASSIGNMENT_TEMPLATE: &str = include_str!("../../templates/assignment.md"); const STACK_TEMPLATE: &str = include_str!("../../templates/stack.md"); @@ -23,8 +17,7 @@ const ASSUMPTIONS_REGISTER_TEMPLATE: &str = include_str!("../../templates/assump const SANITY_CHECKS_TEMPLATE: &str = include_str!("../../templates/sanity_checks.md"); const LIMITING_CASES_TEMPLATE: &str = include_str!("../../templates/limiting_cases.md"); const REFERENCE_DATA_TEMPLATE: &str = include_str!("../../templates/reference_data.md"); -const PLOTS_REVIEW_TEMPLATE: &str = include_str!("../../templates/plots_review.md"); -const DDV_SCENARIOS_TEMPLATE: &str = include_str!("../../templates/ddv_scenarios.md"); +const LISA_CLAUDE_MD: &str = include_str!("../../templates/lisa_claude.md"); pub fn run(project_root: &Path, name: Option, tech: Option) -> Result<()> { let lisa_root = project_root.join(".lisa"); @@ -73,9 +66,8 @@ pub fn run(project_root: &Path, name: Option, tech: Option) -> R String::new() }; - let paths = PathsConfig::default(); - - // Create directory structure + // Create .lisa/ process infrastructure only — no source or test dirs. + // The init agent (Phase 2) resolves project-specific paths. let dirs = [ ".lisa", ".lisa/methodology", @@ -85,23 +77,12 @@ pub fn run(project_root: &Path, name: Option, tech: Option) -> R ".lisa/validation", ".lisa/references/core", ".lisa/references/retrieved", - ".lisa/ddv", - ".lisa/plots", + ".lisa/skills", ".lisa/output", - ]; - let source_dirs: Vec<&str> = paths.source.iter().map(|s| s.as_str()).collect(); - let source_dirs_only: Vec<&&str> = source_dirs.iter().filter(|s| !looks_like_file(s)).collect(); - let test_dirs = [ - paths.tests_ddv.as_str(), - paths.tests_software.as_str(), - paths.tests_integration.as_str(), + ".lisa/prompts/scope", ]; - for dir in dirs - .iter() - .chain(source_dirs_only.iter().copied()) - .chain(test_dirs.iter()) - { + for dir in &dirs { std::fs::create_dir_all(project_root.join(dir)) .with_context(|| format!("Failed to create directory: {}", dir))?; } @@ -152,29 +133,16 @@ pub fn run(project_root: &Path, name: Option, tech: Option) -> R &lisa_root.join("validation/reference-data.md"), REFERENCE_DATA_TEMPLATE, )?; - // Write plots review - write_file(&lisa_root.join("plots/REVIEW.md"), PLOTS_REVIEW_TEMPLATE)?; + // Write skill files + for (filename, content) in crate::prompt::SKILLS { + write_file(&lisa_root.join(format!("skills/{}", filename)), content)?; + } - // Write DDV templates - write_file(&lisa_root.join("ddv/scenarios.md"), DDV_SCENARIOS_TEMPLATE)?; + // Write CLAUDE.md inside .lisa/ so agents can discover artifacts + write_file(&lisa_root.join("CLAUDE.md"), LISA_CLAUDE_MD)?; - // Write .gitkeep files - let lisa_keepdirs = [ - ".lisa/methodology/derivations", - ".lisa/references/core", - ".lisa/references/retrieved", - ".lisa/output", - ]; - for dir in lisa_keepdirs - .iter() - .chain(test_dirs.iter()) - .chain(source_dirs_only.iter().copied()) - { - let keepfile = project_root.join(dir).join(".gitkeep"); - if !keepfile.exists() { - std::fs::write(&keepfile, "")?; - } - } + // Ensure .lisa/ is gitignored + ensure_gitignore(project_root)?; // Write initial state crate::state::save_state(&lisa_root, &crate::state::SpiralState::NotStarted)?; @@ -194,43 +162,42 @@ pub fn run(project_root: &Path, name: Option, tech: Option) -> R println!("Spiral state (auto-managed)"); terminal::print_colored(" .lisa/validation/ ", Color::Cyan); println!("V&V artifacts (auto-managed)"); - terminal::print_colored(" .lisa/ddv/ ", Color::Cyan); - println!("DDV verification scenarios"); - terminal::print_colored(" .lisa/plots/ ", Color::Cyan); - println!("Verification plots"); - let source_display: Vec = paths - .source - .iter() - .map(|s| { - if looks_like_file(s) { - s.clone() - } else { - format!("{}/", s) - } - }) - .collect(); - terminal::print_colored( - &format!(" {:<33}", source_display.join(", ")), - Color::Cyan, - ); - println!("Implementation code"); - terminal::print_colored( - &format!(" {:<33}", format!("{}/", paths.tests_ddv)), - Color::Cyan, - ); - println!("Domain verification tests"); - terminal::print_colored( - &format!(" {:<33}", format!("{}/", paths.tests_software)), - Color::Cyan, - ); - println!("Software quality tests"); - terminal::print_colored( - &format!(" {:<33}", format!("{}/", paths.tests_integration)), - Color::Cyan, - ); - println!("End-to-end tests"); + terminal::print_colored(" .lisa/skills/ ", Color::Cyan); + println!("Engineering skills for agents"); + terminal::print_colored(" .lisa/CLAUDE.md ", Color::Cyan); + println!("Artifact guide for AI agents"); println!(); + // Phase 2: Run the init agent to examine the codebase and resolve paths + terminal::log_phase("INIT AGENT — Examining project structure"); + let init_prompt = prompt::load_prompt(prompt::Phase::Init, &lisa_root); + let init_prompt = prompt::render_prompt( + &init_prompt, + &crate::config::Config::load(project_root)?, + None, + ); + + match agent::run_agent( + &init_prompt, + "opus", + "Init Agent", + true, + Some(&lisa_root.join("last-error.md")), + &[], + 300, // 5 min idle timeout for init agent + ) { + Ok(_result) => { + terminal::log_success("Init agent completed — project structure resolved."); + } + Err(e) => { + terminal::log_warn(&format!("Init agent failed: {}", e)); + terminal::log_warn( + "Paths not resolved. Fill [paths] in lisa.toml manually, or re-run `lisa init`.", + ); + } + } + + println!(); terminal::println_bold(" Next steps:"); println!(" 1. Edit ASSIGNMENT.md with the full assignment"); println!(" 2. Add reference papers to .lisa/references/core/"); @@ -240,6 +207,31 @@ pub fn run(project_root: &Path, name: Option, tech: Option) -> R Ok(()) } +/// Ensure `.lisa/` is listed in the project's `.gitignore`. +fn ensure_gitignore(project_root: &Path) -> Result<()> { + let gitignore_path = project_root.join(".gitignore"); + let entry = ".lisa/"; + + if gitignore_path.exists() { + let content = std::fs::read_to_string(&gitignore_path)?; + // Already present — nothing to do + if content.lines().any(|line| line.trim() == entry) { + return Ok(()); + } + // Append with a preceding newline if the file doesn't end with one + let separator = if content.ends_with('\n') { "" } else { "\n" }; + std::fs::write( + &gitignore_path, + format!("{}{}{}\n", content, separator, entry), + )?; + } else { + std::fs::write(&gitignore_path, format!("{}\n", entry))?; + } + + terminal::log_info("Added .lisa/ to .gitignore"); + Ok(()) +} + fn write_file(path: &Path, content: &str) -> Result<()> { if let Some(parent) = path.parent() { std::fs::create_dir_all(parent)?; @@ -247,19 +239,3 @@ fn write_file(path: &Path, content: &str) -> Result<()> { std::fs::write(path, content).with_context(|| format!("Failed to write {}", path.display()))?; Ok(()) } - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_looks_like_file() { - assert!(looks_like_file("src/lib.rs")); - assert!(looks_like_file("src/my_module/mod.rs")); - assert!(looks_like_file("lib.rs")); - assert!(looks_like_file("path/to/file.txt")); - assert!(!looks_like_file("src")); - assert!(!looks_like_file("src/my_module")); - assert!(!looks_like_file("tests/ddv")); - } -} diff --git a/src/main.rs b/src/main.rs index 91a0818..5f67263 100644 --- a/src/main.rs +++ b/src/main.rs @@ -35,12 +35,19 @@ fn main() -> Result<()> { max_passes, no_pause, verbose, + follow_up, } => { let mut config = load_config()?; if verbose { config.terminal.collapse_output = false; } - orchestrator::run(&config, &project_root(), max_passes, no_pause) + orchestrator::run( + &config, + &project_root(), + max_passes, + no_pause, + follow_up.as_deref(), + ) } cli::Commands::Resume { no_pause, verbose } => { let mut config = load_config()?; @@ -49,60 +56,13 @@ fn main() -> Result<()> { } orchestrator::resume(&config, &project_root(), no_pause) } - cli::Commands::Scope => { - let config = load_config()?; - orchestrator::run_scope_only(&config, &project_root()) - } - cli::Commands::Ddv => { - let config = load_config()?; - orchestrator::run_ddv_agent_only(&config, &project_root()) - } cli::Commands::Status => cmd_status(), cli::Commands::Doctor => cmd_doctor(), - cli::Commands::Finalize => { - let config = load_config()?; - let lisa_root = config.lisa_root(&project_root()); - let state = state::load_state(&lisa_root)?; - match state { - state::SpiralState::PassReview { pass } => { - orchestrator::finalize(&config, &project_root(), pass) - } - state::SpiralState::InPass { pass, ref phase } => { - terminal::log_warn(&format!( - "Finalizing from mid-pass (pass {}, phase {}). \ - The validate phase has not run — review-package.md may not exist.", - pass, phase - )); - orchestrator::finalize(&config, &project_root(), pass) - } - state::SpiralState::Complete { final_pass } => { - terminal::log_info(&format!("Spiral already complete at pass {}.", final_pass)); - Ok(()) - } - _ => { - terminal::log_error("Cannot finalize: no pass has been completed yet."); - Ok(()) - } - } - } cli::Commands::EjectPrompts => cmd_eject_prompts(), - cli::Commands::History => cmd_history(), cli::Commands::Rollback { pass, force } => { let config = load_config()?; orchestrator::rollback(&config, &project_root(), pass, force) } - cli::Commands::Continue { - question, - max_passes, - no_pause, - verbose, - } => { - let mut config = load_config()?; - if verbose { - config.terminal.collapse_output = false; - } - orchestrator::continue_spiral(&config, &project_root(), &question, max_passes, no_pause) - } } } @@ -204,6 +164,88 @@ fn cmd_status() -> Result<()> { let tag_strs: Vec = tags.iter().map(|t| t.to_string()).collect(); println!(" Rollback points: pass {}", tag_strs.join(", ")); } + + // Show pass history table (merged from cmd_history) + let spiral_dir = lisa_root.join("spiral"); + if spiral_dir.exists() { + let mut passes: Vec = Vec::new(); + if let Ok(entries) = std::fs::read_dir(&spiral_dir) { + for entry in entries.filter_map(|e| e.ok()) { + let name = entry.file_name(); + let name_str = name.to_string_lossy(); + if let Some(num_str) = name_str.strip_prefix("pass-") { + if let Ok(num) = num_str.parse::() { + if num > 0 { + passes.push(num); + } + } + } + } + } + passes.sort(); + + if !passes.is_empty() { + println!(); + terminal::println_bold(" Pass History"); + println!(); + println!( + " {:>4} {:<30} {:<8} {:<7} {:<8} Status", + "Pass", "Answer", "Bounds", "Sanity", "Cost" + ); + println!( + " {:>4} {:<30} {:<8} {:<7} {:<8} --------------", + "----", "------------------------------", "--------", "-------", "--------" + ); + + for pass_num in &passes { + let review_path = + lisa_root.join(format!("spiral/pass-{}/review-package.md", pass_num)); + if !review_path.exists() { + continue; + } + let content = match std::fs::read_to_string(&review_path) { + Ok(c) => c, + Err(_) => continue, + }; + + let answer = + review::extract_section_first_line(&content, "## Current Answer") + .unwrap_or_else(|| "-".to_string()); + let answer_trunc = truncate_str(&answer, 30); + + let bounds = + extract_bounds_summary(&content).unwrap_or_else(|| "-".to_string()); + let bounds_trunc = truncate_str(&bounds, 8); + + let sanity = + extract_sanity_summary(&content).unwrap_or_else(|| "-".to_string()); + let sanity_trunc = truncate_str(&sanity, 7); + + let rec = + review::extract_section_first_line(&content, "## Status Assessment") + .or_else(|| { + review::extract_section_first_line( + &content, + "## Recommendation", + ) + }) + .unwrap_or_else(|| "-".to_string()); + + let cost = ledger.pass_cost(*pass_num); + let cost_str = if cost > 0.0 { + format!("${:.4}", cost) + } else { + "-".to_string() + }; + let cost_trunc = truncate_str(&cost_str, 8); + + println!( + " {:>4} {:<30} {:<8} {:<7} {:<8} {}", + pass_num, answer_trunc, bounds_trunc, sanity_trunc, cost_trunc, rec + ); + } + } + } } } println!(); @@ -354,12 +396,13 @@ fn cmd_eject_prompts() -> Result<()> { std::fs::create_dir_all(&prompts_dir)?; let prompts = [ + ("init.md", prompt::PROMPT_INIT), ("scope.md", prompt::PROMPT_SCOPE), ("refine.md", prompt::PROMPT_REFINE), - ("ddv_agent.md", prompt::PROMPT_DDV_AGENT), ("build.md", prompt::PROMPT_BUILD), - ("validate.md", prompt::PROMPT_VALIDATE), + ("audit.md", prompt::PROMPT_AUDIT), ("finalize.md", prompt::PROMPT_FINALIZE), + ("explore.md", prompt::PROMPT_EXPLORE), ]; for (filename, content) in &prompts { @@ -372,121 +415,47 @@ fn cmd_eject_prompts() -> Result<()> { } } - println!(); - terminal::log_info("Prompts ejected to .lisa/prompts/"); - terminal::log_info("Edit them freely — the CLI will use local prompts when present."); - println!(); - - Ok(()) -} - -fn cmd_history() -> Result<()> { - let root = project_root(); - let lisa_root = match load_config() { - Ok(config) => config.lisa_root(&root), - Err(_) => root.join(".lisa"), - }; - - if !lisa_root.exists() { - terminal::log_error("No .lisa/ directory found. Run `lisa init` first."); - return Ok(()); - } - - let spiral_dir = lisa_root.join("spiral"); - if !spiral_dir.exists() { - terminal::log_error("No spiral directory found. Run `lisa run` first."); - return Ok(()); - } - - // Collect pass directories (skip pass-0) - let mut passes: Vec = Vec::new(); - if let Ok(entries) = std::fs::read_dir(&spiral_dir) { - for entry in entries.filter_map(|e| e.ok()) { - let name = entry.file_name(); - let name_str = name.to_string_lossy(); - if let Some(num_str) = name_str.strip_prefix("pass-") { - if let Ok(num) = num_str.parse::() { - if num > 0 { - passes.push(num); - } - } - } + // Also eject scope artifact spec files + let scope_dir = prompts_dir.join("scope"); + std::fs::create_dir_all(&scope_dir)?; + for (filename, content) in prompt::SCOPE_SPECS { + let path = scope_dir.join(filename); + if path.exists() { + terminal::log_warn(&format!(" Skipping scope/{} (already exists)", filename)); + } else { + std::fs::write(&path, content)?; + terminal::log_success(&format!(" Written scope/{}", filename)); } } - passes.sort(); - - if passes.is_empty() { - terminal::log_info("No completed passes found (only pass-0 exists)."); - return Ok(()); - } - - let ledger = usage::load_usage(&lisa_root).unwrap_or_default(); - - println!(); - terminal::println_bold("Lisa Loop — Pass History"); - println!(); - - // Header - println!( - " {:>4} {:<30} {:<8} {:<7} {:<8} Status", - "Pass", "Answer", "DDV", "Sanity", "Cost" - ); - println!( - " {:>4} {:<30} {:<8} {:<7} {:<8} --------------", - "----", "------------------------------", "--------", "-------", "--------" - ); - - for pass in &passes { - let review_path = lisa_root.join(format!("spiral/pass-{}/review-package.md", pass)); - if !review_path.exists() { - continue; - } - let content = match std::fs::read_to_string(&review_path) { - Ok(c) => c, - Err(_) => continue, - }; - - let answer = review::extract_section_first_line(&content, "## Current Answer") - .unwrap_or_else(|| "-".to_string()); - let answer_trunc = truncate_str(&answer, 30); - - let ddv = extract_ddv_summary(&content).unwrap_or_else(|| "-".to_string()); - let ddv_trunc = truncate_str(&ddv, 8); - - let sanity = extract_sanity_summary(&content).unwrap_or_else(|| "-".to_string()); - let sanity_trunc = truncate_str(&sanity, 7); - - let rec = review::extract_section_first_line(&content, "## Status Assessment") - .or_else(|| review::extract_section_first_line(&content, "## Recommendation")) - .unwrap_or_else(|| "-".to_string()); - - let cost = ledger.pass_cost(*pass); - let cost_str = if cost > 0.0 { - format!("${:.4}", cost) + // Also eject skill files + let skills_dir = lisa_root.join("skills"); + std::fs::create_dir_all(&skills_dir)?; + for (filename, content) in prompt::SKILLS { + let path = skills_dir.join(filename); + if path.exists() { + terminal::log_warn(&format!(" Skipping skills/{} (already exists)", filename)); } else { - "-".to_string() - }; - let cost_trunc = truncate_str(&cost_str, 8); - - println!( - " {:>4} {:<30} {:<8} {:<7} {:<8} {}", - pass, answer_trunc, ddv_trunc, sanity_trunc, cost_trunc, rec - ); + std::fs::write(&path, content)?; + terminal::log_success(&format!(" Written skills/{}", filename)); + } } + println!(); + terminal::log_info("Prompts ejected to .lisa/prompts/"); + terminal::log_info("Scope artifact specs ejected to .lisa/prompts/scope/"); + terminal::log_info("Skills ejected to .lisa/skills/"); + terminal::log_info("Edit them freely — the CLI will use local versions when present."); println!(); Ok(()) } -/// Extract DDV test result summary (e.g., "3/4") from review content. -fn extract_ddv_summary(content: &str) -> Option { +/// Extract bounding test result summary (e.g., "3/4") from review content. +fn extract_bounds_summary(content: &str) -> Option { for line in content.lines() { - if line.starts_with("DDV:") { - // Try to extract a fraction like "3/4" or "passed: 3/4" - let text = line.trim_start_matches("DDV:").trim(); - // Look for N/M pattern + if line.starts_with("Bounds:") { + let text = line.trim_start_matches("Bounds:").trim(); if let Some(frac) = extract_fraction(text) { return Some(frac); } diff --git a/src/orchestrator.rs b/src/orchestrator.rs index 9c9f67d..dddb1c8 100644 --- a/src/orchestrator.rs +++ b/src/orchestrator.rs @@ -1,5 +1,6 @@ use anyhow::Result; use crossterm::style::Color; +use std::io::IsTerminal; use std::path::Path; use crate::agent::{self, AgentResult}; @@ -13,18 +14,46 @@ use crate::tasks; use crate::terminal; use crate::usage; -/// Run the full spiral: scope if needed, then iterate passes +/// Run the full spiral: scope if needed, then iterate passes. +/// If the spiral is already complete and `follow_up` is provided (or prompted interactively), +/// continues the spiral with a new question. pub fn run( config: &Config, project_root: &Path, max_passes: Option, no_pause: bool, + follow_up: Option<&str>, ) -> Result<()> { let mut config = config.clone(); if no_pause { config.review.pause = false; } + let lisa_root = config.lisa_root(project_root); + let state = state::load_state(&lisa_root)?; + + // If spiral is complete, handle follow-up continuation + if let SpiralState::Complete { final_pass } = state { + let question = if let Some(q) = follow_up { + q.to_string() + } else if std::io::stdin().is_terminal() { + terminal::log_info(&format!("Spiral complete at pass {}.", final_pass)); + println!(); + let q: String = dialoguer::Input::new() + .with_prompt(" Follow-up question (or Ctrl+C to exit)") + .interact_text()?; + q + } else { + terminal::log_info(&format!( + "Spiral complete at pass {}. Use --follow-up to continue.", + final_pass + )); + return Ok(()); + }; + + return continue_spiral(&config, project_root, &question, max_passes, no_pause); + } + let max = max_passes.unwrap_or(config.limits.max_spiral_passes); if no_pause { @@ -38,14 +67,12 @@ pub fn run( terminal::log_phase(&format!("LISA LOOP — SPIRAL RUN (max {} passes)", max)); ensure_scope_complete(&config, project_root)?; - ensure_ddv_complete(&config, project_root)?; - run_pass_range(&config, project_root, 1, max) -} + // After scope, paths must be resolved (by init agent or scope agent) + let config = reload_config_if_needed(config, project_root)?; + config.validate_paths()?; -/// Run only the scope phase -pub fn run_scope_only(config: &Config, project_root: &Path) -> Result<()> { - run_scope(config, project_root) + run_pass_range(&config, project_root, 1, max) } /// Resume from saved state @@ -73,25 +100,16 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() match state { SpiralState::NotStarted => { terminal::log_info("No previous run found. Starting fresh."); - run(config, project_root, None, no_pause) + run(config, project_root, None, no_pause, None) } SpiralState::Scoping | SpiralState::ScopeReview => { terminal::log_info("Resuming: scope was incomplete."); run_scope(config, project_root)?; - run(config, project_root, None, no_pause) + run(config, project_root, None, no_pause, None) } SpiralState::ScopeComplete => { - terminal::log_info("Scope already complete. Running DDV Agent and spiral passes."); - run(config, project_root, None, no_pause) - } - SpiralState::DdvAgent | SpiralState::DdvAgentReview => { - terminal::log_info("Resuming: DDV Agent was incomplete."); - run_ddv_agent(config, project_root)?; - run(config, project_root, None, no_pause) - } - SpiralState::DdvAgentComplete => { - terminal::log_info("DDV scenarios already complete. Running spiral passes."); - run_pass_range(config, project_root, 1, config.limits.max_spiral_passes) + terminal::log_info("Scope already complete. Running spiral passes."); + run(config, project_root, None, no_pause, None) } SpiralState::InPass { pass, phase } => { resume_from_phase(config, project_root, pass, &phase) @@ -112,11 +130,11 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() if !run_build_loop(config, project_root, pass, 1)? { return Ok(()); } - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; git::push(config)?; git::create_tag(&format!("lisa/pass-{}", pass))?; state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); @@ -128,6 +146,7 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() pass + 1, config.limits.max_spiral_passes, ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } SpiralState::RefineReview { pass } => { @@ -142,11 +161,11 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() if !run_build_loop(config, project_root, pass, 1)? { return Ok(()); } - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; git::push(config)?; git::create_tag(&format!("lisa/pass-{}", pass))?; state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); @@ -158,24 +177,25 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() pass + 1, config.limits.max_spiral_passes, ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } SpiralState::BuildComplete { pass } => { terminal::log_info(&format!( - "Resuming: build complete for pass {}, proceeding to validate.", + "Resuming: build complete for pass {}, proceeding to audit.", pass )); - resume_from_phase(config, project_root, pass, &PassPhase::Validate) + resume_from_phase(config, project_root, pass, &PassPhase::Audit) } - SpiralState::ValidateComplete { pass } => { + SpiralState::AuditComplete { pass } => { terminal::log_info(&format!( - "Resuming: validate complete for pass {}, proceeding to review.", + "Resuming: audit complete for pass {}, proceeding to review.", pass )); git::push(config)?; git::create_tag(&format!("lisa/pass-{}", pass))?; state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); @@ -187,11 +207,36 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() pass + 1, config.limits.max_spiral_passes, ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } SpiralState::PassReview { pass } => { terminal::log_info(&format!("Resuming: review gate of pass {}.", pass)); - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { + ReviewDecision::Finalize => finalize(config, project_root, pass), + ReviewDecision::Quit => { + terminal::log_warn("Stopping after pass review."); + Ok(()) + } + ReviewDecision::Continue | ReviewDecision::Redirect => run_pass_range( + config, + project_root, + pass + 1, + config.limits.max_spiral_passes, + ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), + } + } + SpiralState::Exploring { pass, explore_id } => { + terminal::log_info(&format!( + "Resuming: exploration #{} in pass {} (re-running agent).", + explore_id, pass + )); + // The exploration was interrupted mid-agent. We're on the explore branch. + // Re-run the explore agent (question is preserved in explore dir if available). + run_explore(config, project_root, pass, explore_id)?; + state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); @@ -203,11 +248,56 @@ pub fn resume(config: &Config, project_root: &Path, no_pause: bool) -> Result<() pass + 1, config.limits.max_spiral_passes, ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), + } + } + SpiralState::ExploreReview { pass, explore_id } => { + terminal::log_info(&format!( + "Resuming: explore review for exploration #{} in pass {}.", + explore_id, pass + )); + let original_branch = format!("lisa/pass-{}", pass); + // We may be on the explore branch still, show the review gate + match review::explore_review_gate(pass, explore_id, &lisa_root)? { + review::ExploreDecision::Merge => { + // Try to get back to original branch and merge + let branch_name = format!("lisa/explore-{}-{}", pass, explore_id); + let current = git::current_branch()?; + if current == branch_name { + git::checkout(&original_branch).or_else(|_| git::checkout("main"))?; + git::merge_branch(&branch_name)?; + } + terminal::log_success(&format!("Exploration #{} merged.", explore_id)); + } + review::ExploreDecision::Discard => { + let branch_name = format!("lisa/explore-{}-{}", pass, explore_id); + let current = git::current_branch()?; + if current == branch_name { + git::checkout(&original_branch).or_else(|_| git::checkout("main"))?; + } + let _ = git::delete_branch(&branch_name); + terminal::log_info(&format!("Exploration #{} discarded.", explore_id)); + } + } + state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; + match pass_review_loop(config, project_root, pass, &lisa_root)? { + ReviewDecision::Finalize => finalize(config, project_root, pass), + ReviewDecision::Quit => { + terminal::log_warn("Stopping after pass review."); + Ok(()) + } + ReviewDecision::Continue | ReviewDecision::Redirect => run_pass_range( + config, + project_root, + pass + 1, + config.limits.max_spiral_passes, + ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } SpiralState::Complete { final_pass } => { terminal::log_success(&format!( - "Spiral already complete at pass {}. Use `lisa continue \"\"` to start a follow-up.", + "Spiral already complete at pass {}. Use `lisa run --follow-up \"\"` to continue.", final_pass )); Ok(()) @@ -237,7 +327,7 @@ fn resume_from_phase( if !run_build_loop(config, project_root, pass, 1)? { return Ok(()); } - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; git::push(config)?; } PassPhase::Build { iteration } => { @@ -248,19 +338,19 @@ fn resume_from_phase( if !run_build_loop(config, project_root, pass, *iteration)? { return Ok(()); } - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; git::push(config)?; } - PassPhase::Validate => { + PassPhase::Audit => { terminal::log_info(&format!("Resuming: validate phase at pass {}.", pass)); - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; git::push(config)?; } } git::create_tag(&format!("lisa/pass-{}", pass))?; state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); @@ -272,6 +362,7 @@ fn resume_from_phase( pass + 1, config.limits.max_spiral_passes, ), + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } @@ -309,29 +400,65 @@ fn run_pass_range( )); return Ok(()); } - run_validate(config, project_root, pass)?; + run_audit(config, project_root, pass)?; + + // Generate code-diff.patch capturing this pass's changes + generate_pass_diff(&lisa_root, pass); + git::push(config)?; git::create_tag(&format!("lisa/pass-{}", pass))?; state::save_state(&lisa_root, &SpiralState::PassReview { pass })?; - match review::review_gate(config, pass, &lisa_root)? { + match pass_review_loop(config, project_root, pass, &lisa_root)? { ReviewDecision::Finalize => return finalize(config, project_root, pass), ReviewDecision::Quit => { terminal::log_warn("Stopping after pass review."); return Ok(()); } ReviewDecision::Continue | ReviewDecision::Redirect => continue, + ReviewDecision::Explore => unreachable!("handled in pass_review_loop"), } } terminal::log_warn(&format!( "Reached max spiral passes ({}) without finalization. \ - Run `lisa run --max-passes N` with a higher limit, or `lisa finalize` to finalize current results.", + Run `lisa run --max-passes N` with a higher limit, or `lisa resume` to reach the review gate where you can finalize.", max_pass )); Ok(()) } +/// Generate a code-diff.patch file for the pass, diffing against the previous pass tag. +fn generate_pass_diff(lisa_root: &Path, pass: u32) { + let prev_tag = if pass > 1 { + format!("lisa/pass-{}", pass - 1) + } else { + "lisa/pass-0".to_string() + }; + let diff_path = lisa_root.join(format!("spiral/pass-{}/code-diff.patch", pass)); + let output = std::process::Command::new("git") + .args(["diff", &format!("{}..HEAD", prev_tag)]) + .output(); + if let Ok(out) = output { + if out.status.success() { + let _ = std::fs::write(&diff_path, &out.stdout); + } + } +} + +/// Re-read lisa.toml from disk (agents may have updated [paths] or [commands]). +fn reload_config_if_needed(config: Config, project_root: &Path) -> Result { + match Config::load(project_root) { + Ok(mut fresh) => { + // Preserve runtime overrides (e.g. --no-pause) + fresh.review.pause = config.review.pause; + fresh.terminal.collapse_output = config.terminal.collapse_output; + Ok(fresh) + } + Err(_) => Ok(config), + } +} + /// Return the path to the error log file for a given lisa root. fn error_log(lisa_root: &Path) -> std::path::PathBuf { lisa_root.join("last-error.md") @@ -348,14 +475,50 @@ fn run_agent_with_tracking( pass: u32, ) -> Result { let err_log = error_log(lisa_root); - let result = agent::run_agent( - input, - model, - label, - config.terminal.collapse_output, - Some(&err_log), - &config.agent.extra_args, - )?; + let max_retries = config.limits.max_agent_retries; + + let result = { + let mut last_err = None; + let mut attempt = 0; + + loop { + match agent::run_agent( + input, + model, + label, + config.terminal.collapse_output, + Some(&err_log), + &config.agent.extra_args, + config.limits.idle_timeout_secs, + ) { + Ok(r) => break r, + Err(e) => { + // Only retry on idle timeouts + if e.downcast_ref::() + .is_some_and(|ae| matches!(ae, agent::AgentError::IdleTimeout { .. })) + && attempt < max_retries + { + attempt += 1; + terminal::log_warn(&format!( + "Idle timeout — retrying agent ({}/{})...", + attempt, max_retries + )); + std::thread::sleep(std::time::Duration::from_secs(30)); + last_err = Some(e); + continue; + } + // Non-timeout error or retries exhausted + if attempt > 0 { + terminal::log_error(&format!( + "Agent failed after {} retries. Surfacing error.", + attempt + )); + } + return Err(last_err.unwrap_or(e)); + } + } + } + }; let cumulative = usage::record_invocation( lisa_root, @@ -411,145 +574,16 @@ fn ensure_scope_complete(config: &Config, project_root: &Path) -> Result<()> { if !lisa_root.join("spiral/pass-0/PASS_COMPLETE.md").exists() { terminal::log_info("Pass 0 (scoping) not complete. Running scope first."); run_scope(config, project_root)?; - } else { - terminal::log_info("Pass 0 already complete."); - } - Ok(()) -} - -fn run_ddv_agent(config: &Config, project_root: &Path) -> Result<()> { - let lisa_root = config.lisa_root(project_root); - - terminal::log_phase("DDV AGENT — Writing verification scenarios"); - - if lisa_root.join("ddv/DDV_COMPLETE.md").exists() { - terminal::log_success("DDV scenarios already complete."); - return Ok(()); - } - - state::save_state(&lisa_root, &SpiralState::DdvAgent)?; - std::fs::create_dir_all(lisa_root.join("ddv"))?; - - let input = prompt::build_agent_input(Phase::DdvAgent, config, &lisa_root, 0, None); - let model = Phase::DdvAgent.model_key(config); - - run_agent_with_tracking( - config, - &lisa_root, - &input, - &model, - "DDV Agent", - "ddv_agent", - 0, - )?; - git::commit_all("ddv-agent: verification scenarios written", config)?; - - // DDV review gate — always shown (even when pause = false) - state::save_state(&lisa_root, &SpiralState::DdvAgentReview)?; - loop { - match review::ddv_review_gate(config, &lisa_root)? { - review::DdvDecision::Approve => { - terminal::log_success("DDV scenarios approved. Proceeding to Pass 1."); - break; - } - review::DdvDecision::Refine => { - // Create feedback template if it doesn't exist - let feedback_path = lisa_root.join("ddv/ddv-feedback.md"); - if !feedback_path.exists() { - std::fs::write( - &feedback_path, - "# DDV Feedback\n\n## Coverage Gaps\n-\n\n## Scenario Issues\n-\n\n## Missing Sources\n-\n\n## Other\n-\n", - )?; - } - - review::wait_for_edit( - "Write your DDV feedback in the file below. Describe coverage gaps, scenario issues, or missing sources.", - &feedback_path, - ); - - // Remove completion marker so agent re-runs - let complete_marker = lisa_root.join("ddv/DDV_COMPLETE.md"); - if complete_marker.exists() { - std::fs::remove_file(&complete_marker)?; - } - - terminal::log_info("Re-running DDV Agent with feedback..."); - state::save_state(&lisa_root, &SpiralState::DdvAgent)?; - - let refine_ctx = "DDV REFINEMENT: The human has reviewed your scenarios and provided feedback.\n\ - Read ddv/ddv-feedback.md carefully and update affected scenarios.\n\ - Do not discard previous work — refine it based on the feedback."; - - let input = prompt::build_agent_input( - Phase::DdvAgent, - config, - &lisa_root, - 0, - Some(refine_ctx), - ); - - run_agent_with_tracking( - config, - &lisa_root, - &input, - &model, - "DDV Agent: refinement", - "ddv_agent", - 0, - )?; - git::commit_all("ddv-agent: scenarios refined after human feedback", config)?; - terminal::log_info("DDV scenarios refined. Reviewing again..."); - state::save_state(&lisa_root, &SpiralState::DdvAgentReview)?; - } - review::DdvDecision::Edit => { - let scenarios_path = lisa_root.join("ddv/scenarios.md"); - terminal::log_info("Edit the DDV scenario files directly with any editor."); - println!(); - terminal::print_colored(" Scenarios: ", Color::Cyan); - println!("{}", scenarios_path.display()); - println!(); - print!(" Press Enter when you are done editing..."); - let _ = std::io::Write::flush(&mut std::io::stdout()); - let mut _buf = String::new(); - let _ = std::io::stdin().read_line(&mut _buf); - - // Re-display summary after edit - display_ddv_edit_summary(&lisa_root); - - terminal::log_success( - "DDV scenarios approved (manually edited). Proceeding to Pass 1.", - ); - break; - } - review::DdvDecision::Quit => { - terminal::log_warn("Stopping after DDV Agent."); - return Ok(()); - } + if !lisa_root.join("spiral/pass-0/PASS_COMPLETE.md").exists() { + terminal::log_warn("Scope not completed. Run `lisa run` to try again."); + anyhow::bail!("Scope phase did not complete (user quit or agent failed)."); } - } - - state::save_state(&lisa_root, &SpiralState::DdvAgentComplete)?; - terminal::log_success("DDV Agent complete."); - Ok(()) -} - -fn ensure_ddv_complete(config: &Config, project_root: &Path) -> Result<()> { - let lisa_root = config.lisa_root(project_root); - if !lisa_root.join("ddv/DDV_COMPLETE.md").exists() { - terminal::log_info("DDV scenarios not complete. Running DDV Agent first."); - run_ddv_agent(config, project_root)?; } else { - terminal::log_info("DDV scenarios already complete."); + terminal::log_info("Pass 0 already complete."); } Ok(()) } -/// Public entry point for `lisa ddv` command -pub fn run_ddv_agent_only(config: &Config, project_root: &Path) -> Result<()> { - ensure_scope_complete(config, project_root)?; - run_ddv_agent(config, project_root) -} - fn run_scope(config: &Config, project_root: &Path) -> Result<()> { let lisa_root = config.lisa_root(project_root); @@ -563,6 +597,9 @@ fn run_scope(config: &Config, project_root: &Path) -> Result<()> { state::save_state(&lisa_root, &SpiralState::Scoping)?; std::fs::create_dir_all(lisa_root.join("spiral/pass-0"))?; + // Ensure scope artifact spec files are on disk for the agent to read + prompt::ensure_scope_specs(&lisa_root, config)?; + // Check for existing feedback (resume case) let feedback_path = lisa_root.join("spiral/pass-0/scope-feedback.md"); let extra_context = if feedback_path.exists() { @@ -820,23 +857,6 @@ fn display_scope_edit_summary(lisa_root: &Path) { } } -/// Display a brief summary of DDV artifacts after manual editing. -fn display_ddv_edit_summary(lisa_root: &Path) { - let scenarios_path = lisa_root.join("ddv/scenarios.md"); - if scenarios_path.exists() { - if let Ok(content) = std::fs::read_to_string(&scenarios_path) { - let count = content.lines().filter(|l| l.starts_with("## DDV-")).count(); - terminal::print_colored(" Scenarios: ", Color::Cyan); - println!("{}", count); - let manifest_count = content.lines().filter(|l| l.starts_with("| DDV-")).count(); - if manifest_count > 0 { - terminal::print_colored(" Manifest: ", Color::Cyan); - println!("{} entries", manifest_count); - } - } - } -} - /// Display a brief summary of key artifacts after manual editing. fn display_refine_edit_summary(lisa_root: &Path) { let plan_path = lisa_root.join("methodology/plan.md"); @@ -869,6 +889,7 @@ fn run_refine(config: &Config, project_root: &Path, pass: u32) -> Result<()> { )?; std::fs::create_dir_all(lisa_root.join(format!("spiral/pass-{}", pass)))?; + std::fs::create_dir_all(lisa_root.join(format!("spiral/pass-{}/plots", pass)))?; let prev_pass = pass - 1; let mut extra = format!("Current spiral pass: {}\n", pass); @@ -896,6 +917,20 @@ fn run_refine(config: &Config, project_root: &Path, pass: u32) -> Result<()> { pass, )?; git::commit_all(&format!("refine: pass {}", pass), config)?; + + // Advisory warning if task count exceeds max_tasks_per_pass + let plan_path = lisa_root.join("methodology/plan.md"); + if plan_path.exists() { + if let Ok(counts) = tasks::count_tasks_by_status_for_pass(&plan_path, pass) { + if counts.total > config.limits.max_tasks_per_pass { + terminal::log_warn(&format!( + "Pass {} has {} tasks (limit: {}). Consider refining further to shrink scope.", + pass, counts.total, config.limits.max_tasks_per_pass + )); + } + } + } + state::save_state(&lisa_root, &SpiralState::RefineComplete { pass })?; Ok(()) } @@ -1027,37 +1062,174 @@ fn run_build_loop( Ok(true) } -fn run_validate(config: &Config, project_root: &Path, pass: u32) -> Result<()> { +fn run_audit(config: &Config, project_root: &Path, pass: u32) -> Result<()> { let lisa_root = config.lisa_root(project_root); - terminal::log_phase(&format!("PASS {} — VALIDATE", pass)); + terminal::log_phase(&format!("PASS {} — AUDIT", pass)); state::save_state( &lisa_root, &SpiralState::InPass { pass, - phase: PassPhase::Validate, + phase: PassPhase::Audit, }, )?; std::fs::create_dir_all(lisa_root.join(format!("spiral/pass-{}", pass)))?; let extra = format!("Current spiral pass: {}", pass); - let input = prompt::build_agent_input(Phase::Validate, config, &lisa_root, pass, Some(&extra)); - let model = Phase::Validate.model_key(config); + let input = prompt::build_agent_input(Phase::Audit, config, &lisa_root, pass, Some(&extra)); + let model = Phase::Audit.model_key(config); + run_agent_with_tracking( + config, + &lisa_root, + &input, + &model, + &format!("Audit: pass {}", pass), + "audit", + pass, + )?; + git::commit_all(&format!("audit: pass {}", pass), config)?; + state::save_state(&lisa_root, &SpiralState::AuditComplete { pass })?; + Ok(()) +} + +/// Run the pass review gate in a loop, handling exploration side-branches. +/// Returns the final non-explore decision. +fn pass_review_loop( + config: &Config, + project_root: &Path, + pass: u32, + lisa_root: &Path, +) -> Result { + loop { + match review::review_gate(config, pass, lisa_root)? { + ReviewDecision::Explore => { + let explore_id = next_explore_id(lisa_root, pass); + run_explore(config, project_root, pass, explore_id)?; + state::save_state(lisa_root, &SpiralState::PassReview { pass })?; + continue; + } + other => return Ok(other), + } + } +} + +/// Determine the next exploration ID for a given pass. +fn next_explore_id(lisa_root: &Path, pass: u32) -> u32 { + let pass_dir = lisa_root.join(format!("spiral/pass-{}", pass)); + let mut max_id = 0u32; + if let Ok(entries) = std::fs::read_dir(&pass_dir) { + for entry in entries.filter_map(|e| e.ok()) { + let name = entry.file_name(); + let name_str = name.to_string_lossy(); + if let Some(id_str) = name_str.strip_prefix("explore-") { + if let Ok(id) = id_str.parse::() { + max_id = max_id.max(id); + } + } + } + } + max_id + 1 +} + +/// Run a lightweight exploration on a side-branch. +fn run_explore(config: &Config, project_root: &Path, pass: u32, explore_id: u32) -> Result<()> { + let lisa_root = config.lisa_root(project_root); + let branch_name = format!("lisa/explore-{}-{}", pass, explore_id); + + terminal::log_phase(&format!( + "EXPLORATION — Pass {} (explore #{})", + pass, explore_id + )); + + // Save the current branch so we can return + let original_branch = git::current_branch()?; + + // Create and checkout the exploration branch + git::create_branch(&branch_name)?; + git::checkout(&branch_name)?; + + // Create exploration directory + let explore_dir = lisa_root.join(format!("spiral/pass-{}/explore-{}", pass, explore_id)); + std::fs::create_dir_all(explore_dir.join("plots"))?; + + // Save state + state::save_state(&lisa_root, &SpiralState::Exploring { pass, explore_id })?; + + // Check for saved question (resume case) or prompt interactively + let question_path = explore_dir.join("question.md"); + let question: String = if question_path.exists() { + let saved = std::fs::read_to_string(&question_path)?.trim().to_string(); + if !saved.is_empty() { + terminal::log_info(&format!("Resumed exploration question: {}", saved)); + saved + } else if std::io::stdin().is_terminal() { + dialoguer::Input::new() + .with_prompt(" Exploration question") + .interact_text()? + } else { + anyhow::bail!("Exploration requires an interactive terminal for the question prompt"); + } + } else if std::io::stdin().is_terminal() { + let q: String = dialoguer::Input::new() + .with_prompt(" Exploration question") + .interact_text()?; + std::fs::write(&question_path, &q)?; + q + } else { + anyhow::bail!("Exploration requires an interactive terminal for the question prompt"); + }; + + // Build extra context with the exploration question and explore_id + let extra = format!( + "Exploration question: {}\n\ + Exploration ID: {}\n\ + Exploration directory: {}/spiral/pass-{}/explore-{}/\n", + question, explore_id, config.paths.lisa_root, pass, explore_id + ); + + let input = prompt::build_agent_input(Phase::Explore, config, &lisa_root, pass, Some(&extra)); + let model = Phase::Explore.model_key(config); run_agent_with_tracking( config, &lisa_root, &input, &model, - &format!("Validate: pass {}", pass), - "validate", + &format!("Explore: pass {} #{}", pass, explore_id), + "explore", pass, )?; - git::commit_all(&format!("validate: pass {}", pass), config)?; - state::save_state(&lisa_root, &SpiralState::ValidateComplete { pass })?; + + // Commit exploration results + git::commit_all( + &format!("explore: pass {} #{} — {}", pass, explore_id, question), + config, + )?; + + // Save state for review + state::save_state(&lisa_root, &SpiralState::ExploreReview { pass, explore_id })?; + + // Show the review gate + match review::explore_review_gate(pass, explore_id, &lisa_root)? { + review::ExploreDecision::Merge => { + // Checkout original branch and merge + git::checkout(&original_branch)?; + git::merge_branch(&branch_name)?; + terminal::log_success(&format!("Exploration #{} merged.", explore_id)); + } + review::ExploreDecision::Discard => { + // Checkout original branch and delete the exploration branch + git::checkout(&original_branch)?; + if let Err(e) = git::delete_branch(&branch_name) { + terminal::log_warn(&format!("Could not delete explore branch: {}", e)); + } + terminal::log_info(&format!("Exploration #{} discarded.", explore_id)); + } + } + Ok(()) } -pub fn finalize(config: &Config, project_root: &Path, pass: u32) -> Result<()> { +fn finalize(config: &Config, project_root: &Path, pass: u32) -> Result<()> { let lisa_root = config.lisa_root(project_root); terminal::log_phase("FINALIZING — Producing deliverables"); @@ -1111,7 +1283,7 @@ pub fn finalize(config: &Config, project_root: &Path, pass: u32) -> Result<()> { } } - // Create SPIRAL_COMPLETE.md + // Create SPIRAL_COMPLETE.md (internal marker, stays in .lisa/) let complete_content = format!( "# Spiral Complete\n\n\ The human has finalized the results.\n\n\ @@ -1125,6 +1297,17 @@ pub fn finalize(config: &Config, project_root: &Path, pass: u32) -> Result<()> { &complete_content, )?; + // Generate audit trail artifact in project root + let report_path = project_root.join("LISA-REPORT.md"); + generate_audit_report(&lisa_root, config, pass, &report_path)?; + terminal::log_success(&format!( + "Audit report generated: {}", + report_path.display() + )); + terminal::log_info( + "The report is NOT auto-committed. Review it and commit manually if you want to keep it.", + ); + state::save_state(&lisa_root, &SpiralState::Complete { final_pass: pass })?; git::commit_all( &format!("final: spiral complete — finalized at pass {}", pass), @@ -1134,17 +1317,120 @@ pub fn finalize(config: &Config, project_root: &Path, pass: u32) -> Result<()> { terminal::log_success("Done. Final deliverables produced."); - // Show audit summary if it exists - let audit_path = lisa_root.join("output/audit-summary.md"); - if audit_path.exists() { - println!(); - terminal::log_info(&format!("Audit summary: {}", audit_path.display())); + Ok(()) +} + +/// Generate a standalone markdown audit report summarizing the full spiral history. +fn generate_audit_report( + lisa_root: &Path, + config: &Config, + final_pass: u32, + output_path: &Path, +) -> Result<()> { + use std::fmt::Write; + let mut report = String::new(); + + writeln!(report, "# Lisa Loop — Audit Report").unwrap(); + writeln!(report).unwrap(); + writeln!(report, "**Project:** {} ", config.project.name).unwrap(); + writeln!( + report, + "**Completed:** {} ", + chrono::Local::now().to_rfc3339() + ) + .unwrap(); + writeln!(report, "**Final pass:** {} ", final_pass).unwrap(); + writeln!(report).unwrap(); + + // Assignment + let assignment_path = lisa_root + .parent() + .unwrap_or(lisa_root) + .join("ASSIGNMENT.md"); + if let Ok(content) = std::fs::read_to_string(&assignment_path) { + writeln!(report, "---\n").unwrap(); + writeln!(report, "## Assignment\n").unwrap(); + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + + // Methodology + let methodology_path = lisa_root.join("methodology/methodology.md"); + if let Ok(content) = std::fs::read_to_string(&methodology_path) { + writeln!(report, "---\n").unwrap(); + writeln!(report, "## Final Methodology\n").unwrap(); + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); } + // Per-pass summaries + writeln!(report, "---\n").unwrap(); + writeln!(report, "## Spiral Pass History\n").unwrap(); + + for pass in 0..=final_pass { + let pass_dir = lisa_root.join(format!("spiral/pass-{}", pass)); + if !pass_dir.exists() { + continue; + } + + writeln!(report, "### Pass {}\n", pass).unwrap(); + + // Review package (most useful summary) + let review_pkg = pass_dir.join("review-package.md"); + if let Ok(content) = std::fs::read_to_string(&review_pkg) { + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + + // Progress tracking + let progress = pass_dir.join("progress-tracking.md"); + if let Ok(content) = std::fs::read_to_string(&progress) { + writeln!(report, "#### Progress Tracking\n").unwrap(); + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + + // Validation results + let validation = pass_dir.join("system-validation.md"); + if let Ok(content) = std::fs::read_to_string(&validation) { + writeln!(report, "#### Validation Results\n").unwrap(); + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + } + + // Assumptions register + let assumptions_path = lisa_root.join("methodology/assumptions-register.md"); + if let Ok(content) = std::fs::read_to_string(&assumptions_path) { + writeln!(report, "---\n").unwrap(); + writeln!(report, "## Assumptions Register\n").unwrap(); + writeln!(report, "{}", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + + // Usage/cost if available + let usage_path = lisa_root.join("usage.toml"); + if let Ok(content) = std::fs::read_to_string(&usage_path) { + writeln!(report, "---\n").unwrap(); + writeln!(report, "## Cost Ledger\n").unwrap(); + writeln!(report, "```toml\n{}\n```", content.trim()).unwrap(); + writeln!(report).unwrap(); + } + + writeln!( + report, + "\n---\n*Generated by [Lisa Loop](https://github.com/edmondop/lisa-loop)*" + ) + .unwrap(); + + std::fs::write(output_path, &report)?; Ok(()) } /// Roll back to a previous pass boundary. +/// +/// Code is reset via git tags. Process state (.lisa/) is rolled back on the +/// filesystem since it is gitignored and never committed. pub fn rollback(config: &Config, project_root: &Path, target_pass: u32, force: bool) -> Result<()> { let lisa_root = config.lisa_root(project_root); let tag = format!("lisa/pass-{}", target_pass); @@ -1176,7 +1462,7 @@ pub fn rollback(config: &Config, project_root: &Path, target_pass: u32, force: b // Confirmation prompt if !force { terminal::log_warn(&format!( - "This will reset the repository to the state at pass {}.", + "This will reset code to the state at pass {} and remove later spiral artifacts.", target_pass )); terminal::log_warn("A backup branch will be created at current HEAD."); @@ -1190,23 +1476,53 @@ pub fn rollback(config: &Config, project_root: &Path, target_pass: u32, force: b } } - // Create backup branch + // Create backup branch (for code rollback safety) let timestamp = chrono::Local::now().format("%Y%m%d-%H%M%S"); let backup_branch = format!("lisa/backup/rollback-{}", timestamp); git::create_branch(&backup_branch)?; terminal::log_info(&format!("Backup branch created: {}", backup_branch)); - // Reset to tag + // Reset code to tag git::reset_hard(&tag)?; - terminal::log_success(&format!("Reset to {}", tag)); - - // Restore usage.toml from backup branch (cost history should never be lost) - let usage_rel = format!("{}/usage.toml", config.paths.lisa_root); - if let Ok(Some(content)) = git::show_file_from_ref(&backup_branch, &usage_rel) { - let usage_path = lisa_root.join("usage.toml"); - std::fs::write(&usage_path, &content)?; - git::commit_all("rollback: restore usage ledger", config)?; - terminal::log_info("Usage ledger preserved from before rollback."); + terminal::log_success(&format!("Code reset to {}", tag)); + + // Roll back .lisa/ process state on the filesystem + // Remove spiral pass directories after the target pass + for pass in (target_pass + 1)..=100 { + let pass_dir = lisa_root.join(format!("spiral/pass-{}", pass)); + if pass_dir.exists() { + std::fs::remove_dir_all(&pass_dir)?; + terminal::log_info(&format!("Removed spiral/pass-{}/", pass)); + } else { + break; + } + } + + // Remove completion markers + let complete_marker = lisa_root.join("spiral/SPIRAL_COMPLETE.md"); + if complete_marker.exists() { + std::fs::remove_file(&complete_marker)?; + } + + // Reset state to the end of the target pass + let new_state = if target_pass == 0 { + SpiralState::ScopeComplete + } else { + SpiralState::PassReview { pass: target_pass } + }; + state::save_state(&lisa_root, &new_state)?; + terminal::log_info(&format!("State reset to: {}", new_state)); + + // Delete git tags for passes after the target + for pass in (target_pass + 1)..=100 { + let tag_name = format!("lisa/pass-{}", pass); + if available.contains(&pass) { + let _ = std::process::Command::new("git") + .args(["tag", "-d", &tag_name]) + .output(); + } else { + break; + } } terminal::log_success(&format!( @@ -1217,7 +1533,7 @@ pub fn rollback(config: &Config, project_root: &Path, target_pass: u32, force: b } /// Continue with a follow-up question after a completed spiral. -pub fn continue_spiral( +fn continue_spiral( config: &Config, project_root: &Path, question: &str, @@ -1263,8 +1579,8 @@ pub fn continue_spiral( std::fs::remove_file(&complete_marker)?; } - // Reset state to DdvAgentComplete (scope + DDV scenarios are still valid) - state::save_state(&lisa_root, &SpiralState::DdvAgentComplete)?; + // Reset state to ScopeComplete (scope is still valid) + state::save_state(&lisa_root, &SpiralState::ScopeComplete)?; git::commit_all( &format!( @@ -1288,7 +1604,7 @@ pub fn continue_spiral( final_pass )); - run(&config, project_root, Some(effective_max), no_pause) + run(&config, project_root, Some(effective_max), no_pause, None) } /// Count the number of `## Follow-up` sections in ASSIGNMENT.md content. diff --git a/src/prompt.rs b/src/prompt.rs index 610ec6d..8adbe48 100644 --- a/src/prompt.rs +++ b/src/prompt.rs @@ -1,32 +1,69 @@ use crate::config::Config; +use anyhow::Result; use std::path::Path; // Compiled-in prompts +pub const PROMPT_INIT: &str = include_str!("../prompts/PROMPT_init.md"); pub const PROMPT_SCOPE: &str = include_str!("../prompts/PROMPT_scope.md"); pub const PROMPT_REFINE: &str = include_str!("../prompts/PROMPT_refine.md"); -pub const PROMPT_DDV_AGENT: &str = include_str!("../prompts/PROMPT_ddv_agent.md"); pub const PROMPT_BUILD: &str = include_str!("../prompts/PROMPT_build.md"); -pub const PROMPT_VALIDATE: &str = include_str!("../prompts/PROMPT_validate.md"); +pub const PROMPT_AUDIT: &str = include_str!("../prompts/PROMPT_audit.md"); pub const PROMPT_FINALIZE: &str = include_str!("../prompts/PROMPT_finalize.md"); +pub const PROMPT_EXPLORE: &str = include_str!("../prompts/PROMPT_explore.md"); + +// Compiled-in skill files +pub const SKILL_ENGINEERING_JUDGMENT: &str = include_str!("../skills/engineering_judgment.md"); +pub const SKILL_DIMENSIONAL_ANALYSIS: &str = include_str!("../skills/dimensional_analysis.md"); +pub const SKILL_NUMERICAL_STABILITY: &str = include_str!("../skills/numerical_stability.md"); +pub const SKILL_LITERATURE_GROUNDING: &str = include_str!("../skills/literature_grounding.md"); + +pub const SKILLS: &[(&str, &str)] = &[ + ("engineering-judgment.md", SKILL_ENGINEERING_JUDGMENT), + ("dimensional-analysis.md", SKILL_DIMENSIONAL_ANALYSIS), + ("numerical-stability.md", SKILL_NUMERICAL_STABILITY), + ("literature-grounding.md", SKILL_LITERATURE_GROUNDING), +]; + +// Compiled-in scope artifact specs (read on demand by the scoping agent) +const SCOPE_SPEC_METHODOLOGY: &str = include_str!("../prompts/scope/methodology_spec.md"); +const SCOPE_SPEC_LITERATURE_SURVEY: &str = + include_str!("../prompts/scope/literature_survey_spec.md"); +const SCOPE_SPEC_SPIRAL_PLAN: &str = include_str!("../prompts/scope/spiral_plan_spec.md"); +const SCOPE_SPEC_STACK_SELECTION: &str = include_str!("../prompts/scope/stack_selection_spec.md"); +const SCOPE_SPEC_VALIDATION: &str = include_str!("../prompts/scope/validation_specs.md"); +const SCOPE_SPEC_IMPLEMENTATION_PLAN: &str = + include_str!("../prompts/scope/implementation_plan_spec.md"); +pub const SCOPE_SPECS: &[(&str, &str)] = &[ + ("methodology_spec.md", SCOPE_SPEC_METHODOLOGY), + ("literature_survey_spec.md", SCOPE_SPEC_LITERATURE_SURVEY), + ("spiral_plan_spec.md", SCOPE_SPEC_SPIRAL_PLAN), + ("stack_selection_spec.md", SCOPE_SPEC_STACK_SELECTION), + ("validation_specs.md", SCOPE_SPEC_VALIDATION), + ( + "implementation_plan_spec.md", + SCOPE_SPEC_IMPLEMENTATION_PLAN, + ), +]; #[derive(Debug, Clone, Copy)] pub enum Phase { + Init, Scope, Refine, - DdvAgent, Build, - Validate, + Audit, Finalize, + Explore, } impl Phase { pub fn model_key(&self, config: &Config) -> String { match self { - Phase::Scope => config.models.scope.clone(), + Phase::Init | Phase::Scope => config.models.scope.clone(), Phase::Refine => config.models.refine.clone(), - Phase::DdvAgent => config.models.ddv.clone(), Phase::Build => config.models.build.clone(), - Phase::Validate | Phase::Finalize => config.models.validate.clone(), + Phase::Audit | Phase::Finalize => config.models.audit.clone(), + Phase::Explore => config.models.scope.clone(), } } } @@ -34,12 +71,13 @@ impl Phase { /// Load prompt for a phase. Prefers local .lisa/prompts/ if ejected, otherwise uses compiled-in. pub fn load_prompt(phase: Phase, lisa_root: &Path) -> String { let local_path = match phase { + Phase::Init => lisa_root.join("prompts/init.md"), Phase::Scope => lisa_root.join("prompts/scope.md"), Phase::Refine => lisa_root.join("prompts/refine.md"), - Phase::DdvAgent => lisa_root.join("prompts/ddv_agent.md"), Phase::Build => lisa_root.join("prompts/build.md"), - Phase::Validate => lisa_root.join("prompts/validate.md"), + Phase::Audit => lisa_root.join("prompts/audit.md"), Phase::Finalize => lisa_root.join("prompts/finalize.md"), + Phase::Explore => lisa_root.join("prompts/explore.md"), }; if local_path.exists() { @@ -49,29 +87,37 @@ pub fn load_prompt(phase: Phase, lisa_root: &Path) -> String { } match phase { + Phase::Init => PROMPT_INIT.to_string(), Phase::Scope => PROMPT_SCOPE.to_string(), Phase::Refine => PROMPT_REFINE.to_string(), - Phase::DdvAgent => PROMPT_DDV_AGENT.to_string(), Phase::Build => PROMPT_BUILD.to_string(), - Phase::Validate => PROMPT_VALIDATE.to_string(), + Phase::Audit => PROMPT_AUDIT.to_string(), Phase::Finalize => PROMPT_FINALIZE.to_string(), + Phase::Explore => PROMPT_EXPLORE.to_string(), } } -/// Render the prompt with path substitutions -pub fn render_prompt(prompt: &str, config: &Config) -> String { +/// Render the prompt with path substitutions. +/// `pass` is the current spiral pass number (used for `{{pass}}` placeholder). +pub fn render_prompt(prompt: &str, config: &Config, pass: Option) -> String { let lisa_root = &config.paths.lisa_root; let source_dirs = config.source_dirs_display(); - let tests_ddv = &config.paths.tests_ddv; + let tests_bounds = &config.paths.tests_bounds; let tests_software = &config.paths.tests_software; let tests_integration = &config.paths.tests_integration; + let pass_str = pass.unwrap_or(0).to_string(); prompt .replace("{{lisa_root}}", lisa_root) .replace("{{source_dirs}}", &source_dirs) - .replace("{{tests_ddv}}", tests_ddv) + .replace("{{tests_bounds}}", tests_bounds) .replace("{{tests_software}}", tests_software) .replace("{{tests_integration}}", tests_integration) + .replace( + "{{max_tasks_per_pass}}", + &config.limits.max_tasks_per_pass.to_string(), + ) + .replace("{{pass}}", &pass_str) } /// Build the context preamble that gets prepended to every agent invocation @@ -98,9 +144,9 @@ pub fn build_context_preamble( - Spiral: {}/spiral/ - Validation: {}/validation/ - References: {}/references/ -- Plots: {}/plots/ +- Plots: {}/spiral/pass-{}/plots/ - Source code: {} (deliverable) -- DDV tests: {} +- Bounds tests: {} - Software tests: {} - Integration tests: {} @@ -116,8 +162,9 @@ pub fn build_context_preamble( lisa_root, lisa_root, lisa_root, + current_pass, source_dirs, - config.paths.tests_ddv, + config.paths.tests_bounds, config.paths.tests_software, config.paths.tests_integration, current_pass, @@ -140,6 +187,16 @@ pub fn build_context_preamble( )); } + // Idle timeout heartbeat guidance + ctx.push_str(&format!( + "\n### Heartbeat\n\ + Lisa Loop will kill your process if no tool output is received for {} seconds.\n\ + During long-running operations (compilation, test suites, complex analysis),\n\ + emit periodic heartbeats by running: `echo \"[heartbeat] still working...\"`\n\ + Do this every few minutes during operations that may take a while.\n", + config.limits.idle_timeout_secs + )); + ctx } @@ -152,12 +209,13 @@ pub fn build_agent_input( extra_context: Option<&str>, ) -> String { let phase_name = match phase { + Phase::Init => "Init", Phase::Scope => "Scope", Phase::Refine => "Refine", - Phase::DdvAgent => "DDV Agent", Phase::Build => "Build", - Phase::Validate => "Validate", + Phase::Audit => "Audit", Phase::Finalize => "Finalize", + Phase::Explore => "Explore", }; let has_redirect = if current_pass > 0 { @@ -171,7 +229,7 @@ pub fn build_agent_input( let preamble = build_context_preamble(config, current_pass, phase_name, has_redirect); let prompt = load_prompt(phase, lisa_root); - let rendered = render_prompt(&prompt, config); + let rendered = render_prompt(&prompt, config, Some(current_pass)); let mut input = preamble; if let Some(extra) = extra_context { @@ -183,6 +241,22 @@ pub fn build_agent_input( input } +/// Write scope artifact spec files to .lisa/prompts/scope/ if they don't already exist. +/// Renders {{placeholder}} substitutions so the agent sees concrete paths and values. +/// Preserves user-customized files (same pattern as eject-prompts). +pub fn ensure_scope_specs(lisa_root: &Path, config: &Config) -> Result<()> { + let scope_dir = lisa_root.join("prompts/scope"); + std::fs::create_dir_all(&scope_dir)?; + for (filename, content) in SCOPE_SPECS { + let path = scope_dir.join(filename); + if !path.exists() { + let rendered = render_prompt(content, config, Some(0)); + std::fs::write(&path, rendered)?; + } + } + Ok(()) +} + #[cfg(test)] mod tests { use super::*; @@ -196,17 +270,43 @@ mod tests { #[test] fn test_render_prompt_substitutions() { let config = test_config(); - let prompt = "Read ASSIGNMENT.md and {{tests_ddv}}/ tests."; - let rendered = render_prompt(prompt, &config); - assert_eq!(rendered, "Read ASSIGNMENT.md and tests/ddv/ tests."); + let prompt = "Read ASSIGNMENT.md and {{tests_bounds}}/ tests."; + let rendered = render_prompt(prompt, &config, None); + assert_eq!(rendered, "Read ASSIGNMENT.md and / tests."); } #[test] fn test_render_prompt_source_dirs() { let config = test_config(); let prompt = "Source at {{source_dirs}}."; - let rendered = render_prompt(prompt, &config); - assert_eq!(rendered, "Source at src."); + let rendered = render_prompt(prompt, &config, None); + assert_eq!(rendered, "Source at ."); + } + + #[test] + fn test_render_prompt_substitutions_with_filled_paths() { + let toml_str = r#" +[project] +name = "test" + +[paths] +source = ["src"] +tests_bounds = "tests/bounds" +tests_software = "tests/software" +tests_integration = "tests/integration" +"#; + let config: Config = toml::from_str(toml_str).unwrap(); + let prompt = "Read {{tests_bounds}}/ and {{source_dirs}}."; + let rendered = render_prompt(prompt, &config, Some(1)); + assert_eq!(rendered, "Read tests/bounds/ and src."); + } + + #[test] + fn test_render_prompt_pass_placeholder() { + let config = test_config(); + let prompt = "Plots at .lisa/spiral/pass-{{pass}}/plots/"; + let rendered = render_prompt(prompt, &config, Some(3)); + assert_eq!(rendered, "Plots at .lisa/spiral/pass-3/plots/"); } #[test] @@ -225,6 +325,7 @@ mod tests { let preamble = build_context_preamble(&config, 2, "Build", false); assert!(preamble.contains("Spiral pass: 2")); assert!(preamble.contains("Previous pass results: .lisa/spiral/pass-1/")); + assert!(preamble.contains("Plots: .lisa/spiral/pass-2/plots/")); } #[test] @@ -236,11 +337,83 @@ mod tests { #[test] fn test_compiled_prompts_not_empty() { + assert!(!PROMPT_INIT.is_empty()); assert!(!PROMPT_SCOPE.is_empty()); assert!(!PROMPT_REFINE.is_empty()); - assert!(!PROMPT_DDV_AGENT.is_empty()); assert!(!PROMPT_BUILD.is_empty()); - assert!(!PROMPT_VALIDATE.is_empty()); + assert!(!PROMPT_AUDIT.is_empty()); assert!(!PROMPT_FINALIZE.is_empty()); + assert!(!PROMPT_EXPLORE.is_empty()); + } + + #[test] + fn test_scope_specs_not_empty() { + assert_eq!(SCOPE_SPECS.len(), 6); + for (filename, content) in SCOPE_SPECS { + assert!(!filename.is_empty(), "spec filename is empty"); + assert!( + !content.is_empty(), + "spec content is empty for {}", + filename + ); + } + } + + #[test] + fn test_scope_prompt_references_all_specs() { + for (filename, _) in SCOPE_SPECS { + let reference = format!("prompts/scope/{}", filename); + assert!( + PROMPT_SCOPE.contains(&reference), + "PROMPT_scope.md does not reference {}", + filename, + ); + } + } + + #[test] + fn test_ensure_scope_specs_creates_files() { + let tmp = tempfile::tempdir().unwrap(); + let lisa_root = tmp.path(); + let config = test_config(); + ensure_scope_specs(lisa_root, &config).unwrap(); + + for (filename, _) in SCOPE_SPECS { + let path = lisa_root.join("prompts/scope").join(filename); + assert!(path.exists(), "{} was not created", filename); + let written = std::fs::read_to_string(&path).unwrap(); + // Specs should be rendered — no raw {{lisa_root}} placeholders remaining + assert!( + !written.contains("{{lisa_root}}"), + "{} still contains unrendered {{{{lisa_root}}}} placeholder", + filename, + ); + } + } + + #[test] + fn test_ensure_scope_specs_preserves_existing() { + let tmp = tempfile::tempdir().unwrap(); + let lisa_root = tmp.path(); + let config = test_config(); + let scope_dir = lisa_root.join("prompts/scope"); + std::fs::create_dir_all(&scope_dir).unwrap(); + + // Write a customized version of the first spec + let (filename, _) = SCOPE_SPECS[0]; + let custom = "# My custom spec\n"; + std::fs::write(scope_dir.join(filename), custom).unwrap(); + + ensure_scope_specs(lisa_root, &config).unwrap(); + + // Custom file should be preserved + let content = std::fs::read_to_string(scope_dir.join(filename)).unwrap(); + assert_eq!(content, custom, "existing file was overwritten"); + + // Other files should have been created + for (fname, _) in &SCOPE_SPECS[1..] { + let path = scope_dir.join(fname); + assert!(path.exists(), "{} was not created", fname); + } } } diff --git a/src/review.rs b/src/review.rs index 8abdda3..c9dd8ea 100644 --- a/src/review.rs +++ b/src/review.rs @@ -30,9 +30,16 @@ pub enum ReviewDecision { Finalize, Continue, Redirect, + Explore, Quit, } +#[derive(Debug, PartialEq)] +pub enum ExploreDecision { + Merge, + Discard, +} + #[derive(Debug, PartialEq)] pub enum ScopeDecision { Approve, @@ -48,14 +55,6 @@ pub enum BlockDecision { Abort, } -#[derive(Debug, PartialEq)] -pub enum DdvDecision { - Approve, - Refine, - Edit, - Quit, -} - #[derive(Debug, PartialEq)] pub enum RefineDecision { Approve, @@ -115,8 +114,7 @@ pub fn scope_review_gate(config: &Config, lisa_root: &Path) -> Result Result 0 { - terminal::print_colored(" DDV scenarios:", Color::Cyan); - println!(" {}", ddv_count); - } - } - } - // Acceptance criteria lines if acceptance_path.exists() { if let Ok(content) = std::fs::read_to_string(&acceptance_path) { @@ -219,9 +205,7 @@ pub fn scope_review_gate(config: &Config, lisa_root: &Path) -> Result Result Result { - println!(); - terminal::print_separator(); - terminal::println_bold(" DDV SCENARIOS COMPLETE — REVIEW REQUIRED"); - terminal::print_separator(); - println!(); - - // Show scenario summary - let scenarios_path = lisa_root.join("ddv/scenarios.md"); - if scenarios_path.exists() { - if let Ok(content) = std::fs::read_to_string(&scenarios_path) { - let scenario_count = content.lines().filter(|l| l.starts_with("## DDV-")).count(); - terminal::print_colored(" Scenarios: ", Color::Cyan); - println!("{}", scenario_count); - - // Show first few scenario titles - let titles: Vec<&str> = content - .lines() - .filter(|l| l.starts_with("## DDV-")) - .take(5) - .collect(); - for title in &titles { - println!(" {}", title.trim_start_matches("## ")); - } - if scenario_count > 5 { - println!(" ... and {} more", scenario_count - 5); - } - } - } - - // Show manifest summary (from ## Manifest section in scenarios.md) - if scenarios_path.exists() { - if let Ok(content) = std::fs::read_to_string(&scenarios_path) { - let entry_count = content.lines().filter(|l| l.starts_with("| DDV-")).count(); - if entry_count > 0 { - terminal::print_colored(" Manifest: ", Color::Cyan); - println!("{} entries", entry_count); - } - } - } - - println!(); - terminal::print_colored(" Files:\n", Color::Cyan); - println!(" Scenarios: {}/ddv/scenarios.md", lisa_root.display()); - - println!(); - terminal::print_colored(" [A]", Color::Green); - println!(" APPROVE — accept DDV scenarios and proceed to Pass 1"); - terminal::print_colored(" [R]", Color::Yellow); - println!(" REFINE — write feedback to a file, then the DDV agent re-runs"); - terminal::print_colored(" [E]", Color::Cyan); - println!(" EDIT — edit the scenario files yourself with any editor, then approve"); - terminal::print_colored(" [Q]", Color::Red); - println!(" QUIT — stop the spiral here (resume later with `lisa resume`)"); - println!(); - terminal::print_separator(); - println!(); - - loop { - print!(" Your choice [A/R/E/Q]: "); - io::stdout().flush()?; - let mut choice = String::new(); - io::stdin().read_line(&mut choice)?; - match choice.trim().to_lowercase().as_str() { - "a" => return Ok(DdvDecision::Approve), - "r" => return Ok(DdvDecision::Refine), - "e" => return Ok(DdvDecision::Edit), - "q" => return Ok(DdvDecision::Quit), - _ => println!(" Invalid choice. Enter A, R, E, or Q."), - } - } -} - /// Refine review gate — after each pass's refine phase pub fn refine_review_gate(config: &Config, pass: u32, lisa_root: &Path) -> Result { if !config.review.pause { @@ -460,7 +370,11 @@ pub fn review_gate(config: &Config, pass: u32, lisa_root: &Path) -> Result Result Result Result println!(" Please enter F, C, R, or Q."), + "E" => { + terminal::log_info("EXPLORE — creating a side-branch for investigation."); + return Ok(ReviewDecision::Explore); + } + _ => println!(" Please enter F, C, R, E, or Q."), + } + } +} + +/// Gate shown after an exploration completes. User decides to merge findings or discard. +pub fn explore_review_gate( + pass: u32, + explore_id: u32, + lisa_root: &Path, +) -> Result { + let findings_path = lisa_root.join(format!( + "spiral/pass-{}/explore-{}/findings.md", + pass, explore_id + )); + + println!(); + terminal::print_separator(); + terminal::println_bold(" EXPLORATION REVIEW"); + println!(); + + if findings_path.exists() { + if let Ok(content) = std::fs::read_to_string(&findings_path) { + let lines: Vec<&str> = content.lines().take(15).collect(); + for line in &lines { + println!(" {}", line); + } + if content.lines().count() > 15 { + println!(" ..."); + } + } + } else { + println!(" No findings file produced."); + } + + println!(); + println!(" Findings: {}", findings_path.display()); + println!(); + + terminal::print_colored(" [M]", Color::Green); + println!(" MERGE — merge exploration findings back into the main branch"); + terminal::print_colored(" [D]", Color::Red); + println!(" DISCARD — discard the exploration branch"); + println!(); + + loop { + print!(" Your choice [M/D]: "); + io::stdout().flush()?; + let mut choice = String::new(); + io::stdin().read_line(&mut choice)?; + match choice.trim().to_uppercase().as_str() { + "M" => { + terminal::log_success("MERGE — folding exploration into main branch."); + return Ok(ExploreDecision::Merge); + } + "D" => { + terminal::log_warn("DISCARD — abandoning exploration branch."); + return Ok(ExploreDecision::Discard); + } + _ => println!(" Please enter M or D."), } } } @@ -903,14 +882,6 @@ pub fn extract_methodology_approach_from(content: &str) -> Option { None } -/// Count `## DDV-` headings in ddv/scenarios.md. -pub fn count_ddv_scenarios(content: &str) -> u32 { - content - .lines() - .filter(|l| l.starts_with("## DDV-")) - .count() as u32 -} - fn extract_stack_info(agents_content: &str) -> Option { let mut found = false; for line in agents_content.lines() { @@ -944,7 +915,7 @@ fn display_review_summary(content: &str, _pass: u32) { // Extract test summary for line in content.lines() { - if line.starts_with("DDV:") { + if line.starts_with("Bounds:") { terminal::print_bold(" Tests: "); println!("{}", line); break; @@ -961,24 +932,6 @@ fn display_review_summary(content: &str, _pass: u32) { } } - // DDV Scenario Coverage - if let Some(coverage) = extract_section_first_line(content, "## DDV Scenario Coverage") { - terminal::print_bold(" DDV coverage: "); - println!("{}", coverage); - } - // Highlight re-run recommendation - for line in content.lines() { - if line.contains("Re-run DDV Agent recommended:") { - let text = line.trim(); - if text.to_uppercase().contains("YES") { - terminal::print_colored(&format!(" {}\n", text), Color::Yellow); - } else { - println!(" {}", text); - } - break; - } - } - // Engineering Judgment (HUMAN REVIEW) let judgment_lines = extract_section_lines(content, "## Engineering Judgment", 5); if !judgment_lines.is_empty() { @@ -1106,18 +1059,6 @@ mod tests { ); } - #[test] - fn test_count_ddv_scenarios() { - let content = "# DDV Scenarios\n\n## DDV-001: Basic check\nDetails...\n\n## DDV-002: Boundary\nDetails...\n\n## DDV-003: Convergence\nDetails...\n"; - assert_eq!(count_ddv_scenarios(content), 3); - } - - #[test] - fn test_count_ddv_scenarios_none() { - let content = "# DDV Scenarios\n\n## Manifest\n\nNo scenarios yet.\n"; - assert_eq!(count_ddv_scenarios(content), 0); - } - #[test] fn test_extract_section_first_line() { let content = "## Current Answer\n\nRe = 1.23e5 +/- 2.1%\n\n## Progress\n"; diff --git a/src/state.rs b/src/state.rs index 624e406..7eaf703 100644 --- a/src/state.rs +++ b/src/state.rs @@ -9,15 +9,14 @@ pub enum SpiralState { Scoping, ScopeReview, ScopeComplete, - DdvAgent, - DdvAgentReview, - DdvAgentComplete, InPass { pass: u32, phase: PassPhase }, RefineComplete { pass: u32 }, RefineReview { pass: u32 }, BuildComplete { pass: u32 }, - ValidateComplete { pass: u32 }, + AuditComplete { pass: u32 }, PassReview { pass: u32 }, + Exploring { pass: u32, explore_id: u32 }, + ExploreReview { pass: u32, explore_id: u32 }, Complete { final_pass: u32 }, } @@ -26,7 +25,7 @@ pub enum SpiralState { pub enum PassPhase { Refine, Build { iteration: u32 }, - Validate, + Audit, } impl std::fmt::Display for SpiralState { @@ -36,17 +35,20 @@ impl std::fmt::Display for SpiralState { SpiralState::Scoping => write!(f, "Scoping"), SpiralState::ScopeReview => write!(f, "Scope review"), SpiralState::ScopeComplete => write!(f, "Scope complete"), - SpiralState::DdvAgent => write!(f, "DDV Agent"), - SpiralState::DdvAgentReview => write!(f, "DDV Agent review"), - SpiralState::DdvAgentComplete => write!(f, "DDV Agent complete"), SpiralState::InPass { pass, phase } => write!(f, "Pass {} — {}", pass, phase), SpiralState::RefineComplete { pass } => write!(f, "Pass {} — Refine complete", pass), SpiralState::RefineReview { pass } => write!(f, "Pass {} — Refine review", pass), SpiralState::BuildComplete { pass } => write!(f, "Pass {} — Build complete", pass), - SpiralState::ValidateComplete { pass } => { - write!(f, "Pass {} — Validate complete", pass) + SpiralState::AuditComplete { pass } => { + write!(f, "Pass {} — Audit complete", pass) } SpiralState::PassReview { pass } => write!(f, "Pass {} — Review", pass), + SpiralState::Exploring { pass, explore_id } => { + write!(f, "Pass {} — Exploring (id {})", pass, explore_id) + } + SpiralState::ExploreReview { pass, explore_id } => { + write!(f, "Pass {} — Explore review (id {})", pass, explore_id) + } SpiralState::Complete { final_pass } => write!(f, "Complete (pass {})", final_pass), } } @@ -57,7 +59,7 @@ impl std::fmt::Display for PassPhase { match self { PassPhase::Refine => write!(f, "Refine"), PassPhase::Build { iteration } => write!(f, "Build (iteration {})", iteration), - PassPhase::Validate => write!(f, "Validate"), + PassPhase::Audit => write!(f, "Audit"), } } } @@ -155,8 +157,8 @@ mod tests { } #[test] - fn test_state_roundtrip_ddv_agent() { - let state = SpiralState::DdvAgent; + fn test_state_roundtrip_refine_review() { + let state = SpiralState::RefineReview { pass: 3 }; let file = StateFile { state: state.clone(), }; @@ -166,8 +168,8 @@ mod tests { } #[test] - fn test_state_roundtrip_ddv_agent_review() { - let state = SpiralState::DdvAgentReview; + fn test_state_roundtrip_refine_complete() { + let state = SpiralState::RefineComplete { pass: 2 }; let file = StateFile { state: state.clone(), }; @@ -177,8 +179,8 @@ mod tests { } #[test] - fn test_state_roundtrip_ddv_agent_complete() { - let state = SpiralState::DdvAgentComplete; + fn test_state_roundtrip_build_complete() { + let state = SpiralState::BuildComplete { pass: 3 }; let file = StateFile { state: state.clone(), }; @@ -188,8 +190,8 @@ mod tests { } #[test] - fn test_state_roundtrip_refine_review() { - let state = SpiralState::RefineReview { pass: 3 }; + fn test_state_roundtrip_audit_complete() { + let state = SpiralState::AuditComplete { pass: 1 }; let file = StateFile { state: state.clone(), }; @@ -199,19 +201,11 @@ mod tests { } #[test] - fn test_state_roundtrip_refine_complete() { - let state = SpiralState::RefineComplete { pass: 2 }; - let file = StateFile { - state: state.clone(), + fn test_state_roundtrip_exploring() { + let state = SpiralState::Exploring { + pass: 2, + explore_id: 1, }; - let toml_str = toml::to_string_pretty(&file).unwrap(); - let parsed: StateFile = toml::from_str(&toml_str).unwrap(); - assert_eq!(parsed.state, state); - } - - #[test] - fn test_state_roundtrip_build_complete() { - let state = SpiralState::BuildComplete { pass: 3 }; let file = StateFile { state: state.clone(), }; @@ -221,8 +215,11 @@ mod tests { } #[test] - fn test_state_roundtrip_validate_complete() { - let state = SpiralState::ValidateComplete { pass: 1 }; + fn test_state_roundtrip_explore_review() { + let state = SpiralState::ExploreReview { + pass: 3, + explore_id: 2, + }; let file = StateFile { state: state.clone(), }; @@ -247,8 +244,8 @@ mod tests { "Pass 3 — Build complete" ); assert_eq!( - format!("{}", SpiralState::ValidateComplete { pass: 1 }), - "Pass 1 — Validate complete" + format!("{}", SpiralState::AuditComplete { pass: 1 }), + "Pass 1 — Audit complete" ); assert_eq!( format!( diff --git a/src/tasks.rs b/src/tasks.rs index d784cbc..6a08b54 100644 --- a/src/tasks.rs +++ b/src/tasks.rs @@ -52,6 +52,25 @@ pub fn count_tasks_by_status(plan_path: &Path) -> Result { }) } +pub fn count_tasks_by_status_for_pass(plan_path: &Path, pass: u32) -> Result { + if !plan_path.exists() { + return Ok(TaskCounts::default()); + } + let content = std::fs::read_to_string(plan_path)?; + let tasks = parse_tasks(&content); + let filtered: Vec<&Task> = tasks.iter().filter(|t| t.pass == pass).collect(); + Ok(TaskCounts { + total: filtered.len() as u32, + todo: filtered.iter().filter(|t| t.status == "TODO").count() as u32, + in_progress: filtered + .iter() + .filter(|t| t.status == "IN_PROGRESS") + .count() as u32, + done: filtered.iter().filter(|t| t.status == "DONE").count() as u32, + blocked: filtered.iter().filter(|t| t.status == "BLOCKED").count() as u32, + }) +} + #[derive(Debug, Default)] pub struct TaskCounts { pub total: u32, @@ -254,4 +273,49 @@ mod tests { let hash2 = hash_task_statuses(path).unwrap(); assert_eq!(hash, hash2, "Hash of missing file should be deterministic"); } + + #[test] + fn test_count_tasks_by_status_for_pass() { + let dir = std::env::temp_dir().join("lisa_test_pass_filter"); + std::fs::create_dir_all(&dir).unwrap(); + let plan = dir.join("plan.md"); + + let content = r#"# Implementation Plan + +## Tasks + +### Task 1: Setup +- **Status:** DONE +- **Pass:** 1 + +### Task 2: Core +- **Status:** TODO +- **Pass:** 1 + +### Task 3: Advanced +- **Status:** TODO +- **Pass:** 2 + +### Task 4: Polish +- **Status:** BLOCKED +- **Pass:** 2 +"#; + std::fs::write(&plan, content).unwrap(); + + let pass1 = count_tasks_by_status_for_pass(&plan, 1).unwrap(); + assert_eq!(pass1.total, 2); + assert_eq!(pass1.done, 1); + assert_eq!(pass1.todo, 1); + assert_eq!(pass1.blocked, 0); + + let pass2 = count_tasks_by_status_for_pass(&plan, 2).unwrap(); + assert_eq!(pass2.total, 2); + assert_eq!(pass2.todo, 1); + assert_eq!(pass2.blocked, 1); + + let pass3 = count_tasks_by_status_for_pass(&plan, 3).unwrap(); + assert_eq!(pass3.total, 0); + + std::fs::remove_dir_all(&dir).ok(); + } } diff --git a/templates/ddv_scenarios.md b/templates/ddv_scenarios.md deleted file mode 100644 index 9319f8f..0000000 --- a/templates/ddv_scenarios.md +++ /dev/null @@ -1,19 +0,0 @@ -# DDV Scenarios - -## Manifest - - - -| Scenario | Category | Pass Relevance | Source | Visual | Status | -|----------|----------|----------------|--------|--------|--------| - -## Scenarios - - diff --git a/templates/lisa_claude.md b/templates/lisa_claude.md new file mode 100644 index 0000000..2bf65e1 --- /dev/null +++ b/templates/lisa_claude.md @@ -0,0 +1,105 @@ +# CLAUDE.md — Lisa Loop Artifact Guide + +This directory (`.lisa/`) contains all process artifacts from a Lisa Loop run. +Use this guide to find and interpret results, methods, assumptions, and validation evidence. + +## Quick Start + +- **What was the assignment?** Read `../ASSIGNMENT.md` +- **What's the current status?** Read `state.toml` +- **What methodology was chosen?** Read `methodology/methodology.md` +- **What are the results?** Read `spiral/pass-N/review-package.md` (latest pass) +- **Did the results pass validation?** Read `spiral/pass-N/system-validation.md` + +## Artifact Map + +### Configuration & State + +| File | Purpose | +|------|---------| +| `../lisa.toml` | Project configuration (models, limits, review gates, paths, commands) | +| `state.toml` | Current spiral state machine position | +| `CODEBASE.md` | Auto-discovered project structure summary | +| `STACK.md` | Resolved technology stack and build/test commands | + +### Skills (engineering standards for agents) + +| File | Purpose | +|------|---------| +| `skills/engineering-judgment.md` | Three-level bounding methodology (phenomenon, composition, system) | +| `skills/dimensional-analysis.md` | Unit tracking through computation chains | +| `skills/numerical-stability.md` | Discretisation, convergence, floating point checks | +| `skills/literature-grounding.md` | Optional reference data comparison methodology | + +### Methodology (refined each pass) + +| File | Purpose | +|------|---------| +| `methodology/methodology.md` | Full method specification: governing equations, assumptions, valid range | +| `methodology/plan.md` | Task breakdown with status (TODO/IN_PROGRESS/DONE/BLOCKED) per pass | +| `methodology/assumptions-register.md` | Explicit assumptions and known limitations | +| `methodology/derivations/*.md` | Non-trivial derivations (discretizations, transforms, numerical schemes) | + +### Validation & Verification + +| File | Purpose | +|------|---------| +| `validation/sanity-checks.md` | Order-of-magnitude, trend, conservation, and dimensional checks | +| `validation/limiting-cases.md` | Analytical limiting cases to verify | +| `validation/reference-data.md` | Published reference datasets for comparison | + +### Per-Pass Artifacts (`spiral/pass-N/`) + +| File | Purpose | +|------|---------| +| `acceptance-criteria.md` | (Pass 0 only) Final acceptance targets | +| `literature-survey.md` | (Pass 0 only) Method candidates surveyed | +| `spiral-plan.md` | (Pass 0 only) Scope progression strategy across passes | +| `refine-summary.md` | What methodology changed this pass | +| `execution-report.md` | Intermediate values and outputs from Build | +| `system-validation.md` | Full validation results: bounding audit, test counts, sanity checks | +| `progress-tracking.md` | Cross-pass convergence metrics and coverage | +| `review-package.md` | Human-facing summary with key results and recommendations | +| `plots/REVIEW.md` | Index of all plots with descriptions and assessments | +| `plots/*.png` | Visual evidence (bounding checks, convergence, reference data) | +| `reconsiderations/*.md` | Methodology issues pending adjudication | +| `code-diff.patch` | Code changes vs. previous pass | +| `PASS_COMPLETE.md` | Marker indicating this pass finished | + +### Output + +| File | Purpose | +|------|---------| +| `output/audit-summary.md` | Final audit: deliverables produced, validation status, evidence trail | + +### References + +| Directory | Purpose | +|-----------|---------| +| `references/core/` | User-supplied reference papers and data | +| `references/retrieved/` | Agent-retrieved reference summaries | + +## How to Read Results + +1. Check `state.toml` to see which pass completed last +2. Read `spiral/pass-N/review-package.md` for the high-level summary +3. Read `spiral/pass-N/system-validation.md` for detailed test results and bounding audit +4. Check `spiral/pass-N/plots/REVIEW.md` for visual evidence +5. Read `spiral/pass-N/progress-tracking.md` for convergence across passes +6. Read `methodology/assumptions-register.md` for caveats and limitations + +## How to Interrogate Methods + +- The governing equations and their sources are in `methodology/methodology.md` +- Each non-trivial derivation is documented in `methodology/derivations/` +- Engineering skills in `skills/` define the verification methodology agents follow +- The `literature-survey.md` in pass-0 shows what alternatives were considered and why they were rejected + +## How to Assess Trustworthiness + +- Check bounding test results by level (phenomenon, composition, system) in `system-validation.md` +- Review bounding discipline audit: does every phenomenon have L1 bounds? Every composition L2? +- Review sanity checks in `validation/sanity-checks.md` +- Look for reconsiderations in `spiral/pass-N/reconsiderations/` — these flag unresolved issues +- Compare convergence across passes in `progress-tracking.md` +- Read `methodology/assumptions-register.md` for known limitations diff --git a/templates/plots_review.md b/templates/plots_review.md deleted file mode 100644 index 3dc065c..0000000 --- a/templates/plots_review.md +++ /dev/null @@ -1,12 +0,0 @@ -# Plot Review - - - -## Plots - -## Anomalies