diff --git a/CHANGELOG.md b/CHANGELOG.md index 5699136..e246f42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes to TangleClaw are documented in this file. ## [Unreleased] +### Fixed + +- **`_parseFields` now accepts natural-English heading variants (#201)** — `lib/wrap-steps/ai-content.js:_parseFields` previously required exact-string equality (after `.toLowerCase()`) between the `## Heading` text and the declared `captureField` name. This silently rejected the most natural Markdown a human or AI would write: `step.captureFields: ['nextSteps']` would match `## nextsteps` but NOT `## Next Steps` (space-separated), `## next-steps` (kebab), `## NEXT_STEPS` (screaming snake), or `## Next.Steps`. The handler then returned `{ok:false, status:'blocked', blockers:['Required captureField "nextSteps" missing or empty in AI response']}` and halted the wrap pipeline. Surfaced as Critic n1 on PR #200; the workaround there was to harden the `memory-update` prompt with explicit "use these exact spellings" instructions — works, but every prompt edit is one slip away from re-introducing the block. **Fix:** new `_normalizeFieldKey(s)` helper lowercases and strips every non-`[a-z0-9]` character, applied symmetrically to both sides of the comparison. After normalization, `Next Steps`, `next-steps`, `next_steps`, `NEXT.STEPS`, and `nextSteps` all collapse to the same match key `nextsteps`. Output map keys remain the DECLARED `captureField` name (not the normalized form) so downstream consumers (`_completeV2Wrap`'s summary deriver, the prawduct lockstep / inverse-drift pins in `test/prawduct-aicontent-prompts.test.js`) do not have to also normalize. Normalization is a strict superset of the prior equality contract — every input the old parser matched still matches; only the previously-rejected variants now also match. **Test coverage:** 11 new tests in `test/wrap-pipeline.test.js`'s `_parseFields` describe block — natural-English variants (`## Next Steps`, kebab, screaming snake, dotted), canonical form still works (regression pin against over-correction), multiple mixed-style headings in one response, output-key contract (declared name, not normalized form). Plus a dedicated `_normalizeFieldKey` describe block covering non-alphanumeric stripping, case folding, digit preservation, and defensive inputs (empty, null, undefined, number coercion). The prawduct lockstep / inverse-drift tests are intentionally left untouched — they enforce a STRICTER convention than the parser (single-word PascalCase headings) as a style guide for bundled methodology prompts; the parser is the safety net for community-authored methodologies and for AI responses that drift from canonical. Full suite 2360 / 2360 pass (net +11 from this fix). Closes #201. + ### Added - **YOLO mode in the launch picker for Gemini / Aider / Codex (#209)** — extends the session-launch mode modal beyond Claude Code so every engine with a documented skip-permissions / unattended-execution flag exposes it as a picker option. **Gemini CLI** gains four modes mirroring Claude's structure: `Interactive` (default), `Auto Edit` (`--approval-mode auto_edit`), `Plan Only` (`--approval-mode plan`), and `YOLO` (`--yolo` — auto-approve all actions). **Aider** gains `Interactive` + `YOLO` (`--yes-always`). **Codex** gains `Interactive` + `Full Auto` (`--full-auto` — sandboxed automatic execution, not true YOLO; the label and description call out the distinction). The modal's render gate at `public/landing.js:470` (`Object.keys(launchModes).length > 1`) means each engine needs ≥2 modes to surface the picker, which all three satisfy. Backend wiring (`lib/sessions.js:_buildLaunchCommand`) is already engine-agnostic — no JS changes required, only data. **OpenClaw** was deliberately scoped out: it's TC's in-house wrapper with no upstream YOLO flag to map, so its `launchModes` semantics need to be designed deliberately (follow-up tracked at [#210](https://github.com/Jason-Vaughan/TangleClaw/issues/210)). **Per-engine flag rationale:** all four CLIs (`claude --dangerously-skip-permissions`, `gemini --yolo`, `aider --yes-always`, `codex --full-auto`) verified against the locally installed binary's `--help` output before wiring. **Test coverage:** new `launchModes parity across engines (#209)` describe block in `test/engines.test.js` pinning (a) each engine's YOLO flag args exactly, (b) presence of `warning` text on every YOLO mode, (c) `defaultLaunchMode` key exists in `launchModes`, (d) every engine with `launchModes` has >1 mode so the modal renders, (e) sentinel pin asserting OpenClaw intentionally has no `launchModes` until [#210](https://github.com/Jason-Vaughan/TangleClaw/issues/210) is resolved. Full suite 2349 / 2349 pass (net +6 from this PR). diff --git a/lib/wrap-steps/ai-content.js b/lib/wrap-steps/ai-content.js index 1d3011a..b260198 100644 --- a/lib/wrap-steps/ai-content.js +++ b/lib/wrap-steps/ai-content.js @@ -92,12 +92,37 @@ function _interpolatePrompt(promptTemplate, previousResults) { return promptTemplate.replace(/\{previousMemoryBlock\}/g, memoryText); } +/** + * Normalize a field-name token for matching: lowercase, strip every + * character that isn't `[a-z0-9]`. Used symmetrically on both the + * heading text and the declared captureField so the two only need to + * agree on alphanumeric content. Resolves a known fragility class + * (#201): the parser previously required exact-string equality after + * `.toLowerCase()`, which made `step.captureFields: ['nextSteps']` + * silently reject the most natural Markdown heading `## Next Steps` + * (two words). Normalizing strips spaces, hyphens, underscores, + * punctuation — so `Next Steps`, `next-steps`, `next_steps`, + * `NEXT.STEPS`, and `nextSteps` all collapse to the same match key + * `nextsteps`. + * + * @param {string} s - Raw heading text OR a captureField name + * @returns {string} Alphanumeric-only lowercase key + */ +function _normalizeFieldKey(s) { + return String(s == null ? '' : s).toLowerCase().replace(/[^a-z0-9]/g, ''); +} + /** * Parse captured pane output into `## Heading` sections. Mirrors the * `parseWrapSummary` logic in `lib/sessions.js` but returns the raw * sections map (not a flattened string) so the handler can validate * field-by-field. * + * Heading-to-captureField matching uses `_normalizeFieldKey` on both + * sides — symmetric normalization absorbs natural-English whitespace + * and punctuation variants without requiring the methodology author + * to enumerate synonyms (see #201 rationale). + * * @param {string} rawOutput - Pane scrollback as a single string * @param {string[]} captureFields - Field names expected as `## Heading` * @returns {Record} Map of field name → trimmed section content @@ -113,8 +138,8 @@ function _parseFields(rawOutput, captureFields) { for (const line of lines) { const headingMatch = line.match(/^##\s+(.+)$/); if (headingMatch) { - const heading = headingMatch[1].trim().toLowerCase(); - const matched = captureFields.find((f) => f.toLowerCase() === heading); + const heading = _normalizeFieldKey(headingMatch[1]); + const matched = captureFields.find((f) => _normalizeFieldKey(f) === heading); if (matched) { if (currentField) sections[currentField] = currentContent.join('\n').trim(); currentField = matched; @@ -302,4 +327,4 @@ const _internal = { sleep: defaultSleep }; -module.exports = { run, _internal, _interpolatePrompt, _parseFields }; +module.exports = { run, _internal, _interpolatePrompt, _parseFields, _normalizeFieldKey }; diff --git a/test/wrap-pipeline.test.js b/test/wrap-pipeline.test.js index e2d0218..92775b2 100644 --- a/test/wrap-pipeline.test.js +++ b/test/wrap-pipeline.test.js @@ -858,6 +858,91 @@ describe('wrap-step ai-content — pure helpers (#139 Chunk 5)', () => { assert.equal(parsed.summary, 'kept'); assert.equal(parsed.Other, undefined); }); + + describe('heading normalization (#201 — natural-English variants)', () => { + it('matches "## Next Steps" (space-separated) against captureField "nextSteps"', () => { + const raw = '## Next Steps\n- do x\n- do y\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + assert.equal(parsed.nextSteps, '- do x\n- do y', + '## Next Steps must match nextSteps — the canonical fragility from PR #200 Critic n1'); + }); + + it('matches "## next-steps" (kebab) against "nextSteps"', () => { + const raw = '## next-steps\nbody\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + assert.equal(parsed.nextSteps, 'body'); + }); + + it('matches "## NEXT_STEPS" (screaming snake) against "nextSteps"', () => { + const raw = '## NEXT_STEPS\nbody\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + assert.equal(parsed.nextSteps, 'body'); + }); + + it('matches "## Next.Steps" (dotted) against "nextSteps"', () => { + const raw = '## Next.Steps\nbody\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + assert.equal(parsed.nextSteps, 'body'); + }); + + it('still matches the exact canonical form "## nextSteps"', () => { + const raw = '## nextSteps\nbody\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + assert.equal(parsed.nextSteps, 'body', 'normalization must be a superset of equality — never narrower'); + }); + + it('handles multiple captureFields with mixed-style headings simultaneously', () => { + const raw = + '## Summary\nshipped X\n' + + '## Next Steps\n- a\n- b\n' + + '## Learnings\nnone\n'; + const parsed = aiContent._parseFields(raw, ['summary', 'nextSteps', 'learnings']); + assert.equal(parsed.summary, 'shipped X'); + assert.equal(parsed.nextSteps, '- a\n- b'); + assert.equal(parsed.learnings, 'none'); + }); + + it('returns the captureField under its DECLARED key, not the heading\'s normalized form', () => { + const raw = '## Next Steps\nbody\n'; + const parsed = aiContent._parseFields(raw, ['nextSteps']); + // Map key must be the literal field name the caller passed in + // ('nextSteps'), NOT the normalized form ('nextsteps') — otherwise + // _parseFields-consuming code (the handler's missing-field check, + // captureFields lockstep tests in test/prawduct-aicontent-prompts.test.js) + // would have to also normalize. + assert.equal(parsed.nextSteps, 'body'); + assert.equal(parsed.nextsteps, undefined, + 'output key must be the declared field name, not the normalized matcher key'); + }); + }); + + describe('_normalizeFieldKey (#201 — helper exposed for cross-module checks)', () => { + it('strips every non-alphanumeric character', () => { + assert.equal(aiContent._normalizeFieldKey('Next Steps'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('next-steps'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('next_steps'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('next.steps'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('next/steps'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('next steps'), 'nextsteps'); + }); + + it('lowercases', () => { + assert.equal(aiContent._normalizeFieldKey('NEXTSTEPS'), 'nextsteps'); + assert.equal(aiContent._normalizeFieldKey('NextSteps'), 'nextsteps'); + }); + + it('preserves digits', () => { + assert.equal(aiContent._normalizeFieldKey('field42'), 'field42'); + assert.equal(aiContent._normalizeFieldKey('v1.2 stuff'), 'v12stuff'); + }); + + it('handles defensive inputs', () => { + assert.equal(aiContent._normalizeFieldKey(''), ''); + assert.equal(aiContent._normalizeFieldKey(null), ''); + assert.equal(aiContent._normalizeFieldKey(undefined), ''); + assert.equal(aiContent._normalizeFieldKey(42), '42', 'numbers coerce to strings before normalization'); + }); + }); }); });