Acceptance Test for User Guide

[alexheifetz]

Add acceptance test for Embabel User Guide

Description

Introduce an automated acceptance test that validates the structure and content of the Embabel User Guide. This ensures documentation remains consistent and complete as it evolves.

Goals

Phase 1 — Structural validation (this issue)

• Verify the guide contains all expected chapters/sections by title or heading
• Assert known chapter ordering is preserved
• Fail fast on missing or renamed sections

Phase 2 — Content quality (future)

• Integrate with an eval framework to examine chapter contents more thoroughly (e.g., completeness, accuracy, coherence)

Trigger & Notification

• The acceptance test runs automatically whenever documentation files are touched (e.g., via CI path filter on docs/ or relevant glob)
• On failure, a notification is streamed to the Discord #build channel

Acceptance Criteria

• Test validates presence and order of known chapters in the User Guide
• Test is wired into CI and triggers only on documentation changes
• Failure notifications are delivered to Discord #build channel
• Test is designed to be extensible for future eval-based content checks

[alexheifetz]

User Guide Validator — Acceptance Tests

What's new

Adds UserGuideValidatorAgent to the examples repo — a Java agent that validates live sections of the Embabel User Guide via a 3-step GOAP chain:

1. extractQuery — parses the user's natural-language request into a structured query
2. findRelevantSections — fetches the live guide ToC with a scanner LLM and resolves matching section anchors; performs a structural check (≥ 8 chapters) as a pure-Java guard
3. validateSection — uses a validator LLM to check the section for accuracy and completeness

Testing approach

Coverage is split across two layers:

Unit tests — all 28 known sections, zero LLM cost via FakeOperationContext. Cover prompt shape, anchor resolution, Java guard logic, and ValidationReport structure.

Acceptance tests (this PR) — end-to-end via the live A2A server, scoped to a representative sample to avoid running 28 × (scanner + validator) LLM calls on every CI run:

Test	What it covers
UserGuideStructureTest	Structural guard — asserts guide has ≥ 8 top-level chapters; no LLM call made
UserGuideValidatorAgentTest	agent.guide, blackboard, reference.tools, reference.testing, reference.integrations__a2a

Each section asserts passed:true, presence of summary and sectionTitle, and a Zipkin trace confirming the agent ran end-to-end.