Skip to content

Agent report-back: structured summary after MCP-driven course/deck builds #141

Description

@johnfire

From the same 2026-09-15 strategy conversation as #1 (agents-in-tow as the primary market). If a user's agent is doing the building, the user needs a way to see what it did without reading every tool call.

The idea

Right now build_a_course (packages/mcp/src/prompts/build-a-course.ts) guides a connected AI through clarifying a goal, grounding it in real sources, building a syllabus, and authoring cards — but there's no structured output at the end that the agent hands back to the human. The agent just stops after the last tool call.

Add a "summary" step/tool to the course-building (and possibly deck-building) flow that produces something like: what was built (course/deck names, card counts, difficulty progression), what sources it was grounded in, and why it made the structural choices it made (e.g. "deck 2 gates on deck 1 because X is a prerequisite for Y"). This becomes the trust layer for a user who didn't watch the process happen live.

Why this matters

The whole pitch of "AI builds this for you" only works if the user can verify what got built without re-reading every card themselves. A course showing up in the library with zero explanation is a worse experience than one accompanied by a short rationale the agent can just paste into chat.

Decisions to make

  • New MCP tool vs. a documented expectation in the prompt text? E.g. a summarize_course tool that reads back structured data for the agent to narrate, vs. just instructing build_a_course to compose a summary itself from what it already knows.
  • Does this apply narrowly to Courses, or generally to any MCP-driven content creation (a single deck, add_cards on an existing deck, etc.)?
  • Where does the summary live, if anywhere — is it purely conversational (the agent just tells the user), or should something persist on the course/deck record itself (e.g. a "how this was built" note visible in the web/Android UI too)?

Reuse

Builds on the existing build_a_course prompt and Courses data model (courses, course_decks tables) — this is additive, no schema changes obviously required unless the "persist a rationale" option is chosen.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ideaExploratory idea, not yet committed to

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions