Add first-class structured source editing APIs and expose them through MCP

[HelgeSverre]

Summary

Add first-class structured source editing support to Sema, and expose the same operations through sema mcp so coding agents can edit Sema code as forms instead of raw text.

Sema already has the important building blocks:

• read/string and read/all parse Sema source into form data.
• format/form already prints one form; the public source conversion API should be form->source.
• forms->source should print multiple top-level forms to a source string.
• sema/check-string returns parse/compile diagnostics as data.
• sema mcp already exposes Sema developer tooling to agents.

What is missing is a stable, source-aware edit surface that lets an agent inspect forms, address a target form deterministically, apply a constrained edit, validate the result, and write it back safely.

Motivation

A planned Sema-native agent-evaluation harness needs a structured edit condition. The purpose is to test whether code-as-data edits reduce failure modes common in LLM-generated text diffs:

• unbalanced parentheses or malformed output
• replacing text inside strings/comments by accident
• broad multi-site replacements
• wrong-scope renames
• no-op edits that look successful
• edits that cannot be parsed after the model writes them

This should also be useful outside the experiment. Agents using Sema through MCP should be able to make targeted source changes without rewriting whole files.

Naming direction

Use Sema's existing conversion naming convention for source rendering:

• form->source for one form
• forms->source for multiple top-level forms

Do not introduce Sema API names such as format/forms, format-forms, or forms_to_source for the language/runtime layer. format/form can remain the lower-level formatter primitive or implementation detail, but the structured editing API should read as a conversion from forms back to source.

MCP tool names may need transport-safe snake_case identifiers. If so, forms_to_source is acceptable only as an MCP wrapper name, and its documentation/schema should state that it maps to the Sema API forms->source. Operation names in Sema data should still use :forms->source.

Proposed language/runtime APIs

Exact names are open except for the source conversion direction above. The feature should provide a small stable layer over forms and source files. Possible API shape:

(read/all source)                   ; already exists: parse all top-level forms
(form->source form)                 ; new public conversion over existing format/form
(forms->source forms)               ; new: multiple top-level forms -> source string
(sema/check-string source)          ; already exists: diagnostics as data

(form/at-path forms path)           ; path like [top-level-index child-index ...]
(form/replace-at-path forms path replacement)
(form/walk f form)
(form/find forms predicate-or-query)
(source/forms path)                 ; read + parse + attach metadata where possible
(source/write-forms path forms opts)

The important contract is:

1. Parse a full Sema file into all top-level forms.
2. Address forms deterministically by path.
3. Apply edits to form data rather than arbitrary strings.
4. Re-render valid Sema source through form->source / forms->source.
5. Check parse/compile validity before writing or before reporting success.
6. Return structured metadata about what changed.

Proposed structured edit operations

Initial operations that would cover most agent repair/refactor tasks:

[:replace-symbol
 :rename-binding
 :insert-definition
 :replace-form-at-path
 :wrap-form
 :unwrap-form
 :append-to-list
 :replace-literal
 :rewrite-call
 :forms->source]

Each operation should return a structured result like:

{:ok #t
 :operation :replace-symbol
 :path "src/foo.sema"
 :changed #t
 :parse-valid-before #t
 :parse-valid-after #t
 :diff-preview "..."
 :forms-touched 2}

Failures should be structured too:

{:ok #f
 :operation :rename-binding
 :error {:kind :ambiguous-target
         :message "Symbol appears in multiple binding scopes; provide a form path."}}

MCP exposure

Expose the same surface through sema mcp, probably as one or more tools:

• inspect_forms
• structured_edit
• check_source
• forms_to_source as the transport-safe MCP wrapper for Sema forms->source

The MCP schema should make the operation explicit instead of accepting free-form patches:

{
  "path": "src/foo.sema",
  "operation": "replace-form-at-path",
  "args": {
    "formPath": [0, 2, 1],
    "replacement": "(+ x 1)"
  },
  "dryRun": true
}

The MCP tool should:

• validate paths with the existing path sandbox rules
• support dry-run mode
• return a diff preview
• run sema/check-string on the resulting source
• reject invalid output unless explicitly requested otherwise
• never silently rewrite unrelated files

Source preservation questions

A minimal version can canonicalize edited forms through form->source / forms->source, but the long-term feature should decide how much source trivia to preserve:

• comments
• blank lines
• shebangs
• original string/f-string/regex literal spelling
• top-level spacing
• metadata or reader sugar

If full trivia preservation is too large for v1, the API should document that structured edits canonicalize touched forms and preserve the rest of the file where feasible.

Acceptance criteria

• A Sema script can parse a multi-form .sema file, replace a form by path, render it back with forms->source, and pass sema/check-string.
• forms->source handles multiple top-level forms cleanly.
• At least replace-symbol, replace-form-at-path, and insert-definition are implemented.
• The MCP server exposes a structured edit tool with dry-run, diff preview, and validation.
• Path handling respects --sandbox / allowed paths and rejects traversal or symlink escapes.
• Tests cover maps, vectors, short lambdas, f-strings, regex literals, comments, and multiple top-level forms.
• Ambiguous operations return structured errors instead of guessing.

Why this belongs in Sema

This is a natural extension of Sema's existing code-as-data story. It makes homoiconicity operational for agents: not just (read source) and (eval form), but safe inspect/edit/validate/write workflows that tools and MCP clients can rely on.