Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,15 @@ judgment-pack version
judgment-pack spec validate <pack-or->
judgment-pack spec test-conformance [suite]
judgment-pack spec schema <spec-version>
judgment-pack mcp
```

The namespace is `judgment-pack spec`, not `judgment-pack jps`. JPS remains the name of the
specification and the prefix of its provisional diagnostic codes.

The `mcp` command serves the same offline operations to a Model Context Protocol client over stdio,
so an agent can validate documents as a tool call; see [docs/agent-testing.md](docs/agent-testing.md).

## Install a tagged release

Download the archive for your operating system and architecture from
Expand Down
24 changes: 15 additions & 9 deletions docs/adr/0003-mcp-integration-and-testing-surface.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
status: proposed
status: accepted
date: 2026-07-24
deciders: Brian Jin
---

# MCP is the runtime's integration and testing surface

> **Stub.** The direction and the constraints below are settled; the server itself is not built.
> Acceptance is gated on the Phase 0 review described under "Decision outcome".
> **Accepted.** Phase 0 passed and Phase 1 is built: `internal/mcp` is a hand-rolled stdio MCP
> server wired as the `judgment-pack mcp` subcommand.

## Context and problem statement

Expand Down Expand Up @@ -36,9 +36,9 @@ Chosen direction: **A**, reached in two phases so the premise is proven before c
- **Phase 0 (proven first, no Go): option C as a warm-up.** An agent in the reference client shells
out to `judgment-pack spec validate` to pressure-test whether the runtime's diagnostics are good
enough to close an authoring loop. Review the result, *then* decide Phase 1.
- **Phase 1 (pending review): the MCP server.** Fill the `internal/mcp` seam as a `judgment-pack
mcp` subcommand over stdio, wrapping the existing engine (`validate`, `test_conformance`,
`get_schema`, `describe_runtime`) and evaluating nothing.
- **Phase 1 (built): the MCP server.** `internal/mcp` is a `judgment-pack mcp` subcommand over
stdio, wrapping the existing engine as the `validate`, `test_conformance`, `get_schema`, and
`describe_runtime` tools, and evaluating nothing.

Settled constraints, regardless of phase:

Expand All @@ -48,16 +48,22 @@ Settled constraints, regardless of phase:
- **Examples:** read from a `judgment-pack-spec` checkout; the runtime does not vendor example
packs, because the specification owns them.

Deferred: the MCP Go library choice (official `go-sdk` vs. `mark3labs/mcp-go`) — decided at
implementation time, not now.
Implementation: **hand-rolled, no SDK.** Both Go MCP SDKs (`modelcontextprotocol/go-sdk`,
`mark3labs/mcp-go`) require a Go 1.25 toolchain and pull heavy dependencies — including `oauth2`,
which a keyless offline server never uses — conflicting with this runtime's minimal, offline,
Go 1.21 posture. The MCP surface here is small (four read-only tools over stdio), so the server
speaks MCP's newline-delimited JSON-RPC directly and adds zero dependencies. An SDK belongs in a
future commercial, authenticated API/MCP service (a separate layer per
[0002](0002-language-plurality-at-the-wire.md) and the repository boundary), not in the public
offline runtime.

### Consequences

- Good, because there is no UI, auth, or hosting to build, and BYO-key falls out of the client.
- Good, because it realizes [0002](0002-language-plurality-at-the-wire.md): all-language access
from one Go wire surface.
- Bad, because testing depends on a third-party client's behavior and setup.
- Revisit / promote to `accepted` after the Phase 0 review decides Phase 1.
- Revisit if a commercial, authenticated MCP/API surface is needed; that is a separate service, not this runtime (see the implementation note above).

## More information

Expand Down
5 changes: 3 additions & 2 deletions docs/agent-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ generic Draft 2020-12 validator rather than an agent. Why the runtime, rather th
the integration and testing surface is recorded in
[ADR-0003](adr/0003-mcp-integration-and-testing-surface.md).

Today the agent drives the command-line binary directly. A first-class MCP surface is future work
(ADR-0003); the loop below needs only the CLI.
The agent drives the command-line binary directly below. The runtime also ships an MCP server
(`judgment-pack mcp`, see ADR-0003) exposing the same operations as tools to any MCP client; the
CLI loop here needs neither an SDK nor a running server.

## What you need

Expand Down
19 changes: 18 additions & 1 deletion internal/cli/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (
"github.com/Judgment-Pack/judgment-pack-runtime/internal/describe"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/display"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/fssecure"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/mcp"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/result"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/validation"
)
Expand Down Expand Up @@ -89,7 +90,7 @@ func (a *App) rootCommand() *cobra.Command {
root.SetVersionTemplate("judgment-pack {{.Version}}\n")
root.CompletionOptions.DisableDefaultCmd = true
root.PersistentFlags().BoolVar(&a.pretty, "pretty", false, "indent JSON output")
root.AddCommand(a.versionCommand(), a.specCommand())
root.AddCommand(a.versionCommand(), a.specCommand(), a.mcpCommand())
return root
}

Expand All @@ -107,6 +108,22 @@ func (a *App) specCommand() *cobra.Command {
return spec
}

func (a *App) mcpCommand() *cobra.Command {
return &cobra.Command{
Use: "mcp",
Short: "Serve the offline validator to an MCP client over stdio",
Long: "Run a Model Context Protocol server on standard input and output. It exposes the offline validation, conformance, and description operations as MCP tools; it holds no credential, opens no network connection, and evaluates nothing.",
Args: cobra.NoArgs,
RunE: func(_ *cobra.Command, _ []string) error {
server := mcp.NewServer(a.engine, a.runner)
if err := server.Serve(a.in, a.out, a.errOut); err != nil {
return &handledExit{code: result.ExitIO}
}
return nil
},
}
}

func (a *App) versionCommand() *cobra.Command {
format := "human"
command := &cobra.Command{
Expand Down
16 changes: 16 additions & 0 deletions internal/cli/app_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -268,3 +268,19 @@ func diagnosticCode(t *testing.T, jsonOutput string) string {
}
return output.Diagnostics[0].Code
}

func TestMCPSubcommandRespondsToInitialize(t *testing.T) {
input := `{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}` + "\n"
code, stdout, stderr := runTest(t, []string{"mcp"}, input)
if code != 0 || stderr != "" {
t.Fatalf("exit=%d stderr=%q", code, stderr)
}
var response map[string]any
if err := json.Unmarshal([]byte(strings.TrimSpace(stdout)), &response); err != nil {
t.Fatalf("undecodable response %q: %v", stdout, err)
}
result, ok := response["result"].(map[string]any)
if !ok || result["protocolVersion"] != "2025-06-18" {
t.Fatalf("unexpected initialize response: %#v", response)
}
}
20 changes: 10 additions & 10 deletions internal/mcp/doc.go
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
// Package mcp will expose this runtime over the Model Context Protocol.
// Package mcp exposes this runtime over the Model Context Protocol.
//
// It is a transport adapter only: it maps MCP tool and resource calls onto the
// existing validation, conformance, and describe packages and returns their
// versioned results. It evaluates no condition, resolves no outcome, and adds
// no judgment behavior of its own -- an MCP client reaches exactly the same
// core the CLI reaches, over JSON-RPC instead of argv.
// It is a transport adapter only: it maps MCP tool calls onto the existing
// validation, conformance, and describe packages and returns their versioned
// results. It evaluates no condition, resolves no outcome, opens no network
// connection, and holds no credential -- an MCP client reaches exactly the same
// offline core the CLI reaches, over JSON-RPC on stdio instead of argv.
//
// The adapter is intended to run as an "mcp" subcommand of the single
// judgment-pack binary rather than as a separate program.
//
// This package is a scaffold and currently has no implementation.
// The server speaks MCP's newline-delimited JSON-RPC 2.0 stdio framing directly,
// without a third-party SDK, to keep the runtime's dependency set minimal and
// its build free of a newer-Go-toolchain requirement. It runs as the "mcp"
// subcommand of the single judgment-pack binary.
package mcp
148 changes: 148 additions & 0 deletions internal/mcp/server.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
package mcp

import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"io"

"github.com/Judgment-Pack/judgment-pack-runtime/internal/conformance"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/result"
"github.com/Judgment-Pack/judgment-pack-runtime/internal/validation"
)

// protocolVersion is the MCP revision this server implements. When a client
// requests a version, initialize echoes it back, since this server's tool-only
// surface is version-agnostic; absent one, it answers with this value.
const protocolVersion = "2025-06-18"

// maxMessageBytes bounds one JSON-RPC line. It sits above the 10 MiB document
// limit so a validate call carrying a maximal document still fits, while
// refusing an unbounded line from a misbehaving client.
const maxMessageBytes = 16 * 1024 * 1024

const (
codeParse = -32700
codeMethodNotFound = -32601
codeInvalidParams = -32602
)

// Server maps MCP tool calls onto the offline validation core. It holds no
// network connection and no credential.
type Server struct {
engine *validation.Engine
runner *conformance.Runner
}

// NewServer builds a server over an already-constructed engine and runner.
func NewServer(engine *validation.Engine, runner *conformance.Runner) *Server {
return &Server{engine: engine, runner: runner}
}

type rpcRequest struct {
JSONRPC string `json:"jsonrpc"`
ID json.RawMessage `json:"id,omitempty"`
Method string `json:"method"`
Params json.RawMessage `json:"params,omitempty"`
}

type rpcError struct {
Code int `json:"code"`
Message string `json:"message"`
}

// Serve reads newline-delimited JSON-RPC messages from in, dispatches each, and
// writes one response line per request to out. Notifications receive no
// response. Diagnostic text goes to logw so it never mixes with protocol output
// on out. Serve returns when in reaches EOF.
func (s *Server) Serve(in io.Reader, out, logw io.Writer) error {
scanner := bufio.NewScanner(in)
scanner.Buffer(make([]byte, 0, 64*1024), maxMessageBytes)
encoder := json.NewEncoder(out)
encoder.SetEscapeHTML(false)

for scanner.Scan() {
line := bytes.TrimSpace(scanner.Bytes())
if len(line) == 0 {
continue
}
var request rpcRequest
if err := json.Unmarshal(line, &request); err != nil {
writeMessage(encoder, logw, map[string]any{"jsonrpc": "2.0", "id": nil, "error": &rpcError{Code: codeParse, Message: "Message is not valid JSON."}})
continue
}
if response, ok := s.handle(&request); ok {
writeMessage(encoder, logw, response)
}
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(logw, "mcp: input error:", err)
return err
}
return nil
}

func writeMessage(encoder *json.Encoder, logw io.Writer, response map[string]any) {
if err := encoder.Encode(response); err != nil {
fmt.Fprintln(logw, "mcp: write error:", err)
}
}

// handle routes one message. The bool is false for notifications, which never
// receive a response.
func (s *Server) handle(request *rpcRequest) (map[string]any, bool) {
switch request.Method {
case "notifications/initialized":
return nil, false
case "initialize":
return s.reply(request, s.initializeResult(request.Params), nil)
case "ping":
return s.reply(request, map[string]any{}, nil)
case "tools/list":
return s.reply(request, map[string]any{"tools": toolDefinitions()}, nil)
case "tools/call":
res, rerr := s.callTool(request.Params)
return s.reply(request, res, rerr)
default:
if len(request.ID) == 0 {
return nil, false // ignore unknown notifications
}
return s.reply(request, nil, &rpcError{Code: codeMethodNotFound, Message: "Unknown method: " + request.Method})
}
}

// reply builds a response for a request, or signals no response for a
// notification (a message with no id).
func (s *Server) reply(request *rpcRequest, res any, rerr *rpcError) (map[string]any, bool) {
if len(request.ID) == 0 {
return nil, false
}
response := map[string]any{"jsonrpc": "2.0", "id": request.ID}
if rerr != nil {
response["error"] = rerr
} else {
response["result"] = res
}
return response, true
}

func (s *Server) initializeResult(rawParams json.RawMessage) map[string]any {
version := protocolVersion
var params struct {
ProtocolVersion string `json:"protocolVersion"`
}
if len(rawParams) > 0 {
if err := json.Unmarshal(rawParams, &params); err == nil && params.ProtocolVersion != "" {
version = params.ProtocolVersion
}
}
return map[string]any{
"protocolVersion": version,
"capabilities": map[string]any{"tools": map[string]any{}},
"serverInfo": map[string]any{
"name": result.CLIName,
"version": result.CLIVersion,
},
}
}
Loading