structured-generation-engine

Structured outputs for application boundaries.

This repo takes a target JSON schema, forces a model through that shape, validates the returned payload locally, and retries with explicit validation feedback if the first pass is wrong.

It is built for the boring but expensive problem in AI systems: turning natural-language input into typed payloads that another service can execute without regex cleanup or hand-written parsers.

What Runs Here

• JSON schema or Pydantic model input
• provider-swappable generation backend
• Azure OpenAI backend using a forced tool call whose parameters are the target schema
• local jsonschema validation after every model response
• bounded retry loop with concrete validation errors fed back to the model
• CLI for generation and validation
• checked-in live demo artifacts for nested objects, enums, arrays, booleans, and dates

This is not a token-mask decoder. It is a practical control layer for structured outputs in production APIs.

Live Demo

The checked-in demo set was generated with the live Azure backend using gpt-5-mini.

Scenarios:

• purchase-order: procurement email to nested purchase order JSON
• incident-brief: incident handoff text to typed incident payload

Artifacts:

• demo/input/purchase-order.prompt.txt
• demo/input/purchase-order.schema.json
• demo/input/incident-brief.prompt.txt
• demo/input/incident-brief.schema.json
• demo/output/purchase-order.json
• demo/output/incident-brief.json
• demo/output/demo-summary.json

Rendered demo captures:

Observed summary:

[
  {
    "name": "purchase-order",
    "model": "gpt-5-mini",
    "latency_ms": 20876,
    "total_tokens": 1115,
    "validation_retries": 0
  },
  {
    "name": "incident-brief",
    "model": "gpt-5-mini",
    "latency_ms": 29167,
    "total_tokens": 1516,
    "validation_retries": 0
  }
]

Purchase order excerpt:

{
  "order_id": "AMER-4821",
  "vendor_name": "Northwind Systems",
  "currency": "USD",
  "requested_ship_date": "2026-04-30"
}

Python API

Generate against a raw JSON schema:

from structured_generation_engine import (
    AzureStructuredProvider,
    Settings,
    StructuredGenerationEngine,
)

engine = StructuredGenerationEngine(
    AzureStructuredProvider(Settings.from_env()),
    max_validation_attempts=2,
)

schema = {
    "title": "user_profile",
    "type": "object",
    "properties": {
        "email": {"type": "string", "format": "email"},
        "plan": {"type": "string", "enum": ["free", "pro", "enterprise"]},
        "needs_follow_up": {"type": "boolean"}
    },
    "required": ["email", "plan", "needs_follow_up"],
    "additionalProperties": False
}

result = engine.generate_json(
    user_prompt="Extract the user's plan and email from the support note ...",
    schema=schema,
)

print(result.raw_payload)
print(result.trace.total_tokens)

Generate against a Pydantic model:

from pydantic import BaseModel
from structured_generation_engine import (
    AzureStructuredProvider,
    Settings,
    StructuredGenerationEngine,
)


class RefundDecision(BaseModel):
    ticket_id: str
    approve_refund: bool
    reason_code: str


engine = StructuredGenerationEngine(AzureStructuredProvider(Settings.from_env()))
result = engine.generate_model(
    user_prompt="Normalize the billing escalation into the refund decision model.",
    response_model=RefundDecision,
)

print(result.output.approve_refund)

CLI

Install:

uv sync --extra dev

Generate a typed payload:

export AZURE_OPENAI_ENDPOINT="https://<resource>.openai.azure.com/"
export AZURE_OPENAI_API_KEY="<key>"
export AZURE_OPENAI_API_VERSION="2025-04-01-preview"
export AZURE_OPENAI_DEPLOYMENT="gpt-5-mini"

uv run sge generate \
  --prompt-file demo/input/purchase-order.prompt.txt \
  --schema-file demo/input/purchase-order.schema.json \
  --out /tmp/purchase-order.json \
  --trace-out /tmp/purchase-order.trace.json

Validate an existing payload:

uv run sge validate \
  --schema-file demo/input/purchase-order.schema.json \
  --json-file /tmp/purchase-order.json

Regenerate the checked-in live demo:

uv run python scripts/run_live_demo.py

Design Notes

• provider output is never trusted until the local validator accepts it
• retries are explicit and bounded
• schema enforcement is local, provider choice is swappable
• the retry loop preserves model flexibility while keeping the application contract rigid

If the provider cannot satisfy the schema, the engine raises instead of returning a partially valid object.

Files Worth Reading

• src/structured_generation_engine/engine.py
• src/structured_generation_engine/schema.py
• src/structured_generation_engine/azure_provider.py
• src/structured_generation_engine/cli.py
• scripts/run_live_demo.py
• docs/architecture.md
• docs/azure-foundry.md

Tests

uv run pytest -q