Skip to content

Add contract testing (Schemathesis internal + Pact public)#125

Open
jschoedl wants to merge 2 commits into
optimized-llm-configfrom
contract-testing
Open

Add contract testing (Schemathesis internal + Pact public)#125
jschoedl wants to merge 2 commits into
optimized-llm-configfrom
contract-testing

Conversation

@jschoedl

@jschoedl jschoedl commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

This PR is stacked on #123, merge that first. Afterwards, I'll repoint it to main.

Adds contract testing across both inter-service boundaries, as required by the course best-practices. Each boundary uses the tool that fits its shape:

  • Internal (spring-api → py-recipe/py-help): Schemathesis provider-conformance. Both Python GenAI services are hand-written, so the real risk is provider drift from api/openapi-internal.yaml. Schemathesis derives cases from the spec and checks every response conforms (the LLM is stubbed). It re-syncs itself when the spec changes.
  • Public (web-client → spring-api): consumer-driven Pact. The consumer test exercises the real client code against a mock and generates web-client/pacts/; the provider test replays it against the real Spring app (real JWT/security, Python GenAI clients mocked). Covers every route the client uses, happy and error paths (401/404/409/400).

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Infrastructure / CI
  • Documentation

API changes

  • This PR does not affect the API
  • This PR changes the API → api/openapi.yaml updated and api/scripts/gen-all.sh re-run

Definition of Done

  • CI passes
  • Pre-commit hooks pass locally
  • Relevant tests added or updated

🤖 Generated with Claude Code

@jschoedl @claude
add schemathesis contract tests for genai services
263c1cd 
Verify py-recipe-service and py-help-service responses conform to the
internal OpenAPI contract, run in-process over ASGI with the LLM stubbed.
Both service CI workflows now also trigger on api/openapi-internal.yaml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d442a2e-fa47-40ef-b044-8823ed77a098

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch contract-testing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jschoedl @claude
add pact contract tests for web-client and spring-api
1fd4805 
Consumer-driven Pact over the public REST boundary: the web-client
consumer test generates web-client/pacts/, which the spring-api provider
test verifies against the real app (JWT auth real, Python GenAI clients
mocked). Covers every route the client uses, happy and error paths.
The committed pact is the filesystem handoff; spring-api CI triggers on it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jschoedl jschoedl force-pushed the contract-testing branch from 2e310a6 to 1fd4805 Compare July 2, 2026 19:27
@jschoedl jschoedl deployed to kubernetes July 2, 2026 19:28 — with GitHub Actions Active
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

🚀 Frontend preview ↗  ·  commit 1fd4805570230b5a87ffcb9dee71d0119a4a04f2

@jschoedl jschoedl linked an issue Jul 2, 2026 that may be closed by this pull request
Api contract testing #105
Open
@jschoedl jschoedl requested review from imol-ai and paulwiese and removed request for paulwiese July 2, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@imol-ai imol-ai Awaiting requested review from imol-ai
@paulwiese paulwiese Awaiting requested review from paulwiese

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Api contract testing

1 participant

@jschoedl