A small example repo to create a bug report for https://github.com/schemathesis/schemathesis
Start the API:
python3 -m venv venv
source venv/bin/activate
pip install "fastapi[standard]"
fastapi dev main.pyRun schemathesis:
SCHEMATHESIS_HOOKS="./hook.py" schemathesis --config-file ./schemathesis.toml run ./oas.yaml --url http://127.0.0.1:8000 --wait-for-schema 60Schemathesis v4.21.10
━━━━━━━━━━━━━━━━━━━━━
✅ Loaded specification from ./oas.yaml (in 0.08s)
Base URL: http://127.0.0.1:8000
Specification: Open API 3.1.0
Operations: 1 selected / 1 total
Configuration: /Users/anton/Projects/schemathesis-bug-report/schemathesis
.toml
✅ API capabilities:
Supports NULL byte in headers: ✘
Accepts backslash and control characters in URL paths: ✓
⏭ Examples (in 0.11s)
⏭ 1 skipped
❌ Coverage (in 0.19s)
❌ 1 failed
✅ Fuzzing (in 0.14s)
✅ 1 passed
=================================== FAILURES ===================================
_____________________________ GET /{z}/{x}/{y}.png _____________________________
1. Test Case ID: tejXRm
- API accepted schema-violating request
Invalid data should have been rejected
Expected: 400, 401, 403, 404, 405, 406, 409, 422, 428, 5xx
Invalid component: parameter `y` in path - value greater than maximum
[200] OK:
`{"z":2,"x":0,"y":3}`
Reproduce with:
curl -X GET http://127.0.0.1:8000/2/0/3.png
=================================== SUMMARY ====================================
API Operations:
Selected: 1/1
Tested: 1
Test Phases:
⏭ Examples
❌ Coverage
✅ Fuzzing
⏭ Stateful (not applicable)
Failures:
❌ API accepted schema-violating request: 1
Test cases:
3 generated, 1 found 1 unique failures
Seed: 339635661803335157031709863483724145595
============================== 1 failure in 0.46s ==============================
- The request schemathesis thinks is schema-violating is not actually schema-violating and should pass normally
- Schemathesis version: 4.21.10
- OpenAPI version: 3.1.0