Skip to content

Add integration test harness - #221

Open
matoushybl wants to merge 4 commits into
mainfrom
dev-test-harness
Open

Add integration test harness#221
matoushybl wants to merge 4 commits into
mainfrom
dev-test-harness

Conversation

@matoushybl

@matoushybl matoushybl commented Aug 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This PR adds a test harness usable for making sure the evok API doesn't break when refactoring/fixing problems.

It is based on the following ideas:

  • Record cassettes with requests and responses for specific endpoints/protocols.
  • Use these cassettes to generate a spec (snapshot of a schema).
  • Replay mode validates the cassettes, tests and spec are correct.
  • Compare mode allows for checking a drift in server behavior against a running device.
  • Functional mode runs tests against mocked hardware.

Review recommendations:

  • this code is LLM generated and validated against a running Neuron M103
  • majority of the diff are the recording
  • I made quick review throughout the implementation, forcing the clanker to use types etc. Another source of linting was ruff.
  • It may be possible that the model got poisned with evok's code quality
  • I hope that this code is largely throwaway in the long run, relying on more robust systems instead.

@matoushybl
matoushybl force-pushed the dev-test-harness branch 9 times, most recently from 4b75bd2 to 498681a Compare August 26, 2026 12:49
@matoushybl
matoushybl force-pushed the dev-test-harness branch 3 times, most recently from 610b82d to 182753a Compare August 27, 2026 14:27
@matoushybl matoushybl changed the title Dev test harness Add integration test harness Aug 27, 2026
@matoushybl
feat: Add integration test suite
2d75fba 
Adds integration test suite, suitable for validating new development.
The suite captures the current API state to avoid breaking the API for
existing users, while documenting currently existing problems.
@matoushybl
ci: Automatically run integration tests
73f0c37
@matoushybl
matoushybl marked this pull request as ready for review August 27, 2026 15:21
cleveHEX
cleveHEX approved these changes Aug 31, 2026
View reviewed changes

@cleveHEX cleveHEX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cleveHEX cleveHEX cleveHEX approved these changes
@miroslav-ondra miroslav-ondra Awaiting requested review from miroslav-ondra

Assignees

@matoushybl matoushybl

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@matoushybl @cleveHEX