Skip to content

test: add performance regression guards - #24

Merged
zefutoff merged 1 commit into
mainfrom
test/performance-regressions
Oct 8, 2026
Merged

zefutoff merged 1 commit into
mainfrom
test/performance-regressions

Conversation

@zefutoff

@zefutoff zefutoff commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • add real-engine performance regression coverage
  • verify repeated analyses reuse the initialized QuickJS runtime without reloading Grammalecte assets
  • add tolerant timing budgets for cold start, sentence analysis and paragraph analysis
  • keep the existing opt-in characterization benchmark
  • add a dedicated make performance-check target
  • run performance regression checks in the existing CI quality job
  • document the measured development baseline and regression budgets
  • complete the Phase 4 performance-regression roadmap item

Regression budgets

  • cold first correction: ≤ 3000 ms
  • sentence median: ≤ 75 ms
  • sentence p95: ≤ 200 ms
  • paragraph median: ≤ 150 ms
  • paragraph p95: ≤ 400 ms

The thresholds are intentionally wider than the current development baseline to detect substantial regressions without making CI sensitive to normal host-load variation.

Development baseline

Three representative local runs measured:

  • cold first correction: 809.54–828.45 ms
  • sentence median: 13.48–13.84 ms
  • sentence p95: 14.94–16.45 ms
  • paragraph median: 31.89–32.23 ms
  • paragraph p95: 32.85–34.35 ms

A final regression run measured:

  • cold first correction: 809.58 ms
  • sentence median: 13.90 ms
  • sentence p95: 15.52 ms
  • paragraph median: 32.14 ms
  • paragraph p95: 34.01 ms

Validation

  • make check
  • make performance-check
  • make assemble
  • git diff --check
  • final APK does not request android.permission.INTERNET

Roadmap

Completes:

  • Add performance regression tests

@zefutoff
zefutoff merged commit 15b744d into main Oct 8, 2026
5 checks passed
@zefutoff
zefutoff deleted the test/performance-regressions branch October 8, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant