Skip to content

Test coverage pass: wire the suites that ran nowhere into CI, close the sharpest unit gaps, gate coverage - #2

Merged
mianaz merged 3 commits into
mainfrom
claude/test-coverage-analysis-sr3atm
Aug 8, 2026
Merged

Test coverage pass: wire the suites that ran nowhere into CI, close the sharpest unit gaps, gate coverage#2
mianaz merged 3 commits into
mainfrom
claude/test-coverage-analysis-sr3atm

Conversation

@mianaz

@mianaz mianaz commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Three rounds of the coverage pass that came out of the test-coverage analysis: first make every existing-but-orphaned suite actually execute, then close the highest-value unit gaps, then instrument coverage so regressions stay visible.

1. Suites that silently ran nowhere now gate CI

  • tauri-plugin-os-speech Rust crate: an iOS-target-only dependency with no cargo workspace, so the app crate's cargo test never even compiled it. CI now runs cargo test --manifest-path for it (its desktop.rs stub exists precisely so a host build type-checks), with the rust-cache extended to cover it.
  • The iOS plugin's 43 Swift tests (ios/Tests/OsSpeechCoreTests): blocked from any host run by two independent issues — SwiftPM's aggregate test bundle builds the Tauri-dependent glue target (Tauri imports UIKit unconditionally, so it can't compile for macOS), and the manifest's ../.tauri/tauri-api path dependency is gitignored, so fresh checkouts fail at dependency resolution. New scripts/test-ios-core.mjs stages OsSpeechCore + its tests into a scratch Tauri-free manifest and runs swift test there — the automated form of the manual workaround ios/Package.swift documented. Wired into the macOS CI job; the Package.swift host-testing note now records both blockers.
  • scripts/ joins the root vitest projects glob (a test there could never execute before). check-versions.mjs exports its extraction/comparison logic behind an is-main guard and gets the first scripts suite (14 tests pinning the regexes and the exact-vs-startsWith site rules).

2. Unit gaps closed

  • core: sourceSentence (the shared sentence→clause→word-safe-window narrowing rule with three consumers across two workspaces) and glossaryLookup (mocked away in every suite that touched it — the personal-glossary shadowing logic was never asserted).
  • web: define/translate task modules (truncation ceilings, per-kind defaults, hallucinated-id postFilter, prompt wiring — the only two untested llm/tasks modules), the proxyUrl validator shared with the backup-restore boundary, and DemoEngine — the default engine and first-run experience, whose timer/teardown lifecycle ran in zero tests (every useMeeting suite mocks @/lib/stt away).
  • extension: the mic first-grant page — pins that every track stops before the success copy renders (the hot-mic guard), both zh outcome strings verbatim, and the no-auto-request-on-load contract. permission.ts gains an export {} module marker for tsc.
  • rust: models.rs wire-contract tests — the Rust half of what OsSpeechWireTests.swift already pinned from Swift (explicit-null reason, camelCase vs deliberately-not, Swift envelope unwraps).
  • sidecar: test_agent_prompts.py — tripwires for the zh→en .replace() chains that silently no-op when the zh source drifts; plus the two largest untested blocks in the repo, the HTTP dispatch layers: test_agent_http.py (54 checks — gate ordering: origin/token 403s fire before the body is ever read, asserted via rfile position; the provider stub records zero calls on every rejected path) and test_job_http.py (48 checks — routing, download single-flight 409, terminal-cancel zh string, upload body bytes down to the tmp file, CORS).

3. Coverage becomes a CI gate

@vitest/coverage-v8 as a real devDependency, npm run test:coverage, and the CI web job now runs the instrumented suite. Thresholds (77/72/73/79) sit a couple of points under measured baselines (81.1 stmts / 77.6 branches / 77.1 functions / 82.3 lines) with include-all denominators — never-imported files count against the floor, closing the blind spot where 14 files (page.tsx, the whole extension side panel…) were invisible to a "85% overall" number.

Verification

  • 286 vitest files / 5,146 tests green under the instrumented run; typecheck green; check:versions green; 13/13 sidecar files green; models.rs suite verified in a scratch crate. Test badges restamped 5073 → 5146.
  • First-time-in-CI steps to watch on this PR: the plugin cargo test step and the Swift scratch-runner (no mac toolchain in the dev environment — this PR's macOS job is their first real execution).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KvWjVGboyurjEUYBKx3a38


Generated by Claude Code

claude added 3 commits August 7, 2026 05:04
…est unit gaps close

Item 1 — tests that silently didn't run:
- ci.yml compiles + tests the os-speech plugin crate (an iOS-target-only
  dependency with no workspace, so the app crate's cargo test never even
  built it) and runs the iOS plugin's 43 Swift core tests via
  scripts/test-ios-core.mjs — a scratch Tauri-free manifest that
  sidesteps both host blockers (the glue target's Tauri/UIKit dependency
  and the gitignored ../.tauri path dep that fails resolution on a fresh
  checkout; ios/Package.swift's host-testing note now records both).
- scripts/ joins the root vitest projects glob (a test there could never
  execute before); check-versions.mjs exports its extraction/comparison
  logic behind an is-main guard and gets the first scripts suite.

Item 4 — high-value unit gaps:
- core: sourceSentence (the shared narrowing rule, three consumers, no
  direct test) and glossaryLookup (mocked away in every suite that
  touched it — the personal-glossary shadowing logic was never asserted).
- web: define/translate task modules (truncation ceilings + hallucinated-
  id postFilter, the only two untested llm/tasks) and the proxyUrl
  validator shared with the backup-restore boundary.
- extension: the mic first-grant page — pins that every track stops
  BEFORE the success copy renders (the hot-mic guard) and both outcome
  strings; permission.ts gains an export{} module marker for tsc.
- rust: models.rs wire-contract tests, the Rust half of what
  OsSpeechWireTests.swift already pinned from Swift (explicit-null
  reason, camelCase vs deliberately-not, envelope unwraps).
- sidecar: test_agent_prompts.py — tripwires for the zh->en .replace()
  chains that silently no-op when the zh source drifts; the system-
  prompt builders had never been called by any test.

Verified here: 285 vitest files / 5140 tests green, typecheck green,
11/11 sidecar files green, models.rs suite green in a scratch crate
(the plugin's own cargo test needs the mac toolchain — CI's new step).
The swift scratch-runner can only prove itself on the macos-26 job.
Badges restamped 5073 -> 5140.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWjVGboyurjEUYBKx3a38
…nd coverage itself becomes a CI gate

- lib/stt/demo.ts (the DEFAULT engine — every useMeeting suite mocks
  @/lib/stt away, so its timer/teardown lifecycle ran in zero tests):
  fake-timer suite pins the listening status, cumulative interim
  prefixes, verbatim finals stamped with line-START time, the
  demo_finished idle detail, stop() halting already-scheduled ticks,
  and restart-after-stop.
- @vitest/coverage-v8 becomes a real devDependency with a root
  `test:coverage` script; ci.yml's web job runs it in place of the
  uninstrumented `npm test`.
- Root vitest config carries the coverage contract: include-all
  denominators over the three product source trees (Vitest 4 otherwise
  only reports files a test happened to LOAD — the mechanism that kept
  page.tsx and the whole extension side panel invisible at "85%"),
  scripts/ deliberately out, and floors (77/72/73/79) a few points
  under the measured baselines (81.1/77.6/77.1/82.3) so a landing dark
  module fails while routine refactors don't.

286 files / 5146 tests green under the instrumented run; badges
restamped. Sidecar HTTP-handler suites land separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWjVGboyurjEUYBKx3a38
… largest untested blocks in the repo

- test_agent_http.py (54 checks): make_agent_http_handler's gate ORDER
  is the money — origin 403 and token 403 must fire before the body is
  ever read (rfile position asserted untouched), route 404 before the
  gates, the 413 boundary is > MAX_BODY_BYTES not >=, malformed JSON
  vs valid-JSON-non-dict get their distinct 400s with their real zh
  strings, AgentCallError codes map through ERROR_CODE_STATUS, and the
  provider stub records ZERO calls on every rejected path — a
  reordering bug that kept all the pure-function tests green would now
  fail loudly instead of opening the ride-the-subscription hole.
- test_job_http.py (48 checks): make_job_http_handler's routing —
  /health payload, /jobs listing + 404, download-model 202/400/single-
  flight 409, cancel of a terminal job answers 任务已结束,无法取消,
  ingest-url origin/validation gates, PUT /transcribe upload lands the
  exact body bytes in the tmp file start_job receives (busy path
  deletes it, happy path leaves it for the worker), CORS/OPTIONS.
- Handlers run their REAL do_* code sans socket (Handler.__new__ +
  BytesIO + recorder send_response/send_header), factories closed over
  minimal fakes exposing exactly the surface the handlers touch;
  agent_server.py and whisper_server.py untouched.

13/13 sidecar files green under scripts/test-sidecar.mjs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWjVGboyurjEUYBKx3a38
@mianaz
mianaz merged commit 918d338 into main Aug 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants