ci: typecheck runner tests

[borgesius]

Summary

• extend runner/API CI mypy from src to src tests
• clean up test typing issues: stale ignores, fixture generator return types, fake client casts, and typed JSON/audio fixture helpers
• normalize RimeTTSProvider's voice annotation to match its existing None -> luna fallback behavior

Validation

• parsed .github/workflows/ci.yml
• uv sync --frozen --dev
• uv run ruff format --check .
• uv run ruff check .
• uv run mypy --strict src tests
• uv run pytest -q (582 passed, 1 skipped)

Greptile Summary

This PR extends the runner's mypy strict check from src to src tests, then cleans up all typing issues the expanded check surfaced. No behavioral changes are intended except for RimeTTSProvider, which now officially accepts voice=None and falls back to "luna" in its constructor.

• CI: uv run mypy --strict src tests now covers the full test tree, preventing future type regressions in test code.
• Source fixes: GCS imports are rewritten to import google.cloud.storage as storage for mypy strict compatibility; RimeTTSProvider.__init__ widens voice to str | None with a or "luna" default.
• Test fixes: Stale type: ignore suppressions removed across 12+ test files, fixture generators typed as Generator[None, None, None], fake client assignments replaced with cast(Any, ...), and SecretStr(...) wrappers added to Settings constructors.

Confidence Score: 5/5

Safe to merge — all changes are typing housekeeping and the only behavioral change is the officially documented None→luna fallback in RimeTTSProvider.

The diff is entirely type annotation cleanup: removing stale ignores, adding SecretStr wrappers, correcting generator return types, and widening the rime voice parameter. The single behavioral change (voice=None → "luna") was already the implicit runtime behavior before this PR. No logic paths were restructured.

runner/src/coval_bench/providers/tts/rime.py has a minor dead-code fragment (the or "luna" guard in synthesize() is now unreachable after the constructor change).

Important Files Changed

Filename	Overview
.github/workflows/ci.yml	Extends mypy strict type checking from src to src tests, ensuring tests are now covered by the same strict typing rules.
runner/src/coval_bench/providers/tts/rime.py	Widens voice parameter to `str
runner/src/coval_bench/datasets/loader.py	Switches GCS import from from google.cloud import storage to import google.cloud.storage as storage for mypy strict compatibility; functionally identical.
runner/src/coval_bench/datasets/scripts/build_dataset.py	Same GCS import style fix as loader.py, applied in both the TYPE_CHECKING guard and the runtime _gcs_client() function.
runner/tests/providers/tts/conftest.py	Replaces bare string literals passed to Settings with explicit SecretStr(...) wrappers (eliminating the type: ignore[arg-type] suppressions) and corrects FakeWebSocket.__aiter__ return type to `AsyncIterator[str
runner/tests/providers/test_http_session.py	Fixes fixture generator return type, uses cast(Any, ...) to access private transport internals and inject a _BrokenClient, removing stale type: ignore suppressions.
runner/tests/unit/test_db_writer.py	Fully parameterises AsyncConnectionPool generic with the correct connection and row types, removing two type: ignore[type-arg] suppressions; also refines a null-assertion pattern.
runner/tests/providers/stt/conftest.py	Wraps json.loads return with cast(list[Any], ...) and wraps numpy tobytes() result with bytes() to satisfy mypy, replacing stale type: ignore comments.
runner/tests/unit/test_orchestrator.py	Removes a stale type: ignore[arg-type] on database_url and refines an assertion to use explicit is not None pattern for mypy strict compatibility.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[RimeTTSProvider.__init__ voice str or None] --> B{voice is None?}
    B -- Yes --> C[self._voice = luna]
    B -- No --> D[self._voice = voice]
    C --> E[synthesize: speaker = self._voice always a str]
    D --> E
    E --> F[WebSocket /ws3 request]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[RimeTTSProvider.__init__ voice str or None] --> B{voice is None?}
    B -- Yes --> C[self._voice = luna]
    B -- No --> D[self._voice = voice]
    C --> E[synthesize: speaker = self._voice always a str]
    D --> E
    E --> F[WebSocket /ws3 request]

Loading

Comments Outside Diff (1)

1. runner/src/coval_bench/providers/tts/rime.py, line 76-77 (link)

   After __init__ assigns self._voice = voice or "luna", self._voice is always a non-empty str. The or "luna" guard in synthesize() can never trigger and is dead code left over from before the annotation was widened.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "ci: typecheck runner tests" | Re-trigger Greptile}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
benchmarks	Ready	Preview, Comment	Jun 19, 2026 4:55am

[coderabbitai]

Warning

Review limit reached

@borgesius, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 1 hour, 52 minutes, and 53 seconds. Learn how PR review limits work.

To continue reviewing without waiting, enable usage-based billing in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 286f0df3-5d7d-467a-9ccc-eec59bf3278f

📥 Commits

Reviewing files that changed from the base of the PR and between d90a63e and ca7e065.

📒 Files selected for processing (19)

• .github/workflows/ci.yml
• runner/src/coval_bench/datasets/loader.py
• runner/src/coval_bench/datasets/scripts/build_dataset.py
• runner/src/coval_bench/providers/tts/rime.py
• runner/tests/api/test_providers.py
• runner/tests/conftest.py
• runner/tests/providers/stt/conftest.py
• runner/tests/providers/test_http_session.py
• runner/tests/providers/tts/conftest.py
• runner/tests/providers/tts/test_cartesia.py
• runner/tests/providers/tts/test_elevenlabs.py
• runner/tests/providers/tts/test_gradium.py
• runner/tests/providers/tts/test_hume.py
• runner/tests/providers/tts/test_openai.py
• runner/tests/providers/tts/test_rime.py
• runner/tests/providers/tts/test_smallest.py
• runner/tests/unit/test_dataset_loader.py
• runner/tests/unit/test_db_writer.py
• runner/tests/unit/test_orchestrator.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/benchmarks-runner-mypy-tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}