Skip to content

Strengthen the test suite: 298 → 365 tests - #72

Merged
danielchalmers merged 1 commit into
mainfrom
test-suite-improvements
Aug 1, 2026
Merged

Strengthen the test suite: 298 → 365 tests#72
danielchalmers merged 1 commit into
mainfrom
test-suite-improvements

Conversation

@danielchalmers

@danielchalmers danielchalmers commented Aug 1, 2026

Copy link
Copy Markdown
Owner

A review pass over the whole suite for coverage gaps, redundancy, brittleness, and maintainability. Test-only — the three production bugs this review turned up, and the CI changes, are in #73 on top of this.

The suite was already in good shape — fast, deterministic, no external dependencies, thoughtful fixtures — so this preserves those properties and targets the weaknesses that were actually concentrated somewhere. 365 tests, still ~3s, green in Debug and Release, stable over repeated runs.

Coverage gaps closed

Several surfaces weren't merely untested, they were unreachable — mostly for want of a small seam the codebase already uses elsewhere (clipLoader, backgroundYield, savePathPicker).

Surface Was Now
ClipExporter.ExportAsync zero tests — needs a real ffmpeg process optional runFfmpeg delegate; the not-installed and partial-output-cleanup paths covered
PackageManager no test file at all ExtractFFmpegBin covered
Converters 4 of 13 tested 13 of 13, including null / wrong-type / ConverterParameter branches
VideoPlayerController transport Play/Next/Previous/GoToClip never called covered, plus the open-failure branch and two recovery give-up paths
Deleting the clip that's playing deadlocked on Dispatcher.Invoke, so all 5 delete tests used a null controller uiInvoker seam; the real path is now tested
Scan empty/error states, keyboard transport unreachable covered (no seam needed — the existing clipLoader was enough)
Mp4DurationReader box parsing happy path only truncated moov, size 0 / size 1 forms, largesize nesting
Culture invariance invisible to every test new CultureInvarianceTests

That last one is worth calling out: every test runs under the machine's own culture, where invariant and current formats coincide, so no test could detect a dropped InvariantCulture argument. Dropping one silently returns zero clips for ar-SA users (UmAlQura rejects the filename outright) or mis-dates every clip by 543 years under th-TH.

Fixtures and reliability

  • TestClipFiles wrote uniform 60s chunks spaced 60s apart. Probed duration and nominal spacing were therefore indistinguishable in every timeline calculation — the running sum, the tail computation, and gap detection were mutually interchangeable. It now takes per-chunk durations, so chunk-offset arithmetic is observable.
  • FakeClipMediaSourceBuilder exposed raw Lists that tests indexed while Build() appended from a pool thread. The bookkeeping is now private behind its existing lock and handed out as snapshots.
  • FfconcatMediaSourceBuilderTests no longer leaks a playlist file per test into %TEMP%, and CamStorageTests no longer recursively scans the build output directory.

Removed, merged, and fixed

Tests removed or merged for duplicating a sibling's exact code path, and one that asserted System.Text.Json's default behavior rather than ours. Several others passed for the wrong reason and were fixed — notably two ClipPlaylist tests whose central assertions already held before the act ran (MoveTo(0) on an empty playlist is a no-op, so CurrentIndex.ShouldBe(-1) was asserting the constructor).

The three seams

This PR is test-only with one deliberate exception: ClipExporter gains an optional runFfmpeg delegate, MainWindowViewModel gains an optional uiInvoker, and PackageManager.ExtractFFmpegBin goes privateinternal. The tests literally cannot compile without them, and each defaults to the existing behavior byte-for-byte, so the app is unchanged.

Deliberately deferred

  • CamEvent's two JSON paths disagree on a Z-suffixed timestamp — strict yields Kind=Utc, lenient yields host-local, and which path runs depends on an unrelated field. Latent (Tesla doesn't emit offsets today) but real.
  • ClipTimeline.GetPosition/ToAbsolutePosition/GetChunk and CamClip.Summary have no production callers — only tests keep them alive.
  • Splitting MainWindowViewModelTests.cs and VideoPlayerControllerTests.cs — better as a pure-movement change that doesn't collide with these additions.
  • .ffconcat files still leak into %TEMP% per run from two other test files. The durable fix is an internal PlaylistDirectory override; the app never prunes that directory either.

@danielchalmers danielchalmers changed the title Strengthen the test suite: 298 → 395 tests, and fix three bugs it found Strengthen test suite Aug 1, 2026
Reviewed the whole suite for coverage gaps, redundancy, brittleness, and
maintainability.
The suite was already in good shape (fast, deterministic, no external
dependencies), so this keeps those properties -- still ~3s -- and targets
the concentrated weaknesses.

Coverage gaps closed. Several surfaces were unreachable rather than merely
untested, mostly for want of a small seam the codebase already uses
elsewhere (clipLoader, backgroundYield, savePathPicker):

- ClipExporter.ExportAsync had zero tests. An optional runFfmpeg delegate
  lets its failure and script-cleanup paths run without spawning ffmpeg.
- PackageManager had no test file at all; ExtractFFmpegBin now has one.
- Nine of thirteen converters were untested, including their null and
  wrong-type branches.
- VideoPlayerController's transport commands were never called by any test.
- MainWindowViewModel's scan empty/error states and keyboard transport had
  no coverage, and deleting the clip that is actually playing deadlocked on
  Dispatcher.Invoke, so RunOnUiThread got a uiInvoker seam.
- Mp4DurationReader's box-parsing branches.
- New CultureInvarianceTests pins the four load-bearing InvariantCulture
  arguments. Dropping one silently returns zero clips under ar-SA or
  mis-dates every clip by 543 years under th-TH, and no existing test could
  see it, because every test runs under the machine's own culture.

Fixture and reliability work:

- TestClipFiles wrote 60s chunks spaced 60s apart, which made probed
  duration and nominal spacing indistinguishable in every timeline
  calculation. It now takes per-chunk durations.
- FakeClipMediaSourceBuilder exposed raw Lists that tests read while Build()
  appended off-thread; the bookkeeping is now private behind its lock.
- FfconcatMediaSourceBuilderTests no longer leaks a playlist file per test
  into %TEMP%, and CamStorageTests no longer scans the build output
  directory.

Removed or merged the tests that duplicated a sibling's code path or
asserted framework behavior, and fixed several that passed for the wrong
reason -- notably two ClipPlaylist tests whose central assertions already
held before the act ran.

The three production bug fixes this review turned up, and the CI changes,
are in a follow-up PR so this one stays test-only. The three seams above
are the exception: the tests cannot compile without them, and each defaults
to the existing behavior byte-for-byte.
@danielchalmers
danielchalmers force-pushed the test-suite-improvements branch from aa3b15e to efd5739 Compare August 1, 2026 21:49
@danielchalmers danielchalmers changed the title Strengthen test suite Strengthen the test suite: 298 → 365 tests Aug 1, 2026
@danielchalmers
danielchalmers merged commit d8d3a27 into main Aug 1, 2026
1 check passed
@danielchalmers
danielchalmers deleted the test-suite-improvements branch August 1, 2026 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant