Party/action highlight detection (saliency candidate source)

[nmbrthirteen]

Stacked on #43.

What

Auto-cut highlight clips from footage with no useful transcript (party videos, action). New podcli process --profile party|action picks moments by peak-picking a fused laughter+energy curve instead of reading the transcript.

How

• backend/services/saliency.py — the fusion engine:
  • per-video robust-z normalization (median/MAD, never global) so different recordings compare fairly
  • weighted channel fusion, renormalized over available channels
  • numpy peak-pick with min-gap NMS (no scipy dep)
  • reaction dilation so a single-frame laugh isn't suppressed by a louder energy neighbor
  • reaction-first candidate generation — a detected laugh/cheer is a primary candidate, energy peaks fill the rest
  • 8s backward expansion to capture the setup before a laugh (the funny thing happens before people react)
  • quiet-point boundary snapping
• profiles.py candidate source routes selection: saliency (party/action) vs llm (podcast). Wired into the process pipeline Step 3 and the --profile flag.
• Saliency clips use the render-compatible dict shape, so they flow through the existing review + render fan-out.

Validated

• Deterministic on clean input (identical output across runs).
• Reactions detected and labeled with correct run-up windows; energy peaks supplement.
• Found a real laugh in a synthetic multi-clip video and expanded backwards to the setup.
• Full suite: 375 passed (+24 new saliency/event tests). Same 3 pre-existing test_ai_fallback env failures.

Honest caveats

• Thresholds and channel weights are starting points — they need tuning on real party footage. A synthetic concat of podcast clips is not representative (nearly all speech, one faint chuckle).
• This increment still runs transcription for party videos (the process Step 1 is mandatory). Skipping transcription for saliency profiles on long videos is a follow-up optimization.
• TS-side profile threading (MCP/web UI) is not in this PR — CLI only for now.

See plans/moment-detection.md for the full design and phasing.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a86b686a-3be4-47c0-aff1-b4f17766dc54

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/moment-detection-phase2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}