Update developer guide with v0.12 changes

Document the new behaviors introduced across v0.12.0 and v0.12.1:

- Frame extractor: per-stream tuning (OBJECT/SS threshold 0.15 + 30s
  periodic sampling), PAGE_SIMILARITY_THRESHOLD=35, junk filter with
  _JUNK_DESC_RE + _is_blank_frame pre-check, contiguous renumbering,
  vision-API description cache.
- Note generation: screenshare img_hints include vision cache (fixes
  caption-image mismatch), render_chunk_images refreshes stale copies on
  mtime/size mismatch, MAX_NOTE_CHARS=120000 replaces per-slide cap,
  ThreadPoolExecutor with PARALLEL_SECTIONS=6, translation prompt keeps
  technical terms in English.
- Downloader: stream priority OBJECT > SS > DV.
- Alignment parser: Whisper dot-hallucination filter in _clean_transcript.
- pipeline_worker.py: what it does and why it must be in extraResources
  (v0.12.1 fix for Windows).
- Electron packaging: nsis only on Windows, explicit no-sign on macOS,
  extraResources must match main.js:SCRIPTS.
- NSIS uninstaller: current MessageBox-inside-customUnInstall design plus
  a list of approaches that did not work (save future iterations).
- benchmark.py: three metrics, weighting, and known limitations.
- Data storage table extended with new files (.language marker, image_cache,
  frames dir, .zh.md / .en.md note variants).

Author: nodeeeeee

arch_design.md

@@ -16,10 +16,12 @@ Pipeline Scripts (Python, executed as subprocesses)
     │
     ├── downloader.py            Canvas + Panopto download
     ├── extract_caption.py       Whisper transcription
-    ├── frame_extractor.py       Screen-share frame extraction + dedup
+    ├── frame_extractor.py       Screen-share frame extraction + dedup + junk filter
     ├── semantic_alignment.py    Transcript ↔ slide alignment (FAISS + Viterbi)
     ├── alignment_parser.py      Compact alignment JSON for LLM prompts
-    └── note_generation.py       LLM-based note writing + image filtering
+    ├── pipeline_worker.py       Orchestrates transcribe + frame-extract + align
+    ├── note_generation.py       LLM-based note writing + image filtering
+    └── benchmark.py             Quality evaluation (coverage / image / coherency)
     │
     ▼
 ML Environment (~/.auto_note/venv/)
@@ -39,10 +41,12 @@ materials/  (PDF, PPTX)    videos/  (MP4)
     │               Camera?      Screen share?
     │                    │            │
     │                    ▼            ▼
-    │              captions/    frames/ + captions/
+    │              captions/    frames/<stem>/
     │               (Whisper)    (ffmpeg scene detect
+    │                            + periodic sampling
     │                            + perceptual hash dedup
-    │                            + info-score selection)
+    │                            + junk filter via vision API)
+    │                            + image_cache.json (descriptions)
     │                    │            │
     └────────┬───────────┘            │
              │                        │
@@ -78,11 +82,18 @@ materials/  (PDF, PPTX)    videos/  (MP4)
 
 **Dashboard detail modal**: Clicking a course card opens a modal overlay that lists all transcribed videos with their processing status (caption/alignment/notes). Each video has a delete button that removes the transcript, alignment, note sections, and per-video note files via the `course:deleteVideo` IPC handler.
 
+**Packaging** (`electron/package.json`):
+- **Windows**: NSIS installer only (`target: ["nsis"]`). The `appx` target was removed — it requires Microsoft Store signing credentials that aren't available in CI.
+- **macOS**: Universal DMG (x64 + arm64) with signing explicitly disabled (`identity: null`, `hardenedRuntime: false`, `gatekeeperAssess: false`, `dmg.sign: false`). Without these, users see a "AutoNote.app is damaged and can't be opened" quarantine error. Users still need to right-click → Open on first launch (standard for unsigned apps).
+- **Linux**: AppImage + deb.
+- **`extraResources`** must list every Python script referenced by `main.js:SCRIPTS` or the renderer. Current list: `downloader.py`, `extract_caption.py`, `frame_extractor.py`, `semantic_alignment.py`, `alignment_parser.py`, `note_generation.py`, `pipeline_worker.py`. Missing a script from this list leaves it out of `resources/scripts/` in the packaged app and causes "No such file or directory" errors on users' machines.
+
 ### Pipeline Scripts
 
 #### downloader.py
 - Downloads videos from Panopto and materials from Canvas
-- Tracks download state in `manifest.json` and `download_log.json`
+- **Stream priority**: OBJECT > SS > DV > untagged. OBJECT streams are screen recordings and preferred over camera (DV) when both are available for a lecture.
+- Tracks download state in `manifest.json` and `download_log.json`. Each entry records `stream_tag` (used downstream by `frame_extractor`).
 - Slack mode adds random delays to avoid rate-limiting
 - Smart size filter uses LLM to select relevant files when > 1 GB
 
@@ -91,12 +102,25 @@ materials/  (PDF, PPTX)    videos/  (MP4)
 - Produces timestamped segment-level JSON
 - `--force` flag re-transcribes even if captions already exist
 - Language detection probes from mid-audio for accuracy
+- Whisper hallucination filter: segments whose text is only `. . . . .` (common during silent audio from OBJECT-stream intro/loading screens) are dropped downstream in `alignment_parser._clean_transcript`.
 
 #### frame_extractor.py
-- Classifies videos as screen-share or camera using edge/uniformity heuristics
-- Scene detection via ffmpeg scene filter + periodic sampling fallback
-- **Same-page deduplication**: Groups consecutive frames by perceptual hash similarity (dHash, Hamming distance < 45 bits). From each group, selects the frame with the highest visual information score (edge density on 160x120 grayscale). This ensures incremental bullet reveals keep only the most complete version.
-- Builds timestamp-based alignment JSON compatible with the rest of the pipeline
+- Classifies videos as screen-share or camera using edge/uniformity heuristics. Screen recordings have sharp edges, high brightness, and large uniform regions.
+- **Per-stream extraction tuning** (`detect_scenes(stream_tag=...)`):
+  - For `OBJECT`/`SS` streams (stable slide recordings with subtle text-only changes): scene threshold lowered to 0.15 and periodic samples taken every 30s unconditionally. Without periodic sampling, 2-hour OBJECT streams yield only ~10 frames.
+  - Other streams: default 0.3 threshold, periodic sampling only as fallback when < 5 scene changes detected.
+- **Same-page deduplication**: Groups consecutive frames by perceptual hash similarity (dHash, 16×16 = 256-bit hash, Hamming distance < `PAGE_SIMILARITY_THRESHOLD = 35` bits). Merges incremental bullet reveals while keeping genuinely different pages. From each group, selects the frame with the highest visual information score (edge density on 160x120 grayscale).
+- **Junk-frame filter** (`_JUNK_DESC_RE` + `_is_blank_frame`):
+  - Pre-vision: pure-black or pure-white frames (>95% of samples <15 or >240 in grayscale) are skipped before calling the vision API.
+  - Post-vision: frames whose vision description matches `_JUNK_DESC_RE` (desktop wallpaper, taskbar, Windows 11, loading screens, vision-API refusals, memes, XKCD, four-panel comics) are deleted. Remaining frames are contiguously renumbered on disk and in the alignment so they stay in sync.
+- **Vision API descriptions**: After extraction, `_describe_frames` calls GPT-4o-mini (via `semantic_alignment.ImageDescriber`) to describe each frame. Descriptions are cached in `frames/<stem>/image_cache.json` keyed by `page_N` (0-indexed matching frame_{N+1:03d}.png).
+- Builds timestamp-based alignment JSON compatible with the rest of the pipeline. The `source` field is `"screenshare"` for frame-based alignments and `"slides"` for PDF-based ones.
+
+#### pipeline_worker.py
+- Single subprocess that orchestrates `extract_caption.py` + `frame_extractor.py` + `semantic_alignment.py` for every video in a course.
+- Invoked by the Electron app (`main.js:SCRIPTS.pipeline_worker`) for the "Transcribe + Align" button, letting one progress bar cover the whole pipeline.
+- `_script(name)` resolves script paths by looking in `~/.auto_note/scripts/` first (production install), then the script's own directory (development). This matches how the Electron app syncs packaged `resources/scripts/*.py` into `~/.auto_note/scripts/` on first launch.
+- Must be listed in `electron/package.json:build.extraResources` or the packaged Windows/macOS installer will be missing it.
 
 #### semantic_alignment.py
 - Extracts text from slides (PDF/PPTX/DOCX) with image enrichment for sparse slides
@@ -110,24 +134,38 @@ materials/  (PDF, PPTX)    videos/  (MP4)
 - Viterbi temporal smoothing with forward bias and temporal position prior
 - Off-slide detection for Q&A/demo segments (cosine < threshold)
 - `--force` flag re-aligns even if alignment files already exist
+- `ImageDescriber` is reused by `frame_extractor` to describe screen-capture frames.
 
 #### alignment_parser.py
 - Compresses full alignment JSON (300 KB) into compact per-slide format (30 KB)
-- Cleans filler words from transcripts
+- `_clean_transcript` strips Whisper's dot-hallucination segments (`re.fullmatch(r"[\s.]+", text)`) and common ASR filler words.
 - Used by note_generation to build token-efficient LLM prompts
 
 #### note_generation.py
-- Multi-provider LLM support: OpenAI, Anthropic, Google Gemini, DeepSeek, xAI, Mistral
-- **Language system**: `--language en|zh` CLI flag overrides the `NOTE_LANGUAGE` constant. The `_P(key)` function selects from `_PROMPTS["en"]` or `_PROMPTS["zh"]` dictionaries containing complete prompt sets (system, chunk, slide_only, verify, exam, detail_instructions). Language is selectable per-run from the Pipeline and Generate page dropdowns.
-- Per-lecture chunking: CHAPTER_SIZE slides per LLM call
-- Section caching: each chunk saved as `L{N}_S{ci}.md` for resume support
+- Multi-provider LLM support: OpenAI, Anthropic, Google Gemini, DeepSeek, xAI, Mistral, Claude CLI (`claude -p`)
+- **Language system**: `--language en|zh` CLI flag overrides the `NOTE_LANGUAGE` constant. `_P(key)` always returns English prompts; Chinese output is produced by a separate post-generation `_translate()` call per section.
+- **Translation prompt preserves technical terms in English** when target is Chinese. The prompt lists protected term categories (protocols, crypto, algorithms, proper nouns, code identifiers) and instructs the LLM to translate only connecting prose. Example output: `symmetric key cryptography 方案在 encryption 和 decryption 时使用同一个 key。` This matches how students study in English-taught courses — the Chinese provides narrative, the English terms remain exam-ready.
+- **Per-lecture chunking**: CHAPTER_SIZE slides per LLM call. `MAX_NOTE_CHARS = 120000` is the total prompt-char cap (transcript + slide outlines + image hints) — replaces the older per-slide `MAX_TRANSCRIPT_CHARS` limit that was truncating content too aggressively.
+- **Parallel section generation**: `ThreadPoolExecutor(max_workers=PARALLEL_SECTIONS = 6)` fans out chunk generation + translation concurrently. Each chunk returns `(ci, (content, fresh))` — make sure to unpack as `ci_ret, (content, fresh) = fut.result()`.
+- **Section caching**: each chunk saved as `L{N}_S{ci}.md`. `notes/sections/.language` marker triggers auto-force-regen when the language changes.
+- **Image hints for screen-share lectures** (`make_chunk_prompt`, line 819): now include the cached vision description (`img_cache["page_N"]`) instead of only the transcript context. Without this the LLM invents captions for frames based on what it's writing about, causing captions that don't match the displayed image.
+- **Image rendering** (`LectureData.render_chunk_images`, line 1222): copies source frames from `frames/<stem>/frame_NNN.png` into `notes/images/L{NN}/`. **Refreshes stale copies** when source mtime or size differs from the destination — important after re-extraction, or the displayed image will be an outdated frame with different content than the caption describes.
 - `--force` flag re-generates all sections from scratch
 - Image filtering: multi-step decision pipeline (cache description keywords → title pattern → vision API). Includes all slides with visual elements; only excludes administrative/non-course elements.
 - Self-scoring: coverage, terminology, callouts, code blocks (weighted average)
 - Per-video mode (`--per-video`): one note file per lecture instead of merged
 - Iterative mode: raises detail level until quality target is reached
 - All terminal output (print/tqdm.write) is in English regardless of note language
 
+#### benchmark.py
+- Stand-alone quality evaluator for generated notes. Usage: `python benchmark.py --course ID [--verbose]` or `--note PATH --transcript PATH --image-cache PATH`.
+- **Three metrics, each scored 0-10**:
+  - **Content coverage** (`content_coverage`): extracts key terms (capitalized phrases, acronyms, hyphenated terms appearing ≥2 times) from the transcript; scores the % of those terms that appear in the note.
+  - **Image density** (`image_density`): ratio of images inserted in the note vs. number of content-rich images available in `image_cache.json` (filtered via `_CONTENT_KEYWORDS` to exclude loading-screen/desktop/blank descriptions).
+  - **Logic coherency** (`logic_coherency`): structural checks — sections headings, long image clusters (≥3 consecutive images without intervening prose; pairs are allowed), orphan images (no preceding paragraph within 6 lines), mid-sentence truncation, leaked artifacts (APPROVED, © CS, LLM refusals).
+- **Overall** = `0.4 × coverage + 0.3 × image_density + 0.3 × coherency`.
+- Limitations: coverage extractor matches English terms only, so Chinese notes score lower on coverage even when content is complete. Image density returns 10.0 when no cache exists (division-by-zero fallback for slide-based notes).
+
 ### Force Regenerate Behavior
 
 The "Force regenerate" toggle applies to whichever pipeline steps are selected:
@@ -144,13 +182,39 @@ Without force, the pipeline is incremental: only missing files are processed.
 
 Images pass through multiple filtering layers before appearing in the final notes:
 
-1. **Image hints generation**: Slides with word_count < 80, cached descriptions, or code are offered to the LLM as available images
-2. **LLM prompt instructions**: System prompt instructs to insert all slides with visual elements (diagrams, charts, code, math, etc.) and skip pure text or administrative slides
-3. **Post-generation filter** (`filter_images_pass`):
+1. **Frame extraction** (`frame_extractor.py`):
+   - `_is_blank_frame` pre-vision skip for all-black / all-white frames
+   - `_JUNK_DESC_RE` post-vision filter for desktop wallpaper, loading screens, memes, XKCD, vision-API refusals
+2. **Image hints generation**: Slides with word_count < 80, cached descriptions, or code are offered to the LLM as available images. Screen-share frames always pass through their `image_cache` description so the LLM caption matches the actual frame.
+3. **LLM prompt instructions**: System prompt instructs to insert all slides with visual elements (diagrams, charts, code, math, etc.) and skip pure text or administrative slides
+4. **Post-generation filter** (`filter_images_pass`):
    - Screen-share frames: always kept
    - Cache-verified visual description: kept
    - Title/divider pattern: removed
    - Vision API (GPT-4o-mini): decides uncertain cases; defaults to keep
+5. **Image rendering** (`render_chunk_images`): always refreshes stale copies in `notes/images/L{NN}/` on mtime/size mismatch so the final `.md` never points at stale content.
+
+### NSIS Uninstaller (Windows)
+
+`electron/build/uninstaller.nsh` lets users choose what to keep on uninstall via three sequential `MessageBox` prompts, all inside the `customUnInstall` macro:
+
+1. Keep generated notes and downloaded course files in `%USERPROFILE%\AutoNote`? (default: No)
+2. Keep ML environment (~2 GB) in `%USERPROFILE%\.auto_note\venv`? (default: No)
+3. Keep settings and API keys in `%USERPROFILE%\.auto_note`? (default: No)
+
+Each MessageBox uses the NSIS label-jump idiom: `IDYES keep_label` → skip the following delete commands and jump past them. This keeps the implementation free of `Var` declarations, `${If}/${EndIf}` macros, and `LogicLib` dependencies — all of which turned out to break electron-builder's multi-pass NSIS compile.
+
+Three pragmas suppress warnings that fire harmlessly in one of the two compile passes but would be escalated to errors by CI:
+
+- `!pragma warning disable 6010` — "un.* function not referenced"
+- `!pragma warning disable 6020` — "uninstaller script code but no WriteUninstaller"
+- `!pragma warning disable 8000` — "Uninstall page instfiles not used"
+
+**Things that did not work** during iteration (kept here to save future attempts):
+
+- **Custom `UninstPage` via `!macro customUnInstallPage`**: that macro name is not a real electron-builder hook, so the `un.` function is never referenced and NSIS zeros it out with warning 6010.
+- **`UninstPage custom ...` inside `!macro customHeader`**: valid NSIS location but triggers warning 8000 because it overrides MUI2's default `MUI_UNPAGE_INSTFILES` without a replacement.
+- **`customUnInit` + `Var` + `${If}`**: compiled but failed in one of electron-builder's passes (exact error unavailable without admin log access). Moving the prompts into `customUnInstall` directly succeeded.
 
 ## Data Storage
 
@@ -160,15 +224,20 @@ Images pass through multiple filtering layers before appearing in the final note
 |------|----------|---------|
 | `config.json` | `~/.auto_note/` | Canvas URL, Panopto host, output dir |
 | `*_api.txt` / `*_token.txt` | `~/.auto_note/` | API keys and tokens |
-| `manifest.json` | Output dir root | Video download state tracking |
+| `manifest.json` | `~/.auto_note/` | Video download state + `stream_tag` |
 | `download_log.json` | Per-course | Material download tracking |
 | `captions/*.json` | Per-course | Whisper transcript (timestamped segments) |
+| `frames/<stem>/frame_NNN.png` | Per-course | Extracted screen-share frames |
+| `frames/<stem>/image_cache.json` | Per-course | Vision-API descriptions per frame |
 | `alignment/*.json` | Per-course | Full segment-level alignment |
 | `alignment/*.compact.json` | Per-course | Token-efficient alignment for LLM |
 | `notes/sections/L*_S*.md` | Per-course | Cached per-chunk note sections |
-| `notes/*_notes.md` | Per-course | Final merged/per-video notes |
+| `notes/sections/.language` | Per-course | Current language marker (forces regen on change) |
+| `notes/*_notes.md` | Per-course | Final merged/per-video notes (English default) |
+| `notes/*_notes.zh.md` | Per-course | Chinese version (user renames after `--language zh`) |
+| `notes/*_notes.en.md` | Per-course | Optional English backup before running ZH regen |
 | `notes/*.score.json` | Per-course | Self-score breakdown |
-| `notes/images/L*/` | Per-course | Rendered slide PNGs |
+| `notes/images/L*/` | Per-course | Rendered slide PNGs / copied frames |
 
 ## Testing
 
@@ -181,6 +250,9 @@ Tests are in `test/` and organized by scope:
 | `test_language_and_skip.py` | Language selection, terminal output (no CJK in prints), skip logic with/without --force |
 | `test_note_generation.py` | Note generation specific tests |
 | `test_gui.py` | GUI-specific tests |
-| `electron/test/main.test.js` | Electron main process tests |
+| `electron/test/main.test.js` | Electron main process tests (26 cases) |
 
 Run all offline tests: `python -m pytest test/ -v -k "not integration"`
+Run Electron tests: `cd electron && npx jest test/`
+
+Ad-hoc quality check: `python benchmark.py --course <ID> --verbose` runs the three quality metrics against every note in a course.