chore: sync with getopenscreen/openscreen upstream (v1.6.0) - #1
Merged
Conversation
Decode mono 16k audio from the editor video, run Whisper via Transformers.js, and insert linked text annotations with timing and layout helpers. Adds Vite resolve shims for Node-only imports used by the model stack, optional leading-silence trim for the caption buffer, timeline gap reconciliation for auto-caption regions, and editor i18n. Raises Select content z-index so caption controls stay usable over the video surface. Co-authored-by: Cursor <cursoragent@cursor.com>
- Gate trimSec segment shift when transcription used full buffer retry - Restore UTF-8 autoCaptions strings (ar, es, fr, ja-JP, ko-KR, tr, zh-CN, zh-TW) - Dedupe caption segments after grouping only; stricter chunk dedupe in transcribe - Post-truncate duration, explicit consume merge for web demuxer, trim region shift cleanup - Tests for caption annotation pipeline Co-authored-by: Cursor <cursoragent@cursor.com>
Phrase mode with one merged span now splits wrapCaptionTextByWordBounds lines into separate CaptionSegments with even time allocation across the phrase span (fallback when duration is too short for min spans). Update unit test accordingly. Co-authored-by: Cursor <cursoragent@cursor.com>
Merge siddharthvaddem/openscreen main and resolve conflicts in VideoEditor.tsx (export diagnostics, save-diagnostic handler, auto captions, toolbar chrome) and package-lock.json. Apply Biome formatting after merge so hooks pass. Co-authored-by: Cursor <cursoragent@cursor.com>
Use one Sonner id for the full caption flow, show a distinct transcribing step, yield before Whisper so updates paint, match editor dark chrome on toasts, and keep pointer-events only on toast bodies. Add transcribing strings across locales. Co-authored-by: Cursor <cursoragent@cursor.com>
Scope withoutNodeVersion to Transformers import/pipeline only; reapply trim-region filtering after ignore-trims retries; check AbortSignal after each slice inference before chunk processing. Co-authored-by: Cursor <cursoragent@cursor.com>
Adds `projectFolder` as a key to user preferences, stores the location of the most recently opened project, and prefills it in the next File → Open action. Mirrors the pattern from siddharthvaddem#512 (`exportFolder`). Closes siddharthvaddem#668
Resolves conflicts in userPreferences.ts and userPreferences.test.ts by combining the projectFolder feature with upstream's trayLayout addition.
…iptions # Conflicts: # src/i18n/locales/ar/editor.json # src/i18n/locales/en/editor.json # src/i18n/locales/es/editor.json # src/i18n/locales/fr/editor.json # src/i18n/locales/ja-JP/editor.json # src/i18n/locales/ko-KR/editor.json # src/i18n/locales/tr/editor.json # src/i18n/locales/zh-CN/editor.json # src/i18n/locales/zh-TW/editor.json
Documents the cursor helper binary, permissions, manual test checklist, expected sidecar shape, and known limitations for the darwin native cursor path.
- Unify sidecar key name to camelCase cursorRecordingData (was mixing cursor-recording-data and cursorRecordingData in the same checklist) - Graceful-degradation test now removes both binary copies so the missing-helper fallback is actually exercised (build/ and bin/darwin-*) - Healthy recording section points to <videoPath>.cursor.json sidecar instead of the .openscreen project file, which does not embed cursor data
Resolved conflicts: - LaunchWindow.tsx: kept upstream's removal of openVideoFile/openProjectFile (moved to Studio Dashboard) - VideoEditor.tsx: used upstream's doLoadProject rename while preserving getProjectFolder() arg
The Studio Dashboard's Load Project button was calling loadProjectFile() without the remembered folder, always defaulting to RECORDINGS_DIR. Also save projectFolder preference after a successful load from this path.
…mirror-toggle Add webcam mirror toggle
…selector-test-hooks test: expose source selector hooks
…e-cursor docs: add macOS native cursor capture test pipeline
…imeline Move model load and transcription off the renderer main thread into a dedicated worker (transcribe.worker.ts + transcribeCore.ts) so the editor UI no longer freezes during captioning. Relocate the auto-captions action from the editor header into the timeline toolbar, and add the missing it/pt-BR/ru/vi autoCaptions translations.
…ranscriptions Feature/video transcriptions
…lemetry fix: resolve cursor-sampler telemetry not working
…mber-last-project-folder feat: Add projectFolder to user preferences
Updated package versions to 1.6.0-rc.1 and integrated Tiptap for the NotesWindow component, replacing the previous textarea with a rich text editor. Improved local storage handling for notes and added styles for better UI. Enhanced CSS to hide scrollbars for a cleaner appearance.
Modified the minimum width of the Electron notes window for improved usability. Removed unused CSS styles from NotesWindow.module.css and updated the NotesWindow component to utilize Tailwind CSS for layout and styling, enhancing the overall user interface.
Updated the NotesToolbar component to include tooltips for each formatting button, improving user experience. Integrated localization for toolbar button labels using the useScopedT hook, ensuring accessibility for multiple languages. Added corresponding translations for toolbar actions in various locale files.
Simplified the notes content escaping logic in the NotesWindow component by formatting the code for better readability. This change enhances maintainability while ensuring that notes saved as plain text are correctly wrapped for Tiptap parsing.
- Restore demo screenshot height to 320px (0.2467 made images invisible) - Align macOS permission instructions with System Settings naming - Reuse settings.imageUpload.errorReading in AnnotationSettingsPanel
annotation.failedImageUpload is identical (modulo one Arabic preposition) to imageUpload.failedToUpload. Use the latter so the upload error toast in AnnotationSettingsPanel reuses the same namespace as SettingsPanel. # Conflicts: # src/components/video-editor/AnnotationSettingsPanel.tsx
prerelease.yml: the version bump now lives on release/vX.Y.Z-rc.N only and is NOT merged into main. The RC tag points at the release branch tip, so anything merged into main after the RC cut stays out of the build. promote.yml: checks out the frozen release branch, bumps to the stable version, tags its tip, publishes the release, then opens a release-sync PR into main so the released snapshot eventually lands on main without polluting it during QA. Fixes the v1.6.0 incident where 23 commits (Tiptap, NotesWindow, lint button, AI handoff) landed on main between RC cut (5d7248c) and the stable promote, so the published v1.6.0 stable included untested features.
- git-workflow.md: rewrite the § Release flow section to describe the release/vX.Y.Z-rc.N freeze contract, cherry-pick rules during the RC window, and the post-promote release-sync PR. Add a § Release branches section that lists the three branches per version (rc, sync, stable) and the rules each one follows. - AGENTS.md § Release flow: one-paragraph pointer to the freeze contract. - MEMORY.md: log the v1.6.0 incident + the fix so future agents don't re-introduce the 'tag main instead of release branch' bug.
…arthvaddem#45) The native Windows pause/resume branches added in ca826d9 only paused the screen capture. The webcam MediaRecorder kept recording during a pause, so the editor saw a gap in the screen track and continuous webcam footage through the pause window. Mirror the native macOS pattern: after a successful native pause/resume, call MediaRecorder.pause()/.resume() on the webcam recorder if it's in the matching state. The browser path and the native macOS path already did this correctly.
Exporting a recording larger than ~2 GiB failed with "Failed to read binary file". The renderer loaded the whole source into memory via the read-binary-file IPC (Node `fs.readFile`), which throws ERR_FS_FILE_TOO_LARGE above 2 GiB, so any long recording (e.g. a 2h 1080p60 capture at ~6.6 GB) could never be exported. Even under the cap, a multi-GB ArrayBuffer/Blob would exhaust memory on a typical machine. web-demuxer reads a File on demand, so it never needs the bytes up front. Add a chunked range-read IPC (`get-readable-file-info` + `read-file-chunk`) and a renderer helper that streams large recordings into an OPFS-backed File, handing web-demuxer a disk-backed File instead of an in-memory one. Memory stays flat regardless of length. Small recordings keep the existing single-shot path. Wire the streaming loader into the export decoder and the captions audio extractor; skip the in-memory source-copy fast path and the waveform ArrayBuffer read for oversized files so they degrade gracefully. Verified against a real 6.6 GB / 2h11m recording: OPFS copy peaked at ~137 MB heap, web-demuxer parsed metadata and decoded 1920x1080 frames with heap staying ~27 MB. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review feedback on the >2GB export fix: - OPFS cache pruning now reference-counts live sources and never deletes an entry a demuxer is still reading. releaseLocalSourceFile() is called from StreamingVideoDecoder.destroy() and the captions extractor's finally. Previously pruning kept only the newest entry, which could remove a file still in use by a concurrent export/caption pass. - Extract the shared in-memory size threshold into sourceFileLimits.ts (was duplicated across localSourceFile, extractMono16k, videoExporter). - Report OPFS copy progress as a "preparing" export phase so the dialog reflects the multi-GB copy instead of sitting at 0%. - Add unit coverage for the OPFS streaming/eviction path: chunked copy, cache reuse, and the in-use pruning guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address the follow-up review on the OPFS cache guard: - Reference-count cache entries by cache-entry name (the returned File's .name) instead of by source URL. Keying by URL discarded an older revision's refcount when a new revision (size/mtime change) superseded it, so releasing the old revision could decrement the wrong entry and prematurely drop a still-in-use copy. Callers now release with the File's name (StreamingVideoDecoder.destroy and the captions extractor). - Add a test that a failed chunk read mid-copy throws and removes the partial cache entry, exercising the catch-block cleanup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 1.5 GiB threshold was too high. read-binary-file returns the whole file over IPC, which Electron structured-clones (copies) in the main process, so a ~1 GB recording transiently needs ~2x its size there and hard-crashes a memory-constrained machine (observed on a 16 GB Mac) — well below the 2 GiB fs.readFile cap. Two paths hit this: - Export/demux source load and captions now stream via OPFS above 256 MiB (lowered MAX_IN_MEMORY_SOURCE_BYTES), so moderate recordings no longer ship a giant buffer over IPC. - The trim waveform (loadFileAsArrayBuffer) cannot stream — decodeAudioData needs the whole file — so it now skips recordings above the limit; useAudioPeaks already degrades to no waveform on throw. Fixes an editor-entry crash when opening a ~1 GB recording. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review feedback from siddharthvaddem#74: - Fix the concurrent-prune race: retain the OPFS cache entry BEFORE pruning/writing, so a concurrent materialization's prune pass can no longer delete an entry that is still mid-write. On failure the reference is released and the partial copy removed only if no other in-flight operation still references it. Regression test included. - Cap read-file-chunk requests at 64 MiB on the main-process side so a buggy or compromised renderer cannot force an arbitrarily large Buffer.allocUnsafe. - Infer the small-file MIME type from the extension (mp4/webm/mov/...) instead of hardcoding video/mp4. - Restore the trim waveform for large recordings instead of skipping it: new streaming peaks path demuxes the audio track and folds each decoded AudioData frame straight into min/max buckets (same output format as audioPeaksWorker), so memory stays flat. Verified on a 288 MB file: all 24000 blocks populated, renderer heap 25→27 MB. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… paths Findings from an adversarial multi-agent cross-review of this branch: - Cancelling an export during the "preparing" phase used to leak the OPFS cache reference forever (destroy() ran before sourceCacheName was set) while the multi-GB copy kept streaming in the background. Copies are now abortable: materializeLocalSourceFile takes an AbortSignal, the decoder's cancel()/destroy() and the load timeout abort it, and references are only taken when a caller actually receives the File — no success, no retain, nothing to leak. GifExporter benefits via the same decoder hook. - Concurrent materializations of the same recording (waveform + export) used to race two writables on one OPFS handle and could invalidate the File another consumer was reading. In-flight copies are now deduplicated per cache entry: joiners share one stream (and its progress events), and the underlying copy aborts only when every joined caller has aborted. - The streaming waveform now falls back to a demux-only packet-timestamp scan when the container duration is missing/bogus (MediaRecorder WebM), and always closes its AudioDecoder on error/abort paths. - The caption demuxer path buffers decoded PCM in memory, so oversized sources now cap decoded audio at 30 min and surface `truncated` instead of exhausting the renderer heap on multi-hour recordings. - Stale multi-GB OPFS copies from previous sessions are reclaimed at app startup (previously only pruned during the next large-file load). Tests: in-flight dedup, mid-copy abort + cleanup, and shared-copy survival when one of two joined callers aborts (307 total passing). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Hoist WebmElementMock interface to describe scope in webm-duration.test.ts - Wrap ws.write inside try/finally to ensure streams are always destroyed on failure - Disable pause/restart/cancel HUD buttons while saving to prevent race conditions - Skip setSaving(true) for cancelled/restarted recordings (discardRecordingId guard) - Extract hudDisabledClasses constant to reduce repetition in LaunchWindow.tsx
After cropping, the exported cursor drifted because the recorded sample was projected onto the screen mask rect instead of the rectangle the cropped video is actually painted on. In cover layouts where the cropped video letterboxes inside screenRect (coverOffset != 0), the cursor offset was proportional to the letterbox margin. Track the painted croppedRect in FrameRenderer layoutCache and pass it to projectNativeCursorToLocal so the cursor overlays the cropped video pixels exactly. Adds regression tests for projectNativeCursorToLocal. Fixes siddharthvaddem#64
- Rename cover-letterboxed test to cover-overflowing (fixture is cover, not letterbox) - Update sizeNorm comment to acknowledge the export/preview asymmetry introduced by the croppedRect field: export now uses croppedRect.width, preview still uses screenRect.width. They agree in cover mode but differ in fit-to-height letterbox layouts.
The Windows cursor-sampler (Win32 GetCursorInfo) reports raw x/y in physical screen pixels. For display captures, normalizeSample was dividing by bounds from Electron's \screen\ API, which are in logical pixels (DIP). On a 100% DPI display these coincide and the bug is invisible, but on any high-DPI display (125%, 150%, 200%) the normalized cursor position is wrong by the scale factor. The error then compounds through projectNativeCursorToLocal in the export, producing a visible cursor offset proportional to the DPI ratio. Fix by converting the logical bounds to physical via the display's scaleFactor before normalizing. payload.bounds (from the sampler's GetWindowRect, used for window captures) is already physical and is left as-is. Pre-existing since 1.5.0; never caught because CI is Linux-only and manual Windows smoke tests ran on 100% DPI displays.
The previous fix multiplied bounds.x/y/width/height by a single
scaleFactor to convert from DIPs to physical pixels. That works for the
primary display (origin at 0,0 in the virtual screen) but misplaces the
origin on non-primary or mixed-DPI displays: a secondary 200% DPI
monitor to the right of a 100% primary has DIP bounds {x:1920, y:0,
w:1920, h:1080} but physical bounds {x:1920, y:0, w:3840, h:2160} —
multiplying the origin by 2 would push it to 3840.
Use Electron's \screen.dipToScreenRect(null, bounds)\ instead, which
picks the correct display from the rect's center and handles the
virtual-screen origin correctly across multi-monitor and mixed-DPI
setups. payload.bounds (from the sampler's GetWindowRect) is already
physical and is left as-is.
m8i-51
marked this pull request as ready for review
July 11, 2026 06:39
cursor
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
getopenscreen/openscreen（旧EtienneLescot/openscreen）を upstream として追加し、コミュニティ継続版の最新main（201 コミット、v1.6.0 相当）を fast-forward で取り込みました。Changes
upstreamリモートをhttps://github.com/getopenscreen/openscreen.gitに設定（push は無効化）upstream/mainを fast-forward マージ（競合なし）Notable upstream additions
Verification
npm cinpx tsc --noEmitnpm test— 41 files, 315 tests passed