Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 15 additions & 10 deletions electron-builder.json5
Original file line number Diff line number Diff line change
Expand Up @@ -185,16 +185,21 @@
"filter": ["win32-*/*", "!win32-*/ffmpeg.exe"]
}
],
// Native D3D11 compositor addon (Windows-only). Built by
// `npm run build:native:compositor` (scripts/build-windows-compositor-addon.mjs)
// before packaging. Packed into asar, then auto-unpacked by the top-level
// `asarUnpack: ["**/*.node"]` rule to app.asar.unpacked/... — matches the
// `.asar` -> `.asar.unpacked` rewrite compositorViewService.ts's
// buildCandidatePaths() already applies for packaged builds. Appends to
// (doesn't replace) the top-level `files` list.
"files": [
"electron/native/compositor-view/build"
]
// The D3D11 compositor addon deliberately does NOT travel through `files` into
// app.asar. It ships with the ffmpeg DLLs it links against, via the
// `extraResources` entry above, because it has to sit in the SAME directory as
// them — exactly like the Metal addon on macOS.
//
// It used to be listed here and unpacked to app.asar.unpacked/…, one directory
// away from `electron/native/bin/win32-x64/*.dll`. That only ever worked because
// `ensureFfmpegSharedDllsOnPath` prepends the DLL directory to PATH before the
// require. MSIX ignores PATH when resolving dependent DLLs — it resolves through
// the package graph — so the Store build of 1.9.0 loaded no compositor at all and
// the editor ran with no preview while audio kept playing.
//
// Node loads .node files with LOAD_WITH_ALTERED_SEARCH_PATH, so the addon's own
// directory is searched for its dependencies; colocation makes the load work with
// no PATH at all, on every Windows packaging format.
},
"nsis": {
"oneClick": false,
Expand Down
78 changes: 77 additions & 1 deletion scripts/before-pack.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,56 @@ function checkNativePayload({ dir, required, osLabel, bundleNoun, emptyDirFix })
);
}

/**
* The Windows addon must sit in the SAME directory as the ffmpeg DLLs it links
* against, because that is the only arrangement that loads under MSIX.
*
* The addon dlopens avcodec/avformat/avutil at require() time. While it shipped from
* app.asar.unpacked — one directory away from the DLLs — loading it depended on
* `ensureFfmpegSharedDllsOnPath` prepending their directory to PATH. That works for
* the NSIS installer and does not work under MSIX, which resolves dependent DLLs
* through the package graph and ignores PATH. Measured inside a registered package,
* with the directory correctly on PATH: `require` failed both before and after the
* PATH was set; with the addon beside its DLLs it loaded with no PATH at all.
*
* That shipped as 1.9.0 on the Store: no compositor loaded, so the editor showed no
* preview at all while audio kept playing — and it looked like an app bug, not a
* packaging one, because every file was present and the NSIS build of the same commit
* was fine.
*
* `win.extraResources` ships this directory wholesale (filter `win32-*​/*`), so
* "together here" is the same thing as "together in the installed app".
*/
const WIN_REQUIRED = [
{
match: (name) => name === "compositor_view.node",
what: "the D3D11 compositor addon",
breaks: "the preview renders nothing and every export falls back to the no-op compositor",
fix: FIX,
},
// One requirement per library, not `atLeast: 3` over a combined regex — the same
// trap LINUX_REQUIRED documents above. Several versioned copies of one library
// (avcodec-60/61/62.dll left by an earlier fetch) would satisfy a combined count
// while another library was missing entirely, and the addon would still fail to
// load.
...["avcodec", "avformat", "avutil"].map((library) => ({
match: (name) => new RegExp(`^${library}-\\d+\\.dll$`).test(name),
what: `the ${library} DLL the compositor links`,
breaks: "the addon cannot be loaded at all under MSIX, which ignores PATH",
fix: "Fetch them with:\n\n npm run fetch:ffmpeg",
})),
];

function checkWinNativePayload() {
checkNativePayload({
dir: path.join(ROOT, "electron", "native", "bin", "win32-x64"),
required: WIN_REQUIRED,
osLabel: "Windows",
bundleNoun: "the installer",
emptyDirFix: `${FIX}\n\nThe STT helper and the capture helper are separate builds — see\ntechnical-documentation/engineering/build-and-packaging.md.`,
});
}

function checkMacNativePayload(context) {
checkNativePayload({
dir: path.join(ROOT, "electron", "native", "bin", `darwin-${archTagFor(context)}`),
Expand Down Expand Up @@ -313,7 +363,19 @@ function checkCompositorAddonFreshness(
exports.default = async function beforePack(context) {
const platform = context?.electronPlatformName ?? process.platform;
if (platform === "win32") {
checkCompositorAddonFreshness();
// The copy that ships is the arch-tagged one under electron/native/bin/
// (win.extraResources), beside its ffmpeg DLLs — not the dev copy this hook
// used to be the sole guardian of. Same reasoning as the darwin branch below.
const shipped = path.join(
ROOT,
"electron",
"native",
"bin",
"win32-x64",
"compositor_view.node",
);
checkWinNativePayload();
checkCompositorAddonFreshness(shipped, FIX, "D3D11");
return;
}
if (platform === "darwin") {
Expand Down Expand Up @@ -376,6 +438,20 @@ if (require.main === module) {
HELPER_SOURCE_PATHS,
);
console.log(`Linux native payload complete in electron/native/bin/${tag}, addon up to date.`);
} else if (process.platform === "win32") {
const shipped = path.join(
ROOT,
"electron",
"native",
"bin",
"win32-x64",
"compositor_view.node",
);
checkWinNativePayload();
checkCompositorAddonFreshness(shipped, FIX, "D3D11");
console.log(
"Windows native payload complete in electron/native/bin/win32-x64 (addon beside its ffmpeg DLLs), addon up to date.",
);
} else {
checkCompositorAddonFreshness();
console.log("compositor addon is up to date with its Rust sources.");
Expand Down
29 changes: 29 additions & 0 deletions scripts/build-windows-compositor-addon.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -95,5 +95,34 @@ fs.mkdirSync(BUILD_OUT_DIR, { recursive: true });
const dest = path.join(BUILD_OUT_DIR, "compositor_view.node");
fs.copyFileSync(builtDll, dest);

// Also install next to the ffmpeg DLLs the addon links against, which is the copy
// that actually ships (win `extraResources`, filter `win32-*/*`). macOS has always
// done this — see build-macos-compositor-addon.mjs's archBinDir — and Windows not
// doing it is what broke the Store build of 1.9.0:
//
// The addon dlopens avcodec/avformat/avutil at require() time. Shipped from inside
// app.asar.unpacked it sat in a different directory from those DLLs, so loading it
// depended on `ensureFfmpegSharedDllsOnPath` prepending their directory to PATH.
// That works for the NSIS installer and does NOT work under MSIX: a packaged app
// resolves dependent DLLs through its package graph and ignores PATH. Measured
// inside a registered package, with the directory correctly on PATH:
//
// require BEFORE PATH: FAILED: The specified module could not be found.
// require AFTER PATH: FAILED: The specified module could not be found.
//
// and with the addon sitting beside its DLLs, no PATH involved:
//
// require BEFORE PATH: LOADED OK
//
// Node loads .node files with LOAD_WITH_ALTERED_SEARCH_PATH, so the addon's own
// directory is searched for its dependencies. Colocating removes the PATH mechanism
// rather than repairing it. `buildCandidatePaths` already probes this location
// first, so no loader change is needed.
const archBinDir = path.join(ROOT, "electron", "native", "bin", "win32-x64");
fs.mkdirSync(archBinDir, { recursive: true });
const archDest = path.join(archBinDir, "compositor_view.node");
fs.copyFileSync(builtDll, archDest);

console.log(`Built ${builtDll}`);
console.log(`Copied ${dest}`);
console.log(`Copied ${archDest}`);
26 changes: 25 additions & 1 deletion technical-documentation/engineering/build-and-packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,31 @@ A usable full package depends on generated artifacts that are not committed:
| Native Metal compositor addon | `electron/native/bin/darwin-<arch>/compositor_view.node` (plus a dev copy under `electron/native/compositor-view/build/`) | Rust, Xcode, and the LGPL FFmpeg tree from `fetch:ffmpeg:mac` |
| FFmpeg runtime files | matching `electron/native/bin/<platform>-<arch>/` directory | Downloaded by `fetch:ffmpeg` on Windows; **built from source** by `fetch:ffmpeg:mac` on macOS (~5 min) — BtbN publishes no macOS target and every circulating macOS build is GPL, which would relicense this MIT app |

Electron-builder copies only the matching `electron/native/bin/<platform>-<arch>/` directory into each package. The compositor `.node` file is included by the Windows `files` rule and unpacked from ASAR because native addons cannot be loaded from inside the archive.
Electron-builder copies only the matching `electron/native/bin/<platform>-<arch>/` directory into each package. On both Windows and macOS the compositor `.node` ships from inside that directory, beside the ffmpeg libraries it links against, and never travels through ASAR.

### The compositor addon must sit beside its ffmpeg libraries

This is a hard requirement on Windows, not a tidiness preference.

The addon dlopens `avcodec`/`avformat`/`avutil` at `require()` time. Until 1.9.0 the Windows build shipped it inside `app.asar.unpacked/electron/native/compositor-view/build/`, one directory away from `electron/native/bin/win32-x64/*.dll`, and the gap was bridged at runtime by `ensureFfmpegSharedDllsOnPath` prepending the DLL directory to `PATH` before the require.

That works for the NSIS installer. **It does not work under MSIX**, which resolves an addon's dependent DLLs through the package graph and ignores `PATH`. Measured inside a registered package, with the directory verifiably present and correctly prepended to `PATH`:

```
dllDir existsSync : true
require BEFORE PATH : FAILED: The specified module could not be found.
require AFTER PATH : FAILED: The specified module could not be found.
```

and with the addon sitting beside those same DLLs, no `PATH` involved:

```
Comment on lines +49 to +57

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Specify a language for both diagnostic code fences.

The configured markdown linter reports MD040 for lines 49 and 57. Add text to both opening fences.

Proposed fix
-```
+```text
 dllDir existsSync   : true
 require BEFORE PATH : FAILED: The specified module could not be found.
 require AFTER  PATH : FAILED: The specified module could not be found.

and with the addon sitting beside those same DLLs, no PATH involved:

- +text
require BEFORE PATH : LOADED OK

</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion

🧰 Tools
🪛 markdownlint-cli2 (0.23.2)

[warning] 49-49: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


[warning] 57-57: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@technical-documentation/engineering/build-and-packaging.md` around lines 49 -
57, Add the text language identifier to both diagnostic Markdown code fences in
the documentation, including the fence containing the dllDir/require output and
the subsequent fence containing the LOADED OK output; leave their contents
unchanged.

Source: Linters/SAST tools

require BEFORE PATH : LOADED OK
```

Node loads `.node` files with `LOAD_WITH_ALTERED_SEARCH_PATH`, so the addon's own directory is searched for its dependencies. Colocating removes the `PATH` mechanism rather than repairing it, and works on every Windows packaging format.

This shipped: the 1.9.0 Store build loaded no compositor at all, so the editor opened with a permanently blank preview while audio kept playing — audio comes from the renderer, every frame comes from the addon. It read as an application bug rather than a packaging one, because every file was present in the package and the NSIS build of the same commit was fine. `scripts/before-pack.cjs` now refuses to package unless the addon and at least `avcodec`/`avformat`/`avutil` are in the same directory, on Windows as it already did on macOS.

`electron/native/bin/`, local native build directories, the compositor build output, models, and caches are gitignored. Rebuilding from a source checkout therefore requires the complete platform toolchain and third-party SDKs; running the generic `npm run build` alone does not manufacture missing native artifacts. The Windows compositor's D3D11/FFmpeg prerequisites are described by the source POC in `crates/README.md`, while capture helper lookup and output conventions are documented in `electron/native/README.md`.

Expand Down
Loading