diff --git a/.changelog/NEXT.md b/.changelog/NEXT.md index dfaefcb5dd..cf5f37ddfd 100644 --- a/.changelog/NEXT.md +++ b/.changelog/NEXT.md @@ -25,6 +25,7 @@ TBD - **Image Gen: hi-res presets opened to HiDream + Qwen-Image, plus a Qwen comic-page draft preset.** `client/src/lib/imageGenResolutions.js`'s `compatibilityKey` previously fell through to `flux1` for HIDREAM/QWEN, silently denying their natural `1024×1536` / `1536×1024` / `1536²` hi-res variants. Both runners now return their own keys, the three hi-res entries advertise compatibility for FLUX2 + QWEN + HIDREAM, and the Flux portrait/landscape entries (`832×1216`, `1216×832`) also open to them. Added a Qwen-only `1328×2048 (Qwen comic draft — experimental)` preset for US-comic-trim-aspect draft renders — past Qwen's documented ~1664 long-edge support window but coherent in practice on the 20B MMDiT; pair with a 1.5× upscale for final 1988×3056 output. Partial fix for `[hidream-qwen-client-runner-consumers]`; LoRA-picker side still pending. - **Image Gen preview: step counter under the preview.** The live diffusion preview now shows a `step 32/50 · 64%` row below the image while a render is in flight. The progress payload already carried `step`/`totalSteps` from `imageGen/local.js`; just rendered them. Hidden for Codex / external paths that don't emit per-step events. - **Qwen-Image stepwise preview decode.** `scripts/z_image_turbo.py` gained a Qwen-aware preview-decoder branch: Qwen-Image (and Img2Img / Edit siblings) packs latents to 3D `(B, num_patches, C*4)` like Flux, then unpacks to a 5-D video-VAE layout and unnormalizes with per-channel `latents_mean`/`latents_std` lists from `vae.config` before decoding. The generic `latents/scaling + shift` path matched neither step. New unpack helper (3D→5D) + Qwen-specific decoder; per-channel norm tensors are built CPU-side once and lazily migrated to (device, dtype) per call so the diffusion loop's ~30–50 fires don't rebuild from a list every step. +- **[ref-watch-phosphene-pin-mlx-lazy-eval-cadence-envs] Faster LTX-2 video renders on Apple Silicon.** Tuned the MLX lazy-eval cadence the LTX-2 helper uses so denoise steps run at the cadence phosphene's published M-series perf matrix shows is fastest — also avoids a known post-decode hang the prior setting could exhibit. No UI change; renders just finish sooner. - **Suppress cosmetic CLIP-truncation warning on multi-encoder pipelines.** The "input was truncated because CLIP can only handle N tokens" warning from diffusers was bleeding red into PortOS server logs on long prompts with HiDream, Qwen-Image, FLUX.2, Z-Image, and ERNIE — but on every one of these pipelines CLIP only contributes pooled style vectors derived from the first ~77 tokens while a high-context encoder (T5 / Llama-3.1-8B / Qwen2-VL) reads the full prompt. New `suppress_cosmetic_clip_truncation()` helper in `scripts/_runner_common.py` attaches an idempotent logging filter to the `diffusers` and `transformers` handlers (filter is on handlers, not loggers, because logger-level filters don't apply to propagated child records). Called from `scripts/z_image_turbo.py` after `import diffusers` and from `scripts/flux2_macos.py` after the pipeline-load dispatcher. ## Fixed diff --git a/PLAN.md b/PLAN.md index 18b6cb9d8f..68e156d9b8 100644 --- a/PLAN.md +++ b/PLAN.md @@ -23,7 +23,6 @@ _Batch-cleared 2026-05-25: 23 Next Up items shipped together via parallel sub-ag - [ ] [chrome-canary-followups] **Hardening for the custom-Chrome-binary feature (xhigh code-review 2026-05-25).** _DONE in the v2.10.0 release review: (b) both `spawn()` calls now have `.on('error', …)` listeners; (d) `browser/server.js#loadConfig` now try/catches the `JSON.parse`; (e) `setup-browser.js#loadConfig` now warns + returns `null` and `applyCanaryToConfig` skips the save when the existing config is unreadable; (f) top-level `runCanarySetup()` is now `.catch()`-wrapped. (c) is tracked separately in `[setup-browser-canary-headless]`. Remaining: (a), (g)–(o)._ **High-severity:** (a) On macOS headed mode (the default), `browser/server.js:208` uses `macAppBundle` only and silently ignores `chromePath` — a UI user who fills in `chromePath` for Canary/Chromium/Brave but leaves `macAppBundle` empty gets stock Chrome and the log misreports the binary; couple the two fields in the UI (or auto-derive `macAppBundle` from `chromePath` when the latter is inside a `.app`). (b) Neither `spawn(chromePath, …)` (line 216) nor `spawn('/usr/bin/open', …)` (line 208) has an `.on('error', …)` listener — a typo in `chromePath` emits 'error' with no listener → `uncaughtException` → portos-browser PM2 child crashes and restart-loops. (c) `scripts/setup-browser.js#applyCanaryToConfig` writes `chromePath` + `macAppBundle` but doesn't flip `headless: false`; combined with the seed default of `headless: true` (`data.reference/browser-config.json`), a fresh install accepting Canary runs Canary invisibly. (d) `browser/server.js#loadConfig` does bare `JSON.parse(raw)` with no try/catch — combined with setup-browser's non-atomic `writeFileSync` and the `cachedConfig` race, a partial-file write crashes the supervisor on next start (PM2 restart-loop forever). **Medium-severity:** (e) `setup-browser.js#loadConfig` silently catches all JSON parse errors and returns `{}`, then `applyCanaryToConfig` saves `{chromePath, macAppBundle}` only — wiping every other user-customized key. (f) Top-level `await runCanarySetup()` has no try/catch — any EACCES on data/browser-config.json aborts `npm run setup` / `update.sh` at what was previously a no-op step. (g) Idempotency guard only checks `chromePath`: users who decline get re-prompted on every update, and users who set only `macAppBundle` keep getting re-prompted. (h) `PORTOS_USE_CANARY` only matches literal `'0'`/`'false'` (opt-out) or `'1'`/`'true'` (opt-in) — `'no'`/`'off'`/`'yes'`/`'on'`/`'True'` fall through both branches. (i) `spawnSync(install.cmd, …, { stdio: 'inherit' })` for brew/winget can hang on a sudo password prompt under non-TTY + `PORTOS_USE_CANARY=1` (update.sh stalls). (j) `cachedConfig` in `browserService.js` is stale relative to setup-browser's direct write — GET /api/browser/config returns pre-update values until process restart. (k) `saveConfig` uses bare `writeFileSync` — switch to the canonical `atomicWrite` pattern (`server/lib/fileUtils.js`). **Low-severity:** (l) `spawnSync` failure-status check `result.status !== 0` treats `status: null` (spawn-failure / signal kill) identically to a non-zero exit and never logs `result.error` — masks ENOENT/EPERM/SIGKILL. (m) `optionalPath` Zod schema accepts any string up to 1024 chars; no `.app`/`.exe` sanity check — user pastes the bundle into `chromePath` → spawn() EISDIR. (n) `launchBrowser`'s reuse-existing-Chrome early-return (line 167) fires BEFORE `headlessMode = config.headless === true` (line 170), leaving the module-level default `false` after a PM2 restart that reuses Chrome — /health reports wrong mode (pre-existing, but in a function touched by this change). (o) `detectCanary` on macOS only checks `/Applications/...`; misses per-user `~/Applications/Google Chrome Canary.app` installs (corporate Macs, `HOMEBREW_CASK_OPTS=--appdir=$HOME/Applications`). - [ ] [ref-watch-phosphene-teacache-extend-a2v-denoise] **TeaCache through Extend + A2V Stage-1 denoise loops.** From `reference-watch` review of phosphene (commits `ea98aad8` + `17be2a79`, 2026-05-28). PortOS's `scripts/generate_ltx2.py` invokes `ExtendPipeline.extend_from_video(...)` (~line 378) and `AudioToVideoPipeline.generate_and_save(...)` (~line 417) directly without passing a TeaCache controller, so the slowest two modes in the panel run full denoise. Phosphene reuses the existing Stage-1 calibration (`ti2vid_two_stages._build_teacache_controller(n_steps, thresh=0.5)`) by monkey-patching `guided_denoise_loop` on the `ltx_pipelines_mlx.retake` and `ltx_pipelines_mlx.a2vid_two_stage` modules, gating activation on a per-call module-level config dict, set before the pipeline call and cleared in `finally`. Predicted ~1.2× on extend at default threshold (up to ~3× at 1.5); similar for A2V Stage 1. Fix: install both monkey-patches at the top of `generate_ltx2.py`, wire `_EXTEND_TC_CONFIG` / `_A2V_TC_CONFIG` from the helper's `run_extend`/`run_a2v` paths, and add a `--no-teacache` CLI flag (default-on). **Decision needed:** ride along with the `[ref-watch-phosphene-bump-ltx2-pin-v0148]` pin bump (the calibration constants are part of the same upstream surface that gets renamed) or land independently against today's pin? Scope: small/medium. - [ ] [ref-watch-phosphene-strip-malloc-subprocess-env] **Strip `Malloc*` env vars before spawning Python subprocesses.** From `reference-watch` review of phosphene (commit `0d7cb137`, 2026-05-28). PortOS's `server/services/videoGen/local.js:609` passes `process.env` through to the Python child unchanged (`const childEnv = { ...process.env, ...(await hfTokenEnv()) };`). On macOS, when the user launched PortOS from Pinokio or any tool that exports an empty/zero `MallocStackLogging` (or related `Malloc*`) var, every Python subprocess prints `MallocStackLogging: can't turn off malloc stack logging because it was not enabled` once per child exit — flooding stderr (~50+ lines per gen for image-gen, which fans out into download/helper subprocesses). Fix: add a `stripDebugMallocEnv(env)` helper in `server/lib/processEnv.js` (new) — `Object.fromEntries(Object.entries(env).filter(([k]) => !k.startsWith('Malloc')))` — barrel it from `server/lib/index.js` per the module-org rule, document in `server/lib/README.md`. Then route every Node→Python spawn through it: `server/services/videoGen/local.js:611`, plus the image-gen spawn sites (grep `spawn.*python` under `server/`). Scope: small. -- [ ] [ref-watch-phosphene-pin-mlx-lazy-eval-cadence-envs] **Pin `LTX2_DIT_EVAL_EVERY=1` + `LTX2_GEMMA_EVAL_EVERY=1` at helper boot.** From `reference-watch` review of phosphene (commit `ec1437fe`, 2026-05-28). PortOS's `scripts/generate_ltx2.py` doesn't set either env, so the upstream `ltx-2-mlx` default (`=8`) governs lazy MLX graph eval cadence. Phosphene's M4 Max 64 GB matrix on I2V Balanced 5s / 121 f shows `=8` → ~3 min/step, `=1` → ~7 s/step (~25× faster denoise); `=0` is also fast but extends the documented post-decode hang. Fix: add at the very top of `generate_ltx2.py`, BEFORE any `from ltx_pipelines_mlx import …`, `os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "1"); os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "1")`. Smoke-test all five modes (t2v/i2v/fflf/extend/a2v) on a real Mac before merging since `=1` interacts with the same post-decode hang the existing `[ref-watch-phosphene-sigkill-watchdog-videogen-local]` slug already plans to mitigate. Scope: small. ## Deferred Architecture (human-led planning) diff --git a/scripts/generate_ltx2.py b/scripts/generate_ltx2.py index 4aa71eea37..9f08854710 100644 --- a/scripts/generate_ltx2.py +++ b/scripts/generate_ltx2.py @@ -46,12 +46,13 @@ from typing import NoReturn # Must be set BEFORE any ltx_core_mlx import: ltx_core_mlx.model.transformer.model -# reads LTX2_DIT_EVAL_EVERY at import time. The upstream default (=8) causes -# ~15-25x perf loss on M-series Macs via per-block Metal command-buffer churn; -# =0 disables lazy evaluation and avoids that overhead. setdefault lets a -# caller-supplied env var override these if needed. -os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "0") -os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "0") +# reads LTX2_DIT_EVAL_EVERY at import time. Phosphene's M4 Max 64 GB I2V Balanced +# 5s / 121 f matrix: upstream default =8 runs ~3 min/step (per-block Metal +# command-buffer churn); =1 runs ~7 s/step (~25× faster denoise); =0 is also +# fast but extends the post-decode deallocator hang. =1 wins on both axes. +# setdefault lets a caller-supplied env var override. +os.environ.setdefault("LTX2_DIT_EVAL_EVERY", "1") +os.environ.setdefault("LTX2_GEMMA_EVAL_EVERY", "1") def emit_status(msg: str) -> None: