xrCaptureAtlasEXT capture has wrong/squished aspect on Leia in-process (live display is fine)

[dfattal]

Summary

xrCaptureAtlasEXT ("I" key) produces a wrong / squished-aspect PNG when the
DisplayXR-test Unity player runs in-process on a Leia SR display. The live
display is correct (and has been for months) — this is a capture-only bug.

This issue chronicles the full saga so a Unity-side agent can pick it up. It is the
Unity counterpart of runtime issue DisplayXR/displayxr-runtime#425.

The saga (three stages)

1. Stage 0 — black capture. Original symptom: the Unity capture wrote a PNG
   but it was fully transparent → black. Root-caused (runtime side) to the
   in-process compositor gating the capture dispatch behind !zero_copy while the
   zero-copy path skipped populating the renderer atlas. Fixed in the runtime
   (comp_d3d11, in-process zero-copy capture path). PNG now writes opaque.

2. Stage 1 — fix broke the texture. An early runtime fix attempt forced a
   re-composite to populate the atlas for capture; this corrupted the geometry
   in the captured texture. Reverted; replaced with a direct SRV read.

3. Stage 2 — wrong aspect (current). With the PNG now writing, the captured
   atlas has the wrong aspect ratio — content is horizontally squeezed. This is
   the remaining open bug.

Environment / repro

• Display: Leia SR, 3840×2160, recommendedViewScale = 0.5 × 0.5 (via
  XR_EXT_display_info).
• App: displayxr-unity-test (productName DisplayXR-test), built player,
  in-process (is_service_mode=false), not shell/IPC.
• Window: ~2400×2160 client (squarish) — not 16:9. The Unity window is roughly
  square, with a visible black/pillarbox region.
• Repro: launch the player in-process (no service/shell), press I. PNG lands in
  Pictures\DisplayXR\…_atlas_2_2x1.png. (Synthetic key injection does not work
  for Unity's new Input System — a human keypress is required.)

Confirmed facts (from live runtime logs)

Quantity	Value	Source
Display pixels / scale	3840×2160, 0.5×0.5	XR_EXT_display_info (iface=leia-sr)
Unity per-eye swapchain	1920×1080, arrays=2 (= display × 0.5×0.5)	plugin xrCreateSwapchain log
Unity submitted imageRect	1920×1080 per view	runtime VIEW SIZE MISMATCH
Compositor renderer view (window-based)	1200×1080 (atlas 2400×1080)	comp_d3d11_renderer_resize
DP weave target (= window client)	2400×2160	leia_dp_d3d11_process_atlas
Live display	correct (works for months)	user-confirmed

So Unity sizes its render target / swapchain / imageRect from
recommendedImageRect = display × scale = 1920×1080 (display-relative). The
in-process compositor instead sizes its atlas tile slots window-relative
(window_client × scale ⇒ 2400×0.5 = 1200, 2160×0.5 = 1080). The compositor
composites Unity's 1920×1080 tile into its 1200×1080 slot (X scaled 0.625), so
the internal atlas the capture reads is horizontally squeezed. The DP weave
re-stretches it onto the 2400×2160 window, which is why the live display looks
right but the captured atlas does not.

This also fires a (cosmetic, non-fatal) VIEW SIZE MISMATCH: app_rect=1920x1080 expected=1200x1080 every frame, which disables the zero-copy passthrough path — so
the capture falls through to reading the composited (squeezed) atlas rather than
Unity's pristine 1920×1080 source.

What does NOT work (dead-end — do not repeat)

I tried making the Unity plugin render window-relative tiles to match the
compositor: a new xrEnumerateViewConfigurationViews hook in
native~/displayxr_hooks.cpp rewriting recommendedImageRect from 1920×1080
(display×scale) → 1200×1080 (window-client×scale). The hook worked exactly as
intended (Unity created 1200×1080 swapchains, VIEW SIZE MISMATCH would clear).

But it broke the live display. With Unity rendering 1200×1080, the content
filled only the left ~60% of the window with a large black region on the right
(screenshot below). Unity rendering 1920×1080 is correct for the live weave;
changing it regresses a working path. This change was reverted. Do not alter how
Unity sizes its render target — the bug is in the capture, not the rendering.

Screenshot of the regression (window squarish; cube + sky gradient fill the left
portion, right ~40% black): captured during the failed window-relative attempt.

Likely correct direction (for the next agent)

The live path wants Unity's display-relative 1920×1080. The capture wants a
correct-aspect image. Options, roughly in order of how Unity-local they are:

1. App-side capture of Unity's own source texture. Have the plugin
   (Runtime/DisplayXRScreenshot.cs + native readback) write the PNG from Unity's
   own 1920×1080 per-eye render texture (the pristine, correct-aspect source)
   instead of routing through the runtime's xrCaptureAtlasEXT, which reads the
   compositor's window-squeezed atlas. Trade-off: partially re-introduces app-side
   readback that #425 was trying to unify under the runtime path — coordinate intent
   with the runtime maintainers.
2. Runtime-side capture-only fix (needs runtime coordination, …-runtime#425).
   Make the in-process capture read the app's submitted swapchain at its
   imageRect dims (Unity's 1920×1080 source), not the window-based composited
   atlas. This is the cleanest "capture is faithful to what the app drew" fix but
   lives in the runtime, not this repo.
3. De-squeeze on encode. Resample the captured atlas to the window/canvas aspect
   before writing the PNG. Cosmetic; least principled.

Open questions to resolve first

• Which exact capture path does the Unity (non-zero-copy) case currently take in the
  runtime, and what are the actual dims/content of the bad PNG? (I characterized
  the squeeze from logs + the live screenshot; I did not open the bad PNG this
  session — verify it directly.)
• Is app-side capture acceptable given #425's goal of a runtime-owned capture
  contract, or should this be fixed runtime-side?

Cross-references

• Runtime: DisplayXR/displayxr-runtime#425 (capture naming + opaque alpha + this
  squish).
• Pre-existing uncommitted #425 work in this repo (Runtime/DisplayXRScreenshot.cs,
  DisplayXRFeature.cs, DisplayXRNative.cs, native~/displayxr_extensions.h,
  displayxr_hooks.h, CHANGELOG) is the in-flight matcher/naming change — build on
  it, don't revert it.

[dfattal]

Resolved — root cause was a project setting, not a plugin/runtime bug.

The actual cause of the wrong-aspect capture was the Unity test project's graphics API set to D3D11 instead of D3D12. Switching it to D3D12 produces correct-aspect captures. The "squish" framing in this issue (and the separate-per-eye-swapchain / window-vs-display-scale theories) was a misdiagnosis — the live display was always correct, and the difference was the D3D11 vs D3D12 submission path, fixed in the project config.

For the record, the runtime-side hardening of xrCaptureAtlasEXT (opaque alpha, _atlas_<viewCount>_<cols>x<rows> naming, in-process capture dispatch) landed in DisplayXR/displayxr-runtime#432 (SPEC_VERSION 2) and is verified across native test apps, demos, and the Unity (v1.16.0) + Unreal (v0.3.1) plugins. Unity's capture uses the v2 bare-prefix convention.

Closing as resolved.