Extract the C++ scaffolding layer (displayxr::common) — layer 2 of #396

[dfattal]

⚠️ Re-scoped under epic #396 (2026-06-02)

Investigation found the shared code (esp. the Kooima math display3d_view/camera3d_view) is vendored in six places — not just the two demos — including displayxr-unreal (Private/Native/) and displayxr-unity (native~). The plan changed accordingly:

• One repo, two CMake targets — not a new repo. Repurpose the dormant kooima-projection → displayxr-common, exposing displayxr::math (pure C, linked by the engines too) + displayxr::common (C++ scaffolding, depends on math). Net new repos: 0.
• This issue is now layer 2 (W4) of Epic: shared helper module (displayxr-common) + unified capture API — kill the 6× Kooima/common drift #396: extracting/migrating the C++ scaffolding target. The universal piece — reconciling the math core and adopting it org-wide (engines included) — is W1–W3 in the epic and is the prerequisite for the work here.
• The "new repo / FetchContent by tag / divergence policy / reconcile-first" detail below still applies to the displayxr::common target; read "new displayxr-common repo" as "the displayxr::common target in the renamed repo."

See #396 for the full inventory, architecture, and ordering.

---

Problem

The native C++ demos each carry a checked-in common/ directory (view math,
atlas capture, HUD, input, window manager, window-space-layer UI, logging,
xr-session glue, stb wrappers). These copies were originally seeded from this
runtime repo but are now maintained independently in each demo — so they
drift. The runtime no longer carries these files, so there is no canonical
home, and "common" is a misnomer: the copies are diverging.

Concrete snapshot today (displayxr-demo-modelviewer vs
displayxr-demo-gaussiansplat, ~28 files in common/):

File	Δ lines	Nature
display3d_view.c/.h	183 / 57	mostly gauss's unmerged feat/zdp-clip-soft-fade-pick WIP (near_z/far_z, selftest, center_view, vulkan_flip_y) + a Vulkan [0,1] projection fix landed in modelviewer
xr_session_common.cpp/.h	98 / 31	gauss-side changes
hud_renderer.cpp/.h	31 / 10	gauss-side
input_handler.cpp/.h	8 / 7	gauss-side
stb_image_impl_macos.cpp	—	modelviewer-only (legit app-local)

~20 of the ~28 files are byte-identical. Almost all the drift is accidental
(two checked-in copies edited independently, plus one repo on an unmerged
branch), not essential divergence.

Most recently this bit us in the clip-plane work: the model viewer's raster PBR
pass needed a Vulkan [0,1] projection (the GL [-1,1] form silently clipped
the model once far = ez + 1000·vH), while the gauss splat renderer is immune
(it culls in software, not against the projection planes). That fix had to be
hand-applied to one copy with no mechanism to keep the other in sync.

Why now (motivation beyond de-dup)

The window-space-layer UI is becoming a genuinely reusable building block. With
the #389 fix (VK native compositor now reliably composites many window-space
layers; new windowspace_handle_{d3d11,d3d12,vk,gl}_win test apps added here),
every native app — both demos and this repo's native test apps — wants the
same window-space-layer + HUD + input scaffolding. That's a library, not a
vendored folder.

Goal

Make common/ a versioned dependency rather than a checked-in directory, so
that editing-a-local-copy (the drift failure mode) becomes structurally
impossible, and the same module is reused by:

• displayxr-demo-modelviewer
• displayxr-demo-gaussiansplat
• this repo's native C++ test apps (*_handle_*_win, windowspace_handle_*_win)
• future native demos / test apps

Proposed design

1. New repo displayxr-common (org-level, lightweight), exporting CMake
targets (e.g. displayxr::common). Contents = the genuinely shared files:
view math (display3d_view, camera3d_view, leia_math), atlas capture,
HUD renderer, input handler, window manager + window-space-layer UI helpers,
logging, xr_session_common, stb wrappers, view_params, manifest cmake.

2. Consume via CMake FetchContent pinned to a tag — same pattern this repo

• the demos already use for tinygltf/glm/OpenXR-loader. Rationale vs
  alternatives:

• FetchContent by tag (recommended): plain git clone + cmake works (no
  --recursive); a version bump is a one-line GIT_TAG diff; nothing is
  checked into the consumer tree so the vendored-copy edit is impossible; local
  co-development via FETCHCONTENT_SOURCE_DIR_DISPLAYXRCOMMON=../displayxr-common
  (edit a sibling checkout live, no re-fetch).
• git submodule: source visible in-tree but --recursive/detached-HEAD/
  forgotten-pointer-bump friction; rejected.
• git subtree: no clone pain, but the vendored copy stays editable so drift
  can creep back; rejected.

3. Divergence policy — mechanism, not policy, lives in the lib. App-specific
behavior is expressed at the call site, never via #ifdef APP inside the lib:

• Parameterize (preferred): the clip/cull case is the worked example — the lib
  exposes display3d_compute_view(..., near_offset, far_offset, ...) and outputs
  both the projection matrix and the resolved near_z/far_z. The model
  viewer uses the projection's hardware clip; gauss feeds near_z/far_z into
  its software splat cull. Same call, different consumption — no fork. The
  Vulkan [0,1] projection is correct for both (splats are insensitive to the
  z-range).
• Inject the renderer-specific bit via a small callback/strategy when a param
  isn't enough.
• Keep app-local what is genuinely app-specific (e.g.
  stb_image_impl_macos.cpp). Not everything must live in the lib.
• Never #ifdef MODELVIEWER / #ifdef GAUSS inside the lib — that re-creates
  drift inside the shared repo.

Migration steps

1. Reconcile first (while still vendored): unify display3d_view.{c,h} into
   one file that is the superset — Vulkan [0,1] projection + gauss's
   near_offset/far_offset API + near_z/far_z outputs (+ selftest /
   center_view as they land). Fold in the hud/input/xr_session deltas.
   Land gauss's feat/zdp-clip-soft-fade-pick so there's a single reconciled
   baseline.
2. Create displayxr-common seeded from the reconciled baseline; define the
   CMake target(s); tag v0.1.0.
3. Migrate consumers: delete each common/ dir; add a FetchContent block
   pinned to the tag; wire the target into the existing CMake. Verify Windows +
   macOS builds (CI already runs both per demo).
4. Adopt in this repo's native test apps (*_handle_*_win,
   windowspace_handle_*_win) so the window-space-layer UI is shared, not
   re-copied.
5. Release discipline: displayxr-common gets its own tags; consumers bump
   the pin when ready (independent cadence preserved). Optional CI guard: a hash
   check if any app ever keeps a vendored mirror.

Non-goals

• Monorepo (contradicts the independent release cadence).
• Folding engine-based test apps (Unity/Unreal) — they don't use this C++ code.
• Renaming common/ in place without extraction (documents drift, doesn't
  prevent it).

Open questions

• Home: dedicated displayxr-common repo (recommended) vs a subdir of this
  runtime repo (zero-fetch for runtime test apps, but couples demo builds to the
  large runtime repo).
• Lib surface: exact split between shared lib vs app-local files (esp. the
  stb implementation TU and any platform glue).
• Versioning: tag-per-change vs periodic tags; whether to gate demo releases
  on a minimum displayxr-common version the way they already gate on a minimum
  runtime version.