Skip to content

Android: CNSDK display-processor plug-in (libdxrp050_leia_cnsdk.so)#11

Open
leaiss wants to merge 4 commits into
mainfrom
feat/android-cnsdk-dp
Open

Android: CNSDK display-processor plug-in (libdxrp050_leia_cnsdk.so)#11
leaiss wants to merge 4 commits into
mainfrom
feat/android-cnsdk-dp

Conversation

@leaiss
Copy link
Copy Markdown
Collaborator

@leaiss leaiss commented May 27, 2026

Summary

Companion to displayxr-runtime PR #268, which splits the Android POC along the post-#263 plug-in boundary. The runtime side keeps the cross-cutting infrastructure (is_self_submitting vtable flag, vk_native Android plumbing, audit fixes); CNSDK + driver-side audit fixes live here.

Adds src/drv_leia_android/ as a sibling of the Windows src/drv_leia/, sharing the same plug-in entry contract (xrtPluginNegotiate) but selected at CMake configure time by if(WIN32) / elseif(ANDROID).

Source files (new src/drv_leia_android/)

File Purpose
leia_cnsdk.{cpp,h} CNSDK C-ABI wrapper. Core init (worker thread), interlacer lifecycle, atlas weave, face-tracking snapshot. Audit fixes: B2 (gamma format match), B4 (mm→m), B5 (camera-relative→display-relative), B10 (worker watchdog), B11 (interlacer fail-flag), B12 (device-config cache → also implicitly closes B9).
leia_display_processor_cnsdk.{cpp,h} xrt_display_processor vtable wired to the wrapper. is_self_submitting=true, atlas-mode only (no per-tile blit — CNSDK splits the SBS atlas internally via set_interlace_view_texture_atlas), DXR_HW_DBG + DXR_ATRACE blocks gated on XRT_DEBUG_ANDROID_VERBOSE. Audit fixes: B7 (vkDeviceWaitIdle between weave + HUD record — masked on Android, future-proof), B8 (mono 1×1 passthrough), B14 (pause/resume from session begin/end).
leia_plugin_android.c xrtPluginNegotiate entry point + xrt_plugin_iface vtable. Only create_dp_vk is non-NULL (no D3D/Metal/GL on Android). id = "leia-cnsdk". probe always succeeds (POC pattern — CNSDK device-config is async, so synchronous probing isn't possible at xrCreateInstance time). Minimal leia_android_hmd_create() with hardcoded Lume Pad 2 defaults; CNSDK metrics surface lazily via the DP's get_display_dimensions / get_display_pixel_info once the async core init completes.

Build wiring

  • CMakeLists.txt (top-level): relax if(NOT WIN32) FATAL_ERROR to if(NOT WIN32 AND NOT ANDROID). Branch add_subdirectory between src/drv_leia (Windows) and src/drv_leia_android (Android). Skip the NSIS installer on Android — APK + jniLibs/ is the install path.
  • src/drv_leia_android/CMakeLists.txt: find_package(CNSDK CONFIG REQUIRED) on CNSDK_ROOT env var or CMake var. Builds libdxrp050_leia_cnsdk.so, matching the runtime's plug-in filename convention (libdxrp<NNN>_<id>.so). Symbol-hidden except xrtPluginNegotiate.

Docs

  • docs/cnsdk-android-calibration.md — moved from displayxr-runtime PR #271's docs/cnsdk-axis-calibration branch, with file-path refs rewritten to src/drv_leia_android/…. Three CNSDK convention assumptions (face axis signs/units, tile-to-eye mapping, UV vertical flip) with symptom→fix tables for on-device bring-up.

How this slots into the runtime APK (POC mode)

# 1. Build the plug-in standalone:
export CNSDK_ROOT=/path/to/cnsdk-android-0.7.28
cmake -S . -B build-android \
    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI=arm64-v8a \
    -DANDROID_PLATFORM=android-29 -G Ninja
cmake --build build-android --target dxrp050_leia_cnsdk

# 2. Drop the .so into the runtime APK's jniLibs/ tree:
cp build-android/src/drv_leia_android/libdxrp050_leia_cnsdk.so \
   <runtime-repo>/src/xrt/targets/openxr_android/src/main/jniLibs/arm64-v8a/

# 3. Build the runtime APK (PR #268 head):
./gradlew.bat :src:xrt:targets:openxr_android:assembleInProcessDebug

The runtime's target_plugin_loader.c Android branch (PR #309, commit c96c93ce8 on main) discovers the plug-in via dladdr-of-self + dirname enumeration at xrCreateInstance time. v2 multi-APK discovery (separate vendor APKs each shipping plug-ins) is tracked at runtime #310.

Status

  • ✅ Compiles in the standalone repo against CNSDK 0.7.28 (host-side preflight).
  • ⏳ Not yet wired into the runtime APK's Gradle (manual jniLibs/ drop per the README until multi-module Gradle integration lands — follow-up).
  • ⏳ First hardware install gated on Lume Pad arrival. Bring-up plan: docs/getting-started/android-bringup-checklist.md in the runtime repo.

Test plan

  • cmake -B build-android -DCMAKE_TOOLCHAIN_FILE=... configures cleanly with CNSDK_ROOT set.
  • cmake --build build-android --target dxrp050_leia_cnsdk produces a .so whose only exported symbol is xrtPluginNegotiate (llvm-nm --defined-only --extern-only libdxrp050_leia_cnsdk.so).
  • After dropping the .so into the runtime APK's jniLibs/arm64-v8a/, assembleInProcessDebug produces a single APK that contains both libopenxr_displayxr.so and libdxrp050_leia_cnsdk.so.
  • On Lume Pad install: adb logcat | grep target_plugin_loader shows the loader discovering libdxrp050_leia_cnsdk.so and successfully calling xrtPluginNegotiate.
  • Cube test app from displayxr-runtime PR #269 renders woven output (single first-light test from android-bringup-checklist.md Test A).

🤖 Generated with Claude Code

@leaiss
Copy link
Copy Markdown
Collaborator Author

leaiss commented May 27, 2026

@dfattal — reposting from the closed #5 (which was sourced from my fork; now consolidated to upstream branches). Same content. Android CNSDK plug-in stack #11#14 ready for review, CI green, end-to-end validated on Android-36 emulator up to xrCreateInstance -> XR_SUCCESS. Reviewing #14 alone effectively approves the whole stack since commit SHAs cascade.

leaiss and others added 3 commits June 2, 2026 08:46
…me PR #268)

Companion to displayxr-runtime PR #268, which splits the Android POC
along the post-#263 plug-in boundary. Runtime side keeps the cross-
cutting infrastructure (is_self_submitting vtable flag, vk_native
Android plumbing, audit fixes); CNSDK + drv-side audit fixes live
here.

Adds `src/drv_leia_android/` as a sibling of the Windows drv_leia/,
sharing the same plug-in entry contract (xrtPluginNegotiate) but
selected at CMake configure time by `if(WIN32)` / `elseif(ANDROID)`.

Source files
  leia_cnsdk.{cpp,h}                       — CNSDK C-ABI wrapper:
                                              core init (worker thread),
                                              interlacer lifecycle,
                                              atlas weave, face-tracking
                                              snapshot. Audit fixes
                                              B2/B4/B5/B10/B11/B12.
  leia_display_processor_cnsdk.{cpp,h}     — xrt_display_processor vtable.
                                              is_self_submitting=true,
                                              atlas-mode only (no per-
                                              tile blit), DXR_HW_DBG +
                                              DXR_ATRACE blocks gated
                                              on XRT_DEBUG_ANDROID_VERBOSE.
                                              Audit B7 / B8 / B14.
  leia_plugin_android.c                    — entry point + iface. Only
                                              create_dp_vk is non-NULL
                                              (no D3D/Metal/GL on Android).
                                              id = "leia-cnsdk",
                                              probe always succeeds (POC
                                              pattern, async device init).

Build wiring
  CMakeLists.txt (top-level)              — relax `if(NOT WIN32) FATAL_ERROR`
                                              into `if(NOT WIN32 AND NOT
                                              ANDROID)`; branch
                                              add_subdirectory between
                                              src/drv_leia (Windows) and
                                              src/drv_leia_android (Android).
                                              Skip the NSIS installer on
                                              Android (APK is the install).
  src/drv_leia_android/CMakeLists.txt     — new. find_package(CNSDK CONFIG
                                              REQUIRED) on CNSDK_ROOT;
                                              builds libdxrp050_leia_cnsdk.so
                                              matching the runtime's plug-in
                                              filename convention; symbol-
                                              hidden except xrtPluginNegotiate.

Docs
  docs/cnsdk-android-calibration.md       — moved from displayxr-runtime
                                              docs/cnsdk-axis-calibration
                                              branch per PR #271 plan, with
                                              file-path refs rewritten to
                                              point at src/drv_leia_android/.

Status: compiles in the standalone repo; not yet wired into the runtime
APK's Gradle (manual jniLibs/ drop per README until the multi-module
gradle setup lands). First hardware install gated on Lume Pad arrival.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ndroid

The Android NDK toolchain (android.toolchain.cmake) defaults
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to ONLY, which restricts find_package()
to NDK sysroot dirs and silently ignores anything we add to
CMAKE_PREFIX_PATH outside that. Result:

  find_package(CNSDK CONFIG REQUIRED)

found nothing even with CNSDK_ROOT set + appended to CMAKE_PREFIX_PATH,
breaking the standalone NDK build with "Could not find a package
configuration file provided by 'CNSDK'".

Flip the mode to BOTH at the top of src/drv_leia_android/CMakeLists.txt
so the CNSDK install at CNSDK_ROOT becomes searchable. Mirrors what the
runtime APK's gradle does via cmake.arguments
"-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH".

Also widen .gitignore to cover build-*/ so out-of-tree builds like
build-android/ don't end up staged.

Verified end-to-end:
  cmake -B build-android -G Ninja \
    -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake \
    -DCMAKE_MAKE_PROGRAM=$NDK_CMAKE/bin/ninja.exe \
    -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-29 \
    -DCNSDK_ROOT=$RUNTIME/cnsdk \
    -DDXR_RUNTIME_SOURCE_DIR=$RUNTIME \
    -DEigen3_DIR=$RUNTIME/.../intermediates/eigen/eigen-3.4.0/cmake
  cmake --build build-android --target dxrp050_leia_cnsdk

  Output: libdxrp050_leia_cnsdk.so (1.8 MB, xrtPluginNegotiate exported)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Android CNSDK display-processor factory wired six optional vtable
slots (on_pause, on_resume, is_self_submitting, get_predicted_eye_positions,
get_display_dimensions, get_display_pixel_info) but never set the
struct_size header introduced by ABI v2. The impl is calloc'd, so it
stayed 0 — under the v2 runtime, XRT_DP_HAS_SLOT bounds every optional
slot against struct_size and reads them all as absent. Most damaging:
is_self_submitting=true would be ignored, regressing the self-submitting
atlas path into a per-frame double submit.

Mirrors the drv_leia (Windows) migration in a6c0d7d.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@leaiss leaiss force-pushed the feat/android-cnsdk-dp branch from 923a309 to 2f9b688 Compare June 2, 2026 15:51
@leaiss
Copy link
Copy Markdown
Collaborator Author

leaiss commented Jun 2, 2026

Rebased the whole Android stack (#11#18) onto current main and force-pushed. Two reasons:

  1. ABI major v2 (ADR-020). The stack was based on the old v1.4.1-pinned main; main now pins runtime v1.9.0 (ABI v2). The Android DP factory wired six optional vtable slots (is_self_submitting, on_pause/on_resume, eye positions, display dims/pixel info) but never set the new struct_size header — so under the v2 runtime, XRT_DP_HAS_SLOT would read every one as absent, silently regressing the self-submitting atlas path into a per-frame double-submit. Fixed by setting base.struct_size = sizeof(struct xrt_display_processor), mirroring the drv_leia (Windows) migration in a6c0d7d.
  2. The rebase was otherwise clean (only the top-level CMakeLists.txt pin/guard overlapped; auto-merged).

Verified: builds clean against the v1.9.1 runtime headers (ABI v2) on the NDK r26 arm64-v8a toolchain — libdxrp050_leia_cnsdk.so links, DP TU compiles with 0 errors. Stack is MERGEABLE end-to-end again.

… mono NULL-guard

Post-ABI-v2 review pass:
- leia_plugin_android.c: make the probe_displays omission deliberate with an
  explicit `.probe_displays = NULL` + rationale. struct_size spans the full v2
  iface, so the runtime's gate sees the slot, finds NULL, and uses the
  synthesized primary claim — correct for a single-panel Android device with no
  monitor enumeration (Windows arm implements it over EDID; Android has none).
- leia_display_processor_cnsdk.h: fix stale doc that described the obsolete
  per-tile-blit path; current code is atlas mode (SBS VkImage handed straight to
  leia_cnsdk_weave, no per-view blit).
- leia_display_processor_cnsdk.cpp: NULL-guard vk / vk->main_queue in the mono
  1x1 passthrough fallback before dereferencing main_queue->queue, mirroring the
  atlas-weave path's existing guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant