Android: link CNSDK in-app variant + bundle transitive .so deps into runtime APK#14
Open
leaiss wants to merge 7 commits into
Open
Android: link CNSDK in-app variant + bundle transitive .so deps into runtime APK#14leaiss wants to merge 7 commits into
leaiss wants to merge 7 commits into
Conversation
This was referenced May 27, 2026
…me PR #268)
Companion to displayxr-runtime PR #268, which splits the Android POC
along the post-#263 plug-in boundary. Runtime side keeps the cross-
cutting infrastructure (is_self_submitting vtable flag, vk_native
Android plumbing, audit fixes); CNSDK + drv-side audit fixes live
here.
Adds `src/drv_leia_android/` as a sibling of the Windows drv_leia/,
sharing the same plug-in entry contract (xrtPluginNegotiate) but
selected at CMake configure time by `if(WIN32)` / `elseif(ANDROID)`.
Source files
leia_cnsdk.{cpp,h} — CNSDK C-ABI wrapper:
core init (worker thread),
interlacer lifecycle,
atlas weave, face-tracking
snapshot. Audit fixes
B2/B4/B5/B10/B11/B12.
leia_display_processor_cnsdk.{cpp,h} — xrt_display_processor vtable.
is_self_submitting=true,
atlas-mode only (no per-
tile blit), DXR_HW_DBG +
DXR_ATRACE blocks gated
on XRT_DEBUG_ANDROID_VERBOSE.
Audit B7 / B8 / B14.
leia_plugin_android.c — entry point + iface. Only
create_dp_vk is non-NULL
(no D3D/Metal/GL on Android).
id = "leia-cnsdk",
probe always succeeds (POC
pattern, async device init).
Build wiring
CMakeLists.txt (top-level) — relax `if(NOT WIN32) FATAL_ERROR`
into `if(NOT WIN32 AND NOT
ANDROID)`; branch
add_subdirectory between
src/drv_leia (Windows) and
src/drv_leia_android (Android).
Skip the NSIS installer on
Android (APK is the install).
src/drv_leia_android/CMakeLists.txt — new. find_package(CNSDK CONFIG
REQUIRED) on CNSDK_ROOT;
builds libdxrp050_leia_cnsdk.so
matching the runtime's plug-in
filename convention; symbol-
hidden except xrtPluginNegotiate.
Docs
docs/cnsdk-android-calibration.md — moved from displayxr-runtime
docs/cnsdk-axis-calibration
branch per PR #271 plan, with
file-path refs rewritten to
point at src/drv_leia_android/.
Status: compiles in the standalone repo; not yet wired into the runtime
APK's Gradle (manual jniLibs/ drop per README until the multi-module
gradle setup lands). First hardware install gated on Lume Pad arrival.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ndroid
The Android NDK toolchain (android.toolchain.cmake) defaults
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to ONLY, which restricts find_package()
to NDK sysroot dirs and silently ignores anything we add to
CMAKE_PREFIX_PATH outside that. Result:
find_package(CNSDK CONFIG REQUIRED)
found nothing even with CNSDK_ROOT set + appended to CMAKE_PREFIX_PATH,
breaking the standalone NDK build with "Could not find a package
configuration file provided by 'CNSDK'".
Flip the mode to BOTH at the top of src/drv_leia_android/CMakeLists.txt
so the CNSDK install at CNSDK_ROOT becomes searchable. Mirrors what the
runtime APK's gradle does via cmake.arguments
"-DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH".
Also widen .gitignore to cover build-*/ so out-of-tree builds like
build-android/ don't end up staged.
Verified end-to-end:
cmake -B build-android -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake \
-DCMAKE_MAKE_PROGRAM=$NDK_CMAKE/bin/ninja.exe \
-DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-29 \
-DCNSDK_ROOT=$RUNTIME/cnsdk \
-DDXR_RUNTIME_SOURCE_DIR=$RUNTIME \
-DEigen3_DIR=$RUNTIME/.../intermediates/eigen/eigen-3.4.0/cmake
cmake --build build-android --target dxrp050_leia_cnsdk
Output: libdxrp050_leia_cnsdk.so (1.8 MB, xrtPluginNegotiate exported)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Android CNSDK display-processor factory wired six optional vtable slots (on_pause, on_resume, is_self_submitting, get_predicted_eye_positions, get_display_dimensions, get_display_pixel_info) but never set the struct_size header introduced by ABI v2. The impl is calloc'd, so it stayed 0 — under the v2 runtime, XRT_DP_HAS_SLOT bounds every optional slot against struct_size and reads them all as absent. Most damaging: is_self_submitting=true would be ignored, regressing the self-submitting atlas path into a per-frame double submit. Mirrors the drv_leia (Windows) migration in a6c0d7d. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
leaiss
force-pushed
the
feat/android-cnsdk-dp-bundle-transitive
branch
from
54a525f to
e76cb87
Compare
… mono NULL-guard Post-ABI-v2 review pass: - leia_plugin_android.c: make the probe_displays omission deliberate with an explicit `.probe_displays = NULL` + rationale. struct_size spans the full v2 iface, so the runtime's gate sees the slot, finds NULL, and uses the synthesized primary claim — correct for a single-panel Android device with no monitor enumeration (Windows arm implements it over EDID; Android has none). - leia_display_processor_cnsdk.h: fix stale doc that described the obsolete per-tile-blit path; current code is atlas mode (SBS VkImage handed straight to leia_cnsdk_weave, no per-view blit). - leia_display_processor_cnsdk.cpp: NULL-guard vk / vk->main_queue in the mono 1x1 passthrough fallback before dereferencing main_queue->queue, mirroring the atlas-weave path's existing guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Plug-in discovery contract §4 ("single-symbol export discipline"):
plug-ins must expose only xrtPluginNegotiate via the dynamic symbol
table. The existing C_/CXX_VISIBILITY_PRESET=hidden setting only hides
symbols compiled from THIS target's own sources — symbols pulled in
from linked static libs (aux_util, aux_vk, aux_android jni glue, etc.)
keep whatever visibility they had when compiled, which defaults to
"default" everywhere.
Result before this commit:
$ llvm-nm -D --defined-only --extern-only libdxrp050_leia_cnsdk.so | wc -l
1233 (one intentional + 1232 leaked from static libs)
After:
$ llvm-nm -D --defined-only --extern-only libdxrp050_leia_cnsdk.so | wc -l
1
$ llvm-nm -D --defined-only --extern-only libdxrp050_leia_cnsdk.so
0000000000036970 T xrtPluginNegotiate
LINKER:--exclude-libs,ALL applies a "hidden" visibility filter to every
archive linked into the .so, so static-lib symbols stay private even
when compiled with default visibility. The XRT_PLUGIN_EXPORT macro on
xrtPluginNegotiate marks it as explicitly visible so the runtime
loader's dlsym still resolves it.
Gated on (ANDROID OR UNIX) since --exclude-libs,ALL is a GNU ld /
lld flag. Windows uses /EXPORT def-file discipline elsewhere.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps the multi-arg cmake invocation needed to produce
libdxrp050_leia_cnsdk.so for Android arm64-v8a. Without this script,
a bring-up dev has to remember:
- NDK toolchain file path
- Ninja binary path inside the SDK's bundled cmake (not on PATH)
- CNSDK_ROOT (extracted release tree, not the source checkout)
- DXR_RUNTIME_SOURCE_DIR (sibling runtime checkout)
- Eigen3_DIR (gradle-fetched stub config under the runtime tree)
- ABI + platform settings
All of these drift across machines. The script auto-resolves each from
sensible defaults relative to the script's location, with explicit env-
var overrides documented at the top. Hard failure with actionable error
messages if any required dep is missing (NDK version not installed,
CNSDK not extracted, runtime checkout not next to the plug-in, etc.).
Mirrors scripts/build-windows.bat's shape — same one-line usage pattern,
same env-var convention. Adds a [target] arg supporting:
- dxrp050_leia_cnsdk (default) — build the plug-in .so
- clean — wipe build-android/
Verified end-to-end:
$ scripts/build-android.sh clean
Wiping build-android/
$ scripts/build-android.sh dxrp050_leia_cnsdk
...
=== Built: build-android/src/drv_leia_android/libdxrp050_leia_cnsdk.so ===
-rw-r--r-- 1.7M libdxrp050_leia_cnsdk.so
Drop into the runtime APK's jniLibs/<ABI>/:
cp build-android/src/drv_leia_android/libdxrp050_leia_cnsdk.so ...
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes to make the plug-in self-sufficient on a stock-Android
device (the emulator validation pass surfaced both):
1. src/drv_leia_android/CMakeLists.txt — link
CNSDK::leiaSDK-faceTrackingInApp (in-process face tracking)
instead of CNSDK::leiaSDK (which CNSDKConfig.cmake aliases to
the faceTrackingInService variant, requiring a separate Leia
face-tracking service running). CNSDK 0.7.28's Targets.cmake
only exposes the InService variant — define our own IMPORTED
target around the InApp .so file (which DOES exist on disk at
${CNSDK_ROOT}/lib/<ABI>/libleiaSDK-faceTrackingInApp.so).
Reuse the existing InService target's INTERFACE_INCLUDE_DIRECTORIES
so includes resolve.
2. scripts/build-android.sh — new `install-runtime-jnilibs` target.
Builds the plug-in .so and ALSO copies the CNSDK transitive .so
deps (extracted from the InApp AAR + SNPE third-party AAR) into
the runtime APK's jniLibs/<ABI>/ alongside the plug-in. Without
this, dlopen of libdxrp050_leia_cnsdk.so fails at runtime with
"library libleiaSDK-faceTrackingInApp.so not found" (and then
libblink.so, libSNPE.so, ...). Bundling the full transitive set
into the runtime APK = 16 .so files total, ~80 MB added.
Usage:
scripts/build-android.sh install-runtime-jnilibs
cd <runtime-checkout>
./gradlew :src:xrt:targets:openxr_android:assembleInProcessDebug --rerun-tasks
De-duplicates libc++_shared.so (CNSDK ships its own copy; the
runtime build already ships one — version skew between the two
causes dlopen failures).
Known limitation surfaced during emulator validation: even with all
transitive .so files bundled into the runtime APK, the Android linker
fails dlopen of the plug-in's DT_NEEDED libs because the dlopen
happens in the TEST APP's classloader namespace (clns-7), which only
sees the test app's own /data/app/<test-pkg>/lib/<ABI>/. The fix is
in target_plugin_loader.c (runtime side): use android_dlopen_ext
with a custom namespace whose library_search_paths includes the
RUNTIME APK's lib dir. That's a runtime-side change tracked
separately. On Lume Pad the namespace issue probably doesn't apply
because Leia's preinstalled OS likely places CNSDK at a system path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
leaiss
force-pushed
the
feat/android-cnsdk-dp-bundle-transitive
branch
from
e76cb87 to
cfef49d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacks on PRs #5, #6, #7. Two changes that together make the plug-in self-sufficient on a stock Android device (no system-shipped CNSDK).
1. Link CNSDK in-app variant, not in-service
`src/drv_leia_android/CMakeLists.txt`: switch from `CNSDK::leiaSDK` (which CNSDK 0.7.28's `CNSDKConfig.cmake` aliases to the `faceTrackingInService` variant — requires a separate Leia service running) to a custom `CNSDK::leiaSDK-faceTrackingInApp` target around the in-app .so file. The in-app variant runs face tracking in-process, which is the whole POC plan (single-app, in-process).
CNSDK 0.7.28's `CNSDKTargets.cmake` only defines the InService variant; the InApp .so file exists on disk at `${CNSDK_ROOT}/lib//libleiaSDK-faceTrackingInApp.so` but no CMake target wraps it. This PR adds a local IMPORTED target that does, reusing the InService target's `INTERFACE_INCLUDE_DIRECTORIES` (same headers).
2. Bundle CNSDK transitive .so deps via build-android.sh
`scripts/build-android.sh` gets a new `install-runtime-jnilibs` target that:
Total bundle size: ~80 MB added to the runtime APK (24 MB → ~105 MB). All transitive deps the plug-in needs at runtime.
Without this, `dlopen` of `libdxrp050_leia_cnsdk.so` fails with "library libleiaSDK-faceTrackingInApp.so not found" — the plug-in's DT_NEEDED references chain back through libblink → SNPE.
Verification
Discovered + diagnosed end-to-end on an Android-36 emulator with the runtime broker PR (DisplayXR/displayxr-runtime#332) installed. Sequence of failures as I added pieces:
Known limitation — Android linker namespace
Even with all transitive .so files bundled into the runtime APK, the Android linker fails `dlopen` of the plug-in's DT_NEEDED libs because the `dlopen` happens in the test app's classloader namespace (`clns-7`), which only sees the test app's own `/data/app//lib//`. The plug-in is loaded from the runtime APK's lib dir (a different package), and its transitive deps aren't visible from the test app's namespace.
The real fix is in `target_plugin_loader.c` (runtime side, owned by displayxr-runtime): use `android_dlopen_ext` with a custom namespace whose `library_search_paths` includes the runtime APK's lib dir. Tracked as a separate follow-up.
On Lume Pad the namespace issue probably doesn't apply — Leia's preinstalled OS likely places CNSDK at a system path (`/system/lib/` or `/vendor/lib/`) which is always in the default namespace. Bundling here is for emulator + future non-Leia-hardware Android devices.
Test plan
🤖 Generated with Claude Code