Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

# Build artifacts
build/
build-*/
_package/
out/
*.user
Expand Down
43 changes: 28 additions & 15 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@ set(CMAKE_C_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

if(NOT WIN32)
message(FATAL_ERROR "The Leia SR plug-in is Windows-only.")
if(NOT WIN32 AND NOT ANDROID)
message(FATAL_ERROR
"The Leia plug-in supports two targets:\n"
" - Windows / Leia SR (DisplayXR-LeiaSR.dll, drv_leia)\n"
" - Android / Leia CNSDK (libdxrp050_leia_cnsdk.so, drv_leia_android)\n"
"Configured platform is neither WIN32 nor ANDROID.")
endif()

if(NOT DEFINED BUILD_NUM)
Expand Down Expand Up @@ -64,17 +68,26 @@ else()
FetchContent_MakeAvailable(displayxr_runtime)
endif()

# --- SR SDK -------------------------------------------------------------
if(NOT DEFINED ENV{LEIASR_SDKROOT})
message(FATAL_ERROR
"LEIASR_SDKROOT env var not set. Point at an extracted LeiaSR-SDK-*-win64 directory. "
"scripts/build-windows.bat fetches it from the sr-sdk-v* release on this repo.")
endif()
set(SR_PATH "$ENV{LEIASR_SDKROOT}")
list(APPEND CMAKE_PREFIX_PATH "${SR_PATH}")
find_package(simulatedreality CONFIG REQUIRED GLOBAL)
find_package(srDirectX CONFIG REQUIRED GLOBAL)
message(STATUS "Leia SR SDK at ${SR_PATH}")
if(WIN32)
# --- SR SDK (Windows / Leia SR plug-in) ----------------------------
if(NOT DEFINED ENV{LEIASR_SDKROOT})
message(FATAL_ERROR
"LEIASR_SDKROOT env var not set. Point at an extracted LeiaSR-SDK-*-win64 directory. "
"scripts/build-windows.bat fetches it from the sr-sdk-v* release on this repo.")
endif()
set(SR_PATH "$ENV{LEIASR_SDKROOT}")
list(APPEND CMAKE_PREFIX_PATH "${SR_PATH}")
find_package(simulatedreality CONFIG REQUIRED GLOBAL)
find_package(srDirectX CONFIG REQUIRED GLOBAL)
message(STATUS "Leia SR SDK at ${SR_PATH}")

add_subdirectory(src/drv_leia)
add_subdirectory(installer)
add_subdirectory(src/drv_leia)
add_subdirectory(installer)
elseif(ANDROID)
# --- CNSDK (Android / Leia CNSDK plug-in) --------------------------
# CNSDK setup (find_package(CNSDK CONFIG)) lives inside
# src/drv_leia_android/CMakeLists.txt — it consumes CNSDK_ROOT,
# which the build environment supplies. No NSIS installer on
# Android; the .so ships in the runtime APK's jniLibs/<ABI>/.
add_subdirectory(src/drv_leia_android)
endif()
113 changes: 113 additions & 0 deletions docs/cnsdk-android-calibration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# CNSDK Android Calibration Procedure

The Android POC's CNSDK display processor makes three assumptions about
CNSDK conventions that the SDK docs leave ambiguous. Once a Lume Pad is
attached, validate each in this order — they're listed cheapest-to-test
first. Audit references in brackets.

## 1. Face position axis convention [B15]

**What's assumed:** in `src/drv_leia_android/leia_cnsdk.cpp::leia_cnsdk_get_primary_face`,
CNSDK's `leia_core_get_primary_face` returns a 3-float position in
millimeters with axes x=right / y=up / z=toward viewer, origin at the
camera. The wrapper converts mm→m and subtracts the cached
`cameraCenterX/Y/Z` to land in `xrt_eye_position`'s display-relative
frame.

**What CNSDK actually documents** (cnsdk/include/leia/headTracking/common/types.h):
- `leia_headtracking_detected_face.posePosition` is "Head location in
mm. The origin point is the location of the camera." ✓ matches.
- `leia_headtracking_face.point.pos` is "3D coordinate with camera
transform and Kalman filter applied." Axis convention not stated.
- `posePoseAngle` is "in radians. **The rotation is a left handed
coordinate system.**" Strong hint that positions may also follow a
left-handed system.

**Test:** stand directly in front of the display, motionless. Note the
last reported eye position in logcat (look for `xrLocateViews` HUD
output or any debug printf you add). Then move your head 10 cm
**right**. Re-read.

| Result | What it means | Fix |
|---|---|---|
| x increased by ~0.10 | Right-handed, x=right. Convention matches. ✓ | — |
| x decreased by ~0.10 | Left-handed, x=right OR right-handed with x=left | Flip sign of `out_x` in `leia_cnsdk_get_primary_face` |
| y changed by ~0.10 | Axes are rotated; CNSDK uses landscape vs portrait orientation differently | Swap x/y, possibly negate one |

Repeat for **down** (y) and **toward display** (z).

If the cube appears to track the wrong direction, this is the first
thing to check.

## 2. Tile-to-eye mapping [B17]

**What's assumed:** the SBS atlas's tile (col 0, row 0) is the **left
eye** view and (col 1, row 0) is the **right eye**. The DP's
`blit_atlas_to_per_view` blits tile 0 → `view_img[0]`, tile 1 →
`view_img[1]`. `leia_cnsdk_weave` then calls
`leia_interlacer_vulkan_set_view_for_texture_array(interlacer, 0, view_iv[0])`
and `(interlacer, 1, view_iv[1])` — passing left as view-0, right as
view-1.

**What CNSDK actually does:** undocumented. CNSDK's interlacer treats
view 0 and view 1 as left and right per common convention, but this
isn't explicit in the headers.

**Test:** display the [Khronos OpenXR cube test pattern](https://github.com/KhronosGroup/OpenXR-SDK-Source/tree/main/src/tests/hello_xr)
or any stereo-pair where left and right eyes have visually distinct
content (e.g., a number "1" rendered to left view, "2" to right). Look
at the Lume Pad with one eye covered, then the other.

| What you see | Meaning | Fix |
|---|---|---|
| "1" with left eye, "2" with right | Convention matches. ✓ | — |
| "2" with left eye, "1" with right | Tile-to-eye mapping is swapped | In `leia_cnsdk_weave`, swap the args to `set_view_for_texture_array` (pass right as 0, left as 1). Or in the DP, swap the destination index in `blit_atlas_to_per_view`. |
| Both eyes see "1" and "2" overlaid as a ghost | Interlacing pattern broken — likely a calibration issue, not view assignment | Check `leia_core_get_device_config` is returning the right `displayDots` — out of scope of this doc |

## 3. UV vertical flip [B18]

**What's assumed:** `leia_cnsdk.cpp::leia_cnsdk_weave` calls
`leia_interlacer_set_flip_input_uv_vertical(interlacer, true)` because
Vulkan's NDC is Y-down (origin at top-left) and CNSDK is presumed to
default to OpenGL convention (Y-up, origin bottom-left).

**Test:** display any content with clear top/bottom asymmetry — text
is ideal. A rendered string "HELLO" or just a single capital "A" works.

| What you see | Meaning | Fix |
|---|---|---|
| Right-side-up "HELLO" | UV flip convention is correct ✓ | — |
| Upside-down "OLLEH" / "HELLO" vertically mirrored | CNSDK is already flipping; we're double-flipping | Change `leia_interlacer_set_flip_input_uv_vertical(interlacer, true)` to `false` in `leia_cnsdk.cpp::leia_cnsdk_weave` |
| Letters look correct but offset / cropped | Different bug — not a flip issue. Check view_width / view_height accounting | Out of scope of this doc |

## Combined check

Once all three are pinned down, the cube test app (when it lands —
B13d) should:
1. Render a spinning cube at the depth corresponding to the wearer's
head distance.
2. Move convincingly with head motion (left-right strafing).
3. Have correct horizontal disparity (look closer with one eye than the
other).
4. Show right-side-up text/numbers on debug overlays.

If 1+3 are off, it's #1 (face axes). If 2 is off, it's #1 or #2 (eye
mapping). If 4 is off, it's #3 (UV flip).

## Where the convention assumptions live

| Assumption | File | Function | Audit ref |
|---|---|---|---|
| Face position axes / units | `src/drv_leia_android/leia_cnsdk.cpp` | `leia_cnsdk_get_primary_face` | B15 |
| Tile-to-eye mapping | `src/drv_leia_android/leia_cnsdk.cpp` | `leia_cnsdk_weave` (`set_view_for_texture_array`) | B17 |
| UV vertical flip | `src/drv_leia_android/leia_cnsdk.cpp` | `leia_cnsdk_weave` (`set_flip_input_uv_vertical`) | B18 |

All three are 1–3 line changes once you know the correct values.

## Why these aren't fixed pre-emptively

Each is a 50/50 coin flip in either direction. Picking wrong is worse
than picking right (off-by-180° on axes is more confusing than
matching the spec assumption). Lume Pad on-device test is the
definitive answer; this doc lets you arrive at hardware ready to flip
the right knob.
111 changes: 111 additions & 0 deletions src/drv_leia_android/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Copyright 2026, Leia Inc / DisplayXR
# SPDX-License-Identifier: BSL-1.0
#
# DisplayXR Leia CNSDK plug-in (Android).
#
# Builds the CNSDK-backed display-processor as a shared library named
# `libdxrp050_leia_cnsdk.so`, matching the runtime's Android plug-in
# discovery convention (libdxrp<NNN>_<id>.so, where NNN is the
# probe-order and <id> matches the iface's `id` field). See
# `docs/specs/runtime/plugin-discovery.md` §3.2 in the runtime tree.
#
# The .so is meant to be dropped into the runtime APK's
# `jniLibs/<ABI>/` so the runtime's target_plugin_loader.c (PR #309)
# discovers it via dladdr-of-self + dirname enumeration at
# `xrCreateInstance` time.

if(NOT ANDROID)
message(FATAL_ERROR "drv_leia_android is Android-only.")
endif()

# --- CNSDK ----------------------------------------------------------------
#
# CNSDK ships as an AAR + extracted SDK tree (headers + .so + cmake
# config). The expected layout matches what was previously consumed by
# the runtime POC's openxr_android/build.gradle CNSDK_AAR rule:
#
# ${CNSDK_ROOT}/include/leia/sdk/*.h
# ${CNSDK_ROOT}/share/cmake/CNSDK/CNSDKConfig.cmake
# ${CNSDK_ROOT}/lib/<ABI>/libleiaSDK-faceTrackingInApp.so
#
# CI / dev set CNSDK_ROOT to the extracted release tree. Pinned to
# CNSDK 0.7.28 for the POC.

if(NOT DEFINED CNSDK_ROOT)
if(DEFINED ENV{CNSDK_ROOT})
set(CNSDK_ROOT "$ENV{CNSDK_ROOT}")
else()
message(FATAL_ERROR
"CNSDK_ROOT not set. Extract CNSDK 0.7.28 and set "
"CNSDK_ROOT (CMake var or env var) to the extracted dir. "
"Source: https://github.com/LeiaInc/leiainc.github.io/tree/master/CNSDK/cnsdk-android-0.7.28.zip")
endif()
endif()
# Android NDK toolchain defaults CMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY,
# which restricts find_package to NDK sysroot dirs and ignores
# CMAKE_PREFIX_PATH outside that. Flip it to BOTH so the CNSDK install
# at ${CNSDK_ROOT} is searchable (matches the runtime APK's gradle
# cmake.arguments setup).
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)
list(APPEND CMAKE_PREFIX_PATH "${CNSDK_ROOT}")
find_package(CNSDK CONFIG REQUIRED)
message(STATUS "Leia CNSDK at ${CNSDK_ROOT}")

# --- Plug-in target -------------------------------------------------------

set(DRV_LEIA_ANDROID_SOURCES
leia_cnsdk.cpp
leia_cnsdk.h
leia_display_processor_cnsdk.cpp
leia_display_processor_cnsdk.h
leia_plugin_android.c
)

add_library(dxrp050_leia_cnsdk SHARED ${DRV_LEIA_ANDROID_SOURCES})

target_include_directories(dxrp050_leia_cnsdk PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}
)

target_link_libraries(dxrp050_leia_cnsdk
PRIVATE
# Runtime aux surface (consumed via FetchContent or local
# checkout per top-level CMakeLists.txt).
xrt-interfaces
aux_util
aux_os
aux_math
aux_vk
# CNSDK Vulkan interlacer + face-tracker.
CNSDK::leiaSDK
# Android system libs.
log
android
vulkan
)

# Debug-log + ATrace gating: when XRT_DEBUG_ANDROID_VERBOSE is defined,
# DXR_HW_DBG / DXR_ATRACE blocks compile in. Release builds drop them
# all. The runtime APK's Gradle Debug variant sets this; Release does
# not. For standalone CMake builds, pass -DXRT_DEBUG_ANDROID_VERBOSE=1
# on the cmake command line.

set_target_properties(dxrp050_leia_cnsdk PROPERTIES
OUTPUT_NAME "dxrp050_leia_cnsdk"
PREFIX "lib" # libdxrp050_leia_cnsdk.so (matches discovery convention)
C_VISIBILITY_PRESET hidden
CXX_VISIBILITY_PRESET hidden
VISIBILITY_INLINES_HIDDEN ON
)

# Single-symbol export discipline (plug-in discovery contract §4):
# only xrtPluginNegotiate should be in the dynamic symbol table.
# C_/CXX_VISIBILITY_PRESET=hidden only affects this target's own
# sources, not symbols pulled in from linked static libs (aux_util,
# aux_vk, jni glue, etc.). --exclude-libs,ALL applies a "hidden"
# visibility filter to ALL static libs, so their symbols stay
# private to this .so even if compiled with default visibility.
# XRT_PLUGIN_EXPORT on xrtPluginNegotiate keeps it explicitly visible.
if(ANDROID OR UNIX)
target_link_options(dxrp050_leia_cnsdk PRIVATE "LINKER:--exclude-libs,ALL")
endif()
60 changes: 60 additions & 0 deletions src/drv_leia_android/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# drv_leia_android — DisplayXR Leia CNSDK Plug-in

The Android variant of the Leia display-processor plug-in. Wraps the
**Leia Computational Display SDK (CNSDK)** as an `xrt_display_processor`
implementation, shipping as `libdxrp050_leia_cnsdk.so`.

The runtime's `target_plugin_loader.c` Android branch (PR #309 / commit
`c96c93ce8`) discovers this `.so` at `xrCreateInstance` time via
`dladdr`-of-self + dirname enumeration of files matching the
`libdxrp<NNN>_<id>.so` convention. Bundle this `.so` into the runtime
APK's `jniLibs/<ABI>/` (single-vendor mode); multi-APK discovery is v2
(tracked at runtime #310).

## Files

| File | Purpose |
|---|---|
| `leia_cnsdk.{cpp,h}` | C++ wrapper around the CNSDK C ABI: core init, interlacer lifecycle, face-tracking worker thread, atlas weave entry point, device-config caching. |
| `leia_display_processor_cnsdk.{cpp,h}` | `xrt_display_processor` vtable wired to the wrapper. Advertises `is_self_submitting=true` so the compositor flushes its pre-DP cmd buffer + skips its own post-DP submit. Atlas mode only (per-tile blit removed — CNSDK splits the SBS atlas internally via `set_interlace_view_texture_atlas`). |
| `leia_plugin_android.c` | `xrtPluginNegotiate` entry point + `xrt_plugin_iface` vtable. `create_dp_vk` is the only non-NULL factory slot. |

## Build

```bash
# CNSDK 0.7.28 extracted somewhere on disk:
export CNSDK_ROOT=/path/to/cnsdk

# Configure for arm64-v8a Android via the NDK toolchain:
cmake -S . -B build-android \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=arm64-v8a \
-DANDROID_PLATFORM=android-29 \
-G Ninja

cmake --build build-android --target dxrp050_leia_cnsdk
```

Output: `build-android/src/drv_leia_android/libdxrp050_leia_cnsdk.so`.

Copy that into your runtime APK's
`src/xrt/targets/openxr_android/src/main/jniLibs/arm64-v8a/` before
running `./gradlew :src:xrt:targets:openxr_android:assembleInProcessDebug`.

(End-to-end multi-module Gradle integration is a follow-up — see the
top-level repo README's "Android — POC build flow" section.)

## CNSDK convention assumptions

Three plug-in-side conventions are assumed in this POC (face axis
signs/units, tile-to-eye mapping, UV vertical flip). All three may
need single-line flips after first hardware bring-up — see
[`docs/cnsdk-android-calibration.md`](../../docs/cnsdk-android-calibration.md)
for the symptom-→-fix table.

## Status

POC — not yet validated on a Lume Pad. Compiles clean on the host.
First hardware install is gated on Lume Pad arrival; bring-up plan
lives in the runtime repo at
`docs/getting-started/android-bringup-checklist.md`.
Loading
Loading