DisplayXR — Unreal Plugin

Unreal Engine plugin for rendering on eye-tracked 3D light field displays via the DisplayXR OpenXR runtime. Works with any OpenXR-compatible 3D display.

Table of Contents

• Overview
• Requirements
• Installing the Plugin
• Scene Setup
  • Camera-Centric Mode
  • Display-Centric Mode
• Building Your App
• Documentation
• Related Repositories

New to the plugin? Start with the Display Rig Setup guide — it covers the pawn and camera configuration needed for stereo 3D, plus the differences between the two rig modes. For outstanding work and planned parity with the Unity sibling plugin, see TODO.

---

Overview

The plugin hooks into Unreal's OpenXR pipeline to provide:

• Eye-tracked stereo rendering — Kooima asymmetric frustum projection driven by real-time eye positions from the DisplayXR runtime
• Two stereo rig modes — camera-centric (add to an existing pawn camera) or display-centric (place a virtual display in the scene)
• Zero-copy atlas handoff — UE renders directly into the OpenXR swapchain (see AtlasHandoff)
• Editor preview — a standalone OpenXR session in the editor so you can see stereo output without running PIE (see EditorPreview)

One unified session loads the DisplayXR OpenXR runtime directly on every platform; UE's OpenXR plugin is not a dependency. Platform differences (Windows D3D12, macOS Metal, Android Vulkan) live inside the session and compositor, not in a product-level compile flag. See Architecture.md for the full picture.

Requirements

• Unreal Engine 5.3 or later
• DisplayXR OpenXR runtime installed (see openxr-3d-display)
• Windows, macOS, or Android target
• Visual Studio (Windows) or Xcode (macOS) matching your UE version

Installing the Plugin

1. Create a UE project (or open an existing one).
2. Make a Plugins folder next to the .uproject if one doesn't exist.
3. Clone this repository to Plugins/DisplayXR.
4. Regenerate project files and build the project from your IDE.

Scene Setup

Camera-Centric Mode

Attach a UDisplayXRCamera component to your pawn's camera. Set ConvergenceDistance, LookaroundFactor, and BaselineFactor as desired. The rig rotates with the camera.

Display-Centric Mode

Attach a UDisplayXRDisplay component to an actor representing a virtual display in the scene. Stereo is computed relative to the display's world transform.

See DisplayRigSetup for the full walkthrough.

Building Your App

Use the provided script to package the plugin against a target UE version:

Scripts\PackagePlugin.bat 5.6

Output lands in Packages/DisplayXR_<version>/. Drop the packaged plugin into your shipping project's Plugins/ folder.

For per-project builds, use UAT directly (see Scripts/PackageApp.py for a reference invocation).

Documentation

In-depth docs live in Docs/DisplayXR/:

• QuickStart — install runtime → install plugin → press Play
• Architecture — one-page class hierarchy, ownership, per-frame flow
• AtlasHandoff — zero-copy UE → OpenXR swapchain pipeline
• DisplayRigSetup — pawn/camera rig configuration, input, rig modes
• EditorPreview — current SceneCapture2D-based preview
• EditorPreviewNative — in-flight plan for native FDisplayXRDevice → PIE preview
• EyeTracking — xrLocateViews → Kooima → per-view projection pipeline
• MacSetup — macOS-specific setup quirks
• CompositorIntegration — prior compositor design (historical)
• ADRs — decision records for load-bearing architectural choices
• TODO — outstanding work, parity items, in-flight branches

Related Repositories

• dfattal/openxr-3d-display — DisplayXR OpenXR runtime. Extensions: XR_EXT_display_info, XR_EXT_win32_window_binding, XR_EXT_cocoa_window_binding.
• DisplayXR/displayxr-unity — Unity sibling plugin. Reference implementation for rig patterns, editor preview, and shared Kooima C libraries.