Skip to content

Repository files navigation

SplatKit for React Native

Real-time 3D Gaussian splatting in a React Native view, rendered natively on Metal and Vulkan. Point SplatKitView at an .spz file on disk, give it a collider, and walk through the scene.

npm license: MIT platform: iOS | Android architecture: Fabric

Experimental alpha. The API changes before 1.0, and every release so far is a prerelease published under the next dist-tag. Verified on physical devices only: an iPhone 17 Pro and a Xiaomi Mi 9 (Adreno 640).

Contents

Why

  • Native, not a port. Metal on iOS and Vulkan on Android, behind one Fabric component. No WebGL, no WebView, no JS renderer.
  • No bytes cross the bridge. The view reads the world from an absolute local path and never receives splat data from JavaScript.
  • It scales down. Hierarchical LOD and residency streaming with budgets you set, so a 6M splat capture runs on a phone from 2019.
  • It tells you what it did. Every quality request is validated natively and reported back through onCapabilities and onPolicyEvent, with diagnostics for each fallback. Nothing degrades silently.
  • Walking is real collision. Load a collider mesh and the camera becomes a character that stands on floors, is blocked by walls and climbs steps.
  • The UI is yours. The SDK draws no joystick, no HUD and no buttons. It exposes commands and events; you build the controls.

Requirements

Minimum
React Native 0.87.x (peer dependency ~0.87.1)
React ^19.2.3
Architecture New Architecture (Fabric). SplatKitView has no legacy bridge fallback.
iOS 17.0+, Apple GPU family 7 (A14/M1 or newer), Xcode with CocoaPods
Android API 29+, Vulkan 1.1, arm64-v8a only
Node ^22.13.0, ^24.3.0 or >=26.0.0

Run it on a physical device. The renderer's shaders use SIMD prefix reductions, which the iOS Simulator does not implement, so it reports Metal is unavailable on this device and loads nothing. Emulators are not a supported target on either platform.

Installation

npm install @splatkit/react-native@next
cd ios && pod install

The @next tag is required while the package is in alpha.

iOS. The podspec vendors SplatKitCore.xcframework, which is fetched and checksum-verified when the package is packed, so there is nothing to build. iOS 26 terminates apps that skip the UIScene lifecycle, so a React Native 0.87 template app needs a scene delegate; the example app has one.

Android. The module autolinks through the app's com.facebook.react Gradle plugin, which also runs Fabric Codegen, and pulls io.github.xget7:splatkit-android from Maven Central. Set minSdkVersion = 29 and reactNativeArchitectures=arm64-v8a in the host app.

Quick start

import {useMemo, useState} from 'react';
import {StyleSheet} from 'react-native';
import {
  QualityPreset,
  SplatKitBuilder,
  SplatKitView,
  conservativeCapabilities,
  nativeCapabilitiesFromEvent,
  toNativeViewProps,
} from '@splatkit/react-native';

export function Scene({worldPath}: {worldPath: string}) {
  // An engine reports its real limits through onCapabilities, and those arrive only once it
  // exists. Build the first request against conservativeCapabilities, the narrowest limits
  // every shipped adapter accepts, then rebuild when the real ones arrive.
  const [capabilities, setCapabilities] = useState(conservativeCapabilities);

  const configuration = useMemo(
    () =>
      new SplatKitBuilder()
        .withWorld({requestId: 'lobby', filePath: worldPath, maxShDegree: 3})
        .withPreset(QualityPreset.balanced)
        .build(capabilities),
    [worldPath, capabilities],
  );

  return (
    <SplatKitView
      style={StyleSheet.absoluteFill}
      {...toNativeViewProps(configuration)}
      onCapabilities={event =>
        setCapabilities(nativeCapabilitiesFromEvent(event.nativeEvent))
      }
    />
  );
}

That renders the world and lets one finger drag to look around. Walking needs a collider, which Navigation covers.

Do not ask for limits above conservativeCapabilities before the first onCapabilities arrives. A request an adapter rejects is refused before any engine exists, so no capabilities event ever follows and the view has no way to tell you what it wanted instead.

Loading a world

filePath must be an absolute, readable local path, never a URL, a require() asset or a content URI. Supported formats are .spz, .ply, .lodsplat and tiled worlds that stream progressively.

world is a transaction, not a setting. Native reloads when requestId changes and ignores every other edit to the object, so changing a load-time budget means changing the id too.

Getting a world file onto a device

Worlds are not bundled with the app. Push one into the app's sandbox and point filePath at it.

# Android, into the app's external files directory
adb push world.spz /sdcard/Android/data/<your.application.id>/files/world.spz

# iOS, into the app's Documents directory on a connected device
xcrun devicectl device copy to --device <device-id> --domain-type appDataContainer \
  --domain-identifier <your.bundle.id> --source world.spz --destination Documents/world.spz

Restart the app after copying a new world.

Navigation

The SDK ships no navigation UI. SplatKitView handles a one-finger drag to look and a double tap to toggle the gyroscope; every other control is yours to draw.

  1. Load a collider mesh through the collider prop, {requestId, filePath}, pointing at an absolute path to a collider GLB.
  2. Show your controls when onColliderEvent reports ColliderPhase.ready. ColliderPhase.failed carries errorCode and message.
  3. Shape the walker with the character prop, {eyeHeight, bodyRadius, stepHeight} in meters, applied immediately and to any collider loaded later.
  4. Drive the camera with SplatKitCommands.

Commands go straight to the native view and bypass React's render and commit cycle, so a stick can steer at touch rate without a re-render per frame.

import {useCallback, useRef, useState} from 'react';
import {PanResponder, View} from 'react-native';
import {ColliderPhase, SplatKitCommands, SplatKitView} from '@splatkit/react-native';

const WALK_SPEED = 1.4; // meters per second at full deflection
const RADIUS = 62;
const clamp = (value: number) => Math.max(-1, Math.min(1, value));

export function WalkableScene(props: {colliderPath: string}) {
  const view = useRef<React.ComponentRef<typeof SplatKitView>>(null);
  const [walking, setWalking] = useState(false);

  const stick = useRef(
    PanResponder.create({
      onStartShouldSetPanResponder: () => true,
      onPanResponderMove: (_event, gesture) => {
        const target = view.current;
        if (!target) return;
        SplatKitCommands.setWalkVelocity(
          target,
          clamp(-gesture.dy / RADIUS) * WALK_SPEED,
          clamp(gesture.dx / RADIUS) * WALK_SPEED,
        );
      },
      onPanResponderRelease: () => {
        const target = view.current;
        if (target) SplatKitCommands.setWalkVelocity(target, 0, 0);
      },
    }),
  ).current;

  const onColliderEvent = useCallback(
    (event: {nativeEvent: {phase: ColliderPhase}}) =>
      setWalking(event.nativeEvent.phase === ColliderPhase.ready),
    [],
  );

  return (
    <>
      <SplatKitView
        ref={view}
        collider={{requestId: 'floor', filePath: props.colliderPath}}
        character={{eyeHeight: 1.5, bodyRadius: 0.35, stepHeight: 0.35}}
        onColliderEvent={onColliderEvent}
      />
      {walking && <View {...stick.panHandlers} style={{width: RADIUS * 2, height: RADIUS * 2}} />}
    </>
  );
}

An empty collider.requestId releases walk mode and returns the camera to free look.

Quality and performance

SplatKitBuilder resolves what you ask for against what the host and the adapter allow, and returns a SplatKitConfiguration carrying world, render and performance (requested, effective, diagnostics).

import {
  QualityPreset,
  SplatKitBuilder,
  toNativePolicyProp,
  toNativeViewProps,
} from '@splatkit/react-native';

const configuration = new SplatKitBuilder()
  .withWorld({requestId: 'lobby-1', filePath: '/absolute/path/lobby.spz', maxShDegree: 3})
  .withPreset(QualityPreset.high)
  .withPerformance({lodBudgetSplats: 2_500_000})
  .build(capabilities);

// <SplatKitView {...toNativeViewProps(configuration)} policy={toNativePolicyProp(configuration, revision)} />
  • withWorld() takes world identity, file path and the load-time maximum SH degree.
  • withRender() takes paused.
  • withPreset(preset) replaces the policy with one of the presets below.
  • withPerformance(options) overrides fields on top of the current preset and marks the policy manual.
  • build(capabilities) clamps budgets to the limits, caps shDegree at the world's maxShDegree, and records every fallback in performance.diagnostics. Malformed numbers, invalid enums and unknown options throw.

Presets

Every preset rasterizes in hardware, with frustum culling and early termination on.

Preset Render scale SH LOD budget Residency Tile LOD error px Alpha Sub-pixel Hi-Z Sort
highEnd 1.25 3 4,000,000 4,000,000 16 0.75 1/255 0.35 Yes 32
high 1.0 3 3,000,000 3,000,000 16 1.0 1/255 0.5 Yes 32
balanced 0.85 2 2,000,000 2,000,000 16 1.25 1/255 0.65 No 16
performance 0.65 1 1,000,000 1,000,000 8 2.0 2/255 1.0 No 16

Lower lodErrorPixels sharpens the far field, which is where a captured room looks soft, and costs fill rate. withPerformance({raster: RasterStrategy.hybrid}) opts into experimental screen-tile compositing, which helps only where many large translucent splats overlap one pixel and costs more on sparse scenes. Only the iOS adapter builds hybrid today, and computeTile is not implemented anywhere.

Splat budgets

Knob Set through Effect
lodCapacitySplats withPerformance({lodBudgetSplats}) LOD tree budget. 0 disables tree building. Load-time.
residencyCapacitySplats withPerformance({residencyCapacitySplats}) Resident streaming splats, not a memory guarantee. Load-time.
maxShDegree withWorld() Load-time cap on stored spherical-harmonics degree.
shDegree withPerformance({shDegree}) Draw-time SH degree, 0 to 3, clamped to maxShDegree.
renderScale withPerformance({renderScale}) Render resolution scale, [0.1, 2].

The first two are read while the engine builds the world, so raising one takes a new requestId and a reload. Everything else updates in place. classifyPolicyChange(previous, next) tells you which case you are in before you commit to it.

The policy prop and revisions

policy carries the live renderer policy under a revision. Native revalidates the whole policy and never trusts JavaScript state, so give every change a new positive revision; 0 or less means no policy. Native reapplies the current revision to each new engine, and a world reload creates one, so expect onPolicyEvent again after a reload.

onCapabilities fires once per engine, before that engine's first policy event, and reports the limits, the GPU features and exactly which policy fields the adapter applies. onPolicyEvent reports PolicyPhase.applied, PolicyPhase.warning (a field fell back, message says why) or PolicyPhase.rejected (the previous policy stayed in effect), and always carries the effective values.

API reference

SplatKitView props

Extends the standard ViewProps.

Prop Type Default Description
world WorldRequest - Load transaction: requestId, absolute filePath, maxShDegree, lodCapacitySplats, residencyCapacitySplats. Omit to render nothing.
collider ColliderRequest - {requestId, filePath}. An empty requestId releases walk mode.
character Character - {eyeHeight, bodyRadius, stepHeight} in meters.
policy NativeRenderPolicy - The requested renderer policy, under a revision.
paused boolean false Freezes rendering.
renderScale number 1 Render resolution scale.
shDegree number (0-3) 3 Draw-time spherical-harmonics degree.
linearBlending boolean false Blends in linear light instead of the encoded space training used.
cullMarginDegrees number 10 Angular culling margin for the CPU fallback; the GPU path uses projected bounds.
motionEnabled boolean false Drives the camera from the gyroscope. Ignored where the sensor is missing.
touchLookEnabled boolean true Whether a one-finger drag turns the camera.
lookSensitivity number 0.004 Radians per point dragged.
cameraPoseInterval number 0 Seconds between onCameraPose events. 0 never sends one.

Events

Event Payload
onWorldEvent {requestId, phase: WorldPhase, loadedSplats, errorCode, message}. errorCode and message are set only on failed.
onColliderEvent {requestId, phase: ColliderPhase, errorCode, message}.
onStats {requestId, loadedSplats, drawnSplats, frameMillis, gpuMillis, sortMillis} with a *TimingAvailable flag beside each timing. Throttled to 2 Hz.
onCameraPose {x, y, z, yaw, pitch} in the world's frame, throttled to cameraPoseInterval and sent only when the pose changed.
onPolicyEvent {revision, phase: PolicyPhase, errorCode, message} plus every effective policy field.
onCapabilities The adapter's limits, GPU feature support and applied policy fields. Once per engine.

A *TimingAvailable flag of false means its paired value is not a measurement; pass both through optionalTimingMillis() to get number | null.

Commands

Each takes the view ref first and returns nothing.

Command Signature Notes
setWalkVelocity (ref, forward, right) Meters per second, held until called again. Forward is where the camera looks. Needs walk mode ready.
look (ref, deltaYaw, deltaPitch) Radians. Pitch is clamped, and this is ignored while the gyroscope drives the view.
setCameraPose (ref, x, y, z, yaw, pitch) Teleports. While walking, the camera settles onto the floor under the new point.

Constants

Every enumerated value is a frozen object with a matching type, so there are no bare strings to misspell.

Export Members
QualityPreset highEnd, high, balanced, performance
qualityPresets every preset in order, for rendering a picker
WorldPhase uploaded, frameReady, failed
ColliderPhase ready, failed
PolicyPhase applied, warning, rejected
RasterStrategy hardware, computeTile, hybrid
PerformanceMode auto, manual
PolicyChangeKind none, nativePropUpdate, worldReload, unavailable
conservativeCapabilities the narrowest limits every shipped adapter accepts, for the first request

Builder and helpers

Export Purpose
SplatKitBuilder withWorld(), withRender(), withPerformance(), withPreset(), build(capabilities).
toNativeViewProps(configuration) The world, paused, renderScale and shDegree props.
toNativePolicyProp(configuration, revision) The policy prop for a positive revision.
nativeCapabilitiesFromEvent(event) An onCapabilities payload as a DeviceCapabilities snapshot for build().
classifyPolicyChange(previous, next) Whether a change needs nothing, a prop update, a world reload, or is unavailable.
optionalTimingMillis(available, value) value, or null when unavailable. Zero is a valid measurement.

Validators

validateWorldRequest(request, limits), validateColliderRequest(request), validateCharacter(character) and validateRenderOptions(options) throw on structural violations. None of them touch the filesystem or the GPU.

PerformanceDiagnostic explains every fallback build() made:

code Meaning
limit-clamped Clamped to a host or native limit.
native-option-fallback The adapter does not apply this field; it fell back to the native default.
native-support-unknown Capabilities have not arrived, so support is unknown.
capability-fallback-unavailable The requested raster strategy is unsupported; hardware is the candidate.
fabric-option-unavailable The option has no Fabric prop. targetFps only, which is a request and never enables dynamic quality.

Error codes

Both adapters share these, so a host branches on errorCode without checking the platform.

Code Reported on Meaning
INVALID_REQUEST onWorldEvent, onColliderEvent The request failed structural validation before native tried to load it.
WORLD_LOAD_FAILED onWorldEvent The world file could not be decoded or loaded.
GPU_UNAVAILABLE onWorldEvent The GPU backend could not be initialized.
COLLIDER_LOAD_FAILED onColliderEvent The collider GLB could not be loaded.
INVALID_POLICY onPolicyEvent The policy failed validation.
POLICY_PREPARATION_FAILED onPolicyEvent The policy was valid but native could not prepare it; the previous one stays.

Troubleshooting

The view is blank. Check that world is set and that onWorldEvent fires at all; a view with no world renders nothing. Prefer a physical device: emulators often lack full Vulkan 1.1, which reports GPU_UNAVAILABLE.

The world fails to load. filePath must be absolute, local and readable. Confirm the file is really at that path in the sandbox, and read message for the loader's reason.

Nothing renders and the log mentions Fabric or Codegen. SplatKitView is Fabric-only. Confirm the New Architecture is enabled and that a full rebuild picked up SplatKitSpec.

The scene is sharp up close and soft further out. That is the LOD error threshold. Lower lodErrorPixels below the preset's value, and watch onStats for the fill-rate cost.

The HUD shows no frame rate. A timing whose *TimingAvailable flag is false was not measured, which is not the same as zero. The Vulkan backend reports submitted frames rather than presented ones, and neither backend measures GPU time without timestamp queries.

Example app

apps/react-native is a React Native 0.87.1 app from the community template that installs this package, draws a thumb stick and a stats and quality HUD, and walks a 6M splat capture. Its README lists every change a fresh template needs.

Related

Contributing

This package is generated from a monorepo; see CONTRIBUTING.md.

License

MIT

About

React Native component for real-time Gaussian splatting: load an SPZ scene, pick a quality preset, and walk through it from JavaScript.

Topics

Resources

Contributing

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages