2026-09-06

[markaren]

New this release: imaging sonar, ray-traced AO with bent normals, zero-copy frame export to CUDA, splat point clouds and splat scene serialization, PhysX tendons, and the GeoScene terrain stack.

Breaking changes

• Object3D::lookAt() now applies the camera convention (local -Z toward the target) to sensors. New virtual Object3D::usesCameraLookAtConvention() (default false) is overridden by TracedRasterVisionSensor (so DepthSensor and LidarSensor), PathTracedLidarSensor, CameraSensor and SonarSensor. Code that compensated with the mirror point lookAt(2*position - target) now aims backwards and must pass the target itself. Adding a virtual to Object3D changes its vtable layout.
• Imu defaults are the ICM-42688-P datasheet figures: gyroNoise 0.005 → 4.8869e-5 rad/s/√Hz, accelNoise 0.06 → 6.8647e-4 m/s²/√Hz, both random-walk terms → 0. Seeds are unchanged, so seeded replays keep their streams, but existing scenes get different noise magnitudes.
• LidarModel::OS1_64() and OS0_128() step azimuth at 360/1024° (0.3515625) instead of 0.35°: 1024 columns per revolution instead of 1029.
• BVH::intersects() and BVH::intersect(..., accurate=true) run an exact triangle-triangle test instead of the triangle-AABB overlap approximation. Near-miss pairs that previously reported a hit no longer do. BVH::build(positions, indices) throws on malformed input, and getGeometry() returns nullptr after the array build.
• ArticulationLink's constructor gained a leading const PhysxWorld* parameter.
• PhysxGpuBatch::blockFloatsWrite throws std::runtime_error for unhandled write types (including the four tendon types) instead of returning maxDofs.
• ParticleField::submit gained float dtSec = 0.f, and VulkanRenderer::enableVertexInterop gained bool stableCorrespondence = true. Both source-compatible; both ABI-breaking.
• RoadNetwork::buildBridgeMeshes() and buildGroundChunkMeshes() keep their signatures but ignore texWidth/texHeight. Resolution, tile length, UVs, vertex/index counts and materials now come from MarkingRules; materials are shared per (marking class, surface, width, wear, variant), so mutating one returned mesh's material changes every piece of that class; ground chunks gain a child patch-decal mesh. buildMeshes() keeps the legacy path.
• buildGeoBuildingMeshes() emits up to nine named child meshes (geo_buildings_wood, _masonry, _industrial, _roof_slate, …) instead of the three geo_buildings_walls, _walls_plain, _roof. Lookups by name now fail at runtime rather than at compile time.
• terrain::detail::geoBldRoofPalette() was removed from GeoBuildings.hpp. Buildings with no OSM colour tag render in different colours.
• makeNorwegianSplat() emits eight layers instead of six (new shallow and seabed layers; wetland heightLo moved from sea−5 m to sea−0.5 m). Terrain colour below the waterline changes for existing geo scenes.
• makeGeoProvider() tints terrain albedo from land cover by default: GeoTerrainOptions::landCover defaults to true and pack.flow is always computed, so wet-rock and scree tinting apply to every existing pack.
• loadSogWithLod throws when a multi-level SOG asset's tree leaves do not tile a chunk exactly.
• Rendering output changes: the ocean fine-cascade normal runs at full slope (an undocumented 0.5 factor was removed); setDeferredAO(true) now selects the half-res RT AO + bent-normal pass; TAA history invalidation changes frame-0 output; the foam accumulator is capped at 1024 (THREEPP_OCEAN_FOAM_RES=2048 restores the old density); the GL fogColor uniform is encoded at upload.

Added

Sensors

• SonarSensor (new header threepp/helpers/SonarSensor.hpp) — Vulkan-only forward-looking imaging sonar, traced through the renderer's acceleration structure as one dispatch. Synchronous scan() plus the pipelined scanBegin/scanFired/scanReady/scanCollect.
• SonarModel (new header threepp/helpers/SonarModel.hpp) — renderer-free model with presets Wide130(), OculusM750d(), BlueViewM900(), Gemini720is(), plus SonarImage and SonarReflectivity.
• ImuModel with the preset ImuModel::ICM42688P(); Imu's defaults are that preset.
• VulkanRenderer::EventCameraSource {Shaded, Final} with setEventCameraSource(). Shaded (default) is the deterministic G-buffer Lambert proxy; Final feeds the detector the presented frame box-averaged to sensor resolution.
• Sub-frame DVS event timestamps (ESIM linear ramp) on both the CPU EventCameraSensor and the GPU detector. readEventStreamInto returns events sorted by (t_us, y, x, polarity).

Vulkan renderer

• VulkanRenderer::resetTemporalHistory() — restarts every temporal accumulator on every live view: TAA history and its deviation streak, the deferred shade's GI / momentsSq / reflection / reflAux / shadowVis histories, froxel fog, RTAO, cloud march, the probe-GI store, ReSTIR reservoirs, occlusion-cull visibility bits, the sample index and Halton jitter phase, plus one forced dynamic-BLAS rebuild. Costs one frame of convergence.
• Half-res ray-traced AO and bent normals behind setDeferredAO(true) (rtao.comp): four cosine occlusion rays per half-res pixel, temporally reprojected, depth-aware bilateral upsample, writing AO, bent normal, near-field AO and sky visibility. RTX 4070 at 1920×1080: fjord gpuTotal 29.9 → 22.3 ms, Sponza interior 14.25 → 12.29 ms/frame. New FrameTimings::rtaoMs.
• Zero-copy frame export to CUDA: enableFrameInterop(viewHandle, channels), disableFrameInterop, frameInteropActive, syncFrameInterop, the FrameChannel enum {Color, Depth, Normal, Motion, Ids, Albedo, SplatDepth} and FrameInteropExport. Single-buffered, and disabled with a stderr line if the source images are reallocated. Measured 640×480 Color+Depth+Ids: 4.41 ms host readback vs 0.36 ms zero-copy.
• Underwater rendering — a camera below setFogWaterSurfaceY renders as submerged: murk column on env miss, Snell's window, TIR mirror, murk sun shafts with a caustic proxy. Gated on murkDensity > 0 and a set surface Y.
• MaterialWithAttenuation::scatterColor and scatterDistance (mean free path in metres, 0 = off), consumed by the deferred water body as a single-scattering integral.
• Emitter next-event estimation coverage mode: every small emissive light is sampled at reflected, GI and probe hits.
• MeshStandardMaterial::envMapIntensity and HemisphereLight are read on the deferred path.
• FrameTimings::probeGiMs.
• The pipeline cache is written during the run (atomic temp-file rename, keyed to the device, size-checked after the first frame and every 120th), so a killed process no longer loses it.

Ocean and particles

• Ocean::Options::fetch and DisplacedMesh::Params::fetch — JONSWAP fetch scaling on top of Phillips (0 = fully developed sea), live-tunable like windSpeed.
• Ocean::Look::Fjord.
• DisplacedMesh::HullExclusion gains centerY, pitch and roll: inside the hull footprint the surface is pulled onto the vessel's own waterline plane. All three default to 0, bit-identical to before.
• ParticleField::Config::attributes is functional — one vec4 per particle (rgb linear HDR radiance, a = medium weight), with setAttributes, hostAttributes and attributeSerial. Exported alongside positions by enableParticleFieldInterop.
• ParticleField::Config::hostStableSlots lets a HostRing field declare stable slot identity, which makes BillboardRepr::stretchSeconds velocity streaks work for host-driven fields.
• BillboardRepr gains alphaOver, lit / litPhaseG / litAmbient, volumeExtinction / volumeShadow / volumeAmbient / volumeSunGain, sunGeometryShadow (one ray query per particle against the scene TLAS) and coreWeight. All default to the previous behaviour.
• Submerged sprites are moved to their apparent position for an above-water eye (Snell crossing solved with four Newton steps, optical path length preserved).
• THREEPP_OCEAN_FOAM_RES and THREEPP_OCEAN_FOAM_INTERVAL.

Gaussian splats

• Point rendering — SplatCloud::setPointMix / setPointSize / pointSigmaPixels() draw every splat as an opaque disc; intermediate mixes lerp the projected covariance and opacity toward it. Both backends.
• Point-cloud import — SplatLoader::loadPointCloudPly / parsePointCloudPly / isPointCloudPly with PointCloudOptions and PointCloudInfo, reading binary little/big-endian and ascii PLY. SplatLoader::writePly writes the INRIA layout loadPly reads.
• New header threepp/splats/PointSurface.hpp — splats::buildPointSurface builds a triangle SurfaceMesh on the CPU from point means alone (voxel hash, sparse union-of-balls field, marching cubes, island filter). Works on every backend, unlike the Vulkan-only bakeSurface.
• splats::medianNeighbourSpacing.
• Per-node LOD — SogLoader::describe() returns the SSOG tree's leaves in Info::nodes (NodeInfo, NodeRange); splats::selectLodPerNode gives every leaf its own level, selectLodWholeCloud keeps the previous policy, and selectLod dispatches between them.
• SplatCloud is serialized by ObjectExporter/ObjectLoader as a threeppSplat block (source path, import ops to replay, point-mode look, splat count — never the splat data). A .tpz carries the source file under splats/; a pathless cloud gets a splats/<uuid>.ply sidecar. ObjectLoader::setSplatCloudResolver hands live clouds back by uuid instead of re-reading from disk.

PhysX and robotics

• New header threepp/extras/physx/ArticulationTendon.hpp — SpatialTendon / SpatialAttachment and FixedTendon / TendonJoint over PhysX's own tendon primitives.
• New header threepp/extras/physx/TendonCable.hpp — TendonCable, a routed pull-only cable with via points and wrap surfaces, driven by tension (Mode::Tension) or spool length (Mode::Length), applying the frictionless-pulley free-body force at every via point.
• PhysxWorld::addStaticHeightField (raw pointer and std::vector<float> overloads).
• PhysxVehicleBase::setRoadOverride(wheel, height, mu, vRoad), clearRoadOverride, roadOverrideActive, and suspensionForce(wheel).
• Articulation::links(), nameLink(), findLink(), linkNames(). UrdfArticulation registers each URDF link name against the articulation link that governs it, so a fixed-collapsed child (a tool frame, say) resolves to the link it was welded into.
• ArticulationLink::addTorque, addForceAtPos, worldPoint, axis.

Terrain and geodata

• Six new public headers: terrain/GeoScene.hpp, terrain/CellStreamer.hpp, terrain/CliffShell.hpp, terrain/StraightSkeleton.hpp, terrain/UrbanProps.hpp, vegetation/CanopyForest.hpp.
• terrain::GeoScene::create(opts) assembles the whole geodata stack — pack load, road network, road carve, tiled terrain, band sets, cliff shell, canopy forest, synthetic bathymetry — behind one call, with update(camPos), heightAt(x, z) and Stats. GeoScene::makeBathymetry is a public static usable on any HeightGrid.
• GeoTerrainPack gains optional pack data with has*() guards: GeoBuilding::roof (GeoRoof), landuse (GeoLandUse), canopy (a canopy height model) and flow.
• Measured roofs — GeoBuildingsOptions::measuredRoofs (default true) builds hip / gable / flat roofs from a Felkel straight skeleton, with eaves, fascias, parapets and cornices, reporting through GeoBuildingsStats.
• FacadeMaps gains masonry (upper / ground / plain), industrial, slate-roof and tile-roof sets.
• DetailTexture gains BandKind::Cliff; makeTerrainBandSet() takes an optional rockKind (default BandKind::Rock, so existing calls are unchanged).
• Road surfaces — RoadGenerator::bakeSurfaceMaps produces albedo, tangent-space normal and roughness/metalness with coverage-antialiased markings, wheel-track wear, wandering cracks and repair patches, via SurfaceKind, LinePattern and RoadSurfaceStyle. RoadNetwork gains N302 marking classification (MarkingClass, MarkingRules, classOf()) and shared surface bakes.
• TileTerrain::treeSignature().
• TreeParams::blobOcclusion, blobLatSegs, blobLonSegs.
• scripts/geodata/fetch_norway_terrain.py gains --canopy (lidar canopy height model), --roofs (measured eave/ridge/axis per building), --landuse, --texture (Kartverket raster WMS, needs pillow) and --cache. --out now defaults to the repo's geodata/ root instead of a hard-coded absolute path.

Core and loaders

• BVH::distance(b1, b2, m1, m2, maxDistance), BVH::build(positions, indices), triangleCount(), boundingBox().
• URDFLoader reports the <visual> and <collision> elements the articulation description drops, one line per affected link, to diagnostics() and cerr.
• Object3D::usesCameraLookAtConvention().

Python

• New bindings: BVH / RayHit; curves and 2-D shapes (Curve3, CatmullRomCurve3, Path, Shape, TubeGeometry, ExtrudeGeometry, ShapeGeometry); Flock (the extras/fauna bird flock, with its ~40-field Params, perch pipeline and startle); ParticleField with data_texture / float_texture; PhysxVehicle; soft bodies (add_soft_body, SoftBody, PhysxSoftBodyMaterial); add_static_heightfield; the tendon types; IkSolver / IkOptions / IkResult / IkTask with Robot.set_end_effector; SonarSensor / SonarModel / SonarImage; PathTracedLidarSensor; renderer.scan_lidar; the Vulkan event camera; secondary views (add_view, set_view_camera, set_view_display_rect, read_view_rgb_pixels); GeoScene; SogLoader, select_lod, bake_surface, make_sensor_mesh; and the KHR specular / scatter material knobs.
• VulkanRenderer gains sim_time (None = wall clock), dlss / dlss_available, restir_di, starfield, splat_depth_aov, set_sensor_only_surfaces, set_fog_water_surface_y, reset_temporal_history(), enable_frame_interop / sync_frame_interop, enable_particle_field_interop, and rtao_ms in frame_timings.
• threepp.torch_frames.FrameTensors exposes renderer images as live torch tensors; threepp.cuda_interop gains VkInteropTensor, VkInteropTensorView, VkInteropArray and VkInteropMemory.
• threepp.__version__; threepp.vulkan_validation_active() and vulkan_validation_error_count().
• threepp.rl: GpuSim.apply_link_force, VecTask.on_pre_substep, VecTask.settling, CUDA-graph replay of the per-step torch region (VecTask(graph=True), verify_graph()), and the new rl.raycast and rl.perception modules.
• threepp.terrain_deform.
• threepp.imgui gains plot_lines, dummy, item_rect and a draw list (draw_line, draw_polyline, draw_rect, draw_circle, draw_text).
• BufferGeometry.set_index and apply_matrix4.

Changed

• RoadGenerator::buildSurface(vOffsetMeters = 0, mirrorAlternateTiles = false) — both defaults reproduce the previous geometry byte for byte.
• Water shaded from above uses the scene's declared murk medium instead of the Ocean material's Beer-Lambert plus analytic deep body, so hulls and particle sprites in the same column no longer disagree.
• MeshPhysicalMaterial::specularIntensity scales a water reflection that lands on geometry, leaving the escape-to-sky at full Fresnel.
• Performance, with no intended output change: froxel integrate early-outs (TP_Froxel 0.144 → 0.083 ms), in-place ocean IFFT permute, one ORM tap in post_composite, dead work sunk below the water early-out, and an SSE2 world-matrix kernel plus batched BLAS refits on the CPU scene walk.

Fixed

Vulkan renderer

• Probe GI updated once per view instead of once per frame: its round-robin cursor and ray seed advanced once per attached camera, so the probe grid depended on how many views were attached and a capture replayed with an extra sensor did not reproduce. The prev-store snapshot now copies only the written window (10 MB/frame → ~1.3 MB).
• Vertex interop lifetime: the arm-time rebuild freed buffers whose device addresses the GeometryDesc mirror still named, so every RT hit on an armed mesh read freed memory and eventually hit VK_ERROR_DEVICE_LOST; the per-frame refit built PREFER_FAST_BUILD into PREFER_FAST_TRACE-sized storage and overran the acceleration structure; and a same-count edit to a CPU-owned side attribute under an armed registration destroyed exports a foreign API still held. Records are now sized for both build modes and transplanted rather than evicted.
• The swapchain resize predicate compared canvas.size() against the swapchain extent, so whenever the platform granted a different extent the swapchain plus its vkDeviceWaitIdle was recreated every frame. Secondary views never removeView()d leaked their G-buffer images, colour target, readback buffer and post chain past vkDestroyDevice.
• With setGbufferMsaa(2) or (4) the deferred shade reconstructed world position along the pixel-centre ray while gbuf_resolve writes the dominant sample's depth, putting refraction origins outside glass so the interior trace escaped to the environment.
• traceRadiance shaded any transmission > 0 hit as opaque, so nested glTF glass rendered as a solid block; textured BLEND materials with opacity < 1 rasterized fully opaque.
• A roughness-0.4 surface rendered as a polished mirror whenever the camera or object moved, relaxing back to glossy over ~16 frames at rest.
• Emitter NEE's 4-stratum power CDF could not resolve several small emitters, so only one of a pair of small nav lights was ever sampled and the reflection flipped colour a few times a second.
• Depth of field read its gather radius from the dilated CoC tile with a nearest fetch, so the blur stepped a whole 16×16 tile at a time and produced a hard staircase halo around defocused silhouettes. The tile max is now sampled bilinearly.
• Line and LineSegments overlay pipelines inherited the wireframe pipeline's blend-off attachment state, so transparency, opacity and additive blending never reached the hardware.
• A mesh drawing a partial BufferGeometry::drawRange could be picked for a coarser auto-LOD level whose index set has no correspondence with the range, drawing the whole simplified mesh. Such spans now pin to LOD0.
• readGBufferAOV(SplatDepth) gated on setSplatDepthAov rather than on allocation, so a scene whose splats were correctly occluding overlays reported the AOV as unallocated.

Ocean, particles and splats

• DisplacedMesh::sampleHeight() read a CPU mirror while in-flight frames were still copying into it, so the sampled field was 1–2 frames old and torn depending on GPU pacing (58 of 60 frames of a pinned sim-time sequence differed between paced and unpaced runs).
• The ParticleField glow leg drew into a target with no depth attachment, so embers behind opaque geometry rendered their halo through the wall.
• Three defects in the DensityRepr march, including a medium lit through shadowVis() alone — which reads a shadow map a participating medium never renders into, so a density volume was fully sunlit through its whole depth and read as white ash at any albedo.
• LOD selection submitted no ranges when nothing survived frustum culling, and an empty range list means "draw all splats" to the Vulkan splat pass, so turning the camera away from a scan submitted the entire cloud.

Sensors

• The path-traced lidar back-scatter shader omitted the height-correlated Smith GGX visibility term, so returned intensity climbed at grazing incidence (measured on a flat panel: 0.208 head-on, 0.017 at 55°, rising past 0.16 beyond 80°).
• The GPU event stream buffer was a fixed 256k events, so a frame's worst case could overflow it and truncation was decided by workgroup scheduling on the atomic append. It is now sized to max(256k, width * height * maxEventsPerPixel) per ring slot.
• The event-camera detector reused a stale reference image and accumulator across a disable/enable cycle at the same resolution.

PhysX, terrain and core

• ArticulationLink::addImpulse called addForce with PxForceMode::eIMPULSE, which PhysX documents as invalid for articulation links; it now applies J / fixedTimestep as eFORCE.
• PhysxGpuBatch::writeHost mis-sized host-staged link writes (eLINK_FORCE and eLINK_TORQUE are maxLinks*3, not DOF-sized).
• TileTerrain deadlocked on its own bake budget: bakes that had finished but were waiting on the LOD swap gate kept occupying an in-flight slot, so the streamer could sit permanently at maxBakesInFlight with no tile ever refining.
• --roofs wrote a roof block where the lidar predates or misses the building.
• CurvePath<T>::getPoint() returned NaN unconditionally (a missing return after the segment lookup), so CurvePath / Path / Shape getPoint(), getPointAt() and getSpacedPoints() were unusable. getPoints() was unaffected.
• Frustum::intersectsObject(), intersectsSprite() and intersectsBox() wrote into namespace-scope scratch singletons, so two threads culling against different Frustum instances clobbered each other's scratch. The scratch is now stack-local.
• GLTFLoader validates every accessor index a primitive references and skips just that primitive with a warning, instead of failing the assembly — OpenCASCADE's RWGltf_CafWriter emits such primitives for faces it cannot triangulate.
• Python BufferGeometry.set_attribute reset the draw range on every call, so setting a per-vertex colour after set_index truncated an indexed mesh and it drew a fraction of itself.
• Python make_sensor_mesh raises RuntimeError on a build without the Vulkan backend instead of failing to link, and an empty event-visualisation array no longer breaks the GCC build.

Examples, tooling and docs

• New example examples/vulkan/vulkan_sonar — an imaging sonar on an ROV inside a net pen, with a polar echogram fan and a model combo, demonstrating the sensor-only proxy pattern.
• vulkan_aov_audit gains ~20 options (--scene fjord, --terrain, --events, --lidar, --scene-edit, --fsr, --settle, --reset-history, …); vulkan_lidar_audit gains --sonar and the same scene-edit axis.
• examples/objects/gaussian_splats gains --points, --point-size, --point-mix and point-cloud PLY detection.
• examples/extras/terrain/norway_terrain gains --view presets, --fly, --list-roads, --cam and --fov, and builds an Ålesund capstone scene on the new terrain headers.
• Editor: splat clouds are saved by reference and survive Play; the inspector gains point-mode mix/size and surface method / body / mass / hull controls.
• New Python examples: warp_netpen, warp_prop_vortex, warp_mudsnow_drive / _mpm, warp_mudsnow, warp_explosion, warp_cloth_catch, warp_hydraulic_press, tendon_hand, tendon_probe, stewart_balancer, lidar_sculpt, warp_hull_sculpt, and the calico set (Spot walking a Gaussian-splat scan). python/README.md now indexes every script rather than the fifteen with a table row.
• New measurement harnesses: sensor_audit.py, multiview_bench.py, netpen_e3.py, gazebo_audit/ and isaac_audit/ (the same determinism audit run against Gazebo Harmonic and Isaac Sim 4.5), spot_push_probe.py, spot_gait_probe.py, spot_scan_ab.py.
• Trained checkpoints are checked in, so the play/eval scripts run on a fresh clone: cartpole_swingup.pt and scratch_flat_best.pt.
• doc/vulkan_splats.md documents point mode and the splat depth AOV's allocation rule.

Build

• MSVC RelWithDebInfo rewrites /Ob1 to /Ob2 for the library target (directory-scoped to src/), so it no longer inlines only functions marked inline.
• No new third-party dependencies. CMakePresets.json, vcpkg.json, pyproject.toml, CITATION.cff and everything under .github/ are unchanged. The geodata --texture fetch adds an optional pillow dependency, recorded in geodata/README.md.

Full Changelog: 2026-08-21...2026-09-06

---

This discussion was created from the release 2026-09-06.