Weaving light from the invisible depth of the web.
Depthweaver is a real-time rendering pipeline that transforms live web pages into 2.5D displacement surfaces and area light cookies within a Unity 3D scene. By introducing a novel Composite Depth Scoring System that extracts implicit elevation semantics from CSS/DOM signals (e.g., box-shadow, z-index, transform: translateZ()), the system reconstructs 3D depth information from inherently 2D HTML/CSS content. The extracted depth map drives vertex displacement on a subdivided screen mesh while the color output simultaneously serves as an HDRP RectAreaLight cookie, projecting colored indirect illumination onto surrounding 3D geometry.
This approach differs from prior work (CineShader, CSS3D Renderer, MiDaS) by operating directly on CSS design-system semantics rather than pixel-level inference, achieving event-driven depth updates with zero per-frame computational cost when the DOM is static.
- System Architecture
- Composite Depth Scoring Algorithm
- Dual Texture Pipeline
- Feature Overview
- Project Structure
- Technical Specifications
- Build Instructions
- Keyboard Controls
- Acknowledgments
- License
┌─────────────────────────────────────────────────────────────────────┐
│ UNITY PROCESS (C#) │
│ │
│ ┌─────────────────────┐ ┌─────────────────────────────────┐ │
│ │ CEF Browser │ │ HDRP Rendering Engine │ │
│ │ (Chromium Offscreen)│ │ │ │
│ │ │ │ ┌─────────────┐ ┌───────────┐ │ │
│ │ ┌────────────────┐ │ │ │ Screen Mesh │ │ Rect Area │ │ │
│ │ │ Web Page │ │ │ │ (Displaced) │ │ Light │ │ │
│ │ │ HTML / CSS / JS│ │ │ └──────┬──────┘ └─────┬─────┘ │ │
│ │ └───────┬────────┘ │ │ │ │ │ │
│ │ │ │ │ Depth Map Cookie Tex │ │
│ │ ┌─────┴─────┐ │ │ (Alpha) (RGB) │ │
│ │ │ Offscreen │ │ │ └───────┬───────┘ │ │
│ │ │ Render │ │ │ │ │ │
│ │ └─────┬─────┘ │ │ ┌──────────────┴───────────┐ │ │
│ │ │ │ │ │ 3D Environment │ │ │
│ │ ┌─────┴─────┐ │ │ │ Terrain / Vegetation / │ │ │
│ │ │Color Buffer│────╂─────╂──│ Studio / Showcases │ │ │
│ │ └───────────┘ │ │ └──────────────────────────┘ │ │
│ │ ┌───────────┐ │ │ │ │
│ │ │Depth Canvas│────╂─────╂──→ Displacement Map │ │
│ │ └───────────┘ │ │ │ │
│ └─────────────────────┘ └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
The core contribution of this project is the Composite Depth Score, a weighted aggregation of six CSS/DOM signals that recovers designer-intended elevation from web page structure:
VisualDepth(element) =
w1 × NormalizedDOMDepth(element)
+ w2 × NormalizedStackingContext(element)
+ w3 × BoxShadowElevation(element)
+ w4 × TransformZ(element)
+ w5 × OpacityHint(element)
+ w6 × PositionType(element)
| Signal | Default Weight | Extraction Method | Rationale |
|---|---|---|---|
| DOM Nesting Depth | w1 = 0.25 | parentNode chain traversal |
Hierarchical baseline from document structure |
| Stacking Context | w2 = 0.25 | getComputedStyle().zIndex |
CSS-specified visual ordering |
| Box Shadow Elevation | w3 = 0.30 | box-shadow blur/spread parsing |
Material Design elevation encodes depth intent |
| Transform Z-axis | w4 = 0.10 | transform: translateZ() parsing |
Explicit 3D positioning |
| Opacity Hint | w5 = 0.05 | opacity < 1 detection |
Background layer / overlay detection |
| Position Type | w6 = 0.05 | position: fixed/sticky analysis |
Navigation bars, floating elements |
Box Shadow receives the highest weight (0.30) because modern design systems (Material Design, Tailwind CSS) systematically encode elevation through shadow values:
box-shadow: 0 1px 2px → elevation ~ 1 (subtle surface lift)
box-shadow: 0 4px 8px → elevation ~ 3 (card level)
box-shadow: 0 12px 24px → elevation ~ 6 (modal / dropdown)
box-shadow: 0 24px 48px → elevation ~ 12 (top-level popup)
The system separates color and depth texture production into two independent update paths with distinct refresh rates:
Web Page (CEF Offscreen Render)
│
├── [Every Frame] Color Buffer (BGRA32)
│ ├──→ Screen Mesh Albedo Texture
│ └──→ RectAreaLight Cookie Texture (GPU: saturation + LUT + intensity)
│
└── [DOM Mutation Only] Depth Canvas (Grayscale, 512x512)
│ depth-extractor.js renders depth → DepthTransmitter encodes R-channel
│ → console.log("__DEPTH:512:<base64>")
│ → C++ OnConsoleMessage → Base64 decode → depthBuffer_
│ → C# PollDepthFrame → Texture2D(R8)
│ → DepthTextureProcessor (GPU Gaussian blur)
│
└──→ Screen Mesh Displacement Map
vertex += normal * depth.r * displacementScale
The depth canvas is updated only when the DOM changes (via MutationObserver), while the color buffer is captured every frame. This decoupled architecture yields zero per-frame depth computation cost for static pages. The depth data transfer employs a console message protocol (__DEPTH:<size>:<base64>) to bridge the JavaScript→C++→C# boundary without requiring additional IPC mechanisms.
The system embeds a full Chromium browser via a C++ native plugin for offscreen rendering, with a Strategy Pattern abstraction (IBrowserBackend) enabling runtime browser engine replacement.
Native Plugin (C++17, CEF 145.0.28 / Chromium 145):
- Offscreen rendering via
CefRenderHandler::OnPaint()with configurable resolution and frame rate --single-processmode for macOS ARM64 compatibility (CEF subprocess launcher bypass)- Double-buffered pixel transfer with mutex-guarded front/back swap and coordinate correction (X+Y flip for CEF→Unity coordinate system alignment)
- Dirty rectangle tracking for partial-update optimization
- Full input forwarding: mouse (move, click, wheel), keyboard (key down/up, char)
- JavaScript execution and evaluation bridge with async callback polling
DW_Log()ring buffer (32 KB) for C++ → Unity Console log relay via P/Invoke pollingCefDisplayHandler::OnConsoleMessage()— intercepts depth data transmitted from JavaScript (see Depth Transfer Protocol below)- Cross-platform build system (Windows x64 / macOS ARM64 via CMake)
Unity Bridge (C#):
IBrowserBackend— Abstract interface decoupling browser engine from the pipelineCEFNativeBackend— P/Invoke-based implementation wrapping 30+ native API calls, including depth frame pollingCEFTextureSource—ITextureSourceimplementation with three transfer modes:- Standard:
LoadRawTextureData()single-buffer upload - DoubleBuffered: Pingpong
Texture2D[2]for GPU stall prevention - NativePointer: Direct GPU texture write (platform-specific)
- Standard:
CEFBridge— High-level facade (URL navigation, JS injection, depth weight sync)CEFInputHandler— Raycast-based screen mesh hit detection with UV-to-pixel coordinate mapping
Extensibility: Adding a new browser backend (e.g., Vuplex, Ultralight) requires implementing a single interface (IBrowserBackend) with no changes to the texture pipeline.
A JavaScript module (depth-extractor.js) injected into the CEF browser that generates a 512x512 grayscale depth canvas from live DOM analysis.
Architecture:
SignalRegistry— Plugin-pattern signal extractor registry; new depth signals can be registered at runtime viawindow.__UIShader.registerSignal()without modifying core codeDepthScorer— Weighted aggregation of registered signals into a normalized depth value per elementDepthRenderer— DOM tree traversal with bounding rect projection onto the depth canvasDOMWatcher—MutationObserver+ event listeners (transitionend,animationend,scroll,resize) with configurable debounce (DOM: 50ms, scroll: 100ms)- CSS animation active-state polling for continuous animation detection
Presets: Three built-in weight configurations — balanced, materialDesign, flatDesign — optimized for different design systems.
Depth Transfer Protocol (JS → C++ → C#):
The depth canvas data traverses a multi-stage pipeline from JavaScript to the Unity shader:
depth-extractor.js C++ Native Plugin Unity C#
┌──────────────┐ console.log() ┌──────────────────┐ P/Invoke ┌─────────────────┐
│DepthTransmitter│──────────────────→│OnConsoleMessage()│───────────→│PollDepthFrame() │
│ R-channel │ "__DEPTH:512: │ Base64Decode() │ │ NativeArray<T> │
│ Base64 encode │ <base64>" │ depthBuffer_[] │ │ Texture2D (R8) │
└──────────────┘ └──────────────────┘ └─────────────────┘
DepthTransmitter(JavaScript) — Extracts the R-channel from the 512x512 depth canvas (262,144 bytes), encodes as RFC 4648 Base64, and transmits viaconsole.log("__DEPTH:<size>:<base64>")BrowserClient::OnConsoleMessage()(C++) — Intercepts__DEPTH:prefixed console messages, decodes Base64 via lookup table, stores in mutex-guardeddepthBuffer_with atomicdepthReady_flagCEFTextureSource.PollDepthFrame()(C#) — PollsHasNewDepthFrame, copies depth pixels via GC-pinned buffer, loads intoTexture2D(R8)usingNativeArray<byte>for precise size handlingTexturePipelineManagerroutes the depth texture throughDepthTextureProcessor.ApplyBlur()(GPU Gaussian blur) before applying to the displacement shader
Depth Texture Post-Processing (GPU):
DepthBlur.compute— Separable Gaussian blur (horizontal + vertical, 2-pass) on the depth mapDepthTextureProcessor.cs— Pingpong buffer management with configurable iteration count, integrated intoTexturePipelineManagerdepth path
Weight Tuning:
DepthWeightPreset(ScriptableObject) — JSON serialization, interpolation between presets, factory methodsDepthWeightTuner(Custom Editor) — Real-time sliders with color bars, sum visualization, normalization, preset buttons
The depth map drives physical vertex displacement on a subdivided screen mesh, producing a 2.5D surface where UI elements protrude from the screen plane.
Screen Mesh (Procedural Generation):
- Three-level LOD pool pre-generated at startup:
- High: 511x511 subdivision (262,144 vertices)
- Medium: 255x255 subdivision (65,536 vertices)
- Low: 63x63 subdivision (4,096 vertices)
- 32-bit index buffer support for high-density meshes
- Tangent vectors for normal mapping compatibility
OnLODChangedevent for external system notification- Screen frame generator: 4-edge extruded border with per-face flat shading
HLSL Displacement Shader:
- 3-pass structure: ForwardOnly (color + displacement + normals), ShadowCaster (displaced shadows), DepthForwardOnly (HDRP depth prepass)
- Central-difference normal reconstruction: 4-directional adjacent depth samples for accurate surface gradient
- Edge feathering:
smoothstep-based displacement attenuation at mesh boundaries - Self-emission output for screen glow effect
- Shader Graph Custom Function Node interface (
DisplaceVertex.hlsl) withfloat/halfprecision variants
Runtime Control:
DisplacementController— Real-time shader parameter synchronization (scale, bias, edge falloff, emission intensity)- Camera-distance-based automatic LOD switching with hysteresis to prevent boundary flickering
The web page color output serves as an HDRP RectAreaLight cookie, projecting colored indirect illumination onto surrounding 3D geometry via Unity's native Linearly Transformed Cosines (LTC) integration.
GPU Cookie Processing:
CookieProcess.shader— Single-pass GPU pipeline: saturation boost (BT.709 luminance lerp) → contrast curve (1D LUT) → intensity multiplierAnimationCurve→ 256x1 LUT texture auto-baking with half-texel offset correction- ~100x faster than equivalent CPU
GetPixelsloop
Auto Exposure:
- Progressive downsampling (512 → 64 → 4 → 1) for average luminance computation (BT.709 weighted)
- Configurable min/max intensity multipliers with exponential decay interpolation
- Prevents oversaturation from bright web content
Quadrant Light System:
- 2x2 subdivided
RectAreaLightarray for improved spatial accuracy - GPU Blit with UV scale/offset for per-quadrant cookie cropping (zero CPU cost)
- Automatic main light ↔ quadrant light switching to prevent double illumination
- Shared auto-exposure intensity synchronization
Two interchangeable environment modes for different presentation contexts:
Natural Environment (Procedural):
ProceduralTerrainModule— Heightmap-based terrain generationHDRPSkyModule— Physically-based sky with configurable exposureSunLightModule— Directional sun with shadow cascadesProceduralGrassModule— Terrain-conforming grass instancesAtmosphereModule— Volumetric fog and atmospheric scattering- Modular architecture via
IEnvironmentModuleinterface
Studio Environment (Production):
- Reflective floor: HDRP/Lit with ClearCoat (Smoothness = 0.88)
- Cyclorama backdrop: Procedurally generated arc mesh with configurable arc angle, radius, and segment count
- Floor-to-wall cove transition: Quarter-circle smooth junction with configurable radius
- Showcase objects: Extensible
List<ShowcaseObjectConfig>with 5 material presets (ChromeMirror, DarkVehicle, HumanSkin, GlossyPlastic, MatteWhite) - Fill light: Low-intensity directional (50 lux, cool blue-white)
Orbit Camera Controller:
- Manual mode: Right-click orbit + scroll wheel zoom with configurable sensitivity and clamped pitch
- Auto-cruise mode (C key): Continuous horizontal rotation + sinusoidal vertical oscillation
- Preset transitions:
AnimationCurve-based smooth interpolation between camera presets - Automatic cruise interruption on manual input detection (right-click, scroll)
- Three-state machine: Manual → AutoCruise → Transition
Post-Processing Controller:
- HDRP Volume auto-generation with code-configured overrides
- Effects: Bloom, Vignette, ACES Tonemapping, Color Adjustments, Film Grain, Depth of Field
- Distance-based auto DOF: Activates within 6m, dynamically updates focus distance
Demo Auto-Play System:
DemoScenarios(ScriptableObject) — Data-driven scenario definitions (URL, weight preset, camera preset, displacement scale)- 5 built-in scenarios: Material Design modal, dark mode toggle, animated hero, hover card grid, dashboard
- Coroutine-based scenario cycling with configurable dwell time
- CEF-independent design:
event Action<string> OnLoadURLfor loose coupling
In-Game UI Overlay (Tab key):
- Draggable IMGUI window with sectioned layout
- URL input, depth preset selection grid, displacement/lighting/camera sliders
- Quadrant light toggle, auto-exposure toggle, keyboard shortcut reference
- GC-optimized GUIStyle caching
Performance Profiler (F3 key):
- 120-frame history ring buffer
- FPS average + 1% low (top-percentile sorted)
- Memory:
Profiler.GetTotalAllocatedMemoryLong()/GetMonoUsedSizeLong() - Editor-only: draw calls, triangles, batches via
UnityStats - Pipeline state: color/depth update rates, camera mode, demo scenario
Recording Guide (F5/F6 keys):
- 30-second timeline with 6 keyframe markers for highlight reel composition
- Recording timer with elapsed/remaining display
- Safe area margins and rule-of-thirds grid overlays
Build Automation:
- Menu:
UIShader > Build > Windows x64,macOS (Apple Silicon),All Platforms BuildReport-based result reporting (size, duration, errors/warnings)- CI/CD support:
BuildFromCommandLine()with-platform windows/macos/all - Automatic CEF binary copying (activates when native plugin is available)
depthweaver/
├── Assets/
│ ├── Scripts/
│ │ ├── Core/ # Pipeline orchestration
│ │ │ ├── UIShaderConfig.cs # Central ScriptableObject configuration
│ │ │ ├── TexturePipelineManager.cs # Texture distribution orchestrator
│ │ │ ├── BootstrapManager.cs # Coroutine-based initialization sequence
│ │ │ ├── ITextureSource.cs # Texture provider abstraction
│ │ │ ├── StaticTextureSource.cs # Phase 0 PNG source
│ │ │ └── UIShaderBootstrap.cs # Editor bootstrap utility
│ │ │
│ │ ├── CEF/ # Browser integration (Phase 1)
│ │ │ ├── IBrowserBackend.cs # Browser engine abstraction (Strategy)
│ │ │ ├── NativePluginInterop.cs # P/Invoke declarations (25+ APIs)
│ │ │ ├── CEFNativeBackend.cs # Native backend implementation
│ │ │ ├── CEFTextureSource.cs # ITextureSource for live web pages
│ │ │ ├── CEFBridge.cs # High-level facade
│ │ │ └── CEFInputHandler.cs # Raycast input forwarding
│ │ │
│ │ ├── Rendering/ # Displacement + lighting
│ │ │ ├── ScreenMeshGenerator.cs # 3-level LOD mesh pool
│ │ │ ├── ScreenFrameGenerator.cs # Extruded border frame
│ │ │ ├── DisplacementController.cs # Runtime shader parameter control
│ │ │ ├── ScreenLightController.cs # Main RectAreaLight + auto-exposure
│ │ │ ├── CookieProcessor.cs # GPU cookie post-processing
│ │ │ └── QuadrantLightSystem.cs # 2x2 subdivided area lights
│ │ │
│ │ ├── DepthMap/ # Depth processing (Phase 2)
│ │ │ ├── DepthTextureProcessor.cs # GPU Gaussian blur (pingpong)
│ │ │ └── DepthWeightPreset.cs # Weight presets (ScriptableObject)
│ │ │
│ │ ├── Environment/ # Natural environment modules
│ │ │ ├── IEnvironmentModule.cs # Module interface
│ │ │ ├── NaturalEnvironmentBuilder.cs
│ │ │ ├── ProceduralTerrainModule.cs
│ │ │ ├── HDRPSkyModule.cs
│ │ │ ├── SunLightModule.cs
│ │ │ ├── ProceduralGrassModule.cs
│ │ │ └── AtmosphereModule.cs
│ │ │
│ │ ├── Studio/ # Production studio environment
│ │ │ └── StudioEnvironment.cs
│ │ │
│ │ ├── Camera/ # Camera + post-processing
│ │ │ ├── OrbitCameraController.cs # Orbit / cruise / transition
│ │ │ └── PostProcessController.cs # HDRP Volume auto-setup
│ │ │
│ │ ├── Demo/ # Demo scenarios
│ │ │ ├── DemoScenarios.cs # ScriptableObject data
│ │ │ └── DemoAutoPlay.cs # Auto-play controller
│ │ │
│ │ └── UI/ # Overlays and tools
│ │ ├── UIShaderOverlay.cs # In-game control panel
│ │ ├── PerformanceProfiler.cs # F3 performance overlay
│ │ └── RecordingGuide.cs # F5/F6 recording guide
│ │
│ ├── Shaders/
│ │ ├── UIShaderDisplacement.shader # 3-pass HLSL displacement
│ │ ├── UIShaderDisplacement_Simple.shader # Phase 0 simplified
│ │ ├── DisplaceVertex.hlsl # Shader Graph custom function
│ │ ├── DepthBlur.compute # Separable Gaussian blur
│ │ └── CookieProcess.shader # GPU cookie processing
│ │
│ ├── JavaScript/
│ │ └── depth-extractor.js # Composite Depth Score engine
│ │
│ └── Editor/
│ ├── HDRPSettingsValidator.cs # HDRP asset validation
│ ├── ProjectStructureSetup.cs # Folder structure generator
│ ├── StudioSceneSetupWizard.cs # Scene builder wizard
│ ├── DepthWeightTuner.cs # Weight tuning inspector
│ └── BuildHelper.cs # Cross-platform build automation
│
└── NativePlugin/
├── CMakeLists.txt # Cross-platform build (VS 2022 / Xcode)
└── src/
├── cef_plugin.h # C API declarations
├── cef_plugin.cpp # API implementation + global state
├── render_handler.h/cpp # Offscreen CefRenderHandler
├── browser_client.h/cpp # CefClient + CefDisplayHandler + depth reception
└── browser_app.h/cpp # CefApp process handler
| Metric | Value |
|---|---|
| Total Source Files | 58+ |
| Total Lines of Code | ~13,000+ |
| Languages | C# (45%), HLSL (25%), JavaScript (20%), C++ (10%) |
| Render Resolution | 512 x 512 (configurable) |
| Max Mesh Subdivision | 511 x 511 (262,144 vertices) |
| LOD Levels | 3 (High / Medium / Low) |
| Depth Signals | 6 (extensible via plugin registry) |
| Depth Update Trigger | DOM Mutation (event-driven, 0 cost when static) |
| Depth Transfer | Base64 via console.log → C++ OnConsoleMessage → C# Texture2D (R8) |
| Depth Post-Processing | GPU separable Gaussian blur (compute shader, configurable iterations) |
| Cookie Processing | GPU Blit (single pass: saturation + LUT contrast + intensity) |
| Auto-Exposure | Progressive downsample (512 → 1, BT.709 weighted) |
| Target Performance | 60 FPS at 1080p (RTX 3060 tier) |
| Platform Support | Windows x64, macOS ARM64 |
| Unity Version | 6.3 LTS (6000.3.10f1) |
| Render Pipeline | HDRP (High Definition Render Pipeline) |
| CEF Version | 145.0.28+g51162e8 (Chromium 145.0.7632.160) |
| CEF Process Mode | --single-process (macOS ARM64 compatibility) |
- Unity 6.3 LTS (6000.3.10f1) or later with HDRP package
- CMake 3.20+ (for native plugin)
- Windows: Visual Studio 2022 with C++ desktop workload
- macOS: Xcode 15+ with Command Line Tools
- CEF binary distribution v145.0.28+ (download from cef-builds.spotifycdn.com)
# macOS ARM64 (verified)
cd NativePlugin
cmake -B build -G Xcode \
-DCMAKE_OSX_ARCHITECTURES=arm64 \
-DCEF_ROOT="../cef_binary_145.0.28+g51162e8+chromium-145.0.7632.160_macosarm64_minimal"
cmake --build build --config Release
# Windows x64
cd NativePlugin
cmake -B build -G "Visual Studio 17 2022" -A x64 \
-DCEF_ROOT="path/to/cef_binary_windows64"
cmake --build build --config Release- Open the project in Unity 6.3 LTS with HDRP
- Verify HDRP Asset settings: Area Lights, Shadows, SSR, SSAO enabled
- Create
UIShaderConfigasset:Create > UIShader > Config - Build studio scene:
UIShader > Build Studio Scene - Place native plugin binaries in
Assets/Plugins/(platform-specific subfolders) - Place CEF framework in the appropriate location for your platform
Unity Menu → UIShader > Build > Windows x64
Unity Menu → UIShader > Build > macOS (Apple Silicon)
| Key | Action |
|---|---|
| Tab | Toggle in-game UI overlay |
| C | Toggle auto-cruise camera |
| 1-4 | Camera preset transitions |
| P | Toggle demo auto-play |
| Left/Right Arrow | Previous/next demo scenario |
| F3 | Toggle performance profiler |
| F5 | Toggle recording guide |
| F6 | Start/stop recording timer |
| Right-click + Drag | Orbit camera |
| Scroll Wheel | Zoom camera |
This project was inspired by CineShader by Lusion, which demonstrated the rendering technique of using Shadertoy GLSL output as both a displacement map and area light cookie in a virtual studio environment. Depthweaver extends this concept to arbitrary web pages through the introduction of the Composite Depth Scoring System.
This project is licensed under the Apache License 2.0. See the LICENSE file for details.
Copyright 2025 seadonggyun4
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0