[Avatar] Explore a shared compiled mesh for WebGL editing and SVG export

[NWYLZW]

Status

Deferred architecture exploration for the oneworks-ai/avatar package. This issue records findings and a possible direction; it is not an implementation decision and no migration work is currently planned.

Problem

The SVG-native renderer works well for simple front-facing avatars, but complex 3D-style heads expose systemic problems at oblique angles:

• intersecting parts can reveal seams, hard ownership lines, or incorrect whole-part depth changes;
• screen-space decal fixes can detach color regions from the real curved surface;
• lowering interactive ownership resolution can introduce drag-time holes or jitter;
• rebuilding projected SVG paths every frame puts projection, path generation, React updates, SVG parsing, and rasterization on the main thread.

Reducing SVG DOM nodes alone does not solve this. An isolated prototype used only 4–7 final SVG paths and still missed the desired frame budget.

Local proof of concept

A local-only prototype was built without production integration:

1. Geometry/topology changes compile primitives into a stable, watertight triangle mesh.
2. Camera rotation reuses the compiled mesh instead of recompiling geometry.
3. Embedded attachment roots are clipped during compilation.
4. Continuous skin volumes use controlled smooth unions.
5. Separate anatomy such as a pointed beak remains independent geometry.
6. Color regions and mouth lines are evaluated in object space on the actual target surface.
7. The current pose can be projected into a small number of SVG paths.

Fixtures included intersecting spheres, embedded ears, a fused muzzle, a pointed bird beak, and a cat-style double muzzle. Across fixed multi-angle captures, the visual hypothesis held: no connected background cracks, no screen-space floating markings, correct rear-face hiding, and stable geometry caching.

Representative local measurements:

• geometry compile: approximately 53–66 ms when shape/topology changes;
• steady projection: approximately 7–8 ms;
• final output: approximately 4–7 semantic SVG paths;
• real drag rAF p95: approximately 35–42 ms;
• no geometry recompilation during view-only dragging.

The visual architecture is promising, but the CPU/SVG interaction path is not production-ready.

Possible future architecture

Avatar Definition
        ↓
Shared geometry compiler
        ↓
Compiled 3D mesh + surface semantics
  ├─ Three.js/WebGL renderer for the interactive editor
  └─ SVG projector/exporter for the current pose

Both backends would share compiled geometry, camera parameters, object-space markings, materials, semantic ownership, and cache invalidation rules. The editor would not regenerate SVG on every pointer move. SVG would be generated on demand or after settling, while PNG/GIF/animated previews could use the GPU renderer.

Important constraints

• SVG export must remain genuine editable vector output, not a raster image wrapped in SVG.
• WebGL and SVG must not implement separate geometry or decal rules.
• Pure color regions must remain surface semantics, not additional geometry.
• Continuous skin/fur volumes and separate anatomy require different compile rules.
• Existing Avatar Definition, URL, SDK, and saved-avatar compatibility must be evaluated before migration.
• A hybrid renderer must preserve deterministic exports and user-selected poses.

Open questions

• How closely can WebGL raster output and SVG export match at silhouettes and anti-aliased junctions?
• Should projection/export run in a Worker/WASM module?
• Can one object-space marking representation compile reliably to both shaders and SVG surface paths?
• What cache/invalidation model should be used for continuously edited geometry?
• How should hit testing and selected-part overlays share the compiled mesh?
• Is the maintenance cost justified compared with limiting avatar topology/rotation?

Acceptance criteria for revisiting

• One compiled mesh is the only geometry truth for both renderers.
• No connected background gaps, hard third-color seams, or floating markings across a fixed 26-direction matrix.
• Interactive drag reaches an agreed frame budget without reducing ownership topology.
• Geometry changes compile off the main thread or without visible long tasks.
• Exported SVG remains deterministic and editable.
• Pixel comparisons cover front, ±30°, ±60°, ±85°, profile, rear, and pitch ±30°.
• At least one real animal with fused soft volumes and one with independent hard anatomy pass browser review.
• No breaking public-schema migration is required without an explicit versioning plan.

Related work

• Avatar draft PR: feat: expand animal avatars and surface-aware rendering #22