unflatten peelstack

Flat image in, separated layers out — background.png plus one RGBA cutout per element, every layer complete.

The problem

A flat image has no parts. The moment you want something in it to move — parallax on a wallpaper, animating one element, recomposing a scene — you need it separated into layers, and each layer has to be whole.

That last word is where it gets hard. Segmentation only ever gives you the visible part of an object: anything standing in front of it punches a hole straight through its silhouette, and the pixels it was hiding were never drawn in the first place. Cut a figure out of a scene and the wall behind it is simply missing. Slide the figure aside and the hole shows.

Fixing that by hand means painting in content that never existed, once per layer, for every image. This does it automatically.

How

Elements come off nearest-first: peel the top element, reconstruct what it covered, then extract the next one from the filled image — so by the time a layer is lifted, whatever was hiding it is already gone and its hidden parts have been painted back in. Amodal completion falls out of the ordering rather than needing a dedicated model. What remains at the end is the background plate.

Layers are full canvas size with straight alpha, so they drop into any editor at the origin and line up.

Demo

Two public-domain paintings, run end to end. Both source images are CC0 — see Credits.

What peeling buys you that a cutout doesn't

Hiroshige, Kinryūsan Temple at Asakusa (1856). The title cartouches are printed on top of the lantern, so the segmenter returns a lantern silhouette with holes punched straight through it. Because the cartouches are nearer, they come off first and the fill reconstructs what they covered — so the lantern layer arrives whole. 61,855 pixels, 12.4% of that layer, were never in the source image.

That is the part a cutout cannot do. Everything else here — the silhouette, the soft edge — segmentation and matting already give you.

The layers themselves

Renoir, Two Sisters (On the Terrace) (1881), separated into four layers plus a plate. Each is full canvas size with straight alpha, so they stack back at the origin with no offsets: recomposite PSNR 32.7 dB, worst edge rim 0.109, occlusion seam 0.010.

And what it costs

What remains once every element is removed. The plate is the hardest thing the pipeline makes, because most of it was never visible: the fill behind the lantern scores a texture ratio of 0.43, below the 0.5 smear gate, and Stage F routes this image to review for exactly that reason. Reconstructed regions are softer than the original — a deliberate trade, since the alternative is a diffusion model inventing an object into the gap where nothing can see it.

Reproducing them

.venv/bin/python -m layerengine.cli asakusa.png --out out/asakusa \
  --plan docs/demo/asakusa.plan.json --max-side 1920

Both runs use a hand-written plan rather than a generated one. Left to itself the planner asked for falling snow and lantern cords, which are diffuse and thread-thin — this pipeline's two weakest cases — and eight elements became eighteen layers of compounding fill drift.

Install

python -m venv .venv
.venv/bin/pip install -e ".[dev]"      # core + tests, no torch
.venv/bin/pip install -e ".[models]"   # torch / transformers / pymatting
./scripts/setup_da3.sh                 # Depth Anything 3 (read the header first)

Run

cp .env.example .env      # a key for whichever plan provider you want
.venv/bin/python -m layerengine.cli IMAGE --out out/name

Without --plan, a vision model writes the scene plan and saves it to <out>/plan.json so the run stays reproducible. --fill picks the fill backend: auto (default), lama, sdxl, classical.

Pipeline

A plan → B segment → C depth & occlusion order → D matte → E peel & fill → F QA → G package. One module per stage under src/layerengine/, models behind swappable backends in backends/.

Notes

• Images are float32 (H, W, 4) in [0, 1], straight (non-premultiplied) alpha.
• Depth is disparity-like: larger means nearer.
• .venv/bin/python -m pytest runs the offline suite; -m models adds the backend contract tests, which download weights.

Design rationale and measurements: docs/design-notes.md. Spec: docs/layer-decomposition-pipeline-spec-v2.md.

Credits

The demo images are public domain and were chosen so this repository carries no licensing encumbrance:

• Utagawa Hiroshige, Kinryūsan Temple at Asakusa (1856) — The Met, Open Access (CC0). Cropped to the printed block.
• Pierre-Auguste Renoir, Two Sisters (On the Terrace) (1881) — Art Institute of Chicago, public domain (CC0).

Both artworks are long out of copyright, and both museums additionally release their scans under CC0, so the reproductions carry no separate claim either.

License

MIT — see LICENSE.