Skip to content

Ruffle Flash Integration

Jump to bottom
chameleonxxl edited this page May 17, 2026 · 2 revisions

dirplayer-rs uses Ruffle — an open-source Flash Player runtime — to render and script Flash (.swf) cast members embedded in Director movies. Each Flash member becomes a Ruffle instance hosted in a hidden <div>; pixels are read back and composited into the dirplayer canvas so the SWF appears as a regular Director sprite. ActionScript ↔ Lingo is bridged via a set of JavaScript shims that mirror Director's Flash Lingo API on top of Ruffle's player object.

⚠ Custom Ruffle fork required, but coexists with stock Ruffle.

The Lingo↔Flash bridge depends on patches we maintain in our Ruffle fork (the dirplayer-integration branch of chameleonxxl/ruffle). The patches add:

  • A triggerLingoCallbackOnScript extern that fires from ExternalInterface.call, plus __dirplayer_ref_N object handle storage on _root so AS-returned objects can be re-resolved by Lingo;
  • dirplayerDispatchPointer for injecting Director-side mouseDown/Up/Move events into the offscreen Ruffle canvas;
  • dirplayerCallOpenUrl (+ dirplayer_addOpenUrlHandler) so getURL("event: …") routes into the host movie's Lingo event chain instead of opening a tab;
  • DirPlayer-specific player methods extending the AVM1 ExternalInterface shape (setCallback, callable FlashObjectRef property chains, etc.).

Co-existence with stock Ruffle on the same page is supported. The public surface of the fork is namespaced under the dirplayer_ prefix — custom element tag, global (window.dirplayer_RufflePlayer), loader bundle (dirplayer_ruffle.js), webpack chunk-loading global, and every wasm-bindgen JS extern. A page that already has stock Ruffle loaded (browser extension, <script src="ruffle.js"> etc.) won't collide with us. Conversely, calling stock Ruffle through our bridge would silently break the advanced Lingo paths (setCallback, callable FlashObjectRefs, method chains) — they need the patches.

This page documents:

  • The architectural roles of each side (Ruffle, JS bridge, WASM)
  • Every Lingo Flash API we forward to Ruffle (and how)
  • The ActionScript-side lingo callback machinery
  • Frame buffer compositing
  • Per-page configuration knobs you'll likely touch

For setup details (where Ruffle is loaded from, data-ruffle-url, __dirplayerFlashConfig), see the Polyfill / Embedding wiki page.

Architecture

                ┌─────────────────────────────────────────────────────┐
                │                Lingo movie                          │
                │   sprite(N).getVariable("/:foo")                    │
                │   member("clip").goToFrame(5)                       │
                │   sprite.fooMethod(arg1, arg2)   ← Flash methods    │
                └─────────────────────────────────────────────────────┘
                                       │ Lingo bytecode
                                       ▼
   ┌────────────────────────────────────────────────────────────────────┐
   │           dirplayer-rs WASM  (vm-rust/src/...)                     │
   │  ┌─────────────────────────────┐                                   │
   │  │ sprite.rs       gotoFrame,  │                                   │
   │  │                 stop, play, │   wasm-bindgen externs,           │
   │  │                 hitTest, …  │   resolved to window.ruffle*      │
   │  │ flash_object.rs getVariable │                                   │
   │  │                 setVariable │                                   │
   │  │                 callFunction│                                   │
   │  │                 (FlashObject│                                   │
   │  │                  property   │                                   │
   │  │                  chains)    │                                   │
   │  └─────────────────────────────┘                                   │
   └────────────────────────────────────────────────────────────────────┘
                                       │ window.ruffle* JS bridge
                                       ▼
   ┌────────────────────────────────────────────────────────────────────┐
   │         flashPlayerManager.ts  (JS bridge)                         │
   │  per-sprite map: spriteNum → FlashInstance                         │
   │   - one Ruffle <player> per sprite (multiple sprites can share a   │
   │     single Flash cast member and still have independent playheads) │
   │   - hidden <div>, canvas frame capture loop                        │
   │   - SWF signature scan (FWS / CWS / ZWS), strip Director header    │
   │   - GetVariable / SetVariable / GotoFrame / CallFunction / hitTest │
   │   - Pending-op queue: gotoFrame/play/stop/rewind that arrive       │
   │     before SWF is ready are queued, drained at ready-time          │
   │   - Playhead inheritance from sibling sprites of the same member   │
   └────────────────────────────────────────────────────────────────────┘
                                       │ Ruffle's selfhosted API
                                       ▼
   ┌────────────────────────────────────────────────────────────────────┐
   │                       Ruffle player                                │
   │  AS1 / AS2 / AS3 SWF runtime, Canvas2D renderer (forced)           │
   │  XMLSocket → window.dirplayerResolveSocketUrl → ws-tcp proxy       │
   │  wmode='transparent' so SWFs can layer over Director sprites       │
   └────────────────────────────────────────────────────────────────────┘
                                       │ canvas.getContext('2d').getImageData
                                       ▼
   ┌────────────────────────────────────────────────────────────────────┐
   │   WASM frame buffer  ─►  rendering.rs → BitmapManager              │
   │     player.flash_frame_buffers: HashMap<i16, BitmapRef>            │
   │     (i16 key = sprite number; one entry per Flash sprite)          │
   │   composited as a regular sprite by the score renderer             │
   └────────────────────────────────────────────────────────────────────┘

Two reasons we render Ruffle off-screen and copy pixels in:

  1. Director compositing — Flash members are sprites, ink modes apply, they may be rotated/skewed, occluded by other sprites, etc. We need the rendered frames as raw RGBA so the score renderer can run them through the same pipeline as bitmaps.
  2. Ruffle's WebGL renderer can't be read back from another WebGL context. We force renderer: "canvas" so we can use getImageData(). Performance cost is real but acceptable.

Lingo APIs we forward to Ruffle

All bindings live in src/services/flashPlayerManager.ts. The WASM side declares matching wasm_bindgen externs (mostly in vm-rust/src/player/handlers/datum_handlers/sprite.rs and .../flash_object.rs).

Frame / playback control

These are exposed on the sprite datum (sprite(N).gotoFrame(2), sprite(N).stop(), …), as well as on the Flash member directly. All bridge calls take spriteNum (not (castLib, castMember)) because each Flash sprite has its own Ruffle instance — see the architecture note above.

Lingo call JS bridge Ruffle op Notes
sprite(N).gotoFrame(frameOrLabel) ruffleGoToFrame(spriteNum, frameOrLabel: str) numeric → player.GotoFrame(frame, false) (gotoAndPlay); label → AS1 _root.gotoAndPlay(label) via CallFunction Director's gotoFrame is gotoAndPlay-semantic — seek then keep playing. Accepts both integer frame indices AND string labels (sprite(N).gotoFrame("warm0")).
the frame of sprite N = X ruffleGoToFrameAndStop(spriteNum, frameOrLabel: str) numeric → player.GotoFrame(frame, true) (gotoAndStop), behind a microtask-pin; label → AS1 _root.gotoAndStop(label) The frame property setter is gotoAndStop-semantic — seek and halt at the target. Distinct from gotoFrame method above. Microtask-pin prevents Ruffle's render-loop RAF from ticking past the target between the goto and the stop.
sprite(N).stop() / hold ruffleStop(spriteNum) player.pause()
sprite(N).play() / resume rufflePlay(spriteNum) player.play() then GotoFrame(_currentframe, false) to clear any MovieClip-internal stopped flag set by AS A bare player.play() only flips the player-level paused flag; if the SWF's own AS called stop() on a MovieClip, only GotoFrame(_, false) (gotoAndPlay) clears the MovieClip's internal stopped flag and lets the playhead actually advance.
sprite(N).rewind() ruffleRewind(spriteNum) player.GotoFrame(1, true)
isPlaying ruffleIsPlaying(spriteNum) player.IsPlaying()
frameCount ruffleGetFrameCount(spriteNum) player.TotalFrames
currentFrame ruffleGetCurrentFrame(spriteNum) player.CurrentFrame + 1 Director-side rebased to 1
callFrame frame ruffleCallFrame(spriteNum, frame) player.CallFrame(frame - 1)
findLabel "name" ruffleFindLabel(spriteNum, label) scan via GetVariable("_root._currentlabel")
hitTest x, y ruffleHitTest(spriteNum, x, y) bounding-box hit test Coordinate space matches sprite rect

Variable get/set (AS1/AS2 timeline vars)

Lingo JS Ruffle
sprite(N).getVariable("/:foo") ruffleGetVariable(spriteNum, path) player.GetVariable(path)
sprite(N).setVariable("/:foo", v) ruffleSetVariable(spriteNum, path, value) player.SetVariable(path, value)

The path syntax is Flash's classic timeline notation (/:varname, level0:foo, _root.foo, myMC.bar, …). The bridge does light path normalisation via translateLevel0() so _level0/whatever is rewritten into the form Ruffle understands.

getFlashProperty / setFlashProperty

Director's numeric Flash-property accessors are mapped to AS1 globals:

# Property Notes
0 / 1 _x / _y
2 / 3 _xscale / _yscale
4 _currentframe get-only
5 _totalframes get-only
6 / 7 _alpha / _visible
8 / 9 _width / _height
10 _rotation
11 _target get-only
12 _framesloaded get-only
13 _name
14 _droptarget get-only
15 _url get-only
16 _highquality
17 _focusrect get-only
18 _soundbuftime
19 _quality get-only
20 / 21 _xmouse / _ymouse get-only

Internally getFlashProperty(target, propNum) is rewritten to GetVariable("<target>:<propName>") (or /:<propName> if target is empty). Get-only props are silently dropped on setFlashProperty.

tellTarget / endTellTarget

Lingo JS bridge Implementation
tellTarget "/myMC" … end tellTarget ruffleTellTarget(spriteNum, target, action) The bridge rewrites contained gotoFrame/stop/play/setVariable calls into target-prefixed Ruffle ops; classic AS1 tellTarget semantics.

callFunction / Flash methods

ExternalInterface-style invocation of an AS-side function:

result = sprite(N).fooMethod(1, "two", [3, 4])

→ ruffleCallFunction(spriteNum, "fooMethod", argsXml). Args are encoded as flash: XML (the same format Director's official Flash Asset Xtra used) and decoded inside Ruffle. The return value is unwrapped: strings, numbers, booleans become Lingo primitives; objects become a FlashObjectRef for further property access.

Property chains via FlashObjectRef

When Ruffle returns an object reference (e.g. _root.someMC), the WASM side wraps it in a FlashObjectRef carrying the path string. Lingo property access (flashObj.x, flashObj.foo.bar) walks the path one hop at a time:

  • FlashObjectDatumHandlers::get_prop calls ruffleGetVariable(spriteNum, "<path>.<prop>") and decides the result type at runtime (string / int / float / bool / nested object). The sprite number is resolved by scanning the score for a sprite whose member matches the FlashObjectRef's (cast_lib, cast_member).
  • FlashObjectDatumHandlers::call calls ruffleCallFunction with the full dotted path.

The __ruffle_path: marker (visible in FlashObjectRef.to_string()) is how Lingo string(flashObj) reports the inspector form.

setCallback(flashObj, "asMethod", "lingoHandler", lingoTarget)

Registers a callback so that ActionScript can call Lingo back (via Flash's ExternalInterface.call("asMethod", …)). Implementation:

  1. Lingo invokes sprite(N).setCallback(...).
  2. WASM (sprite.rs) calls dirplayer_ruffleRegisterLingoCallback(movieClipPath, asMethod, lingoCastLib, lingoCastMember, lingoHandler, flashCastLib, flashCastMember).
  3. The fork's Ruffle stores the mapping (in LINGO_CALLBACKS) and intercepts ExternalInterface.call on the matching MovieClip, packaging the AS args and posting them to window.triggerLingoCallbackOnScript(...).
  4. triggerLingoCallbackOnScript is a thin shim that decodes and forwards into the WASM-exported trigger_lingo_callback_on_script, which finds the script by (cl, cm) and runs the handler with the decoded args.

Mouse-event forwarding (Director click → SWF)

Since the Ruffle canvas is hidden (offscreen) for compositing, real browser pointer events never reach the SWF — AS1 button handlers (on (press), on (release)) wouldn't fire on Director-side clicks. The mouseDown / mouseUp command handlers in commands.rs forward the click into Ruffle's input pipeline via dirplayer_ruffleDispatchMouse → dirplayerDispatchPointer (the synthetic input extern in the fork).

Gating: forwarding only happens if get_sprite_at(scripted=true) returns None at the click point. If Director has a script behavior on the sprite, that script is the intended handler and we must NOT also inject the click into Flash — otherwise animation-only Flash sprites (prepareFrame-driven, no AS1 buttons) get frozen by stray on (press) { stop() } handlers that happen to be in their SWF.

getURL("event: …") → Lingo event chain

Director's Flash Asset Xtra intercepts SWF getURL("event: send #done") URLs and feeds the body into the host movie's Lingo event chain. The fork's Ruffle implements this hook: WebNavigatorBackend::navigate_to_url short-circuits any URL starting with event: and calls dirplayerCallOpenUrl on the JS host. flashPlayerManager.ts registers an open-URL handler that parses the body as send <handler> [args…] and dispatches a global event (walks active sprite behaviors → frame script → movie scripts). See lib.rs::dispatch_flash_event.

pausedAtStart (Flash member property)

The Flash member property pausedAtStart is parsed from the FlashInfo chunk and honoured at instance load: after Ruffle finishes init we pin the SWF at frame 1 (forced paint then gotoAndStop) so it doesn't visibly cycle through autoplay until Lingo explicitly calls sprite(N).play(). Director's Flash sprites that ship "paused" rely on this — a tile poster sprite that should display frame 1 only and never animate.

Frame buffer ↔ Director sprite

flashPlayerManager.startFrameCapture() runs a requestAnimationFrame loop that reads pixels from Ruffle's internal canvas via ctx.getImageData() and pushes the RGBA bytes to WASM (update_flash_frame(spriteNum, w, h, rgba)). The WASM side (vm-rust/src/lib.rs) keeps a per-sprite map:

player.flash_frame_buffers: HashMap<i16, BitmapRef>

When spriteNum already has an associated BitmapRef, the existing bitmap is updated in place (so the Bitmap ID is stable and the texture cache can keep its entry). Otherwise a new bitmap is allocated.

The texture cache key in texture_cache.rs also includes the bitmap's image_ref so multiple sprites sharing one Flash member (storyscramble's 3 story tiles all reference cast 2:1) don't collapse into a single cached texture.

The score renderer (rendering.rs) and the WebGL2 pipeline (rendering_gpu/webgl2/mod.rs) treat this entry as a regular bitmap source for the sprite — so ink, blend, rotation, and skew apply just like bitmap sprites. With the following Flash-specific carve-outs:

  • blend is forced to 100 for any Flash member, regardless of sprite-level blend or directToStage. Director's D8-era Flash compositor ignored sprite blend for Flash members; without this override, score-data blend values (e.g. blend=50) wash out Flash sprites that should render at full opacity.
  • Ink 36 (Background Transparent) trusts the embedded alpha for Flash bitmaps (is_flash=true on the TextureSource::Bitmap variant). Ruffle's wmode='transparent' already encodes transparency in the alpha channel via the SWF stage, so layering a secondary bg-color key on top would destructively strip artwork whose fill matches sprite.bg_color (storyscramble's white-filled chat bubble + sprite bg=white → entire bubble erased). Plain authored 32-bit bitmaps with embedded alpha keep the bg-color key path because that's the only way ink 36 can actually do its job for them.
  • centerRegPoint is computed from the real SWF FrameSize (parse_swf_dimensions) → reg_point = (w/2, h/2). The cached FlashInfo.reg_point field was unreliable enough that the earlier /2-based fallback shifted sprites toward the top-left by a quarter of their dimensions.

SWF signature handling

Director embeds the raw SWF inside its own cast-member chunk, sometimes with a small header prefix. Before handing data to Ruffle, the bridge scans the first bytes for the SWF magic FWS / CWS / ZWS and slices from that offset. If no signature is found, the load is aborted with a warning — Ruffle would fail anyway.

This handles legacy Director files that wrap the SWF data; the Director Xtra used to do the same scan.

XMLSocket / network plumbing

Ruffle's AS sockets (XMLSocket, the Socket AS3 class, ActionScript SOCKET events) are routed through dirplayer's socket proxy bridge, not opened directly:

  1. AS calls XMLSocket.connect("host", port).
  2. Ruffle's socketProxy config (set on every Ruffle instance from getSocketProxyConfig()) maps (host, port) → a WebSocket URL.
  3. Ruffle opens that WebSocket; a server-side TCP relay forwards bytes to the original target.

The same socketProxy array is used by dirplayer's own Multiuser Xtra sockets via window.dirplayerResolveSocketUrl. See Polyfill / Embedding → socketProxy.

The JS-side fetchRewriteRules (applyFetchRewrite()) also intercepts fetch() calls to rewrite host/port — used when a SWF hard-codes URLs that aren't reachable from the dev page.

Currently implemented vs. not yet

✅ Implemented

  • gotoFrame — accepts both numeric frame indices AND string labels (sprite(N).gotoFrame("warm0")). Method form is gotoAndPlay-semantic; the the frame of sprite N = X property setter is the gotoAndStop variant via dirplayer_ruffleGoToFrameAndStop.
  • stop, play, rewind, hold, callFrame, isPlaying, frameCount, currentFrame, findLabel
  • getVariable, setVariable (timeline /: paths and target-prefixed)
  • callFunction / dotted method invocation, with ExternalInterface-XML argument encoding
  • Numeric getFlashProperty / setFlashProperty (props 0–21)
  • tellTarget / endTellTarget
  • setCallback (AS → Lingo callback registration)
  • hitTest (bounding-box)
  • Frame buffer capture → Director compositing (ink, blend, rotation, skew supported via the bitmap path; Flash-specific blend=100 + ink-36 alpha-trust carve-outs documented above)
  • Per-sprite Ruffle instances — multiple sprites that share one Flash cast member each get an independent player + playhead (storyscramble's 3 story tiles share cast 2:1 but display poster frames 2/4/6 simultaneously). Sibling-playhead inheritance on instance creation so a fresh instance starts at the frame its siblings already advanced to.
  • Pending-op queue — goto/play/stop/rewind calls that arrive before the SWF is ready are queued, drained at ready-time so beginSprite-time scripting works on still-loading Flash sprites.
  • pausedAtStart member property honoured — the SWF is pinned at frame 1 after load until Lingo explicitly calls play.
  • Mouse forwarding — Director's mouseDown/Up forwarded into the offscreen Ruffle canvas via dirplayerDispatchPointer, gated on no Director-side script being present at the click location.
  • getURL("event: …") routing — SWF event-URLs feed back into the host movie's Lingo event chain (event: send #done → on done) instead of opening browser tabs.
  • SWF magic detection (FWS / CWS / ZWS)
  • XMLSocket via __dirplayerFlashConfig.socketProxy
  • Fetch rewrites via __dirplayerFlashConfig.fetchRewriteRules
  • Loading-state gate (isFlashLoading()) — the WASM frame loop waits for in-progress Flash instances to come up before scripting them. Sound-channel fades keep ticking via tick_sound_manager during these waits so a sound fadeIn started just before a Flash load doesn't sit at gain=0 for the entire load duration.

⚠ Partial

  • hitTest is bounding-box only — Director's hitTest returned 0 (transparent) / 1 (frame) / 2 (drawing) / 3 (clickable shape); we always return a single boolean.
  • Numeric Flash properties 22+ (extended set in MX2004) are not mapped — the bridge silently no-ops on unknown propNum.
  • tellTarget rewrites are limited to the subset of inner commands we forward; complex AS1 dialects with computed targets may not match Director's exact resolution.

❌ Not implemented

  • AS3-only ExternalInterface argument types beyond JSON-able primitives — for example AS3 ByteArray, MovieClip references passed as arguments, etc. Strings / numbers / booleans / arrays / plain objects work.
  • stageToFlash / flashToStage coordinate transforms — these haven't come up in the games we run; raise an issue with a reproducer if you hit one.
  • viewScale, originMode, streamMode, loop, actionsEnabled, bufferSize, imageEnabled, static member properties — most are no-ops; we always render with autoplay: on, unmuteOverlay: hidden, splashScreen: false.
  • Direct WebGL renderer for Flash content — we force Canvas2D so we can read pixels back. A future fast path would render Ruffle to a separate WebGL FBO and texture-share with dirplayer's GL context.

Files of interest

File Role
src/services/flashPlayerManager.ts Whole JS bridge: instance lifecycle, frame capture, every window.ruffle* shim
vm-rust/src/player/handlers/datum_handlers/sprite.rs Flash sprite Lingo methods (gotoFrame, getVariable, callFunction, setCallback, …)
vm-rust/src/player/handlers/datum_handlers/flash_object.rs FlashObjectRef property/method access — bridges chained AS1/AS2 object references
vm-rust/src/lib.rs trigger_lingo_callback_on_script (AS → Lingo callback entry point) + Flash frame buffer ingestion
polyfill/src/standalone.tsx Polyfill init: loads Ruffle relative to the polyfill script (or data-ruffle-url)

Debugging tips

  • Set __dirplayerFlashConfig.logLevel = "info" or "debug" to get Ruffle's own log output mixed into the console (see Polyfill / Embedding).
  • Loading state — if Lingo accesses a Flash sprite before its SWF has finished initialising, the bridge sets a flashAccessBeforeReady flag so the next frame waits. Check isFlashLoading() from the console to see if anything's still pending.
  • Frame capture not running? Ensure the SWF actually rendered a canvas (Ruffle creates <canvas> inside the player's shadow root or light DOM). The bridge looks in both; if neither has a canvas after 500 ms, capture won't start.
  • __vm console helpers — see Debug Console Commands. Flash- specific: window.dirplayer_flashInstances is the live per-sprite Map<spriteKey, FlashInstance> from flashPlayerManager.ts — inspect from the console to see what's loaded, which sprite holds which Ruffle player, the underlying canvas, etc.

Clone this wiki locally