FableLoom asset manifest omits typed playback videos, stranding hold loops and exit clips during peer sync

[atomantic]

Problem

In server/services/sharing/peerSyncAssets.js:558-579, buildFableLoomAssetManifest(loom) builds the list of rendered scene media assets (images and videos) to advertise on peer-sync push payloads and in media integrity audits:

export async function buildFableLoomAssetManifest(loom) {
  const nodes = (Array.isArray(loom?.episodes) ? loom.episodes : [])
    .flatMap((episode) => (Array.isArray(episode?.nodes) ? episode.nodes : []));
  const dedup = new Map();
  const imageNames = [...new Set(
    nodes.map((node) => (isStr(node?.image) ? node.image : null)).filter(Boolean),
  )];
  for (const entry of await Promise.all(imageNames.map((name) => hashImageForManifest(name)))) {
    if (entry) dedup.set(`${entry.kind}:${entry.filename}`, entry);
  }
  const videoIds = [...new Set(
    nodes.map((node) => (isStr(node?.videoHistoryId) ? node.videoHistoryId : null)).filter(Boolean),
  )];
  if (videoIds.length > 0) {
    const entries = await Promise.all(videoIds.map((id) =>
      hashSimpleAsset(collectionVideoRefToFilename(id), 'video', PATHS.videos)));
    for (const entry of entries) {
      if (entry) dedup.set(`${entry.kind}:${entry.filename}`, entry);
    }
  }
  return [...dedup.values()];
}

Notice that videoIds only extracts node?.videoHistoryId.

However, FableLoom scene nodes support typed playback assets (server/lib/fableLoomPlayback.js:50-52 and server/services/fableLoom/records.js:1053-1105) stored in node.playbackAssets:

• playbackAssets.entryVideoHistoryId
• playbackAssets.holdLoopVideoHistoryIds (array of video history IDs)
• playbackAssets.exitByTransition ({ [transitionId]: videoHistoryId } map)

Each of these references an on-disk <id>.mp4 video under PATHS.videos. Because buildFableLoomAssetManifest only scans node.videoHistoryId, hold loops and transition exit clips are never gathered into assetManifest.

Trigger

1. An author generates or attaches typed playback assets to scene nodes in FableLoom (e.g. hold loop videos or transition exit clips via attachNodePlaybackAsset).
2. The story syncs to a peer via peer-sync push (server/services/sharing/peerSyncPush.js:831).
3. peerSyncPush calls buildFableLoomAssetManifest(record). The generated assetManifest contains the storyboard images and primary node.videoHistoryId, but omits all video history IDs referenced in node.playbackAssets.holdLoopVideoHistoryIds and node.playbackAssets.exitByTransition.
4. The receiving peer pulls missing assets according to the manifest via diffAssetManifestAgainstLocal. Because the hold loops and exit clips were never listed in the manifest, the receiving peer never requests or downloads them.
5. When a user on the receiving peer opens the player or launches a hosted session (server/services/fableLoom/hostedSession.js:68-73), the player looks for the hold loop or exit video files in PATHS.videos and encounters missing files / playback failure.
6. Similarly, the media integrity check (server/services/sharing/integrity.js:30) fails to detect that these playback videos are missing from disk.

Impact

Rendered video files are stranded on the originating machine. Synced peers receive JSON story metadata pointing to hold loop and transition exit videos that do not exist on the peer's filesystem, leading to broken playback in the viewer and hosted sessions.

Fix

In server/services/sharing/peerSyncAssets.js:

• In buildFableLoomAssetManifest(loom): Expand videoIds collection to gather all video IDs referenced across scene nodes:
  • node?.videoHistoryId
  • node?.playbackAssets?.entryVideoHistoryId
  • Elements of node?.playbackAssets?.holdLoopVideoHistoryIds
  • Values of Object.values(node?.playbackAssets?.exitByTransition || {})
• In server/services/sharing/peerSyncAssets.test.js:
  • Extend the buildFableLoomAssetManifest test suite to assert that video history IDs in playbackAssets.holdLoopVideoHistoryIds, playbackAssets.exitByTransition, and playbackAssets.entryVideoHistoryId are included in the generated asset manifest.

Alternative rejected: Synthesizing videoHistoryId to hold multiple IDs. Rejected because videoHistoryId is a single string for backwards-compatibility; typed multi-clip assets belong in playbackAssets.

Dispatch labels justification: model:light + effort:low because this is a self-contained, mechanical change in one helper file (peerSyncAssets.js) and its adjacent test (peerSyncAssets.test.js), requiring straightforward extraction and deduplication of strings from an already-sanitized object structure.

Acceptance Criteria

• buildFableLoomAssetManifest collects video history IDs from node.videoHistoryId, node.playbackAssets.entryVideoHistoryId, node.playbackAssets.holdLoopVideoHistoryIds, and node.playbackAssets.exitByTransition.
• Duplicate video IDs referenced in both node.videoHistoryId and node.playbackAssets are deduplicated.
• server/services/sharing/peerSyncAssets.test.js passes with tests covering manifest extraction of hold loop and transition exit video assets.

[Bryandero98]

I'd like to work on this — could you assign it to me?

[Bryandero98]

Fixed and merged in #6110.

[atomantic]

Already fixed on main.

buildFableLoomAssetManifest in server/services/sharing/peerSyncAssets.js now flat-maps every node's videoHistoryId, playbackAssets.entryVideoHistoryId, playbackAssets.holdLoopVideoHistoryIds, and Object.values(playbackAssets.exitByTransition) into a single Set, so hold loops and transition exit clips are advertised on peer-sync push payloads and picked up by the media integrity audit.

Evidence — commit 26effe868 ("fix: fableLoom asset manifest omits playbackAssets video clips (#6006)"), merged via #6110.

Acceptance criteria verified against origin/main:

• All four sources collected.
• Deduplicated via the surrounding new Set(...) — covered by the test "deduplicates a video id referenced by both node.videoHistoryId and playbackAssets".
• server/services/sharing/peerSyncAssets.test.js carries coverage for typed-playback-only nodes (entry-1 / hold-1 / exit-1) plus the dedup case.

Closing as completed. No further work needed.