Skip to content

Latest commit

 

History

History
3057 lines (2273 loc) · 62.2 KB

File metadata and controls

3057 lines (2273 loc) · 62.2 KB

Agent Effects

MVP Product Requirements Document

Document status: Draft v1.0 Date: 26 July 2026 Owner: Duncan Trevithick Working repository: agent-effects Primary CLI: agent-effects Optional CLI alias: afx Project file extension: .afx.yaml


1. Executive summary

Agent Effects is an agent-first motion-graphics engine that converts structured story, scene and animation descriptions into rendered video.

A compiled Rust application will interpret declarative project files, render typography and graphics natively, and stream raw frames directly to FFmpeg. AI coding agents will create and modify the project files without writing Rust, controlling a constrained library of story structures, semantic compositions, layouts, animation presets, transitions and visual effects.

The initial product will support both:

  • short 30–60 second motion-graphics explainers
  • long-form 5–15 minute editorial and YouTube videos composed from independently renderable scenes

The MVP will focus on:

  • kinetic typography
  • simple vector graphics
  • statistics and comparisons
  • lists and process diagrams
  • photographs and editorial collage layouts
  • website and app screenshots
  • stylised browser and device frames
  • screenshot callouts and annotations
  • constrained maps of Spain
  • textured backgrounds
  • drop shadows and layered depth
  • narration-aligned timing
  • deterministic scene transitions
  • bounded-memory long-form rendering
  • automated visual feedback for AI agents

Agent Effects is not intended to reproduce After Effects.

Its purpose is to prove that an AI agent can autonomously create a coherent, attractive and efficiently rendered motion-graphics video using a small, opinionated visual language.


2. Problem

Current browser-based video systems such as HyperFrames and similar HTML-to-video pipelines incur substantial overhead when capturing each rendered frame.

In the current workflow:

  • approximately 94% of render time is spent capturing frames
  • each frame capture takes approximately 750 milliseconds
  • Chromium must lay out, paint and rasterise the scene
  • a screenshot must then be extracted from the browser
  • the result may be compressed, transferred and decoded before video encoding

At 30 frames per second, an 11-minute video contains approximately 19,800 frames.

At 750 milliseconds per capture, frame extraction alone would take approximately:

19,800 × 0.75 seconds = 14,850 seconds
                           = 247.5 minutes
                           = 4 hours 7 minutes

This excludes other rendering, media processing and encoding work.

Rewriting only the browser controller in Rust would not remove the underlying bottleneck because Chromium would still perform the layout, rasterisation and screenshot extraction.

Replacing HTML with low-level Rust or GPU code would solve part of the performance problem, but it would create an authoring problem. An AI agent should not need to implement text shaping, geometry, shaders, compositing or frame generation for every video.

Agent Effects therefore needs to solve two problems simultaneously:

  1. Eliminate the browser screenshot boundary.
  2. Give AI agents a constrained and discoverable language for creating attractive videos.

3. Product vision

Create an agent-first motion-graphics system with:

  • an After Effects-like internal rendering model
  • a Canva-like collection of semantic components
  • a declarative language designed for AI coding agents
  • a compiled native Rust renderer
  • deterministic rendering
  • reusable branded visual systems
  • automated structural and perceptual feedback
  • a lightweight human review and refinement interface

The long-term workflow should allow a user to provide:

  • a script
  • a narration track
  • a transcript
  • a visual theme
  • photographs
  • screenshots
  • maps or supporting data
  • a target duration
  • an aspect ratio

An agent should then be able to:

  1. identify the central message
  2. construct a beat sheet
  3. organise beats into scenes
  4. choose semantic compositions
  5. create the project DSL
  6. validate it
  7. render representative frames
  8. inspect visual and motion feedback
  9. revise the project
  10. render the final video

4. Product promise

Give an AI agent a script, narration, visual theme and assets. Receive a polished short- or long-form motion-graphics video without browser screenshot capture, per-video Rust compilation or manual After Effects work.

The Rust renderer is compiled when the engine changes.

Individual videos remain data:

agent-effects render project.afx.yaml --output video.mp4

5. Product principles

5.1 Meaning before motion

Motion should reinforce the meaning, hierarchy and rhythm of the story.

The system should discourage arbitrary animation used solely for visual variety.

A word such as “collapse” may compress or fall. A statistic such as “94%” may dominate the frame. A transition representing resolution may simplify or settle the composition.

5.2 Story before scenes

The agent should identify what the audience needs to understand before choosing visual layouts.

The primary storytelling unit is a beat, not a scene.

5.3 Semantic authoring before low-level control

Agents should usually choose concepts such as:

  • headline
  • statistic
  • comparison
  • evidence
  • reveal
  • emphasise
  • connect
  • resolve

They should not normally position every rectangle or manually define every keyframe.

5.4 Strong defaults over unlimited flexibility

A small collection of polished components should outperform a broad system with inconsistent visual results.

5.5 Presets before parameters

The agent should prefer:

entrance:
  preset: words-rise
  feel: confident

over:

position:
  from: [0, 72]
  to: [0, 0]
opacity:
  from: 0
  to: 1
duration: 0.57
easing:
  cubic_bezier: [0.16, 1, 0.3, 1]

Explicit control remains available as an escape hatch.

5.6 Deterministic rendering

The same project, engine version, assets, font files and random seed should produce the same frames.

5.7 The project file is the source of truth

The CLI, UI and agents must all operate on the same declarative project files.

The UI must not create important hidden state that cannot be represented in the DSL.

5.8 Fast feedback before final rendering

Agents should inspect:

  • validation results
  • representative still frames
  • contact sheets
  • low-resolution previews
  • motion telemetry
  • optional video-model critique

before producing a full-quality final render.

5.9 Bounded resource use

Long videos must not require retaining all rendered frames, decoded media or scene surfaces in memory.

5.10 Progressive complexity

The authoring system should expose four levels:

  1. complete scene templates
  2. semantic compositions
  3. animation presets
  4. explicit layers and keyframes

6. Target users

6.1 Primary user

A developer or technical creator using an AI coding agent to produce short or long-form explainer, educational, marketing, property or editorial videos.

6.2 Secondary users

  • YouTube creators
  • SaaS marketing teams
  • automated content systems
  • developer-tool companies
  • educational-content producers
  • agencies producing repeated branded videos
  • humans reviewing agent-generated videos

7. Jobs to be done

7.1 Creator job

Given a script, narration and visual theme, create an attractive motion-graphics video without manually animating every layer.

7.2 Story agent job

Turn a script or transcript into a clear visual argument made from beats, scenes and recurring motifs.

7.3 Motion agent job

Discover the available visual language, produce a valid project, render previews, inspect feedback and improve the result without modifying the Rust renderer.

7.4 Reviewer job

Quickly understand the story, pacing and visual problems, then approve or adjust the generated video.

7.5 Engine developer job

Add reusable visual capabilities without breaking the appearance of existing projects.


8. MVP hypothesis

A constrained system focused on kinetic typography, editorial graphics and reusable semantic compositions can cover most of the visual needs of:

  • short product explainers
  • technical explainers
  • property and location videos
  • narrated YouTube essays
  • SaaS marketing clips
  • educational videos
  • social-video derivatives

The MVP succeeds if an AI agent can autonomously produce a coherent video containing:

  • a clear hook
  • an understandable narrative arc
  • readable kinetic typography
  • images or screenshots
  • statistics, comparisons or simple processes
  • deliberate transitions
  • narration-aligned timing
  • consistent visual styling
  • no significant layout or rendering errors

The agent should accomplish this without:

  • writing Rust
  • compiling Rust per video
  • controlling Chromium
  • manually capturing screenshots
  • creating arbitrary shaders
  • using a traditional After Effects timeline
  • retaining the entire video in memory

9. Benchmark project

The primary MVP benchmark should be an approximately 11-minute editorial video containing:

  • kinetic typography
  • multiple sections and narrative beats
  • maps of Spain
  • highlighted provinces such as Almería and León
  • map markers and simple zooms
  • photographs of properties
  • property-listing screenshots
  • browser-window compositions
  • Polaroid-style image cards
  • tape and sticker overlays
  • paper-like textured backgrounds
  • drop shadows
  • simple diagrams and statistics
  • narration-synchronised transitions
  • long settled still-image intervals

This benchmark is intended to test:

  • long-form project organisation
  • constrained map support
  • editorial image treatments
  • screenshot presentation
  • memory-bounded rendering
  • scene-level caching
  • resumability
  • real rendering performance

10. MVP scope

10.1 Story construction

The MVP includes:

  • story goal
  • audience
  • objective
  • central takeaway
  • tone
  • narrative arc
  • ordered beats
  • beat purposes
  • desired audience responses
  • beat-to-scene grouping
  • semantic relationships between scenes
  • recurring visual motifs
  • pacing metadata

10.2 Primitive visual elements

The MVP includes:

  • text
  • numbers
  • rectangles
  • rounded rectangles
  • circles
  • ellipses
  • lines
  • arrows
  • connectors
  • SVG assets
  • raster images
  • groups
  • clipping masks
  • solid backgrounds
  • gradient backgrounds
  • raster-image backgrounds
  • tiled texture backgrounds
  • deterministic noise or grain overlays

10.3 Typography

The MVP includes:

  • font family
  • font weight
  • font style
  • font size
  • line height
  • tracking
  • word spacing
  • automatic wrapping
  • automatic fitting
  • minimum font-size constraints
  • emphasis spans
  • highlighted words
  • foreground and accent colours
  • left, centre and right alignment
  • line splitting
  • word splitting
  • character splitting
  • text masks
  • safe-area validation

10.4 Images and editorial collage

The MVP includes:

  • image placement
  • image cropping
  • cover and contain fitting
  • image scale and rotation
  • rounded corners
  • image borders
  • drop shadows
  • image tint
  • Polaroid-style cards
  • photo stacks
  • taped photographs
  • overlapping paper panels
  • caption labels
  • decorative PNG and SVG overlays
  • collage layout presets

10.5 Browser and device frames

The MVP includes constrained semantic wrappers for:

  • desktop browser
  • generic laptop
  • generic phone

Browser-frame support includes:

  • stylised browser chrome
  • configurable toolbar visibility
  • URL-bar text
  • page title
  • macOS-style window controls
  • screenshot viewport
  • rounded corners
  • border
  • drop shadow
  • viewport crop and pan
  • optional cursor
  • optional callout
  • optional highlighted region

These are graphic compositions, not functional browsers.

10.6 Constrained maps

The MVP includes:

  • imported SVG maps with targetable element IDs
  • built-in maps of Spain
  • Spanish autonomous communities
  • Spanish provinces
  • region highlighting
  • region dimming
  • point markers
  • pulsing markers
  • text labels
  • simple routes
  • country-to-region zoom
  • predefined map camera bounds
  • pan and zoom transitions
  • path fill, stroke and opacity animation

The MVP does not include a complete geographic-information system.

10.7 Motion

The MVP includes:

  • position
  • scale
  • rotation
  • opacity
  • colour
  • blur
  • clipping and reveal progress
  • image crop position
  • image crop scale
  • background position
  • texture opacity
  • texture scale
  • shadow properties
  • SVG fill and stroke
  • map camera bounds
  • browser viewport transforms
  • named easing curves
  • custom cubic Bézier curves
  • deterministic springs
  • stagger
  • parent-child transforms
  • basic motion paths
  • constant-speed path traversal
  • orient-to-path
  • scene transitions
  • basic motion blur where performance permits

10.8 Semantic compositions

The MVP includes:

  • headline
  • question
  • statement
  • statistic
  • quote
  • bullet list
  • ranked list
  • comparison
  • before-and-after
  • steps
  • process
  • callout
  • screenshot
  • screenshot with callouts
  • browser frame
  • device frame
  • photo card
  • photo stack
  • SVG map
  • location map
  • simple route map
  • end card

10.9 Audio and timing

The MVP includes:

  • narration audio
  • external music track
  • external sound-effects track
  • word-level transcript timing
  • phrase markers
  • named timeline markers
  • alignment to words and phrases
  • scene-duration validation
  • minimum settled-reading-time warnings
  • audio muxing through FFmpeg
  • scene timing independent of media decoding

10.10 Rendering and review

The MVP includes:

  • frame rendering at an arbitrary timestamp
  • scene-specific rendering
  • time-range rendering
  • contact-sheet generation
  • low-resolution preview generation
  • full-resolution MP4 rendering
  • chunked long-form rendering
  • resumable rendering
  • machine-readable diagnostics
  • profiling output
  • optional Kimi video-critique bridge
  • local review UI

11. Explicit MVP exclusions

The MVP does not include:

  • arbitrary HTML or CSS rendering
  • live browser capture
  • live webpages
  • complete global mapping
  • map tiles
  • satellite imagery
  • road routing
  • arbitrary address geocoding
  • terrain rendering
  • advanced 3D
  • character rigging
  • particle systems
  • arbitrary shader authoring
  • freehand vector illustration
  • complex path morphing
  • full Lottie authoring
  • full Rive authoring
  • After Effects project compatibility
  • third-party effects plug-ins
  • full keyframe timeline editing
  • Bézier graph editing
  • collaborative cloud editing
  • cloud rendering
  • mobile editing
  • generative image or video models embedded in the renderer
  • a complete charting library

Simple bars, progress indicators and numeric comparisons may be represented through primitive or semantic components without introducing a full chart engine.


12. Product model

Agent Effects should distinguish storytelling from rendering.

Project
└── Story
    └── Sections
        └── Beats
            └── Scenes
                └── States
                    └── Compositions
                        └── Elements
                            └── Animations

12.1 Project

The complete video, including:

  • metadata
  • theme
  • assets
  • story
  • scenes
  • output settings
  • narration
  • shared motifs
  • renderer version
  • random seed

12.2 Story

The communication objective and narrative progression.

12.3 Section

A long-form organisational unit such as:

  • introduction
  • context
  • case study
  • geographic comparison
  • evidence
  • conclusion

Sections group related scenes and permit long-form navigation.

12.4 Beat

One meaningful change in what the audience understands or feels.

A beat should be removable only at the cost of weakening the argument.

12.5 Scene

A visually coherent interval delivering one or more related beats.

A new scene is appropriate when there is a meaningful change in:

  • layout
  • subject
  • dominant object
  • argument stage
  • background environment
  • visual perspective
  • narrative energy

12.6 State

A meaningful visual stage within a scene.

Example:

  1. display the whole rendering pipeline
  2. highlight frame capture
  3. dim the other stages
  4. enlarge the 94% statistic

These remain part of one scene because they form one continuous visual idea.

12.7 Composition

A reusable visual sentence structure such as:

  • statistic
  • comparison
  • process
  • map
  • browser frame

12.8 Element

A drawable object such as:

  • text
  • shape
  • line
  • image
  • SVG
  • marker
  • connector

12.9 Animation

A time-dependent change to an element, group, composition or camera property.


13. DSL architecture

Agent Effects should use three conceptual representations.

13.1 Story document

Describes what the video communicates.

story:
  audience: "People considering cheap property in Spain"
  objective: "Explain where cheap properties exist and the trade-offs"
  takeaway: "Low prices often reflect location, condition or legal complexity"
  tone: "curious, informative and practical"

13.2 Scene plan

Describes how the story will be visualised.

beats:
  - id: almeria-prices
    type: evidence
    message: "Some of Spain's lowest property prices are in Almería"
    emphasis: "Almería"
    desired_response: curiosity

scenes:
  - id: almeria-map
    beats: [almeria-prices]

    composition:
      type: location-map
      map: spain-provinces
      highlight: ["Almería"]

13.3 Render intermediate representation

Contains resolved:

  • layers
  • transforms
  • bounds
  • shaped text
  • asset references
  • timing
  • keyframes
  • effects
  • clipping relationships
  • scene transitions

This representation is generated by the compiler and consumed by the Rust renderer.

Agents should rarely edit it directly.


14. Authoring format

YAML will be the primary human- and agent-facing format.

Reasons:

  • concise
  • readable
  • easy to diff
  • supports comments
  • suitable for AI generation
  • maps cleanly to strongly typed Rust structures
  • easier to inspect than raw animation JSON

The Rust type definitions are the source of truth.

Recommended libraries:

  • serde
  • serde_yaml
  • schemars
  • tagged enums for composition types
  • explicit project-version migration

Every project must declare:

version: 1

Unknown fields should produce warnings or errors rather than being silently ignored.


15. File structure

Short projects may use one file:

project.afx.yaml
assets/

Long projects should support modular scene files:

project.afx.yaml
theme.afx.yaml

scenes/
├── 001-hook.afx.yaml
├── 002-context.afx.yaml
├── 003-spain-map.afx.yaml
├── 004-almeria.afx.yaml
├── 005-listing-example.afx.yaml
└── 006-conclusion.afx.yaml

assets/
├── narration.wav
├── narration.words.json
├── paper-grain.webp
├── spain-provinces.svg
├── listing-01.webp
└── property-01.webp

The root project may reference scene files:

version: 1

video:
  width: 1920
  height: 1080
  fps: 30

scenes:
  - include: "scenes/001-hook.afx.yaml"
  - include: "scenes/002-context.afx.yaml"
  - include: "scenes/003-spain-map.afx.yaml"

Benefits:

  • smaller agent-editable files
  • clearer Git diffs
  • independent validation
  • scene-level caching
  • scene-level rendering
  • easier recovery
  • parallel preparation
  • selective invalidation

16. Example project

version: 1

video:
  width: 1920
  height: 1080
  fps: 30
  duration: 38
  seed: 42817

theme:
  background: "#F1EDE3"
  foreground: "#1D2827"
  muted: "#6F7975"
  accent: "#EF694D"
  secondary: "#55B7AD"
  font_family: "Inter"
  display_font_family: "Fraunces"
  motion_style: editorial
  layout_style: documentary

background:
  colour: "@theme.background"
  texture:
    source: "assets/paper-grain.webp"
    mode: tile
    opacity: 0.16
    scale: 0.8

story:
  audience: "People looking for affordable property in Spain"
  objective: "Show why some regions appear unusually cheap"
  takeaway: "Cheap property is available, but location and paperwork matter"
  tone: "curious, practical and slightly playful"

narration:
  audio: "assets/narration.wav"
  words: "assets/narration.words.json"

beats:
  - id: hook
    type: question
    message: "Can you really buy a house in Spain for under €50,000?"

  - id: map-context
    type: context
    message: "The cheapest listings cluster in specific inland regions"

  - id: almeria-example
    type: example
    message: "Almería has low-priced properties close to the coast"

  - id: tradeoff
    type: contrast
    message: "But the cheapest homes often need work or have paperwork risks"

scenes:
  - id: opening
    beats: [hook]
    from: 0
    to: 5

    composition:
      type: question
      text: "Can you really buy a house\nin Spain for under €50,000?"
      emphasis: "€50,000"
      layout: editorial-left

    entrance:
      preset: lines-reveal
      feel: curious

  - id: national-map
    beats: [map-context]
    from: 5
    to: 12

    composition:
      type: location-map
      map: spain-provinces

      highlight:
        - province: "Almería"
          colour: "@theme.accent"

        - province: "León"
          colour: "@theme.secondary"

      markers:
        - place: "Almería"
          label: true

        - place: "León"
          label: true

    transition_in:
      relation: reveal
      preset: camera-push

  - id: listing-example
    beats: [almeria-example]
    from: 12
    to: 23

    composition:
      type: browser-frame
      style: minimal-macos
      url: "idealista.com"
      screenshot: "assets/listing-01.webp"

      viewport:
        fit: cover
        position: [0.5, 0.28]

      callouts:
        - target: [0.72, 0.31]
          text: "€43,000"

    effects:
      - type: drop-shadow
        offset: [0, 24]
        blur: 42
        opacity: 0.22

  - id: tradeoff-collage
    beats: [tradeoff]
    from: 23
    to: 38

    composition:
      type: photo-stack

      photos:
        - source: "assets/property-01.webp"
          rotation: -4
          caption: "Low price"

        - source: "assets/property-02.webp"
          rotation: 3
          caption: "Needs work"

        - source: "assets/documents.webp"
          rotation: -1
          caption: "Check the paperwork"

    transition_in:
      relation: contrast
      preset: split-wipe

17. Story primitives

17.1 Beat types

The MVP vocabulary includes:

  • hook
  • question
  • context
  • claim
  • evidence
  • contrast
  • cause
  • mechanism
  • example
  • reveal
  • consequence
  • solution
  • proof
  • summary
  • call to action

17.2 Desired audience responses

  • curiosity
  • surprise
  • concern
  • understanding
  • confidence
  • relief
  • urgency
  • excitement
  • amusement

17.3 Scene relationships

  • continue
  • contrast
  • cause
  • reveal
  • resolve
  • summarise
  • escalate
  • return

17.4 Motion intentions

  • introduce
  • emphasise
  • connect
  • separate
  • replace
  • compare
  • accumulate
  • reduce
  • reveal
  • focus
  • de-emphasise
  • transform
  • conclude
  • settle

17.5 Visual motifs

A motif is a recurring visual object or structure that creates continuity.

Examples:

  • a map marker
  • a horizontal pipeline
  • a moving dot
  • a paper card
  • a price tag
  • a highlighted word
  • a route line
  • a photograph frame

Example:

motifs:
  - id: location-pin
    type: map-marker
    style: outlined
    continuity: persistent

18. Scene model

A scene contains:

  • ID
  • section
  • associated beats
  • start and end timing
  • composition
  • optional states
  • transition in
  • transition out
  • pacing metadata
  • scene-local assets
  • optional cached static surfaces

Example:

scene:
  id: pipeline-breakdown
  beats:
    - capture-cause

  states:
    - id: full-pipeline
      at: 0

    - id: highlight-capture
      at: 1.2

    - id: reveal-statistic
      at: 2.4

A scene should not be created merely because one word changes.

Scenes are visual chapters. States are meaningful stages within those chapters.


19. Semantic compositions

19.1 Headline

Presents one dominant phrase with optional emphasis and supporting text.

19.2 Question

Presents an opening or transitional question.

19.3 Statement

Presents a declarative idea without numeric dominance.

19.4 Statistic

Presents a number, percentage, currency amount or duration as the primary visual.

19.5 Quote

Presents attributed speech or a source excerpt.

19.6 Bullet list

Presents a short unordered sequence.

19.7 Ranked list

Presents ordered items with numerical or visual ranking.

19.8 Comparison

Presents two or more contrasting values or concepts.

19.9 Before-and-after

Shows transformation from one state to another.

19.10 Steps or process

Presents ordered stages, flow or cause-and-effect.

19.11 Callout

Attaches a label to an element, image coordinate or map location.

19.12 Screenshot

Displays a screenshot with crop, pan, zoom and annotation.

19.13 Browser frame

Displays a screenshot inside stylised browser chrome.

19.14 Device frame

Displays a screenshot inside a constrained laptop or phone frame.

19.15 Photo card

Displays an image as a designed physical object.

19.16 Photo stack

Displays overlapping images with rotation, depth and captions.

19.17 SVG map

Displays an imported targetable SVG map.

19.18 Location map

Displays a built-in map with named regions and markers.

19.19 Route map

Displays a simple origin-to-destination route.

19.20 End card

Presents a summary, brand, call to action or final statement.


20. Kinetic typography principles

Agent Effects should encode the following defaults.

20.1 Meaning drives motion

Animation should support the meaning of the phrase.

20.2 Animate phrases rather than transcripts

The system should not default to presenting every narrated word as animated subtitles.

20.3 Maintain hierarchy

Each scene should usually have:

  • one primary phrase or object
  • one supporting phrase or object
  • optional accents

20.4 Preserve settled reading time

Important text should normally remain still and legible for longer than it animates.

20.5 Synchronise to speech

Important visual changes should align to stressed words, phrase boundaries or deliberate pauses.

20.6 Use contrast between movement and stillness

Constant motion weakens emphasis.

20.7 Use appropriate easing

Linear motion should be reserved for intentionally mechanical movement.

20.8 Preserve continuity

Important words or objects should carry between scenes where this strengthens the visual argument.

20.9 Compose the complete frame

Typography should respect:

  • safe areas
  • negative space
  • alignment
  • balance
  • line length
  • visual eye flow

20.10 Maintain legibility during movement

The renderer should discourage:

  • excessive rotation
  • prolonged blur
  • unreadably fast motion
  • long periods at tiny scale
  • overlapping moving text

20.11 Use repetition with variation

A video should have a coherent motion grammar rather than unrelated effects in every scene.

20.12 Prefer restraint

Most polished typography can be produced with:

  • translation
  • scale
  • opacity
  • clipping
  • stagger
  • colour
  • blur
  • line drawing

21. Kinetic typography defaults

The MVP should encode or warn around these guidelines:

  1. One dominant phrase per scene.
  2. No more than two primary movement directions at once.
  3. Typical entrance duration of 300–700 milliseconds.
  4. Important text remains settled longer than its entrance.
  5. Entrances default to ease-out.
  6. Exits default to ease-in.
  7. Word and line stagger are preferred over character stagger.
  8. Emphasis generally applies to 10–25% of words.
  9. Safe margins are enforced.
  10. Major ideas receive brief pauses.
  11. Important moving text resolves to a sharp still state.
  12. Simultaneous competing emphasis is discouraged.

These are defaults and diagnostics, not rigid creative restrictions.


22. Motion primitives

22.1 Animatable properties

  • position
  • scale
  • rotation
  • opacity
  • fill colour
  • stroke colour
  • blur
  • tracking
  • line spacing
  • clipping bounds
  • mask progress
  • line-drawing progress
  • image crop
  • image scale
  • image rotation
  • background position
  • texture opacity
  • texture scale
  • shadow offset
  • shadow blur
  • shadow opacity
  • SVG fill
  • SVG stroke
  • SVG opacity
  • map camera bounds
  • browser viewport position
  • browser viewport scale
  • effect parameters

22.2 Temporal interpolation

  • linear
  • sine
  • cubic
  • quartic
  • quintic
  • exponential
  • back
  • elastic
  • bounce
  • custom cubic Bézier
  • deterministic spring

22.3 Semantic motion feels

  • calm
  • confident
  • energetic
  • urgent
  • playful
  • heavy
  • elegant
  • mechanical
  • dramatic
  • documentary
  • understated

Semantic values map to tested combinations of:

  • duration
  • easing
  • overshoot
  • stagger
  • travel distance
  • blur
  • motion intensity

22.4 Spatial motion

  • straight-line interpolation
  • cubic Bézier path
  • Catmull–Rom path
  • points-through-path generation
  • arc-length lookup
  • constant-speed path traversal
  • orient-to-path
  • parent-relative transforms

22.5 Staggering

Support stagger by:

  • line
  • word
  • character
  • list item
  • process step
  • child element
  • map marker
  • photo card

23. Initial preset library

The MVP should launch with approximately 25 polished presets.

23.1 Text entrances

  • words-rise
  • lines-reveal
  • masked-slide
  • blur-resolve
  • scale-reveal
  • tracking-tighten
  • split-reveal
  • fade-slide

23.2 Emphasis

  • scale-emphasis
  • colour-emphasis
  • highlight-sweep
  • underline-draw
  • pulse-once
  • isolate-and-dim
  • count-up

23.3 Images and editorial objects

  • photo-drop
  • card-stack
  • tape-in
  • image-pan
  • focus-crop
  • browser-zoom
  • callout-pop

23.4 Supporting graphics

  • line-draw
  • progress-fill
  • ring-progress
  • connector-grow
  • marker-pulse
  • region-highlight

23.5 Scene transitions

  • push-left
  • push-up
  • split-wipe
  • directional-wipe
  • shared-word
  • shared-object
  • simplify
  • camera-push
  • settle
  • paper-slide

Each preset must have:

  • valid targets
  • default duration
  • supported parameters
  • semantic description
  • visual preview
  • canonical example
  • stable version behaviour

24. Layer effects

The MVP includes:

  • drop shadow
  • blur
  • border
  • stroke
  • corner radius
  • clipping mask
  • colour tint
  • limited blend modes
  • deterministic noise
  • texture overlay

Drop-shadow example:

effects:
  - type: drop-shadow
    offset: [0, 18]
    blur: 32
    opacity: 0.22

Supported drop-shadow animation properties:

  • horizontal offset
  • vertical offset
  • blur radius
  • opacity
  • optional spread

Effects should use caching where possible.


25. SVG and map requirements

25.1 SVG import

The importer must retain or expose:

  • element IDs
  • element type
  • document bounds
  • element bounds
  • transforms
  • fill
  • stroke
  • opacity

25.2 SVG inspection

The CLI must allow agents to inspect targetable elements:

agent-effects inspect-svg assets/spain.svg --list-ids

Example result:

{
  "elements": [
    {
      "id": "almeria",
      "type": "path",
      "bounds": [742.1, 611.4, 63.8, 54.2]
    },
    {
      "id": "leon",
      "type": "path",
      "bounds": [389.2, 251.5, 78.7, 69.3]
    }
  ]
}

25.3 SVG targeting

composition:
  type: svg-map
  source: "assets/spain-provinces.svg"

  regions:
    - selector: "#almeria"
      fill: "@theme.accent"

    - selector: "#leon"
      fill: "@theme.secondary"

25.4 Missing selectors

Errors should suggest close matches:

{
  "error": "svg_selector_not_found",
  "selector": "#almeria",
  "source": "assets/spain-provinces.svg",
  "available_close_matches": [
    "#almería",
    "#province-almeria"
  ]
}

25.5 Built-in Spain map

The MVP should ship with a stable map package containing:

  • Spain outline
  • autonomous-community boundaries
  • province boundaries
  • canonical stable IDs
  • named camera bounds
  • major city coordinates where useful

25.6 Map animations

  • fade in
  • draw outline
  • highlight region
  • dim other regions
  • marker appearance
  • marker pulse
  • route drawing
  • full-country to province zoom
  • label attachment
  • map pan

26. Rendering architecture

Project YAML
    ↓
Parser
    ↓
Schema validation
    ↓
Story and scene compiler
    ↓
Resolved render IR
    ↓
Timeline evaluator
    ↓
Layout and text shaping
    ↓
Native vector and image renderer
    ↓
Bounded pixel buffers
    ↓
FFmpeg stdin
    ↓
Video output

26.1 Recommended technology

  • Rust
  • serde
  • schemars
  • skia-safe for initial 2D rendering
  • resvg or equivalent for SVG
  • FFmpeg subprocess receiving raw frames
  • optional Rayon parallelism
  • bounded producer-consumer queue
  • CPU-backed renderer first
  • GPU-backed Skia or wgpu optimisation after correctness

26.2 No per-video compilation

The Rust engine is compiled only when capabilities change.

A video is interpreted data:

agent-effects render project.afx.yaml

26.3 Frame production

For each frame:

  1. identify the active scene and state
  2. evaluate keyframes and expressions
  3. resolve layout
  4. shape required text
  5. load or reuse required assets
  6. render the scene
  7. copy pixels into a bounded output buffer
  8. send the frame to FFmpeg
  9. reuse or release the buffer

27. Long-form rendering architecture

27.1 Bounded-memory rendering

The renderer must not retain all frames.

A 1920×1080 RGBA frame requires approximately 8 MB.

Retaining 19,800 frames would require roughly 158 GB before additional overhead.

Agent Effects must instead use a small bounded pool of reusable frame buffers.

27.2 Scene-by-scene compilation

Global resident information may include:

  • output configuration
  • theme
  • font registry
  • asset manifest
  • transcript timing index
  • scene manifest

Scene-local render information should be released after the scene finishes unless required by a transition or cache.

27.3 Time-range rendering

agent-effects render project.afx.yaml \
  --from 240 \
  --to 300 \
  --output section.mp4

27.4 Scene rendering

agent-effects render project.afx.yaml \
  --scene almeria-property-example

27.5 Chunked rendering

Long renders should support safe chunks, initially aligned to scene boundaries.

agent-effects render project.afx.yaml \
  --chunk-duration 20

Automatic chunk boundaries must not cut through unsafe transitions.

27.6 Resumable rendering

The renderer should produce a manifest:

{
  "engine_version": "0.1.0",
  "project_hash": "abc123",
  "completed_chunks": [
    {
      "from_frame": 0,
      "to_frame": 899,
      "output": ".agent-effects/chunks/0000.mkv"
    }
  ]
}

Rendering can resume when:

  • the project hash is compatible
  • relevant assets are unchanged
  • the engine version is compatible
  • completed chunks pass validation

27.7 Selective invalidation

Changing one scene should ideally invalidate:

  • that scene
  • dependent transition frames
  • affected final assembly metadata

It should not require rerendering unrelated scenes.

27.8 Chunk transitions

Safe approaches include:

  • chunking only at transition-safe scene boundaries
  • rendering overlap frames
  • treating transitions as separate bridge scenes
  • rerendering a small boundary range

The first MVP implementation may restrict chunks to safe scene boundaries.


28. Audio architecture

The renderer should not require fully decoded audio to remain in memory.

Permitted approaches include:

  • passing original audio directly to FFmpeg
  • reading transcript timings without decoding audio
  • decoding only required windows
  • assembling visual chunks before final audio muxing
  • using external audio files during final encoding

The MVP should support:

  • narration
  • music
  • sound effects
  • volume envelopes
  • fade in and fade out
  • delayed starts
  • final muxing

Advanced audio editing remains out of scope.


29. Asset caching

The engine should cache:

  • shaped text runs
  • loaded fonts
  • decoded SVG structures
  • raster images
  • browser chrome
  • static textures
  • map geometry
  • reusable shadow surfaces
  • unchanged scene layers
  • static composition surfaces

Caching must be bounded through:

  • configurable limits
  • least-recently-used eviction
  • scene-aware release
  • disk-backed cache where justified
  • project and asset hashes

Caching must not compromise deterministic output.


30. Static-hold optimisation

Long-form editorial videos may contain intervals where most or all visual content is unchanged.

The DSL may declare:

hold:
  from: 18.2
  to: 21.8

Optimisations may include:

  • reusing the previous rendered frame
  • reusing unchanged layer surfaces
  • skipping layout work
  • skipping repeated text shaping
  • minimising repeated image processing

Output timing and frame cadence must remain correct.


31. Expressions and references

The MVP should support a constrained expression or reference system.

Preferred semantic references:

anchor:
  to: headline.right
  offset: [40, 0]

Timeline references:

start: "@word:Almería - 0.1"

Property references:

position:
  x: "= headline.right + 40"
  y: "= headline.center_y"

Arbitrary Rust or unrestricted JavaScript must not be embedded in project files.

A small safe expression evaluator may be used.


32. Validation and diagnostics

32.1 Structural errors

  • invalid YAML
  • unsupported project version
  • missing required field
  • unknown composition
  • unknown preset
  • invalid property type
  • missing asset
  • missing font
  • circular reference
  • invalid scene timing
  • unresolved include

32.2 Visual warnings

  • text outside safe area
  • text overflow
  • fitted text below minimum size
  • insufficient contrast
  • overlapping text
  • element outside canvas
  • callout target outside image
  • missing SVG selector
  • map label outside viewport
  • browser screenshot crop outside image
  • shadow clipping

32.3 Motion warnings

  • important text settled too briefly
  • excessive character stagger
  • too many simultaneous animations
  • abrupt position jump
  • abrupt velocity discontinuity
  • spring not settled before exit
  • transition overlap
  • motion path discontinuity
  • unreadably fast moving text
  • conflicting emphasis

32.4 Story warnings

  • scene without associated beat
  • beat not used by any scene
  • repeated composition without variation
  • no conclusion
  • excessively dense scene
  • too many unrelated ideas in one scene
  • long interval without meaningful visual change

32.5 Educational errors

Errors should contain:

  • exact DSL path
  • received value
  • expected values
  • close matches
  • documentation command
  • suggested correction

Example:

{
  "error": "unknown_preset",
  "path": "scenes[1].entrance.preset",
  "received": "smooth-pop",
  "suggestions": [
    "scale-reveal",
    "fade-slide",
    "photo-drop"
  ],
  "docs_command": "agent-effects docs scale-reveal"
}

33. CLI requirements

The CLI is the primary interface for coding agents.

33.1 Core commands

agent-effects init
agent-effects validate project.afx.yaml
agent-effects docs <component-or-preset>
agent-effects schema
agent-effects examples search "<query>"
agent-effects inspect project.afx.yaml
agent-effects inspect-svg assets/map.svg
agent-effects frame project.afx.yaml --time 3.2
agent-effects contact-sheet project.afx.yaml
agent-effects preview project.afx.yaml
agent-effects render project.afx.yaml
agent-effects critique project.afx.yaml

33.2 Long-form commands

agent-effects render project.afx.yaml --scene <id>
agent-effects render project.afx.yaml --from <seconds> --to <seconds>
agent-effects render project.afx.yaml --chunk-duration 20
agent-effects render project.afx.yaml --resume

33.3 validate

Returns structured errors and warnings.

33.4 docs

Displays concise machine- and human-readable documentation for one component, property or preset.

33.5 schema

Outputs the complete generated JSON Schema.

33.6 examples search

Returns a small number of canonical examples relevant to a natural-language query.

33.7 inspect

Outputs resolved scene, layer, timing and asset metadata.

33.8 inspect-svg

Outputs targetable SVG IDs, types and bounds.

33.9 frame

Renders one frame at an arbitrary timestamp.

33.10 contact-sheet

Renders representative frames with:

  • timestamps
  • scene IDs
  • beat labels
  • issue markers

33.11 preview

Renders a lightweight video, for example:

  • 854×480
  • 15 fps
  • H.264
  • simplified effects where permitted

33.12 render

Produces the final output.

33.13 critique

Combines:

  • deterministic validation
  • visual diagnostics
  • contact-sheet output
  • low-resolution preview
  • optional video-model critique
  • machine-readable recommendations

34. Agent enablement

Agents should not be expected to memorise the DSL.

34.1 AGENTS.md

Every project should include concise instructions:

# Agent Effects project

Edit `project.afx.yaml` and files under `scenes/`.

Prefer semantic compositions and named presets.
Avoid explicit coordinates and keyframes unless required.
Preserve the visual theme and recurring motifs.

After editing:

1. Run `agent-effects validate project.afx.yaml`
2. Render and inspect the contact sheet
3. Render the affected scene or preview
4. Run `agent-effects critique project.afx.yaml`
5. Address high-severity issues
6. Render the final video only after the preview passes

34.2 Discoverability tools

Agents must be able to:

  • list compositions
  • describe a composition
  • list presets
  • describe a preset
  • inspect SVGs
  • search examples
  • validate projects
  • render specific scenes
  • inspect warnings
  • inspect source paths

34.3 Canonical cookbook

The repository should include approximately 25–30 excellent examples.

Suggested examples:

  • headline word rise
  • line reveal question
  • dominant statistic
  • price counter
  • quote emphasis
  • ranked list
  • three-step process
  • before-and-after comparison
  • browser screenshot callout
  • browser viewport zoom
  • phone-frame animation
  • photo card drop
  • taped photo collage
  • paper-stack transition
  • Spain region highlight
  • province zoom
  • location markers
  • route animation
  • shared-word transition
  • branded end card
  • narration-aligned scene
  • multi-file long-form project
  • resumable chunked render

Examples should be curated, not generated in bulk.

34.4 Schema generation

The Rust types are the source of truth.

Documentation and JSON Schema should be generated from the same definitions to prevent drift.


35. Agent visual-feedback loop

The agent feedback system should have four layers.

35.1 Structural validation

Checks whether the project is syntactically and semantically valid.

35.2 Contact sheet

Used to inspect:

  • visual hierarchy
  • composition
  • scene progression
  • text overflow
  • settled states
  • map framing
  • image cropping
  • narrative continuity

35.3 Low-resolution preview

Used to inspect:

  • pacing
  • easing
  • transitions
  • motion continuity
  • narration alignment
  • visual rhythm

35.4 Deterministic telemetry

Used to detect:

  • overlaps
  • bounds violations
  • movement discontinuities
  • insufficient reading time
  • unsettled springs
  • timing conflicts
  • static intervals
  • expensive effects
  • scene render cost

36. Optional Kimi critique bridge

Because some coding-agent environments may not pass MP4 files directly to a multimodal model, Agent Effects should provide an optional bridge.

agent-effects critique may:

  1. render a lightweight MP4
  2. upload it directly through the Kimi API
  3. request timestamped structured criticism
  4. save the result to .agent-effects/critique.json

Example:

{
  "issues": [
    {
      "start": 16.2,
      "end": 17.1,
      "severity": "medium",
      "scene": "national-map",
      "problem": "The zoom reaches Almería before the label appears",
      "suggestion": {
        "label_start": 16.4,
        "zoom_duration": 0.8
      }
    }
  ]
}

The coding agent can then patch the affected scene.

Video-model feedback is advisory. Deterministic validation remains authoritative for measurable problems.


37. Human review UI

The MVP should include a local browser application.

The interface should function as an inspection and refinement tool, not an After Effects clone.

37.1 Layout

┌───────────────┬──────────────────────────────┬───────────────────┐
│ Sections      │                              │ Inspector         │
│ and scenes    │        Video preview         │                   │
│               │                              │ Scene             │
│ Introduction  │                              │ Composition       │
│ Spain map     │                              │ Motion preset     │
│ Almería       │                              │ Source DSL path   │
├───────────────┴──────────────────────────────┴───────────────────┤
│ Narration | Scene 1 | Scene 2 | Scene 3 | Issues                │
└─────────────────────────────────────────────────────────────────┘

37.2 Preview

  • play
  • pause
  • timeline scrubbing
  • frame stepping
  • half-speed playback
  • safe-area guides
  • element bounds
  • low-resolution preview
  • full-resolution still
  • jump to scene
  • jump to issue

37.3 Scene list

Display:

  • section
  • scene name
  • associated beat
  • start and end
  • composition type
  • warning count
  • cache state
  • render status

37.4 Timeline

Display:

  • narration duration
  • waveform where available
  • scene boundaries
  • word or phrase markers
  • issue markers
  • chunk boundaries
  • current playhead

37.5 Inspector

Initially read-only:

  • scene ID
  • beat ID
  • composition
  • source file
  • source DSL path
  • resolved bounds
  • active animation
  • preset
  • asset references
  • render timing

37.6 Issues panel

Display:

  • severity
  • timestamp
  • affected scene
  • element
  • DSL path
  • deterministic or model source
  • suggested fix

37.7 Semantic editing

A later MVP stage may support:

  • editing text
  • changing a layout
  • selecting a preset
  • adjusting scene duration
  • changing emphasis
  • choosing theme tokens

All UI edits must patch the DSL.


38. UI exclusions

The MVP UI does not include:

  • freehand drawing
  • full drag-and-drop authoring
  • complex keyframe timelines
  • easing graph editor
  • arbitrary path editor
  • general vector illustration
  • advanced masking
  • 3D editing
  • hidden database-only projects

Visual dragging may be added later as an escape hatch.


39. Performance requirements

Performance claims must be benchmarked rather than assumed.

39.1 Benchmark scene classes

Typography scene

  • text
  • masks
  • simple shapes
  • limited blur
  • no large images

Editorial collage scene

  • several high-resolution images
  • clipping
  • rotation
  • shadows
  • textured background

Map scene

  • complex SVG
  • region highlights
  • labels
  • pan and zoom

Browser-frame scene

  • browser chrome
  • large screenshot
  • crop and scale
  • callouts
  • shadow

Static-hold scene

  • mostly unchanged composition
  • long narration interval

39.2 Preview targets

On a suitable modern desktop:

  • generate a representative contact sheet for a 30-second project in under 10 seconds
  • render an arbitrary still without browser startup
  • begin low-resolution preview generation promptly
  • support random-access frame rendering
  • avoid evaluating preceding frames for deterministic springs
  • render only affected scenes during iteration

39.3 Final-render targets

  • eliminate the approximately 750 ms browser screenshot-extraction step
  • avoid intermediate PNG sequences by default
  • avoid base64 screenshot transfer
  • stream raw frames directly to FFmpeg
  • maintain bounded memory
  • support interruption and resumption
  • begin encoding before the entire project is evaluated
  • substantially outperform the Chromium workflow on representative projects

39.4 Aspirational long-form target

An aspirational target is to render the representative 11-minute editorial video in approximately 5–10 minutes on suitable modern hardware.

This is not an acceptance requirement until validated by benchmarks.

Rendering performance will depend on:

  • resolution
  • frame rate
  • image sizes
  • shadows
  • blur
  • motion blur
  • SVG complexity
  • CPU
  • GPU
  • media codecs
  • FFmpeg configuration

39.5 Profiling

The renderer should report time spent in:

  • project parsing
  • scene compilation
  • timeline evaluation
  • layout
  • text shaping
  • SVG rendering
  • image decoding
  • image resizing
  • effects
  • compositing
  • pixel transfer
  • FFmpeg encoding

40. Reliability requirements

  • no frame-ordering errors
  • no silent asset failures
  • no silent missing fonts
  • no silent unsupported fields
  • clear version incompatibility errors
  • deterministic random effects
  • bounded memory
  • cancellable renders
  • resumable long renders
  • atomic chunk output
  • corrupted chunks rejected
  • reproducible output for the same project and engine version
  • validation before final render

41. MVP user journey

41.1 Inputs

The user provides:

  • script or narration
  • word-level transcript where available
  • visual theme
  • fonts
  • images
  • screenshots
  • map requirements
  • target duration
  • target aspect ratio

41.2 Agent process

  1. Extract the central takeaway.
  2. Identify the audience.
  3. Create a beat sheet.
  4. Select the story arc.
  5. Group beats into sections and scenes.
  6. Select recurring motifs.
  7. Choose semantic compositions.
  8. Select transitions and motion intentions.
  9. Generate project and scene files.
  10. Validate the project.
  11. Render a contact sheet.
  12. Inspect and fix layout problems.
  13. Render affected scenes.
  14. Render a low-resolution preview.
  15. Run deterministic critique.
  16. Optionally run video-model critique.
  17. Patch the project.
  18. Render final chunks.
  19. Assemble and mux the final video.

41.3 Human process

  1. Open the local review UI.
  2. Review the beat and scene structure.
  3. Watch the preview.
  4. Inspect issue markers.
  5. Review map framing, screenshot crops and typography.
  6. Give natural-language feedback or edit project files.
  7. Approve the final render.

42. Success metrics

42.1 Primary success metric

Percentage of agent-generated videos accepted after no more than two critique-and-revision cycles.

Initial targets:

  • at least 70% structurally and visually usable
  • at least 40% accepted with only minor human changes

42.2 Supporting quality metrics

  • percentage of beats represented clearly
  • number of scenes with layout warnings
  • number of unreadable text intervals
  • percentage of scenes using semantic compositions
  • percentage of animations using named presets
  • human rating of visual coherence
  • human rating of story clarity
  • human rating of pacing
  • consistency across videos using the same theme

42.3 Agent metrics

  • first-draft validation-error count
  • percentage of errors autonomously resolved
  • number of agent retries caused by unclear documentation
  • number of low-level escape-hatch uses
  • number of project changes per accepted video
  • time from script to first preview
  • number of critique cycles

42.4 Rendering metrics

  • median render time per frame
  • render-to-video-duration ratio
  • peak memory usage
  • contact-sheet generation time
  • preview generation time
  • final assembly time
  • cache hit rate
  • number of scenes rerendered after a local change
  • successful render-resume rate

43. MVP acceptance criteria

The MVP is ready for internal use when all of the following are true.

43.1 Core engine

  1. A compiled Rust executable can load a versioned .afx.yaml project.
  2. Videos do not require recompiling Rust.
  3. Text, shapes, SVGs and images render correctly.
  4. Raw frames can be streamed directly to FFmpeg.
  5. Output frame order is correct.
  6. Rendering is deterministic.

43.2 Story and scenes

  1. Projects can define beats, sections, scenes and states.
  2. Beats can be associated with scenes.
  3. Scene relationships can influence transition selection.
  4. Long projects can be split across scene files.

43.3 Typography and motion

  1. Line-, word- and character-level kinetic typography works.
  2. At least 20 polished presets are available.
  3. Easing, stagger and deterministic springs work.
  4. Basic motion paths work.
  5. Parent-child transforms work.
  6. Important typography diagnostics are implemented.

43.4 Semantic compositions

  1. At least 15 semantic compositions are available.
  2. Browser-frame compositions can display and animate screenshots.
  3. Photo-card and photo-stack compositions work.
  4. Screenshot callouts work.
  5. Drop shadows and textured backgrounds work.

43.5 Maps

  1. SVG element IDs can be inspected.
  2. SVG elements can be targeted from the DSL.
  3. A built-in Spain map is available.
  4. At least two provinces can be highlighted independently.
  5. Markers and labels can be attached.
  6. The map can animate from country to province framing.

43.6 Audio and timing

  1. Narration audio can be muxed.
  2. Word-level timings can be imported.
  3. Scene entrances can reference word or phrase markers.
  4. Reading-time warnings are generated.

43.7 Long-form rendering

  1. A 15-minute project renders with bounded memory.
  2. Scenes and time ranges can be rendered independently.
  3. Long renders can be chunked.
  4. Interrupted renders can resume.
  5. Frames are not retained for the full video.
  6. Static intervals can reuse cached results.
  7. A scene change invalidates only affected chunks where possible.

43.8 Agent workflow

  1. The agent can inspect schema and documentation through the CLI.
  2. The agent can search canonical examples.
  3. Validation errors include corrective guidance.
  4. Contact sheets can be generated.
  5. Low-resolution previews can be generated.
  6. Machine-readable critique can be generated.
  7. The repository includes at least 25 canonical examples.

43.9 Review UI

  1. The local UI can play a preview.
  2. It displays sections, scenes and issues.
  3. It can jump to timestamps.
  4. It displays source DSL paths.
  5. The DSL remains the source of truth.

43.10 End-to-end benchmark

  1. An AI agent can create a coherent 30-second video without changing Rust.
  2. An AI agent can create or modify a multi-scene long-form project.
  3. The representative 11-minute video can be recreated using supported features.
  4. The native pipeline substantially outperforms the existing Chromium capture workflow.
  5. Performance results are supported by profiler output rather than assumptions.

44. Build sequence

Milestone 1: Repository and project model

  • Rust workspace
  • versioned schema
  • YAML parsing
  • project includes
  • asset manifest
  • CLI skeleton
  • structured errors

Milestone 2: Renderer skeleton

  • Skia surface
  • frame timeline
  • text rendering
  • rectangles and circles
  • images
  • raw FFmpeg pipeline
  • single-frame rendering

Milestone 3: Motion system

  • keyframes
  • easing
  • interpolation
  • springs
  • staggering
  • groups
  • parenting
  • masks
  • basic paths

Milestone 4: Kinetic typography

  • text shaping
  • wrapping
  • fitting
  • emphasis spans
  • line splitting
  • word splitting
  • character splitting
  • typography presets
  • text diagnostics

Milestone 5: Images and effects

  • cropping
  • image transforms
  • corner radius
  • border
  • drop shadow
  • texture backgrounds
  • deterministic grain
  • collage layouts

Milestone 6: Semantic compositions

  • headline
  • question
  • statistic
  • quote
  • list
  • comparison
  • before-and-after
  • process
  • callout
  • screenshot
  • browser frame
  • photo card
  • photo stack
  • end card

Milestone 7: SVG and maps

  • SVG import
  • stable element IDs
  • bounds inspection
  • path targeting
  • Spain map package
  • province highlights
  • markers
  • labels
  • map camera
  • zoom transitions

Milestone 8: Story and scene compiler

  • story metadata
  • beats
  • sections
  • scenes
  • states
  • relationships
  • motifs
  • semantic motion intentions
  • render IR

Milestone 9: Long-form architecture

  • modular scene files
  • scene-by-scene compilation
  • bounded frame buffers
  • scene rendering
  • range rendering
  • chunks
  • manifests
  • resumability
  • selective invalidation

Milestone 10: Agent workflow

  • generated schema
  • CLI docs
  • example search
  • educational errors
  • AGENTS.md
  • canonical cookbook

Milestone 11: Feedback system

  • contact sheets
  • preview rendering
  • telemetry
  • issue reports
  • profiler
  • optional Kimi bridge

Milestone 12: Review UI

  • Axum local server
  • scene list
  • preview playback
  • timeline
  • issues panel
  • read-only inspector
  • source-path display

Milestone 13: End-to-end benchmark

Use the target 11-minute editorial video and require an AI coding agent to:

  • create the beat and scene structure
  • construct project files
  • use browser frames
  • use maps
  • create collage scenes
  • validate
  • inspect previews
  • revise
  • render chunks
  • produce the final video

45. Suggested repository structure

agent-effects/
├── Cargo.toml
├── AGENTS.md
├── README.md
├── crates/
│   ├── agent-effects-core/
│   ├── agent-effects-schema/
│   ├── agent-effects-story/
│   ├── agent-effects-layout/
│   ├── agent-effects-text/
│   ├── agent-effects-svg/
│   ├── agent-effects-maps/
│   ├── agent-effects-render/
│   ├── agent-effects-ffmpeg/
│   ├── agent-effects-diagnostics/
│   ├── agent-effects-cli/
│   └── agent-effects-server/
├── ui/
├── presets/
├── examples/
├── maps/
│   └── spain/
├── schemas/
├── tests/
├── benchmarks/
└── docs/

46. Post-MVP roadmap

Phase 2: Product and diagram explainers

  • automatic connectors
  • flow diagrams
  • browser-page scrolling
  • cursor paths
  • click indicators
  • product-tour templates
  • persistent shared elements
  • richer screenshot annotations

Phase 3: Simple charts

  • animated number
  • progress bar
  • progress ring
  • bar chart
  • line chart
  • simple donut chart
  • labels
  • annotations
  • restrained axes

Phase 4: Expanded maps

  • European countries
  • world map
  • GeoJSON ingestion
  • more projections
  • route arcs
  • richer label placement
  • additional country packages

Phase 5: Advanced effects

  • path morphing
  • particles
  • richer 2.5D camera
  • advanced compositing
  • custom GPU effects
  • Rive playback
  • Lottie playback

Phase 6: Visual editing

  • drag with snapping
  • resizing
  • anchor controls
  • basic motion-path editing
  • basic keyframe editing
  • before-and-after comparison
  • semantic prompt-based scene revision

47. Key risks and mitigations

47.1 The DSL becomes too low-level

Risk: Agents frequently resort to coordinates and explicit keyframes.

Mitigation: Track escape-hatch usage and add semantic components for repeated patterns.

47.2 The DSL becomes too broad

Risk: Attempting to reproduce all motion-graphics functionality delays the useful product.

Mitigation: Maintain the editorial explainer focus and explicit exclusions.

47.3 Presets look repetitive

Risk: Videos feel templated.

Mitigation: Support controlled variation in layout, direction, duration, emphasis, composition and theme.

47.4 Agents generate valid but unattractive projects

Risk: Schema compliance does not ensure visual quality.

Mitigation: Use strong defaults, curated examples, contact sheets, diagnostics and video critique.

47.5 Story quality remains weak

Risk: Attractive scenes fail to form a coherent argument.

Mitigation: Require a takeaway and beat sheet before scene generation.

47.6 Text rendering differs across systems

Risk: Missing or differing fonts change layout.

Mitigation: Require explicit approved fonts, verify availability and record font hashes in render manifests.

47.7 Maps become a hidden large project

Risk: Geographic support expands into a full GIS system.

Mitigation: Limit MVP support to built-in Spain maps and targetable SVGs.

47.8 Shadows and blur reduce performance

Risk: Editorial effects undermine native-rendering gains.

Mitigation: Cache static effects, provide quality tiers and profile each effect independently.

47.9 Chunk boundaries produce visual defects

Risk: Cross-scene transitions break during incremental rendering.

Mitigation: Restrict early chunks to safe boundaries and support overlap handles later.

47.10 Native rendering is slower than expected

Risk: Text shaping, image processing, shadows or FFmpeg remain expensive.

Mitigation: Benchmark representative scenes, optimise measured bottlenecks and avoid unverified performance claims.

47.11 UI development distracts from the renderer

Risk: Building a visual editor delays the core engine.

Mitigation: Begin with CLI, then read-only UI, then semantic editing.


48. Decisions

The MVP will:

  • be called Agent Effects
  • use the repository name agent-effects
  • use .afx.yaml project files
  • compile Rust once rather than per video
  • use native rendering rather than Chromium
  • begin with Skia rather than raw wgpu
  • pipe raw frames directly to FFmpeg
  • support short and long-form videos
  • use semantic story beats and compositions
  • focus on kinetic typography and editorial graphics
  • include images, collage layouts and screenshot callouts
  • include browser and constrained device frames
  • include targetable SVG maps
  • include built-in Spain province maps
  • include textured backgrounds and drop shadows
  • support modular scene files
  • use bounded scene-by-scene rendering
  • support resumable chunks
  • include a CLI as the primary agent interface
  • include a lightweight local review UI
  • use contact sheets and previews for feedback
  • optionally integrate Kimi as a video critic
  • preserve project files as the source of truth
  • expose low-level keyframes only as an escape hatch
  • benchmark performance rather than promising assumed native speeds

49. Final product definition

Agent Effects is:

  • a compiled Rust motion-graphics renderer
  • an agent-facing declarative language
  • a story and scene compiler
  • a curated library of semantic visual compositions
  • a preset motion system
  • a memory-safe long-form rendering pipeline
  • a validation and critique system
  • a lightweight review interface

Its first repeatable outcome is:

An AI agent can reliably construct, inspect, revise and render a polished short or long-form editorial motion-graphics video using story beats, semantic scenes, targetable media and tested animation presets.

The central product principle is:

The agent decides what the story means, which ideas matter and how they relate. Agent Effects turns that intent into coherent, tasteful and efficiently rendered motion graphics.