Skip to content

EXTENSIONS.md

Latest commit

 

History

History
310 lines (245 loc) · 10.6 KB

EXTENSIONS.md

File metadata and controls

310 lines (245 loc) · 10.6 KB

Recordly Extension API

Go to https://www.marketplace.recordly.dev/extensions for full, regularly updated documentation

Recordly extensions run in the editor renderer and use a permission-gated host API. They can draw into the render pipeline, react to playback and export events, register cursor effects, add settings panels, and contribute packaged assets such as frames, wallpapers, and cursor styles.

Quick Start

For local user-installed extensions, use Extensions -> Open Directory in the app. Recordly stores them in the app userData/extensions directory. This repo also includes installable example bundles under extension-examples/.

Minimum Extension

my-extension/
  recordly-extension.json
  index.js

{
  "id": "com.example.my-extension",
  "name": "My Extension",
  "version": "1.0.0",
  "description": "A short description",
  "author": "Your Name",
  "license": "MIT",
  "main": "index.js",
  "permissions": ["render"]
}
export function activate(api) {
  api.log("Hello from my extension");
}

export function deactivate() {}

TypeScript

Extensions can be authored in TypeScript. Import the RecordlyExtensionAPI type from types.ts in this repo for full IDE auto-complete and compile-time checks:

import type { RecordlyExtensionAPI } from "./types";

export function activate(api: RecordlyExtensionAPI) {
  api.registerRenderHook("final", (ctx) => {
    ctx.ctx.fillStyle = "rgba(255,255,255,0.1)";
    ctx.ctx.fillRect(0, 0, ctx.width, 30);
  });
}

export function deactivate() {}

Recordly loads the main entry from the manifest, which must be a .js file. If you author in TypeScript, compile or bundle to JavaScript before packaging. A tsconfig.json with "module": "ESNext" and "target": "ESNext" works well since extensions run in a Chromium renderer.

Manifest Screenshots

Extensions can include screenshots in the manifest to show preview images in the marketplace:

{
  "id": "com.example.my-extension",
  "screenshots": [
    "screenshots/preview-1.png",
    "screenshots/preview-2.png"
  ]
}

Paths are relative to the extension root. The marketplace displays screenshots in a carousel on both the in-app detail modal and the web detail page.

Manifest

recordly-extension.json is validated both when the extension loads and when a zip is uploaded to the marketplace.

Field Type Required Description
id string Yes Unique identifier, for example yourname.cool-effect
name string Yes Human-readable name shown in the UI
version string Yes Strict semver version, for example 1.0.0
description string Yes One-line summary
author string No Author or organisation
homepage string No HTTPS homepage or repository URL
license string No SPDX license identifier
engine string No Minimum supported Recordly version
icon string No Relative path to a PNG icon
screenshots string[] No Relative paths to preview images shown in the marketplace
main string Yes Relative entry point JS file
permissions string[] Yes Required capabilities
contributes object No Metadata for packaged frames, cursor styles, sounds, wallpapers, and webcam frames

contributes is metadata only today. Recordly does not auto-register runtime behavior from the manifest. Use activate() to call APIs such as registerFrame(), registerWallpaper(), registerCursorStyle(), registerSettingsPanel(), and playSound().

Permissions

Permission Grants access to
render Render hook registration
cursor Cursor telemetry and cursor effect registration
audio Bundled sound playback
timeline Playback and timeline events
ui Settings panels and device frame registration
assets Bundled asset resolution plus wallpaper and cursor style registration
export Export lifecycle events

Render Pipeline

Render hooks draw into hookCtx.ctx, a CanvasRenderingContext2D, at specific phases.

Phase Preview Export Notes
background Reserved Reserved Exists in the type surface but is not dispatched yet
post-video Yes Yes Runs inside the scene transform
post-zoom Yes Yes Runs inside the scene transform
post-cursor Yes Yes Runs inside the scene transform
post-webcam Yes Yes Runs after the built-in transform
post-annotations Yes Yes Runs after the built-in transform
final Yes Yes Last pass for HUD-style overlays

Inside vs Outside the Scene Transform

  • post-video, post-zoom, and post-cursor already follow zoom and motion in preview and export.
  • post-webcam, post-annotations, and final run after Recordly restores the canvas transform. Use sceneTransform manually if you want those overlays to move with the scene.

RenderHookContext

{
  width: number;
  height: number;
  timeMs: number;
  durationMs: number;
  cursor: { cx: number; cy: number; interactionType?: string } | null;
  smoothedCursor?: {
    cx: number;
    cy: number;
    trail: Array<{ cx: number; cy: number }>;
  } | null;
  ctx: CanvasRenderingContext2D;
  videoLayout?: {
    maskRect: { x: number; y: number; width: number; height: number };
    borderRadius: number;
    padding: number;
  };
  zoom?: { scale: number; focusX: number; focusY: number; progress: number };
  sceneTransform?: { scale: number; x: number; y: number };
  shadow?: { enabled: boolean; intensity: number };
  getPixelColor(x: number, y: number): { r: number; g: number; b: number; a: number };
  getAverageSceneColor(): { r: number; g: number; b: number; a: number };
  getEdgeAverageColor(edgeWidth?: number): { r: number; g: number; b: number; a: number };
  getDominantColors(count?: number): Array<{ r: number; g: number; b: number; frequency: number }>;
}

Use videoLayout.maskRect and videoLayout.borderRadius for scene-relative sizing. That gives you true scene borders and correctly shaped rounded corners instead of canvas-wide overlays.

Cursor Effects

Cursor effect callbacks run each frame after a click until they return false.

api.registerCursorEffect((ctx) => {
  const progress = ctx.elapsedMs / 400;
  if (progress >= 1) return false;

  const sceneWidth = ctx.videoLayout?.maskRect.width ?? ctx.width;
  const x = ctx.cx * ctx.width;
  const y = ctx.cy * ctx.height;
  const radius = sceneWidth * 0.03 * progress;

  ctx.ctx.beginPath();
  ctx.ctx.arc(x, y, radius, 0, Math.PI * 2);
  ctx.ctx.stroke();
  return true;
});

CursorEffectContext now includes videoLayout, zoom, and sceneTransform, so effects can scale relative to the scene instead of the full canvas.

API Surface

Registration

api.registerRenderHook(phase, hook);
api.registerCursorEffect(effect);
api.registerFrame(frame);
api.registerWallpaper(wallpaper);
api.registerCursorStyle(cursorStyle);
api.registerSettingsPanel(panel);
api.on(event, handler);

Every registration returns a dispose function.

Settings

api.getSetting(settingId);
api.setSetting(settingId, value);
api.onSettingChange((settingId, value) => {});
api.getAllSettings();

Supported settings field types are toggle, slider, select, color, and text.

Assets and Audio

api.resolveAsset("images/overlay.png");
api.playSound("sounds/click.mp3", { volume: 0.8 });
api.log("hello", payload);

Read-only Queries

api.getVideoInfo();
api.getVideoLayout();
api.getCursorAt(timeMs);
api.getSmoothedCursor();
api.getZoomState();
api.getShadowConfig();
api.getKeystrokesInRange(startMs, endMs);
api.getAspectRatio();
api.getActiveFrame();
api.isExtensionActive(extensionId);
api.getPlaybackState();
api.getCanvasDimensions();
api.drawIcon(ctx, "Sparkle", 100, 100, 20, "#2563EB", "regular");

Drawing Icons

Extensions can draw icons from Recordly's bundled Phosphor icon set directly on a canvas context:

api.drawIcon(
  ctx,
  "ArrowClockwise", // icon name from @phosphor-icons/react
  120,              // x (center)
  80,               // y (center)
  18,               // size in px
  "#ffffff",        // color
  "bold",           // optional weight: thin | light | regular | bold | fill
);

This is useful for lightweight overlays and avoids bundling your own icon assets.

Settings Panels

api.registerSettingsPanel({
  id: "my-settings",
  label: "My Extension",
  icon: "sparkles",
  parentSection: "cursor",
  fields: [
    { id: "enabled", label: "Enable", type: "toggle", defaultValue: true },
    { id: "size", label: "Size", type: "slider", defaultValue: 1, min: 0.1, max: 3, step: 0.1 },
    {
      id: "style",
      label: "Style",
      type: "select",
      defaultValue: "ripple",
      options: [{ label: "Ripple", value: "ripple" }, { label: "Pulse", value: "pulse" }],
    },
    { id: "color", label: "Color", type: "color", defaultValue: "#2563EB" },
  ],
});

Use parentSection to nest your panel inside an existing area such as cursor or scene.

Events

Event Permission Description
playback:timeupdate timeline Fires each playback tick
playback:play timeline Playback started
playback:pause timeline Playback paused
cursor:click cursor Cursor click detected
cursor:move cursor Cursor move detected
timeline:region-added timeline Region added
timeline:region-removed timeline Region removed
export:start export Export started
export:frame export A frame was rendered during export
export:complete export Export finished

Frames, Wallpapers, and Cursor Styles

  • registerFrame() is the runtime API for device frames. Use a draw(ctx, width, height) function when possible for resolution-independent output.
  • registerWallpaper() contributes scene backgrounds.
  • registerCursorStyle() contributes cursor image packs.
  • All three rely on packaged files relative to the extension root.

Lifecycle

  1. Discovery: Recordly scans built-in extensions and the user extensions directory.
  2. Activation: activate(api) runs and you register hooks, effects, panels, and assets.
  3. Runtime: Registered callbacks execute in preview and export according to their phase.
  4. Deactivation: deactivate() runs and all registrations are automatically disposed.

Examples

  • extension-examples/webadderall.more-wallpapers shows a user-installable wallpaper bundle that registers 180 packaged wallpapers through registerWallpaper().