refactor: switch from mux/ai package to 🤐 by walkerfrankenberg · Pull Request #230 · muxinc/stream.new

walkerfrankenberg · 2026-03-17T19:07:12Z

Overview

Replace the @mux/ai package and its dual OpenAI/Hive moderation providers with direct calls to the Mux Robots API. Migrate all HTTP calls from got to native fetch, fix the resulting test breakages, and add missing response.ok checks.

What was changed

lib/robots-client.ts (new) — HTTP client for the Mux Robots API (/jobs/moderate, /jobs/summarize, /jobs/ask-questions, polling via /jobs/:id)
types/robots.ts (new) — TypeScript types for Robots API job responses and outputs, replacing @mux/ai/workflows types
workflows/process-mux-ai.ts — Rewrote the workflow to use robots-client instead of @mux/ai. Removed caption-hook machinery (no longer needed); moderation now uses a single provider instead of OpenAI + Hive
app/api/webhooks/mux/route.ts (new, replaces mux-ai/route.ts) — Simplified webhook handler; only handles video.asset.ready, removed track.ready/track.errored caption-hook logic
lib/moderation-action.ts — Switched from got to fetch, collapsed dual-provider logic into single moderationResult, added response.ok check on Airtable call
lib/slack-notifier.ts — Switched from got to fetch, merged dual OpenAI/Hive score display into single moderation block, added response.ok checks on all Slack webhook calls
app/api/report/route.ts — Switched from got to fetch, added response.ok check on Airtable call
lib/moderation-action.spec.ts — Updated types, set Airtable env vars in beforeEach, hardcoded nock URLs to match
lib/slack-notifier.spec.ts — Updated types, added @jest-environment node for fetch availability
package.json — Removed @mux/ai, upgraded nock to v14 (native fetch interception)

Suggested review order

types/robots.ts — new type definitions that everything else depends on
lib/robots-client.ts — new API client; check auth, polling, and error handling
workflows/process-mux-ai.ts — core workflow rewrite
app/api/webhooks/mux/route.ts + deleted mux-ai/route.ts — webhook simplification
lib/moderation-action.ts + lib/slack-notifier.ts + app/api/report/route.ts — got → fetch migration and response.ok fixes
lib/moderation-action.spec.ts + lib/slack-notifier.spec.ts — test fixes

Note

Medium Risk
Moderation and auto-deletion flow is reworked to a new Robots job/polling integration and webhook entrypoint, which could affect content deletion and notifications if job status handling or auth is misconfigured. Signature verification behavior is tightened in production, which may reject webhooks if env configuration is incomplete.

Overview
Replaces the @mux/ai dual-provider (OpenAI/Hive) workflow with direct Mux Robots API jobs for moderate, summarize, and ask-questions, including new lib/robots-client.ts, Robots output types, and polling/normalization logic in workflows/process-mux-ai.ts.

Simplifies the Mux webhook surface by deleting app/api/webhooks/mux-ai/route.ts (caption hook + track event handling) and adding app/api/webhooks/mux/route.ts that only starts the workflow on video.asset.ready, while requiring signature verification in production.

Migrates outbound HTTP from got to native fetch across Airtable/Slack/reporting code paths, adding res.ok checks and removing lib/got-client.ts; Slack moderation messaging is collapsed to a single moderation score block, tests are updated accordingly, and @mux/ai/got deps are removed (with nock bumped to v14).

^{Written by Cursor Bugbot for commit 124d498. This will update automatically on new commits. Configure here.}

vercel · 2026-03-17T19:07:22Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
stream-new	Ready	Preview, Comment	Mar 25, 2026 9:18pm

walkerfrankenberg · 2026-03-17T19:08:00Z

lib/robots-client.ts

+
+type FetchFn = typeof globalThis.fetch;
+
+function toBase64(str: string): string {


it was erroring when I tried to use btoa 😕

errorName: 'ReferenceError', errorStack: 'ReferenceError: btoa is not defined\n' + ' at getAuthHeader (lib/robots-client.ts:5:4)\n' + ' at createModerationJob (lib/robots-client.ts:21:27)\n' + ' at moderateAndSummarize (workflows/process-mux-ai.ts:62:45)\n' +

is this a node-like vs. browser-like/Web API issue? should we use Buffer here? e.g. const encoded = Buffer.from("string-to-encode").toString('base64');

lib/robots-client.ts

workflows/process-mux-ai.ts

lib/slack-notifier.ts

cjpillsbury

Left some comments but saving approve for @philcluff on his return.

cjpillsbury · 2026-03-20T15:27:23Z

workflows/process-mux-ai.ts

 import { sendSlackModerationResult, sendSlackSummarizationResult, sendSlackAutoDeleteMessage } from '../lib/slack-notifier';
 import { checkAndAutoDelete, checkAndAutoDeleteWatchParty } from '../lib/moderation-action';
-import type { CaptionHookPayload, CaptionStatus } from '../types';
+import { createModerationJob, createSummarizeJob, createAskQuestionsJob, getJobStatus } from '../lib/robots-client';


question(non-blocking): Should we have kept the creation + polling per workflow abstracted away from this module to denoise and keep the code similar to directly using robots? I can envision future cases where we are building/dog-fooding things in stream.new defined in our SDK but not yet supported by robots (or possibly even never intended to be supported in robots?)

next-env.d.ts

app/api/webhooks/mux/route.ts

cjpillsbury · 2026-03-20T15:35:20Z

app/api/webhooks/mux/route.ts

+    }
+
+    // Event type not handled
+    return NextResponse.json({ message: 'Event type not handled' });


question(non-blocking): Should this be a 200 (noticed the equivalent from the old impl is the same here)

I believe so, non-2xx responses are considered delivery failures and will cause us to retry the webhook. Returning a non-2xx for an event type we intentionally don't handle would cause unnecessary retries

package.json

lib/robots-client.ts

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

cursor · 2026-03-25T21:38:18Z

next-env.d.ts

 /// <reference types="next" />
 /// <reference types="next/image-types/global" />
-import "./.next/types/routes.d.ts";
+import "./.next/dev/types/routes.d.ts";


Accidentally committed auto-generated dev path change

Low Severity

The import path changed from .next/types/routes.d.ts to .next/dev/types/routes.d.ts. The developer confirmed this was auto-generated by running next dev and was not an intentional change. This file states "NOTE: This file should not be edited." Committing a dev-specific generated path adds noise to the diff and may diverge from what next build produces in CI.

refactor: switch from mux/ai package to robots API

620db13

walkerfrankenberg commented Mar 17, 2026

View reviewed changes

philcluff changed the title ~~refactor: switch from mux/ai package to robots API~~ refactor: switch from mux/ai package to 🤐 Mar 17, 2026

cursor bot reviewed Mar 17, 2026

View reviewed changes

lib/robots-client.ts Outdated Show resolved Hide resolved

workflows/process-mux-ai.ts Show resolved Hide resolved

lib/slack-notifier.ts Show resolved Hide resolved

walkerfrankenberg marked this pull request as draft March 17, 2026 19:18

tweak

82220c3

vercel bot deployed to Preview March 17, 2026 22:03 View deployment

address comments, clean up

f63c5c0

vercel bot deployed to Preview March 17, 2026 22:24 View deployment

walkerfrankenberg marked this pull request as ready for review March 17, 2026 22:25

walkerfrankenberg requested a review from philcluff March 17, 2026 23:59

cjpillsbury reviewed Mar 20, 2026

View reviewed changes

address comments

e8097f7

vercel bot deployed to Preview March 25, 2026 00:09 View deployment

cursor bot reviewed Mar 25, 2026

View reviewed changes

package.json Show resolved Hide resolved

lib/robots-client.ts Show resolved Hide resolved

move to step

f46672c

vercel bot deployed to Preview March 25, 2026 21:15 View deployment

remove got

124d498

vercel bot deployed to Preview March 25, 2026 21:18 View deployment

cursor bot reviewed Mar 25, 2026

View reviewed changes


		type FetchFn = typeof globalThis.fetch;

		function toBase64(str: string): string {

Conversation

walkerfrankenberg commented Mar 17, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

What was changed

Suggested review order

Uh oh!

vercel bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

walkerfrankenberg Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

cjpillsbury Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cjpillsbury left a comment

Choose a reason for hiding this comment

Uh oh!

cjpillsbury Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cjpillsbury Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

walkerfrankenberg Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Mar 25, 2026

Choose a reason for hiding this comment

Accidentally committed auto-generated dev path change

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

walkerfrankenberg commented Mar 17, 2026 •

edited by cursor bot

Loading

vercel bot commented Mar 17, 2026 •

edited

Loading