diff --git a/.gitignore b/.gitignore old mode 100755 new mode 100644 diff --git a/.opencode/skills/motion/SKILL.md b/.opencode/skills/motion/SKILL.md deleted file mode 100755 index 8ad8b65..0000000 --- a/.opencode/skills/motion/SKILL.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -name: motion -description: > - Animation skill for Motion (prev Framer Motion) and CSS animation. Provides: animation best practices (including specific advice for vanilla JS, React, Vue, Base UI and Radix), documentation and example search, CSS spring and bounce generation, MotionScore code and runtime performance audits, and the visual transition editor. Use when writing animations, working with Motion (motion, motion/react, motion-v, framer-motion), animating a UI, writing CSS linear() springs, auditing performance/jank/layout thrash via code or runtime, searching Motion docs or examples, adding a Motion UI section, or upgrading between Motion versions. -argument-hint: "[subcommand or question, e.g. 'audit src/Modal.tsx', 'spring bounce 0.3', 'upgrade', 'how do I animate a list']" ---- - -# Motion - -Animation for the web, done properly. - -- [Animation best practices](best-practices/index.md): "Animate this button", "Fade this layer in", "Animate this Vue component". Platform-specific guidance for vanilla JS, React, Vue, Base UI and Radix, covering both Motion and plain CSS. -- [Documentation, examples and Motion UI search](codex/index.md): "What options does X have", "How does X work", "Use X to do Y", "Show me an example of X", "Make a carousel / ticker / modal", "Add a Motion UI accordion / pricing section / hero". -- [CSS spring and bounce generation](css-spring/index.md): "Generate a CSS spring with a bounce of 0.5 over 0.3s", "Make this bouncier", "Give me a bounce easing". -- [MotionScore performance audit](performance-audit/index.md): "Audit src/Modal.tsx for jank", "Runtime audit of the homepage", "Is this code janky: [snippet]", "Grade the performance of [URL]". You may also run audits proactively and report what you find. Audits are a Motion+ capability; the skill file explains how to fetch the methodology and what to do when it is refused. -- [Transition preview](transition-preview/index.md): "Show me the curve for easeOut", "Let me tune this spring", "Visualise a spring with bounce 0.5". - -## Upgrading Motion - -"/motion upgrade", "migrate from framer-motion", "upgrade to Motion 12" and -similar all resolve through documentation search — there is no separate tool. - -1. **Read the installed version first.** Check `package.json` for `motion`, - `framer-motion` or `motion-v` before searching. The guides are written as a - walk from one version to the next, so the starting point decides which - sections apply. -2. Search the codex for `upgrade` on the project's platform. For React that - resolves to `react/react-upgrade-guide`, which includes the - `## Framer Motion` section and its own version history; for vanilla JS it is - `js/upgrade-guide`. Coming from GSAP, search `migrate from gsap`. -3. **Read the whole page and follow it in order. Do not summarise it.** Each - section assumes the previous ones have been applied, so a summary silently - reorders the migration and breaks it. -4. Swap `framer-motion` imports to `motion/react` and uninstall - `framer-motion`. They must never both be installed. - -## Tiers - -Best practices, search and easing generation work without an account. The -rest is tiered, and the tools say so when you reach them: - -- **A Motion account** (free): saving a transition. Run the Motion+ MCP - server, signed in from the editor's MCP settings. -- **Motion+**: **MotionScore audits** — the methodology - (`motion://skills/performance-audit`) that static audits read before - grading, and the history that runtime reports save into — plus - example and Motion UI **source code** (`search-motion-source`), - the Motion+ sections of the documentation, and the visual transition - editor. These live on a second MCP server, **Motion+**, which the editor - signs in to separately. Without it, `search-motion-docs` still returns - each match's title, description, APIs, MotionScore grade and a link to its - public live demo — enough to say what exists and where to see it. Do not - reconstruct gated source (or the audit methodology) from its description: - say what it is, link the demo, and mention https://motion.dev/plus once. - -## If the Motion MCP server is unavailable - -`best-practices/` is self-contained and works with no server at all — use it -directly. Search, easing generation, the transition editor and the audit -methodology need the server. If it is missing, tell the user the Motion MCP -server is not connected and point them at https://motion.dev/docs/ai-kit. diff --git a/.opencode/skills/motion/best-practices/base-ui.md b/.opencode/skills/motion/best-practices/base-ui.md deleted file mode 100755 index 9a5f014..0000000 --- a/.opencode/skills/motion/best-practices/base-ui.md +++ /dev/null @@ -1,106 +0,0 @@ -# Animating Base UI with Motion for React - -Rules for integrating Motion animations with Base UI components. - -## Adding Animations - -Pass a `motion` component via the Base UI `render` prop: - -```jsx - - } -> -``` - -**Don't** use the function/spread props approach — it causes type errors. - -## Exit Animations - -### Standard Approach - -For most components, use `AnimatePresence` with the `exit` prop as usual: - -```jsx - - {open && ( - - } - /> - )} - -``` - -### Self-Managing Components - -Some Base UI components (e.g. `ContextMenu`, `Popover`) control their own conditional rendering. For exit animations on these: - -1. **Hoist their open state** with `useState`: - ```jsx - const [open, setOpen] = useState(false) - - return ( - - ``` - -2. **Add `keepMounted` to `Portal`** and wrap with `AnimatePresence`: - ```jsx - - {open && ( - - ``` - -3. **Add exit animation** via `render` prop on a `motion` component: - ```jsx - - } - > - ``` - -### Full Example - -```jsx -function App() { - const [open, setOpen] = useState(false) - - return ( - - Open menu - - {open && ( - - - - } - > - {/* Children */} - - - - )} - - - ) -} -``` - -**Note:** `Portal` keeps the tree mounted as long as Base UI detects animations via `element.getAnimations()`. Motion runs `opacity`, `transform`, `filter`, and `clipPath` via hardware acceleration — ensure at least one of these is used for exit animations. diff --git a/.opencode/skills/motion/best-practices/index.md b/.opencode/skills/motion/best-practices/index.md deleted file mode 100755 index 497a80d..0000000 --- a/.opencode/skills/motion/best-practices/index.md +++ /dev/null @@ -1,75 +0,0 @@ -# Animation best practices - -## Platform-specific rules - -- [React](react.md) -- [Vue](vue.md) -- [Vanilla JS](motion.md) -- [Base UI](base-ui.md) - -## Universal rules (all platforms) - -### Performance - -#### Execution speed - -Inside functions that run every animation frame (rAF callbacks, `useTransform` callbacks, pointer move callbacks, `onUpdate`, `frame.render` etc): - -- Avoid object allocation. Prefer mutation where safe. -- Prefer `for` loops over `forEach` of `map`, unless function callback can be pre-allocated. -- Avoid `Object.entries`, `Object.values`. - -#### Animating via `transform` vs independent transforms - -Motion can animate transforms either via `transform` or `x`, `y`, `scale` etc. - -```javascript -animate(element, { transform: "scale(2)" }) -animate(element, { scale: 2 }) -``` - -```jsx - - -``` - -Prefer `transform` as these animations will run via WAAPI. Use independent transforms when: - -- Some transforms have different transition settings -- Some transforms need to be passed in as motion values - Note: Passing `transform` in as a motion value will also disable WAAPI animations, so no need to prefer it if you would resort to this. -- Defining transforms via `style` prop -- Use independent transforms when you have competing/composable transforms: - -```javascript -animate(element, { x: 100 }) - -hover(() => { - animate(element, { scale: 1.2 }) - return () => animate(element, { scale: 1 }) -}) -``` - -```jsx - -``` - -#### will-change - -When animating with CSS `transition` or Motion independent transforms `x`, `y`, `scale` etc, set `will-change` on the animating properties so the browser promotes the element to its own compositor layer. Use it sparingly and remove it once the animation finishes. - -When animating with CSS `animation` or Motion via `transform`, this is unnecessary — the layer is promoted automatically by the browser. - -### Design - -In general, prefer physics-based springs for physical motion such as `x`, `rotate` etc. Especially when it could be interrupted. - -Non-numerical values won't use spring physics so you can use more predictable settings like `type: "spring", bounce: 0.2, visualDuration: 0.4` - -Consider the kind of interface you are building. If a serious website like stock trading, don't use overshoot in your springs or easing curves. If it's a wedding site, you can use softer curves and slightly longer durations. - -### API best practice - -#### MotionValues - -- Never use `motionValue.onChange(update)` — always use `motionValue.on("change", update)` diff --git a/.opencode/skills/motion/best-practices/motion.md b/.opencode/skills/motion/best-practices/motion.md deleted file mode 100755 index d0b59c8..0000000 --- a/.opencode/skills/motion/best-practices/motion.md +++ /dev/null @@ -1,25 +0,0 @@ -# Motion (Vanilla JS / HTML / TypeScript) - -Rules for using Motion in vanilla JavaScript, TypeScript, and HTML projects. - -## Importing - -- Import from `motion`, never from `framer-motion`. - -## `animate` - -`animate` has three valid syntaxes: - -1. **MotionValue**: `animate(motionValue, targetValue, options)` -2. **Plain value**: `animate(originValue, targetValue, options)` — add `onUpdate` to `options` -3. **Element/object**: `animate(objectOrElement, values, options)` - -When animating motion values, don't track the current animation in a variable — use `value.stop()` to end the current animation. Starting a new animation on the same value automatically cancels the previous one. - -## Easing - -Easing is defined via the `ease` option using camelCase: `easeOut`, `easeInOut`, `circOut`, etc. Not `ease-out` or `ease-in-out`. - -## API guidance - -The latest docs are available via the Motion MCP. Check the [Codex](../codex/index.md) documentation. diff --git a/.opencode/skills/motion/best-practices/react.md b/.opencode/skills/motion/best-practices/react.md deleted file mode 100755 index 93cee15..0000000 --- a/.opencode/skills/motion/best-practices/react.md +++ /dev/null @@ -1,49 +0,0 @@ -# Motion for React - -Rules for using Motion in React and TypeScript projects. Framer Motion is now called Motion for React — all Framer Motion knowledge applies. - -## Importing - -- **Never** import from `framer-motion`. -- Import from `motion/react` in client components. -- In server components, import `motion` like: `import * as motion from "motion/react-client"` -- Files marked `"use client"` must import from `"motion/react"`. -- The `animate` function: import from `"motion/react"` in React files, from `"motion"` elsewhere. - -## MotionValues - -- **Never** read from a `MotionValue` in a render. Only read in effects/callbacks. - - OK: `useTransform(() => value.get())` - - Bad: `propName={value.get()}` - -## React Patterns - -- Compose chains of `useTransform`, `useSpring`, `useMotionValue`, and `useVelocity` rather than complex imperative logic -- Prefer `willChange` over `transform: translateZ(0)` -- When animating MotionValues: - - Use `animate()` to animate the source MotionValue directly - - Don't use the `transition` prop when values are driven by MotionValues via `style` - - Derived values (via `useTransform`, `useSpring`) automatically follow the source animation - -## `useTransform` - -Two current syntaxes: - -1. `useTransform(value, inputRange, outputRange, options)` — prefer this -2. `useTransform(() => otherMotionValue.get() * 2)` — function syntax - -**Deprecated** (never use): `useTransform(value, (latestValue) => newValue)` - -## Radix Integration - -When integrating with Radix: - -- Add animations via `asChild` + a `motion` component child (`motion.div`, `motion.li`) -- For exit/layout animations, hoist Radix state into `useState` (`open`/`onOpenChange`, `value`/`onValueChange`) -- Conditionally render the Radix component as child of `AnimatePresence` -- The component accepting `forceMount` is what goes inside `AnimatePresence`, and `forceMount` must be set -- Only apply `forceMount` on Radix components, never on DOM elements - -## API guidance - -The latest docs are available via the Motion MCP. Check the [Codex](../codex/index.md) documentation. diff --git a/.opencode/skills/motion/best-practices/vue.md b/.opencode/skills/motion/best-practices/vue.md deleted file mode 100755 index 94db1ca..0000000 --- a/.opencode/skills/motion/best-practices/vue.md +++ /dev/null @@ -1,37 +0,0 @@ -# Motion for Vue - -Rules for using Motion in Vue projects. - -## Importing - -- Always import from `motion-v` and nothing else. -- Import components and functions: `import { motion, useMotionValue } from 'motion-v'` - -## Patterns - -- Don't read MotionValue directly in templates — use `watch` or callbacks instead -- Use `ref` for state management -- Use `:style` for dynamic styles in templates -- Compose `useTransform`, `useSpring`, `useMotionValue`, and `useVelocity` rather than complex conditionals -- Prefer `willChange` over `transform: translateZ(0)` -- When using MotionValues: - - Use `animate()` to animate the source MotionValue directly - - Don't use `transition` prop when values are driven by MotionValues via `:style` - - Derived values (via `useTransform`, `useSpring`) automatically follow the source animation - -## `useTransform` - -Two syntaxes: - -1. `useTransform(value, inputRange, outputRange, options)` — prefer this -2. `useTransform(() => otherMotionValue.get() * 2)` - -## Component Integration - -- Wrap HTML elements with motion components (`motion.div`, `motion.li`) -- For exit/layout animations, use `v-if`/`v-show` with `AnimatePresence` -- Use `ref` or `reactive` for state management - -## API guidance - -The latest docs are available via the Motion MCP. Check the [Codex](../codex/index.md) documentation. diff --git a/.opencode/skills/motion/codex/index.md b/.opencode/skills/motion/codex/index.md deleted file mode 100755 index d33bb52..0000000 --- a/.opencode/skills/motion/codex/index.md +++ /dev/null @@ -1,95 +0,0 @@ -# Codex: Documentation, examples & Motion UI search - -The Motion Codex finds the official Motion API documentation, working code examples, and Motion UI components and sections. - -Call it **before** implementing any non-trivial animation. Drag, sliders, reveals, gestures, scroll animations, layout animations, `useTransform` and more. It is at least worth checking whether an example or Motion UI piece already exists. Then build from the result rather than writing from memory. - -## Two servers - -The plugin registers two MCP servers, and which tools you have tells you what -you can deliver: - -- **Motion** is always available, needs no account, and carries - `search-motion-docs` and `generate-css-easing`. -- **Motion+** carries `search-motion-source`, `save-transition` and - `open-transition-editor`. Its tools appear only once the editor is signed - in to it *and* the account has Motion+. - -**Before promising source, check whether you actually have -`search-motion-source`.** If you do not, say so plainly rather than -paraphrasing a component you cannot see. See "When source is unavailable". - -## 1. Search - -``` -search-motion-docs({ platform, searchTerm }) -``` - -- **platform** (required) — exactly one of `"js"`, `"react"`, `"vue"`. There is no `ts`, `html`, `svelte`, etc. -- **searchTerm** (required) — the component or concept to find, e.g. `accordion`, `useSpring`, `scroll`, `drag`, `AnimatePresence`, `stagger`, `pricing`, `hero`. - -### Search by concept, not by the word "animation" - -The tool strips `animate`, `animation`, `animations` and `animated` from the query. A search of only those words returns "too generic". Search the _thing_ being animated or the _API_ needed: - -- ✅ `scroll`, `drag`, `accordion`, `useSpring`, `shared layout` -- ❌ `animation`, `animate a component` - -Matching is fuzzy and typo-tolerant, so close terms still hit. Minimum 2 characters. - -## 2. Return type - -A short set of adaptation rules, followed by MCP **resource links** and, where content is gated, a metadata block instead. - -- Up to **3 docs** first, for API and option lookups — `motion://docs/{platform}/{id}`. Available to everyone. -- Up to **5 examples** — `motion://examples/{platform}/{id}`. -- **Motion UI** (`platform: "react"` only): components and sections — `motion://ui/react/{id}`. Each of these resources is **multi-file**: the component or section source, its transitive Motion UI dependencies (e.g. `ui-theme`), and `motion.theme.ts`. Reading one returns the complete paste-ready files. -- The signed-in user's own saved transitions, as JSON. - -**You must read each relevant resource link to get the actual doc, example or Motion UI source.** Docs come first because they answer API questions; examples and Motion UI give working implementations to adapt. - -If nothing matches, broaden the term and search again — results are capped and fuzzy, not exhaustive. - -## 2a. Fetching source - -``` -search-motion-source({ platform, searchTerm }) -``` - -Motion+ only, on the Motion+ server. Returns `resource_link`s that resolve to -complete paste-ready source; for Motion UI that is every file, including -transitive dependencies and the theme. - -Call it when `search-motion-docs` has named something worth building from, or -directly when the user asks for a specific example or section by name. - -### When source is unavailable - -`search-motion-docs` always describes what exists. It never returns source: -that is `search-motion-source`, and you only have that tool when this editor -is signed in to the Motion+ server with a Motion+ account. - -If you do not have it, **say so in your reply** rather than quietly building -something approximate: - -> The Motion+ examples that match are [names], with demos at [links]. Their -> source needs Motion+ (https://motion.dev/plus). If you already have it, sign -> in to the Motion+ MCP server from Settings, MCP, Motion+, Log in. - -Handle that honestly: - -- **Tell the user what exists and link the demo.** The demo pages (`examples.motion.dev/...`, `motion.dev/ui/sections/...`, `motion.dev/ui/components/...`) are public and run the real thing. -- **Do not reconstruct the source from the description.** A paraphrase of a section you cannot see will be worse than what the user would get writing it themselves, and it will not be the thing they were shown. -- **Mention https://motion.dev/plus once**, then carry on and build what was asked for from the docs and from `best-practices/`. A gated result is not a dead end; it is one route among several. -- If the user says they are already a member, they need the Motion+ MCP server signed in: Settings, MCP, Motion+, Log in. `search-motion-source` appears once that is done. - -## 3. Implement - -The response embeds adaptation rules. Follow them: - -- Adapt colours, fonts and styling to the host project; match its conventions (use Tailwind classes in a Tailwind project, and so on). -- Install any referenced packages. -- **Never import from `framer-motion`** — only from `motion`. Migrate any existing `framer-motion` imports. -- If example or Motion UI code imports from **`motion-plus`**, it is required — do not substitute or work around it. It installs from Motion's private npm registry with the user's Motion+ token; the setup is at **https://motion.dev/docs/react-motion-plus-installation**. Tell the user to generate a token at **https://motion.dev/dashboard/tokens**. Never ask them to paste a token into chat. -- **Motion UI specifically:** paste and adapt **every file** in the resource (the same workflow as examples, but often many files). Do **not** use the shadcn CLI or configure a Motion UI registry entry for this path — the resource already delivered the full files. If `motion.theme.ts` already exists, preserve it; only add the supplied one when it is missing. Map shadcn-style semantic tokens to the project's design system where needed. Preserve animation structure and reduced-motion behaviour. -- **Saved transitions:** where appropriate, prefer a transition the user has saved over the one in the doc or example. Choose sensibly — no very bouncy springs on a stock-trading dashboard. diff --git a/.opencode/skills/motion/css-spring/index.md b/.opencode/skills/motion/css-spring/index.md deleted file mode 100755 index a5c1e19..0000000 --- a/.opencode/skills/motion/css-spring/index.md +++ /dev/null @@ -1,70 +0,0 @@ -# Generate a CSS spring or bounce - -Springs and bounces are not native CSS easings, so Motion approximates them by -sampling the curve into a `linear()` easing function. One tool covers both. - -## Usage - -``` -generate-css-easing({ kind, duration, bounce }) -``` - -- **kind** — `"spring"` (default) for the usual springy settle, or `"bounce"` - for a ball landing on a hard surface. -- **duration** (seconds) — the **perceptual** duration: how long the motion - appears to take. Defaults to `0.4` for a spring and `1` for a bounce. -- **bounce** (0 to 1) — how much the spring overshoots. `0` is a firm settle - with no overshoot, `1` is maximum wobble. Defaults to `0.2`. - -### The one thing that is easy to get wrong - -`bounce` means two different things in the same sentence, so read carefully: - -- As a **kind**, `"bounce"` is the gravity-like bouncing-ball easing. -- As a **parameter**, `bounce` is the springiness of a spring. - -When `kind` is `"bounce"`, the `bounce` parameter is ignored — the feel of a -bounce is controlled by duration alone. - -### Reading the result - -The tool returns the ` ` half of a CSS transition, so use it -as `transition: ;`. - -For a spring, that duration is **longer** than the one you asked for, because -it includes the settle after the motion has visually arrived. Time any sibling -animations off the duration you asked for, not the one that came back: - -```css -/* generate-css-easing({ kind: "spring", duration: 0.2, bounce: 0.3 }) */ -transition: - opacity 0.2s linear, - transform 0.35s linear(0, 0.28, 0.78, 1.04, ...); -``` - -### Choosing values - -- Snappy or quick: around `0.2s` -- Normal: `0.3s` to `0.4s` -- Slow or heavy: around `1s` -- Bounces read better long. `1s` feels like normal gravity; shorter feels - heavier, longer feels lighter or lower-gravity. -- Match the product. A stock-trading interface should not overshoot. A - wedding site can afford softer curves and longer durations. - -### Examples - -> "Generate a bouncy spring for a modal entrance" - -→ `generate-css-easing({ kind: "spring", duration: 0.35, bounce: 0.4 })` - -> "Make this drop like it hits the floor" - -→ `generate-css-easing({ kind: "bounce", duration: 1 })` - -## Only for CSS - -This is for hand-written CSS. Inside Motion, use a spring transition directly — -`{ type: "spring", visualDuration: 0.4, bounce: 0.2 }` — rather than pasting a -sampled curve. The real spring can be interrupted mid-flight and pick up the -current velocity; a `linear()` approximation cannot. diff --git a/.opencode/skills/motion/performance-audit/index.md b/.opencode/skills/motion/performance-audit/index.md deleted file mode 100755 index eaa1693..0000000 --- a/.opencode/skills/motion/performance-audit/index.md +++ /dev/null @@ -1,44 +0,0 @@ -# MotionScore performance audit - -MotionScore grades every animation by its render-pipeline cost, from S -(compositor-only, near-zero) down to F (forced synchronous layout every -frame). Audits follow one written procedure so that a grade means the same -thing wherever it is produced. - -## Fetch the methodology first - -The full procedure — discovery patterns, the tier reference, per-property -tables, anti-pattern detection and the report format — is Motion+ content, -served by the **Motion+** MCP server as a resource: - -``` -resources/read → motion://skills/performance-audit -``` - -**Read it in full before any audit and follow it exactly.** Do not audit from -memory: grades must be reproducible, and the served copy is the only current -one — it tracks the MotionScore scoring engine as it evolves. - -## If the read is refused - -- **Not signed in**: tell the user to sign in to the Motion+ MCP server from - the editor's MCP settings (in Cursor: Settings, MCP, Motion+, Log in). -- **Signed in without Motion+**: MotionScore audits are a Motion+ - capability. Say so plainly and mention https://motion.dev/plus once. Do - not improvise a MotionScore grade from general knowledge. - -## Runtime audits - -When the prompt names a URL (a dev server, a deployed page) or asks for a -"runtime" audit, run: - -``` -npx motionscore --agent -``` - -Static and runtime audits triangulate well: run both and merge findings as -the methodology describes. - -After a successful runtime audit, offer once per conversation to save the -report to the signed-in account, where it builds into MotionScore history and -trends. Never withhold or trim the report over it. diff --git a/.opencode/skills/motion/transition-preview/index.md b/.opencode/skills/motion/transition-preview/index.md deleted file mode 100755 index b647c55..0000000 --- a/.opencode/skills/motion/transition-preview/index.md +++ /dev/null @@ -1,51 +0,0 @@ -# Transition preview - -Numbers are a poor way to describe how something feels. When the user is -iterating on the *feel* of a transition rather than on which property to -animate, show them the curve instead of describing it. - -## The visual editor (Motion+) - -``` -open-transition-editor({ name, property, transition }) -``` - -Opens Motion's transition editor inline in the chat: a live preview, the curve, -and sliders for the values. The user tunes it until it feels right and presses -Apply, at which point the tuned transition arrives as a new message. - -- **Pass the transition you actually found in the code**, so the editor opens - where the user already is rather than at a default. -- **name** labels the editor, e.g. `"Card hover"`. -- **property** drives the preview, e.g. `"transform"`, `"opacity"`. -- When Apply comes back, **write those exact values into the source.** Do not - re-derive or round them; the user chose them by eye. - -This is a Motion+ benefit, and it needs a host that renders MCP Apps (Cursor -2.6 and later). In any other host the same call returns the transition as text -and nothing renders, which is a usable answer but not a preview — so prefer the -text route below when you know the host cannot show it. - -## Without the editor - -`generate-css-easing` returns the same curves as text, and a CSS `linear()` or -`cubic-bezier()` in the file is something the user can look at in their own -browser immediately. See [css-spring/index.md](../css-spring/index.md). - -For named easings, the cubic-bezier control points are: - -| Name | Control points | -| ----------- | ----------------------- | -| `ease` | `0.25, 0.1, 0.25, 1` | -| `easeIn` | `0.42, 0, 1, 1` | -| `easeOut` | `0, 0, 0.58, 1` | -| `easeInOut` | `0.42, 0, 0.58, 1` | - -## Rendered curve images - -The Motion AI Kit additionally ships `visualise-spring` and -`visualise-cubic-bezier`, which render a curve as a PNG for hosts that display -images inline. They are not part of this plugin. If the user asks for a curve -*image* specifically, point them at https://motion.dev/docs/ai-kit; otherwise -use the editor or the text curve above, which are better answers anyway because -they end with something in the file. diff --git a/CardReference.jsx b/CardReference.jsx deleted file mode 100755 index d754369..0000000 --- a/CardReference.jsx +++ /dev/null @@ -1,309 +0,0 @@ -import { useState, useRef, useEffect } from "react"; -import { motion } from "motion/react"; -import { LogIn } from "lucide-react"; - -const cards = [ - { - description: "Lana Del Rey", - title: "Summertime Sadness", - src: "https://assets.aceternity.com/demos/lana-del-rey.jpeg", - ctaText: "Play", - - content: ( -

- Lana Del Rey, an iconic American singer-songwriter, is celebrated for - her melancholic and cinematic music style. Born Elizabeth Woolridge - Grant in New York City, she has captivated audiences worldwide with her - haunting voice and introspective lyrics.

Her songs often - explore themes of tragic romance, glamour, and melancholia, drawing - inspiration from both contemporary and vintage pop culture. -

- ), - }, - { - description: "The Weeknd", - title: "Blinding Lights", - - src: "https://upload.wikimedia.org/wikipedia/en/thumb/e/e6/The_Weeknd_-_Blinding_Lights.png/250px-The_Weeknd_-_Blinding_Lights.png", - ctaText: "Play", - - content: ( -

- The Weeknd, born Abel Tesfaye, is a Canadian artist known for blending - R&B, pop, and synthwave into his unique sound. His track "Blinding - Lights" became a global sensation, defining the sound of the 2020s with - its retro vibes and emotional depth. -

- ), - }, - { - description: "Billie Eilish", - title: "Happier Than Ever", - src: "https://i.scdn.co/image/ab67616d0000b2732a038d3bf875d23e4aeaa84e", - ctaText: "Play", - - content: ( -

- Billie Eilish redefined pop music with her haunting vocals and dark, - intimate lyrics. Her song “Happier Than Ever” showcases her range and - emotional storytelling, transforming from quiet melancholy to raw - intensity. -

- ), - }, - { - description: "Taylor Swift", - title: "All Too Well", - src: "https://i.scdn.co/image/ab67616d0000b273da5d5aeeabacacc1263c0f4b", - ctaText: "Play", - - content: ( -

- Taylor Swift is celebrated for her narrative songwriting and evolution - across genres. “All Too Well (10 Minute Version)” is a masterclass in - storytelling, capturing heartbreak with vivid emotional detail. -

- ), - }, - { - description: "Post Malone", - title: "Circles", - src: "https://i.scdn.co/image/ab6761610000f178b645c02cfa05103299775097", - ctaText: "Play", - - content: ( -

- Post Malone blends hip-hop, rock, and pop effortlessly. “Circles” is a - reflective anthem about love and repetition, showcasing his melodic - instincts and laid-back vocal delivery. -

- ), - }, -]; - -const navItems = [ - { - title: "Home", - href: "/", - }, - { - title: "About", - href: "/about", - }, - { - title: "Contact", - href: "/contact", - }, - { - title: "Login", - href: "/login", - }, -]; - -const ExpandableCardList = () => { - const [currentCard, setCurrentCard] = useState(null); - const [hovered, setHovered] = useState(null); - - const cardRef = useRef(null); - - // TODO: Missing keyboard accessibility (Escape key to close) - // Improvement: Add keyboard event listener for Escape key - // Example: - // const handleKeyDown = useCallback((e) => { - // if (e.key === 'Escape' && currentCard) setCurrentCard(null); - // }, [currentCard]); - - useEffect(() => { - // Only add the listener if a card is currently open - if (!currentCard) return; - - // TODO: Event handler should be memoized with useCallback - // Improvement: Move handleClickOutside outside useEffect and wrap with useCallback - // This prevents recreating the function on every render - const handleClickOutside = (event) => { - // Check if the click is outside the card element - if (cardRef.current && !cardRef.current.contains(event.target)) { - setCurrentCard(null); // Closes the card - } - }; - - // Add event listener to the whole document - document.addEventListener("mousedown", handleClickOutside); - - // Cleanup function to remove the event listener - return () => { - document.removeEventListener("mousedown", handleClickOutside); - }; - }, [currentCard]); - - //NOTE: - // The main concept of the switching between list and detailed view is implemented using layoutId prop from motion library. When a card is clicked, its layoutId matches the one in the detailed view, enabling a smooth transition i.e. from list to detailed view and vice versa since both views share the 'same' layoutId. - - const handleCardClick = (card) => { - setCurrentCard(card); - }; - - return ( -
- {/* TODO: Missing body scroll lock when modal is open */} - {/* Improvement: Prevent background scrolling when card is expanded */} - {/* Example: Use useEffect to toggle overflow: hidden on body */} - - {currentCard && ( - <> - - - - - -
-
- - {currentCard.title} - - - {currentCard.description} - -
- - - {currentCard.ctaText} - -
- - - {currentCard.content} - -
- - )} - -
- - -
- {cards.map((card, idx) => ( - { - handleCardClick(card); - }} - key={card.title} - className="flex cursor-pointer items-center justify-between rounded-lg bg-white px-7 py-5 shadow-sm transition-colors hover:bg-gray-100" - > -
- -
- - {card.title} - - - {card.description} - -
-
- -
- - {card.ctaText} - -
-
- ))} -
-
-
- ); -}; - -export default ExpandableCardList; \ No newline at end of file diff --git a/IMPLEMENTATION_SUMMARY.md b/IMPLEMENTATION_SUMMARY.md deleted file mode 100755 index 8388ebd..0000000 --- a/IMPLEMENTATION_SUMMARY.md +++ /dev/null @@ -1,263 +0,0 @@ -# Onboarding Logo Transition - Implementation Summary - -## Overview - -Implemented a polished shared-layout transition for the MindStream onboarding flow using Motion for React. The logo smoothly animates from centered position (step 1) to the header (steps 2+), creating a continuous, premium experience. - -## Architecture - -### Component Structure - -``` -SidePanel.jsx -└── PanelShell (with headerProps) - ├── PanelHeader (conditionally shows logo) - │ └── Logo (small, in header) - └── OnboardingState (wrapped in MotionConfig) - └── StepWelcome - └── Logo (medium, centered) -``` - -### Key Components - -#### 1. Logo Component (`src/components/ui/Logo.jsx`) -- Uses `motion.img` from Motion for React -- Shares `layoutId="onboarding-logo"` for layout transitions -- Two size variants: - - `small` (w-5): Header position - - `medium` (w-28): Hero centered position -- No opacity changes - pure layout transition - -#### 2. PanelHeader (`src/components/layout/PanelHeader.jsx`) -- Conditionally renders Logo when `showLogo={true}` -- Logo positioned left of "mindstream" text with 2.5 gap -- Flexbox layout ensures smooth text shifting - -#### 3. PanelShell (`src/components/layout/PanelShell.jsx`) -- Accepts `headerProps` for flexible configuration -- Spreads props to PanelHeader for customization -- Maintains existing functionality for other states - -#### 4. OnboardingState (`src/panel/states/OnboardingState.jsx`) -- Wrapped entire return in `MotionConfig` for consistent timing -- Added `onStepChange` callback to notify parent -- `useEffect` to trigger callback when step changes -- Calls parent with current step number - -#### 5. SidePanel (`src/panel/SidePanel.jsx`) -- Tracks `onboardingStep` in state -- Passes `headerProps={{ showLogo: onboardingStep > 1 }}` -- Provides `onStepChange={setOnboardingStep}` callback - -## Motion Configuration - -```jsx - -``` - -- **Duration**: 600ms - calm, intentional, and elegant -- **Easing**: Cubic bezier `[0.25, 0.1, 0.25, 1]` - smooth ease-in-out -- **No spring/bounce**: Subtle and minimal, like Linear/Raycast -- **Choreographed sequence**: Logo animates first (0-600ms), then content fades in (400-700ms) - -## Animation Choreography - -### Stage 1: Logo Transition (0-600ms) -- Logo moves from center to header -- Wordmark smoothly slides to the right using Motion's `layout` prop -- Only the logo animates during this phase -- Page content remains static - -### Stage 2: Content Fade-in (400-700ms) -- **Only applies to step 1 → step 2 transition** -- Starts at 400ms (overlap with last 200ms of logo animation) -- Next page content fades in with 300ms duration -- Ensures logo reaches destination before content fully appears - -### Later Steps (2→3, 3→4, etc.) -- No logo animation, so no choreography delay needed -- Content fades in immediately (300ms duration, no delay) -- Maintains responsive feel for subsequent pages - -This conditional approach applies staging only where the shared logo animation exists. - -## How It Works - -### The Magic of layoutId + Simultaneous Rendering - -Motion's shared layout animation requires **both elements to exist in the DOM simultaneously**: - -1. **Critical Fix**: Header logo is **always rendered** during onboarding (even on step 1) - - On step 1: Hidden via `opacity-0 pointer-events-none absolute` - - On step 2+: Visible in normal header position - -2. **Step 1 State**: - - Centered logo: visible (w-28) with `layoutId="onboarding-logo"` - - Header logo: hidden but in DOM (w-5) with same `layoutId="onboarding-logo"` - -3. **User clicks "Get Started"**: Step changes from 1 → 2 - -4. **Step 2 State**: - - Centered logo: unmounts - - Header logo: becomes visible (opacity-0 → opacity-100 removed) - - **Motion sees both logos existed** and animates between positions - -5. **Motion interpolates**: - - Position (center → top-left header) - - Scale (w-28 → w-5) - - Opacity of wrapper (for header logo reveal) - -### Why Previous Implementation Failed - -The original implementation conditionally rendered the header logo only when `onboardingStep > 1`. This meant: -- Step 1: Only centered logo exists -- User clicks "Get Started" -- Step 2: Centered logo unmounts, header logo mounts -- **No overlap = no shared layout animation** - -The fix ensures both logos exist simultaneously during the critical transition moment. - -### Preventing Initial Animation While Enabling Transitions - -The challenge: Enable shared layout animation during transitions without triggering it on mount. - -Solution using **AnimatePresence with mode="wait"**: -- Wraps the entire step content container with a key based on current step -- On initial render (step 1): Only centered logo exists, no animation -- On transition (step 1 → 2): AnimatePresence delays unmount of step 1 content -- During this delay: Both logos briefly coexist, enabling shared layout animation -- Header logo wrapped in AnimatePresence to coordinate mounting -- After transition: Only step 2 content exists - -This approach provides clean enter/exit coordination without requiring both elements to exist on initial mount. - -### No Opacity Transitions - -The logo itself never changes opacity: -- No wrapper opacity transitions -- Only position and scale animate via `layoutId` -- AnimatePresence coordinates entrance/exit timing - -### Progress Indicator Flicker Fix - -The progress bar flickered because `visibleSteps` depends on `backendReachable`: -- Initially: `backendReachable = null` -- After backend check: `backendReachable = true/false` -- This caused step count to change from undefined → 3 or 4 - -Fixed by defaulting `needsConfigStep = true` when `backendReachable === null`, ensuring progress bar always shows 4 steps initially. - -### State Flow - -``` -User clicks "Get Started" - ↓ -OnboardingState: setStep(2) - ↓ -useEffect triggers: onStepChange?.(2) - ↓ -SidePanel: setOnboardingStep(2) - ↓ -PanelShell receives: headerProps={{ showLogo: true }} - ↓ -PanelHeader renders: - ↓ -Motion sees layoutId match and animates -``` - -## File Changes - -### New Files -- `src/components/ui/Logo.jsx` - Shared logo component with Motion - -### Modified Files -- `src/components/layout/PanelHeader.jsx` - Added logo rendering logic -- `src/components/layout/PanelShell.jsx` - Added headerProps passthrough -- `src/panel/states/OnboardingState.jsx` - Added MotionConfig and step callback -- `src/panel/SidePanel.jsx` - Added step tracking and header props - -### Documentation -- `ANIMATION_TEST_GUIDE.md` - Manual testing guide - -## Design Principles Applied - -### 1. Single Element Illusion -- Uses `layoutId` to make it feel like the same logo moving -- No opacity transitions or fade in/out -- Continuous visual element throughout - -### 2. Premium Animation Quality -- Smooth cubic-bezier easing (not bouncy springs) -- 400ms duration (neither too fast nor slow) -- Subtle and minimal (doesn't draw attention) -- Inspired by Linear, Raycast, Arc Browser - -### 3. Clean Architecture -- Shared Logo component (DRY principle) -- Props-based configuration (flexible) -- Unidirectional data flow (predictable) -- Minimal changes to existing code (low risk) - -### 4. Performance -- Motion handles layout calculations efficiently -- No manual DOM manipulation -- React reconciliation optimized by Motion -- Smooth 60fps animations - -## Motion vs CSS Transitions - -### Why Motion? -- ✅ Handles complex layout changes automatically -- ✅ Shared element transitions across components -- ✅ Smooth interruption of ongoing animations -- ✅ Better performance for layout animations -- ✅ Declarative API (simpler code) - -### Why not CSS? -- ❌ Can't animate between different DOM positions easily -- ❌ Would require manual coordinate calculations -- ❌ Harder to handle component mount/unmount -- ❌ More complex state management - -## Testing - -Run the build: -```bash -npm run build -``` - -Load extension in Chrome: -1. `chrome://extensions/` -2. Enable Developer mode -3. Load unpacked → select `dist` folder -4. Clear storage to reset onboarding -5. Test forward/backward navigation - -See `ANIMATION_TEST_GUIDE.md` for detailed test cases. - -## Future Enhancements - -### Potential Improvements -1. Add keyboard navigation support (arrow keys) -2. Preload animation for smoother first render -3. Add reduced-motion media query support -4. Consider adding subtle scale effect on hover -5. Implement page transitions for step content - -### Alternative Approaches Considered -1. **CSS-only**: Rejected - too complex, less smooth -2. **Separate logos with opacity**: Rejected - not a true shared element -3. **FLIP animation**: Rejected - Motion handles it better -4. **React Spring**: Rejected - too bouncy, harder to control - -## References - -- CardReference.jsx - Reference implementation for Motion patterns -- Motion for React docs: https://motion.dev/docs/react-quick-start -- Shared layout animations: https://motion.dev/docs/react-layout-animations - -## Conclusion - -The implementation successfully creates a polished, continuous onboarding experience using Motion's shared layout transitions. The logo elegantly morphs from the welcome screen's hero position to its permanent home in the header, establishing visual continuity and premium feel. - -The architecture is clean, maintainable, and follows React best practices while leveraging Motion's powerful animation primitives. The result feels similar to high-quality products like Linear and Raycast. diff --git a/MINDSTREAM_PROJECT_SUMMARY.md b/MINDSTREAM_PROJECT_SUMMARY.md old mode 100755 new mode 100644 index fb562f4..0929b46 --- a/MINDSTREAM_PROJECT_SUMMARY.md +++ b/MINDSTREAM_PROJECT_SUMMARY.md @@ -107,7 +107,7 @@ graph TD LLM[Gemini API: Script Generator] TTS[TTS Engine: edge-tts or KittenTTS] Assets[(Asset Library: bg videos + ambient audio)] - Compositor[MovieLite / FFmpeg: Video Compositor] + Compositor[MoviePy: Video Compositor] JobQueue -- reads emotion result --> MLWorker JobQueue -- sends emotion + context --> LLM @@ -139,9 +139,9 @@ graph TD - **Script Generation (Phase 3):** Google Gemini (`gemini-3.5-flash`) — returns a structured JSON payload with `script` (full spoken text) and `subtitles` (array of short display phrases) in a single API call, eliminating the need for a separate transcription step. - **TTS (Phase 3):** Xiaomi MiMo API (`mimo-v2.5-tts`, voice `Dean`) via `/v1/chat/completions`. Audio returned as base64-encoded MP3 in the response body — no streaming required. - **Subtitle Timing (Phase 3):** Purely local, proportional word-count distribution. Total TTS audio duration (read via `AudioFileClip.duration`) is divided across subtitle phrases proportionally by word count. Subtitles start at 0s with no delay, keeping them synced with the near-zero-latency MiMo TTS. Pause-weighting adds ~15% extra time to phrases ending with sentence punctuation (`.`, `!`, `?`, `…`). No upload to Gemini, no Whisper, no AssemblyAI — free-tier safe. -- **Subtitle Rendering:** Subtitles overlaid using MovieLite/FFmpeg compositor at bottom of frame, uppercase, yellow text (`#FFFF00`) with black stroke. +- **Subtitle Rendering:** MoviePy `SubtitlesClip` overlaid at vertical position `1700` (bottom ~11% of 1920px frame), font size `80`, uppercase, yellow text (`#FFFF00`) with black 3px stroke. - **Video Search:** Pexels API with cinematic/moody query terms extracted by Gemini from the script. Fallback terms (`moody nature`, `dusk calm`, `foggy forest`) used if primary queries return no results. -- **Media Rendering (Phase 3):** Python + `MovieLite`/`FFmpeg` (4x faster than MoviePy with support for `normal` background preset and `fast` multi-worker preset), compositing Pexels-sourced video clips + Edge-TTS/MiMo audio + ambient audio (15% volume) + proportional subtitles into a 9:16 (720×1280 or 1080×1920) MP4. +- **Media Rendering (Phase 3):** Python + `MoviePy`/`FFmpeg`, compositing Pexels-sourced video clips + Xiaomi MiMo TTS audio + ambient audio (15% volume) + proportional subtitles into a 9:16 (1080×1920) MP4. - **Asset Library:** A local folder (`assets/audio/`) of ambient audio tracks organised by emotion. Background videos are fetched dynamically from Pexels per generation (no pre-built video library needed). ## 7. Detailed Implementation Workflow @@ -306,7 +306,7 @@ const assets = ASSET_MAP[emotion.label] || ASSET_MAP["neutral"]; **Sourcing:** Pexels, Pixabay (free stock footage), or AI-generated (RunwayML, etc.) -#### 3.4: Video Composition (MovieLite) +#### 3.4: Video Composition (MoviePy) **File:** `backend/workers/reel_compositor.py` ```python @@ -352,7 +352,7 @@ generate_reel( - Duration distributed across phrases proportionally by word count (not character count) - Negative lead offset (-0.15s) ensures subtitles appear slightly before audio - Sentence-ending punctuation gets ~15% extra duration for natural pauses -- SRT written to `output/audio/.srt`, loaded by MovieLite subtitle overlays +- SRT written to `output/audio/.srt`, loaded by MoviePy `SubtitlesClip` - Rendered as uppercase yellow text (`#FFFF00`), font size 80, black 3px stroke - Positioned at y=1700 (bottom ~11% of 1920px frame) - **No upload to Gemini files API, no Whisper, no AssemblyAI — free-tier safe** @@ -524,7 +524,7 @@ Extension polls `GET /jobs/:id`, sees `status: "ready"`, fires notification. - [ ] Install dependencies: `moviepy`, `edge-tts` (or `kittentts`) - [ ] Generate TTS from script → `test_audio.wav` - [ ] Create a test asset: `assets/backgrounds/frustrated.mp4` (download from Pexels) -- [ ] Composite with MovieLite → output `test_reel.mp4` +- [ ] Composite with MoviePy → output `test_reel.mp4` - [ ] Verify: 9:16 video, ~30-40s duration, audio plays correctly **Success criteria:** Can generate a watchable reel from hardcoded inputs. @@ -580,7 +580,7 @@ Extension polls `GET /jobs/:id`, sees `status: "ready"`, fires notification. - ✅ Xiaomi MiMo TTS (`Dean` voice) generates MP3 audio - ✅ Pexels API fetches dynamic cinematic video clips based on Gemini-extracted keywords - ✅ Proportional local subtitle timing with negative lead offset — no STT/transcription/upload -- ✅ MovieLite composites clips + TTS + ambient audio + subtitles into 9:16 MP4 +- ✅ MoviePy composites clips + TTS + ambient audio + subtitles into 9:16 MP4 - ✅ Subtitles positioned at y=1700 (bottom of frame), font size 80 - ✅ Job status updated to `ready` or `failed` accordingly - ✅ DNF-style progress display with: diff --git a/README.md b/README.md old mode 100755 new mode 100644 diff --git a/backend/.env.example b/backend/.env.example old mode 100755 new mode 100644 diff --git a/backend/.gitignore b/backend/.gitignore old mode 100755 new mode 100644 index eb79a35..ed26d05 --- a/backend/.gitignore +++ b/backend/.gitignore @@ -19,7 +19,6 @@ output/reels/*.mp4 output/temp/*.mp4 output/temp/*.mp3 output/temp/*.wav -output/render_profile_calibration.json # Keep directory structure with .gitkeep files !output/audio/.gitkeep diff --git a/backend/README.md b/backend/README.md old mode 100755 new mode 100644 index 3c19cf2..502adc4 --- a/backend/README.md +++ b/backend/README.md @@ -52,46 +52,6 @@ export PEXELS_API_KEY='...' python reel_generator.py ``` -### Render presets - -`normal` is the default and is intended for background generation while the -browser and desktop remain responsive. It renders at 720x1280/24fps with the -balanced encoder setting. On Linux, it also runs in a hardware-aware CPU cgroup -and at a lower scheduling priority. A four-physical-core machine receives a -150% aggregate budget (up to one and a half CPUs), which leaves substantial -desktop capacity without throttling the compositor and encoder into a long -serial export. - -Normal intentionally uses one MovieLite worker. Its aggregate CPU quota and lower -scheduling priority keep it background-friendly, while Linux is free to place -that work on any available CPU rather than pinning it to CPU 0. On the -reference laptop, two- and three-worker exports were slower under moderate -aggregate CPU quotas and increased peak memory substantially because MovieLite -renders and merges one encoded part per worker. - -`fast` starts with up to two workers when current RAM headroom permits it. -MovieLite renders one encoded part per process and then merges them, so the -calibration reserves a physical core for the desktop and only accepts -additional workers when they make a meaningful measured improvement. Linux -places the selected workers naturally; no preset pins them to particular CPU -IDs. -It keeps the same 720x1280/24fps output as Normal but uses MovieLite's faster -x264 profile and up to two measured-useful encoder threads. That is a modest -compression trade-off for faster generation, not a resolution or frame-rate -reduction. -On its first Fast reel, MindStream benchmarks safe worker counts against a -short sample of the downloaded footage, caches the fastest meaningful result, -and reuses it until the hardware or output profile changes. - -```bash -./test.sh --preset normal -./test.sh --preset fast -./test.sh --preset fast --recalibrate-presets - -# When using the Express server, Normal remains the default. -MINDSTREAM_REEL_PRESET=fast npm start -``` - **What happens:** 1. Loads `data/sample_emotion_result.json` (frustrated emotion) 2. Generates philosophical script with Gemini diff --git a/backend/assets/audio/.gitkeep b/backend/assets/audio/.gitkeep old mode 100755 new mode 100644 diff --git a/backend/assets/audio/README.md b/backend/assets/audio/README.md old mode 100755 new mode 100644 diff --git a/backend/assets/audio/distracted.mp3 b/backend/assets/audio/distracted.mp3 old mode 100755 new mode 100644 diff --git a/backend/assets/audio/fatigued.mp3 b/backend/assets/audio/fatigued.mp3 old mode 100755 new mode 100644 diff --git a/backend/assets/backgrounds/.gitkeep b/backend/assets/backgrounds/.gitkeep old mode 100755 new mode 100644 diff --git a/backend/benchmark_render_presets.py b/backend/benchmark_render_presets.py deleted file mode 100755 index 6c793b0..0000000 --- a/backend/benchmark_render_presets.py +++ /dev/null @@ -1,199 +0,0 @@ -"""Benchmark MovieLite worker/quota combinations with a fixed local export. - -Run from backend after selecting an existing video and narration file: - venv/bin/python benchmark_render_presets.py --video path/to/clip.mp4 --audio path/to/audio.mp3 -""" - -import argparse -import contextlib -import json -import os -import shutil -import subprocess -import sys -import time -from pathlib import Path -from typing import Any, Dict, List, Optional, Tuple - - -PROFILES: Tuple[Tuple[str, int, int], ...] = ( - ("one-worker-50", 1, 50), - ("two-workers-50", 2, 50), - ("two-workers-60", 2, 60), - ("two-workers-75", 2, 75), - ("three-workers-75", 3, 75), -) - - -def _read_cpu_usage(cgroup_path: Path) -> int: - values = dict(line.split() for line in (cgroup_path / "cpu.stat").read_text().splitlines()) - return int(values["usage_usec"]) - - -def _cgroup_path(unit: str) -> Optional[Path]: - result = subprocess.run( - ["systemctl", "--user", "show", unit, "--property=ControlGroup", "--value"], - capture_output=True, - text=True, - check=False, - ) - group = result.stdout.strip() - return Path("/sys/fs/cgroup") / group.lstrip("/") if group else None - - -def _unit_is_active(unit: str) -> bool: - return ( - subprocess.run( - ["systemctl", "--user", "is-active", "--quiet", unit], check=False - ).returncode - == 0 - ) - - -def _run_child(args: argparse.Namespace) -> int: - os.chdir(Path(__file__).parent) - from reel_generator import ReelGenerator - - generator = ReelGenerator(preset=" ") - generator._spinner = lambda _: contextlib.nullcontext() - # Benchmark Linux scheduling without affinity: let the kernel place workers. - generator.preset_cfg["writer_processes"] = args.workers - generator.preset_cfg["cpu_affinity"] = () - generator.composite_reel( - video_paths=[args.video], - tts_path=args.audio, - output_path=args.output, - subtitle_list=["A short representative export", "for preset comparison"], - tts_duration=args.duration, - ) - return 0 - - -def _benchmark_profile( - name: str, - workers: int, - quota: int, - args: argparse.Namespace, - output_dir: Path, -) -> Dict[str, Any]: - unit = f"mindstream-benchmark-{os.getpid()}-{name}.service" - output_path = output_dir / f"{name}.mp4" - command = [ - "systemd-run", - "--user", - "--quiet", - "--no-block", - "--unit", - unit, - "-p", - f"CPUQuota={quota}%", - sys.executable, - str(Path(__file__).resolve()), - "--child", - "--workers", - str(workers), - "--duration", - str(args.duration), - "--video", - str(Path(args.video).resolve()), - "--audio", - str(Path(args.audio).resolve()), - "--output", - str(output_path), - ] - subprocess.run(command, check=True) - - cgroup = None - for _ in range(50): - cgroup = _cgroup_path(unit) - if cgroup and (cgroup / "cpu.stat").exists(): - break - time.sleep(0.05) - if not cgroup or not (cgroup / "cpu.stat").exists(): - raise RuntimeError(f"Could not inspect benchmark cgroup for {name}") - - start_time = time.monotonic() - last_time = start_time - start_usage = _read_cpu_usage(cgroup) - last_usage = start_usage - peak_cpu_percent = 0.0 - peak_memory_bytes = 0 - - while _unit_is_active(unit): - time.sleep(0.1) - now = time.monotonic() - try: - usage = _read_cpu_usage(cgroup) - except FileNotFoundError: - # The scope exited between the active-state check and this sample. - break - elapsed = now - last_time - if elapsed > 0: - peak_cpu_percent = max( - peak_cpu_percent, (usage - last_usage) / 1_000_000 / elapsed * 100 - ) - memory_peak = cgroup / "memory.peak" - if memory_peak.exists(): - peak_memory_bytes = max(peak_memory_bytes, int(memory_peak.read_text())) - last_time = now - last_usage = usage - - total_seconds = time.monotonic() - start_time - # A transient scope may be collected immediately after it stops. The final - # active-state sample is therefore the authoritative value to retain. - final_usage = last_usage - memory_peak = cgroup / "memory.peak" - if memory_peak.exists(): - peak_memory_bytes = max(peak_memory_bytes, int(memory_peak.read_text())) - - if not output_path.exists() or output_path.stat().st_size == 0: - raise RuntimeError(f"{name} did not produce an output file") - - return { - "profile": name, - "workers": workers, - "quota_percent": quota, - "wall_seconds": round(total_seconds, 2), - "average_cpu_percent": round( - (final_usage - start_usage) / 1_000_000 / total_seconds * 100, 1 - ), - "peak_cpu_percent": round(peak_cpu_percent, 1), - "peak_memory_mib": round(peak_memory_bytes / 1024 / 1024, 1), - "output": str(output_path), - } - - -def _run_benchmark(args: argparse.Namespace) -> int: - if not shutil.which("systemd-run"): - raise RuntimeError("This benchmark requires systemd-run and cgroup v2") - - output_dir = Path(args.output_dir).resolve() - output_dir.mkdir(parents=True, exist_ok=True) - results: List[Dict[str, Any]] = [] - - for name, workers, quota in PROFILES: - print(f"Benchmarking {name}: {workers} worker(s), {quota}% CPU quota...") - results.append(_benchmark_profile(name, workers, quota, args, output_dir)) - - print(json.dumps(results, indent=2)) - return 0 - - -def main() -> int: - parser = argparse.ArgumentParser(description="Benchmark MovieLite render profiles") - parser.add_argument("--video", required=True) - parser.add_argument("--audio", required=True) - parser.add_argument("--output-dir", default="/tmp/mindstream-render-benchmarks") - parser.add_argument("--duration", type=float, default=8.0) - parser.add_argument("--workers", type=int) - parser.add_argument("--output") - parser.add_argument("--child", action="store_true") - args = parser.parse_args() - - if args.child: - return _run_child(args) - return _run_benchmark(args) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/backend/data/sample_emotion_result.json b/backend/data/sample_emotion_result.json old mode 100755 new mode 100644 index d92a518..b7f31fc --- a/backend/data/sample_emotion_result.json +++ b/backend/data/sample_emotion_result.json @@ -1,7 +1,7 @@ { "job_id": "sample-job-001", "emotion": { - "label": "frustrated", + "label": "neutral", "confidence": 0.88 }, "context": { @@ -17,7 +17,7 @@ "recent_switches": 5 }, "current_tab_category": "productivity", - "user_name": "Binary", + "user_name": "Prashant", "local_weather": "Cloudy, 20°C", "time_of_day": "Evening", "session_duration_minutes": 5, diff --git a/backend/movielite_docs/advanced.md b/backend/movielite_docs/advanced.md deleted file mode 100755 index e760c85..0000000 --- a/backend/movielite_docs/advanced.md +++ /dev/null @@ -1,619 +0,0 @@ -# Advanced Usage Guide - -This guide covers advanced topics for using movielite effectively. - -## Table of Contents - -- [Custom Effect Development](#custom-effect-development) -- [Performance Optimization](#performance-optimization) -- [Memory Management](#memory-management) -- [Advanced Audio Processing](#advanced-audio-processing) -- [Complex Compositions](#complex-compositions) -- [Masking and Compositing](#masking-and-compositing) -- [Integration with Other Libraries](#integration-with-other-libraries) - ---- - -## Custom Effect Development - -### Creating Custom Visual Effects - -You can create custom visual effects by subclassing `GraphicEffect`: - -```python -from movielite.vfx.base import GraphicEffect -from movielite.core import GraphicClip -import cv2 -import numpy as np - -class CustomVignetteEffect(GraphicEffect): - """Custom vignette effect with adjustable parameters.""" - - def __init__(self, intensity: float = 0.7, color: tuple = (0, 0, 0)): - """ - Args: - intensity: Darkness intensity (0.0 to 1.0) - color: RGB color tuple for vignette (default: black) - """ - self.intensity = intensity - self.color = color - - def apply(self, clip: GraphicClip) -> None: - """Apply the vignette effect to the clip.""" - - # Cache the vignette mask - cache = {} - - def vignette_transform(frame: np.ndarray, t: float) -> np.ndarray: - h, w = frame.shape[:2] - cache_key = (w, h) - - if cache_key not in cache: - # Create radial gradient - center_x, center_y = w // 2, h // 2 - Y, X = np.ogrid[:h, :w] - dist = np.sqrt(((X - center_x) / w) ** 2 + ((Y - center_y) / h) ** 2) - - # Create mask - mask = np.clip(1.0 - dist, 0, 1) - mask = 1.0 - (1.0 - mask) * self.intensity - mask = np.stack([mask] * 3, axis=2) - - cache[cache_key] = mask - - # Apply vignette - result = frame.astype(np.float32) * cache[cache_key] - return result.astype(np.uint8) - - clip.add_transform(vignette_transform) - -# Usage -from movielite import VideoClip, VideoWriter - -clip = VideoClip("video.mp4") -clip.add_effect(CustomVignetteEffect(intensity=0.8, color=(20, 20, 30))) - -writer = VideoWriter("output.mp4", fps=clip.fps, size=clip.size) -writer.add_clip(clip) -writer.write() -clip.close() -``` - -### Creating Custom Audio Effects - -```python -from movielite.afx.base import AudioEffect -from movielite.audio import AudioClip -import numpy as np - -class EchoEffect(AudioEffect): - """Add echo effect to audio.""" - - def __init__(self, delay: float = 0.3, decay: float = 0.5): - """ - Args: - delay: Echo delay in seconds - decay: Echo decay factor (0.0 to 1.0) - """ - self.delay = delay - self.decay = decay - - def apply(self, clip: AudioClip) -> None: - """Apply echo effect to the audio clip.""" - - def echo_transform(samples: np.ndarray, t: float, sr: int) -> np.ndarray: - # Calculate delay in samples - delay_samples = int(self.delay * sr) - - # Create echo buffer - result = samples.copy() - - # Add delayed and decayed samples - if len(samples) > delay_samples: - result[delay_samples:] += samples[:-delay_samples] * self.decay - - # Normalize to prevent clipping - max_val = np.abs(result).max() - if max_val > 1.0: - result = result / max_val - - return result - - clip.add_transform(echo_transform) - -# Usage -from movielite import AudioClip - -audio = AudioClip("audio.mp3") -audio.add_effect(EchoEffect(delay=0.5, decay=0.4)) -``` - -### Creating Custom Transitions - -```python -from movielite.vtx.base import Transition -from movielite.core import GraphicClip - -class SlideTransition(Transition): - """Slide transition between two clips.""" - - def __init__(self, duration: float, direction: str = "left"): - """ - Args: - duration: Duration of the transition - direction: Slide direction ("left", "right", "up", "down") - """ - self.duration = duration - self.direction = direction - - def apply(self, clip1: GraphicClip, clip2: GraphicClip) -> None: - """Apply slide transition.""" - - # Validate overlap - self._validate_clips_have_overlap(clip1, clip2, self.duration) - - # Get overlap region - overlap_start = clip2.start - overlap_end = min(clip1.end, clip2.start + self.duration) - - # Modify clip positions - original_pos1 = clip1._position - original_pos2 = clip2._position - - def position_func_clip1(t): - base_pos = original_pos1(t) - - if clip1.start <= t < overlap_end: - # During transition, slide out - progress = (t - overlap_start) / self.duration - - if self.direction == "left": - offset_x = int(-clip1.size[0] * progress) - return (base_pos[0] + offset_x, base_pos[1]) - elif self.direction == "right": - offset_x = int(clip1.size[0] * progress) - return (base_pos[0] + offset_x, base_pos[1]) - - return base_pos - - def position_func_clip2(t): - base_pos = original_pos2(t) - - if overlap_start <= t < overlap_end: - # During transition, slide in - progress = (t - overlap_start) / self.duration - - if self.direction == "left": - offset_x = int(clip2.size[0] * (1 - progress)) - return (base_pos[0] + offset_x, base_pos[1]) - elif self.direction == "right": - offset_x = int(-clip2.size[0] * (1 - progress)) - return (base_pos[0] + offset_x, base_pos[1]) - - return base_pos - - clip1.set_position(position_func_clip1) - clip2.set_position(position_func_clip2) -``` - ---- - -## Performance Optimization - -### Using Multiprocessing - -For long videos, use multiprocessing to parallelize rendering: - -```python -from movielite import VideoClip, VideoWriter, VideoQuality -import multiprocessing - -clip = VideoClip("long_video.mp4") - -writer = VideoWriter("output.mp4", fps=clip.fps, size=clip.size) -writer.add_clip(clip) - -# Use all available CPU cores -num_processes = multiprocessing.cpu_count() -writer.write(processes=num_processes, video_quality=VideoQuality.HIGH) - -clip.close() -``` - -### Caching Expensive Computations - -For static transformations, cache results: - -```python -import cv2 -import numpy as np - -class CachedGrayscaleEffect: - def __init__(self): - self.cache = {} - - def __call__(self, frame: np.ndarray, t: float) -> np.ndarray: - # Use frame memory address as cache key - frame_id = id(frame) - - if frame_id not in self.cache: - gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) - self.cache[frame_id] = cv2.cvtColor(gray, cv2.COLOR_GRAY2BGR) - - # Warning: you should have a maximum amount of frames stored in your cache - return self.cache[frame_id] - -clip.add_transform(CachedGrayscaleEffect()) -``` - ---- - -## Advanced Audio Processing - -### Complex Audio Mixing - -```python -from movielite import AudioClip, VideoClip, VideoWriter, afx -import numpy as np - -# Load multiple audio tracks -video = VideoClip("video.mp4") -music = AudioClip("background_music.mp3", start=0, volume=0.3) -narration = AudioClip("voiceover.mp3", start=5, volume=1.0) -sfx1 = AudioClip("explosion.wav", start=10, volume=0.8) -sfx2 = AudioClip("applause.wav", start=15, volume=0.6) - -# Apply effects to each track -music.add_effect(afx.FadeIn(2.0)).add_effect(afx.FadeOut(3.0)) -narration.add_effect(afx.FadeIn(0.5)) - -# Duck music during narration (custom volume curve) -def ducking_curve(t): - if 5 <= t < 25: # Narration period - return 0.2 # Reduce music to 20% - return 0.5 # Normal music at 50% - -music.set_volume_curve(ducking_curve) - -# Mix all tracks -writer = VideoWriter("output.mp4", fps=video.fps, size=video.size) -writer.add_clip(video) -writer.add_clip(music) -writer.add_clip(narration) -writer.add_clip(sfx1) -writer.add_clip(sfx2) -writer.write() - -video.close() -``` - -### Custom Audio Filters - -```python -import numpy as np -from scipy import signal - -def apply_lowpass_filter(cutoff_freq: float = 1000, order: int = 5): - """Create a lowpass filter transform.""" - - def lowpass_transform(samples: np.ndarray, t: float, sr: int) -> np.ndarray: - # Design Butterworth lowpass filter - nyquist = sr / 2 - normal_cutoff = cutoff_freq / nyquist - b, a = signal.butter(order, normal_cutoff, btype='low', analog=False) - - # Apply filter to each channel - filtered = np.zeros_like(samples) - for ch in range(samples.shape[1]): - filtered[:, ch] = signal.lfilter(b, a, samples[:, ch]) - - return filtered - - return lowpass_transform - -# Usage -audio = AudioClip("audio.mp3") -audio.add_transform(apply_lowpass_filter(cutoff_freq=2000, order=4)) -``` - ---- - -## Complex Compositions - -### Picture-in-Picture Effect - -```python -from movielite import VideoClip, VideoWriter, vfx - -# Main video -main_video = VideoClip("main.mp4", start=0) - -# Small overlay video -pip_video = VideoClip("overlay.mp4", start=0) -pip_video.set_size(width=320, height=180) # Resize to small size -pip_video.set_position((main_video.size[0] - 340, 20)) # Top-right corner -pip_video.add_effect(vfx.FadeIn(0.5)) -pip_video.add_effect(vfx.FadeOut(0.5)) - -# Compose -writer = VideoWriter("output.mp4", fps=30, size=main_video.size) -writer.add_clip(main_video) -writer.add_clip(pip_video) -writer.write() - -main_video.close() -pip_video.close() -``` - -### Split Screen - -```python -from movielite import VideoClip, VideoWriter, ImageClip - -# Create black background -bg = ImageClip.from_color((0, 0, 0), size=(1920, 1080), duration=10) - -# Left video -left_video = VideoClip("left.mp4") -left_video.set_size(width=960, height=1080) -left_video.set_position((0, 0)) - -# Right video -right_video = VideoClip("right.mp4") -right_video.set_size(width=960, height=1080) -right_video.set_position((960, 0)) - -# Compose -writer = VideoWriter("split_screen.mp4", fps=30, size=(1920, 1080)) -writer.add_clip(bg) -writer.add_clip(left_video) -writer.add_clip(right_video) -writer.write() - -left_video.close() -right_video.close() -``` - -### Animated Text - -```python -from movielite import VideoClip, TextClip, VideoWriter -from pictex import Canvas -import math - -video = VideoClip("background.mp4") - -canvas = Canvas().font_size(80).color("white").background_color("transparent") -text = TextClip("Animated Title", start=0, duration=5, canvas=canvas) - -# Animate position (bounce effect) -def animated_position(t): - # Bounce in from top - if t < 1.0: - y = int(-100 + (video.size[1] // 2) * (1 - math.cos(t * math.pi))) - else: - y = video.size[1] // 2 - - x = video.size[0] // 2 - text.size[0] // 2 - return (x, y) - -text.set_position(animated_position) - -# Animate opacity -text.set_opacity(lambda t: min(1.0, t / 0.5)) - -writer = VideoWriter("output.mp4", fps=video.fps, size=video.size) -writer.add_clip(video) -writer.add_clip(text) -writer.write() - -video.close() -``` - ---- - -## Integration with Other Libraries - -### Using OpenCV Filters - -```python -import cv2 -import numpy as np -from movielite import VideoClip, VideoWriter - -clip = VideoClip("video.mp4") - -def apply_bilateral_filter(frame: np.ndarray, t: float) -> np.ndarray: - """Apply bilateral filter for edge-preserving smoothing.""" - return cv2.bilateralFilter(frame, d=9, sigmaColor=75, sigmaSpace=75) - -clip.add_transform(apply_bilateral_filter) - -writer = VideoWriter("filtered.mp4", fps=clip.fps, size=clip.size) -writer.add_clip(clip) -writer.write() -clip.close() -``` - -### Using NumPy for Advanced Effects - -```python -import numpy as np -from movielite import VideoClip, VideoWriter - -clip = VideoClip("video.mp4") - -def color_temperature_shift(frame: np.ndarray, t: float) -> np.ndarray: - """Shift color temperature (warmer).""" - result = frame.copy().astype(np.float32) - - # Increase red channel - result[:, :, 2] = np.clip(result[:, :, 2] * 1.1, 0, 255) - - # Decrease blue channel - result[:, :, 0] = np.clip(result[:, :, 0] * 0.9, 0, 255) - - return result.astype(np.uint8) - -clip.add_transform(color_temperature_shift) - -writer = VideoWriter("warm.mp4", fps=clip.fps, size=clip.size) -writer.add_clip(clip) -writer.write() -clip.close() -``` - -### Using Pillow for Image Processing - -```python -from PIL import Image, ImageEnhance -import numpy as np -from movielite import VideoClip, VideoWriter -import cv2 - -clip = VideoClip("video.mp4") - -def enhance_colors(frame: np.ndarray, t: float) -> np.ndarray: - """Use Pillow to enhance colors.""" - # Convert BGR to RGB - rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) - - # Convert to PIL Image - pil_image = Image.fromarray(rgb_frame) - - # Enhance color - enhancer = ImageEnhance.Color(pil_image) - enhanced = enhancer.enhance(1.5) # 1.5x color saturation - - # Convert back to numpy array (BGR) - rgb_array = np.array(enhanced) - return cv2.cvtColor(rgb_array, cv2.COLOR_RGB2BGR) - -clip.add_transform(enhance_colors) - -writer = VideoWriter("enhanced.mp4", fps=clip.fps, size=clip.size) -writer.add_clip(clip) -writer.write() -clip.close() -``` - -### Using scipy for Audio Processing - -```python -from scipy.signal import butter, filtfilt -import numpy as np -from movielite import AudioClip - -def bandpass_filter(lowcut: float, highcut: float, order: int = 5): - """Create a bandpass filter for audio.""" - - def filter_transform(samples: np.ndarray, t: float, sr: int) -> np.ndarray: - nyquist = sr / 2 - low = lowcut / nyquist - high = highcut / nyquist - - b, a = butter(order, [low, high], btype='band') - - filtered = np.zeros_like(samples) - for ch in range(samples.shape[1]): - filtered[:, ch] = filtfilt(b, a, samples[:, ch]) - - return filtered - - return filter_transform - -audio = AudioClip("audio.mp3") -audio.add_transform(bandpass_filter(lowcut=200, highcut=3000)) -``` - ---- - -## Masking and Compositing - -Masking is a powerful feature for creating advanced compositing effects where one clip's visibility is controlled by another clip's luminance values. - -### Basic Masking - -```python -from movielite import VideoClip, ImageClip, VideoWriter - -video = VideoClip("waves.mp4") -mask = ImageClip("mask.png", duration=video.duration) - -# Apply mask - white areas of mask = visible, black = transparent -video.set_mask(mask) - -writer = VideoWriter("masked.mp4", fps=video.fps, size=video.size) -writer.add_clip(video) -writer.write() - -video.close() -``` - -### Text Masking - -```python -from movielite import VideoClip, TextClip, VideoWriter -from pictex import Canvas - -video = VideoClip("colorful.mp4", duration=5) - -# Create text as mask -canvas = Canvas().font_size(200).color("white").background_color("transparent") -text = TextClip("MASKED", duration=5, canvas=canvas) -text.set_position((video.size[0] // 2 - text.size[0] // 2, - video.size[1] // 2 - text.size[1] // 2)) - -# Video only visible through text -video.set_mask(text) - -writer = VideoWriter("text_masked.mp4", fps=video.fps, size=video.size) -writer.add_clip(video) -writer.write() - -video.close() -``` - -### Animated Masks - -```python -import numpy as np -from movielite import VideoClip, TextClip, VideoWriter -from pictex import Canvas - -video = VideoClip("waves.mp4", duration=10) - -# Create animated text mask -canvas = Canvas().font_size(200).color("white").background_color("transparent") -text = TextClip("Hello World!", duration=10, canvas=canvas) - -# Animate position (wave motion) -text.set_position(lambda t: ( - 960 - text.size[0] // 2, - 500 + int(20 * np.sin(2 * np.pi * (t / 10.0))) -)) - -# Animate scale (grow) -text.set_scale(lambda t: 1.0 + 0.4 * (t / 10.0)) - -video.set_mask(text) -video.set_size(1920, 1080) - -writer = VideoWriter("animated_mask.mp4", fps=30, size=(1920, 1080)) -writer.add_clip(video) -writer.write() - -video.close() -``` - ---- - -## Best Practices - -1. **Profile your effects**: Time-intensive effects should be optimized or cached. - -2. **Use appropriate data types**: Keep frames as uint8 until final processing to save memory. - -3. **Leverage multiprocessing**: For videos longer than 60 seconds, use multiple processes if is possible. - -4. **Cache static transformations**: If a transformation doesn't depend on time, cache its result. diff --git a/backend/movielite_docs/api.md b/backend/movielite_docs/api.md deleted file mode 100755 index e0b9343..0000000 --- a/backend/movielite_docs/api.md +++ /dev/null @@ -1,1077 +0,0 @@ -# API Reference - -Complete API reference for movielite library. - -## Table of Contents - -- [Core Classes](#core-classes) - - [MediaClip](#mediaclip) - - [GraphicClip](#graphicclip) - - [VideoClip](#videoclip) - - [AlphaVideoClip](#alphavideoclip) - - [ImageClip](#imageclip) - - [TextClip](#textclip) - - [AudioClip](#audioclip) - - [VideoWriter](#videowriter) -- [Visual Effects (vfx)](#visual-effects-vfx) - - [Fade Effects](#fade-effects) - - [Blur Effects](#blur-effects) - - [Color Effects](#color-effects) - - [Zoom Effects](#zoom-effects) - - [Glitch Effects](#glitch-effects) - - [Other Effects](#other-effects) -- [Audio Effects (afx)](#audio-effects-afx) -- [Transitions (vtx)](#transitions-vtx) -- [Enumerations](#enumerations) -- [Utilities](#utilities) - ---- - -## Core Classes - -### MediaClip - -Base class for all media clips (visual and audio). - -**Constructor:** -```python -MediaClip(start: float, duration: float) -``` - -**Parameters:** -- `start` (float): Start time in the composition (seconds) -- `duration` (float): Duration of the clip (seconds) - -**Properties:** -- `start` (float): Start time in seconds -- `duration` (float): Duration in seconds -- `end` (float): End time in seconds (start + duration) - -**Methods:** - -#### set_start(start: float) -> Self -Set the start time of this clip in the composition. - -**Parameters:** -- `start` (float): Start time in seconds (must be >= 0) - -**Returns:** Self for chaining - -**Raises:** ValueError if start is negative - ---- - -#### set_duration(duration: float) -> Self -Set the duration of this clip. - -**Parameters:** -- `duration` (float): Duration in seconds (must be > 0) - -**Returns:** Self for chaining - -**Raises:** ValueError if duration is not positive - ---- - -#### set_end(end: float) -> Self -Set the end time of this clip in the composition. Adjusts duration to match. - -**Parameters:** -- `end` (float): End time in seconds (must be > start) - -**Returns:** Self for chaining - -**Raises:** ValueError if end is not greater than start - ---- - -### GraphicClip - -Base class for all visual/graphic clips (video, image, text). - -Inherits from [MediaClip](#mediaclip). - -**Constructor:** -```python -GraphicClip(start: float, duration: float) -``` - -**Properties:** -- `position` (Callable[[float], Tuple[int, int]]): Position function -- `opacity` (Callable[[float], float]): Opacity function -- `scale` (Callable[[float], float]): Scale function -- `size` (Tuple[int, int]): Size of the clip (width, height) -- `has_any_transform` (bool): Whether any transformations are applied - -**Methods:** - -#### set_position(value: Union[Callable[[float], Tuple[int, int]], Tuple[int, int]]) -> Self -Set the position of the clip. - -**Parameters:** -- `value`: Either a tuple (x, y) or a function that takes time and returns (x, y) - -**Returns:** Self for chaining - -**Example:** -```python -# Static position -clip.set_position((100, 100)) - -# Animated position -clip.set_position(lambda t: (int(100 + t * 50), 100)) -``` - ---- - -#### set_opacity(value: Union[Callable[[float], float], float]) -> Self -Set the opacity of the clip. - -**Parameters:** -- `value`: Either a float (0-1) or a function that takes time and returns opacity - -**Returns:** Self for chaining - -**Example:** -```python -# Static opacity -clip.set_opacity(0.5) - -# Animated opacity -clip.set_opacity(lambda t: min(1.0, t / 2.0)) -``` - ---- - -#### set_scale(value: Union[Callable[[float], float], float]) -> Self -Set the scale of the clip. - -**Parameters:** -- `value`: Either a float or a function that takes time and returns scale - -**Returns:** Self for chaining - -**Example:** -```python -# Static scale -clip.set_scale(0.5) - -# Animated scale -clip.set_scale(lambda t: 1.0 + t * 0.1) -``` - ---- - -#### set_size(width: Optional[int] = None, height: Optional[int] = None) -> Self -Set the size of the clip, maintaining aspect ratio if only one dimension is provided. - -**Parameters:** -- `width` (Optional[int]): Target width -- `height` (Optional[int]): Target height - -**Returns:** Self for chaining - -**Raises:** ValueError if both width and height are None or invalid - -**Example:** -```python -# Explicit size -clip.set_size(width=1280, height=720) - -# Maintain aspect ratio (width only) -clip.set_size(width=1280) - -# Maintain aspect ratio (height only) -clip.set_size(height=720) -``` - ---- - -#### set_mask(mask: GraphicClip) -> Self -Set a mask for this clip. The mask determines which pixels are visible. - -The mask is converted to grayscale, where white (255) means fully visible and black (0) means fully transparent. Gray values create partial transparency. - -**Parameters:** -- `mask` (GraphicClip): A GraphicClip to use as mask - -**Returns:** Self for chaining - -**Examples:** - -Simple image mask: -```python -image = ImageClip("photo.png") -mask = ImageClip("mask.png") -image.set_mask(mask) -``` - -Animated text mask: -```python -import numpy as np -from movielite import VideoClip, TextClip -from pictex import Canvas - -# Video to be masked -video = VideoClip("waves.mp4", start=0, duration=10) - -# Create animated text mask -canvas = Canvas().font_size(200).color("white").background_color("transparent") -text = TextClip("Hello World!", start=0, duration=10, canvas=canvas) - -# Animate the mask -text.set_position(lambda t: ( - 960 - text.size[0] // 2, - 500 + int(20 * np.sin(2 * np.pi * (t / text.duration))) -)) -text.set_scale(lambda t: 1.0 + 0.4 * (t / text.duration)) - -# Apply mask - video only visible through text shape -video.set_mask(text) -``` - ---- - -#### add_transform(callback: Callable[[np.ndarray, float], np.ndarray]) -> Self -Apply a custom transformation to each frame at render time. - -**Parameters:** -- `callback` (Callable): Function that takes (frame, time) and returns transformed frame - -**Returns:** Self for chaining - -**Note:** The frame must be in BGR or BGRA format and uint8 type. - -**Example:** -```python -def make_grayscale(frame, t): - gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) - return cv2.cvtColor(gray, cv2.COLOR_GRAY2BGR) - -clip.add_transform(make_grayscale) -``` - ---- - -#### add_effect(effect: GraphicEffect) -> Self -Apply a visual effect to this clip. - -**Parameters:** -- `effect` (GraphicEffect): A GraphicEffect instance to apply - -**Returns:** Self for chaining - -**Example:** -```python -from movielite import vfx -clip.add_effect(vfx.FadeIn(2.0)).add_effect(vfx.FadeOut(1.5)) -``` - ---- - -#### add_transition(next_clip: GraphicClip, transition: Transition) -> Self -Apply a transition effect between this clip and another clip. - -**Parameters:** -- `next_clip` (GraphicClip): The clip to transition to -- `transition` (Transition): A Transition instance to apply - -**Returns:** Self for chaining - -**Example:** -```python -from movielite import vtx -clip1.add_transition(clip2, vtx.CrossFade(0.5)) -``` - ---- - -#### get_frame(t_rel: float) -> np.ndarray -Get the frame at a relative time within the clip (abstract method). - -**Parameters:** -- `t_rel` (float): Relative time within the clip (0 to duration) - -**Returns:** Frame as numpy array (BGR or BGRA, uint8) - ---- - -#### render(bg: np.ndarray, t_global: float) -> np.ndarray -Render this clip onto a background at a given global time. - -**Parameters:** -- `bg` (np.ndarray): Background frame (BGR or BGRA format, float32 type) -- `t_global` (float): Global time in seconds - -**Returns:** Background with this clip rendered on top - -**Note:** Modifies the background in-place. - ---- - -#### close() -Closes the graphic clip and releases any resources. - ---- - -### VideoClip - -A video clip that loads and processes frames in BGR format (no alpha channel). - -Inherits from [GraphicClip](#graphicclip). - -**Constructor:** -```python -VideoClip( - path: str, - start: float = 0, - duration: Optional[float] = None, - offset: float = 0 -) -``` - -**Parameters:** -- `path` (str): Path to the video file -- `start` (float): Start time in the composition (seconds) -- `duration` (Optional[float]): Duration to use from the video (if None, uses full duration) -- `offset` (float): Start offset within the video file (seconds) - -**Properties:** -- `fps` (float): Frames per second of the video -- `audio` (AudioClip): Audio track associated with this video - -**Methods:** - -#### subclip(start: float, end: float) -> VideoClip -Extract a subclip from this video. - -**Parameters:** -- `start` (float): Start time within this clip (seconds) -- `end` (float): End time within this clip (seconds) - -**Returns:** New VideoClip instance - -**Raises:** ValueError if range is invalid - -**Example:** -```python -clip = VideoClip("video.mp4") -segment = clip.subclip(10, 20) # Extract seconds 10-20 -``` - ---- - -#### loop(enabled: bool = True) -> Self -Enable or disable looping for this video clip. - -**Parameters:** -- `enabled` (bool): Whether to enable looping - -**Returns:** Self for chaining - -**Example:** -```python -clip.loop(True) # Video will restart when it reaches the end -``` - ---- - -#### set_offset(offset: float) -> Self -Set the offset within the source video file. - -**Parameters:** -- `offset` (float): Offset in seconds - -**Returns:** Self for chaining - ---- - -### AlphaVideoClip - -A video clip that loads and processes frames in BGRA format (with alpha channel). - -Inherits from [VideoClip](#videoclip). - -**Note:** AlphaVideoClip has a performance penalty (~33% more memory per frame) compared to VideoClip. Only use this when you need transparency support. - -**Constructor:** -```python -AlphaVideoClip( - path: str, - start: float = 0, - duration: Optional[float] = None, - offset: float = 0 -) -``` - -**Parameters:** Same as [VideoClip](#videoclip) - ---- - -### ImageClip - -An image clip that displays a static image for a given duration. - -Inherits from [GraphicClip](#graphicclip). - -**Constructor:** -```python -ImageClip( - source: Union[str, np.ndarray], - start: float = 0, - duration: float = 5.0 -) -``` - -**Parameters:** -- `source` (Union[str, np.ndarray]): Either a file path or a numpy array (RGB/RGBA) -- `start` (float): Start time in the composition (seconds) -- `duration` (float): How long to display the image (seconds) - -**Class Methods:** - -#### from_color(color: tuple, size: tuple, start: float = 0, duration: float = 5.0) -> ImageClip -Create a solid color image clip. - -**Parameters:** -- `color` (tuple): RGB or RGBA tuple (0-255) -- `size` (tuple): (width, height) -- `start` (float): Start time in seconds -- `duration` (float): Duration in seconds - -**Returns:** ImageClip instance - -**Example:** -```python -# Create a red background -red_bg = ImageClip.from_color( - color=(255, 0, 0), - size=(1920, 1080), - duration=10 -) -``` - ---- - -### TextClip - -A text clip that renders text using the pictex library. - -Inherits from [GraphicClip](#graphicclip). - -**Constructor:** -```python -TextClip( - text: str, - start: float = 0, - duration: float = 5.0, - canvas: Optional[Canvas] = None -) -``` - -**Parameters:** -- `text` (str): The text to render -- `start` (float): Start time in the composition (seconds) -- `duration` (float): How long to display the text (seconds) -- `canvas` (Optional[Canvas]): A pictex Canvas instance with styling configured - -**Properties:** -- `text` (str): The text content - -**Example:** -```python -from pictex import Canvas, LinearGradient, Shadow - -canvas = ( - Canvas() - .font_family("Arial") - .font_size(60) - .color("white") - .padding(20) - .background_color(LinearGradient(["#2C3E50", "#FD746C"])) - .border_radius(10) -) - -text = TextClip("Hello World", duration=3, canvas=canvas) -``` - ---- - -### AudioClip - -An audio clip that can be overlaid on video. - -Inherits from [MediaClip](#mediaclip). - -**Constructor:** -```python -AudioClip( - path: str, - start: float = 0, - duration: Optional[float] = None, - volume: float = 1.0, - offset: float = 0 -) -``` - -**Parameters:** -- `path` (str): Path to the audio file -- `start` (float): Start time in the composition (seconds) -- `duration` (Optional[float]): Duration to use (if None, uses full audio duration) -- `volume` (float): Volume multiplier (0.0 to 1.0+) -- `offset` (float): Start offset within the audio file (seconds) - -**Properties:** -- `path` (str): Path to the audio file -- `volume` (float): Volume multiplier -- `offset` (float): Offset within the source audio file -- `sample_rate` (int): Sample rate in Hz -- `channels` (int): Number of audio channels (1=mono, 2=stereo) -- `has_audio` (bool): Whether this clip has actual audio - -**Methods:** - -#### get_samples(start: float = 0, end: Optional[float] = None) -> np.ndarray -Get audio samples as numpy array. - -**Parameters:** -- `start` (float): Start time relative to this clip's offset (seconds) -- `end` (Optional[float]): End time relative to this clip's offset (seconds) - -**Returns:** Numpy array of shape (n_samples, n_channels) with float32 values in [-1, 1] - -**Note:** For memory-efficient processing of long audio, use `iter_chunks()` instead. - ---- - -#### iter_chunks(chunk_duration: float = 5.0) -> Iterator[Tuple[np.ndarray, float]] -Iterate over audio chunks sequentially. - -**Parameters:** -- `chunk_duration` (float): Duration of each chunk in seconds - -**Yields:** Tuple of (processed_samples, chunk_start_time) - -**Example:** -```python -for samples, start_time in audio.iter_chunks(chunk_duration=10.0): - # Process each chunk - process_audio(samples) -``` - ---- - -#### add_transform(callback: Callable[[np.ndarray, float, int], np.ndarray]) -> Self -Apply a custom transformation to audio samples at render time. - -**Parameters:** -- `callback` (Callable): Function that takes (samples, time, sample_rate) and returns transformed samples - -**Returns:** Self for chaining - -**Example:** -```python -def apply_reverb(samples, t, sr): - # Apply custom reverb effect - return reverb_filter(samples, sr) - -audio.add_transform(apply_reverb) -``` - ---- - -#### fade_in(duration: float) -> Self -Apply a linear fade-in effect. - -**Parameters:** -- `duration` (float): Fade duration in seconds - -**Returns:** Self for chaining - ---- - -#### fade_out(duration: float) -> Self -Apply a linear fade-out effect. - -**Parameters:** -- `duration` (float): Fade duration in seconds - -**Returns:** Self for chaining - ---- - -#### set_volume(volume: float) -> Self -Set the volume of this audio clip. - -**Parameters:** -- `volume` (float): Volume multiplier (0.0 to 1.0+) - -**Returns:** Self for chaining - ---- - -#### set_volume_curve(curve: Union[Callable[[float], float], float]) -> Self -Set a volume curve that changes over time. - -**Parameters:** -- `curve`: Either a float (constant volume) or a function that takes time and returns volume - -**Returns:** Self for chaining - -**Example:** -```python -# Gradual volume increase -audio.set_volume_curve(lambda t: min(1.0, t / 5.0)) -``` - ---- - -#### subclip(start: float, end: float) -> AudioClip -Extract a subclip from this audio. - -**Parameters:** -- `start` (float): Start time within this clip (seconds) -- `end` (float): End time within this clip (seconds) - -**Returns:** New AudioClip instance - ---- - -#### loop(enabled: bool = True) -> Self -Enable or disable looping for this audio clip. - -**Parameters:** -- `enabled` (bool): Whether to enable looping - -**Returns:** Self for chaining - ---- - -#### add_effect(effect: AudioEffect) -> Self -Apply an audio effect to this clip. - -**Parameters:** -- `effect` (AudioEffect): An AudioEffect instance to apply - -**Returns:** Self for chaining - -**Example:** -```python -from movielite import afx -clip.add_effect(afx.FadeIn(2.0)).add_effect(afx.FadeOut(1.5)) -``` - ---- - -### VideoWriter - -Write clips to a video file. - -**Constructor:** -```python -VideoWriter( - output_path: str, - fps: float = 30, - size: Optional[Tuple[int, int]] = None, - duration: Optional[float] = None -) -``` - -**Parameters:** -- `output_path` (str): Path where the final video will be saved -- `fps` (float): Frames per second for the output video -- `size` (Optional[Tuple[int, int]]): Video dimensions (width, height). If None, auto-calculated from clips -- `duration` (Optional[float]): Total duration in seconds. If None, auto-calculated from clips - -**Methods:** - -#### add_clip(clip: MediaClip) -> VideoWriter -Add a media clip to the composition. - -**Parameters:** -- `clip` (MediaClip): MediaClip to add (VideoClip, AudioClip, ImageClip, TextClip, etc.) - -**Returns:** Self for chaining - ---- - -#### add_clips(clips: List[MediaClip]) -> VideoWriter -Add multiple media clips to the composition. - -**Parameters:** -- `clips` (List[MediaClip]): List of MediaClip instances to add - -**Returns:** Self for chaining - ---- - -#### write(processes: int = 1, video_quality: VideoQuality = VideoQuality.MIDDLE) -> None -Render and write the final video. - -**Parameters:** -- `processes` (int): Number of processes to use for parallel rendering -- `video_quality` (VideoQuality): Quality preset for encoding - -**Example:** -```python -from movielite import VideoWriter, VideoQuality - -writer = VideoWriter("output.mp4", fps=30, size=(1920, 1080)) -writer.add_clip(clip) -writer.write(processes=8, video_quality=VideoQuality.HIGH) -``` - ---- - -## Visual Effects (vfx) - -All visual effects inherit from `GraphicEffect` and are applied using `clip.add_effect(effect)`. - -### Fade Effects - -#### FadeIn -Gradually increases opacity from 0 to the clip's original opacity. - -```python -from movielite import vfx -clip.add_effect(vfx.FadeIn(duration=2.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the fade in seconds - ---- - -#### FadeOut -Gradually decreases opacity from the clip's original opacity to 0. - -```python -clip.add_effect(vfx.FadeOut(duration=1.5)) -``` - -**Parameters:** -- `duration` (float): Duration of the fade in seconds - ---- - -### Blur Effects - -#### Blur -Apply Gaussian blur to the clip. - -```python -clip.add_effect(vfx.Blur(intensity=5.0, animated=False)) -``` - -**Parameters:** -- `intensity` (float): Blur intensity (kernel size). Higher = more blur -- `animated` (bool): If True, blur increases from 0 to intensity over duration -- `duration` (Optional[float]): Duration of the blur animation (required if animated=True) - ---- - -#### BlurIn -Starts blurred and gradually becomes sharp. - -```python -clip.add_effect(vfx.BlurIn(duration=2.0, max_intensity=15.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the blur-in effect -- `max_intensity` (float): Maximum blur intensity at the start - ---- - -#### BlurOut -Starts sharp and gradually becomes blurred. - -```python -clip.add_effect(vfx.BlurOut(duration=2.0, max_intensity=15.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the blur-out effect -- `max_intensity` (float): Maximum blur intensity at the end - ---- - -### Color Effects - -#### Saturation -Adjust saturation of the clip. - -```python -clip.add_effect(vfx.Saturation(factor=1.5)) -``` - -**Parameters:** -- `factor` (float): Saturation multiplier (0.0=grayscale, 1.0=no change, >1.0=more saturated) - ---- - -#### Brightness -Adjust brightness of the clip. - -```python -clip.add_effect(vfx.Brightness(factor=1.2)) -``` - -**Parameters:** -- `factor` (float): Brightness multiplier (1.0=no change, >1.0=brighter, <1.0=darker) - ---- - -#### Contrast -Adjust contrast of the clip. - -```python -clip.add_effect(vfx.Contrast(factor=1.3)) -``` - -**Parameters:** -- `factor` (float): Contrast multiplier (1.0=no change, >1.0=more contrast, <1.0=less contrast) - ---- - -#### BlackAndWhite / Grayscale -Convert clip to black and white. - -```python -clip.add_effect(vfx.BlackAndWhite()) -# or -clip.add_effect(vfx.Grayscale()) -``` - ---- - -#### Sepia -Apply sepia tone effect. - -```python -clip.add_effect(vfx.Sepia(intensity=1.0)) -``` - -**Parameters:** -- `intensity` (float): Intensity of the sepia effect (0.0 to 1.0) - ---- - -### Zoom Effects - -#### ZoomIn -Gradually scales up the clip from a smaller size. - -```python -clip.add_effect(vfx.ZoomIn(duration=3.0, from_scale=0.5, to_scale=1.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the zoom effect -- `from_scale` (float): Starting scale (0.5 = 50% size) -- `to_scale` (float): Ending scale (1.0 = 100% size) - ---- - -#### ZoomOut -Gradually scales down the clip. - -```python -clip.add_effect(vfx.ZoomOut(duration=3.0, from_scale=1.0, to_scale=0.5)) -``` - -**Parameters:** -- `duration` (float): Duration of the zoom effect -- `from_scale` (float): Starting scale -- `to_scale` (float): Ending scale - ---- - -#### KenBurns -Slow zoom and pan animation for cinematic effect. - -```python -clip.add_effect(vfx.KenBurns( - duration=None, # Uses entire clip duration - start_scale=1.0, - end_scale=1.2, - start_position=(0, 0), - end_position=(100, 50) -)) -``` - -**Parameters:** -- `duration` (Optional[float]): Duration of the effect (None = entire clip duration) -- `start_scale` (float): Starting zoom level -- `end_scale` (float): Ending zoom level -- `start_position` (Tuple[int, int]): Starting position (x, y) -- `end_position` (Tuple[int, int]): Ending position (x, y) - ---- - -### Glitch Effects - -#### Glitch -Digital distortion artifacts simulating VHS glitches. - -```python -clip.add_effect(vfx.Glitch( - intensity=0.5, - rgb_shift=True, - horizontal_lines=True, - scan_lines=False -)) -``` - -**Parameters:** -- `intensity` (float): Intensity of the glitch effect (0.0 to 1.0) -- `rgb_shift` (bool): Enable RGB channel shifting -- `horizontal_lines` (bool): Enable horizontal displacement lines -- `scan_lines` (bool): Enable scan line artifacts - ---- - -#### ChromaticAberration -RGB channel separation for lens distortion effect. - -```python -clip.add_effect(vfx.ChromaticAberration(intensity=5.0)) -``` - -**Parameters:** -- `intensity` (float): Intensity of the aberration in pixels - ---- - -#### Pixelate -Reduce resolution to create a blocky appearance. - -```python -clip.add_effect(vfx.Pixelate(block_size=10)) -``` - -**Parameters:** -- `block_size` (int): Size of pixelation blocks in pixels - ---- - -### Other Effects - -#### Vignette -Darken the edges of the frame. - -```python -clip.add_effect(vfx.Vignette(intensity=0.5, radius=0.8)) -``` - -**Parameters:** -- `intensity` (float): Darkness intensity at the edges (0.0 to 1.0) -- `radius` (float): Radius of the bright center area (0.0 to 1.0) - ---- - -## Audio Effects (afx) - -All audio effects inherit from `AudioEffect` and are applied using `audio.add_effect(effect)`. - -### FadeIn -Gradually increases volume from 0 to the clip's original volume. - -```python -from movielite import afx -audio.add_effect(afx.FadeIn(duration=2.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the fade in seconds - ---- - -### FadeOut -Gradually decreases volume from the clip's original volume to 0. - -```python -audio.add_effect(afx.FadeOut(duration=1.5)) -``` - -**Parameters:** -- `duration` (float): Duration of the fade in seconds - ---- - -## Transitions (vtx) - -All transitions inherit from `Transition` and are applied using `clip1.add_transition(clip2, transition)`. - -### CrossFade -Smooth opacity-based transition between two clips. - -```python -from movielite import vtx -clip1.add_transition(clip2, vtx.CrossFade(duration=0.5)) -``` - -**Parameters:** -- `duration` (float): Duration of the crossfade in seconds - -**Note:** Requires clips to overlap by at least `duration` seconds. - ---- - -### BlurDissolve -Transition with blur effect during the dissolve. - -```python -clip1.add_transition(clip2, vtx.BlurDissolve(duration=1.0, max_blur=15.0)) -``` - -**Parameters:** -- `duration` (float): Duration of the transition in seconds -- `max_blur` (float): Maximum blur intensity at the midpoint - ---- - -## Enumerations - -### VideoQuality - -Quality presets for video encoding. - -```python -from movielite import VideoQuality - -# Available values: -VideoQuality.LOW # Fastest encoding, lowest quality -VideoQuality.MIDDLE # Balanced (default) -VideoQuality.HIGH # Slower encoding, better quality -VideoQuality.VERY_HIGH # Slowest encoding, best quality -``` - ---- - -## Utilities - -### get_logger() -Get the movielite logger instance. - -```python -from movielite import get_logger - -logger = get_logger() -logger.info("Processing video...") -``` - ---- - -### set_log_level(level: int) -Set logging level. - -```python -from movielite import set_log_level -import logging - -set_log_level(logging.DEBUG) -``` - -**Parameters:** -- `level` (int): Logging level (e.g., logging.DEBUG, logging.INFO, logging.WARNING) diff --git a/backend/output/audio/.gitkeep b/backend/output/audio/.gitkeep old mode 100755 new mode 100644 diff --git a/backend/output/audio/sample-job-001.srt b/backend/output/audio/sample-job-001.srt new file mode 100644 index 0000000..12b8daf --- /dev/null +++ b/backend/output/audio/sample-job-001.srt @@ -0,0 +1,127 @@ +1 +00:00:00,000 --> 00:00:01,474 +Prashant, my dear boy. + +2 +00:00:01,474 --> 00:00:03,077 +I see you've been deep + +3 +00:00:03,077 --> 00:00:04,680 +in that 'Install Real Linux + +4 +00:00:04,680 --> 00:00:06,282 +Desktop on Any Android Phone' + +5 +00:00:06,282 --> 00:00:07,885 +video on YouTube, for nearly + +6 +00:00:07,885 --> 00:00:09,488 +47 minutes. That's a long + +7 +00:00:09,488 --> 00:00:10,770 +stretch, especially at this + +8 +00:00:10,770 --> 00:00:12,372 +time of night, with the + +9 +00:00:12,372 --> 00:00:13,975 +rain coming down steady at + +10 +00:00:13,975 --> 00:00:15,578 +22 degrees outside. No wonder + +11 +00:00:15,578 --> 00:00:17,180 +you’re feeling it now, that + +12 +00:00:17,180 --> 00:00:18,783 +deep fatigue. It’s like a + +13 +00:00:18,783 --> 00:00:20,385 +warm candle, burning so brightly, + +14 +00:00:20,385 --> 00:00:21,988 +just starting to flicker, its + +15 +00:00:21,988 --> 00:00:23,591 +flame gentle and low. You + +16 +00:00:23,591 --> 00:00:25,193 +poured yourself into that, and + +17 +00:00:25,193 --> 00:00:26,796 +it's so easy to get + +18 +00:00:26,796 --> 00:00:28,399 +pulled in, isn't it? To + +19 +00:00:28,399 --> 00:00:30,001 +lose track of time when + +20 +00:00:30,001 --> 00:00:31,604 +you’re focused. That’s just being + +21 +00:00:31,604 --> 00:00:33,206 +human, my boy. Now, just + +22 +00:00:33,206 --> 00:00:34,809 +gently place one hand on + +23 +00:00:34,809 --> 00:00:36,412 +your chest, and the other + +24 +00:00:36,412 --> 00:00:38,014 +on your stomach. Feel the + +25 +00:00:38,014 --> 00:00:39,617 +quiet rise and fall of + +26 +00:00:39,617 --> 00:00:41,220 +your own breath. Just for + +27 +00:00:41,220 --> 00:00:42,822 +a moment. No need to + +28 +00:00:42,822 --> 00:00:44,425 +do anything else. Just let + +29 +00:00:44,425 --> 00:00:46,028 +yourself be. The coding can + +30 +00:00:46,028 --> 00:00:47,630 +wait. The rain will still + +31 +00:00:47,630 --> 00:00:49,233 +fall. Just rest, Prashant. You've + +32 +00:00:49,233 --> 00:00:49,970 +earned it. diff --git a/backend/output/reels/.gitkeep b/backend/output/reels/.gitkeep old mode 100755 new mode 100644 diff --git a/backend/package-lock.json b/backend/package-lock.json old mode 100755 new mode 100644 diff --git a/backend/package.json b/backend/package.json old mode 100755 new mode 100644 diff --git a/backend/reel_generator.py b/backend/reel_generator.py old mode 100755 new mode 100644 index 1f3a712..a351af2 --- a/backend/reel_generator.py +++ b/backend/reel_generator.py @@ -5,28 +5,16 @@ # Suppress tqdm progress bars from MovieLite (must be before imports) import os - -os.environ["TQDM_DISABLE"] = "1" - -# Limit BLAS/OpenMP threading at import time so numpy doesn't saturate all cores. -# The preset system overrides ffmpeg threads at runtime, but BLAS threads must be -# set before OpenBLAS initializes (i.e. before numpy is imported). -os.environ.setdefault("OPENBLAS_NUM_THREADS", "1") -os.environ.setdefault("OPENBLAS64_NUM_THREADS", "1") # scipy_openblas64 uses this -os.environ.setdefault("MKL_NUM_THREADS", "1") -os.environ.setdefault("OMP_NUM_THREADS", "1") -os.environ.setdefault("NUMEXPR_NUM_THREADS", "1") +os.environ['TQDM_DISABLE'] = '1' import re import json import asyncio import requests import base64 -import shutil import sys import time import threading -from dataclasses import dataclass from typing import List, Dict, Any, Optional, Tuple from concurrent.futures import ThreadPoolExecutor, as_completed from dotenv import load_dotenv @@ -40,244 +28,27 @@ print("Run: pip install movielite pictex google-generativeai python-dotenv") exit(1) -# Suppress third-party warnings for cleaner ux +# Suppress third-party warnings for cleaner CLI output import warnings - -warnings.filterwarnings("ignore", category=UserWarning) -warnings.filterwarnings("ignore", message=".*FontNotFoundWarning.*") -warnings.filterwarnings("ignore", message=".*fontconfig.*") -warnings.filterwarnings("ignore", message=".*resource_tracker.*") +warnings.filterwarnings('ignore', category=UserWarning) +warnings.filterwarnings('ignore', message='.*FontNotFoundWarning.*') +warnings.filterwarnings('ignore', message='.*fontconfig.*') +warnings.filterwarnings('ignore', message='.*resource_tracker.*') import logging - -logging.basicConfig(level=logging.ERROR) -for logger_name in ["movielite", "moviepy", "imageio", "urllib3", "google", "httpx", "httpcore"]: - _l = logging.getLogger(logger_name) - _l.setLevel(logging.ERROR) - _l.propagate = False +logging.getLogger('movielite').setLevel(logging.ERROR) +logging.getLogger('movielite').propagate = False import contextlib +# Load environment variables from .env file load_dotenv() - -@dataclass(frozen=True) -class HardwareProfile: - """CPU topology available to this process, respecting cpuset limits.""" - - logical_cpus: Tuple[int, ...] - physical_core_groups: Tuple[Tuple[int, ...], ...] - - @property - def logical_core_count(self) -> int: - return len(self.logical_cpus) - - @property - def physical_core_count(self) -> int: - return len(self.physical_core_groups) - - -def _detect_hardware_profile() -> HardwareProfile: - """Return available logical CPUs grouped by physical core where possible.""" - try: - logical_cpus = tuple(sorted(os.sched_getaffinity(0))) - except (AttributeError, OSError): - logical_cpus = tuple(range(max(1, os.cpu_count() or 1))) - if not logical_cpus: - logical_cpus = (0,) - - # Linux exposes sibling threads through sysfs. Keeping siblings together lets - # Fast use a small number of complete physical cores instead of all threads. - groups: Dict[Tuple[str, str], List[int]] = {} - try: - for cpu in logical_cpus: - topology_dir = f"/sys/devices/system/cpu/cpu{cpu}/topology" - with open(os.path.join(topology_dir, "physical_package_id")) as fh: - package_id = fh.read().strip() - with open(os.path.join(topology_dir, "core_id")) as fh: - core_id = fh.read().strip() - groups.setdefault((package_id, core_id), []).append(cpu) - except OSError: - groups = {} - - if groups: - physical_core_groups = tuple( - tuple(cpus) for _, cpus in sorted(groups.items(), key=lambda item: min(item[1])) - ) - else: - # Without topology information, treat each available CPU as a core. This - # remains safe because Fast is deliberately capped below all-core usage. - physical_core_groups = tuple((cpu,) for cpu in logical_cpus) - - return HardwareProfile(logical_cpus, physical_core_groups) - -# --------------------------------------------------------------------------- -# Speed / resource presets -# -# Each preset defines a user experience, not a fixed resource allocation. -# Render process counts are derived from detected hardware at init time. -# --------------------------------------------------------------------------- -PRESETS = { - # Background-friendly. Resource limits are resolved from the CPU topology. - "normal": { - "frame_size": (720, 1280), - "target_fps": 24, - "video_quality": "middle", - "download_workers": 1, - "download_chunk_size": 131072, - }, - # Faster while intentionally leaving most of the machine to the user. - "fast": { - "frame_size": (720, 1280), - "target_fps": 24, - # MovieLite maps this to x264 ultrafast / CRF 23. At 720p it is a - # practical Fast trade-off, while Normal retains the balanced encoder. - "video_quality": "low", - "cpu_quota_percent": None, - "download_workers": 2, - "download_chunk_size": 262144, - }, -} - -CALIBRATION_VERSION = 2 -CALIBRATION_SAMPLE_SECONDS = 6.0 -CALIBRATION_MIN_SPEEDUP = 0.08 -CALIBRATION_WORKER_MEMORY_BYTES = 768 * 1024 * 1024 - - -def _available_memory_bytes() -> Optional[int]: - """Return available RAM on Linux without treating cached memory as unavailable.""" - try: - values = {} - with open("/proc/meminfo") as fh: - for line in fh: - key, value = line.split(":", 1) - values[key] = int(value.strip().split()[0]) * 1024 - return values.get("MemAvailable") - except (OSError, ValueError, IndexError): - return None - - -def _candidate_worker_counts(hardware: HardwareProfile) -> Tuple[int, ...]: - """Return safe worker candidates without consuming every physical core.""" - # Reserve one physical core for the desktop whenever the machine has more - # than one. MovieLite's short-reel split/merge overhead also makes more than - # four candidates counterproductive to benchmark at generation time. - cpu_limit = max(1, hardware.physical_core_count - 1) - memory_available = _available_memory_bytes() - memory_limit = ( - max(1, memory_available // CALIBRATION_WORKER_MEMORY_BYTES) - if memory_available is not None - else 2 - ) - max_workers = min(4, cpu_limit, memory_limit) - return tuple(range(1, max_workers + 1)) - - -def _normal_cpu_quota_percent(hardware: HardwareProfile) -> int: - """Choose a responsive aggregate CPU budget from available core topology.""" - # A compositor and encoder can use CPU concurrently. On a four-core machine, - # 150% lets that pipeline make progress while still leaving most capacity to - # the desktop; smaller machines scale down instead of using the same quota. - return min(150, max(100, 50 + 25 * hardware.physical_core_count)) - - -def _resolve_preset(preset: str, hardware: HardwareProfile) -> Dict[str, Any]: - """Resolve a preset into limits appropriate for this machine. - - MovieLite creates one Python compositor and one libx264 encoder per render - process. Its implementation also merges every part afterwards, so benchmarked - scaling is useful through two workers but additional workers mainly add memory - pressure and encoder contention for MindStream's short portrait reels. - """ - if preset not in PRESETS: - raise ValueError(f"Unknown preset '{preset}'. Choose from: {', '.join(PRESETS)}") - - config = dict(PRESETS[preset]) - if preset == "normal": - # A single compositor plus the cgroup quota keeps Normal lightweight. - # Leave placement to the kernel scheduler rather than pinning CPU 0. - config.update( - writer_processes=1, - ffmpeg_threads=1, - cpu_affinity=(), - niceness=10, - cpu_quota_percent=_normal_cpu_quota_percent(hardware), - ) - return config - - # Fast preset uses multi-worker rendering and full physical core count for FFmpeg encoding - fast_worker_limit = max(2, min(4, hardware.physical_core_count)) - config.update( - writer_processes=fast_worker_limit, - ffmpeg_threads=max(2, hardware.physical_core_count), - cpu_affinity=(), - niceness=0, - ) - return config - - -def _run_in_cpu_limited_scope(preset: str) -> Optional[int]: - """Re-exec Normal in a user cgroup so its average CPU use is truly capped. - - Affinity and niceness are useful scheduling hints, but neither prevents an - idle machine from running one core at 100%. systemd's CPUQuota covers the - Python renderer and every MovieLite/FFmpeg child process on Linux. - """ - quota_percent = ( - _normal_cpu_quota_percent(_detect_hardware_profile()) - if preset == "normal" - else PRESETS[preset].get("cpu_quota_percent") - ) - if ( - not quota_percent - or sys.platform != "linux" - or os.getenv("MINDSTREAM_CPU_QUOTA_APPLIED") == "1" - ): - return None - - systemd_run = shutil.which("systemd-run") - if not systemd_run: - print("WARNING: CPU quota unavailable; continuing with affinity limits only.") - return None - - command = [ - systemd_run, - "--user", - "--scope", - "--quiet", - "-p", - f"CPUQuota={quota_percent}%", - "env", - "MINDSTREAM_CPU_QUOTA_APPLIED=1", - sys.executable, - os.path.abspath(__file__), - *sys.argv[1:], - ] - print(f"Normal preset: applying a {quota_percent}% CPU quota.") - try: - import subprocess - - return subprocess.run(command, check=False).returncode - except OSError as exc: - print(f"WARNING: Could not apply CPU quota ({exc}); continuing normally.") - return None - - -# TODO: ambient audio class ReelGenerator: - """Complete generation pipeline with multi-source video search and ambient audio.""" + """Complete reel generation pipeline with multi-source video search and ambient audio.""" - def __init__( - self, - gemini_key: str = None, - pexels_key: str = None, - pixabay_key: str = None, - coverr_key: str = None, - mimo_key: str = None, - preset: str = "normal", - recalibrate_presets: bool = False, - ): - # load API keys if not provided + def __init__(self, gemini_key: str = None, pexels_key: str = None, + pixabay_key: str = None, coverr_key: str = None, mimo_key: str = None): + # Load API keys from environment self.gemini_key = gemini_key or os.getenv("GEMINI_API_KEY") self.groq_key = os.getenv("GROQ_API_KEY") self.pexels_key = pexels_key or os.getenv("PEXELS_API_KEY") @@ -289,17 +60,6 @@ def __init__( self.script_provider = os.getenv("SCRIPT_MODEL_PROVIDER", "gemini").lower() self.script_model = os.getenv("SCRIPT_MODEL_NAME", "gemini-2.0-flash-exp") - # Speed / resource preset - self.preset = preset - self.hardware = _detect_hardware_profile() - self.physical_cores = self.hardware.physical_core_count - self.preset_cfg = _resolve_preset(preset, self.hardware) - self._niceness_applied = False - self.recalibrate_presets = recalibrate_presets - - # Reuse a single HTTP session across all API calls (connection pooling) - self._http = requests.Session() - # Validate required keys if self.script_provider == "gemini" and not self.gemini_key: raise ValueError("GEMINI_API_KEY required (set in .env)") @@ -320,34 +80,28 @@ def __init__( os.makedirs(os.path.join(self.output_dir, "audio"), exist_ok=True) os.makedirs(os.path.join(self.output_dir, "reels"), exist_ok=True) os.makedirs(self.temp_dir, exist_ok=True) - self.calibration_path = os.path.join( - self.output_dir, "render_profile_calibration.json" - ) - + # Ambient audio mapping (placeholder files - replace with real audio later) self.ambient_music = { "frustrated": "assets/audio/frustrated.mp3", # Dark ambient, subtle rain - "fatigued": "assets/audio/fatigued.mp3", # Soft piano, gentle pads + "fatigued": "assets/audio/fatigued.mp3", # Soft piano, gentle pads "distracted": "assets/audio/distracted.mp3", # Calm waves, subtle wind - "anxious": "assets/audio/anxious.mp3", # Breathing sounds, soft hum - "neutral": "assets/audio/neutral.mp3", # White noise, minimal drone + "anxious": "assets/audio/anxious.mp3", # Breathing sounds, soft hum + "neutral": "assets/audio/neutral.mp3", # White noise, minimal drone } @contextlib.contextmanager def _spinner(self, message: str): """Context manager that shows a braille spinner with the given message.""" stop = False - def _run(): idx = 0 while not stop: - sys.stdout.write( - f"\r{self.BRAILLE_CHARS[idx % len(self.BRAILLE_CHARS)]} {message}" - ) + sys.stdout.write(f'\r{self.BRAILLE_CHARS[idx % len(self.BRAILLE_CHARS)]} {message}') sys.stdout.flush() idx += 1 time.sleep(0.1) - sys.stdout.write("\r" + " " * 50 + "\r") + sys.stdout.write('\r' + ' ' * 50 + '\r') sys.stdout.flush() thread = threading.Thread(target=_run, daemon=True) @@ -369,12 +123,12 @@ def generate_script(self, emotion: str, context: Dict[str, Any]) -> Dict[str, An - "subtitles": list of short phrases (4-6 words each) that together cover the whole script in order """ - activity = context.get("active_tab_category", "browsing") - time_of_day = context.get("time_of_day", "the day") - duration = context.get("session_duration_minutes", 0) - idle_time = context.get("idle_minutes_since_last_activity", 0) - user_name = context.get("user_name", "friend") - local_weather = context.get("local_weather", "calm") + activity = context.get("active_tab_category", "browsing") + time_of_day = context.get("time_of_day", "the day") + duration = context.get("session_duration_minutes", 0) + idle_time = context.get("idle_minutes_since_last_activity", 0) + user_name = context.get("user_name", "friend") + local_weather = context.get("local_weather", "calm") # Build activity description more generically activity_desc = activity @@ -427,31 +181,30 @@ def generate_script(self, emotion: str, context: Dict[str, Any]) -> Dict[str, An # Call the appropriate LLM based on provider if self.script_provider == "groq": - response = self._http.post( + response = requests.post( "https://api.groq.com/openai/v1/chat/completions", headers={ "Authorization": f"Bearer {self.groq_key}", - "Content-Type": "application/json", + "Content-Type": "application/json" }, json={ "model": self.script_model, "messages": [ - { - "role": "system", - "content": "You are a wise, warm elder who creates mindfulness scripts in JSON format.", - }, - {"role": "user", "content": prompt}, + {"role": "system", "content": "You are a wise, warm elder who creates mindfulness scripts in JSON format."}, + {"role": "user", "content": prompt} ], "temperature": 0.8, - "response_format": {"type": "json_object"}, + "response_format": {"type": "json_object"} }, - timeout=30, + timeout=30 ) response.raise_for_status() raw = response.json()["choices"][0]["message"]["content"] else: response = self.client.models.generate_content( - model=self.script_model, contents=prompt, config={"temperature": 0.9} + model=self.script_model, + contents=prompt, + config={"temperature": 0.9} ) raw = response.text.strip() # Strip markdown fences if the model ignores the instruction @@ -463,11 +216,9 @@ def generate_script(self, emotion: str, context: Dict[str, Any]) -> Dict[str, An data = json.loads(raw) if "script" not in data or "subtitles" not in data: - raise ValueError( - f"LLM JSON missing required keys. Got: {list(data.keys())}" - ) + raise ValueError(f"LLM JSON missing required keys. Got: {list(data.keys())}") - script = data["script"].strip() + script = data["script"].strip() subtitles = [p.strip() for p in data["subtitles"] if p.strip()] if not script: @@ -483,7 +234,7 @@ def generate_script(self, emotion: str, context: Dict[str, Any]) -> Dict[str, An def extract_video_keywords(self, script: str, emotion: str) -> List[str]: """Extract 3-5 cinematic/moody video search terms from the script.""" - + prompt = f"""From this mindfulness script about the emotion "{emotion}", extract 3-5 search terms to find matching stock video footage. Script: @@ -499,18 +250,15 @@ def extract_video_keywords(self, script: str, emotion: str) -> List[str]: try: if self.script_provider == "groq": - response = self._http.post( + response = requests.post( "https://api.groq.com/openai/v1/chat/completions", - headers={ - "Authorization": f"Bearer {self.groq_key}", - "Content-Type": "application/json", - }, + headers={"Authorization": f"Bearer {self.groq_key}", "Content-Type": "application/json"}, json={ "model": self.script_model, "messages": [{"role": "user", "content": prompt}], - "temperature": 0.7, + "temperature": 0.7 }, - timeout=20, + timeout=20 ) response.raise_for_status() text = response.json()["choices"][0]["message"]["content"] @@ -518,10 +266,10 @@ def extract_video_keywords(self, script: str, emotion: str) -> List[str]: response = self.client.models.generate_content( model=self.script_model, contents=prompt, - config={"temperature": 0.9}, + config={"temperature": 0.9} ) text = response.text.strip() - + text = text.replace("```json", "").replace("```", "").strip() keywords = json.loads(text) if isinstance(keywords, list) and keywords: @@ -536,35 +284,24 @@ def extract_video_keywords(self, script: str, emotion: str) -> List[str]: # Step 3 — Multi-source video search + download # ----------------------------------------------------------------------- - def search_pexels_videos( - self, keyword: str, orientation: str = "portrait" - ) -> Optional[str]: + def search_pexels_videos(self, keyword: str, orientation: str = "portrait") -> Optional[str]: """Search Pexels and return a direct download URL for the best match.""" try: - resp = self._http.get( + resp = requests.get( "https://api.pexels.com/videos/search", headers={"Authorization": self.pexels_key}, - params={ - "query": keyword, - "orientation": orientation, - "size": "medium", - "per_page": 20, - }, + params={"query": keyword, "orientation": orientation, "size": "medium", "per_page": 20}, timeout=10, ) resp.raise_for_status() videos = resp.json().get("videos", []) - + if not videos: return None # Prefer HD (height >= 1080) portrait files for video in videos: - for f in sorted( - video.get("video_files", []), - key=lambda x: x.get("height", 0), - reverse=True, - ): + for f in sorted(video.get("video_files", []), key=lambda x: x.get("height", 0), reverse=True): if f.get("height", 0) >= 720: return f.get("link") @@ -578,21 +315,21 @@ def search_pixabay_videos(self, keyword: str) -> Optional[str]: """Search Pixabay (fallback source) and return a direct download URL.""" if not self.pixabay_key: return None - + try: - resp = self._http.get( + resp = requests.get( "https://pixabay.com/api/videos/", params={ "key": self.pixabay_key, "q": keyword, "video_type": "all", - "per_page": 20, + "per_page": 20 }, timeout=10, ) resp.raise_for_status() videos = resp.json().get("hits", []) - + if not videos: return None @@ -602,7 +339,7 @@ def search_pixabay_videos(self, keyword: str) -> Optional[str]: return video["videos"]["medium"]["url"] elif "small" in video.get("videos", {}): return video["videos"]["small"]["url"] - + return None except Exception as e: return None @@ -611,10 +348,10 @@ def search_coverr_videos(self, keyword: str) -> Optional[str]: """Search Coverr (fallback source) and return a direct download URL.""" if not self.coverr_key: return None - + try: # Coverr API endpoint (based on common API patterns) - resp = self._http.get( + resp = requests.get( "https://api.coverr.co/videos", headers={"Authorization": f"Bearer {self.coverr_key}"}, params={"query": keyword, "per_page": 20}, @@ -622,7 +359,7 @@ def search_coverr_videos(self, keyword: str) -> Optional[str]: ) resp.raise_for_status() videos = resp.json().get("videos", []) - + if not videos: return None @@ -630,7 +367,7 @@ def search_coverr_videos(self, keyword: str) -> Optional[str]: for video in videos: if "url" in video: return video["url"] - + return None except Exception as e: # Coverr API might have different structure, fail gracefully @@ -638,12 +375,11 @@ def search_coverr_videos(self, keyword: str) -> Optional[str]: def _download_video(self, url: str, dest: str) -> bool: try: - r = self._http.get(url, stream=True, timeout=60) + r = requests.get(url, stream=True, timeout=60) r.raise_for_status() - - chunk_size = self.preset_cfg["download_chunk_size"] + with open(dest, "wb") as fh: - for chunk in r.iter_content(chunk_size=chunk_size): + for chunk in r.iter_content(chunk_size=65536): fh.write(chunk) return True except Exception as e: @@ -657,50 +393,48 @@ def download_videos_for_script(self, keywords: List[str], job_id: str) -> List[s if not keywords: print("No keywords extracted — cannot download videos") return [] - + def download_single_keyword(i: int, kw: str) -> Optional[str]: """Try all sources for a keyword: Pexels → Pixabay → Coverr""" url = None - + # Try Pexels first url = self.search_pexels_videos(kw) if url: source = "Pexels" - + # Fallback to Pixabay if not url and self.pixabay_key: url = self.search_pixabay_videos(kw) if url: source = "Pixabay" - + # Fallback to Coverr if not url and self.coverr_key: url = self.search_coverr_videos(kw) if url: source = "Coverr" - + if not url: return None - + dest = os.path.join(self.temp_dir, f"{job_id}_clip_{i}.mp4") if self._download_video(url, dest): return dest return None - + # Download videos in parallel with DNF-style progress paths = [] completed = 0 total = len(keywords) width = 20 - - with ThreadPoolExecutor( - max_workers=min(self.preset_cfg["download_workers"], len(keywords)) - ) as executor: + + with ThreadPoolExecutor(max_workers=min(4, len(keywords))) as executor: futures = { - executor.submit(download_single_keyword, i, kw): (i, kw) + executor.submit(download_single_keyword, i, kw): (i, kw) for i, kw in enumerate(keywords) } - + for future in as_completed(futures): i, kw = futures[future] try: @@ -713,40 +447,26 @@ def download_single_keyword(i: int, kw: str) -> Optional[str]: # DNF-style progress bar matching the format of other steps pct = int(100 * completed / total) if total > 0 else 0 filled = int(width * completed / total) if total > 0 else 0 - bar = "━" * filled + " " * (width - filled) + bar = '━' * filled + ' ' * (width - filled) color = self.CYAN if completed < total else self.GREEN # Format exactly like _progress_bar_dnf with green checkmark prefix desc = f"{'Downloading footage':<26}" if completed >= total: # Final line with green checkmark - print( - f"\r{self.GREEN}✓{self.RESET} {desc}{color}{pct:3d}% |{bar}| {completed}/{total}{self.RESET}", - end="", - flush=True, - ) + print(f"\r{self.GREEN}✓{self.RESET} {desc}{color}{pct:3d}% |{bar}| {completed}/{total}{self.RESET}", end='', flush=True) else: # In-progress with spinner (updates per item) - spinner_char = self.BRAILLE_CHARS[ - completed % len(self.BRAILLE_CHARS) - ] - print( - f"\r{self.CYAN}{spinner_char}{self.RESET} {desc}{color}{pct:3d}% |{bar}| {completed}/{total}{self.RESET}", - end="", - flush=True, - ) - + spinner_char = self.BRAILLE_CHARS[completed % len(self.BRAILLE_CHARS)] + print(f"\r{self.CYAN}{spinner_char}{self.RESET} {desc}{color}{pct:3d}% |{bar}| {completed}/{total}{self.RESET}", end='', flush=True) + print() # newline after progress # Sort paths by clip number to maintain order - paths.sort( - key=lambda p: ( - int(re.search(r"clip_(\d+)", p).group(1)) if "clip_" in p else 999 - ) - ) - + paths.sort(key=lambda p: int(re.search(r'clip_(\d+)', p).group(1)) if 'clip_' in p else 999) + if not paths: print("No videos could be downloaded from any source") return [] - + return paths # ----------------------------------------------------------------------- @@ -755,10 +475,11 @@ def download_single_keyword(i: int, kw: str) -> Optional[str]: async def _generate_tts_async(self, script: str, output_path: str) -> str: loop = asyncio.get_event_loop() - + + response = await loop.run_in_executor( None, - lambda: self._http.post( + lambda: requests.post( "https://api.xiaomimimo.com/v1/chat/completions", headers={ "Authorization": f"Bearer {self.mimo_key}", @@ -772,7 +493,7 @@ async def _generate_tts_async(self, script: str, output_path: str) -> str: timeout=90, ), ) - + if response.status_code != 200: raise RuntimeError( f"MiMo TTS API error {response.status_code}: {response.text[:300]}" @@ -780,7 +501,7 @@ async def _generate_tts_async(self, script: str, output_path: str) -> str: audio_b64 = response.json()["choices"][0]["message"]["audio"]["data"] with open(output_path, "wb") as fh: fh.write(base64.b64decode(audio_b64)) - + return output_path def generate_tts(self, script: str, output_path: str) -> str: @@ -796,12 +517,9 @@ def generate_tts(self, script: str, output_path: str) -> str: def _srt_ts(seconds: float) -> str: """Convert float seconds → SRT timestamp string HH:MM:SS,mmm.""" ms = max(0, int(round(seconds * 1000))) - h = ms // 3_600_000 - ms %= 3_600_000 - m = ms // 60_000 - ms %= 60_000 - s = ms // 1_000 - ms %= 1_000 + h = ms // 3_600_000; ms %= 3_600_000 + m = ms // 60_000; ms %= 60_000 + s = ms // 1_000; ms %= 1_000 return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}" @staticmethod @@ -820,31 +538,31 @@ def build_srt(self, subtitles: List[str], audio_duration: float) -> str: if not subtitles: return "" - PAUSE_BONUS = 0.20 # Add 20% extra time for phrases ending with punctuation - + PAUSE_BONUS = 0.20 # Add 20% extra time for phrases ending with punctuation + # Use character count (better proxy for speech duration than word count) weights = [] for phrase in subtitles: # Count characters (excluding spaces) as base weight w = max(1, len(phrase.replace(" ", ""))) - + # Add pause time for sentence endings if self._phrase_has_pause(phrase): w += PAUSE_BONUS * w - + weights.append(w) total_weight = sum(weights) - available = audio_duration + available = audio_duration - lines = [] - cursor = 0.0 + lines = [] + cursor = 0.0 for idx, (phrase, weight) in enumerate(zip(subtitles, weights), start=1): phrase_dur = available * (weight / total_weight) - start = cursor - end = cursor + phrase_dur - cursor = end + start = cursor + end = cursor + phrase_dur + cursor = end lines.append(str(idx)) lines.append(f"{self._srt_ts(start)} --> {self._srt_ts(end)}") @@ -862,322 +580,65 @@ def build_srt(self, subtitles: List[str], audio_duration: float) -> str: # ----------------------------------------------------------------------- @staticmethod - def _parse_srt_content(content: str) -> List[Tuple[float, float, str]]: - """Parse SRT content string and return list of (start, end, text) tuples.""" - content = content.strip() - blocks = re.split(r"\n\n+", content) + def parse_srt(srt_path: str) -> List[Tuple[float, float, str]]: + """ + Parse SRT file and return list of (start_time, end_time, text) tuples. + Times are in seconds (float). + """ + with open(srt_path, 'r', encoding='utf-8') as f: + content = f.read().strip() + + # Split by double newlines (subtitle blocks) + blocks = re.split(r'\n\n+', content) subtitles = [] - + for block in blocks: - lines = block.strip().split("\n") + lines = block.strip().split('\n') if len(lines) < 3: continue - + + # Line 0: sequence number (ignore) + # Line 1: timestamp (00:00:01,234 --> 00:00:03,456) + # Line 2+: subtitle text + timestamp_line = lines[1] - match = re.match( - r"(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})", - timestamp_line, - ) + match = re.match(r'(\d{2}):(\d{2}):(\d{2}),(\d{3})\s*-->\s*(\d{2}):(\d{2}):(\d{2}),(\d{3})', timestamp_line) if not match: continue - + + # Parse start time h1, m1, s1, ms1, h2, m2, s2, ms2 = map(int, match.groups()) start_time = h1 * 3600 + m1 * 60 + s1 + ms1 / 1000.0 end_time = h2 * 3600 + m2 * 60 + s2 + ms2 / 1000.0 - text = " ".join(lines[2:]) + + # Join remaining lines as text + text = ' '.join(lines[2:]) + subtitles.append((start_time, end_time, text)) - + return subtitles - @staticmethod - def parse_srt(srt_path: str) -> List[Tuple[float, float, str]]: - """Parse SRT file and return list of (start_time, end_time, text) tuples.""" - with open(srt_path, "r", encoding="utf-8") as f: - content = f.read() - return ReelGenerator._parse_srt_content(content) - - def _create_subtitle_canvas(self, frame_width: int) -> Canvas: + def _create_subtitle_canvas(self) -> Canvas: """Create styled canvas for subtitle text (MovieLite/pictex).""" - scale = frame_width / 1080 return ( Canvas() .font_family("Poppins") - .font_size(round(50 * scale)) + .font_size(50) .color("#FFFF00") # Yellow - .text_shadows( - Shadow( - offset=(round(2 * scale), round(2 * scale)), - blur_radius=round(3 * scale), - color="black", - ) - ) - .padding(round(20 * scale)) + .text_shadows(Shadow(offset=(2, 2), blur_radius=3, color="black")) + .padding(20) ) - def _resize_to_portrait( - self, clip: ml.VideoClip, frame_size: Tuple[int, int] - ) -> ml.VideoClip: + def _resize_to_portrait(self, clip: ml.VideoClip) -> ml.VideoClip: """ - Resize to the preset's portrait output size using MovieLite. + Resize to exactly 1080×1920 (9:16) using MovieLite. MovieLite's set_size() maintains aspect ratio and crops/pads automatically. """ # MovieLite's set_size will resize maintaining aspect ratio # If the source aspect ratio doesn't match, it will crop center - clip.set_size(width=frame_size[0], height=frame_size[1]) + clip.set_size(width=1080, height=1920) return clip - def _calibration_signature(self) -> Dict[str, Any]: - """Describe the machine and output profile that affect worker scaling.""" - return { - "logical_cpus": list(self.hardware.logical_cpus), - "physical_core_groups": [ - list(group) for group in self.hardware.physical_core_groups - ], - "frame_size": list(self.preset_cfg["frame_size"]), - "target_fps": self.preset_cfg["target_fps"], - "video_quality": self.preset_cfg["video_quality"], - "movielite_version": getattr(ml, "__version__", "unknown"), - } - - def _affinity_for_workers(self, workers: int) -> Tuple[int, ...]: - # Process count is the resource control. The kernel can then place work - # naturally instead of permanently reserving arbitrary CPU IDs. - return () - - def _load_calibrated_workers(self) -> Optional[int]: - if self.recalibrate_presets or not os.path.exists(self.calibration_path): - return None - try: - with open(self.calibration_path, encoding="utf-8") as fh: - cached = json.load(fh) - profile = cached.get("profiles", {}).get(self.preset) - if ( - cached.get("version") != CALIBRATION_VERSION - or cached.get("signature") != self._calibration_signature() - or not profile - ): - return None - workers = int(profile["workers"]) - return workers if workers in _candidate_worker_counts(self.hardware) else None - except (OSError, ValueError, TypeError, KeyError, json.JSONDecodeError): - return None - - def _save_calibrated_workers( - self, workers: int, timings: Dict[int, float] - ) -> None: - payload: Dict[str, Any] = { - "version": CALIBRATION_VERSION, - "signature": self._calibration_signature(), - "profiles": {}, - } - try: - if os.path.exists(self.calibration_path): - with open(self.calibration_path, encoding="utf-8") as fh: - existing = json.load(fh) - if existing.get("signature") == payload["signature"]: - payload["profiles"] = existing.get("profiles", {}) - except (OSError, ValueError, TypeError, json.JSONDecodeError): - pass - - payload["profiles"][self.preset] = { - "workers": workers, - "timings_seconds": {str(key): round(value, 3) for key, value in timings.items()}, - } - temporary_path = f"{self.calibration_path}.tmp" - try: - with open(temporary_path, "w", encoding="utf-8") as fh: - json.dump(payload, fh, indent=2) - os.replace(temporary_path, self.calibration_path) - except OSError: - try: - os.remove(temporary_path) - except OSError: - pass - - def _benchmark_render_workers(self, video_path: str, workers: int) -> float: - """Render a short clip using a candidate worker count and return wall time.""" - frame_width, frame_height = self.preset_cfg["frame_size"] - output_path = os.path.join( - self.temp_dir, f"calibration_{self.preset}_{workers}_{os.getpid()}.mp4" - ) - clip = None - started = time.perf_counter() - try: - with self._limit_render_resources( - self.preset_cfg["ffmpeg_threads"], - self._affinity_for_workers(workers), - self.preset_cfg["niceness"], - ): - clip = ml.VideoClip(video_path) - sample_duration = min(CALIBRATION_SAMPLE_SECONDS, clip.duration) - if sample_duration <= 0: - raise RuntimeError("Video has no usable duration for calibration") - self._resize_to_portrait(clip, (frame_width, frame_height)) - clip.set_duration(sample_duration) - writer = ml.VideoWriter( - output_path, - fps=self.preset_cfg["target_fps"], - size=(frame_width, frame_height), - duration=sample_duration, - ) - writer.add_clip(clip) - writer.write( - processes=workers, - video_quality=( - ml.VideoQuality.HIGH - if self.preset_cfg["video_quality"] == "high" - else ml.VideoQuality.MIDDLE - ), - ) - return time.perf_counter() - started - finally: - if clip is not None and hasattr(clip, "close"): - clip.close() - try: - os.remove(output_path) - except OSError: - pass - - def _configure_calibrated_workers(self, video_path: str) -> None: - """Select a measured Fast worker count once per hardware/output profile.""" - if self.preset != "fast": - return - - cached_workers = self._load_calibrated_workers() - if cached_workers is not None: - self.preset_cfg["writer_processes"] = cached_workers - self.preset_cfg["cpu_affinity"] = self._affinity_for_workers(cached_workers) - return - - candidates = _candidate_worker_counts(self.hardware) - if len(candidates) == 1: - self.preset_cfg["writer_processes"] = candidates[0] - self.preset_cfg["cpu_affinity"] = () - print(" Fast: using one render worker while memory headroom is limited.") - return - - print("Calibrating Fast for this machine (one-time)...") - timings: Dict[int, float] = {} - for workers in candidates: - try: - timings[workers] = self._benchmark_render_workers(video_path, workers) - except Exception: - continue - - if not timings: - print(" Calibration skipped; using the safe Fast fallback.") - return - - selected_workers = min(timings) - selected_time = timings[selected_workers] - for workers in sorted(timings): - if workers == selected_workers: - continue - candidate_time = timings[workers] - if candidate_time <= selected_time * (1 - CALIBRATION_MIN_SPEEDUP): - selected_workers = workers - selected_time = candidate_time - - self.preset_cfg["writer_processes"] = selected_workers - self.preset_cfg["cpu_affinity"] = self._affinity_for_workers(selected_workers) - self._save_calibrated_workers(selected_workers, timings) - print(f" Fast calibrated: {selected_workers} render worker(s).") - - @contextlib.contextmanager - def _limit_render_resources( - self, - ffmpeg_threads: int, - cpu_affinity: Tuple[int, ...], - niceness: int, - ): - """Apply the preset's CPU budget to MovieLite and its child processes. - - MovieLite forks render workers, so the affinity, niceness, and Popen patch - are inherited by frame rendering, libx264 encoding, audio work, and merging. - """ - import subprocess - - original_popen = subprocess.Popen - - _patch_count = 0 # track how many calls we intercepted - - def _patched_popen(args, *posargs, **kwargs): - nonlocal _patch_count - if isinstance(args, (list, tuple)) and args: - # Match "ffmpeg" by basename (handles /usr/bin/ffmpeg too) - prog = os.path.basename(str(args[0])) - if prog == "ffmpeg" and "-threads" not in args: - args = list(args) - # Find last occurrence of "-i" to place -threads as an output option (after input) - last_i = -1 - for idx, arg in enumerate(args): - if str(arg) == "-i": - last_i = idx - insert_idx = ( - (last_i + 2) - if (last_i != -1 and last_i + 1 < len(args)) - else (len(args) - 1) - ) - args.insert(insert_idx, "-threads") - args.insert(insert_idx + 1, str(ffmpeg_threads)) - _patch_count += 1 - return original_popen(args, *posargs, **kwargs) - - # MovieLite's compositor calls OpenCV for every frame. Keep it single - # threaded so one render worker cannot exceed its preset CPU budget. - try: - import cv2 - - saved_cv2_threads = cv2.getNumThreads() - cv2.setNumThreads(1) - except Exception: - saved_cv2_threads = None - - saved_affinity = None - if cpu_affinity and hasattr(os, "sched_setaffinity"): - try: - saved_affinity = os.sched_getaffinity(0) - os.sched_setaffinity(0, set(cpu_affinity)) - except (OSError, PermissionError): - saved_affinity = None - - subprocess.Popen = _patched_popen - if niceness > 0 and not self._niceness_applied: - # This is intentionally applied only to Normal. Linux niceness cannot - # be raised again without elevated privileges, but this worker exits - # after generation and all MovieLite children inherit the lower priority. - try: - os.nice(niceness) - self._niceness_applied = True - except (OSError, PermissionError): - pass - - try: - yield - finally: - subprocess.Popen = original_popen - - if saved_cv2_threads is not None: - try: - import cv2 - - cv2.setNumThreads(saved_cv2_threads) - except Exception: - pass - - if saved_affinity is not None and hasattr(os, "sched_setaffinity"): - try: - os.sched_setaffinity(0, saved_affinity) - except (OSError, PermissionError): - pass - - if _patch_count > 0: - print( - f" (patched {_patch_count} ffmpeg calls → threads={ffmpeg_threads})" - ) - else: - print(f" (WARNING: ffmpeg patch did not fire — threads not limited)") - def composite_reel( self, video_paths: List[str], @@ -1185,211 +646,187 @@ def composite_reel( output_path: str, subtitle_list: List[str], ambient_path: Optional[str] = None, - tts_duration: float = 0.0, ) -> str: """ - Composite reel using MovieLite. + Composite reel using MovieLite (4x faster than MoviePy). Concatenates video clips, mixes audio, and overlays subtitles. - Wrapped entirely in _limit_render_resources so every MovieLite stage - respects the selected CPU budget. """ - ffmpeg_threads = self.preset_cfg["ffmpeg_threads"] - cpu_affinity = self.preset_cfg["cpu_affinity"] - niceness = self.preset_cfg["niceness"] - - with self._limit_render_resources(ffmpeg_threads, cpu_affinity, niceness): - # Use pre-computed duration to avoid re-loading the TTS audio - if tts_duration <= 0: - tts_audio_tmp = ml.AudioClip(tts_path) - tts_duration = tts_audio_tmp.duration - tts_audio_tmp.close() - duration = tts_duration - - target_fps = self.preset_cfg["target_fps"] - writer_procs = self.preset_cfg["writer_processes"] - frame_width, frame_height = self.preset_cfg["frame_size"] - quality_str = self.preset_cfg["video_quality"] - video_quality = ( - ml.VideoQuality.HIGH - if quality_str == "high" - else ( - ml.VideoQuality.LOW - if quality_str == "low" - else ml.VideoQuality.MIDDLE - ) + # Get TTS audio duration to determine video length + tts_audio = ml.AudioClip(tts_path) + duration = tts_audio.duration + + # --- Phase 1: Process video clips with smooth playback optimization --- + time_per_clip = duration / len(video_paths) + processed_clips = [] + current_time = 0.0 # Track cumulative time to avoid gaps + + TARGET_FPS = 30 # Standardize all clips to 30fps for smooth playback + + for i, path in enumerate(video_paths): + clip = ml.VideoClip(path) + + # CRITICAL FIX 1: Resize BEFORE setting duration/fps to avoid frame inconsistencies + clip = self._resize_to_portrait(clip) + + # CRITICAL FIX 2: Get the actual source FPS and standardize to TARGET_FPS + # This prevents jitter from FPS mismatches between clips + source_fps = getattr(clip, 'fps', 30) + + # CRITICAL FIX 3: If clip is too short, use looping instead of freezing last frame + # This creates smoother transitions + if clip.duration < time_per_clip: + clip.loop(True) # Enable looping for short clips + + # CRITICAL FIX 4: Set exact duration to prevent gaps/overlaps + clip.set_duration(time_per_clip) + clip.set_start(current_time) + + # Move to next clip's start time + current_time += time_per_clip + + processed_clips.append(clip) + + print(f"{self.GREEN}✓{self.RESET} Processing video clips") + + # --- Phase 2: Setup audio --- + print(f"{self.GREEN}✓{self.RESET} Synchronizing audio") + + # Add TTS audio + tts_audio.set_start(0) + + # Add ambient audio if provided + audio_clips = [tts_audio] + if ambient_path and os.path.exists(ambient_path): + ambient_audio = ml.AudioClip(ambient_path, start=0, volume=0.12) + ambient_audio.set_duration(duration) + ambient_audio.loop(True) # Loop if shorter than TTS + audio_clips.append(ambient_audio) + + # --- Phase 3: Generate and parse subtitles --- + srt_content = self.build_srt(subtitle_list, duration) + srt_path = tts_path.replace(".mp3", ".srt") + with open(srt_path, "w", encoding="utf-8") as fh: + fh.write(srt_content) + + # Parse SRT and create TextClip for each subtitle + subtitle_clips = [] + canvas = self._create_subtitle_canvas() + + for start_time, end_time, text in self.parse_srt(srt_path): + adjusted_start = max(0, start_time - 0.2) + adjusted_end = max(adjusted_start + 0.1, end_time - 0.2) + + text_clip = ml.TextClip( + text, + start=adjusted_start, + duration=adjusted_end - adjusted_start, + canvas=canvas ) - # --- Phase 1: Process video clips --- - time_per_clip = duration / len(video_paths) - processed_clips = [] - current_time = 0.0 - - for i, path in enumerate(video_paths): - clip = ml.VideoClip(path) - clip = self._resize_to_portrait(clip, (frame_width, frame_height)) - - if clip.duration < time_per_clip: - clip.loop(True) - - clip.set_duration(time_per_clip) - clip.set_start(current_time) - current_time += time_per_clip - processed_clips.append(clip) - - print(f"{self.GREEN}✓{self.RESET} Processing video clips") - - # --- Phase 2: Setup audio --- - print(f"{self.GREEN}✓{self.RESET} Synchronizing audio") - - tts_audio = ml.AudioClip(tts_path) - tts_audio.set_start(0) - - audio_clips = [tts_audio] - if ambient_path and os.path.exists(ambient_path): - ambient_audio = ml.AudioClip(ambient_path, start=0, volume=0.12) - ambient_audio.set_duration(duration) - ambient_audio.loop(True) - audio_clips.append(ambient_audio) - - # --- Phase 3: Generate subtitles in memory (no disk write) --- - srt_content = self.build_srt(subtitle_list, duration) - subtitle_entries = self._parse_srt_content(srt_content) - - subtitle_clips = [] - canvas = self._create_subtitle_canvas(frame_width) - - for start_time, end_time, text in subtitle_entries: - adjusted_start = max(0, start_time - 0.2) - adjusted_end = max(adjusted_start + 0.1, end_time - 0.2) - - text_clip = ml.TextClip( - text, - start=adjusted_start, - duration=adjusted_end - adjusted_start, - canvas=canvas, - ) - - text_width = text_clip.size[0] - text_clip.set_position( - ((frame_width - text_width) // 2, int(frame_height * 0.86)) - ) - subtitle_clips.append(text_clip) - - print( - f"{self.GREEN}✓{self.RESET} Rendering {len(subtitle_clips)} subtitle segments\n" + text_width = text_clip.size[0] + text_clip.set_position(((1080 - text_width) // 2, 1650)) + subtitle_clips.append(text_clip) + + print(f"{self.GREEN}✓{self.RESET} Rendering {len(subtitle_clips)} subtitle segments\n") + + # --- Phase 5: Export --- + self._print_step(5, 5, "Exporting final video") + + with self._spinner("Exporting..."): + # Create writer with optimized settings for smooth playback + writer = ml.VideoWriter( + output_path, + fps=30, + size=(1080, 1920), + duration=duration ) - - # --- Phase 4: Export --- - self._print_step(5, 5, "Exporting final video") - - with self._spinner("Exporting..."): - writer = ml.VideoWriter( - output_path, - fps=target_fps, - size=(frame_width, frame_height), - duration=duration, - ) - - for clip in processed_clips: - writer.add_clip(clip) - for audio_clip in audio_clips: - writer.add_clip(audio_clip) - for sub_clip in subtitle_clips: - writer.add_clip(sub_clip) - - writer.write(processes=writer_procs, video_quality=video_quality) - - print(f"{self.GREEN}✓{self.RESET} Reel generated successfully") - - # Cleanup all held resources - all_clips = processed_clips + audio_clips + subtitle_clips - for clip in all_clips: - try: - if hasattr(clip, "close"): - clip.close() - except Exception: - pass - - return output_path + + for clip in processed_clips: + writer.add_clip(clip) + for audio_clip in audio_clips: + writer.add_clip(audio_clip) + for sub_clip in subtitle_clips: + writer.add_clip(sub_clip) + + writer.write(processes=4, video_quality=ml.VideoQuality.HIGH) + + print(f"{self.GREEN}✓{self.RESET} Reel generated successfully") + + # Cleanup - MovieLite clips have close() method, but it's optional + # They auto-cleanup when garbage collected + try: + for clip in processed_clips: + if hasattr(clip, 'close'): + clip.close() + except Exception: + pass # Ignore cleanup errors + + return output_path # ----------------------------------------------------------------------- # Main pipeline # ----------------------------------------------------------------------- # ANSI color codes - CYAN = "\033[96m" - GREEN = "\033[92m" - RED = "\033[91m" - RESET = "\033[0m" + CYAN = '\033[96m' + GREEN = '\033[92m' + RED = '\033[91m' + RESET = '\033[0m' # Braille spinner for multi-stage loading - BRAILLE_CHARS = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"] + BRAILLE_CHARS = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏'] def _print_step(self, step: int, total: int, desc: str): """Print step header in DNF style.""" print(f"[{step}/{total}] {desc}") - def _progress_bar_dnf( - self, current: int, total: int, desc: str, color: str = "" - ) -> str: + def _progress_bar_dnf(self, current: int, total: int, desc: str, color: str = '') -> str: """Generate DNF-style progress bar with proper vertical alignment.""" width = 20 percentage = (current / total * 100) if total > 0 else 0 filled = int(width * current / total) if total > 0 else 0 - bar = "━" * filled + " " * (width - filled) + bar = '━' * filled + ' ' * (width - filled) return f"{desc:<25} {color}{percentage:3.0f}% |{bar}| {current}/{total}{self.RESET}" - - def generate_reel( - self, - job_id: str, - emotion: str, - context: Dict[str, Any], - header: bool = True, - output_filename: Optional[str] = None, - ) -> Dict[str, Any]: + + def generate_reel(self, job_id: str, emotion: str, context: Dict[str, Any], header: bool = True) -> Dict[str, Any]: result: Dict[str, Any] = { "success": False, "job_id": job_id, "reel_path": None, "script": None, "keywords": None, - "preset": self.preset, "error": None, } try: # Phase 1: Script generation self._print_step(1, 5, "Generating personalized script") - provider_name = "Groq" if self.script_provider == "groq" else "Gemini" - print(f"Provider : {provider_name} ({self.script_model})") - + print(f"Provider : Gemini ({self.script_model})") + with self._spinner("Generating script..."): - script_data = self.generate_script(emotion, context) - script = script_data["script"] + script_data = self.generate_script(emotion, context) + script = script_data["script"] subtitle_list = script_data["subtitles"] result["script"] = script - + desc = f"{'Script generated':<26}" - print( - f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET}\n" - ) + print(f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET}\n") # Phase 2: Video search self._print_step(2, 5, "Finding supporting visuals") print(f"Provider : Pexels") - + with self._spinner("Searching videos..."): keywords = self.extract_video_keywords(script, emotion) result["keywords"] = keywords if not keywords: raise RuntimeError("Could not extract video keywords from script") - + def _truncate(kw, maxlen=20): - return kw if len(kw) <= maxlen else kw[: maxlen - 1] + "…" - + return kw if len(kw) <= maxlen else kw[:maxlen-1] + "…" keywords_display = " • ".join(_truncate(kw) for kw in keywords[:4]) print(f"Keywords : {keywords_display}") - + video_paths = self.download_videos_for_script(keywords, job_id) if not video_paths: raise RuntimeError("No videos could be downloaded") @@ -1398,46 +835,36 @@ def _truncate(kw, maxlen=20): # Phase 3: TTS generation self._print_step(3, 5, "Generating narration") print(f"Provider : MiMo (Dean)") - + with self._spinner("Generating narration..."): tts_path = os.path.join(self.output_dir, "audio", f"{job_id}.mp3") self.generate_tts(script, tts_path) - - # Compute TTS duration once (avoids reloading in composite_reel) - tts_duration = 0.0 + + # Calculate audio duration for display try: audio = ml.AudioClip(tts_path) - tts_duration = audio.duration - duration_display = int(tts_duration) + duration = int(audio.duration) audio.close() desc = f"{'Narration generated':<26}" - print( - f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET} ({duration_display}s)\n" - ) + print(f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET} ({duration}s)\n") except: desc = f"{'Narration generated':<26}" - print( - f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET}\n" - ) + print(f"{self.GREEN}✓{self.RESET} {desc}{self.GREEN}100% |{'━' * 20}| 1/1{self.RESET}\n") + # Phase 4: Timeline preparation self._print_step(4, 5, "Preparing final composition") - self._configure_calibrated_workers(video_paths[0]) - reel_name = output_filename if output_filename else f"{job_id}.mp4" - reel_path = os.path.join(self.output_dir, "reels", reel_name) - ambient_path = self.ambient_music.get( - emotion, self.ambient_music.get("neutral") - ) + reel_path = os.path.join(self.output_dir, "reels", f"{job_id}.mp4") + ambient_path = self.ambient_music.get(emotion, self.ambient_music.get("neutral")) self.composite_reel( video_paths=video_paths, tts_path=tts_path, output_path=reel_path, subtitle_list=subtitle_list, ambient_path=ambient_path, - tts_duration=tts_duration, ) result["reel_path"] = reel_path - result["success"] = True + result["success"] = True # Clean up temp video clips for p in video_paths: @@ -1449,7 +876,6 @@ def _truncate(kw, maxlen=20): except Exception as exc: result["error"] = str(exc) import traceback - traceback.print_exc() return result @@ -1459,42 +885,23 @@ def _truncate(kw, maxlen=20): # CLI entry point # --------------------------------------------------------------------------- - def main() -> int: import argparse + print("MindStream — Phase 3: Reel Generation\n") + print("Checking environment...") + parser = argparse.ArgumentParser(description="MindStream Reel Generator") - parser.add_argument("--job-id", required=False, help="Job ID") - parser.add_argument("--emotion", required=False, help="Emotion label") - parser.add_argument("--context", required=False, help="Context JSON string") - parser.add_argument("--output-filename", required=False, help="Output MP4 filename") - parser.add_argument( - "--preset", - required=False, - default="normal", - choices=list(PRESETS.keys()), - help="Speed/resource preset (default: normal)", - ) - parser.add_argument( - "--recalibrate-presets", - action="store_true", - help="Ignore cached Fast calibration and measure worker scaling again", - ) + parser.add_argument("--job-id", required=False, help="Job ID") + parser.add_argument("--emotion", required=False, help="Emotion label") + parser.add_argument("--context", required=False, help="Context JSON string") args = parser.parse_args() - scoped_exit_code = _run_in_cpu_limited_scope(args.preset) - if scoped_exit_code is not None: - return scoped_exit_code - - print("MindStream — Reel Generation\n") - try: - gen = ReelGenerator( - preset=args.preset, recalibrate_presets=args.recalibrate_presets - ) - print(f"Preset: {args.preset.title()}\n") + gen = ReelGenerator() + print(f"{gen.GREEN}✓{gen.RESET} Environment ready\n") except ValueError as e: - print("\033[91m✗\033[0m Environment check failed\n") + print(f"{gen.RED}✗{gen.RESET} Environment check failed\n") print(f"Error: {e}", file=sys.stderr) return 1 @@ -1506,12 +913,8 @@ def main() -> int: except json.JSONDecodeError as e: print(f"Invalid --context JSON: {e}", file=sys.stderr) return 1 - print( - f"Job : {args.job_id} | {args.emotion.title()} | {ctx.get('user_name', 'User')}\n" - ) - result = gen.generate_reel( - job_id=args.job_id, emotion=args.emotion, context=ctx, header=False, output_filename=args.output_filename - ) + print(f"Job : {args.job_id} | {args.emotion.title()} | {ctx.get('user_name', 'User')}\n") + result = gen.generate_reel(job_id=args.job_id, emotion=args.emotion, context=ctx, header=False) else: sample = "data/sample_emotion_result.json" if not os.path.exists(sample): @@ -1520,14 +923,12 @@ def main() -> int: print(f"Input : {sample}") with open(sample) as fh: data = json.load(fh) - print( - f"Job : {data['job_id']} | {data['emotion']['label'].title()} | {data['context'].get('user_name', 'User')}\n" - ) + print(f"Job : {data['job_id']} | {data['emotion']['label'].title()} | {data['context'].get('user_name', 'User')}\n") result = gen.generate_reel( job_id=data["job_id"], emotion=data["emotion"]["label"], context=data["context"], - header=False, + header=False ) elapsed = time.time() - start_time @@ -1543,6 +944,5 @@ def main() -> int: print(f"Error: {result['error']}", file=sys.stderr) return 1 - if __name__ == "__main__": - sys.exit(main()) + sys.exit(main()) \ No newline at end of file diff --git a/backend/requirements.txt b/backend/requirements.txt old mode 100755 new mode 100644 diff --git a/backend/server.js b/backend/server.js old mode 100755 new mode 100644 index c4bcb05..0560889 --- a/backend/server.js +++ b/backend/server.js @@ -6,24 +6,6 @@ const os = require('os'); const { spawn } = require('child_process'); const chokidar = require('chokidar'); -// Load environment variables from backend/.env if present so process.env has keys -const envPath = path.join(__dirname, '.env'); -if (fs.existsSync(envPath)) { - const envConfig = fs.readFileSync(envPath, 'utf8'); - for (const line of envConfig.split('\n')) { - const trimmed = line.trim(); - if (!trimmed || trimmed.startsWith('#')) continue; - const eqIdx = trimmed.indexOf('='); - if (eqIdx > 0) { - const key = trimmed.slice(0, eqIdx).trim(); - const value = trimmed.slice(eqIdx + 1).trim().replace(/^["']|["']$/g, ''); - if (key && !process.env[key]) { - process.env[key] = value; - } - } - } -} - const app = express(); app.use(cors()); app.use(express.json()); @@ -31,14 +13,6 @@ app.use(express.json()); // In-memory job state tracker const jobs = {}; -// Active single job tracking -let activeJobId = null; - -const REEL_PRESET = process.env.MINDSTREAM_REEL_PRESET || 'normal'; -if (!['normal', 'fast'].includes(REEL_PRESET)) { - throw new Error("MINDSTREAM_REEL_PRESET must be 'normal' or 'fast'"); -} - // Cache for results written before check-in call finishes (to avoid race conditions) const pendingResults = {}; @@ -53,59 +27,10 @@ try { console.error(`[server] Failed to create capture folder: ${err.message}`); } -/** - * Safely cancels a running or pending job and terminates its spawned Python process. - */ -function cancelJob(jobId, reason = 'Job cancelled by user') { - const job = jobs[jobId]; - if (!job) return; - - if (job.status === 'cancelled') return; - - console.log(`[server] Cancelling Job ${jobId}: ${reason}`); - - if (job.process) { - try { - job.process.kill('SIGTERM'); - } catch (err) { - console.error(`[server] Error killing process for Job ${jobId}: ${err.message}`); - } - job.process = null; - } - - job.status = 'cancelled'; - job.cancelled_at = new Date().toISOString(); - job.error = reason; - - if (activeJobId === jobId) { - activeJobId = null; - } -} - // Helper to run reel generator Python script -function triggerReelGeneration(jobId, emotion, context, preset) { - const job = jobs[jobId]; - if (!job || job.status === 'cancelled') { - console.log(`[server] Skipping generation for cancelled/missing Job ${jobId}`); - return; - } - - // Enforce single active job: cancel any other running job - if (activeJobId && activeJobId !== jobId && jobs[activeJobId] && ['processing_emotion', 'processing_reel'].includes(jobs[activeJobId].status)) { - cancelJob(activeJobId, 'Superseded by new job generation'); - } - - activeJobId = jobId; - const selectedPreset = preset || job.preset || REEL_PRESET; - console.log(`[server] Spawning ${selectedPreset} reel worker for Job ${jobId} (Emotion: ${emotion})...`); - job.status = 'processing_reel'; - - // Construct clean, human-readable reel filename - const now = new Date(); - const dateStr = now.toISOString().slice(0, 10); - const timeStr = now.toTimeString().slice(0, 8).replace(/:/g, '-'); - const cleanFilename = `mindstream_${dateStr}_${timeStr}_${emotion.toLowerCase()}.mp4`; - job.reel_filename = cleanFilename; +function triggerReelGeneration(jobId, emotion, context) { + console.log(`[server] Spawning reel worker for Job ${jobId} (Emotion: ${emotion})...`); + jobs[jobId].status = 'processing_reel'; const contextStr = JSON.stringify(context); const pythonPath = path.join(__dirname, 'venv', 'bin', 'python'); @@ -115,44 +40,29 @@ function triggerReelGeneration(jobId, emotion, context, preset) { scriptPath, '--job-id', jobId, '--emotion', emotion, - '--context', contextStr, - '--preset', selectedPreset, - '--output-filename', cleanFilename + '--context', contextStr ], { - cwd: __dirname, - // PYTHONUNBUFFERED=1 forces Python stdout/stderr to flush immediately line-by-line - env: { ...process.env, PYTHONUNBUFFERED: '1' } + cwd: __dirname }); - job.process = worker; - - // Stream python stdout directly to terminal without prepending [reel:xxxx] + // Pipe Python stdout straight to the server terminal so you can follow progress worker.stdout.on('data', (data) => { - if (job.status === 'cancelled') return; - process.stdout.write(data); + process.stdout.write(`[reel:${jobId.slice(0,8)}] ${data}`); }); let stderrBuf = ''; worker.stderr.on('data', (data) => { - if (job.status === 'cancelled') return; stderrBuf += data.toString(); - process.stderr.write(data); + // Also print stderr live so tracebacks appear immediately + process.stderr.write(`[reel:${jobId.slice(0,8)}:ERR] ${data}`); }); worker.on('close', (code) => { - job.process = null; - - if (job.status === 'cancelled') { - console.log(`[server] Job ${jobId} worker process terminated (cancelled).`); - return; - } - if (code === 0) { - console.log(`[server] ✓ Reel ready for Job ${jobId} -> ${cleanFilename}`); + console.log(`[server] ✓ Reel ready for Job ${jobId}`); jobs[jobId] = { - ...jobs[jobId], status: 'ready', - reel_url: `http://localhost:4000/reels/${cleanFilename}`, + reel_url: `http://localhost:4000/reels/${jobId}.mp4`, emotion_label: emotion, completed_at: new Date().toISOString() }; @@ -160,42 +70,29 @@ function triggerReelGeneration(jobId, emotion, context, preset) { const snippet = stderrBuf.slice(-400); console.error(`[server] ✗ Reel worker exited with code ${code}`); jobs[jobId] = { - ...jobs[jobId], status: 'failed', error: `Worker exited ${code}: ${snippet}`, completed_at: new Date().toISOString() }; } - - if (activeJobId === jobId) { - activeJobId = null; - } }); } // POST /check-in app.post('/check-in', (req, res) => { - const { session_id, context, clip_path, preset } = req.body; + const { session_id, context, clip_path } = req.body; if (!session_id) { return res.status(400).json({ error: 'Missing session_id' }); } - // Cancel any existing running job to ensure single active generation session - if (activeJobId && activeJobId !== session_id && jobs[activeJobId] && ['processing_emotion', 'processing_reel'].includes(jobs[activeJobId].status)) { - cancelJob(activeJobId, 'Superseded by new check-in session'); - } - - activeJobId = session_id; - const selectedPreset = preset || REEL_PRESET; - console.log(`[server] New check-in request. Session ID: ${session_id}, Preset: ${selectedPreset}, Clip: ${clip_path}`); + console.log(`[server] New check-in request. Session ID: ${session_id}, Clip: ${clip_path}`); + // Create job entry jobs[session_id] = { status: 'processing_emotion', context: context || {}, clip_path: clip_path || null, - preset: selectedPreset, - created_at: new Date().toISOString(), - process: null + created_at: new Date().toISOString() }; // Check if we already received the emotion detection result for this clip file @@ -207,32 +104,14 @@ app.post('/check-in', (req, res) => { delete pendingResults[baseName]; if (result.emotion && result.emotion.label) { - triggerReelGeneration(session_id, result.emotion.label, context, selectedPreset); - res.json({ job_id: session_id }); - return; + triggerReelGeneration(session_id, result.emotion.label, context); } else { jobs[session_id].status = 'failed'; jobs[session_id].error = result.error || 'Emotion detection failed'; - res.json({ job_id: session_id }); - return; } } } - // Development Fallback: If no real Phase 2 (friend's script) writes a _result.json - // within 2 seconds, auto-generate a fallback emotion so testing works end-to-end. - if (process.env.MINDSTREAM_MOCK_EMOTION !== 'false') { - setTimeout(() => { - const job = jobs[session_id]; - if (job && job.status === 'processing_emotion') { - const mockEmotions = ['neutral', 'anxious', 'fatigued', 'distracted', 'frustrated']; - const fallbackEmotion = mockEmotions[Math.floor(Math.random() * mockEmotions.length)]; - console.log(`[server] [Phase 2 Simulation] No Phase 2 result JSON detected yet. Auto-generating mock emotion '${fallbackEmotion}' for Job ${session_id}...`); - triggerReelGeneration(session_id, fallbackEmotion, context, selectedPreset); - } - }, 2000); - } - res.json({ job_id: session_id }); }); @@ -242,35 +121,10 @@ app.get('/jobs/:id', (req, res) => { if (!job) { return res.status(404).json({ error: 'Job not found' }); } - const { process: _proc, ...jobData } = job; - res.json(jobData); -}); - -// POST /jobs/:id/cancel -app.post('/jobs/:id/cancel', (req, res) => { - const jobId = req.params.id; - const job = jobs[jobId]; - if (!job) { - return res.status(404).json({ error: 'Job not found' }); - } - cancelJob(jobId, 'Cancelled via API request'); - res.json({ status: 'cancelled', job_id: jobId }); -}); - -// GET /health — returns server status and which required API keys are configured. -app.get('/health', (req, res) => { - res.json({ - status: 'ok', - keys: { - gemini: !!process.env.GEMINI_API_KEY, - pexels: !!process.env.PEXELS_API_KEY, - mimo: !!process.env.MIMO_API_KEY, - groq: !!process.env.GROQ_API_KEY, // optional - pixabay: !!process.env.PIXABAY_API_KEY, // optional - }, - }); + res.json(job); }); +// Serve compiled reel assets app.use('/reels', express.static(path.join(__dirname, 'output', 'reels'))); // Start the Chokidar directory watcher (watching for Phase 2 result JSON files) @@ -279,29 +133,28 @@ chokidar.watch(CAPTURE_FOLDER, { ignored: /capture_.*\.webm$/ }) if (!filePath.endsWith('_result.json')) return; console.log(`[server] Detected new result JSON file: ${filePath}`); - const baseName = path.basename(filePath, '_result.json'); + const baseName = path.basename(filePath, '_result.json'); // e.g. capture_2026-07-18T10-45-00 try { const fileContent = fs.readFileSync(filePath, 'utf-8'); const result = JSON.parse(fileContent); + // Find the corresponding check-in job const jobId = Object.keys(jobs).find(id => { const job = jobs[id]; return job.clip_path && job.clip_path.includes(baseName); }); if (jobId) { - const job = jobs[jobId]; - if (job && job.status !== 'cancelled') { - console.log(`[server] Found matching job ${jobId} for result: ${baseName}`); - if (result.emotion && result.emotion.label) { - triggerReelGeneration(jobId, result.emotion.label, job.context, job.preset); - } else { - job.status = 'failed'; - job.error = result.error || 'Emotion detection failed'; - } + console.log(`[server] Found matching job ${jobId} for result: ${baseName}`); + if (result.emotion && result.emotion.label) { + triggerReelGeneration(jobId, result.emotion.label, jobs[jobId].context); + } else { + jobs[jobId].status = 'failed'; + jobs[jobId].error = result.error || 'Emotion detection failed'; } } else { + // Cache it, in case the extension check-in payload POST hasn't completed yet console.log(`[server] Job not found for result: ${baseName}. Pre-caching result...`); pendingResults[baseName] = result; } diff --git a/backend/test.sh b/backend/test.sh index cfd6d23..35ca219 100755 --- a/backend/test.sh +++ b/backend/test.sh @@ -3,8 +3,6 @@ # # Usage: # ./test.sh # Run with default sample (data/sample_emotion_result.json) -# ./test.sh --preset fast # Run the same sample with the Fast preset -# ./test.sh --preset fast --recalibrate-presets # Re-measure Fast worker scaling # ./test.sh --emotion anxious --context '{"active_tab_category":"social_media",...}' # # How it works: @@ -30,9 +28,9 @@ fi source venv/bin/activate -echo "Preparing backend..." +echo "Checking environment..." pip install -q -r requirements.txt 2>/dev/null || pip install -r requirements.txt 2>/dev/null -echo -e "\033[92m✓\033[0m Ready" +echo -e "\033[92m✓\033[0m Environment ready" # # --- API keys (set via env or use defaults) --- # if [ -z "$GEMINI_API_KEY" ]; then @@ -50,7 +48,7 @@ echo -e "\033[92m✓\033[0m Ready" # --- Run --- if [ $# -gt 0 ]; then # Custom args passed — forward to reel_generator.py - # echo "Running sample with selected options..." + echo "Running with custom args: $@" python reel_generator.py "$@" else # No args — run with default sample data diff --git a/backend/test_presets.py b/backend/test_presets.py deleted file mode 100755 index d4f6471..0000000 --- a/backend/test_presets.py +++ /dev/null @@ -1,132 +0,0 @@ -"""Regression tests for MindStream's hardware-aware render presets.""" - -import unittest -from unittest.mock import patch - -from reel_generator import ( - CALIBRATION_WORKER_MEMORY_BYTES, - HardwareProfile, - PRESETS, - _candidate_worker_counts, - _normal_cpu_quota_percent, - _resolve_preset, -) - - -class PresetResolutionTests(unittest.TestCase): - def test_only_normal_and_fast_are_available(self): - self.assertEqual(set(PRESETS), {"normal", "fast"}) - - def test_normal_is_scheduler_managed_background_work(self): - hardware = HardwareProfile( - logical_cpus=(0, 1, 2, 3, 4, 5, 6, 7), - physical_core_groups=((0, 4), (1, 5), (2, 6), (3, 7)), - ) - - normal = _resolve_preset("normal", hardware) - - self.assertEqual(normal["writer_processes"], 1) - self.assertEqual(normal["ffmpeg_threads"], 1) - self.assertEqual(normal["cpu_affinity"], ()) - self.assertGreater(normal["niceness"], 0) - self.assertEqual(normal["cpu_quota_percent"], 150) - self.assertEqual(normal["frame_size"], (720, 1280)) - self.assertEqual(normal["target_fps"], 24) - - def test_fast_starts_with_two_workers_when_hardware_has_headroom(self): - hardware = HardwareProfile( - logical_cpus=tuple(range(16)), - physical_core_groups=tuple((i, i + 8) for i in range(8)), - ) - - with patch( - "reel_generator._available_memory_bytes", - return_value=3 * CALIBRATION_WORKER_MEMORY_BYTES, - ): - fast = _resolve_preset("fast", hardware) - - self.assertEqual(fast["writer_processes"], 2) - self.assertEqual(fast["ffmpeg_threads"], 2) - self.assertEqual(fast["cpu_affinity"], ()) - self.assertEqual(fast["niceness"], 0) - self.assertIsNone(fast["cpu_quota_percent"]) - self.assertEqual(fast["frame_size"], (720, 1280)) - - def test_fast_degrades_safely_on_one_core(self): - hardware = HardwareProfile(logical_cpus=(0,), physical_core_groups=((0,),)) - - with patch( - "reel_generator._available_memory_bytes", - return_value=3 * CALIBRATION_WORKER_MEMORY_BYTES, - ): - fast = _resolve_preset("fast", hardware) - - self.assertEqual(fast["writer_processes"], 1) - self.assertEqual(fast["ffmpeg_threads"], 1) - self.assertEqual(fast["cpu_affinity"], ()) - - def test_fast_leaves_a_core_available_on_two_core_hardware(self): - hardware = HardwareProfile( - logical_cpus=(0, 1, 2, 3), physical_core_groups=((0, 2), (1, 3)) - ) - - with patch( - "reel_generator._available_memory_bytes", - return_value=3 * CALIBRATION_WORKER_MEMORY_BYTES, - ): - fast = _resolve_preset("fast", hardware) - - self.assertEqual(fast["writer_processes"], 1) - self.assertEqual(fast["cpu_affinity"], ()) - - def test_fast_reduces_its_first_run_fallback_when_memory_is_tight(self): - hardware = HardwareProfile( - logical_cpus=tuple(range(8)), - physical_core_groups=((0, 4), (1, 5), (2, 6), (3, 7)), - ) - - with patch( - "reel_generator._available_memory_bytes", - return_value=CALIBRATION_WORKER_MEMORY_BYTES, - ): - fast = _resolve_preset("fast", hardware) - - self.assertEqual(fast["writer_processes"], 1) - self.assertEqual(fast["cpu_affinity"], ()) - - def test_calibration_candidates_reserve_a_core_and_respect_memory(self): - hardware = HardwareProfile( - logical_cpus=tuple(range(8)), - physical_core_groups=((0, 4), (1, 5), (2, 6), (3, 7)), - ) - - with patch( - "reel_generator._available_memory_bytes", - return_value=3 * CALIBRATION_WORKER_MEMORY_BYTES, - ): - self.assertEqual(_candidate_worker_counts(hardware), (1, 2, 3)) - - with patch( - "reel_generator._available_memory_bytes", - return_value=CALIBRATION_WORKER_MEMORY_BYTES, - ): - self.assertEqual(_candidate_worker_counts(hardware), (1,)) - - def test_normal_quota_scales_down_for_smaller_hardware(self): - one_core = HardwareProfile(logical_cpus=(0,), physical_core_groups=((0,),)) - two_cores = HardwareProfile( - logical_cpus=(0, 1, 2, 3), physical_core_groups=((0, 2), (1, 3)) - ) - - self.assertEqual(_normal_cpu_quota_percent(one_core), 100) - self.assertEqual(_normal_cpu_quota_percent(two_cores), 100) - - def test_presets_use_the_same_output_profile(self): - self.assertEqual(PRESETS["fast"]["frame_size"], PRESETS["normal"]["frame_size"]) - self.assertEqual(PRESETS["fast"]["target_fps"], PRESETS["normal"]["target_fps"]) - self.assertEqual(PRESETS["normal"]["video_quality"], "middle") - self.assertEqual(PRESETS["fast"]["video_quality"], "low") - - -if __name__ == "__main__": - unittest.main() diff --git a/eslint.config.js b/eslint.config.js old mode 100755 new mode 100644 diff --git a/index.html b/index.html old mode 100755 new mode 100644 diff --git a/opencode.json b/opencode.json deleted file mode 100755 index 6414a62..0000000 --- a/opencode.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "$schema": "https://opencode.ai/config.json", - "mcp": { - "motion": { - "type": "remote", - "url": "https://mcp.motion.dev", - "enabled": true - }, - "motion-plus": { - "type": "remote", - "url": "https://mcp.motion.dev/plus", - "enabled": true - } - } -} \ No newline at end of file diff --git a/package-lock.json b/package-lock.json old mode 100755 new mode 100644 index c2ac2c1..dd3e479 --- a/package-lock.json +++ b/package-lock.json @@ -9,8 +9,6 @@ "version": "0.0.0", "dependencies": { "@tailwindcss/vite": "^4.3.2", - "lucide-react": "^1.27.0", - "motion": "^12.43.0", "react": "^19.2.7", "react-dom": "^19.2.7", "tailwindcss": "^4.3.2" @@ -1246,16 +1244,16 @@ } }, "node_modules/brace-expansion": { - "version": "5.0.9", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz", - "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==", + "version": "5.0.7", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz", + "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==", "dev": true, "license": "MIT", "dependencies": { "balanced-match": "^4.0.2" }, "engines": { - "node": "20 || >=22" + "node": "18 || 20 || >=22" } }, "node_modules/browserslist": { @@ -1693,33 +1691,6 @@ "dev": true, "license": "ISC" }, - "node_modules/framer-motion": { - "version": "12.43.0", - "resolved": "https://registry.npmjs.org/framer-motion/-/framer-motion-12.43.0.tgz", - "integrity": "sha512-1eaL3RvR/kAlbG7UYcpMptEyzPoENO0c6w7ZnB3/hh2vSAz/6uGAFn6fdoqTBguNstf3MsFhJHsD/0DHiclG+g==", - "license": "MIT", - "dependencies": { - "motion-dom": "^12.43.0", - "motion-utils": "^12.39.0", - "tslib": "^2.4.0" - }, - "peerDependencies": { - "@emotion/is-prop-valid": "*", - "react": "^18.0.0 || ^19.0.0", - "react-dom": "^18.0.0 || ^19.0.0" - }, - "peerDependenciesMeta": { - "@emotion/is-prop-valid": { - "optional": true - }, - "react": { - "optional": true - }, - "react-dom": { - "optional": true - } - } - }, "node_modules/fsevents": { "version": "2.3.3", "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", @@ -2205,15 +2176,6 @@ "yallist": "^3.0.2" } }, - "node_modules/lucide-react": { - "version": "1.27.0", - "resolved": "https://registry.npmjs.org/lucide-react/-/lucide-react-1.27.0.tgz", - "integrity": "sha512-rJicGl/3Fly/E0rOH1YmPZ6e49JCnKknh1ox1vpHnkfjujAkKA6sqUZvH3MTAaXXjgexyUwgNwTJzTtYuAFYJw==", - "license": "ISC", - "peerDependencies": { - "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" - } - }, "node_modules/magic-string": { "version": "0.30.21", "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", @@ -2239,47 +2201,6 @@ "url": "https://github.com/sponsors/isaacs" } }, - "node_modules/motion": { - "version": "12.43.0", - "resolved": "https://registry.npmjs.org/motion/-/motion-12.43.0.tgz", - "integrity": "sha512-BQgQbSa9Hn3/mtbib0MK53y6JSANa+YKUKlaYnWzAVDH424RYQ5LVpV3pNiWH00BA2z4ojsSdMzqT7g2FQwjuQ==", - "license": "MIT", - "dependencies": { - "framer-motion": "^12.43.0", - "tslib": "^2.4.0" - }, - "peerDependencies": { - "@emotion/is-prop-valid": "*", - "react": "^18.0.0 || ^19.0.0", - "react-dom": "^18.0.0 || ^19.0.0" - }, - "peerDependenciesMeta": { - "@emotion/is-prop-valid": { - "optional": true - }, - "react": { - "optional": true - }, - "react-dom": { - "optional": true - } - } - }, - "node_modules/motion-dom": { - "version": "12.43.0", - "resolved": "https://registry.npmjs.org/motion-dom/-/motion-dom-12.43.0.tgz", - "integrity": "sha512-azKON4d9S65PEoFUiQTMTgPheEmzf2QngdRc50AKfJp9Q9mmcBVw22c8eMq9k8kxOFHdL7+WZY7N/5F/lwiDag==", - "license": "MIT", - "dependencies": { - "motion-utils": "^12.39.0" - } - }, - "node_modules/motion-utils": { - "version": "12.39.0", - "resolved": "https://registry.npmjs.org/motion-utils/-/motion-utils-12.39.0.tgz", - "integrity": "sha512-8nadJAJjTtqRkmRF36FoJTrywK9nnFmnPwnSMyxaOCU7GDjN9RTMJIxx9De8ErM+vpPhMccr/6fo5WciyQLnMQ==", - "license": "MIT" - }, "node_modules/ms": { "version": "2.1.3", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", @@ -2288,9 +2209,9 @@ "license": "MIT" }, "node_modules/nanoid": { - "version": "3.3.18", - "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.18.tgz", - "integrity": "sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w==", + "version": "3.3.15", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.15.tgz", + "integrity": "sha512-y7Wygv/7mEOvxTuEQDB8StXdMRBWf1kR/tlhAzBRUFkB2jfcLOAxO/SHmOO2zgz1pVgK29/kyupn059/bCHdjA==", "funding": [ { "type": "github", @@ -2411,9 +2332,9 @@ } }, "node_modules/postcss": { - "version": "8.5.26", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.26.tgz", - "integrity": "sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ==", + "version": "8.5.16", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.16.tgz", + "integrity": "sha512-vuwillviilfKZsg0VGj5R/YwwcHx4SLsIOI/7K6mQkWx+l5cUHTjj5g0AasTBcyXsbfTgrwsUNmVUb5xVwyPwg==", "funding": [ { "type": "opencollective", @@ -2430,7 +2351,7 @@ ], "license": "MIT", "dependencies": { - "nanoid": "^3.3.17", + "nanoid": "^3.3.12", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" }, @@ -2599,7 +2520,8 @@ "version": "2.8.1", "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", - "license": "0BSD" + "license": "0BSD", + "optional": true }, "node_modules/type-check": { "version": "0.4.0", diff --git a/package.json b/package.json old mode 100755 new mode 100644 index cfb0716..7ab3432 --- a/package.json +++ b/package.json @@ -11,8 +11,6 @@ }, "dependencies": { "@tailwindcss/vite": "^4.3.2", - "lucide-react": "^1.27.0", - "motion": "^12.43.0", "react": "^19.2.7", "react-dom": "^19.2.7", "tailwindcss": "^4.3.2" diff --git a/public/assets/logo.png b/public/assets/logo.png deleted file mode 100755 index 302efc6..0000000 Binary files a/public/assets/logo.png and /dev/null differ diff --git a/public/icons/icon-128.png b/public/icons/icon-128.png old mode 100755 new mode 100644 index 2b255ec..12036db Binary files a/public/icons/icon-128.png and b/public/icons/icon-128.png differ diff --git a/public/icons/icon-16.png b/public/icons/icon-16.png new file mode 100644 index 0000000..186af1d Binary files /dev/null and b/public/icons/icon-16.png differ diff --git a/public/icons/icon-256.png b/public/icons/icon-256.png new file mode 100644 index 0000000..ca94a00 Binary files /dev/null and b/public/icons/icon-256.png differ diff --git a/public/icons/icon-32.png b/public/icons/icon-32.png new file mode 100644 index 0000000..051d4c2 Binary files /dev/null and b/public/icons/icon-32.png differ diff --git a/public/icons/icon-48.png b/public/icons/icon-48.png old mode 100755 new mode 100644 index 6ff6d61..862d163 Binary files a/public/icons/icon-48.png and b/public/icons/icon-48.png differ diff --git a/public/icons/icon-512.png b/public/icons/icon-512.png old mode 100755 new mode 100644 diff --git a/public/icons/icon-96.png b/public/icons/icon-96.png deleted file mode 100755 index d653aee..0000000 Binary files a/public/icons/icon-96.png and /dev/null differ diff --git a/public/manifest.json b/public/manifest.json old mode 100755 new mode 100644 index 9150ac9..7ef070a --- a/public/manifest.json +++ b/public/manifest.json @@ -7,8 +7,9 @@ "action": { "default_title": "Mind Stream", "default_icon": { + "16": "icons/icon-16.png", + "32": "icons/icon-32.png", "48": "icons/icon-48.png", - "96": "icons/icon-96.png", "128": "icons/icon-128.png" } }, @@ -16,14 +17,7 @@ "service_worker": "background.js", "type": "module" }, - "permissions": [ - "sidePanel", - "storage", - "alarms", - "notifications", - "tabs", - "downloads" - ], + "permissions": ["sidePanel", "storage", "alarms", "notifications", "tabs", "downloads"], "host_permissions": [ "http://localhost/*", "http://localhost:*/*", @@ -34,8 +28,11 @@ "default_path": "index.html" }, "icons": { + "16": "icons/icon-16.png", + "32": "icons/icon-32.png", "48": "icons/icon-48.png", - "96": "icons/icon-96.png", - "128": "icons/icon-128.png" + "128": "icons/icon-128.png", + "256": "icons/icon-256.png", + "512": "icons/icon-512.png" } } diff --git a/src/App.jsx b/src/App.jsx old mode 100755 new mode 100644 diff --git a/src/background/index.js b/src/background/index.js old mode 100755 new mode 100644 index 1aba43e..f98d9ca --- a/src/background/index.js +++ b/src/background/index.js @@ -140,20 +140,9 @@ chrome.windows.onRemoved.addListener(async (windowId) => { } }); -// --- Lifecycle: ensure recurring pulse alarm exists ------------------- -async function ensurePulseAlarm() { - const alarm = await chrome.alarms.get(ALARM_NAMES.PULSE); - if (!alarm) { - chrome.alarms.create(ALARM_NAMES.PULSE, { periodInMinutes: PULSE_THRESHOLD_MINUTES }); - } -} - +// --- Lifecycle: set up the recurring pulse alarm on install ----------- chrome.runtime.onInstalled.addListener(() => { - ensurePulseAlarm(); -}); - -chrome.runtime.onStartup?.addListener(() => { - ensurePulseAlarm(); + chrome.alarms.create(ALARM_NAMES.PULSE, { periodInMinutes: PULSE_THRESHOLD_MINUTES }); }); // --- Alarms -------------------------------------------------------------- @@ -190,7 +179,6 @@ async function handlePulseTick() { iconUrl: "icons/icon-128.png", title: "Quick check-in?", message: "Want a quick snap check-in to reset your focus?", - buttons: [{ title: "Start Check-in" }, { title: "Not Now" }], priority: 1, }); } @@ -202,33 +190,11 @@ function createReadyNotification() { iconUrl: "icons/icon-128.png", title: "Your snap is ready", message: "Wanna have a look?", - buttons: [{ title: "Watch Reel" }, { title: "Later" }], priority: 1, }); } /** Polls the local backend for job completion (project summary §7, Phase 3). */ -let activePollTimeout = null; - -async function pollActiveJob() { - if (activePollTimeout) { - clearTimeout(activePollTimeout); - activePollTimeout = null; - } - - const cycle = await getCycle(); - if (cycle.cycle_status !== CYCLE_STATUS.PENDING || !cycle.job_id) { - return; - } - - await handleJobPoll(); - - const nextCycle = await getCycle(); - if (nextCycle.cycle_status === CYCLE_STATUS.PENDING && nextCycle.job_id) { - activePollTimeout = setTimeout(pollActiveJob, 3000); - } -} - async function handleJobPoll() { const cycle = await getCycle(); if (cycle.cycle_status !== CYCLE_STATUS.PENDING || !cycle.job_id) { @@ -248,10 +214,6 @@ async function handleJobPoll() { if (data.status === "ready") { chrome.alarms.clear(ALARM_NAMES.JOB_POLL); - if (activePollTimeout) { - clearTimeout(activePollTimeout); - activePollTimeout = null; - } await setCycle({ cycle_status: CYCLE_STATUS.READY, reel_url: data.reel_url ?? null, @@ -261,10 +223,6 @@ async function handleJobPoll() { createReadyNotification(); } else if (data.status === "failed") { chrome.alarms.clear(ALARM_NAMES.JOB_POLL); - if (activePollTimeout) { - clearTimeout(activePollTimeout); - activePollTimeout = null; - } await setCycle({ cycle_status: CYCLE_STATUS.FAILED, error_message: data.error ?? "Something went wrong while generating your reel.", @@ -286,13 +244,6 @@ async function handleJobPoll() { } } -// Check on worker startup if a job was already in progress -getCycle().then((cycle) => { - if (cycle.cycle_status === CYCLE_STATUS.PENDING && cycle.job_id) { - pollActiveJob(); - } -}); - // --- Notification clicks -------------------------------------------------- chrome.notifications.onClicked.addListener(async (notificationId) => { chrome.notifications.clear(notificationId); @@ -352,37 +303,41 @@ chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => { return true; // keep the message channel open for the async response } - if (message?.type === MESSAGE_TYPES.CANCEL_CHECKIN || message?.type === MESSAGE_TYPES.CANCEL_GENERATION) { + if (message?.type === MESSAGE_TYPES.CANCEL_CHECKIN) { + // The confirm step's "Not now" — discard the clip, reset the cycle. + // The window closes itself; onRemoved above is a no-op by the time it + // fires since capture_window_id is already cleared here. chrome.alarms.clear(ALARM_NAMES.JOB_POLL); - getCycle().then((cycle) => { - if (cycle.job_id) { - fetch(API_ROUTES.CANCEL_JOB(cycle.job_id), { method: "POST" }).catch((err) => - console.warn("[background] Failed to cancel job on backend:", err) - ); - } - setCycle({ - cycle_status: CYCLE_STATUS.IDLE, - capture_window_id: null, - cycle_started_at: null, - job_id: null, - clip_path: null, - error_message: null, - }).then(() => sendResponse({ ok: true })); - }); + setCycle({ + cycle_status: CYCLE_STATUS.IDLE, + capture_window_id: null, + cycle_started_at: null, + job_id: null, + clip_path: null, + }).then(() => sendResponse({ ok: true })); + return true; + } + + if (message?.type === MESSAGE_TYPES.CANCEL_GENERATION) { + // User wants to abort the in-progress generation and start over. + // Reset the cycle to idle so a new check-in can begin. + chrome.alarms.clear(ALARM_NAMES.JOB_POLL); + setCycle({ + cycle_status: CYCLE_STATUS.IDLE, + capture_window_id: null, + cycle_started_at: null, + job_id: null, + clip_path: null, + error_message: null, + }).then(() => sendResponse({ ok: true })); return true; } if (message?.type === MESSAGE_TYPES.CLIP_SAVED) { - // Save clip_path immediately so window removal knows a clip was recorded - setCycle({ clip_path: message.clipPath }); - - // 1. Get active tab info and settings - Promise.all([ - getActiveTabInfo(), - chrome.storage.local.get(STORAGE_KEYS.SETTINGS), - ]).then(async ([tabInfo, { [STORAGE_KEYS.SETTINGS]: settings }]) => { - // 2. Build check-in payload with settings - const payload = buildCheckInPayload({ tabInfo, settings }); + // 1. Get active tab info + getActiveTabInfo().then(async (tabInfo) => { + // 2. Build check-in payload + const payload = buildCheckInPayload({ tabInfo }); payload.clip_path = message.clipPath; try { @@ -403,12 +358,11 @@ chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => { // 4. Update the cycle status with job_id and clip_path await setCycle({ clip_path: message.clipPath, job_id: data.job_id }); - pollActiveJob(); } catch (err) { console.error("[background] Failed to check-in with Express server:", err); await setCycle({ cycle_status: CYCLE_STATUS.FAILED, - error_message: "Could not connect to the local reel generation server on port 4000. Start it by running 'npm start' in the backend directory." + error_message: "Could not connect to the local reel generation server. Make sure it is running on port 4000." }); } }); diff --git a/src/capture/CaptureWindow.jsx b/src/capture/CaptureWindow.jsx old mode 100755 new mode 100644 diff --git a/src/capture/capture.html b/src/capture/capture.html old mode 100755 new mode 100644 diff --git a/src/capture/main.jsx b/src/capture/main.jsx old mode 100755 new mode 100644 diff --git a/src/components/layout/PanelHeader.jsx b/src/components/layout/PanelHeader.jsx old mode 100755 new mode 100644 index 9e3823e..1b0172f --- a/src/components/layout/PanelHeader.jsx +++ b/src/components/layout/PanelHeader.jsx @@ -1,48 +1,35 @@ -import { motion } from "motion/react"; -import Logo from "../ui/Logo.jsx"; - -export default function PanelHeader({ - title = "mindstream", - progress, - showStep = false, - step = 1, - totalSteps = 4, - showLogo = false, - isOnboarding = false, - onboardingStep = 1, -}) { - const shouldShowLogo = isOnboarding || showLogo; - const showLogoInHeader = - shouldShowLogo && (!isOnboarding || onboardingStep > 1); +export default function PanelHeader({ progress }) { + const hasProgress = progress != null && progress >= 0; return ( -
-
-
- {showLogoInHeader && } - - {title} - -
- - {showStep && ( - - {step}/{totalSteps} - - )} +
+
+ mindstream
- - {progress != null && ( -
+ + + + {/* Thin progress bar that replaces the bottom border during countdown */} + {hasProgress && ( +
)} -
+ ); } diff --git a/src/components/layout/PanelShell.jsx b/src/components/layout/PanelShell.jsx old mode 100755 new mode 100644 index 1ccabc6..f18ff43 --- a/src/components/layout/PanelShell.jsx +++ b/src/components/layout/PanelShell.jsx @@ -1,21 +1,15 @@ +import SprocketRail from "./SprocketRail.jsx"; import PanelHeader from "./PanelHeader.jsx"; -export default function PanelShell({ state, progress, headerProps, children }) { - const isPlayer = state === "player"; - - if (isPlayer) { - return ( -
- {children} -
- ); - } - +export default function PanelShell({ state, progress, children }) { return ( -
- -
- {children} +
+ +
+ +
+ {children} +
); diff --git a/src/components/layout/SprocketRail.jsx b/src/components/layout/SprocketRail.jsx new file mode 100644 index 0000000..571dbdc --- /dev/null +++ b/src/components/layout/SprocketRail.jsx @@ -0,0 +1,14 @@ +export default function SprocketRail() { + return ( +