Add CUDA backend for Linux/NVIDIA (GB10-tested) and h3c studio web UI - #43
Open
matrixfede wants to merge 20 commits into
Open
Add CUDA backend for Linux/NVIDIA (GB10-tested) and h3c studio web UI#43matrixfede wants to merge 20 commits into
matrixfede wants to merge 20 commits into
Conversation
…ons and benchmarks
A FastAPI backend and a React front end drive the same h3 binary the CLI does, so a video can be made without the command line. Every one of the 42 flags is reachable; Create shows only what a first-time user needs, and each choice that changes the wait carries its own estimate, learnt from the jobs this machine has actually finished. The backend runs one job at a time — a single GPU, 27 GB at the DiT peak — streams progress and live denoising previews over SSE, and keeps uploads in a library that later jobs can reuse. A generation no longer occupies the page: a monitoring strip keeps it visible while the next one is composed. main.c gains --preview-dir, the only change to the C: it writes the previews preview_denoise already produces to files, and leaves the mp4 byte-identical. Docker builds h3 for CUDA and ships the binary with the service; the 465 GB checkpoint is bind-mounted read-only and never enters an image. The service has no authentication and binds to the loopback address by default. An optional post-processing stage can hand the finished video to an external program. No model, no runtime and no download URL is included. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DELETE /api/jobs/{id} removes the row and the whole data/jobs/<id> directory:
mp4, poster, log and previews. Only a job in a terminal state can go, so the
worker is never left writing into a directory being removed; a queued or
running one answers 409 and says to stop it first.
The directory goes before the row. If it cannot be removed the video stays
listed and can be deleted again — a visible remnant beats gigabytes that
nothing points at any more.
In the UI the control is quiet until it is asked: on a take card and on the
open take, one click turns it into "Delete for good?" with Yes and Keep.
There is no bin and no undo, which is exactly why it asks.
Deleting a completed job also removes it from the sample the time estimate
corrects itself against, which is the accepted cost of not keeping ghosts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
install.sh checks the prerequisites, puts the repository in place and writes a .env. By default it downloads nothing: the 465 GB checkpoint and the optional face-swapping runtime are separate questions, and both can be declined. It is not written to be piped from a URL into a shell. --with-model runs the hf download and hf cache verify commands the README already documented, refuses to start when the filesystem cannot hold the checkpoint, and leaves a complete snapshot alone. --with-faceswap fetches the FaceFusion runtime — the program, not the models, which it downloads itself — after printing what its licence allows and what consent it demands. Nothing about it happens by default. The plugin contract carries only --input and --output, so pointing H3_FACESWAP_CMD straight at FaceFusion would have been broken: scripts/faceswap-facefusion.sh adapts the two, and takes the face to put in from H3_FACESWAP_SOURCE. docs/POSTPROCESSING.md said this repository contains no download URLs. With --with-faceswap that is no longer true, so it now says what is true instead: no models, no weights, one runtime URL, opt-in, and the container path left unwired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Composing was five labelled blocks, each with its CLI flag, its helper text and a grid of cards. It is now a line of direction set in a display serif and four words that can be changed — length, shape, quality, variation — with the wait on the same line. Each word opens where it stands, and the panel that opens is the only thing on the page that pops. Measured against the previous UI, in the composing view: 45 boxes to 12, 40 borders to 10, 89 lines of secondary text to 6. The whole of it now ends 305px down a 900px window, so a first video needs no scrolling and no panels. The three disclosure sections became one panel with three tabs, and every CLI flag left the composing surface: they live under Expert with the exact name they have on the command line, and the test that used to demand a visible flag in Create now forbids one — across Create, Fine-tune, References and the photo slots. --first-frame, --last-frame and --preview-dir are named in Expert next to a line saying where their control actually is. On the stage, the perforation rail moved under the developing frame and now does the work of three readouts: film edge, progress bar and current phase. The palette is unchanged. Boldness spent in one place: the type and the emptiness around the picture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Research in docs/mockup/UX_RESEARCH.md maps ten patterns seen across Runway, Sora, Luma, Kling, Pika, Flow and Hailuo to concrete choices; five are adopted and five are declined with reasons. The mockup v4 fixes the look; the code then takes it over without losing a single option: - attachments hang off the prompt as chips (anchors, references, add) - takes form a grid that plays on hover, actions over the moving picture - the developing preview grows to dominate the render - the wait rides on the Make button - files dropped anywhere on the page land in the library 158 backend tests stay green; verified with a real render started from the Create surface alone inside the container.
The studio is ready to be reached by more than one trusted person: - Schema migrations become versioned (user_version): users, sessions and invites, an owner column on jobs and assets, a login-attempt counter, and per-owner deduplication for uploads. Existing databases migrate in place. - Every /api/* call needs a session cookie. Passwords are argon2id; the first registration becomes the administrator, every account after that needs a single-use invite; five wrong passwords pause a username. - What you made is yours: foreign jobs, uploads and streams answer 404 — indistinguishable from not existing. The administrator sees and manages everything, including invites, password resets and account deletion. - The UI gains a door (sign in / register with invite), the signed-in name in the header, and a People tab for the administrator. Verified end to end in the container with two real users: one video made by the invited user, invisible to nobody but visible to the admin, cleaned up afterwards. Security sections of README and docs/WEBUI.md rewritten.
The People tab existed but lived inside the collapsed Everything else panel. The administrator now has a People entry in the header, next to their name, that opens the panel straight onto accounts and invites. Non-admins get no entry at all; nothing changed in the endpoints.
The administrator no longer walks in through the door: it is declared by the deployment. H3_ADMIN_USERNAME and H3_ADMIN_PASSWORD create the account once, on the first start of an empty database (backfilling whatever existed before accounts); afterwards the environment has no say and passwords are managed from the People tab. Registration loses its last exception: every account is made with a single-use invite. The compose requires the password, the docs and the sign-in screen say so, and the backend logs when neither an account nor a password exists. Also: three SVG logo concepts for 'h3c studio' (the frame, the perforation three, the leader) at favicon/header sizes in both themes, awaiting a choice before integration.
Concept A — the frame — now signs the app: the mark sits in the header and atop the sign-in screen, drawn with currentColor so it follows both themes, and the favicon is the same mark in the printed magenta. Page title and API title become 'h3c studio'. Verified in the container: mark rendered on the login screen in both themes, favicon served, title correct, clean console.
The People door in the header now opens a dedicated view: accounts, invites, password resets and deletions stand on their own page, with the running job's strip still visible and the gallery out of the way. The Everything else panel loses its People tab and is back to Picture, Reference material and Expert. Verified in the container in both themes.
- The restart sweep no longer assumes a running job's h3 is dead: jobs record their process pid, and a restart stops whatever is still alive before declaring the job failed (migration v5). - Post-processing plugins run in their own session and a timeout takes the whole group down, helpers included. - A take whose video cannot be read shows a quiet placeholder instead of a broken picture; long captions no longer spill out of their card. - logs/ moves to .gitignore proper. - Documentation updated to the UI as it is today (composition sentence, Expert panel, People page, env-defined administrator), the UI is named 'h3c studio' throughout, argon2-cffi joins the third-party notices, and the install note that --branch is required goes away.
The CI backend job installed no argon2-cffi although auth.py imports it; the backend package missed python-multipart that UploadFile needs, and setuptools discovered tests/ and tools/ as packages, so a plain pip install failed; install.sh never checked python3 (the .env writer uses it), required ffmpeg on the Docker path where the image carries it, and said nothing about the ICU the host build links; the compose header still claimed the UI has no authentication; and four files referenced plan artefacts that are no longer committed.
Compose grew a required-variable marker when the administrator moved into .env (T129); the validation step only interpolates, so a placeholder that never reaches a running container is enough.
The state flips to 'running' at the claim and the pid is recorded in the write right after Popen (T105 design, so cancel cannot orphan a spawn). On a loaded CI runner the fake shell child starts slowly enough that the test read the pid inside that legitimate window and saw NULL. Poll for it: the invariant is that a running job ends up with its process recorded, which is exactly what the restart sweep relies on.
The first fix covered the direct assertion but the restart-sweep test reads the pid the same way, one write before it exists: on a loaded runner it failed exactly where its sibling had. One _wait_pid helper now expresses the real invariant (a running job ends up with its process recorded) and both call sites use it.
Extract the approved logo concept A into docs/assets (mark + banner SVG, both themes via prefers-color-scheme), capture five studio screenshots from a synthetic demo instance, compose a 1280x640 social preview, and restructure the README so h3c studio and its quick start sit above the CLI tutorial. No CLI content removed: Platforms, Tutorial, Installing, Tests and Implementation sections are byte-identical, only reordered.
matrixfede
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a native CUDA backend for Linux/NVIDIA plus h3c studio, a self-hosted web front end for the same binary.
Supersedes my earlier three-commit shape: the branch is now 20 commits (CUDA port → hardening → web UI → installer/docs/CI → branding). Related to #42. Independent of #45 — "H3 Studio" there is a different project; "h3c studio" here is the browser UI in this PR.
What is in here
1. CUDA backend (
h3_gpu.himplementation)CUDNN_ROOT/CUDNN_FRONTEND_ROOT, with a custom tiled attention fallback when cuDNN is absent.2. Portable host code
3. h3c studio (web UI)
h3binary: every CLI option is reachable from the browser, generation progress streams live with a weighted progress bar, uploads (photos/clips) live in a reusable library.127.0.0.1by default; auth is mandatory in the shippeddocker-compose.yml.docs/WEBUI.md, post-processing notes:docs/POSTPROCESSING.md.4. Installer
install.shchecks prerequisites, clones, writes.env; it downloads nothing (the ~465 GB checkpoint and the optional face-swap runtime are separate, declined-by-default questions) and is meant to be read, not piped.5. Tests and CI
make test, includes the Metal/MLX parity path), 182 backend tests, frontend test suite, Docker-compose smoke test; a CI workflow runs the JS/Python jobs on push (CPU-only runners — see caveat below).Measured on NVIDIA GB10 (DGX Spark, CUDA 13.0, driver 595.84)
Median of three runs, released checkpoint, max-quality preset (1024x576, 107 frames, 50 steps/layers) — reproducible details in
docs/GB10_PROFILE.md:--ssd-streaming: exact (SSIM 1.0) low-memory mode, DiT peak 27.06 GB → 1.63 GB at +37.6% load+denoise time — kept opt-in, not defaultAPI coverage and thresholds:
docs/CUDA_GPU_API_INVENTORY.md.Known limits (stated up front)
--use-int8-row-fc2is a Metal/M5 specialization and a measured CUDA no-op today; there is no verified FP8 execution path../scripts/verify.sh allon the GPU box remains the release gate.Suggested review path
README.md(hero +h3c studiosection, then the CUDA notes in "Platforms and prerequisites")git log --oneline upstream/main..HEADfor the commit groupingdocs/GB10_PROFILE.mdfor the measurement protocolwebui/+docker-compose.ymlfor the studioHappy to split this into smaller PRs (e.g. CUDA backend vs. web UI) if a single 134-file review is too much — say the word.