Repository navigation
refactor: play everything through Vivid's own player bridge and Lucid Engine - #139
Merged
Merged
Conversation
Specifies, by behaviour, the Swift bridge between Lucid Engine's adapter and libmpv: the caller contract and exact payload types, threading and locking, the mpv lifecycle and options, the video surface and the engine's video-output contract, Match Content and HDR, external displays, Picture in Picture, errors, constants, Vivid's own changes by commit, deliberately omitted paths, known quirks, parity hazards and a test checklist. It is the only description of the current bridge that the Vivid-written replacement is built from. It was written and audited by people who read the current code, and the replacement is written by someone who has not.
… [skip release] Adds a new Swift bridge between Lucid Engine's adapter and libmpv, written from docs/playback/player-bridge-spec.md without reference to the current bridge. It is organised as a centre (VividPlayerCoreBase and VividPlayerCore: the mpv context, event loop, ordered delivery and coalescing, requests and disposal) with separate parts for the video surface, Match Content and HDR, external displays and app lifecycle. Both bridges are compiled into every build. VividBridgeSelection.swift picks one with the VIVID_OWN_BRIDGE compilation condition, which is off by default, so shipped behaviour is unchanged. The adapter, the subtitle inventory, the session recorder and the coalescing tests now reach the bridge through those selection names.
… review [skip release] Adds section 17 (internal component files, the wakeup guard nesting, the tvOS last-resort screen, answers to the implementer's questions, distinct start-up check locations and the order of the internal video observations) and corrects three video-output passages the engine rewrite no longer supports.
…Transport patch [skip release]
… release] engine/lucid.py builds dav1d 1.5.3, libass 0.17.5, FFmpeg n8.0.1 and mpv v0.41.0 from pinned upstream archives plus the patch series under engine/patches, stages the interim prebuilt dependencies listed in engine/pins.ini, packages static xcframeworks and writes a provenance report. The FFmpeg series adds a patch that requests Metal-compatible VideoToolbox pixel buffers; the mpv series carries upstream commit 1388a453 unchanged.
…nch [skip release]
…ide [skip release]
… tvOS [skip release] Upstream ao_avfoundation compiles the macOS-only CoreAudio device helpers whenever it is enabled, so -Davfoundation=enabled fails on tvOS. Add Vivid patch mpv/0002, which keeps that code to macOS, and keep ninja going after the first failing file so one run reports every error.
…[skip release] The AVFoundation and AudioUnit outputs share ao_coreaudio_chmap.h, which also declared the macOS-only device queries. Extend mpv patch 0002 to guard those with CA_HAVE_HAL while keeping the layout helpers AVFoundation uses.
…lease] The build facts list filters as the registry symbols Libavfilter defines, which include scale2ref because vf_scale.o defines it, although the configure line never enables it. Compare enabled filters without scale2ref, and compare the defined registry symbols with the facts list less delogo.
…es [skip release] Widen the matrix to iOS device, iOS simulator, tvOS device and tvOS simulator, all arm64, and correct the framework count in the guide.
…kip release] Rejects a non-finite frame rate before the early Match Content write, gives each start-up check its own console location, registers the internal video observations in the order the spec now fixes, keeps the new host view on a repeated initialise, and picks the default Dolby Vision compatibility id from the full profile. The shared teardown marks the core terminal before it drains requests. Also narrows what subclasses can override, marks the lock-protected helpers as Sendable, adds tvOS test seams for the display criteria write, moves the PiP subtitle option into the core, renames internal helpers, corrects two comments and states exactly what has been tested.
Adds conversion, core, display criteria and engine tests that use the new bridge types directly, so CI exercises them whichever bridge the app selects. They cover value conversion and its budgets, ordered delivery and coalescing, exactly-once request completions, the pause cache, lifecycle, the video surface, Match Content derivations and the shared tvOS request identity, the external display role test, the console format, and real mpv playback in the simulator. The files are added to both test targets.
…mpv [skip release] The AVFoundation audio output calls cfstr_get_cstr() from osdep/utils-mac.c, which upstream meson only builds with the macOS cocoa feature, so the static libmpv could not be linked into an app. mpv patch 0002 now builds utils-mac.c with that output whenever cocoa is off. Its notification name log line is unchanged, and macOS builds with cocoa are unchanged.
…e [skip release] lucid.py check now resolves every undefined symbol in the built and pinned archives against the link set, the compiler runtime and the SDK's .tbd stubs, and links a small program that calls mpv_create, mpv_initialize, mpv_terminate_destroy and avformat_configuration, dead-stripped, against the frameworks, prebuilts and app-link.txt. It also checks that Libmpv keeps the AVFoundation notification name log line.
…spec [skip release] Corrects the adapter's audio-option rows, the zoom row's reason, the list of moments the engine waits on the main thread and the cover-art case for audio-only loads, recorded as amendments 17.7 to 17.10.
…p release] [skip ci] Engine implementers work on their own experiment/lucid-engine-* branches, so run the Lucid Engine workflow on pushes to those as well as to experiment/lucid-engine-build. The path filters are unchanged, and the concurrency group stays per branch, so a newer push cancels only that branch's running build.
The behaviour contract Vivid's own mpv and FFmpeg patches are written from: the interface Vivid relies on, the AVFoundation audio and video outputs, Dolby Vision in FFmpeg's VideoToolbox path, libdovi without aborts, the other kept behaviour, the dropped patches, build provenance and the tests. Linked from the build guide, with the patch number ranges.
… [skip release] libdovi was the last engine library linked as a third-party prebuilt that FFmpeg and libplacebo feed media data to, and upstream builds its C library to abort the process on any panic. Build it here instead, as the eleventh framework, Libdovi: - Pin the dovi_tool libdovi-3.3.2 source, the SHA-256 of its dolby_vision Cargo.lock, Rust 1.98.0 stable (rustc, cargo and the standard library for the runner and each slice, which now ships for tvOS too, so no nightly or -Zbuild-std) and cargo-c 0.10.25. fetch downloads every crate the lock file lists from static.crates.io by its checksum, and the build runs offline with --frozen from those vendored crates. - Build the static library with cargo-c first, so FFmpeg and mpv find its dovi.pc. Remove the LLVM bitcode Rust's prebuilt standard library embeds, which Xcode's nm and linker cannot read, with the toolchain's own rust-objcopy. Replace [prebuilt:libdovi] with a reference entry that check uses to require the same header and exported C API as today's prebuilt. - Add Vivid's patch 0001 (MIT, like libdovi): unwinding plus catch_unwind in every exported function, counts checked against the remaining input before memory is reserved, no write of a field holding u64::MAX (the bit writer would first write 512 MiB of zeros), no header when the display management query that libplacebo does not null-check would fail, and a null buffer treated as a parse error. - Add lucid.py test-libdovi and a CI job for it: libdovi built for the runner unpatched and patched, and about 19,400 malformed and valid RPUs through every entry point and through FFmpeg's and libplacebo's call sequences, one process per case. The patched build must never abort and must match the unpatched build wherever that does not abort; the unpatched build must abort on each abort class of the patch specification. Packaging waits for it. - Check per slice: libdovi's form and minimum OS, panic runtime, missing bitcode, header and exports against today's prebuilt, and the libdovi functions libplacebo and FFmpeg import. The test link now calls libdovi. Reproducible archives: set ZERO_AR_DATE=1 and SOURCE_DATE_EPOCH (mpv v0.41.0's commit time, which fixes mpv's build date) for every build, check that every archive member has a zero timestamp and the expected build date, and package twice in the packaging job, failing unless the zips match. Build only the tvOS device slice for this first run; the next commit restores all four slices.
…oducible builds [skip release] Add libdovi to what the pipeline builds, with its pins (Rust 1.98.0, cargo-c 0.10.25 and the locked crates), the patch and why it is sound, the C API test and its corpus, the reproducibility measures, the branch pattern the workflow now runs on, the new checks and the differences from today's prebuilt.
…source [skip release] The tvOS device slice and the libdovi C API test passed in run 38366707132 (229 of 229 slice checks, no failures in the corpus), so restore the iOS device, iOS simulator and tvOS simulator slices.
… release] Add FFmpeg patches 0010 to 0012 from the Lucid Engine patch specification (sections 5 and 6.6): an explicit sample size for every VideoToolbox sample, native Dolby Vision HEVC decoding for profiles 5, 7 and 8 with the profile 7 to 8.1 conversion through libdovi 3.3.2 in its own videotoolbox_dovi.c, and verbose levels for two hevcdec messages. Each is LGPL-2.1-or-later and applies with zero fuzz on top of the existing series.
…ip release] Add mpv patch 0040 from the Lucid Engine patch specification (section 6.1): once hardware decoding has output a frame, a run of decoding errors ends it only after two keyframes fail to recover it or after 240 errors. Probing and copy-back failures still fall back at once. LGPL-2.1-or-later.
…lice [skip release] lucid.py gives FFmpeg libdovi's include path and -ldovi from the staged dovi.pc through CFLAGS and LDFLAGS only, so the configure line is unchanged, and records the flags. lucid.py check now requires the libdovi and per-frame HDR metadata references in Libavcodec, the verbose hevcdec levels, mpv 0040's recovery line and no fault substring in the decoder patches' messages. The new lucid.py test step builds the Dolby Vision host tests from the patched FFmpeg tree, against a scripted libdovi and against libdovi 3.3.2, and fails on any failed check or an empty suite.
…ild guide [skip release]
…ks [skip release] Vivid's own mpv patches for the AVFoundation audio output, written from the patch specification and applied on top of patches 0001 and 0002: - 0010 carries E-AC-3 at four times the codec rate; - 0011 wakes the output chain after a hard reset; - 0012 rebuilds the output around a shared centre (options, queue, audio session, passthrough block, trace edges, prefetch guard); - 0013 adds the decoded PCM sink with the bounded tvOS mode; - 0014 adds passthrough start-up supervision and clock anchoring; - 0015 passes AC-3 and E-AC-3 through AVPlayer and a resource loader. The output's decisions are plain C behind operation tables, so engine/tests/ao_avfoundation_vectors.c drives the shipped code through fakes. `lucid.py test-audio` builds it on the runner from each slice's patched mpv tree and runs every vector of section 9.3, and `lucid.py check` now requires that result, the 25 frozen log formats (with the standalone fallback line checked against a doctored copy), the option and Vivid markers, max-lookahead on tvOS only, and the renderer notification and selectors the output needs. www.gnu.org joins the reference hosts for the LGPL file headers.
xcodebuild -create-xcframework lists the slices in a different order from run to run, so the four-slice packaging in run 38367673194 gave different zips when packaged twice and the new reproducibility check failed it. Rewrite the Info.plist with the slices sorted by library identifier and its keys sorted, so identical frameworks give identical zips.
…ord libdovi flags [skip release] Patch 0011 now keeps the previous session's frame flags until the new VideoToolbox session has been created, so frames output during a restart are not briefly treated as plain HEVC, and releases any session object before the plain HEVC retry. The provenance summary lists the libdovi flags FFmpeg was built with.
…ease] Brings in libdovi 3.3.2 built from source with Vivid's crash-proof patch. FFmpeg now takes libdovi's flags from the dovi.pc that build installs, and lucid.py test builds the same patched libdovi for the build host instead of looking for a host library in the old prebuilt, reads the RPU samples from the pinned libdovi source's own test assets, and passes every RPU of the libdovi corpus through FFmpeg's profile 7 conversion. The copied RPU vectors are no longer needed.
…bass under cache pressure [skip release]
…t frame is rendered [skip release] A static event with glyphs no font provides that draws nothing visible (fully transparent, off screen or clipped to nothing) has no image on screen to keep its event render cache entry. When the images on screen exceed the 32 MiB budget, the cut at the start of every frame dropped that entry, so the event was rendered and its fonts searched again on every frame. Patch 0010 now gives every entry a frame uses a reference of its own, held by the renderer until the next frame has been rendered, as the frame's images hold theirs, and released then or when the renderer is freed. Such an event is searched for once per appearance, plus once per added fallback face, however full the cache is, and a transparent line whose glyphs all resolve is not rendered again either. Nothing is held by events, so freeing an event or a track leaves nothing behind. The libass host test gains a visible baseline of four drawings and three boxed signs that keeps the cache over its budget on its own, and four lines on top of it: the unassigned code point transparent, off screen and clipped to nothing, and a transparent CJK line. Their 12 new checks fail with the previous 0010 and pass with this one (82 scenarios, 10,862 scenario checks).
… invisible-line pressure tests [skip release]
…ilt from [skip release]
…de's note [skip release]
…ranch [skip release] Brings engine/, the Lucid Engine build workflow and the engine guides and patch specification onto this branch. The previous engine's patches, scripts, workflows and licence texts stay deleted.
… source [skip release] Both platform specs and Package.resolved move from the 0.1.0-interim.1 prerelease (11d60dc) to release 0.1.0 (53ebeac), which Vivid's pipeline built from upstream source in full, including libass with its parity patches and the eleven libraries that were interim prebuilt binaries.
…n main and pull requests [skip release] A new always-run workflow checks on every pull request and push that no previous third-party player code or package returns and that both apps pin the same Lucid Engine package, because the player regression workflow only runs for app changes. The Lucid Engine build now also runs on pushes to main and on pull requests that change engine/ or its workflow, so engine changes stay tested once this lands.
…source [skip release] The in-app licence headers, the acknowledgements overview and the third-party notices now describe release 0.1.0: every library built from upstream source by Vivid's pipeline, each library's patches and their licences (mpv, FFmpeg and libbluray LGPL-2.1-or-later, libass ISC including its two upstream patches, libdovi MIT, uavs3d BSD-3-Clause), and where the corresponding source lives. The upstream licence texts are unchanged and match the package's LICENSES folder; the licence test records the new header hashes.
…on this branch [skip release] The engine guides, the patch and bridge specifications, the documentation index, the contribution and agent guides and the release guides now describe the final state: the app pins release 0.1.0, built entirely from upstream source by the pipeline in engine/, which is tested on main and on pull requests and published by hand with engine/publish_package.py. Links that pointed at commits on the two experiment branches are now relative.
blurbery
changed the title
blurbery
marked this pull request as ready for review
October 13, 2026 23:06
|
This PR is large and would use a significant portion of your monthly review quota. Comment |
Owner
Author
|
@cubic-dev-ai review |
@blurbery I have started the AI code review. It will take a few minutes to complete. |
|
Claude review of the new commits 5cb2aad..f970015 I reviewed only the new commit, which changes one line in docs/playback/lucid-engine-patch-spec.md (line 1739). That line now says no library comes as a prebuilt binary apart from the Rust standard library inside libdovi, from the pinned I reviewed the one change since 5cb2aad, the Rust standard library exception on line 1739 of claude-sonnet-5-5, high effort, 8 turns, about US$0.17 of API credit |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 197 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
…rrect engine docs [skip release] Review follow-ups for the engine's host tests, its publishing tool and the docs. No engine patch, build option or app code changes. - test_layers: the off-main helper's completion flag is now an atomic stored with release and loaded with acquire, so the values the worker block writes are safely visible on the main thread. A volatile flag gave no such ordering. - test_centre_tv: the tvOS default-mode case only examines and uninitialises the VO when preinit succeeded. - test_pixels: the 10-bit and 12-bit upload checks now cover every luma sample and chroma pair of each plane, not half of them. - test_frame: the frame-size case now also requires the composite kind, a blend into the decoder's buffer whenever the cue reaches the frame and none when it is wholly off it, so a dropped cue can no longer pass. - publish_package.py refuses a package checkout with uncommitted or untracked files before writing anything, since it replaces LICENSES/ and the package's own files there. - Docs: the notices and distribution guide now say the Rust standard library inside libdovi comes from Rust's pinned prebuilt rust-std archives; the shipped slices are arm64 only, with MoltenVK's simulator x86_64 build thinned away; the adapter track test runs in the iOS jobs only; and the library build facts record that uavs3d's VERSION_SHA1 is its own tag commit, 0e20d2c2, as patch 0001 sets it.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
1 task done
…t part of Lucid Engine [skip release]
…d engine guides [skip release]
…ec [skip release]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Every commit here carries
[skip release]. I'll choose the release marker for the merge commit when I merge.Problem and reproduction
Vivid's player rested on two pieces of other people's work. The Swift bridge between the player and libmpv was adapted from another app, and the engine underneath it (libmpv, FFmpeg and the libraries they use) came as a third-party prebuilt package. That package built mpv and FFmpeg as GPL with OpenSSL, and device builds needed an extra step to swap in a separately rebuilt libmpv carrying my audio patch. I couldn't rebuild exactly what shipped, and the player couldn't carry Vivid's own licence and Apple distribution permission all the way through.
There's no user-facing bug to reproduce. The goal is that every part of the player is mine or built by me from upstream source, with nothing from the previous bridge or package left in the tree.
Implementation and impact
docs/playback/player-bridge-spec.md).VividPlayerCoreBaseandVividPlayerCoreown the mpv context, event loop, ordered delivery, requests and teardown. Conversion, requests, the video surface, Match Content and EDR, external displays and lifecycle observation sit beside them iniosApp/iosApp/Playback/MPV/.LucidTLSturns on certificate verification when FFmpeg is built with Secure Transport. The previous bridge files and their notice are removed.engine/builds every library from pinned upstream source with my patch series: mpv 0.41.0 and FFmpeg 8.0.1 (both LGPL-2.1-or-later as built, Secure Transport instead of OpenSSL), libass 0.17.5, dav1d, libdovi, FreeType, FriBidi, HarfBuzz, libunibreak, Little CMS, shaderc, MoltenVK, libplacebo, uavs3d, libbluray and uchardet. The patches are:0010to0016: Vivid's AVFoundation audio output (decoded PCM, the bounded tvOS mode, AC-3 and E-AC-3 passthrough, AirPlay start-up supervision and clock anchoring);0020to0025: Vivid's AVFoundation video output (timing and pacing, the frame path, subtitles, layers and observers);0010to0012and mpv0040: decoder and Dolby Vision work for VideoToolbox;0001(an error instead of an abort on malformed input), libbluray0001(building for iOS and tvOS) and uavs3d0001(its version without Git history);0001and0002(Secure Transport without a private API, Metal-compatible pixel buffers) and mpv0001(an unchanged upstream spdif fix) and0002(building the audio output for iOS and tvOS);057a5334and upstream pull request #951, both unchanged, plus my0010, so static subtitles render and cache as before.blurbery/lucid-engine0.1.0 (commit53ebeac) serves the 21 frameworks and MoltenVK that run 38458337996 built atfcb6782, withNOTICE,LICENSES/and a README naming that source commit.iosApp/project-ios.yml,iosApp/project.ymlandPackage.resolved. The previous package, its patches, its install script, its workflows and its licence texts are removed. Open Source Licences (and the website licences page built from the same files) andTHIRD_PARTY_NOTICES.mdnow list every library as built from upstream source, each patch and its licence, and where the corresponding source lives:engine/atfcb6782, and the package README.mainand on pull requests that changeengine/or the workflow.VividPlayerCoreTests,VividPlayerConversionTests,VividPlayerDisplayCriteriaTests,VividPlayerEngineTests,PlaybackEndRecoveryTestsandVividEndOfStreamConfirmationTests(MpvPropertyCoalescingTestswent with the old bridge). In the engine pipeline: audio output vectors, video output tests, decoder suites, a malformed Dolby Vision RPU corpus through patched and unpatched libdovi, and libass scenarios.Affects playback on iPhone, iPad and Apple TV for every server. Deliberately unchanged: the player screens, controls, settings and design, the Silo, Emby and Jellyfin cores, and Vivid's driver options.
Risks: this replaces the whole playback stack in one go, and nothing has been played on a physical device yet. The areas most likely to differ are AirPlay and HDMI audio timing (the new audio output), Match Content, HDR and Dolby Vision on real displays, VideoToolbox decoding, long sessions, and libass cost with heavy styled subtitles now that rendering stays on one thread. Reverting this PR's merge brings the previous player back.
Validation
How parity was established:
10faa2a): configuration and symbols match; the audio output gives the same results across 30 scenarios on iOS-like and tvOS-like host builds, apart from free log wording; the decoder matches across 49 scenarios against real macOS VideoToolbox; the video output's timing and picture path show 0 differences (11,744 picture checks); libass is pixel-identical on the parity corpus (23,280 renders), and the libass series added since restores the previous change reporting and font fallback behaviour. No regression was found. Behaviour reviews of the new bridge against the old one found nothing that would regress.1fc0705):SHA256SUMSbyte for byte.Performance
Not benchmarked. Host comparisons show the same timing and pacing behaviour, but frame times, CPU and memory on devices haven't been measured yet.
Testing status
macos-15with Xcode 26.3, unsigned Release device builds, and the engine pipeline onmacos-26with Xcode 26.6. Locally: Xcode 27.0 with the existing iPhone 18 Pro and Apple TV 4K simulators (iOS 27.0 and tvOS 27.0).1fc0705.testCompletionsCalledOffMainArriveLaterOnMainpassed 200 of 200 iterations, the fourVividPlayer*Testsclasses passed 3 iterations (399 tests each), and the playback test classes passed (334 on iPhone, 116 on Apple TV, 0 failures).Design impact and approval
Documentation impact
New:
docs/playback/player-bridge-spec.md,docs/playback/lucid-engine-patch-spec.md,docs/cores/lucid-engine-architecture.md,docs/cores/lucid-engine-build.md,docs/cores/lucid-engine-build-facts.mdanddocs/cores/lucid-engine-libs-build-facts.md.Updated:
docs/cores/player-engine.md,docs/README.md,docs/playback/README.md,docs/release/distribution.md,docs/release/versioning.md,CONTRIBUTING.md,AGENTS.mdandTHIRD_PARTY_NOTICES.md.In-app licences:
iosApp/Resources/OpenSourceLicenses/(overview, every engine library's header, mpv and FFmpeg moved to their LGPL texts, OpenSSL and the previous bridge's licence removed), withscripts/tests/licence-texts-tests.pyupdated. The website licences page is generated from these files.No privacy policy change: what the app sends and stores is unchanged.
I checked the docs this change affects and updated them in this PR, or explained above why none needed changes.
AI disclosure
I implemented these changes and used Claude Opus as a tool.