fix: sidecar and remote URL modes for fragmented BMFF signing#4
Open
santi-ferreiro wants to merge 147 commits into
Open
fix: sidecar and remote URL modes for fragmented BMFF signing#4santi-ferreiro wants to merge 147 commits into
santi-ferreiro wants to merge 147 commits into
Conversation
Alert callers to an error by returning -1 if there was a failure reading the settings string
…#1904) * fix: Micro API * fix: Docs Updated documentation for C2paReader creation to clarify context usage. * fix: Review feedback * fix: stray docs ender --------- Co-authored-by: Tania Mathern <tania.mathern@gmail.comn>
Fix x86_64-linux-android build failure caused by glibc mismatch Use a separate CARGO_TARGET_DIR for Android cross builds to prevent host-compiled build scripts from being reused inside the cross Docker container. When the host and target architectures match (x86_64), Cargo reuses cached build scripts that depend on a newer glibc than the container provides, causing link failures. This was previously fixed in d38dec9 by disabling caching entirely, but that fix was reverted in 352a968 when CI tooling was standardized. This change isolates the cross build target directory instead, preserving caching for all targets. Applied to both library-release.yml and tier-2.yml workflows. See: cross-rs/cross#724 Failed run: https://github.com/contentauth/c2pa-rs/actions/runs/21963594140
* Integration with final feature set * Disable test until I can craft a realistic example. * Fix up some AI generated comments * Build fixes * Address PR comments. Build fixes * Add test simulating complete flow for variable Merkle leaf generation. * Address PR comments * Remove test code * Fix docs * Doc fix * another docs fix * Minor API tweak * Resolve PR comments Fix unwanted logging --------- Co-authored-by: Gavin Peacock <gpeacock@adobe.com>
* Check signature for OCSP responses * Some OCSP fixes * clippy fixes * zip crate update * Fix string formatting.
* feat: Add support for emscripten builds to ffi
Builds with
```
cargo +nightly build -p c2pa-c-ffi --target wasm32-unknown-emscripten -Z build-std=std,panic_abort --no-default-features --features "rust_native_crypto,file_io,fetch_remote_manifests"
```
Requires the following `.cargo/config.toml`
```
[target.wasm32-unknown-emscripten]
rustflags = [
"-C", "target-feature=+atomics,+bulk-memory",
"-C", "link-arg=-pthread",
"-C", "link-arg=-sUSE_PTHREADS=1",
"-C", "link-arg=-sSIDE_MODULE=1",
"-C", "link-arg=-flto",
]
```
* Pass local settings to from_fragmented_files * Resolve issue of bmffHash.mismatch due to verify_store being called on init segment alone * Set store.embedded or store.remote_url explicitly like in from_stream. --------- Co-authored-by: mauricefisher64 <92736594+mauricefisher64@users.noreply.github.com>
* fix: Windows Build flags * Update cargo build command in make_release.ps1 Removed 'fetch_remote_manifests' feature from cargo build command. * fix: Changes to trigger release * fix: Fix that flake * fix: Appease clippy * Retrigger tests
…for validation state (contentauth#1624) * fix: Allow untrusted failure code to be a Valid validation state * fix: Apply same restrictions on ingredient deltas as active manifest for validation state * fix: only allow untrusted ingredients for a valid validation state * fix: any->all for ingredient delta failures to be untrusted for valid state * test: add many permutations of tests for validation state * style: fix formatting
Add a `cargo-lock` job to the Tier 1A workflow that runs `cargo check --locked` to verify Cargo.lock is in sync with Cargo.toml. Closes contentauth#1734
* docs: Convert embeddable API reference to table format Replace long-form API reference section with a concise table that links to docs.rs for each method. Improves scanability and reduces duplication by keeping detailed documentation in the API docs. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * General cleanup and copy edits * Small wording tweak * Add caveat and other sm edits * Wording * Add embed. api to release notes, remove some old/outdated stuff from the rel notes * Comment out BoxHash stuff per Gavin --------- Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
contentauth#1885) (contentauth#1887) * fix: remove exponential memory growth from nested claim reconstruction (contentauth#1885) * fix: address review feedback for mixed ingredients, imports, and error logging * fix: remove eager manifest_data assertions and simplify set_manifest_data calls * fix: restore eager manifest_data population using efficient flat store builder The previous commit removed eager populate_ingredient_manifest_data from Reader::with_store, which broke test_builder_ca_jpg across 4 CI jobs. The eager populate was not the exponential source — that was the nested claim reconstruction in Store::from_jumbf_impl (already fixed). Restore eager population using the new build_ingredient_store (flat, O(n) per call with visited set). Also add a guard in into_builder to skip redundant work when manifest_data is already populated. * fix: add a ? to from_assertion --------- Co-authored-by: Gavin Peacock <gpeacock@adobe.com> Co-authored-by: tmathern <60901087+tmathern@users.noreply.github.com>
… deps on c2pa-rs were already updated (contentauth#1941)
fix: Pin atree
contentauth#1953) * chore: speed up make test with optional nextest support and some other tweaks, adds test-sdk * updated comments on profile.dev The tests should run faster at opt level 1 with a slightly slower build This also moves debug to 1 instead of 2/debug which means full symbols are not added Most of the time we don't need them, but if you want to do full debugging, set this to debug/2
…hat e… (contentauth#1956) * Add compatibility for source assets that report mdat box sizes that extend beyond the end of the content. * simplification * remove unneeded library
…ate `CrJsonReader` (contentauth#1919) * initial commit of crJson * removed all refs to jpegtrust and its specializations, cleaned up tests, etc. * MAJOR cleanup of the schema and the documentation & tests to reflect it * clean up the text a bit more * replaced current validation_status with validationResults * updated to latest schema and made sure that the code outputs compliant json * fixed another issue with icons * add support for getting the timestamp out in the JSON * fix issue with v1 vs. v2 claims * updates to use camelCase instead of snake_case * Enhance CrJSON schema and reader to include jsonGenerator and detailed certificate information. Updated schema to require manifests and jsonGenerator fields, and modified CrJsonReader to build and include jsonGenerator in the output. Improved claim signature handling to incorporate certificate details and timestamp information. * added updated schema and then revised implementation to match * chore: add Reader.cr_json() and Reader.cr_json_value(), no need for cr_json_reader. * refactor: update crJSON schema and validation results structure - Renamed "validationResults" to "validationInfo" in the schema to better reflect its purpose. - Enhanced the validationInfo to include a summary of validation information and validation time. - Updated the CrJsonExporter to build and include validationInfo in the output. - Modified the validationResults structure to clarify the distinction between document-level and per-manifest validation results. - Added support for ingredient deltas in the manifest validation results. - Updated tests to ensure compliance with the new schema structure. * refactor: update crJSON schema and validation results structure - Removed the document-level `validationInfo` in favor of per-manifest `validationResults`. - Introduced `manifestValidationResults` to encapsulate validation status codes and validation time for each manifest. - Updated `CrJsonExporter` to build validation results per manifest, ensuring each manifest's validation status is accurately represented. - Adjusted tests to reflect the new schema structure and validation results handling. * Add crJSON output support: typed structs, --crjson CLI flag, schema compliance tests - Rewrite crjson.rs using typed serde::Serialize structs (CrJsonClaim, CrJsonManifest, CrJsonSignature, etc.) for explicit schema compliance - Eliminate Manifest dependency; drive output directly from Store/Claim APIs - Add Reader::crjson() / crjson_checked() methods - Add --crjson flag to c2patool (conflicts with --detailed) - Add jsonschema dev-dependency (default-features = false for WASM compat) and rewrite schema_compliance tests to validate against the crJSON spec - Consolidate hash_assertions tests; delete redundant hash_encoding.rs - Revert validation_results.rs regression from commit 26fa080 * Update crJSON schema and validation structure - Removed the `date` field from the `jsonGenerator` object in the crJSON schema. - Added `specVersion` to the validation results, ensuring it follows the SemVer format. - Updated the `validationTime` field to be required in the validation results. - Adjusted the `CrJsonExporter` to reflect these schema changes, including updates to the handling of validation results and hash encoding. - Modified tests to ensure compliance with the new schema, including checks for the `b64'` prefix in hash fields. * Refactor crjson module visibility and clean up resource store hash handling Change crjson module from pub to pub(crate) (internal implementation detail) Remove manual hash/alg preservation from resource_store.rs; use add_with instead Move crJSON-schema.json from cli/schemas/ to sdk/tests/fixtures/schemas/ and update the reference Add CLI integration test for --crjson flag output Add check-format, check-docs, and clippy as prerequisites to test-sdk Makefile target Minor code formatting fixes (rustfmt-style rewraps across crjson.rs, hash_assertions.rs, schema_compliance.rs) --------- Originally authored-by: Leonard Rosenthol <lrosenth@adobe.com>
Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.103.9 to 0.103.10. - [Release notes](https://github.com/rustls/webpki/releases) - [Commits](rustls/webpki@v/0.103.9...v/0.103.10) --- updated-dependencies: - dependency-name: rustls-webpki dependency-version: 0.103.10 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* FLAC support * Moved Architecture.md to docs folder * Update links in Architecture.md to correct paths * Add the correct flac sample * Mark test-only methods as test-only Move the flac asset-handler documentation into docs/formats folder. * refactor: Extract shared ID3v2 logic into `id3_helper` module (#2) * Refactor common code between mp3 and flac * Add more tests for mp3_io * Rename id3_audio to id3_helper --------- Co-authored-by: Gopalakrishna Sharma <1406458+mpgopala@users.noreply.github.com> * Update flac.md with id3_helper details * More refactoring * Fix clippy and fmt errors * Fix clippy errors (Attempt 2) * Fix fmt errors (Attempt 2) * Review comments 1. Remove dependency on metaflac as it was validating the entire flac file. Validating only the initial header is enough. 2. Remove Architecture.md. Will add it in a separate PR. 3. Error handling. * Review comments 1. Remove dependency on metaflac as it was validating the entire flac file. Validating only the initial header is enough. 2. Remove Architecture.md. Will add it in a separate PR. 3. Error handling. * Format fixes * Return OK if the stream has TooManyManifestStores from add_required_frame * Update rustls-webpki version to 0.103.10 to fix vulnerability audit failure * A proper upgrade for rustls-webpki * Removing testing details Removing testing details as it already exists in a single file (flac_io.rs).
ci: memory benchmarks with codspeed
…d in BMFF UUID box (contentauth#2097)
…er (contentauth#2070) * fix: Harden against unbounded HTTP response body read in DID web server * Perform review comments Fix clippy errors
chore(c2pa): release v0.81.0
…tauth#2107) * fix: Redaction fix * fix: Unit tests * fix: Appease clippy * chore: Add more tests for redaction (contentauth#2109) fix: So many more tests * fix: Appease clippy
…tauth#2101) Co-authored-by: tmathern <60901087+tmathern@users.noreply.github.com>
…ontentauth#2099) * fix: Harden against integer underflow attack in GIF XMP validation * Perform review comments Fix code format issues
Ignasio-Qualabs
approved these changes
May 8, 2026
* ci: Update `cargo-udeps` setup * build: add env_logger as dev-dependency
* Allow boxes with incomplete content to succeed. * formatting
* feat: large and complex benchmark suite * ci: fix bucket name * fix: use small benchmarks only for now * fix: use new non-deprecated reader/builder functions * ci: split each bench target into their own job * ci: selective memory benches * ci: run on benchmark label * ci: set benchmark binary permissions after downloading from artifacts * ci: cache codspeed build * ci: update actions * ci: use actions cache instead of artifacts and fix chmod * ci: don't cache codspeed build, needs unique builds per runner * ci: add benches and run_args to matrix * ci: restore build sharing and disable memory jpeg for now * ci: rename to complex-read, use zstd, bring back med and large benches * ci: add binary ingredient tree bench to complex read * ci: download fixtures in pre-process job and run each format (sign/read) in its own runner * ci: check cache then save in pre-process steps * ci: simplify * ci: parallel format benches * ci: short circuit build step if cached already * ci: split complex read into simulation and memory jobs * fix: remove large SVG benchmarks for now as they are too slow * fix: complex-read use c2pa manifests instead of svg * ci: separate assets in S3 into groups * fix: try loading bench assets or skip * ci: change codspeed output key (cache bug, old one is reserved) and combine complex read sim,mem * ci: only run full suite given benchmark label, otherwise run small and complex * style: format Rust * ci: split complex into two separate benchmarks so small benches are quick * chore: remove old bench assets * ci: fix codspeed build cache * ci: fix codspeed bench filter label * docs: remove comment
* fix: Reduce allocations * Delete sdk/tests/dhat_sign_loop.rs * Delete sdk/dhat-heap.json
…ntauth#2132) * refactor: use `serde_json` directly to merge settings rather than `config` crate * refactor: remove `config` crate and use `serde_json` directly * test: add perf and memory settings benchmarks * fix: add max recursion depth for merging JSON
…bility with `config` crate (contentauth#2138) fix: preserve backwards compatibility with case-insensitive `config` crate
…th_progress range validation (contentauth#2140)
refactor: remove duplication in embeddable async methods
Bring in upstream compressed manifest support (prefer_compress_manifests), fragmented BMFF signing refactor, and recent SDK/CLI changes while preserving fork features: VOD fragmented sidecar/remote signing, live video labels, and default-on manifest compression with --no-compress-manifest opt-out. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes three bugs in the fragmented BMFF (DASH/VOD) signing pipeline exposed when using the
fragmentsubcommand with--sidecaror--remote:--sidecarmode was silently ignored:save_to_bmff_fragmentedalways embedded the JUMBF in the init segment regardless of theRemoteManifestmode on the claim, and returnedOk(()), discarding the bytes. No.c2pasidecar file was ever written.--remote <url>mode was silently ignored for the same reason. The XMP reference that points verifiers to the remote manifest was never embedded in the init segment, and no.c2pafile was written for upload.verify_fragmentedin the CLI stopped at the first validation failure, hiding any subsequent errors.Closes contentauth/c2pa-rs#1952
Changes
sdk/src/store.rssave_to_bmff_fragmentednow returnsResult<Vec<u8>>(the signed JUMBF bytes) instead ofResult<()>. Reads theRemoteManifestmode from the provenance claim before any mutable borrow and branches:NoRemote/EmbedWithRemote: embeds JUMBF in the init segment viafinish_saveas before.SideCar/Remote: patches the signature directly into the in-memory JUMBF bytes, restores the clean init segment from the source path (no JUMBF embedded), and embeds the XMP URL forRemotemode.start_save_bmff_fragmentedaccepts a newremote_url: Option<&str>parameter and embeds the XMP reference into the init segment beforeinithashis computed, so the XMP bytes are covered by the BMFF Merkle hash.match pc.remote_manifest()that silently skippedSideCar/Remotemodes; uses the pre-computedembed_jumbfflag instead.sdk/src/builder.rssign_fragmented_filesreturn type changed fromResult<()>toResult<Vec<u8>>to propagate the signed JUMBF bytes from the store to the caller.cli/src/main.rssign_fragmentednow acceptssidecar: boolandremote_url: Option<&str>. Captures the JUMBFVec<u8>fromsign_fragmented_filesand writes it as<stem>.c2panext to the signed init segment when either flag is active. Prints an upload reminder when--remoteis used.args.sidecar/args.remotebefore they are consumed by the builder setup and threads them through tosign_fragmented.verify_fragmentednow logs all validation failures instead of stopping at the first.sdk/examples/fragmented_bmff.rsResult<Vec<u8>>return type fromsign_fragmented_files.Test plan
Manual end-to-end
Requires an init segment (
BigBuckBunny_2s_init.mp4) and fragment files (*.m4s) with amanifest.json.Plain embed (unchanged behaviour)
c2patool input/BigBuckBunny_2s_init.mp4 \ --manifest manifest.json \ --output out/embed \ fragment --fragments_glob "*.m4s"Sidecar — JUMBF written to
.c2pa, init segment kept cleanc2patool input/BigBuckBunny_2s_init.mp4 \ --manifest manifest.json \ --output out/sidecar \ --sidecar \ fragment --fragments_glob "*.m4s"Remote — JUMBF + XMP URL embedded in init,
.c2pawritten for uploadVerify (all modes)
c2patool out/embed/input/BigBuckBunny_2s_init.mp4 \ fragment --fragments_glob "*.m4s"--sidecar.c2pa--remote <url>.c2pa