Initial reimplementation of composefs-c by cgwalters · Pull Request #225 · composefs/composefs-rs

cgwalters · 2026-02-06T16:34:56Z

Basically starting on composefs/composefs#423

3 key goals:

Compatible CLI interfaces
Compatible EROFS output format (this is a big deal!)
Next: Compatible C shared library (ugly and messy)

Assisted-by: OpenCode (Claude Sonnet 4)

cgwalters · 2026-02-06T21:42:13Z

There's definitely some sub-tasks to this and pieces that we need to break out. One that I'm realizing is that the dumpfile format is hardcoded to sha256-12. I guess we can just auto-detect from length (like we're doing in other places) but the more I think about this the more I feel we need to formalize it (as is argued in #224 )

So how about a magic comment in the dumpfile like

# format: sha512-12

or so?

cgwalters · 2026-03-04T19:03:01Z

Let's make the format layout a choice to avoid breaking sealed UKIs as is today

cgwalters · 2026-05-18T15:14:11Z

bootc-dev/bootc#1812 is related to this - we need to fix how we generate the EROFS. The problem in a nutshell is that https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9ed50b8231e37b1ae863f5dec8153b98d9f389b4 fixed a legitimate bug, but it also made the kernel parser less strict than it needed to be to fix the bug; we now accept symlinks that cross a block boundary, whereas before we didn't.

alexlarsson · 2026-06-02T09:51:27Z

+    /// Generate only V1 EROFS (default for new repos; C-tool compatible).
+    pub const V1_ONLY: FormatSet = FormatSet(0b01);
+    /// Generate both V1 and V2 EROFS (used by bootc and other multi-format consumers).
+    pub const BOTH: FormatSet = FormatSet(0b11);


Maybe BOTH is not a great name if we ever consider adding a V3.

(And if we add V0)

Yeah, I guess const V1_V2?

alexlarsson · 2026-06-02T10:05:56Z

Also, it seems like the default is still V2, both for "cfsctl init", and the #default for the FormatVersion enum.

alexlarsson · 2026-06-02T10:06:31Z

Other than the above though, I think this is a good start and we should try to get it merged.

cgwalters · 2026-06-02T12:17:10Z

Also, it seems like the default is still V2, both for "cfsctl init", and the #default for the FormatVersion enum.

Yeah, I want to make flipping the default a clearly separate PR.

alexlarsson · 2026-06-03T08:18:20Z

This now looks good to me. If I had one complaint it would be that the last commit is a bit too large. It would make sense to at least split out the mtim_nsec change, and the RepoConfig change to separate commits, and then split the main commit into v1 API support and CLI support.

But I'm fine with this not split up too.

Add set_write_concurrency() to Repository for overriding the default parallelism. Add read_filesystem_with_semaphore() as a public entry point that accepts an explicit Semaphore, and refactor the internal read_filesystem_impl() to centralize semaphore selection. Prep for wiring up --threads in mkcomposefs. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>

The patch recipe referenced crates/cfsctl which was never a valid path; the crate has always been named composefs-ctl. Also relax the clean-tree check to allow untracked files (only committed changes need to match the pinned revision). Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>

import_oci_layout() was opening the layout directory before calling ensure_writable(), so pulling into a read-only repo produced a misleading ENOENT error instead of a clear 'not writable' message. Move the write check to the top of the function, matching the existing skopeo pull path. Fixes privileged_pull_readonly_repo integration test. Signed-off-by: Colin Walters <walters@verbum.org>

For compatibility with the C composefs, we need to support writing directly to a flat XX/DIGEST path, without a leading `objects/`. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>

The script hardcoded /usr/share/edk2/ovmf/OVMF_CODE.fd which is only present on Fedora. Probe a list of common paths (Ubuntu's ovmf package uses /usr/share/ovmf/OVMF.fd, Arch uses /usr/share/edk2/x64/OVMF.4m.fd) so the script works across distros without manual adjustment. Also add -machine q35, required on newer QEMU builds (e.g. RHEL10/CentOS Stream 10) where the default pc-i440fx machine type doesn't pair well with OVMF for EFI boot. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>

composefs-setup-root validates that the repo's meta.json has fs-verity enabled before trusting the repo. The dracut hook was only enabling verity on the content objects, so setup-root would see the repo as insecure and refuse to proceed. Switch the working directory to /sysroot/composefs (instead of the objects subdirectory) so we can enable verity on meta.json in addition to all the content objects. Also quote the loop variable and use the full relative path for clarity. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>

The 30s default is tight on slower hardware (e.g. CentOS Stream 10 with OVMF pflash init overhead) — the VM boots successfully but just barely misses the window. 60s gives enough headroom while still being short enough to catch genuinely broken VMs. CI on Ubuntu with KVM acceleration boots well under 30s so the extra budget costs nothing. Assisted-by: OpenCode (claude-sonnet-4-6@default) Signed-off-by: Colin Walters <walters@verbum.org>

…info CLI Add support for generating V1 EROFS images compatible with the C composefs tools (mkcomposefs/composefs-info 1.0.8+). V1 uses compact inodes, BFS layout, and a simpler on-disk structure. Adds --erofs-version flag to cfsctl, new mkcomposefs and composefs-info compatibility subcommands, and RepositoryConfig for cleaner repo initialization. Note: this commit does not compile with --features oci (the default) until the following commit migrates OCI crate callers. Assisted-by: OpenCode (Claude Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org> ci: Point bootc revdep CI at temporary branch Update the bootc reverse dependency CI to use the cgwalters/bootc adapt-composefs-rs-api branch which adapts bootc to the composefs-rs API changes (RepositoryConfig, FormatVersion, st_mtim_nsec). Assisted-by: OpenCode (Claude Sonnet 4) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters · 2026-06-03T15:32:08Z

If I had one complaint it would be that the last commit is a bit too large. It would make sense to at least split out the mtim_nsec change, and the RepoConfig change to separate commits, and then split the main commit into v1 API support and CLI support.

Yeah, I tried that, it got messy. This PR is itself the result of various splits too (a whole large sidequest here was e.g. #256 )

cgwalters · 2026-06-03T15:34:52Z

BTW this repo is setup so that pushing to a PR drops review approvals, so you'll need to readd

alexlarsson · 2026-06-10T13:02:45Z

This is now in https://github.com/composefs/composefs-run

cgwalters force-pushed the composefs-c-compat branch 2 times, most recently from 4d43b61 to 1871128 Compare February 11, 2026 22:09

cgwalters mentioned this pull request Feb 12, 2026

doc/plans: Update OCI sealing spec (kernel sigs, flattened layers) #224

Open

cgwalters mentioned this pull request Mar 6, 2026

Factor out a composefs-erofs crate #247

Closed

cgwalters force-pushed the composefs-c-compat branch 3 times, most recently from 8a5c48d to 9cb1923 Compare March 11, 2026 01:41

This was referenced Mar 11, 2026

cfsctl: Add cmd to get object ref for a file #236

Merged

WIP: Rough draft for updated generic OCI sealing #226

Draft

cgwalters force-pushed the composefs-c-compat branch 4 times, most recently from 6eda766 to dc1fed7 Compare March 17, 2026 15:31

This was referenced Mar 17, 2026

composefs-info: Write 0 for directory sizes in dump output composefs/composefs#434

Open

dumpfile: Various canonicalization fixes, limit symlink length #270

Merged

cgwalters force-pushed the composefs-c-compat branch from dc1fed7 to 9a845fa Compare March 17, 2026 23:10

cgwalters force-pushed the composefs-c-compat branch from 9a845fa to 9823c67 Compare March 31, 2026 13:49

cgwalters force-pushed the composefs-c-compat branch 4 times, most recently from a8d6802 to 25cbbb1 Compare May 3, 2026 21:40

Copilot AI mentioned this pull request May 6, 2026

tar: Preserve PAX mtime nanoseconds #291

Closed

8 tasks

cgwalters force-pushed the composefs-c-compat branch 2 times, most recently from 895ccd1 to 8eeec80 Compare May 11, 2026 11:59

cgwalters force-pushed the composefs-c-compat branch 2 times, most recently from ad51b62 to 4081268 Compare May 18, 2026 18:28

cgwalters marked this pull request as ready for review May 18, 2026 19:00

alexlarsson reviewed Jun 2, 2026

View reviewed changes

cgwalters force-pushed the composefs-c-compat branch 5 times, most recently from 0675856 to d4a07bc Compare June 2, 2026 22:10

alexlarsson reviewed Jun 3, 2026

View reviewed changes

Comment thread crates/composefs-ctl/src/lib.rs Outdated

alexlarsson previously approved these changes Jun 3, 2026

View reviewed changes

cgwalters added 8 commits June 3, 2026 07:23

fs: Add ObjectStore trait and FlatDigestStore

e50ab2f

For compatibility with the C composefs, we need to support writing directly to a flat XX/DIGEST path, without a leading `objects/`. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>

cgwalters dismissed alexlarsson’s stale review via 234f87f June 3, 2026 12:32

cgwalters force-pushed the composefs-c-compat branch from d4a07bc to 234f87f Compare June 3, 2026 12:32

alexlarsson approved these changes Jun 3, 2026

View reviewed changes

cgwalters added this pull request to the merge queue Jun 3, 2026

Merged via the queue into composefs:main with commit 6e5a4ff Jun 3, 2026
32 of 34 checks passed

Johan-Liebert1 mentioned this pull request Jun 8, 2026

fix(deps): update rust bootc-dev/bootc#2226

Open

1 task

Conversation

cgwalters commented Feb 6, 2026

Uh oh!

cgwalters commented Feb 6, 2026

Uh oh!

cgwalters commented Mar 4, 2026

Uh oh!

cgwalters commented May 18, 2026

Uh oh!

alexlarsson Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

alexlarsson Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

alexlarsson commented Jun 2, 2026

Uh oh!

alexlarsson commented Jun 2, 2026

Uh oh!

cgwalters commented Jun 2, 2026

Uh oh!

Uh oh!

alexlarsson commented Jun 3, 2026

Uh oh!

cgwalters commented Jun 3, 2026

Uh oh!

cgwalters commented Jun 3, 2026

Uh oh!

Uh oh!

alexlarsson commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants