diff --git a/e2e/LLVM_VM_SMOKE_TIMINGS.md b/e2e/LLVM_VM_SMOKE_TIMINGS.md index 247dec5..3d370e8 100644 --- a/e2e/LLVM_VM_SMOKE_TIMINGS.md +++ b/e2e/LLVM_VM_SMOKE_TIMINGS.md @@ -27,27 +27,28 @@ subset of the VM `llvm-tblgen` graph. ## Latest Checked-In Result -- Generated: `2026-08-01 18:24:59 EDT` +- Generated: `2026-08-01 23:15:01 EDT` - Command: `ACTIOND_BAZEL_BUILD_FLAGS="--jobs=32 --distdir=/private/tmp/actiond-audit-distdir --override_repository=llvm++osx+macos_sdk=/private/tmp/actiond-audit-macos-sdk" ACTIOND_LLVM_SMOKE_MAC_HOST=0 e2e/run_llvm_vm_smoke.sh` -- Output root: `/var/folders/p4/xn8y5q_j24l5xwgwd_jx5c340000gn/T/actiond-llvm-vm-smoke.G6vg1w` +- Output root: `/var/folders/p4/xn8y5q_j24l5xwgwd_jx5c340000gn/T/actiond-llvm-vm-smoke.J4l9IY` - Workload: `@llvm-project//llvm:llvm-tblgen`, jobs=8 - VM warmup target: `//e2e:llvm_exec_warmup` - Target and VM host platform: `@llvm//platforms:linux_arm64_musl` - Build mode: `-c opt --strip=always --stripopt=--strip-all` -- VM warmup: `90.262s`; `2207 processes: 190 internal, 2017 remote` -- Measured VM build: `98.569s`; `2310 processes: 4 action cache hit, 204 internal, 2106 remote` +- VM warmup: `92.411s`; `2207 processes: 190 internal, 2017 remote` +- Measured VM build: `109.140s`; `2310 processes: 4 action cache hit, 204 internal, 2106 remote` - Measured executions and timing records: `2106` - Mac-host baseline: not run (`ACTIOND_LLVM_SMOKE_MAC_HOST=0`) -The previous checked-in run used the same jobs=8 and completed the warmup in -`82.129s` and measured build in `92.336s`. The current measured build is 6.8% -slower with the same 2,106 actions, while its warmup is 9.9% slower with the -same 2,017 actions. A repeat with the previous code completed the warmup in -`63.992s` and the measured build in `86.002s`. Intermediate runs on the same -machine completed the warmup/measured builds in `96.032s`/`104.393s` and -`94.614s`/`180.238s`. The large changes between identical action sets coincide -with host CPU and memory contention; compare matched CI actiond/native timings -for performance changes. +The prior checked-in run completed the warmup in `97.375s` and the measured +build in `144.161s` with the same 2,017 warmup actions and 2,106 measured +actions; the current measured build is 24.3% faster. An intermediate run after +the first ten changes completed the warmup in `151.588s` and the measured +build in `155.934s`. Earlier runs of the same action sets completed in +`90.262s`/`98.569s`, `82.129s`/`92.336s`, `63.992s`/`86.002s`, +`96.032s`/`104.393s`, and `94.614s`/`180.238s`. Host CPU and memory contention +produce substantial variation, so these local elapsed times do not isolate an +`actiondfs` improvement or regression; compare matched CI actiond/native +timings instead. Earlier checked-in runs completed the warmup and measured build in `102.136s`/`108.831s` and `106.289s`/`138.481s`. A run before action isolation @@ -61,58 +62,93 @@ Counters include both warmup and measured builds. | Counter | Prior | Current | | ------------------------------------ | ----------: | ----------: | -| actiondfs mounts | 4,122 | 4,122 | -| root directory parses | 1,405 | 1,405 | -| directory loads | 151,235 | 169,617 | -| cached directory requests | 169,617 | 169,617 | -| cached directory hits | 162,998 | 163,010 | -| cached directory misses | 6,619 | 6,607 | -| cached directory races | 13 | 1 | -| directory blob reads | 6,618 | 6,606 | -| staged ensure-directory calls | 15,996 | 15,996 | +| actiondfs mounts | 4,123 | 4,123 | +| root directory parses | 1,407 | 1,406 | +| directory loads | 169,665 | 169,665 | +| cached directory requests | 169,665 | 169,665 | +| cached directory hits | 163,042 | 163,046 | +| cached directory misses | 6,623 | 6,619 | +| cached directory races | 4 | 0 | +| cached file records | 16,723 | 16,722 | +| cached directory records | 12,937 | 12,934 | +| directory blob reads | 6,622 | 6,618 | +| staged ensure-directory calls | 16,000 | 15,999 | | staged ensure-directory components | 16 | 16 | | staged ensure-directory existing | 0 | 0 | | staged ensure-directory created | 16 | 16 | -| staged parent dentry hits | 15,980 | 15,980 | -| blob-path cache hits | 460,692 | 460,692 | -| blob-path cache misses | 6,883 | 6,883 | +| staged parent dentry hits | 15,984 | 15,983 | +| blob-path cache hits | 460,968 | 460,970 | +| blob-path cache misses | 6,972 | 6,970 | | blob-path cache evictions | 0 | 0 | -| blob-path cache insertion races | 0 | 0 | -| staged backing open attempts | 15,825 | 15,825 | -| staged backing open lookup | 0.297 ms | 0.324 ms | +| blob-path cache insertion races | 2 | 0 | +| backing reads | 415,500 | 415,500 | +| input backing-file acquisitions | 472,567 | 472,567 | +| mmap calls | 53,239 | 53,239 | +| staged write calls | 37,194 | 37,194 | +| staged backing open attempts | 15,828 | 15,827 | +| staged backing open lookup | 2.606 ms | 0.296 ms | | staged `copy_file_range` successes | 3,828 | 3,828 | | staged `copy_file_range` fallbacks | 0 | 0 | The workload retained the same 2,017 warmup actions and 2,106 measured actions. -All 163,010 cached-directory hits used RCU instead of the global directory-cache -mutex, and all 18,382 staged input-directory merges searched only cached input -directories. The 169,617 cached-directory requests were unchanged: merged input -directories deferred metadata loading until first access instead of removing -directory-cache requests. The 472,190 input backing-file acquisitions used RCU, -and 75,358 staged positive or negative inode lookups avoided exclusive backing -directory locks. Resolving 6,883 CAS cache misses and 6,606 directory blobs -directly avoided 13,489 temporary `PATH_MAX` pathname allocations, or about -52.7 MiB of cumulative allocation. Root inode ownership reduced each of the -4,122 superblock allocations from 128 bytes to 96 bytes, and unhashed root -inodes avoided global inode-hash insertion on each mount. - -Staged inode ownership followed the backing inode through the staged mount -idmap, preserving sandbox uid and gid checks. Unsupported extended attributes -were skipped without bypassing normal inode permission checks, and backing -filesystem security flags were inherited only when the backing filesystem -advertised the same guarantees. Staged create, mkdir, unlink, rename, size -updates, mmap, and child lookups reported zero failures. +The 163,046 cached-directory hits used RCU without a per-node mutex. The +1,407,098 lookups retained the staged-negative filter, while cached input inode +lookups used `iget5_locked_rcu` without the global inode-hash lookup lock; +hashed inode identity and filesystem watches were preserved. Cached-directory +checks were inlined without constructing the directory-parser stack frame, and +input inode numbers used cached-child indexes instead of hashing filenames. +Immutable input inodes reused their mount root's timestamps. + +The 16,722 cached file records and 12,934 cached directory records stored +32-byte binary SHA-256 digests instead of 64-byte hexadecimal digests, saving +948,992 bytes across those 29,656 records. Each cached directory used one +sorted child vector instead of separate file, directory, and symlink vectors; +packed filenames reused unused child-vector capacity when possible and needed +a separate allocation only when that capacity was insufficient. +Cached-directory records borrowed persistent child digests, compared matching +digest pointers before bytes, rejected full cache-key mismatches before digest +comparisons, and skipped sorting already ordered children. +Each filesystem node omitted its former directory-loading mutex, and each of +the 4,123 mounts borrowed its mount-option buffer. + +The 472,567 input backing-file acquisitions borrowed open backing files without +per-operation references. The 415,500 synchronous backing reads, 53,239 mmap +calls, and 37,194 synchronous staged writes avoided redundant credential swaps. +Staged creates reused VFS negative lookup results, initialized staged ownership +directly, and retained backing-inode ownership through the staged mount idmap. +Directory-blob closes and blob-cache entry releases omitted deferred close and +release workqueues. Staged create, mkdir, unlink, rename, size updates, mmap, +and child lookups reported zero failures. + +Input inode owners used `current_fsuid_fsgid`, while identity-mapped staged +inodes copied backing UID/GID directly instead of performing idmap +conversions. Synchronous backing reads and staged writes constructed backing +credential contexts only for asynchronous or direct-I/O requests, and immutable +input reads let `iov_iter_truncate` clamp the requested byte count directly. +Directory attributes used `generic_fillattr` directly, while regular files and +symlinks used the VFS generic attribute fallback. Staged writes and +`copy_file_range` skipped `file_remove_privs` only when `IS_NOSEC` confirmed no +security attributes required processing. + +All 18,388 staged input-directory merges searched cached input directories, and +75,376 staged positive or negative inode lookups avoided exclusive backing +directory locks. Resolving 6,970 blob-path cache misses and 6,618 directory +blobs directly avoided 13,588 temporary `PATH_MAX` pathname allocations, or +about 53.1 MiB of cumulative allocation. Unsupported extended attributes were +skipped without bypassing inode permission checks, and backing filesystem +security flags were inherited only when the backing filesystem advertised the +same guarantees. ## VM Stage Timing All timing values are milliseconds. -| Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | -| --------------------- | ----: | -----: | -----: | ------: | -------: | ------: | --------: | -| total | 7.179 | 24.020 | 34.068 | 93.793 | 1725.016 | 315.365 | 11505.363 | -| input fetch/setup | 0.400 | 0.582 | 0.668 | 1.053 | 2.103 | 1.269 | 84.561 | -| execute | 6.238 | 22.263 | 31.895 | 91.312 | 1722.654 | 312.755 | 11499.724 | -| output upload/collect | 0.026 | 0.136 | 0.258 | 1.312 | 5.180 | 1.341 | 121.919 | +| Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | +| --------------------- | ----: | -----: | -----: | -----: | -------: | ------: | --------: | +| total | 4.379 | 23.305 | 31.561 | 84.933 | 1998.589 | 358.241 | 11136.456 | +| input fetch/setup | 0.423 | 0.589 | 0.643 | 1.031 | 2.596 | 1.749 | 480.404 | +| execute | 3.626 | 21.362 | 29.330 | 80.134 | 1993.589 | 354.955 | 11131.297 | +| output upload/collect | 0.022 | 0.138 | 0.280 | 1.604 | 5.317 | 1.537 | 106.624 | ## Runner Timing @@ -122,12 +158,12 @@ and lazy actiondfs input reads. All timing values are milliseconds. | Runner Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | | -------------- | ----: | -----: | -----: | -----: | -------: | ------: | --------: | -| parent prepare | 0.057 | 0.105 | 0.135 | 0.199 | 0.412 | 0.280 | 55.961 | -| fork | 0.438 | 0.723 | 0.819 | 1.902 | 7.894 | 2.690 | 143.889 | -| child setup | 0.004 | 2.816 | 4.654 | 8.657 | 30.668 | 9.209 | 220.184 | -| process/io | 0.002 | 15.844 | 23.161 | 64.357 | 1716.234 | 299.968 | 11479.045 | -| wait | 0.000 | 0.000 | 0.000 | 0.000 | 3.570 | 0.385 | 23.227 | -| stdio digest | 0.000 | 0.000 | 0.001 | 0.001 | 0.001 | 0.001 | 0.135 | +| parent prepare | 0.061 | 0.101 | 0.121 | 0.156 | 0.452 | 0.225 | 33.856 | +| fork | 0.283 | 0.515 | 0.589 | 0.977 | 6.401 | 2.939 | 217.086 | +| child setup | 0.008 | 2.190 | 3.563 | 6.238 | 22.170 | 7.461 | 322.330 | +| process/io | 1.689 | 15.803 | 21.989 | 54.066 | 1987.070 | 343.441 | 11127.559 | +| wait | 0.000 | 0.000 | 0.000 | 0.000 | 4.653 | 0.681 | 32.240 | +| stdio digest | 0.000 | 0.000 | 0.001 | 0.001 | 0.001 | 0.001 | 0.106 | ## actiondfs Counters @@ -135,77 +171,77 @@ These `/proc/actiondfs_stats` counters cover the VM lifetime. | Counter | Value | | ----------------------------- | --------------: | -| mounts | 4,122 | -| root directory parses | 1,405 | -| directory loads | 169,617 | -| cached directory requests | 169,617 | -| cached directory hits | 163,010 | -| cached directory misses | 6,607 | -| cached directory races | 1 | -| lookups | 1,406,668 | -| lookup hits | 841,778 | -| lookup negative | 564,890 | -| blob open attempts | 474,181 | +| mounts | 4,123 | +| root directory parses | 1,406 | +| directory loads | 169,665 | +| cached directory requests | 169,665 | +| cached directory hits | 163,046 | +| cached directory misses | 6,619 | +| cached directory races | 0 | +| lookups | 1,407,098 | +| lookup hits | 842,187 | +| lookup negative | 564,911 | +| blob open attempts | 474,558 | | blob open stale retries | 0 | -| blob-path cache hits | 460,692 | -| blob-path cache misses | 6,883 | +| blob-path cache hits | 460,970 | +| blob-path cache misses | 6,970 | | blob-path cache evictions | 0 | | blob-path cache insertion races | 0 | -| node blob cache hits | 472,190 | -| node blob cache misses | 467,575 | -| backing reads | 415,332 | -| backing read bytes | 1,303,012,282 | +| node blob cache hits | 472,567 | +| node blob cache misses | 467,940 | +| backing reads | 415,500 | +| backing read bytes | 1,303,975,362 | | backing read stale retries | 0 | -| mmap calls | 53,030 | -| mmap bytes | 1,900,832,624,640 | +| mmap calls | 53,239 | +| mmap bytes | 1,901,676,589,056 | | mmap failures | 0 | -| directory blob reads | 6,606 | -| directory blob bytes | 2,720,139 | +| directory blob reads | 6,618 | +| directory blob bytes | 2,731,662 | ## Staged Filesystem Counters | Counter | Value | | ------------------------------------ | ----------: | -| staged ensure-directory calls | 15,996 | +| staged ensure-directory calls | 15,999 | | staged ensure-directory components | 16 | | staged ensure-directory existing | 0 | | staged ensure-directory created | 16 | -| staged parent dentry hits | 15,980 | -| staged child lookups | 87,568 | -| staged child lookup hits | 35,533 | -| staged child lookup negative | 52,035 | +| staged parent dentry hits | 15,983 | +| staged child lookups | 87,589 | +| staged child lookup hits | 35,540 | +| staged child lookup negative | 52,049 | | staged child lookup failures | 0 | -| staged inode lookups | 1,406,668 | -| staged inode unstaged-parent skips | 1,331,310 | -| staged inode lookup hits | 35,518 | -| staged inode lookup negative | 39,840 | +| staged inode lookups | 1,407,098 | +| staged inode unstaged-parent skips | 1,331,722 | +| staged inode lookup hits | 35,524 | +| staged inode lookup negative | 39,852 | | staged inode lookup failures | 0 | -| staged inode input-directory merges | 18,382 | -| staged backing open attempts | 15,825 | +| staged inode input-directory merges | 18,388 | +| staged backing open attempts | 15,827 | | staged backing open failures | 0 | -| staged backing open total | 12.790 ms | -| staged backing open lookup | 0.324 ms | -| staged backing open file | 10.718 ms | -| staged create calls/successes | 11,981 | +| staged backing open total | 19.004 ms | +| staged backing open lookup | 0.296 ms | +| staged backing open file | 16.781 ms | +| staged create calls/successes | 11,983 | | staged create failures | 0 | | staged mkdir calls/successes | 198 | | staged mkdir failures | 0 | -| staged unlink calls/successes | 15 | +| staged unlink calls/successes | 16 | | staged unlink failures | 0 | -| staged rename calls/successes | 3,817 | +| staged rename calls/successes | 3,818 | | staged rename failures | 0 | -| staged size updates/successes | 3,874 | +| staged size updates/successes | 3,875 | | staged size update failures | 0 | -| staged write calls | 37,065 | -| staged write bytes | 105,766,927 | -| staged mmap calls | 30 | -| staged mmap bytes | 2,084,864 | +| staged write calls | 37,194 | +| staged write bytes | 106,356,427 | +| staged mmap calls | 31 | +| staged mmap bytes | 2,088,960 | | staged mmap failures | 0 | | staged `copy_file_range` successes | 3,828 | | staged `copy_file_range` bytes | 31,630,764 | | staged `copy_file_range` fallbacks | 0 | -The staged-negative filter skipped 1,331,310 lookups with known-unstaged +The staged-negative filter skipped 1,331,722 lookups with known-unstaged parents. All 3,828 `copy_file_range` attempts succeeded without a buffered fallback. @@ -213,14 +249,14 @@ fallback. | Counter | Value | | ------------------------------------ | ---------: | -| CAS put-file calls | 7,947 | -| CAS promotion attempts | 7,947 | -| CAS promotion success | 2,087 | +| CAS put-file calls | 7,948 | +| CAS promotion attempts | 7,948 | +| CAS promotion success | 2,088 | | CAS existing-blob hits | 5,860 | -| CAS promoted bytes | 31,718,871 | -| CAS digest bytes | 84,052,823 | -| CAS promotion digest | 661.226 ms | -| CAS promotion rename | 174.472 ms | +| CAS promoted bytes | 32,204,511 | +| CAS digest bytes | 84,538,463 | +| CAS promotion digest | 686.880 ms | +| CAS promotion rename | 142.628 ms | | CAS cross-device fallbacks | 0 | | CAS permission fallbacks | 0 | | CAS put-file copy calls | 0 | @@ -229,19 +265,19 @@ fallback. | Counter | Value | | ------------------------------------ | ---------: | -| ByteStream file reads | 4,038 | -| ByteStream file bytes | 57,614,606 | +| ByteStream file reads | 4,039 | +| ByteStream file bytes | 57,718,456 | | ByteStream buffered reads | 0 | -| gRPC response tasks started | 36,229 | +| gRPC response tasks started | 36,254 | | gRPC response tasks failed | 0 | | gRPC response concurrency waits | 0 | -| gRPC data frames | 37,011 | -| gRPC file payload frames | 6,902 | -| gRPC `sendfile` attempts/successes | 6,902 | -| gRPC `sendfile` bytes | 57,614,606 | +| gRPC data frames | 37,041 | +| gRPC file payload frames | 6,909 | +| gRPC `sendfile` attempts/successes | 6,909 | +| gRPC `sendfile` bytes | 57,718,456 | | gRPC `sendfile` fallbacks | 0 | -The measured TCP-to-vsock bridge logged two connections, 28.15 MiB from client +The measured TCP-to-vsock bridge logged two connections, 28.29 MiB from client to guest, 37.37 MiB from guest to client, and zero read/write pump errors. -ByteStream reads remained file-backed and all 6,902 gRPC `sendfile` attempts +ByteStream reads remained file-backed and all 6,909 gRPC `sendfile` attempts succeeded. diff --git a/kernel/actiondfs/actiondfs.c b/kernel/actiondfs/actiondfs.c index a819ed3..aa67698 100644 --- a/kernel/actiondfs/actiondfs.c +++ b/kernel/actiondfs/actiondfs.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -20,28 +19,31 @@ #include #include #include +#include #include #include #include #include +#include #include #include #include #include #include #include +#include #include #include #include #include #include +#include #include #include #include #include #include #include -#include #ifndef ACTIONDFS_ENABLE_STATS #define ACTIONDFS_ENABLE_STATS 0 @@ -60,16 +62,26 @@ #define ACTIONDFS_BLOB_PATH_CACHE_BITS 14 #define ACTIONDFS_BLOB_PATH_CACHE_MAX 16384 #define ACTIONDFS_PROC_STATS "actiondfs_stats" +#define ACTIONDFS_HASH_LEN 32 #define ACTIONDFS_HASH_HEX_LEN 64 -#define ACTIONDFS_EMPTY_SHA256 "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" #define ACTIONDFS_UNKNOWN_SIZE (~(u64)0) +static const u8 actiondfs_empty_sha256[ACTIONDFS_HASH_LEN] = { + 0xe3, 0xb0, 0xc4, 0x42, 0x98, 0xfc, 0x1c, 0x14, + 0x9a, 0xfb, 0xf4, 0xc8, 0x99, 0x6f, 0xb9, 0x24, + 0x27, 0xae, 0x41, 0xe4, 0x64, 0x9b, 0x93, 0x4c, + 0xa4, 0x95, 0x99, 0x1b, 0x78, 0x52, 0xb8, 0x55, +}; + struct actiondfs_cached_child { char *name; u64 size; u16 name_len; umode_t mode; - char hash[ACTIONDFS_HASH_HEX_LEN]; + union { + u8 hash[ACTIONDFS_HASH_LEN]; + const char *target; + }; }; struct actiondfs_cached_children { @@ -81,21 +93,20 @@ struct actiondfs_cached_children { struct actiondfs_cached_dir { struct hlist_node hnode; struct dentry *cas_root; - char hash[ACTIONDFS_HASH_HEX_LEN]; + const u8 *hash; u32 size; - struct actiondfs_cached_children files; - struct actiondfs_cached_children dirs; - struct actiondfs_cached_children symlinks; + u32 directory_count; + char *names; + struct actiondfs_cached_children children; }; struct actiondfs_blob_path_cache_entry { struct hlist_node hnode; - struct list_head list; union { + struct list_head list; struct rcu_head rcu; - struct work_struct release_work; }; - const char *hash; + const u8 *hash; struct dentry *cas_root; struct dentry *dentry; }; @@ -115,7 +126,6 @@ struct actiondfs_node { u64 stage_generation; }; const struct actiondfs_cached_child *input_child; - struct mutex load_lock; struct actiondfs_node *parent; struct dentry *stage_dentry; struct actiondfs_cached_dir *cached_dir; @@ -124,7 +134,7 @@ struct actiondfs_node { struct actiondfs_sb_info { struct path cas_path; struct path stage_path; - char root_hash[ACTIONDFS_HASH_HEX_LEN]; + u8 root_hash[ACTIONDFS_HASH_LEN]; }; struct actiondfs_mount_options { @@ -396,7 +406,6 @@ static DEFINE_HASHTABLE(actiondfs_blob_path_cache, ACTIONDFS_BLOB_PATH_CACHE_BIT static LIST_HEAD(actiondfs_blob_path_cache_list); static DEFINE_MUTEX(actiondfs_blob_path_cache_lock); static size_t actiondfs_blob_path_cache_count; -static struct workqueue_struct *actiondfs_blob_path_cache_release_wq; static const struct inode_operations actiondfs_dir_iops; static const struct inode_operations actiondfs_file_iops; @@ -404,9 +413,9 @@ static const struct inode_operations actiondfs_symlink_iops; static const struct file_operations actiondfs_dir_fops; static const struct file_operations actiondfs_file_fops; static struct dentry *actiondfs_get_cached_blob_dentry( - struct actiondfs_sb_info *sbi, const char *hash); + struct actiondfs_sb_info *sbi, const u8 *hash); static void actiondfs_drop_cached_blob_path(struct actiondfs_sb_info *sbi, - const char *hash); + const u8 *hash); static struct actiondfs_sb_info *actiondfs_sbi(struct super_block *sb) { @@ -419,6 +428,12 @@ static void actiondfs_copy_stage_owner(struct inode *inode, struct mnt_idmap *idmap = mnt_idmap(actiondfs_sbi(inode->i_sb)->stage_path.mnt); + if (idmap == &nop_mnt_idmap) { + inode->i_uid = real_inode->i_uid; + inode->i_gid = real_inode->i_gid; + return; + } + inode->i_uid = vfsuid_into_kuid(i_uid_into_vfsuid(idmap, real_inode)); inode->i_gid = vfsgid_into_kgid(i_gid_into_vfsgid(idmap, real_inode)); } @@ -449,17 +464,10 @@ static struct actiondfs_node *actiondfs_alloc_node(umode_t mode) static u64 actiondfs_input_child_ino(struct actiondfs_node *parent, const struct actiondfs_cached_child *child) { - u64 hash = 1469598103934665603ULL; - size_t i; + u64 index = child - parent->cached_dir->children.entries; + u64 hash = parent->ino * 11400714819323198485ULL; - hash ^= parent->ino; - hash *= 1099511628211ULL; - hash ^= S_ISDIR(child->mode) ? 'd' : 'f'; - hash *= 1099511628211ULL; - for (i = 0; i < child->name_len; i++) { - hash ^= (u8)child->name[i]; - hash *= 1099511628211ULL; - } + hash ^= index + 1; /* * Keep synthetic input inode numbers separate from staged and root @@ -748,18 +756,10 @@ actiondfs_find_cached_child(struct actiondfs_node *dir, const char *name, size_t len) { struct actiondfs_cached_dir *cached = dir->cached_dir; - struct actiondfs_cached_child *child; if (!cached) return NULL; - - child = actiondfs_find_cached_child_in(&cached->files, name, len); - if (child) - return child; - child = actiondfs_find_cached_child_in(&cached->dirs, name, len); - if (child) - return child; - return actiondfs_find_cached_child_in(&cached->symlinks, name, len); + return actiondfs_find_cached_child_in(&cached->children, name, len); } static struct actiondfs_node * @@ -778,46 +778,43 @@ actiondfs_materialize_cached_child(struct actiondfs_node *parent, node->size = record->size; if (S_ISLNK(record->mode)) node->link_target = record->name + record->name_len + 1; - else - mutex_init(&node->load_lock); return node; } -static bool actiondfs_cached_children_overlap( - const struct actiondfs_cached_children *lhs, - const struct actiondfs_cached_children *rhs) +static int actiondfs_compare_cached_children(const void *lhs, const void *rhs) +{ + const struct actiondfs_cached_child *left = lhs; + const struct actiondfs_cached_child *right = rhs; + + return actiondfs_compare_name(left->name, left->name_len, + right->name, right->name_len); +} + +static int actiondfs_sort_cached_children(struct actiondfs_cached_dir *dir) { - const struct actiondfs_cached_child *lhs_entry = lhs->entries; - const struct actiondfs_cached_child *rhs_entry = rhs->entries; - size_t lhs_count = lhs->count; - size_t rhs_count = rhs->count; + struct actiondfs_cached_children *children = &dir->children; + size_t i; - while (lhs_count && rhs_count) { - int cmp = actiondfs_compare_name(lhs_entry->name, - lhs_entry->name_len, - rhs_entry->name, - rhs_entry->name_len); + for (i = 1; i < children->count; i++) { + int cmp = actiondfs_compare_cached_children( + &children->entries[i - 1], &children->entries[i]); if (!cmp) - return true; - if (cmp < 0) { - lhs_entry++; - lhs_count--; - } else { - rhs_entry++; - rhs_count--; - } + return -EEXIST; + if (cmp > 0) + goto sort; } - return false; -} + return 0; -static int -actiondfs_validate_no_cross_type_cached_duplicates(struct actiondfs_cached_dir *dir) -{ - if (actiondfs_cached_children_overlap(&dir->files, &dir->dirs) || - actiondfs_cached_children_overlap(&dir->files, &dir->symlinks) || - actiondfs_cached_children_overlap(&dir->dirs, &dir->symlinks)) - return -EEXIST; +sort: + sort_nonatomic(children->entries, children->count, + sizeof(*children->entries), + actiondfs_compare_cached_children, NULL); + for (i = 1; i < children->count; i++) { + if (!actiondfs_compare_cached_children( + &children->entries[i - 1], &children->entries[i])) + return -EEXIST; + } return 0; } @@ -836,18 +833,15 @@ static int actiondfs_valid_component(const char *name, size_t len) static void actiondfs_free_cached_children( struct actiondfs_cached_children *children) { - size_t i; - - for (i = 0; i < children->count; i++) - kfree(children->entries[i].name); kfree(children->entries); } static void actiondfs_free_cached_dir(struct actiondfs_cached_dir *dir) { - actiondfs_free_cached_children(&dir->files); - actiondfs_free_cached_children(&dir->dirs); - actiondfs_free_cached_children(&dir->symlinks); + if (dir->names && + dir->names != (char *)(dir->children.entries + dir->children.count)) + kvfree(dir->names); + actiondfs_free_cached_children(&dir->children); dput(dir->cas_root); kfree(dir); } @@ -866,35 +860,11 @@ static void actiondfs_destroy_dir_cache(void) mutex_unlock(&actiondfs_dir_cache_lock); } -static void actiondfs_free_blob_path_cache_entry(struct actiondfs_blob_path_cache_entry *entry) -{ - dput(entry->dentry); - kfree(entry); -} - -static void actiondfs_release_blob_path_cache_entry_work(struct work_struct *work) -{ - struct actiondfs_blob_path_cache_entry *entry = - container_of(work, struct actiondfs_blob_path_cache_entry, - release_work); - - actiondfs_free_blob_path_cache_entry(entry); -} - -static void actiondfs_release_blob_path_cache_entry_rcu(struct rcu_head *rcu) -{ - struct actiondfs_blob_path_cache_entry *entry = - container_of(rcu, struct actiondfs_blob_path_cache_entry, rcu); - - INIT_WORK(&entry->release_work, - actiondfs_release_blob_path_cache_entry_work); - queue_work(actiondfs_blob_path_cache_release_wq, &entry->release_work); -} - static void actiondfs_release_blob_path_cache_entry( struct actiondfs_blob_path_cache_entry *entry) { - call_rcu(&entry->rcu, actiondfs_release_blob_path_cache_entry_rcu); + dput(entry->dentry); + kfree_rcu(entry, rcu); } static void actiondfs_unlink_blob_path_cache_entry_locked( @@ -924,8 +894,9 @@ static int actiondfs_append_cached_child(struct actiondfs_cached_children *child size_t name_len, umode_t mode, u64 size, - const char *hash, - const char *target) + const u8 *hash, + const char *target, + u32 previous) { struct actiondfs_cached_child *entries; struct actiondfs_cached_child *child; @@ -937,8 +908,8 @@ static int actiondfs_append_cached_child(struct actiondfs_cached_children *child err = actiondfs_valid_component(name, name_len); if (err) return err; - if (children->count) { - child = &children->entries[children->count - 1]; + if (previous != U32_MAX) { + child = &children->entries[previous]; if (actiondfs_compare_name(child->name, child->name_len, name, name_len) >= 0) return -EINVAL; @@ -962,41 +933,70 @@ static int actiondfs_append_cached_child(struct actiondfs_cached_children *child } child = &children->entries[children->count]; - child->name = kmalloc(name_len + 1 + (target ? size + 1 : 0), - GFP_KERNEL); - if (!child->name) - return -ENOMEM; - memcpy(child->name, name, name_len); - child->name[name_len] = '\0'; - if (target) { - memcpy(child->name + name_len + 1, target, size); - child->name[name_len + 1 + size] = '\0'; - } + child->name = (char *)name; child->name_len = name_len; child->mode = mode; child->size = size; - if (!target) - memcpy(child->hash, hash, ACTIONDFS_HASH_HEX_LEN); + if (target) + child->target = target; + else + memcpy(child->hash, hash, ACTIONDFS_HASH_LEN); children->count++; return 0; } -static int actiondfs_valid_hash(const char *hash, size_t len) +static int actiondfs_pack_cached_dir_names(struct actiondfs_cached_dir *dir) { + struct actiondfs_cached_children *children = &dir->children; + size_t total = 0; + size_t available; + char *next; size_t i; - if (len != ACTIONDFS_HASH_HEX_LEN) - return -EINVAL; - for (i = 0; i < ACTIONDFS_HASH_HEX_LEN; i++) { - if (!isxdigit(hash[i])) - return -EINVAL; + for (i = 0; i < children->count; i++) { + struct actiondfs_cached_child *child = &children->entries[i]; + + if (check_add_overflow(total, (size_t)child->name_len + 1, + &total)) + return -EOVERFLOW; + if (S_ISLNK(child->mode) && + check_add_overflow(total, (size_t)child->size + 1, &total)) + return -EOVERFLOW; + } + if (!total) + return 0; + + available = (size_t)(children->capacity - children->count) * + sizeof(*children->entries); + if (total <= available) { + dir->names = (char *)(children->entries + children->count); + } else { + dir->names = kvmalloc(total, GFP_KERNEL); + if (!dir->names) + return -ENOMEM; + } + next = dir->names; + for (i = 0; i < children->count; i++) { + struct actiondfs_cached_child *child = &children->entries[i]; + const char *target = S_ISLNK(child->mode) ? child->target : NULL; + + memcpy(next, child->name, child->name_len); + child->name = next; + next += child->name_len; + *next++ = '\0'; + if (target) { + memcpy(next, target, child->size); + child->target = next; + next += child->size; + *next++ = '\0'; + } } return 0; } -static bool actiondfs_is_empty_sha256(const char *hash) +static bool actiondfs_is_empty_sha256(const u8 *hash) { - return !memcmp(hash, ACTIONDFS_EMPTY_SHA256, ACTIONDFS_HASH_HEX_LEN); + return !memcmp(hash, actiondfs_empty_sha256, ACTIONDFS_HASH_LEN); } static bool actiondfs_retry_stale(int err, unsigned int *attempts) @@ -1023,15 +1023,17 @@ static bool actiondfs_retry_counted_stale(int err, unsigned int *attempts, #endif static struct dentry *actiondfs_lookup_cas_blob( - struct actiondfs_sb_info *sbi, const char *hash) + struct actiondfs_sb_info *sbi, const u8 *hash) { struct mnt_idmap *idmap = mnt_idmap(sbi->cas_path.mnt); - struct qstr shard_name = QSTR_LEN(hash, 2); - struct qstr blob_name = QSTR_LEN(hash, ACTIONDFS_HASH_HEX_LEN); + char hash_hex[ACTIONDFS_HASH_HEX_LEN]; + struct qstr shard_name = QSTR_LEN(hash_hex, 2); + struct qstr blob_name = QSTR_LEN(hash_hex, ACTIONDFS_HASH_HEX_LEN); struct dentry *shard; struct dentry *blob; int err; + bin2hex(hash_hex, hash, ACTIONDFS_HASH_LEN); shard = lookup_one_positive_unlocked(idmap, &shard_name, sbi->cas_path.dentry); if (IS_ERR(shard)) @@ -1076,7 +1078,7 @@ static struct dentry *actiondfs_lookup_cas_blob( } static struct file *actiondfs_open_directory_blob(struct actiondfs_sb_info *sbi, - const char *hash) + const u8 *hash) { unsigned int stale_attempts = 0; struct file *file; @@ -1106,7 +1108,7 @@ static struct file *actiondfs_open_directory_blob(struct actiondfs_sb_info *sbi, } static struct file *actiondfs_open_backing_cas_blob(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, const struct path *user_path, u64 expected_size) { @@ -1168,46 +1170,14 @@ static struct file *actiondfs_open_backing_cas_blob(struct actiondfs_sb_info *sb static struct file *actiondfs_get_node_blob_file(struct file *actiondfs_file) { - struct file __rcu **slot = - (struct file __rcu **)&actiondfs_file->private_data; - struct file *file; + struct file *file = actiondfs_file->private_data; - rcu_read_lock(); - file = get_file_rcu(slot); - rcu_read_unlock(); if (!file) return ERR_PTR(-EBADF); actiondfs_stat_inc(ACTIONDFS_STAT_NODE_BLOB_CACHE_HITS); return file; } -static int actiondfs_reopen_node_blob_if_current(struct actiondfs_sb_info *sbi, - struct actiondfs_node *node, - struct file *actiondfs_file, - struct file *current_file) -{ - const char *hash = node->input_child->hash; - struct file *replacement; - int err = 0; - - mutex_lock(&node->load_lock); - if (actiondfs_file->private_data == current_file) { - actiondfs_drop_cached_blob_path(sbi, hash); - replacement = actiondfs_open_backing_cas_blob( - sbi, hash, file_user_path(actiondfs_file), node->size); - if (IS_ERR(replacement)) { - err = PTR_ERR(replacement); - } else { - rcu_assign_pointer( - *(struct file __rcu **)&actiondfs_file->private_data, - replacement); - fput(current_file); - } - } - mutex_unlock(&node->load_lock); - return err; -} - static struct file *actiondfs_open_staged_backing(struct actiondfs_sb_info *sbi, struct file *actiondfs_file, int flags) @@ -1256,19 +1226,36 @@ static int actiondfs_staged_backing_flags(const struct file *file) return O_RDONLY; } +static noinline_for_stack ssize_t actiondfs_backing_read_iter_slow( + struct file *file, struct iov_iter *to, struct kiocb *iocb) +{ + struct backing_file_ctx ctx = { + .cred = current_cred(), + }; + + return backing_file_read_iter(file, to, iocb, iocb->ki_flags, &ctx); +} + +static __always_inline ssize_t actiondfs_backing_read_iter( + struct file *file, struct iov_iter *to, struct kiocb *iocb) +{ + if (is_sync_kiocb(iocb) && !(iocb->ki_flags & IOCB_DIRECT)) { + rwf_t flags = (__force rwf_t)(iocb->ki_flags & + (IOCB_NOWAIT | IOCB_HIPRI | IOCB_DSYNC | + IOCB_SYNC | IOCB_APPEND)); + + return vfs_iter_read(file, to, &iocb->ki_pos, flags); + } + return actiondfs_backing_read_iter_slow(file, to, iocb); +} + static ssize_t actiondfs_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct inode *inode = file_inode(iocb->ki_filp); struct actiondfs_node *node = inode->i_private; - struct actiondfs_sb_info *sbi = actiondfs_sbi(inode->i_sb); struct file *file; size_t requested = iov_iter_count(to); - size_t wanted; ssize_t nread; - unsigned int stale_attempts = 0; - struct backing_file_ctx ctx = { - .cred = current_cred(), - }; if (!requested) return 0; @@ -1279,8 +1266,7 @@ static ssize_t actiondfs_read_iter(struct kiocb *iocb, struct iov_iter *to) total_start = actiondfs_stat_time_start(); actiondfs_stat_inc(ACTIONDFS_STAT_STAGE_READ_CALLS); file = iocb->ki_filp->private_data; - nread = backing_file_read_iter(file, to, iocb, iocb->ki_flags, - &ctx); + nread = actiondfs_backing_read_iter(file, to, iocb); if (nread > 0) actiondfs_stat_add(ACTIONDFS_STAT_STAGE_READ_BYTES, (u64)nread); @@ -1294,28 +1280,14 @@ static ssize_t actiondfs_read_iter(struct kiocb *iocb, struct iov_iter *to) if (iocb->ki_pos >= node->size) return 0; - wanted = min_t(u64, (u64)requested, node->size - iocb->ki_pos); - iov_iter_truncate(to, wanted); + iov_iter_truncate(to, node->size - iocb->ki_pos); - do { - file = actiondfs_get_node_blob_file(iocb->ki_filp); - if (IS_ERR(file)) - return PTR_ERR(file); - - actiondfs_stat_inc(ACTIONDFS_STAT_BACKING_READS); - nread = backing_file_read_iter(file, to, iocb, iocb->ki_flags, - &ctx); - if (nread == -ESTALE) { - int err = actiondfs_reopen_node_blob_if_current( - sbi, node, iocb->ki_filp, file); + file = actiondfs_get_node_blob_file(iocb->ki_filp); + if (IS_ERR(file)) + return PTR_ERR(file); - if (err) - nread = err; - } - fput(file); - } while (actiondfs_retry_counted_stale( - nread, &stale_attempts, - ACTIONDFS_STAT_BACKING_READ_STALE_RETRIES)); + actiondfs_stat_inc(ACTIONDFS_STAT_BACKING_READS); + nread = actiondfs_backing_read_iter(file, to, iocb); if (nread > 0) actiondfs_stat_add(ACTIONDFS_STAT_BACKING_READ_BYTES, (u64)nread); @@ -1353,6 +1325,40 @@ static void actiondfs_stage_end_write(struct kiocb *iocb, ssize_t written) actiondfs_stat_add(ACTIONDFS_STAT_STAGE_WRITE_BYTES, (u64)written); } +static noinline_for_stack ssize_t actiondfs_backing_write_iter_slow( + struct file *file, struct iov_iter *from, struct kiocb *iocb) +{ + struct backing_file_ctx ctx = { + .cred = current_cred(), + .end_write = actiondfs_stage_end_write, + }; + + return backing_file_write_iter(file, from, iocb, + iocb->ki_flags, &ctx); +} + +static __always_inline ssize_t actiondfs_backing_write_iter( + struct file *file, struct iov_iter *from, struct kiocb *iocb) +{ + ssize_t written; + + if (!is_sync_kiocb(iocb) || (iocb->ki_flags & IOCB_DIRECT)) + return actiondfs_backing_write_iter_slow(file, from, iocb); + if (!iov_iter_count(from)) + return 0; + if (!IS_NOSEC(file_inode(iocb->ki_filp))) { + written = file_remove_privs(iocb->ki_filp); + if (written) + return written; + } + written = vfs_iter_write(file, from, &iocb->ki_pos, + (__force rwf_t)(iocb->ki_flags & + (IOCB_NOWAIT | IOCB_HIPRI | IOCB_DSYNC | + IOCB_SYNC | IOCB_APPEND))); + actiondfs_stage_end_write(iocb, written); + return written; +} + static ssize_t actiondfs_write_iter(struct kiocb *iocb, struct iov_iter *from) { struct inode *inode = file_inode(iocb->ki_filp); @@ -1360,10 +1366,6 @@ static ssize_t actiondfs_write_iter(struct kiocb *iocb, struct iov_iter *from) struct file *file; ssize_t nwritten; u64 total_start; - struct backing_file_ctx ctx = { - .cred = current_cred(), - .end_write = actiondfs_stage_end_write, - }; if (node->origin != ACTIONDFS_NODE_STAGED) return -EROFS; @@ -1372,8 +1374,7 @@ static ssize_t actiondfs_write_iter(struct kiocb *iocb, struct iov_iter *from) actiondfs_stat_inc(ACTIONDFS_STAT_STAGE_WRITE_CALLS); file = iocb->ki_filp->private_data; inode_lock(inode); - nwritten = backing_file_write_iter(file, from, iocb, - iocb->ki_flags, &ctx); + nwritten = actiondfs_backing_write_iter(file, from, iocb); inode_unlock(inode); actiondfs_stat_add_elapsed(ACTIONDFS_STAT_STAGE_WRITE_TOTAL_NS, total_start); @@ -1426,17 +1427,17 @@ static ssize_t actiondfs_copy_file_range(struct file *file_in, loff_t pos_in, real_out = file_out->private_data; inode_lock(inode_out); - copied = file_remove_privs(file_out); - if (copied) - goto out_unlock; + if (!IS_NOSEC(inode_out)) { + copied = file_remove_privs(file_out); + if (copied) + goto out_unlock; + } copied = vfs_copy_file_range(real_in, pos_in, real_out, pos_out, len, 0); if (copied > 0) actiondfs_sync_staged_inode(inode_out, file_inode(real_out)); out_unlock: inode_unlock(inode_out); - if (node_in->origin == ACTIONDFS_NODE_INPUT) - fput(real_in); out: if (copied >= 0) { @@ -1458,12 +1459,11 @@ static ssize_t actiondfs_splice_read(struct file *actiondfs_file, loff_t *ppos, { struct inode *inode = file_inode(actiondfs_file); struct actiondfs_node *node = inode->i_private; - struct actiondfs_sb_info *sbi = actiondfs_sbi(inode->i_sb); struct file *file; + struct kiocb backing_iocb; loff_t pos = *ppos; size_t wanted; ssize_t nread; - unsigned int stale_attempts = 0; struct backing_file_ctx ctx = { .cred = current_cred(), }; @@ -1497,31 +1497,17 @@ static ssize_t actiondfs_splice_read(struct file *actiondfs_file, loff_t *ppos, wanted = min_t(u64, (u64)len, node->size - pos); - do { - struct kiocb backing_iocb; - - file = actiondfs_get_node_blob_file(actiondfs_file); - if (IS_ERR(file)) - return PTR_ERR(file); - - init_sync_kiocb(&backing_iocb, actiondfs_file); - backing_iocb.ki_pos = pos; - actiondfs_stat_inc(ACTIONDFS_STAT_SPLICE_READS); - nread = backing_file_splice_read(file, &backing_iocb, pipe, - wanted, flags, &ctx); - if (nread > 0) - pos = backing_iocb.ki_pos; - if (nread == -ESTALE) { - int err = actiondfs_reopen_node_blob_if_current( - sbi, node, actiondfs_file, file); + file = actiondfs_get_node_blob_file(actiondfs_file); + if (IS_ERR(file)) + return PTR_ERR(file); - if (err) - nread = err; - } - fput(file); - } while (actiondfs_retry_counted_stale( - nread, &stale_attempts, - ACTIONDFS_STAT_SPLICE_READ_STALE_RETRIES)); + init_sync_kiocb(&backing_iocb, actiondfs_file); + backing_iocb.ki_pos = pos; + actiondfs_stat_inc(ACTIONDFS_STAT_SPLICE_READS); + nread = backing_file_splice_read(file, &backing_iocb, pipe, wanted, + flags, &ctx); + if (nread > 0) + pos = backing_iocb.ki_pos; if (nread > 0) { *ppos = pos; @@ -1530,24 +1516,29 @@ static ssize_t actiondfs_splice_read(struct file *actiondfs_file, loff_t *ppos, return nread; } +static int actiondfs_backing_mmap(struct file *file, + struct vm_area_struct *vma) +{ + if (!can_mmap_file(file)) + return -ENODEV; + vma_set_file(vma, file); + return vfs_mmap(file, vma); +} + static int actiondfs_mmap(struct file *actiondfs_file, struct vm_area_struct *vma) { struct inode *inode = file_inode(actiondfs_file); struct actiondfs_node *node = inode->i_private; - struct actiondfs_sb_info *sbi = actiondfs_sbi(inode->i_sb); struct file *file; int err; - struct backing_file_ctx ctx = { - .cred = current_cred(), - }; if (node->origin == ACTIONDFS_NODE_STAGED) { u64 total_start = actiondfs_stat_time_start(); actiondfs_stat_inc(ACTIONDFS_STAT_STAGE_MMAP_CALLS); file = actiondfs_file->private_data; - err = backing_file_mmap(file, vma, &ctx); + err = actiondfs_backing_mmap(file, vma); if (err) actiondfs_stat_inc(ACTIONDFS_STAT_STAGE_MMAP_FAILURES); else @@ -1565,21 +1556,12 @@ static int actiondfs_mmap(struct file *actiondfs_file, } actiondfs_stat_inc(ACTIONDFS_STAT_MMAPS); - err = backing_file_mmap(file, vma, &ctx); - if (err) { + err = actiondfs_backing_mmap(file, vma); + if (err) actiondfs_stat_inc(ACTIONDFS_STAT_MMAP_FAILURES); - if (err == -ESTALE) { - int reopen_err = actiondfs_reopen_node_blob_if_current( - sbi, node, actiondfs_file, file); - - if (reopen_err) - err = reopen_err; - } - } else { + else actiondfs_stat_add(ACTIONDFS_STAT_MMAP_BYTES, (u64)(vma->vm_end - vma->vm_start)); - } - fput(file); return err; } @@ -1658,8 +1640,9 @@ static int actiondfs_pb_skip(const u8 *data, size_t len, size_t *pos, u64 wire) } struct actiondfs_reapi_digest { - const char *hash; + u8 hash[ACTIONDFS_HASH_LEN]; u64 size; + bool present; }; struct actiondfs_parsed_child { @@ -1681,8 +1664,8 @@ static int actiondfs_parse_reapi_digest(const u8 *data, size_t len, size_t pos = 0; int err; - digest->hash = NULL; digest->size = 0; + digest->present = false; while (pos < len) { const u8 *field; size_t field_len; @@ -1699,10 +1682,11 @@ static int actiondfs_parse_reapi_digest(const u8 *data, size_t len, err = actiondfs_pb_read_len(data, len, &pos, &field, &field_len); if (err) return err; - err = actiondfs_valid_hash((const char *)field, field_len); - if (err) - return err; - digest->hash = (const char *)field; + if (field_len != ACTIONDFS_HASH_HEX_LEN || + hex2bin(digest->hash, (const char *)field, + ACTIONDFS_HASH_LEN)) + return -EINVAL; + digest->present = true; break; case 2: if ((key & 7) != 0) @@ -1720,7 +1704,7 @@ static int actiondfs_parse_reapi_digest(const u8 *data, size_t len, } } - return digest->hash ? 0 : -EINVAL; + return digest->present ? 0 : -EINVAL; } static int actiondfs_parse_reapi_child_fields(const u8 *data, size_t len, @@ -1787,47 +1771,52 @@ static int actiondfs_parse_reapi_child_fields(const u8 *data, size_t len, return -EINVAL; if (S_ISLNK(mode)) return out->symlink.target ? 0 : -EINVAL; - return out->digest.hash ? 0 : -EINVAL; + return out->digest.present ? 0 : -EINVAL; } static int actiondfs_parse_reapi_cached_child(struct actiondfs_cached_dir *parent, const u8 *data, size_t len, - umode_t mode) + umode_t mode, u32 *previous) { struct actiondfs_parsed_child child; - struct actiondfs_cached_children *children; bool regular; int err; err = actiondfs_parse_reapi_child_fields(data, len, &child, mode); if (err) return err; - if (S_ISLNK(mode)) - return actiondfs_append_cached_child( - &parent->symlinks, child.name, child.name_len, + if (S_ISLNK(mode)) { + err = actiondfs_append_cached_child( + &parent->children, child.name, child.name_len, S_IFLNK | 0777, child.symlink.target_len, - NULL, child.symlink.target); + NULL, child.symlink.target, *previous); + if (!err) + *previous = parent->children.count - 1; + return err; + } regular = S_ISREG(mode); - if (regular) { - children = &parent->files; + if (regular) mode |= child.executable ? 0555 : 0444; - } else { - children = &parent->dirs; + else mode |= ACTIONDFS_DIR_MODE; - } - err = actiondfs_append_cached_child(children, child.name, child.name_len, + err = actiondfs_append_cached_child(&parent->children, + child.name, child.name_len, mode, child.digest.size, - child.digest.hash, NULL); - if (!err) + child.digest.hash, NULL, *previous); + if (!err) { + *previous = parent->children.count - 1; + if (!regular) + parent->directory_count++; actiondfs_stat_inc(regular ? ACTIONDFS_STAT_CACHED_FILE_RECORDS : ACTIONDFS_STAT_CACHED_DIR_RECORDS); + } return err; } static int actiondfs_read_cas_blob_once(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, u64 expected_size, u8 **out, size_t *out_len) @@ -1883,12 +1872,12 @@ static int actiondfs_read_cas_blob_once(struct actiondfs_sb_info *sbi, err = 0; out_close: - filp_close(file, NULL); + fput(file); return err; } static int actiondfs_read_cas_blob(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, u64 expected_size, u8 **out, size_t *out_len) @@ -1903,7 +1892,7 @@ static int actiondfs_read_cas_blob(struct actiondfs_sb_info *sbi, return err; } -static unsigned long actiondfs_digest_cache_key(const char *hash) +static unsigned long actiondfs_digest_cache_key(const u8 *hash) { unsigned long key; @@ -1913,7 +1902,7 @@ static unsigned long actiondfs_digest_cache_key(const char *hash) static struct actiondfs_blob_path_cache_entry * actiondfs_find_blob_path_cache(struct actiondfs_sb_info *sbi, - const char *hash) + const u8 *hash) { struct actiondfs_blob_path_cache_entry *entry; unsigned long key = actiondfs_digest_cache_key(hash); @@ -1922,7 +1911,8 @@ actiondfs_find_blob_path_cache(struct actiondfs_sb_info *sbi, lockdep_is_held(&actiondfs_blob_path_cache_lock)) { if (entry->cas_root == sbi->cas_path.dentry && (entry->hash == hash || - !memcmp(entry->hash, hash, ACTIONDFS_HASH_HEX_LEN))) + (actiondfs_digest_cache_key(entry->hash) == key && + !memcmp(entry->hash, hash, ACTIONDFS_HASH_LEN)))) return entry; } return NULL; @@ -1944,7 +1934,7 @@ static void actiondfs_evict_blob_path_cache_one_locked(void) } static void actiondfs_insert_blob_path_cache(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, struct dentry *dentry) { struct actiondfs_blob_path_cache_entry *entry; @@ -1979,15 +1969,15 @@ static void actiondfs_insert_blob_path_cache(struct actiondfs_sb_info *sbi, } static struct dentry *actiondfs_get_cached_blob_dentry( - struct actiondfs_sb_info *sbi, const char *hash) + struct actiondfs_sb_info *sbi, const u8 *hash) { struct actiondfs_blob_path_cache_entry *entry; struct dentry *dentry; rcu_read_lock(); entry = actiondfs_find_blob_path_cache(sbi, hash); - if (entry) { - dentry = dget(entry->dentry); + if (entry && lockref_get_not_dead(&entry->dentry->d_lockref)) { + dentry = entry->dentry; rcu_read_unlock(); actiondfs_stat_inc(ACTIONDFS_STAT_BLOB_PATH_CACHE_HITS); return dentry; @@ -2004,7 +1994,7 @@ static struct dentry *actiondfs_get_cached_blob_dentry( } static void actiondfs_drop_cached_blob_path(struct actiondfs_sb_info *sbi, - const char *hash) + const u8 *hash) { struct actiondfs_blob_path_cache_entry *entry; @@ -2019,7 +2009,7 @@ static void actiondfs_drop_cached_blob_path(struct actiondfs_sb_info *sbi, static struct actiondfs_cached_dir * actiondfs_find_cached_dir(struct actiondfs_sb_info *sbi, - const char *hash) + const u8 *hash) { struct actiondfs_cached_dir *entry; unsigned long key = actiondfs_digest_cache_key(hash); @@ -2027,14 +2017,16 @@ actiondfs_find_cached_dir(struct actiondfs_sb_info *sbi, hash_for_each_possible_rcu(actiondfs_dir_cache, entry, hnode, key, lockdep_is_held(&actiondfs_dir_cache_lock)) { if (entry->cas_root == sbi->cas_path.dentry && - !memcmp(entry->hash, hash, ACTIONDFS_HASH_HEX_LEN)) + (entry->hash == hash || + (actiondfs_digest_cache_key(entry->hash) == key && + !memcmp(entry->hash, hash, ACTIONDFS_HASH_LEN)))) return entry; } return NULL; } static int actiondfs_build_cached_dir(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, u64 expected_size, struct actiondfs_cached_dir **out) { @@ -2042,18 +2034,28 @@ static int actiondfs_build_cached_dir(struct actiondfs_sb_info *sbi, u8 *buffer; size_t len; size_t pos = 0; + size_t hash_bytes; + u32 previous[3] = { U32_MAX, U32_MAX, U32_MAX }; int err; - entry = kzalloc(sizeof(*entry), GFP_KERNEL); + hash_bytes = hash == sbi->root_hash ? ACTIONDFS_HASH_LEN : 0; + entry = kzalloc(sizeof(*entry) + hash_bytes, GFP_KERNEL); if (!entry) return -ENOMEM; entry->cas_root = dget(sbi->cas_path.dentry); - memcpy(entry->hash, hash, ACTIONDFS_HASH_HEX_LEN); + if (hash_bytes) { + u8 *owned_hash = (u8 *)(entry + 1); + + memcpy(owned_hash, hash, ACTIONDFS_HASH_LEN); + entry->hash = owned_hash; + } else { + entry->hash = hash; + } err = actiondfs_read_cas_blob(sbi, hash, expected_size, &buffer, &len); if (err) goto fail; - if (!memcmp(hash, sbi->root_hash, ACTIONDFS_HASH_HEX_LEN)) + if (!memcmp(hash, sbi->root_hash, ACTIONDFS_HASH_LEN)) actiondfs_stat_inc(ACTIONDFS_STAT_ROOT_DIR_PARSES); entry->size = len; actiondfs_stat_inc(ACTIONDFS_STAT_CACHED_DIR_BUILDS); @@ -2083,7 +2085,8 @@ static int actiondfs_build_cached_dir(struct actiondfs_sb_info *sbi, err = actiondfs_parse_reapi_cached_child( entry, field, field_len, (key >> 3) == 1 ? S_IFREG : - (key >> 3) == 2 ? S_IFDIR : S_IFLNK); + (key >> 3) == 2 ? S_IFDIR : S_IFLNK, + &previous[(key >> 3) - 1]); if (err) goto out_buffer; break; @@ -2094,7 +2097,10 @@ static int actiondfs_build_cached_dir(struct actiondfs_sb_info *sbi, } } - err = actiondfs_validate_no_cross_type_cached_duplicates(entry); + err = actiondfs_sort_cached_children(entry); + if (err) + goto out_buffer; + err = actiondfs_pack_cached_dir_names(entry); if (err) goto out_buffer; @@ -2110,7 +2116,7 @@ static int actiondfs_build_cached_dir(struct actiondfs_sb_info *sbi, } static int actiondfs_get_cached_dir(struct actiondfs_sb_info *sbi, - const char *hash, + const u8 *hash, u64 expected_size, struct actiondfs_cached_dir **out) { @@ -2157,29 +2163,30 @@ static int actiondfs_get_cached_dir(struct actiondfs_sb_info *sbi, return 0; } -static int actiondfs_ensure_loaded(struct super_block *sb, - struct actiondfs_node *dir) +static noinline_for_stack int actiondfs_load_dir(struct super_block *sb, + struct actiondfs_node *dir) { struct actiondfs_sb_info *sbi = actiondfs_sbi(sb); struct actiondfs_cached_dir *cached; - const char *hash; - int err = 0; + const u8 *hash; + int err; + + hash = dir->input_child ? dir->input_child->hash : sbi->root_hash; + err = actiondfs_get_cached_dir(sbi, hash, dir->size, &cached); + if (err) + return err; + if (!cmpxchg_release(&dir->cached_dir, NULL, cached)) + actiondfs_stat_inc(ACTIONDFS_STAT_DIR_LOADS); + return 0; +} +static __always_inline int actiondfs_ensure_loaded(struct super_block *sb, + struct actiondfs_node *dir) +{ if (dir->origin == ACTIONDFS_NODE_STAGED || smp_load_acquire(&dir->cached_dir)) return 0; - - mutex_lock(&dir->load_lock); - if (!smp_load_acquire(&dir->cached_dir)) { - actiondfs_stat_inc(ACTIONDFS_STAT_DIR_LOADS); - hash = dir->input_child ? dir->input_child->hash : sbi->root_hash; - err = actiondfs_get_cached_dir(sbi, hash, dir->size, - &cached); - if (!err) - smp_store_release(&dir->cached_dir, cached); - } - mutex_unlock(&dir->load_lock); - return err; + return actiondfs_load_dir(sb, dir); } static int actiondfs_parse_options(struct actiondfs_mount_options *opts, @@ -2208,7 +2215,7 @@ static int actiondfs_parse_options(struct actiondfs_mount_options *opts, if (!opts->cas_root || !opts->root_hash) return -EINVAL; - if (actiondfs_valid_hash(opts->root_hash, strlen(opts->root_hash))) + if (strlen(opts->root_hash) != ACTIONDFS_HASH_HEX_LEN) return -EINVAL; if (opts->root_size != ACTIONDFS_UNKNOWN_SIZE && opts->root_size > ACTIONDFS_MAX_DIRECTORY_PROTO_SIZE) @@ -2219,10 +2226,11 @@ static int actiondfs_parse_options(struct actiondfs_mount_options *opts, static int actiondfs_test_input_inode(struct inode *inode, void *data) { struct actiondfs_node *candidate = data; - struct actiondfs_node *existing = inode->i_private; + struct actiondfs_node *existing = READ_ONCE(inode->i_private); - return existing && existing->parent == candidate->parent && - existing->input_child == candidate->input_child; + return existing && + READ_ONCE(existing->input_child) == candidate->input_child && + READ_ONCE(existing->parent) == candidate->parent; } static int actiondfs_set_input_inode(struct inode *inode, void *data) @@ -2238,12 +2246,24 @@ static void actiondfs_init_inode(struct inode *inode, struct actiondfs_node *node) { inode->i_ino = node->ino; - inode_init_owner(&nop_mnt_idmap, inode, NULL, node->mode); - if (node->stage_dentry) + if (node->stage_dentry) { + inode->i_mode = node->mode; actiondfs_copy_stage_owner(inode, d_inode(node->stage_dentry)); + } else { + inode->i_mode = node->mode; + current_fsuid_fsgid(&inode->i_uid, &inode->i_gid); + } inode_has_no_xattr(inode); inode->i_private = node; - simple_inode_init_ts(inode); + if (node->input_child) { + struct inode *root = d_inode(inode->i_sb->s_root); + + inode_set_atime_to_ts(inode, inode_get_atime(root)); + inode_set_mtime_to_ts(inode, inode_get_mtime(root)); + inode_set_ctime_to_ts(inode, inode_get_ctime(root)); + } else { + simple_inode_init_ts(inode); + } if (S_ISLNK(node->mode)) { inode->i_op = &actiondfs_symlink_iops; @@ -2270,8 +2290,8 @@ static struct inode *actiondfs_iget(struct super_block *sb, bool input_child = node->input_child != NULL; if (input_child) - inode = iget5_locked(sb, node->ino, actiondfs_test_input_inode, - actiondfs_set_input_inode, node); + inode = iget5_locked_rcu(sb, node->ino, actiondfs_test_input_inode, + actiondfs_set_input_inode, node); else inode = iget_locked(sb, node->ino); if (!inode) @@ -2398,10 +2418,10 @@ static struct inode *actiondfs_lookup_staged_inode(struct inode *dir, mode = real_inode->i_mode; if (S_ISDIR(mode)) { mode = S_IFDIR | (mode & S_IALLUGO); - if (parent->cached_dir) - input_child = actiondfs_find_cached_child_in( - &parent->cached_dir->dirs, dentry->d_name.name, - dentry->d_name.len); + input_child = actiondfs_find_cached_child( + parent, dentry->d_name.name, dentry->d_name.len); + if (input_child && !S_ISDIR(input_child->mode)) + input_child = NULL; } else if (S_ISREG(mode)) { mode = S_IFREG | (mode & S_IALLUGO); size = i_size_read(real_inode); @@ -2548,12 +2568,6 @@ static int actiondfs_create_staged_child(struct inode *dir, return -EINVAL; } - err = actiondfs_ensure_loaded(dir->i_sb, parent); - if (err) - return err; - if (actiondfs_find_cached_child(parent, dentry->d_name.name, - dentry->d_name.len)) - return -EROFS; inode = actiondfs_prealloc_staged_inode(dir->i_sb, parent, mode, target, target_len); if (IS_ERR(inode)) @@ -2865,7 +2879,7 @@ static int actiondfs_open(struct inode *inode, struct file *file) struct file *backing_file; if (node->origin == ACTIONDFS_NODE_INPUT) { - const char *hash = node->input_child->hash; + const u8 *hash = node->input_child->hash; if (file->f_mode & FMODE_WRITE) return -EROFS; @@ -2880,8 +2894,7 @@ static int actiondfs_open(struct inode *inode, struct file *file) } if (IS_ERR(backing_file)) return PTR_ERR(backing_file); - rcu_assign_pointer(*(struct file __rcu **)&file->private_data, - backing_file); + file->private_data = backing_file; return 0; } @@ -3161,7 +3174,7 @@ static int actiondfs_dir_release(struct inode *inode, struct file *file) static bool actiondfs_emit_cached_children( struct actiondfs_node *dir, struct dir_context *ctx, struct actiondfs_cached_children *children, - loff_t *base, unsigned int d_type) + loff_t *base) { size_t index = 0; @@ -3177,7 +3190,8 @@ static bool actiondfs_emit_cached_children( struct actiondfs_cached_child *child = &children->entries[index]; if (!dir_emit(ctx, child->name, child->name_len, - actiondfs_input_child_ino(dir, child), d_type)) + actiondfs_input_child_ino(dir, child), + fs_umode_to_dtype(child->mode))) return false; actiondfs_stat_inc(ACTIONDFS_STAT_READDIR_ENTRIES); ctx->pos = *base + index + 1; @@ -3224,12 +3238,8 @@ static int actiondfs_iterate_shared(struct file *file, struct dir_context *ctx) if (dir->cached_dir) { struct actiondfs_cached_dir *cached = dir->cached_dir; - if (!actiondfs_emit_cached_children(dir, ctx, &cached->files, - &base, DT_REG) || - !actiondfs_emit_cached_children(dir, ctx, &cached->dirs, - &base, DT_DIR) || - !actiondfs_emit_cached_children(dir, ctx, &cached->symlinks, - &base, DT_LNK)) + if (!actiondfs_emit_cached_children(dir, ctx, + &cached->children, &base)) return 0; } @@ -3254,9 +3264,7 @@ static int actiondfs_dir_getattr(struct mnt_idmap *idmap, err = actiondfs_ensure_loaded(inode->i_sb, node); if (err) return err; - err = simple_getattr(idmap, path, stat, request_mask, query_flags); - if (err) - return err; + generic_fillattr(&nop_mnt_idmap, request_mask, inode, stat); cached = node->cached_dir; stage_dentry = READ_ONCE(node->stage_dentry); @@ -3273,7 +3281,7 @@ static int actiondfs_dir_getattr(struct mnt_idmap *idmap, return 0; } - input_dir_count = cached->dirs.count; + input_dir_count = cached->directory_count; if (!stage_dentry || backing_nlink == 2) stat->nlink = 2 + input_dir_count; else if (!input_dir_count) @@ -3284,13 +3292,11 @@ static int actiondfs_dir_getattr(struct mnt_idmap *idmap, } static const struct inode_operations actiondfs_file_iops = { - .getattr = simple_getattr, .setattr = actiondfs_setattr, }; static const struct inode_operations actiondfs_symlink_iops = { .get_link = simple_get_link, - .getattr = simple_getattr, .setattr = actiondfs_setattr, }; @@ -3400,7 +3406,6 @@ static int actiondfs_fill_super(struct super_block *sb, struct fs_context *fc) err = -ENOMEM; goto fail; } - mutex_init(&root->load_lock); root->ino = 1; err = actiondfs_parse_options(&opts, fc->fs_private); @@ -3421,7 +3426,9 @@ static int actiondfs_fill_super(struct super_block *sb, struct fs_context *fc) } else { sb->s_flags |= SB_RDONLY; } - memcpy(sbi->root_hash, opts.root_hash, ACTIONDFS_HASH_HEX_LEN); + err = hex2bin(sbi->root_hash, opts.root_hash, ACTIONDFS_HASH_LEN); + if (err) + goto fail; root->size = opts.root_size; root_inode = new_inode(sb); @@ -3457,6 +3464,10 @@ static int actiondfs_parse_monolithic(struct fs_context *fc, void *data) if (!data) return -EINVAL; + if (fc->oldapi) { + fc->fs_private = data; + return 0; + } options = kstrdup(data, GFP_KERNEL); if (!options) @@ -3469,7 +3480,8 @@ static int actiondfs_parse_monolithic(struct fs_context *fc, void *data) static void actiondfs_free_context(struct fs_context *fc) { - kfree(fc->fs_private); + if (!fc->oldapi) + kfree(fc->fs_private); } static const struct fs_context_operations actiondfs_context_ops = { @@ -3507,30 +3519,17 @@ static int __init actiondfs_init(void) { int err; - actiondfs_blob_path_cache_release_wq = alloc_workqueue( - ACTIONDFS_FS_NAME "-blob-cache", WQ_UNBOUND | WQ_MEM_RECLAIM, 0); - if (!actiondfs_blob_path_cache_release_wq) - return -ENOMEM; - err = register_filesystem(&actiondfs_fs_type); if (err) - goto fail_workqueue; + return err; #if ACTIONDFS_ENABLE_STATS if (!proc_create_single(ACTIONDFS_PROC_STATS, 0444, NULL, actiondfs_stats_show)) { - err = -ENOMEM; - goto fail_filesystem; + unregister_filesystem(&actiondfs_fs_type); + return -ENOMEM; } #endif return 0; - -#if ACTIONDFS_ENABLE_STATS -fail_filesystem: - unregister_filesystem(&actiondfs_fs_type); -#endif -fail_workqueue: - destroy_workqueue(actiondfs_blob_path_cache_release_wq); - return err; } static void __exit actiondfs_exit(void) @@ -3540,8 +3539,6 @@ static void __exit actiondfs_exit(void) #endif unregister_filesystem(&actiondfs_fs_type); actiondfs_destroy_blob_path_cache(); - rcu_barrier(); - destroy_workqueue(actiondfs_blob_path_cache_release_wq); actiondfs_destroy_dir_cache(); } diff --git a/test/STRESS_TIMINGS.md b/test/STRESS_TIMINGS.md index 367913a..c30d562 100644 --- a/test/STRESS_TIMINGS.md +++ b/test/STRESS_TIMINGS.md @@ -1,22 +1,22 @@ # Executor Timing Summary -- Generated: `2026-08-01 18:19:48 EDT` +- Generated: `2026-08-01 23:08:06 EDT` - Mode: `vm` - Execute records parsed: `37` - Unique action digests: `37` -- Source log: `/var/folders/p4/xn8y5q_j24l5xwgwd_jx5c340000gn/T/actiond-vm-e2e.z0b6EM/darwin-actiond-vm.log` -- Command: `ACTIOND_E2E_KEEP_TMP=1 ACTIOND_E2E_ACTIONDFS_STATS_PATH=/private/tmp/actiondfs-round-seven-final.stats ACTIOND_REPO_BAZEL_FLAGS="--config=remote --config=executor_timing_logs --jobs=32 --distdir=/private/tmp/actiond-audit-distdir --override_repository=llvm++osx+macos_sdk=/private/tmp/actiond-audit-macos-sdk" tools/e2e.sh vm` +- Source log: `/private/var/folders/p4/xn8y5q_j24l5xwgwd_jx5c340000gn/T/actiond-vm-e2e.d5iHEX/darwin-actiond-vm.log` +- Command: `ACTIOND_E2E_KEEP_TMP=1 ACTIOND_REPO_BAZEL_FLAGS="--config=remote --config=executor_timing_logs" tools/e2e.sh vm` ## Stage Timing All timing values are milliseconds unless noted. -| Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of summed total | -| --------------------- | ----: | -----: | -----: | ------: | ------: | -----: | ------: | --------------------: | -| total | 1.448 | 19.294 | 46.697 | 110.509 | 179.615 | 66.307 | 211.392 | 100.0% | -| input fetch/setup | 0.237 | 0.302 | 0.321 | 0.535 | 2.819 | 0.753 | 7.749 | 1.1% | -| execute | 1.106 | 18.792 | 45.921 | 109.871 | 157.196 | 62.818 | 210.164 | 94.7% | -| output upload/collect | 0.069 | 0.166 | 0.341 | 0.609 | 6.771 | 2.735 | 62.242 | 4.1% | +| Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of summed total | +| --------------------- | ----: | -----: | -----: | -----: | -----: | -----: | ------: | --------------------: | +| total | 1.886 | 12.985 | 17.967 | 26.434 | 54.587 | 23.937 | 105.540 | 100.0% | +| input fetch/setup | 0.232 | 0.321 | 0.399 | 0.697 | 4.007 | 1.078 | 9.242 | 4.5% | +| execute | 1.538 | 10.747 | 16.541 | 24.216 | 51.809 | 21.702 | 102.521 | 90.7% | +| output upload/collect | 0.060 | 0.139 | 0.486 | 1.117 | 4.199 | 1.157 | 10.068 | 4.8% | ## Mount And Output Counts @@ -30,143 +30,143 @@ All timing values are milliseconds unless noted. | Stress Case | Actions | Total p25 | Total p50 | Total p75 | Total p95 | Input p50 | Execute p50 | Output p50 | Mounts p50 | | -------------------------- | ------: | --------: | --------: | --------: | --------: | --------: | ----------: | ---------: | ---------: | -| aggregate | 1 | 20.198 | 20.198 | 20.198 | 20.198 | 0.297 | 19.833 | 0.069 | 4 | -| bare_individual_files | 4 | 33.319 | 54.248 | 86.023 | 122.113 | 0.311 | 41.220 | 0.627 | 4 | -| filesystem_regression | 1 | 186.060 | 186.060 | 186.060 | 186.060 | 2.613 | 182.837 | 0.609 | 4 | -| generated_file_producer | 1 | 72.237 | 72.237 | 72.237 | 72.237 | 1.088 | 66.935 | 4.214 | 4 | -| generated_individual_files | 4 | 15.871 | 17.821 | 34.459 | 69.757 | 0.310 | 17.252 | 0.324 | 4 | -| generated_tree_producer | 1 | 178.003 | 178.003 | 178.003 | 178.003 | 0.302 | 115.459 | 62.242 | 4 | -| generated_tree_reuse | 8 | 36.673 | 62.610 | 110.657 | 113.499 | 0.495 | 62.052 | 0.177 | 4 | -| mixed_all | 1 | 114.812 | 114.812 | 114.812 | 114.812 | 0.353 | 111.190 | 3.268 | 4 | -| nested_individual_files | 8 | 46.193 | 69.945 | 91.947 | 185.322 | 0.319 | 67.985 | 0.491 | 4 | -| pids_one_regression | 1 | 18.152 | 18.152 | 18.152 | 18.152 | 0.312 | 17.704 | 0.135 | 4 | -| source_dir_tree | 4 | 14.190 | 17.024 | 52.322 | 131.588 | 0.286 | 16.558 | 0.253 | 4 | -| symlink_input_consumer | 1 | 7.427 | 7.427 | 7.427 | 7.427 | 0.308 | 6.867 | 0.251 | 4 | -| timeout_recovery | 1 | 14.744 | 14.744 | 14.744 | 14.744 | 0.569 | 14.049 | 0.126 | 4 | -| unknown | 1 | 1.448 | 1.448 | 1.448 | 1.448 | 0.237 | 1.106 | 0.105 | 2 | +| aggregate | 1 | 12.985 | 12.985 | 12.985 | 12.985 | 0.300 | 12.625 | 0.060 | 4 | +| bare_individual_files | 4 | 22.180 | 25.206 | 27.570 | 28.142 | 0.406 | 23.426 | 0.368 | 4 | +| filesystem_regression | 1 | 105.540 | 105.540 | 105.540 | 105.540 | 0.947 | 102.521 | 2.071 | 4 | +| generated_file_producer | 1 | 42.392 | 42.392 | 42.392 | 42.392 | 9.242 | 24.742 | 8.408 | 4 | +| generated_individual_files | 4 | 10.012 | 10.295 | 12.001 | 15.814 | 0.323 | 9.114 | 0.443 | 4 | +| generated_tree_producer | 1 | 45.624 | 45.624 | 45.624 | 45.624 | 0.377 | 35.180 | 10.068 | 4 | +| generated_tree_reuse | 8 | 15.657 | 18.662 | 21.268 | 22.079 | 0.580 | 17.102 | 0.213 | 4 | +| mixed_all | 1 | 12.815 | 12.815 | 12.815 | 12.815 | 0.321 | 10.046 | 2.448 | 4 | +| nested_individual_files | 8 | 16.920 | 19.062 | 29.772 | 39.481 | 0.455 | 16.524 | 0.784 | 4 | +| pids_one_regression | 1 | 16.760 | 16.760 | 16.760 | 16.760 | 1.012 | 15.606 | 0.142 | 4 | +| source_dir_tree | 4 | 16.374 | 22.200 | 42.434 | 80.835 | 0.553 | 21.679 | 0.166 | 4 | +| symlink_input_consumer | 1 | 5.609 | 5.609 | 5.609 | 5.609 | 0.331 | 5.061 | 0.217 | 4 | +| timeout_recovery | 1 | 22.593 | 22.593 | 22.593 | 22.593 | 0.643 | 21.843 | 0.107 | 4 | +| unknown | 1 | 1.886 | 1.886 | 1.886 | 1.886 | 0.232 | 1.538 | 0.117 | 2 | ## Visible Overhead Estimate `process/io` is excluded here because it is mostly the action process runtime plus stdout/stderr drain. -| Metric | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of summed total | -| ------------------------- | ----: | -----: | -----: | -----: | -----: | -----: | ------: | --------------------: | -| fixed overhead, no wait | 1.122 | 13.555 | 31.299 | 38.040 | 84.384 | 35.131 | 186.093 | 53.0% | -| fixed overhead, with wait | 1.122 | 13.555 | 31.815 | 38.040 | 96.734 | 35.901 | 186.093 | 54.1% | +| Metric | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of summed total | +| ------------------------- | ----: | ----: | -----: | -----: | -----: | -----: | -----: | --------------------: | +| fixed overhead, no wait | 1.394 | 8.799 | 13.486 | 19.203 | 33.924 | 16.445 | 76.739 | 68.7% | +| fixed overhead, with wait | 1.394 | 8.799 | 15.393 | 21.165 | 35.607 | 17.050 | 76.739 | 71.2% | ## Runner Timing These values split the `execute` bucket. `child setup` includes successful `execve`; `process/io` starts after close-on-exec confirmation and includes action runtime and stdout/stderr drain. -| Runner Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of execute | -| -------------- | ----: | ----: | -----: | -----: | ------: | -----: | ------: | ---------------: | -| parent prepare | 0.030 | 0.113 | 3.474 | 15.014 | 23.418 | 7.602 | 33.307 | 12.1% | -| fork | 0.121 | 0.303 | 0.653 | 2.076 | 10.871 | 2.123 | 12.015 | 3.4% | -| child setup | 0.197 | 5.594 | 12.433 | 28.355 | 57.755 | 21.917 | 184.087 | 34.9% | -| process/io | 0.318 | 2.938 | 6.884 | 45.981 | 124.798 | 30.363 | 154.147 | 48.3% | -| wait | 0.000 | 0.000 | 0.000 | 0.000 | 2.697 | 0.770 | 15.438 | 1.2% | -| stdio digest | 0.000 | 0.000 | 0.000 | 0.001 | 0.002 | 0.001 | 0.009 | 0.0% | - -| Stress Case | Digest | Parent Prep | Fork | Child Setup | Process/IO | Wait | Stdio Digest | Setup Signaled | -| -------------------------- | -------------- | ----------: | -----: | ----------: | ---------: | -----: | -----------: | -------------- | -| aggregate | `097d7589806b` | 0.109 | 0.303 | 16.791 | 2.607 | 0.000 | 0.001 | True | -| bare_individual_files | `90cce354641e` | 16.998 | 2.076 | 52.792 | 58.782 | 0.000 | 0.002 | True | -| bare_individual_files | `cfea754f9880` | 11.678 | 8.589 | 7.016 | 34.918 | 0.000 | 0.000 | True | -| bare_individual_files | `d227993ae34f` | 0.125 | 0.460 | 12.805 | 4.277 | 2.439 | 0.000 | True | -| bare_individual_files | `faa36aaa12ef` | 7.630 | 0.300 | 6.573 | 5.617 | 0.000 | 0.009 | True | -| filesystem_regression | `d6e431f72b37` | 15.310 | 12.015 | 0.197 | 154.147 | 1.115 | 0.000 | True | -| generated_file_producer | `0ab29dfeee67` | 0.105 | 2.194 | 24.369 | 40.254 | 0.000 | 0.000 | True | -| generated_individual_files | `8949618893dd` | 13.240 | 2.434 | 5.594 | 54.267 | 2.233 | 0.001 | True | -| generated_individual_files | `8e7053a36faf` | 8.184 | 0.449 | 3.598 | 6.884 | 0.000 | 0.001 | True | -| generated_individual_files | `ac62596d2eae` | 0.091 | 0.219 | 11.867 | 2.938 | 0.000 | 0.000 | True | -| generated_individual_files | `d7beec3fc619` | 0.104 | 0.332 | 11.629 | 3.266 | 0.000 | 0.000 | True | -| generated_tree_producer | `f8cb8cf8722c` | 7.494 | 0.445 | 7.026 | 84.991 | 15.438 | 0.001 | True | -| generated_tree_reuse | `4f9d33590a97` | 0.123 | 0.323 | 17.583 | 45.981 | 0.000 | 0.000 | True | -| generated_tree_reuse | `ad080d1d4b46` | 16.020 | 0.850 | 17.171 | 0.677 | 0.000 | 0.000 | True | -| generated_tree_reuse | `b82ea83117e0` | 24.852 | 2.076 | 7.014 | 73.544 | 2.800 | 0.000 | True | -| generated_tree_reuse | `bdbccf204e0e` | 0.228 | 0.602 | 15.987 | 1.602 | 0.000 | 0.000 | True | -| generated_tree_reuse | `c222b682e1e5` | 15.568 | 11.700 | 28.903 | 51.000 | 2.671 | 0.000 | True | -| generated_tree_reuse | `cc2d17d56174` | 15.014 | 10.664 | 6.815 | 27.547 | 0.000 | 0.001 | True | -| generated_tree_reuse | `ec3c5569c6cf` | 33.307 | 0.409 | 77.610 | 2.884 | 0.000 | 0.000 | True | -| generated_tree_reuse | `f58a3550a687` | 0.093 | 0.665 | 33.534 | 0.873 | 0.000 | 0.002 | True | -| mixed_all | `ff909e41f5d7` | 22.656 | 0.242 | 8.827 | 77.636 | 1.788 | 0.000 | True | -| nested_individual_files | `515cc9e83e3d` | 1.190 | 1.479 | 32.946 | 33.272 | 0.000 | 0.000 | True | -| nested_individual_files | `6b51fb6999bc` | 23.059 | 7.960 | 37.940 | 6.362 | 0.000 | 0.000 | True | -| nested_individual_files | `816552fd7392` | 0.124 | 0.653 | 184.087 | 25.280 | 0.000 | 0.001 | True | -| nested_individual_files | `97febd68ffa3` | 12.919 | 0.294 | 28.355 | 4.331 | 0.000 | 0.001 | True | -| nested_individual_files | `a85c70689335` | 0.075 | 0.291 | 36.883 | 29.797 | 0.000 | 0.000 | True | -| nested_individual_files | `c595420ad290` | 0.105 | 3.134 | 39.664 | 0.859 | 0.000 | 0.000 | True | -| nested_individual_files | `f697e529f79f` | 0.137 | 0.941 | 4.858 | 130.155 | 0.000 | 0.001 | True | -| nested_individual_files | `f96dbd47704b` | 20.254 | 2.316 | 4.668 | 10.081 | 0.000 | 0.001 | True | -| pids_one_regression | `9270a8412a96` | 0.127 | 0.281 | 16.083 | 1.188 | 0.000 | 0.000 | True | -| source_dir_tree | `64c12911a3b8` | 3.576 | 0.916 | 2.688 | 4.486 | 0.000 | 0.000 | True | -| source_dir_tree | `67e7d6014831` | 3.474 | 0.509 | 4.735 | 5.519 | 0.000 | 0.001 | True | -| source_dir_tree | `eaf41c0a7f6e` | 6.942 | 0.903 | 3.083 | 7.851 | 0.000 | 0.000 | True | -| source_dir_tree | `fc7ef573df10` | 0.113 | 0.296 | 26.877 | 123.459 | 0.000 | 0.001 | True | -| symlink_input_consumer | `f22dfeb3cc2a` | 0.108 | 0.811 | 1.295 | 4.627 | 0.000 | 0.001 | True | -| timeout_recovery | `bc69c98c3696` | 0.129 | 0.298 | 12.433 | 1.158 | 0.000 | 0.000 | True | -| unknown | `e6380fd0c4ff` | 0.030 | 0.121 | 0.628 | 0.318 | 0.000 | 0.000 | True | +| Runner Stage | Min | p25 | p50 | p75 | p95 | Mean | Max | Share of execute | +| -------------- | ----: | ----: | ----: | -----: | -----: | ----: | -----: | ---------------: | +| parent prepare | 0.039 | 0.129 | 0.154 | 6.001 | 14.192 | 3.974 | 30.510 | 18.3% | +| fork | 0.143 | 0.346 | 0.463 | 1.037 | 5.559 | 1.483 | 15.454 | 6.8% | +| child setup | 0.008 | 1.626 | 5.090 | 11.872 | 19.394 | 8.753 | 74.287 | 40.3% | +| process/io | 0.474 | 2.012 | 3.644 | 5.844 | 14.223 | 6.837 | 83.844 | 31.5% | +| wait | 0.000 | 0.000 | 0.000 | 0.000 | 4.609 | 0.605 | 4.742 | 2.8% | +| stdio digest | 0.000 | 0.000 | 0.000 | 0.001 | 0.001 | 0.000 | 0.003 | 0.0% | + +| Stress Case | Digest | Parent Prep | Fork | Child Setup | Process/IO | Wait | Stdio Digest | Setup Signaled | +| -------------------------- | -------------- | ----------: | -----: | ----------: | ---------: | ----: | -----------: | -------------- | +| aggregate | `b685ab4a0d00` | 0.110 | 0.276 | 9.863 | 2.348 | 0.000 | 0.001 | True | +| bare_individual_files | `24c5cd3f112a` | 13.991 | 0.348 | 4.009 | 5.844 | 0.000 | 0.000 | True | +| bare_individual_files | `2623700af2b1` | 0.146 | 0.463 | 14.002 | 12.122 | 0.000 | 0.001 | True | +| bare_individual_files | `43f037c27458` | 14.996 | 0.722 | 3.040 | 1.851 | 1.962 | 0.000 | True | +| bare_individual_files | `feeb2d55641f` | 0.122 | 0.283 | 7.266 | 10.812 | 0.000 | 0.001 | True | +| filesystem_regression | `d132925a0328` | 0.123 | 3.175 | 15.351 | 83.844 | 0.000 | 0.000 | True | +| generated_file_producer | `e84ec75ce42e` | 0.154 | 8.245 | 5.814 | 8.351 | 2.104 | 0.000 | True | +| generated_individual_files | `118b63159b38` | 4.064 | 1.021 | 0.886 | 2.487 | 0.000 | 0.000 | True | +| generated_individual_files | `242c99eb2c28` | 2.561 | 1.778 | 3.018 | 2.371 | 0.000 | 0.000 | True | +| generated_individual_files | `78230e4a6f6d` | 3.444 | 1.892 | 3.274 | 7.660 | 0.000 | 0.000 | True | +| generated_individual_files | `ec087db43ffd` | 3.041 | 2.295 | 0.877 | 1.571 | 0.000 | 0.001 | True | +| generated_tree_producer | `d33821cebf3e` | 0.133 | 0.346 | 18.122 | 16.559 | 0.000 | 0.001 | True | +| generated_tree_reuse | `0bbda7aff823` | 10.205 | 0.554 | 2.021 | 2.012 | 0.000 | 0.000 | True | +| generated_tree_reuse | `28f302b72b2d` | 9.988 | 0.622 | 0.918 | 2.879 | 0.000 | 0.000 | True | +| generated_tree_reuse | `3fa11b1bc05f` | 9.799 | 0.394 | 1.626 | 4.833 | 4.586 | 0.000 | True | +| generated_tree_reuse | `63a12c93411c` | 0.122 | 0.447 | 1.838 | 4.593 | 0.000 | 0.000 | True | +| generated_tree_reuse | `654f662800a0` | 10.530 | 1.287 | 0.008 | 4.334 | 0.000 | 0.000 | True | +| generated_tree_reuse | `740cb972a7d0` | 0.143 | 0.528 | 13.035 | 4.378 | 0.000 | 0.000 | True | +| generated_tree_reuse | `9de4f80099e4` | 10.471 | 0.987 | 0.460 | 2.529 | 4.699 | 0.000 | True | +| generated_tree_reuse | `ce4b28b15b03` | 0.135 | 0.581 | 11.689 | 5.576 | 0.000 | 0.000 | True | +| mixed_all | `05385b608139` | 0.131 | 0.318 | 1.045 | 8.530 | 0.000 | 0.001 | True | +| nested_individual_files | `0576f836a969` | 0.129 | 2.737 | 13.501 | 8.299 | 4.307 | 0.000 | True | +| nested_individual_files | `0ba5a66718e5` | 10.286 | 0.503 | 3.945 | 1.783 | 0.000 | 0.000 | True | +| nested_individual_files | `1dddb1244c87` | 0.100 | 0.288 | 11.872 | 1.837 | 0.000 | 0.003 | True | +| nested_individual_files | `5043fa575aa6` | 0.140 | 0.233 | 2.211 | 1.442 | 4.742 | 0.000 | True | +| nested_individual_files | `79f9b55d3fb8` | 0.181 | 0.410 | 10.182 | 5.684 | 0.000 | 0.001 | True | +| nested_individual_files | `c10c168c3ea7` | 3.846 | 0.414 | 1.603 | 4.816 | 0.000 | 0.000 | True | +| nested_individual_files | `df42d8f5a591` | 6.001 | 0.455 | 20.712 | 1.628 | 0.000 | 0.000 | True | +| nested_individual_files | `f1239aa39480` | 30.510 | 0.444 | 9.207 | 2.468 | 0.000 | 0.000 | True | +| pids_one_regression | `f4c69ffb2998` | 0.294 | 0.354 | 13.795 | 1.119 | 0.000 | 0.001 | True | +| source_dir_tree | `280e6ad6e0fe` | 0.120 | 4.888 | 9.557 | 2.830 | 0.000 | 0.000 | True | +| source_dir_tree | `35fb441c98d9` | 0.442 | 1.037 | 5.090 | 4.158 | 0.000 | 0.000 | True | +| source_dir_tree | `685a1ba5f47a` | 0.117 | 0.262 | 74.287 | 13.639 | 0.000 | 0.001 | True | +| source_dir_tree | `6a2a37eadcc5` | 0.120 | 15.454 | 8.903 | 1.408 | 0.000 | 0.001 | True | +| symlink_input_consumer | `2698574104c6` | 0.144 | 0.345 | 0.904 | 3.644 | 0.000 | 0.000 | True | +| timeout_recovery | `dc6f23f617c1` | 0.148 | 0.336 | 19.064 | 2.254 | 0.000 | 0.001 | True | +| unknown | `9d7a2c7012de` | 0.039 | 0.143 | 0.863 | 0.474 | 0.000 | 0.000 | True | ## VM Bridge Timing These are raw TCP-to-vsock pump connection measurements logged by `darwin-actiond serve-vm`. -- Bridge connections logged: `10` -- Total client to guest bytes: `1.47 MiB` +- Bridge connections logged: `14` +- Total client to guest bytes: `1.51 MiB` - Total guest to client bytes: `0.30 MiB` - Pump errors: read=`0`, write=`0` | Bridge Metric | Min | p25 | p50 | p75 | p95 | Mean | Max | | ---------------------- | ------: | -------: | -------: | -------: | -------: | -------: | -------: | -| connection elapsed | 186.088 | 1190.171 | 1324.212 | 2915.719 | 5545.221 | 2203.577 | 5575.022 | -| client to guest KiB | 0.0 | 19.6 | 141.4 | 184.7 | 431.1 | 150.8 | 581.9 | -| guest to client KiB | 0.0 | 1.3 | 21.6 | 46.7 | 86.5 | 31.0 | 103.5 | -| client to guest reads | 1 | 12 | 92 | 260 | 314 | 132.1 | 339 | -| client to guest writes | 1 | 12 | 92 | 260 | 314 | 132.1 | 339 | -| guest to client reads | 0 | 10 | 91 | 198 | 246 | 108.7 | 270 | -| guest to client writes | 0 | 10 | 91 | 198 | 246 | 108.7 | 270 | +| connection elapsed | 242.059 | 1063.933 | 1132.733 | 2243.634 | 5052.708 | 1803.113 | 5109.920 | +| client to guest KiB | 0.0 | 30.9 | 79.8 | 122.4 | 317.5 | 110.6 | 587.5 | +| guest to client KiB | 0.0 | 2.2 | 18.1 | 20.3 | 74.8 | 22.1 | 114.3 | +| client to guest reads | 1 | 13 | 112 | 161 | 202 | 100.6 | 212 | +| client to guest writes | 1 | 13 | 112 | 161 | 202 | 100.6 | 212 | +| guest to client reads | 0 | 14 | 62 | 78 | 116 | 57.0 | 147 | +| guest to client writes | 0 | 14 | 62 | 78 | 116 | 57.0 | 147 | ## Per Action | Stress Case | Digest | Total | Input | Execute | Output | Bind Mounts | Outputs | | -------------------------- | -------------- | ------: | ----: | ------: | -----: | ----------: | -------------: | -| aggregate | `097d7589806b` | 20.198 | 0.297 | 19.833 | 0.069 | 4 | 1 file, 0 dir | -| bare_individual_files | `90cce354641e` | 131.135 | 0.288 | 130.680 | 0.166 | 4 | 1 file, 1 dir | -| bare_individual_files | `cfea754f9880` | 70.986 | 7.749 | 62.231 | 1.005 | 4 | 1 file, 1 dir | -| bare_individual_files | `d227993ae34f` | 20.741 | 0.319 | 20.171 | 0.250 | 4 | 1 file, 1 dir | -| bare_individual_files | `faa36aaa12ef` | 37.511 | 0.303 | 20.208 | 17.000 | 4 | 1 file, 1 dir | -| filesystem_regression | `d6e431f72b37` | 186.060 | 2.613 | 182.837 | 0.609 | 4 | 1 file, 1 dir | -| generated_file_producer | `0ab29dfeee67` | 72.237 | 1.088 | 66.935 | 4.214 | 4 | 97 file, 0 dir | -| generated_individual_files | `8949618893dd` | 78.581 | 0.375 | 77.865 | 0.341 | 4 | 1 file, 1 dir | -| generated_individual_files | `8e7053a36faf` | 19.752 | 0.300 | 19.146 | 0.306 | 4 | 1 file, 1 dir | -| generated_individual_files | `ac62596d2eae` | 15.891 | 0.291 | 15.140 | 0.460 | 4 | 1 file, 1 dir | -| generated_individual_files | `d7beec3fc619` | 15.812 | 0.321 | 15.358 | 0.133 | 4 | 1 file, 1 dir | -| generated_tree_producer | `f8cb8cf8722c` | 178.003 | 0.302 | 115.459 | 62.242 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `4f9d33590a97` | 64.459 | 0.303 | 64.028 | 0.128 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `ad080d1d4b46` | 36.264 | 0.651 | 34.749 | 0.864 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `b82ea83117e0` | 111.101 | 0.637 | 110.325 | 0.139 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `bdbccf204e0e` | 19.132 | 0.455 | 18.506 | 0.171 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `c222b682e1e5` | 110.509 | 0.535 | 109.871 | 0.103 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `cc2d17d56174` | 60.761 | 0.387 | 60.077 | 0.297 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `ec3c5569c6cf` | 114.790 | 0.375 | 114.232 | 0.183 | 4 | 1 file, 1 dir | -| generated_tree_reuse | `f58a3550a687` | 36.810 | 0.723 | 35.231 | 0.856 | 4 | 1 file, 1 dir | -| mixed_all | `ff909e41f5d7` | 114.812 | 0.353 | 111.190 | 3.268 | 4 | 1 file, 1 dir | -| nested_individual_files | `515cc9e83e3d` | 72.038 | 0.302 | 68.909 | 2.826 | 4 | 1 file, 1 dir | -| nested_individual_files | `6b51fb6999bc` | 76.960 | 1.043 | 75.415 | 0.502 | 4 | 1 file, 1 dir | -| nested_individual_files | `816552fd7392` | 211.392 | 0.308 | 210.164 | 0.919 | 4 | 1 file, 1 dir | -| nested_individual_files | `97febd68ffa3` | 46.697 | 0.297 | 45.921 | 0.479 | 4 | 1 file, 1 dir | -| nested_individual_files | `a85c70689335` | 67.853 | 0.330 | 67.062 | 0.460 | 4 | 1 file, 1 dir | -| nested_individual_files | `c595420ad290` | 44.681 | 0.355 | 43.797 | 0.528 | 4 | 1 file, 1 dir | -| nested_individual_files | `f697e529f79f` | 136.907 | 0.308 | 136.126 | 0.472 | 4 | 1 file, 1 dir | -| nested_individual_files | `f96dbd47704b` | 41.547 | 3.642 | 37.486 | 0.419 | 4 | 1 file, 1 dir | -| pids_one_regression | `9270a8412a96` | 18.152 | 0.312 | 17.704 | 0.135 | 4 | 1 file, 1 dir | -| source_dir_tree | `64c12911a3b8` | 12.502 | 0.288 | 11.698 | 0.516 | 4 | 1 file, 1 dir | -| source_dir_tree | `67e7d6014831` | 14.753 | 0.283 | 14.325 | 0.145 | 4 | 1 file, 1 dir | -| source_dir_tree | `eaf41c0a7f6e` | 19.294 | 0.330 | 18.792 | 0.172 | 4 | 1 file, 1 dir | -| source_dir_tree | `fc7ef573df10` | 151.404 | 0.284 | 150.786 | 0.334 | 4 | 1 file, 1 dir | -| symlink_input_consumer | `f22dfeb3cc2a` | 7.427 | 0.308 | 6.867 | 0.251 | 4 | 1 file, 1 dir | -| timeout_recovery | `bc69c98c3696` | 14.744 | 0.569 | 14.049 | 0.126 | 4 | 1 file, 1 dir | -| unknown | `e6380fd0c4ff` | 1.448 | 0.237 | 1.106 | 0.105 | 2 | 1 file, 0 dir | +| aggregate | `b685ab4a0d00` | 12.985 | 0.300 | 12.625 | 0.060 | 4 | 1 file, 0 dir | +| bare_individual_files | `24c5cd3f112a` | 27.332 | 2.971 | 24.216 | 0.144 | 4 | 1 file, 1 dir | +| bare_individual_files | `2623700af2b1` | 28.285 | 0.413 | 26.756 | 1.117 | 4 | 1 file, 1 dir | +| bare_individual_files | `43f037c27458` | 23.081 | 0.283 | 22.636 | 0.161 | 4 | 1 file, 1 dir | +| bare_individual_files | `feeb2d55641f` | 19.477 | 0.399 | 18.503 | 0.575 | 4 | 1 file, 1 dir | +| filesystem_regression | `d132925a0328` | 105.540 | 0.947 | 102.521 | 2.071 | 4 | 1 file, 1 dir | +| generated_file_producer | `e84ec75ce42e` | 42.392 | 9.242 | 24.742 | 8.408 | 4 | 97 file, 0 dir | +| generated_individual_files | `118b63159b38` | 9.516 | 0.276 | 8.485 | 0.755 | 4 | 1 file, 1 dir | +| generated_individual_files | `242c99eb2c28` | 10.178 | 0.303 | 9.743 | 0.131 | 4 | 1 file, 1 dir | +| generated_individual_files | `78230e4a6f6d` | 16.768 | 0.343 | 16.294 | 0.131 | 4 | 1 file, 1 dir | +| generated_individual_files | `ec087db43ffd` | 10.412 | 0.536 | 7.827 | 2.049 | 4 | 1 file, 1 dir | +| generated_tree_producer | `d33821cebf3e` | 45.624 | 0.377 | 35.180 | 10.068 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `0bbda7aff823` | 15.830 | 0.596 | 15.124 | 0.110 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `28f302b72b2d` | 15.136 | 0.289 | 14.735 | 0.113 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `3fa11b1bc05f` | 22.070 | 0.697 | 21.263 | 0.110 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `63a12c93411c` | 7.636 | 0.313 | 7.021 | 0.302 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `654f662800a0` | 17.689 | 0.564 | 16.192 | 0.933 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `740cb972a7d0` | 22.083 | 3.158 | 18.115 | 0.810 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `9de4f80099e4` | 19.634 | 0.324 | 19.186 | 0.124 | 4 | 1 file, 1 dir | +| generated_tree_reuse | `ce4b28b15b03` | 21.001 | 2.270 | 18.013 | 0.718 | 4 | 1 file, 1 dir | +| mixed_all | `05385b608139` | 12.815 | 0.321 | 10.046 | 2.448 | 4 | 1 file, 1 dir | +| nested_individual_files | `0576f836a969` | 29.748 | 0.345 | 28.995 | 0.408 | 4 | 1 file, 1 dir | +| nested_individual_files | `0ba5a66718e5` | 17.951 | 0.542 | 16.541 | 0.869 | 4 | 1 file, 1 dir | +| nested_individual_files | `1dddb1244c87` | 14.939 | 0.321 | 14.131 | 0.486 | 4 | 1 file, 1 dir | +| nested_individual_files | `5043fa575aa6` | 17.580 | 7.401 | 8.830 | 1.349 | 4 | 1 file, 1 dir | +| nested_individual_files | `79f9b55d3fb8` | 20.173 | 0.519 | 16.507 | 3.147 | 4 | 1 file, 1 dir | +| nested_individual_files | `c10c168c3ea7` | 11.994 | 0.608 | 10.722 | 0.663 | 4 | 1 file, 1 dir | +| nested_individual_files | `df42d8f5a591` | 29.843 | 0.310 | 28.834 | 0.698 | 4 | 1 file, 1 dir | +| nested_individual_files | `f1239aa39480` | 44.671 | 0.391 | 42.671 | 1.609 | 4 | 1 file, 1 dir | +| pids_one_regression | `f4c69ffb2998` | 16.760 | 1.012 | 15.606 | 0.142 | 4 | 1 file, 1 dir | +| source_dir_tree | `280e6ad6e0fe` | 17.967 | 0.394 | 17.426 | 0.147 | 4 | 1 file, 1 dir | +| source_dir_tree | `35fb441c98d9` | 11.597 | 0.712 | 10.747 | 0.139 | 4 | 1 file, 1 dir | +| source_dir_tree | `685a1ba5f47a` | 90.436 | 0.897 | 88.363 | 1.175 | 4 | 1 file, 1 dir | +| source_dir_tree | `6a2a37eadcc5` | 26.434 | 0.316 | 25.931 | 0.186 | 4 | 1 file, 1 dir | +| symlink_input_consumer | `2698574104c6` | 5.609 | 0.331 | 5.061 | 0.217 | 4 | 1 file, 1 dir | +| timeout_recovery | `dc6f23f617c1` | 22.593 | 0.643 | 21.843 | 0.107 | 4 | 1 file, 1 dir | +| unknown | `9d7a2c7012de` | 1.886 | 0.232 | 1.538 | 0.117 | 2 | 1 file, 0 dir | ## Notes diff --git a/tools/e2e_action_tool.zig b/tools/e2e_action_tool.zig index f16839a..0d67b86 100644 --- a/tools/e2e_action_tool.zig +++ b/tools/e2e_action_tool.zig @@ -936,6 +936,36 @@ fn exerciseFileMetadata(io: std.Io, dir: std.Io.Dir) !void { ((try file.stat(io)).permissions.toMode() & 0o6000) == 0o6000, "staged chmod did not expose setuid/setgid permissions", ); + const zero_written = linux.write(file.handle, payload.ptr, 0); + switch (linux.errno(zero_written)) { + .SUCCESS => {}, + else => return filesystemFailure("zero-length staged write failed"), + } + try requireFilesystem(zero_written == 0, "zero-length staged write returned a nonzero byte count"); + try requireFilesystem( + ((try file.stat(io)).permissions.toMode() & 0o6000) == 0o6000, + "zero-length staged write cleared setuid/setgid permissions", + ); + + const written = linux.write(file.handle, payload.ptr, 1); + switch (linux.errno(written)) { + .SUCCESS => {}, + else => return filesystemFailure("staged write with setuid/setgid permissions failed"), + } + try requireFilesystem(written == 1, "staged permission regression write returned an unexpected byte count"); + try requireFilesystem( + ((try file.stat(io)).permissions.toMode() & 0o6000) == 0, + "staged write did not clear setuid/setgid permissions", + ); + + switch (linux.errno(linux.fchmod(file.handle, 0o6755))) { + .SUCCESS => {}, + else => return filesystemFailure("restoring setuid/setgid permissions before staged truncate failed"), + } + try requireFilesystem( + ((try file.stat(io)).permissions.toMode() & 0o6000) == 0o6000, + "staged chmod did not restore setuid/setgid permissions before truncate", + ); try file.setLength(io, payload.len); try requireFilesystem( ((try file.stat(io)).permissions.toMode() & 0o6000) == 0,