From 1080caf76264daaa7aaef07edaa69ee3f0c7e120 Mon Sep 17 00:00:00 2001 From: Jeremy Huntwork Date: Sun, 23 Aug 2026 11:35:59 -0400 Subject: [PATCH 1/2] Authenticate canonical store permission classes Store identity now distinguishes the read, execute, and special permission classes that survive admission, so packages with materially different runtime access cannot share a store path. Write bits, ownership, and other environment-dependent metadata remain outside identity. Manifest v5 selects the additive v4 identity while every prior manifest and hash format keeps its original meaning. Archive extraction now preserves ordinary directory permissions so packaged intent and verified realized state remain identical. --- docs/design/specification-details.md | 24 +++-- src/activation.zig | 42 +++------ src/extract.zig | 31 +++++-- src/hash.zig | 127 +++++++++++++++++++++++++-- src/import.zig | 31 +------ src/install.zig | 28 +++--- src/manifest.zig | 81 +++++++++++++++-- src/packaging.zig | 70 +++++++++++++-- src/verify.zig | 65 ++++---------- 9 files changed, 340 insertions(+), 159 deletions(-) diff --git a/docs/design/specification-details.md b/docs/design/specification-details.md index 0fa8b77..bf29843 100644 --- a/docs/design/specification-details.md +++ b/docs/design/specification-details.md @@ -101,8 +101,8 @@ Mere persists a number of formats, several of which are signed or content-addres | Format | Location | Discriminator | Written | Accepted | | --- | --- | --- | --- | --- | -| Store content hash (§1) | store path name | manifest format and `schema_version` | v3 | v1, transitional, v2, v3 | -| Package manifest (§17) | `.mere/manifest.v1` through `.mere/manifest.v4` | `schema_version` field and filename | v4 | v1, v2, v3, v4 | +| Store content hash (§1) | store path name | manifest format and `schema_version` | v4 | v1, transitional, v2, v3, v4 | +| Package manifest (§17) | `.mere/manifest.v1` through `.mere/manifest.v5` | `schema_version` field and filename | v5 | v1, v2, v3, v4, v5 | | Manifest signature (§5) | `.mere/manifest.vN.sig` | manifest format; v4 envelope magic/version/algorithm | domain-separated v2 envelope | legacy raw Ed25519, domain-separated v2 | | Key file | `*.pub`, `*.key` | `MEREKEY` magic, version and algorithm bytes | v1 / Ed25519 | v1 / Ed25519 | | Generation manifest (§6) | `/` | `schema_version` field | 2 | 2 only | @@ -138,6 +138,12 @@ path_len = u32 LE path = UTF-8 bytes, '/' separators, no leading slash type_tag = 1 byte: 0x10=file, 0x11=dir, 0x12=symlink +For v3, after type_tag when type is file or directory: + special_bits = 1 byte: (mode & 07000) >> 9 + +For v4, after type_tag when type is file or directory: + canonical_mode = u16 LE: mode & 07555 + If file (0x10): exec_bit = 1 byte: 0x00 (not executable) or 0x01 (executable) content_len = u64 LE @@ -164,8 +170,7 @@ If dir (0x11): - Example: `/mere/store/9f2c3a...64chars...-nginx-1.24.0/` **Metadata location**: -- `.mere/manifest.v1` (binary) and `.mere/manifest.v1.sig` (signature) in the `.mere/` subdirectory -- Both manifest files are **excluded from the content hash** (see spec #4) because the manifest contains the hash it describes and the signature authenticates that manifest +- `.mere/manifest.v1` through `.mere/manifest.v5` and their signatures are excluded from the content hash (see spec #4) because the selected manifest contains the hash it describes and its signature authenticates that manifest - `.mere/meta.kdl` is canonical package intent metadata and **is included in the content hash** - `.mere/projection.v1` is a derived index and **is excluded from the content hash**; it is regenerated and validated from the package contents and metadata - The `content_hash` field in the manifest and the hash in the store path are **identical** (same 32 bytes, displayed as 64 hex chars in path) @@ -176,17 +181,18 @@ The store content hash **MUST** incorporate: - File bytes - Path names - File type (file / directory / symlink) -- setuid, setgid, and sticky bits on files and directories (v3) +- setuid, setgid, and sticky bits on files and directories (v3 and v4) +- User/group/other read and execute permission classes on files and directories (v4) The store content hash **MUST NOT** incorporate: -- Read/write permission bits (other than executable) +- Write permission bits (v4 normalizes them away because admission removes them) - Ownership (uid/gid) - Timestamps - ACLs or extended attributes -**Normative invariant**: Two payloads that differ only in non-executable read/write permission bits or ownership are considered *identical content*. Under v3, two payloads that differ in a setuid, setgid, or sticky bit are different content. +**Normative invariant**: Two v4 payloads that differ only in write permission bits or ownership are considered *identical content*. Two v4 payloads that differ in a read, execute, setuid, setgid, or sticky bit are different content. Earlier variants retain their frozen identity rules. -Implementations MUST preserve extracted permission bits when unpacking archives. Special bits on files and directories participate in v3 identity; symlink modes do not. Read/write bits outside executable and ownership remain outside identity because they are environment-dependent. +Implementations MUST preserve extracted permission bits when unpacking archives. V4 authenticates the canonical post-hardening mode `mode & 07555` for files and directories; symlink modes do not participate. Ownership remains outside identity because privileged and unprivileged admission intentionally establish different owners. --- @@ -2047,6 +2053,8 @@ The package manifest is the authoritative source of package metadata, using a de **Field ordering is part of the canonical definition** - do not reorder fields. +The byte layout is shared by manifest formats v1-v5; the filename and `schema_version` select the immutable verification contract. New packages write `.mere/manifest.v5`, use the existing domain-separated signature envelope, and bind `content_hash` to store identity v4. Manifest v4 retains store identity v3 and MUST NOT be reinterpreted. + **Required fields (v1)**: - `schema_version`: Must be `1` - `name`: Package name (UTF-8, no null bytes) diff --git a/src/activation.zig b/src/activation.zig index 44598b8..3d90e56 100644 --- a/src/activation.zig +++ b/src/activation.zig @@ -553,41 +553,19 @@ fn validateGenerationStorePaths( return ctx.fail(ActivationError.InvalidInput, pkg.store_path, "invalid content hash length in manifest"); } - const format: package_manifest.Format = blk: { - const v4_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, package_manifest.MANIFEST_V4_FILENAME }) catch { - return ctx.fail(ActivationError.OutOfMemory, pkg.store_path, "failed to construct v4 manifest path"); - }; - defer ctx.allocator.free(v4_manifest_path); - const has_v4 = blk_v4: { - std.Io.Dir.accessAbsolute(path_mod.currentIo(), v4_manifest_path, .{}) catch break :blk_v4 false; - break :blk_v4 true; - }; - if (has_v4) break :blk .v4; - - const v3_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, package_manifest.MANIFEST_V3_FILENAME }) catch { - return ctx.fail(ActivationError.OutOfMemory, pkg.store_path, "failed to construct v3 manifest path"); - }; - defer ctx.allocator.free(v3_manifest_path); - const has_v3 = blk_v3: { - std.Io.Dir.accessAbsolute(path_mod.currentIo(), v3_manifest_path, .{}) catch break :blk_v3 false; - break :blk_v3 true; - }; - if (has_v3) break :blk .v3; - - const v2_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, package_manifest.MANIFEST_V2_FILENAME }) catch { - return ctx.fail(ActivationError.OutOfMemory, pkg.store_path, "failed to construct v2 manifest path"); - }; - defer ctx.allocator.free(v2_manifest_path); - const has_v2 = blk_v2: { - std.Io.Dir.accessAbsolute(path_mod.currentIo(), v2_manifest_path, .{}) catch break :blk_v2 false; - break :blk_v2 true; - }; - break :blk if (has_v2) .v2 else .v1; + const format = package_manifest.detectFormat(ctx.allocator, pkg.store_path) catch |err| { + return ctx.fail(switch (err) { + package_manifest.ManifestError.OutOfMemory => ActivationError.OutOfMemory, + package_manifest.ManifestError.PermissionDenied => ActivationError.PermissionDenied, + package_manifest.ManifestError.InvalidInput => ActivationError.InvalidInput, + else => ActivationError.FileSystem, + }, pkg.store_path, "failed to detect package manifest format"); }; - const computed = switch (format) { + const computed = switch (format.storeHashFormat()) { .v1 => hash.calculateStoreContentHash(ctx.allocator, pkg.store_path, null), .v2 => hash.calculateStoreContentHashV2(ctx.allocator, pkg.store_path, null), - .v3, .v4 => hash.calculateStoreContentHashV3(ctx.allocator, pkg.store_path, null), + .v3 => hash.calculateStoreContentHashV3(ctx.allocator, pkg.store_path, null), + .v4 => hash.calculateStoreContentHashV4(ctx.allocator, pkg.store_path, null), }; const computed_hash = computed catch |err| { return ctx.fail(switch (err) { diff --git a/src/extract.zig b/src/extract.zig index 9625710..b05e9a2 100644 --- a/src/extract.zig +++ b/src/extract.zig @@ -638,22 +638,39 @@ fn mergeStagedTree(ctx: *Context, staged_dir: []const u8, target_abs: []const u8 defer ctx.allocator.free(src_path); var slot = try openDestSlot(ctx, target_root, rel_path, false); defer slot.close(); - try copyDirectoryTimes(ctx, src_path, slot, rel_path); + try copyDirectoryMetadata(ctx, src_path, slot, rel_path); } } -fn copyDirectoryTimes( +fn copyDirectoryMetadata( ctx: *Context, src_path: []const u8, dst: DestSlot, diag_path: []const u8, ) ExtractError!void { - var src_dir = std.Io.Dir.openDirAbsolute(p.currentIo(), src_path, .{}) catch |err| { - return ctx.fail(mapFsError(err), src_path, "failed to open staged directory for timestamp copy"); + const io = p.currentIo(); + var src_dir = std.Io.Dir.openDirAbsolute(io, src_path, .{}) catch |err| { + return ctx.fail(mapFsError(err), src_path, "failed to open staged directory for metadata copy"); + }; + defer src_dir.close(io); + const stat = src_dir.stat(io) catch |err| { + return ctx.fail(mapFsError(err), src_path, "failed to stat staged directory for metadata copy"); + }; + + var dst_dir = dst.dir.openDir(io, dst.name, .{ + .iterate = true, + .follow_symlinks = false, + }) catch |err| switch (err) { + error.SymLinkLoop => return ctx.fail(ExtractError.InvalidInput, diag_path, "directory metadata target is a symlink"), + else => return ctx.fail(mapFsError(err), diag_path, "failed to open directory for metadata copy"), }; - defer src_dir.close(p.currentIo()); - const stat = src_dir.stat(p.currentIo()) catch |err| { - return ctx.fail(mapFsError(err), src_path, "failed to stat staged directory for timestamp copy"); + defer dst_dir.close(io); + // The staged merge creates destination directories with process defaults. + // Restore ordinary archive permissions here; special bits remain governed + // by SpecialBitRestorePolicy and are applied separately when requested. + const ordinary_mode = stat.permissions.toMode() & @as(std.posix.mode_t, 0o777); + dst_dir.setPermissions(io, std.Io.File.Permissions.fromMode(ordinary_mode)) catch |err| { + return ctx.fail(mapFsError(err), diag_path, "failed to restore directory permissions"); }; const name_z = ctx.allocator.dupeZ(u8, dst.name) catch { diff --git a/src/hash.zig b/src/hash.zig index 046bbea..b83bf97 100644 --- a/src/hash.zig +++ b/src/hash.zig @@ -108,6 +108,12 @@ const EntryType = enum(u8) { const ENTRY_TAG: u8 = 0x01; +const ModeIdentity = enum { + none, + special, + canonical, +}; + /// The v1 store identity: realized payload only. This is retained for /// compatibility with all packages published before metadata-aware identity. pub fn calculateStoreContentHash( @@ -115,7 +121,7 @@ pub fn calculateStoreContentHash( dir_path: []const u8, diag: ?*HashDiag, ) HashError![]const u8 { - return calculateTreeHashInternal(allocator, dir_path, diag, false, false, false, null); + return calculateTreeHashInternal(allocator, dir_path, diag, false, false, .none, null); } /// The transitional v0.18.0 identity: payload plus meta.kdl, without a @@ -126,7 +132,7 @@ pub fn calculateTransitionalMetadataContentHash( dir_path: []const u8, diag: ?*HashDiag, ) HashError![]const u8 { - return calculateTreeHashInternal(allocator, dir_path, diag, false, true, false, null); + return calculateTreeHashInternal(allocator, dir_path, diag, false, true, .none, null); } /// The versioned metadata-aware store identity used by new packages. @@ -135,7 +141,7 @@ pub fn calculateStoreContentHashV2( dir_path: []const u8, diag: ?*HashDiag, ) HashError![]const u8 { - return calculateTreeHashInternal(allocator, dir_path, diag, false, true, false, "mere-store-content-v2\x00"); + return calculateTreeHashInternal(allocator, dir_path, diag, false, true, .none, "mere-store-content-v2\x00"); } /// The v3 metadata-aware store identity includes setuid, setgid, and sticky @@ -146,7 +152,18 @@ pub fn calculateStoreContentHashV3( dir_path: []const u8, diag: ?*HashDiag, ) HashError![]const u8 { - return calculateTreeHashInternal(allocator, dir_path, diag, false, true, true, "mere-store-content-v3\x00"); + return calculateTreeHashInternal(allocator, dir_path, diag, false, true, .special, "mere-store-content-v3\x00"); +} + +/// The v4 store identity authenticates the mode that survives admission: +/// user/group/other read and execute classes plus setuid, setgid, and sticky. +/// Write bits are normalized away because store hardening removes them. +pub fn calculateStoreContentHashV4( + allocator: std.mem.Allocator, + dir_path: []const u8, + diag: ?*HashDiag, +) HashError![]const u8 { + return calculateTreeHashInternal(allocator, dir_path, diag, false, true, .canonical, "mere-store-content-v4\x00"); } pub fn calculateBuildSnapshotHash( @@ -154,7 +171,7 @@ pub fn calculateBuildSnapshotHash( dir_path: []const u8, diag: ?*HashDiag, ) HashError![]const u8 { - return calculateTreeHashInternal(allocator, dir_path, diag, true, false, false, null); + return calculateTreeHashInternal(allocator, dir_path, diag, true, false, .none, null); } fn calculateTreeHashInternal( @@ -163,7 +180,7 @@ fn calculateTreeHashInternal( diag: ?*HashDiag, include_mtime: bool, include_metadata: bool, - include_special_bits: bool, + mode_identity: ModeIdentity, domain: ?[]const u8, ) HashError![]const u8 { if (!path.isValidInputPath(dir_path)) { @@ -264,9 +281,17 @@ fn calculateTreeHashInternal( }; hasher.update(&[_]u8{type_tag}); - if (include_special_bits and entry.kind != .sym_link) { - const special: u8 = @intCast((entry.mode & 0o7000) >> 9); - hasher.update(&[_]u8{special}); + switch (mode_identity) { + .none => {}, + .special => if (entry.kind != .sym_link) { + const special: u8 = @intCast((entry.mode & 0o7000) >> 9); + hasher.update(&[_]u8{special}); + }, + .canonical => if (entry.kind != .sym_link) { + const canonical_mode: u16 = @intCast(entry.mode & 0o7555); + const canonical_mode_le = std.mem.nativeToLittle(u16, canonical_mode); + hasher.update(&std.mem.toBytes(canonical_mode_le)); + }, } if (include_mtime) { @@ -998,3 +1023,87 @@ test "calculateStoreContentHashV3 distinguishes special bits while v2 does not" try std.testing.expectEqualStrings(v2_plain, v2_setuid); try std.testing.expect(!std.mem.eql(u8, v3_plain, v3_setuid)); } + +test "calculateStoreContentHashV4 authenticates canonical permission classes" { + const th = @import("test_helpers.zig"); + var test_env = try th.createTestEnv(); + defer { + test_env.cleanup(); + std.testing.allocator.destroy(test_env); + } + + const keys = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, ".mere", "keys" }); + defer test_env.ctx.allocator.free(keys); + try path.deleteTreeAbsolute(keys); + + const file_path = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "tool" }); + defer test_env.ctx.allocator.free(file_path); + var file = try std.Io.Dir.createFileAbsolute(path.currentIo(), file_path, .{}); + try file.writeStreamingAll(path.currentIo(), "payload"); + file.close(path.currentIo()); + file = try path.openExistingFile(file_path); + defer file.close(path.currentIo()); + + try file.setPermissions(path.currentIo(), .fromMode(0o755)); + const public_exec = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(public_exec); + + try file.setPermissions(path.currentIo(), .fromMode(0o555)); + const hardened_public_exec = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(hardened_public_exec); + try std.testing.expectEqualStrings(public_exec, hardened_public_exec); + + try file.setPermissions(path.currentIo(), .fromMode(0o500)); + const owner_exec_v3 = try calculateStoreContentHashV3(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(owner_exec_v3); + const owner_exec_v4 = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(owner_exec_v4); + + try file.setPermissions(path.currentIo(), .fromMode(0o501)); + const other_exec_v3 = try calculateStoreContentHashV3(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(other_exec_v3); + const other_exec_v4 = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(other_exec_v4); + + try std.testing.expectEqualStrings(owner_exec_v3, other_exec_v3); + try std.testing.expect(!std.mem.eql(u8, owner_exec_v4, other_exec_v4)); + + try file.setPermissions(path.currentIo(), .fromMode(0o644)); + const public_read = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(public_read); + try file.setPermissions(path.currentIo(), .fromMode(0o600)); + const owner_read = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(owner_read); + try std.testing.expect(!std.mem.eql(u8, public_read, owner_read)); +} + +test "calculateStoreContentHashV4 fixed protocol vector" { + const th = @import("test_helpers.zig"); + var test_env = try th.createTestEnv(); + defer { + test_env.cleanup(); + std.testing.allocator.destroy(test_env); + } + + const keys = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, ".mere", "keys" }); + defer test_env.ctx.allocator.free(keys); + try path.deleteTreeAbsolute(keys); + + const dir_path = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "private" }); + defer test_env.ctx.allocator.free(dir_path); + try path.ensureDirExists(dir_path); + var dir = try std.Io.Dir.openDirAbsolute(path.currentIo(), dir_path, .{ .iterate = true }); + try dir.setPermissions(path.currentIo(), .fromMode(0o750)); + dir.close(path.currentIo()); + + const file_path = try std.fs.path.join(test_env.ctx.allocator, &.{ dir_path, "tool" }); + defer test_env.ctx.allocator.free(file_path); + var file = try std.Io.Dir.createFileAbsolute(path.currentIo(), file_path, .{}); + try file.writeStreamingAll(path.currentIo(), "payload\n"); + try file.setPermissions(path.currentIo(), .fromMode(0o4750)); + file.close(path.currentIo()); + + const actual = try calculateStoreContentHashV4(test_env.ctx.allocator, test_env.path, null); + defer test_env.ctx.allocator.free(actual); + try std.testing.expectEqualStrings("dc7d87316164e2afb509a4d42e7e95671577ad4bb6f670bcce67e1988ca0b114", actual); +} diff --git a/src/import.zig b/src/import.zig index 501c17d..8a164c6 100644 --- a/src/import.zig +++ b/src/import.zig @@ -209,31 +209,7 @@ pub const ManifestResult = struct { }; fn detectManifestFormat(ctx: *Context, temp_dir: []const u8) !manifest.Format { - const v4_path = try std.fs.path.join(ctx.allocator, &.{ temp_dir, manifest.MANIFEST_V4_FILENAME }); - defer ctx.allocator.free(v4_path); - const has_v4 = blk: { - std.Io.Dir.accessAbsolute(p.currentIo(), v4_path, .{}) catch break :blk false; - break :blk true; - }; - if (has_v4) return .v4; - - const v3_path = try std.fs.path.join(ctx.allocator, &.{ temp_dir, manifest.MANIFEST_V3_FILENAME }); - defer ctx.allocator.free(v3_path); - const has_v3 = blk: { - std.Io.Dir.accessAbsolute(p.currentIo(), v3_path, .{}) catch break :blk false; - break :blk true; - }; - if (has_v3) return .v3; - - const v2_path = try std.fs.path.join(ctx.allocator, &.{ temp_dir, manifest.MANIFEST_V2_FILENAME }); - defer ctx.allocator.free(v2_path); - const has_v2 = blk: { - std.Io.Dir.accessAbsolute(p.currentIo(), v2_path, .{}) catch break :blk false; - break :blk true; - }; - if (has_v2) return .v2; - - return .v1; + return manifest.detectFormat(ctx.allocator, temp_dir); } const PreparedImport = struct { extract: ExtractResult, @@ -404,10 +380,11 @@ fn appendJsonEscaped(buf: *std.ArrayList(u8), allocator: std.mem.Allocator, valu } fn computeAndVerifyContentHash(ctx: *Context, temp_dir: []const u8, pkg_manifest: *const manifest.PackageManifestV1, format: manifest.Format) !void { - const computed_hash = switch (format) { + const computed_hash = switch (format.storeHashFormat()) { .v1 => try hash.calculateStoreContentHash(ctx.allocator, temp_dir, null), .v2 => try hash.calculateStoreContentHashV2(ctx.allocator, temp_dir, null), - .v3, .v4 => try hash.calculateStoreContentHashV3(ctx.allocator, temp_dir, null), + .v3 => try hash.calculateStoreContentHashV3(ctx.allocator, temp_dir, null), + .v4 => try hash.calculateStoreContentHashV4(ctx.allocator, temp_dir, null), }; defer ctx.allocator.free(computed_hash); diff --git a/src/install.zig b/src/install.zig index 05be5df..37516e2 100644 --- a/src/install.zig +++ b/src/install.zig @@ -2281,22 +2281,13 @@ fn preVerifyManifest( ctx.debug("partial-extracting manifest for pre-verification", .{}); var format: manifest.Format = .v1; - extract.fileInto(ctx, cache_path, verify_dir, manifest.MANIFEST_V4_FILENAME) catch {}; - const v4_probe_path = try std.fs.path.join(ctx.allocator, &.{ verify_dir, manifest.MANIFEST_V4_FILENAME }); - defer ctx.allocator.free(v4_probe_path); - if (path.fileExists(v4_probe_path)) { - format = .v4; - } else { - extract.fileInto(ctx, cache_path, verify_dir, manifest.MANIFEST_V3_FILENAME) catch {}; - const v3_probe_path = try std.fs.path.join(ctx.allocator, &.{ verify_dir, manifest.MANIFEST_V3_FILENAME }); - defer ctx.allocator.free(v3_probe_path); - if (path.fileExists(v3_probe_path)) { - format = .v3; - } else { - extract.fileInto(ctx, cache_path, verify_dir, manifest.MANIFEST_V2_FILENAME) catch {}; - const v2_probe_path = try std.fs.path.join(ctx.allocator, &.{ verify_dir, manifest.MANIFEST_V2_FILENAME }); - defer ctx.allocator.free(v2_probe_path); - if (path.fileExists(v2_probe_path)) format = .v2; + for (manifest.formats_newest_first) |candidate| { + extract.fileInto(ctx, cache_path, verify_dir, candidate.manifestFilename()) catch {}; + const probe_path = try std.fs.path.join(ctx.allocator, &.{ verify_dir, candidate.manifestFilename() }); + defer ctx.allocator.free(probe_path); + if (path.fileExists(probe_path)) { + format = candidate; + break; } } @@ -2440,10 +2431,11 @@ fn stageAndValidatePayload( // Compute content hash from realized payload and canonical package metadata (spec #1/#4) var hash_diag: hash.HashDiag = .{}; defer hash_diag.deinit(ctx.allocator); - var content_hash: []const u8 = switch (format) { + var content_hash: []const u8 = switch (format.storeHashFormat()) { .v1 => hash.calculateStoreContentHash(ctx.allocator, staging_dir, &hash_diag), .v2 => hash.calculateStoreContentHashV2(ctx.allocator, staging_dir, &hash_diag), - .v3, .v4 => hash.calculateStoreContentHashV3(ctx.allocator, staging_dir, &hash_diag), + .v3 => hash.calculateStoreContentHashV3(ctx.allocator, staging_dir, &hash_diag), + .v4 => hash.calculateStoreContentHashV4(ctx.allocator, staging_dir, &hash_diag), } catch |err| { const action = hash_diag.action orelse "compute content hash"; const path_label = hash_diag.path orelse staging_dir; diff --git a/src/manifest.zig b/src/manifest.zig index b28f4dc..dd34d0a 100644 --- a/src/manifest.zig +++ b/src/manifest.zig @@ -12,6 +12,7 @@ pub const SCHEMA_VERSION: u32 = 1; pub const SCHEMA_VERSION_V2: u32 = 2; pub const SCHEMA_VERSION_V3: u32 = 3; pub const SCHEMA_VERSION_V4: u32 = 4; +pub const SCHEMA_VERSION_V5: u32 = 5; pub const META_DIR = ".mere"; pub const MANIFEST_FILENAME = ".mere/manifest.v1"; pub const MANIFEST_SIG_FILENAME = ".mere/manifest.v1.sig"; @@ -21,14 +22,24 @@ pub const MANIFEST_V3_FILENAME = ".mere/manifest.v3"; pub const MANIFEST_V3_SIG_FILENAME = ".mere/manifest.v3.sig"; pub const MANIFEST_V4_FILENAME = ".mere/manifest.v4"; pub const MANIFEST_V4_SIG_FILENAME = ".mere/manifest.v4.sig"; +pub const MANIFEST_V5_FILENAME = ".mere/manifest.v5"; +pub const MANIFEST_V5_SIG_FILENAME = ".mere/manifest.v5.sig"; pub const META_KDL_FILENAME = ".mere/meta.kdl"; pub const PROJECTION_FILENAME = ".mere/projection.v1"; +pub const StoreHashFormat = enum { + v1, + v2, + v3, + v4, +}; + pub const Format = enum { v1, v2, v3, v4, + v5, pub fn manifestFilename(self: Format) []const u8 { return switch (self) { @@ -36,6 +47,7 @@ pub const Format = enum { .v2 => MANIFEST_V2_FILENAME, .v3 => MANIFEST_V3_FILENAME, .v4 => MANIFEST_V4_FILENAME, + .v5 => MANIFEST_V5_FILENAME, }; } @@ -45,6 +57,7 @@ pub const Format = enum { .v2 => MANIFEST_V2_SIG_FILENAME, .v3 => MANIFEST_V3_SIG_FILENAME, .v4 => MANIFEST_V4_SIG_FILENAME, + .v5 => MANIFEST_V5_SIG_FILENAME, }; } @@ -54,21 +67,29 @@ pub const Format = enum { .v2 => SCHEMA_VERSION_V2, .v3 => SCHEMA_VERSION_V3, .v4 => SCHEMA_VERSION_V4, + .v5 => SCHEMA_VERSION_V5, }; } pub fn signatureFormat(self: Format) sign.ManifestSignatureFormat { return switch (self) { .v1, .v2, .v3 => .legacy_raw, - .v4 => .domain_v2, + .v4, .v5 => .domain_v2, }; } - pub fn usesStoreHashV3(self: Format) bool { - return self == .v3 or self == .v4; + pub fn storeHashFormat(self: Format) StoreHashFormat { + return switch (self) { + .v1 => .v1, + .v2 => .v2, + .v3, .v4 => .v3, + .v5 => .v4, + }; } }; +pub const formats_newest_first = [_]Format{ .v5, .v4, .v3, .v2, .v1 }; + pub const PackageManifestV1 = struct { schema_version: u32, created_at: u64, @@ -205,6 +226,19 @@ pub const PackageManifestV1 = struct { } }; +pub fn detectFormat(allocator: std.mem.Allocator, dir_path: []const u8) ManifestError!Format { + for (formats_newest_first) |format| { + const manifest_path = std.fs.path.join(allocator, &.{ dir_path, format.manifestFilename() }) catch { + return ManifestError.OutOfMemory; + }; + defer allocator.free(manifest_path); + if (std.Io.Dir.accessAbsolute(path.currentIo(), manifest_path, .{})) |_| { + return format; + } else |_| {} + } + return .v1; +} + pub fn readManifestFile(ctx: *Context, dir_path: []const u8) ManifestError![]u8 { return readManifestFileForFormat(ctx, dir_path, .v1); } @@ -271,6 +305,10 @@ pub fn writeManifestV4(ctx: *Context, dir_path: []const u8, manifest: *const Pac return writeManifestForFormat(ctx, dir_path, manifest, secret_key, .v4); } +pub fn writeManifestV5(ctx: *Context, dir_path: []const u8, manifest: *const PackageManifestV1, secret_key: []const u8) ManifestError!void { + return writeManifestForFormat(ctx, dir_path, manifest, secret_key, .v5); +} + fn writeManifestForFormat(ctx: *Context, dir_path: []const u8, input: *const PackageManifestV1, secret_key: []const u8, format: Format) ManifestError!void { var manifest_copy = input.*; manifest_copy.schema_version = format.schemaVersion(); @@ -523,11 +561,44 @@ test "readManifestFile reports InvalidInput when manifest is missing" { try std.testing.expectError(ManifestError.InvalidInput, readManifestFile(&test_env.ctx, package_dir)); } -test "manifest v4 separates signature format from store hash identity" { +test "manifest formats separate signature and store hash identities" { try std.testing.expectEqual(@as(u32, 4), Format.v4.schemaVersion()); try std.testing.expectEqualStrings(MANIFEST_V4_FILENAME, Format.v4.manifestFilename()); try std.testing.expectEqualStrings(MANIFEST_V4_SIG_FILENAME, Format.v4.signatureFilename()); try std.testing.expectEqual(sign.ManifestSignatureFormat.domain_v2, Format.v4.signatureFormat()); try std.testing.expectEqual(sign.ManifestSignatureFormat.legacy_raw, Format.v3.signatureFormat()); - try std.testing.expect(Format.v4.usesStoreHashV3()); + try std.testing.expectEqual(StoreHashFormat.v3, Format.v4.storeHashFormat()); + + try std.testing.expectEqual(@as(u32, 5), Format.v5.schemaVersion()); + try std.testing.expectEqualStrings(MANIFEST_V5_FILENAME, Format.v5.manifestFilename()); + try std.testing.expectEqualStrings(MANIFEST_V5_SIG_FILENAME, Format.v5.signatureFilename()); + try std.testing.expectEqual(sign.ManifestSignatureFormat.domain_v2, Format.v5.signatureFormat()); + try std.testing.expectEqual(StoreHashFormat.v4, Format.v5.storeHashFormat()); +} + +test "detectFormat selects the newest manifest without fallback" { + const th = @import("test_helpers.zig"); + var test_env = try th.createTestEnv(); + defer { + test_env.cleanup(); + std.testing.allocator.destroy(test_env); + } + + const package_dir = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "pkg" }); + defer test_env.ctx.allocator.free(package_dir); + const meta_dir = try std.fs.path.join(test_env.ctx.allocator, &.{ package_dir, META_DIR }); + defer test_env.ctx.allocator.free(meta_dir); + try path.ensureDirExists(meta_dir); + + const v4_path = try std.fs.path.join(test_env.ctx.allocator, &.{ package_dir, MANIFEST_V4_FILENAME }); + defer test_env.ctx.allocator.free(v4_path); + var v4_file = try std.Io.Dir.createFileAbsolute(path.currentIo(), v4_path, .{}); + v4_file.close(path.currentIo()); + try std.testing.expectEqual(Format.v4, try detectFormat(test_env.ctx.allocator, package_dir)); + + const v5_path = try std.fs.path.join(test_env.ctx.allocator, &.{ package_dir, MANIFEST_V5_FILENAME }); + defer test_env.ctx.allocator.free(v5_path); + var v5_file = try std.Io.Dir.createFileAbsolute(path.currentIo(), v5_path, .{}); + v5_file.close(path.currentIo()); + try std.testing.expectEqual(Format.v5, try detectFormat(test_env.ctx.allocator, package_dir)); } diff --git a/src/packaging.zig b/src/packaging.zig index 518fe6b..b98d300 100644 --- a/src/packaging.zig +++ b/src/packaging.zig @@ -329,11 +329,11 @@ pub const Packager = struct { return self.fail(config.staging_dir, "failed to write meta.kdl", PackagingError.FileSystem); }; - // meta.kdl is part of the versioned v3 store identity. Manifest v4 - // keeps that identity while introducing domain-separated signatures. + // meta.kdl is part of the v4 store identity. Manifest v5 selects that + // identity while retaining the domain-separated signature envelope. self.ctx.allocator.free(content_hash); - content_hash = hash.calculateStoreContentHashV3(self.ctx.allocator, config.staging_dir, null) catch { - return self.fail(config.staging_dir, "failed to compute v3 content hash", PackagingError.CreationFailed); + content_hash = hash.calculateStoreContentHashV4(self.ctx.allocator, config.staging_dir, null) catch { + return self.fail(config.staging_dir, "failed to compute v4 content hash", PackagingError.CreationFailed); }; if (pkg.content_hash.len > 0) self.ctx.allocator.free(pkg.content_hash); pkg.content_hash = self.ctx.allocator.dupe(u8, content_hash) catch |err| { @@ -347,7 +347,7 @@ pub const Packager = struct { return self.fail(content_hash, "invalid content hash hex", PackagingError.InvalidInput); }; pkg_manifest.content_hash = final_content_hash_bytes; - manifest.writeManifestV4(self.ctx, config.staging_dir, &pkg_manifest, &secret_key.key) catch { + manifest.writeManifestV5(self.ctx, config.staging_dir, &pkg_manifest, &secret_key.key) catch { self.ctx.allocator.free(content_hash); return self.fail(config.staging_dir, "failed to write final manifest", PackagingError.FileSystem); }; @@ -467,6 +467,52 @@ pub const Packager = struct { const testing = std.testing; const test_helpers = @import("test_helpers.zig"); +test "package archive round trip preserves ordinary permission classes" { + var test_env = try test_helpers.createTestEnv(); + defer { + test_env.cleanup(); + std.testing.allocator.destroy(test_env); + } + + const staging = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "mode-staging" }); + defer test_env.ctx.allocator.free(staging); + const private_dir = try std.fs.path.join(test_env.ctx.allocator, &.{ staging, "private" }); + defer test_env.ctx.allocator.free(private_dir); + try path_mod.ensureDirExists(private_dir); + var private_handle = try std.Io.Dir.openDirAbsolute(path_mod.currentIo(), private_dir, .{ .iterate = true }); + try private_handle.setPermissions(path_mod.currentIo(), .fromMode(0o750)); + private_handle.close(path_mod.currentIo()); + + const secret_path = try std.fs.path.join(test_env.ctx.allocator, &.{ private_dir, "secret" }); + defer test_env.ctx.allocator.free(secret_path); + var secret = try std.Io.Dir.createFileAbsolute(path_mod.currentIo(), secret_path, .{}); + try secret.writeStreamingAll(path_mod.currentIo(), "secret\n"); + try secret.setPermissions(path_mod.currentIo(), .fromMode(0o640)); + secret.close(path_mod.currentIo()); + + var source_root = try std.Io.Dir.openDirAbsolute(path_mod.currentIo(), staging, .{ .iterate = true }); + defer source_root.close(path_mod.currentIo()); + const source_dir_stat = try source_root.statFile(path_mod.currentIo(), "private", .{ .follow_symlinks = false }); + const source_file_stat = try source_root.statFile(path_mod.currentIo(), "private/secret", .{ .follow_symlinks = false }); + try testing.expectEqual(@as(u32, 0o750), source_dir_stat.permissions.toMode() & 0o777); + try testing.expectEqual(@as(u32, 0o640), source_file_stat.permissions.toMode() & 0o777); + + const archive_path = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "modes.pkg.tar.zst" }); + defer test_env.ctx.allocator.free(archive_path); + try archive.createPackageArchive(&test_env.ctx, staging, archive_path); + + const extracted = try std.fs.path.join(test_env.ctx.allocator, &.{ test_env.path, "mode-extracted" }); + defer test_env.ctx.allocator.free(extracted); + try extract.intoPreservingSpecialBits(&test_env.ctx, archive_path, extracted); + + var extracted_dir = try std.Io.Dir.openDirAbsolute(path_mod.currentIo(), extracted, .{ .iterate = true }); + defer extracted_dir.close(path_mod.currentIo()); + const dir_stat = try extracted_dir.statFile(path_mod.currentIo(), "private", .{ .follow_symlinks = false }); + const file_stat = try extracted_dir.statFile(path_mod.currentIo(), "private/secret", .{ .follow_symlinks = false }); + try testing.expectEqual(@as(u32, 0o750), dir_stat.permissions.toMode() & 0o777); + try testing.expectEqual(@as(u32, 0o640), file_stat.permissions.toMode() & 0o777); +} + test "Packager creates package artifacts with metadata independently" { var test_env = try test_helpers.createTestEnv(); defer { @@ -534,6 +580,20 @@ test "Packager creates package artifacts with metadata independently" { var _sigf = try std.Io.Dir.openFileAbsolute(path_mod.currentIo(), manifest_sig_path, .{}); defer _sigf.close(path_mod.currentIo()); + // The final v5 manifest selects store identity v4. + const manifest_v5_path = try std.fs.path.join(test_env.ctx.allocator, &.{ staging_dir, manifest.MANIFEST_V5_FILENAME }); + defer test_env.ctx.allocator.free(manifest_v5_path); + var manifest_v5_file = try std.Io.Dir.openFileAbsolute(path_mod.currentIo(), manifest_v5_path, .{}); + defer manifest_v5_file.close(path_mod.currentIo()); + const manifest_v5_stat = try manifest_v5_file.stat(path_mod.currentIo()); + const manifest_v5_bytes = try test_env.ctx.allocator.alloc(u8, @intCast(manifest_v5_stat.size)); + defer test_env.ctx.allocator.free(manifest_v5_bytes); + _ = try manifest_v5_file.readPositionalAll(path_mod.currentIo(), manifest_v5_bytes, 0); + const decoded_v5 = try manifest.PackageManifestV1.decodeForSchema(manifest_v5_bytes, manifest.SCHEMA_VERSION_V5); + const decoded_hash = try decoded_v5.contentHashHex(test_env.ctx.allocator); + defer test_env.ctx.allocator.free(decoded_hash); + try std.testing.expectEqualStrings(result.content_hash, decoded_hash); + const projection_path = try std.fs.path.join(test_env.ctx.allocator, &.{ staging_dir, manifest.PROJECTION_FILENAME }); defer test_env.ctx.allocator.free(projection_path); var _projf = try std.Io.Dir.openFileAbsolute(path_mod.currentIo(), projection_path, .{}); diff --git a/src/verify.zig b/src/verify.zig index c34362e..c28d400 100644 --- a/src/verify.zig +++ b/src/verify.zig @@ -228,25 +228,13 @@ fn verifyStore( continue; }; - const format: manifest.Format = blk: { - const v4_path = std.fs.path.join(ctx.allocator, &.{ entry_path, manifest.MANIFEST_V4_FILENAME }) catch { - return ctx.fail(VerifyError.OutOfMemory, entry_path, "failed to construct v4 manifest path"); - }; - defer ctx.allocator.free(v4_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v4_path, .{})) |_| break :blk .v4 else |_| {} - - const v3_path = std.fs.path.join(ctx.allocator, &.{ entry_path, manifest.MANIFEST_V3_FILENAME }) catch { - return ctx.fail(VerifyError.OutOfMemory, entry_path, "failed to construct v3 manifest path"); - }; - defer ctx.allocator.free(v3_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v3_path, .{})) |_| break :blk .v3 else |_| {} - - const v2_path = std.fs.path.join(ctx.allocator, &.{ entry_path, manifest.MANIFEST_V2_FILENAME }) catch { - return ctx.fail(VerifyError.OutOfMemory, entry_path, "failed to construct v2 manifest path"); - }; - defer ctx.allocator.free(v2_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v2_path, .{})) |_| break :blk .v2 else |_| {} - break :blk .v1; + const format = manifest.detectFormat(ctx.allocator, entry_path) catch |err| { + return ctx.fail(switch (err) { + manifest.ManifestError.OutOfMemory => VerifyError.OutOfMemory, + manifest.ManifestError.PermissionDenied => VerifyError.PermissionDenied, + manifest.ManifestError.InvalidInput => VerifyError.InvalidInput, + else => VerifyError.FileSystem, + }, entry_path, "failed to detect package manifest format"); }; const manifest_path = std.fs.path.join(ctx.allocator, &.{ entry_path, format.manifestFilename() }) catch { return ctx.fail(VerifyError.OutOfMemory, entry_path, "failed to construct manifest path"); @@ -329,10 +317,11 @@ fn verifyStore( }; if (full_hash) { - const computed = switch (format) { + const computed = switch (format.storeHashFormat()) { .v1 => hash.calculateStoreContentHash(ctx.allocator, entry_path, null), .v2 => hash.calculateStoreContentHashV2(ctx.allocator, entry_path, null), - .v3, .v4 => hash.calculateStoreContentHashV3(ctx.allocator, entry_path, null), + .v3 => hash.calculateStoreContentHashV3(ctx.allocator, entry_path, null), + .v4 => hash.calculateStoreContentHashV4(ctx.allocator, entry_path, null), }; const computed_hash = computed catch { result.store_issues += 1; @@ -523,36 +512,16 @@ fn verifyProfileManifestPackages( } if (full_hash) { - const format: manifest.Format = blk: { - const v4_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, manifest.MANIFEST_V4_FILENAME }) catch { - result.profile_issues += 1; - try addProfileIssue(ctx, result, pkg.store_path, profile_name, realization_name, "failed to construct manifest path"); - continue; - }; - defer ctx.allocator.free(v4_manifest_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v4_manifest_path, .{})) |_| break :blk .v4 else |_| {} - - const v3_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, manifest.MANIFEST_V3_FILENAME }) catch { - result.profile_issues += 1; - try addProfileIssue(ctx, result, pkg.store_path, profile_name, realization_name, "failed to construct manifest path"); - continue; - }; - defer ctx.allocator.free(v3_manifest_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v3_manifest_path, .{})) |_| break :blk .v3 else |_| {} - - const v2_manifest_path = std.fs.path.join(ctx.allocator, &.{ pkg.store_path, manifest.MANIFEST_V2_FILENAME }) catch { - result.profile_issues += 1; - try addProfileIssue(ctx, result, pkg.store_path, profile_name, realization_name, "failed to construct manifest path"); - continue; - }; - defer ctx.allocator.free(v2_manifest_path); - if (std.Io.Dir.accessAbsolute(path_mod.currentIo(), v2_manifest_path, .{})) |_| break :blk .v2 else |_| {} - break :blk .v1; + const format = manifest.detectFormat(ctx.allocator, pkg.store_path) catch { + result.profile_issues += 1; + try addProfileIssue(ctx, result, pkg.store_path, profile_name, realization_name, "failed to detect manifest format"); + continue; }; - const computed = switch (format) { + const computed = switch (format.storeHashFormat()) { .v1 => hash.calculateStoreContentHash(ctx.allocator, pkg.store_path, null), .v2 => hash.calculateStoreContentHashV2(ctx.allocator, pkg.store_path, null), - .v3, .v4 => hash.calculateStoreContentHashV3(ctx.allocator, pkg.store_path, null), + .v3 => hash.calculateStoreContentHashV3(ctx.allocator, pkg.store_path, null), + .v4 => hash.calculateStoreContentHashV4(ctx.allocator, pkg.store_path, null), }; const computed_hash = computed catch { result.profile_issues += 1; From b9e1efa3e51a265661f56d04c2037bb6874b348a Mon Sep 17 00:00:00 2001 From: Jeremy Huntwork Date: Sun, 23 Aug 2026 11:36:56 -0400 Subject: [PATCH 2/2] Make the new identity byte contract explicit The specification now records every store-hash domain prefix and clarifies the directory record shape, so independent implementations can reproduce v4 without inferring protocol bytes from source. It also identifies both manifest formats that use the domain-separated signature envelope. --- docs/design/specification-details.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/design/specification-details.md b/docs/design/specification-details.md index bf29843..5bbb416 100644 --- a/docs/design/specification-details.md +++ b/docs/design/specification-details.md @@ -103,7 +103,7 @@ Mere persists a number of formats, several of which are signed or content-addres | --- | --- | --- | --- | --- | | Store content hash (§1) | store path name | manifest format and `schema_version` | v4 | v1, transitional, v2, v3, v4 | | Package manifest (§17) | `.mere/manifest.v1` through `.mere/manifest.v5` | `schema_version` field and filename | v5 | v1, v2, v3, v4, v5 | -| Manifest signature (§5) | `.mere/manifest.vN.sig` | manifest format; v4 envelope magic/version/algorithm | domain-separated v2 envelope | legacy raw Ed25519, domain-separated v2 | +| Manifest signature (§5) | `.mere/manifest.vN.sig` | manifest format; v4/v5 envelope magic/version/algorithm | domain-separated v2 envelope | legacy raw Ed25519, domain-separated v2 | | Key file | `*.pub`, `*.key` | `MEREKEY` magic, version and algorithm bytes | v1 / Ed25519 | v1 / Ed25519 | | Generation manifest (§6) | `/` | `schema_version` field | 2 | 2 only | | Realization manifest | named profile `root/` | `schema_version` field | 1 | 1 only | @@ -129,7 +129,7 @@ Requirements: The store path is `--` where hash is BLAKE3 of the realized payload. -**Algorithm**: Use a single incremental BLAKE3 hasher. Walk entries in deterministic (lexicographic) order. For each entry, feed a canonical record with explicit length-prefixed boundaries. +**Algorithm**: Use a single incremental BLAKE3 hasher. V1 and the transitional identity have no domain prefix; v2 begins with `mere-store-content-v2\0`, v3 with `mere-store-content-v3\0`, and v4 with `mere-store-content-v4\0`. Walk entries in deterministic (lexicographic) order. For each entry, feed a canonical record with explicit length-prefixed boundaries. **Record format per entry**: ``` @@ -154,7 +154,7 @@ If symlink (0x12): target = symlink target bytes If dir (0x11): - (nothing else - just the entry_tag + path + type_tag) + (no content fields; v3/v4 retain the mode field described above) ``` **Rules**: