Skip to content

meta: add Af2Desc metadata storer for CDM SDFS AF2 DESC attribute#2

Open
riaarora-boop wants to merge 1 commit into
feat/posix-same-dir-tmpfrom
feat/meta-af2desc
Open

meta: add Af2Desc metadata storer for CDM SDFS AF2 DESC attribute#2
riaarora-boop wants to merge 1 commit into
feat/posix-same-dir-tmpfrom
feat/meta-af2desc

Conversation

@riaarora-boop

Copy link
Copy Markdown

Summary:

Adds Af2Desc, a new MetadataStorer implementation for CDM SDFS. It packs must-store S3 object metadata (etag, content-type, content-encoding, user metadata, etc.) into the single sdfs_af2_file_desc xattr so it rides the AF2 partition snapshot alongside the object data.

Key design points:

  • Only the sdfs_af2_file_desc key works on SDFS FUSE — generic xattr keys are rejected and getxattr is unimplemented. Af2Desc packs all metadata into one JSON blob under this key.
  • Serves reads from a write-through in-process cache since SDFS cannot read xattrs back after write. Cold-cache misses return ErrNoSuchKey, which callers already handle as "empty."
  • Non-allowlisted keys are accepted-and-dropped so the blob stays within the MJF value-size cap.

Exposes --af2-desc and --af2-desc-max-bytes flags on the posix subcommand. Mutually exclusive with --sidecar and --nometa.

Test Plan:

  • go test ./backend/meta/... passes (unit tests cover store/retrieve/delete/rename/cache invalidation on a normal xattr-capable filesystem)
  • Verified on SDFS on-cluster: setxattr with raw sdfs_af2_file_desc key succeeds; with user. prefix returns ENOTSUP

JIRA Issues:

CDM-557243 (subtask of CDM-557238)

Revert Plan:

Revert this PR. --af2-desc defaults to false so existing deployments using xattr or sidecar are unaffected.

Implements Af2Desc, a MetadataStorer that packs must-store S3 object
metadata (etag, content-type, user metadata) into the single AF2 DESC
file attribute (sdfs_af2_file_desc xattr) so it rides the AF2 partition
snapshot. Serves reads from a write-through in-process cache because SDFS
cannot read xattrs back after write.

Exposes --af2-desc and --af2-desc-max-bytes flags on the posix subcommand.
Mutually exclusive with --sidecar and --nometa.

Jira: CDM-557243
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant