posix: implement Af2MPUHandler (write-at-offset, stash-the-tail) for AF2/SDFS#4
Open
riaarora-boop wants to merge 2 commits into
Open
posix: implement Af2MPUHandler (write-at-offset, stash-the-tail) for AF2/SDFS#4riaarora-boop wants to merge 2 commits into
riaarora-boop wants to merge 2 commits into
Conversation
…AF2/SDFS Adds Af2MPUHandler, a concrete MPUHandler for Rubrik CDM SDFS AF2 channels. Instead of staging one file per part and concatenating at Complete, every part is written directly at its final byte offset in a single data file, revealed with a same-directory rename — one write per byte, no staging copy. Stride inference (from part #1 or from the max of any two distinct parts) places full-size parts at their final offset immediately. Parts that arrive before the stride is resolvable, or that are shorter than the confirmed stride (the tail), are appended to a per-upload stash file and folded at Complete. No fsync: durability is deferred to the AF2 checkpoint (ack-on-write). Also extends MPUHandler with CreateMultipartUpload so the Af2 path can capture upload metadata as plain files (AF2 DESC xattr is not writable on directories). ListMultipartUploads updated to use readMpObjName which tries the plain-file path first, falling back to the meta storer. Jira: CDM-557245 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
CompleteMultipartUpload no longer re-reads the whole object. validateRevealMpu verifies each part against its stored ETag and accumulates the full-object CRC64NVME from per-part CRCs recorded during UploadPart (new otterMpuPart.CRC), so Complete is O(parts) not O(bytes) — fixes the ~16 min / 100 GiB stall that exceeded the S3 client timeout. Restore fsync durability, now that the SDFS RS-sync regression (CDM-550864, reverted by scaledata/sdmain#226685) is fixed: - writeAtomic → writeDurableAtomic: fsync temp fd + parent dir - uploadPartAtOffset: tf.Sync() before index commit - completeMultipartAtOffset: df.Sync() after each stash fold - PutObject: f.Sync() postprocess hook before commit link Jira: CDM-557245 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Adds
Af2MPUHandler, a concreteMPUHandlerfor Rubrik CDM SDFS AF2 channels. Activated automatically whenSameDirTmp=true.Write-at-offset: Instead of staging one file per part and concatenating at Complete (two full-size writes), every part is written directly at its computed final byte offset in a single data file, revealed with a same-directory rename at Complete — one write per byte, no staging copy.
Stash-the-tail: Parts that arrive before the stride is resolvable (or that are shorter than the confirmed stride) are appended to a per-upload stash file and folded into the data file at Complete with an fsync before the index flip.
O(1) CompleteMultipartUpload: Each part's CRC64NVME is recorded in the per-upload index at
UploadParttime (otterMpuPart.CRC).validateRevealMpuaccumulates the full-object CRC from stored per-part values with no data re-read — O(parts), not O(bytes). Fixes a ~16 min stall at 100 GiB that exceeded S3 client timeouts.Fsync durability:
writeDurableAtomicfsyncs the temp fd and parent dir before returning.uploadPartAtOffsetfsyncs the part data before committing its index record. Fold steps fsync the data file before flipping the stash flag.PutObjectfsyncs via the postprocess hook. The SDFS RS-sync regression that originally required removing fsync (CDM-550864) has been reverted (scaledata/sdmain#226685).Test Plan:
go test ./backend/posix/...passes (MPU write-at-offset, stash-the-tail, stride inference, O(1) Complete, abort/list all covered)JIRA Issues:
CDM-557245 (subtask of CDM-557238)
Revert Plan:
Revert this PR.
Af2MPUHandleris only activated whenSameDirTmp=true; standard posix deployments are unaffected.