BACKUP of CAS tables to S3 on the same endpoint

[k-morozov]

BACKUP of a table on a CAS to an S3 destination that shares the pool's endpoint failed outright, and the code path that failed would have silently corrupted the backup had it not failed.

BackupWriterS3::copyFileFromDisk asks S3 to copy the source object server-side. That assumes the file is that object, whole, from byte 0. On a CAS disk neither half holds:

• A small per-part file (checksums.txt, count.txt, columns.txt, ...) is an inline manifest entry with no object of its own. getStorageObjects returns a sized placeholder with an empty remote key, deliberately poisoned so a reader that bypasses the read path fails loudly rather than reading someone else's bytes. The copy passed that empty key to CopyObject, so every BACKUP died with Invalid argument - these files exist in every part.

• A blob object is [envelope][payload] and the payload is the file. BlobLocation carries the payload offset, but getStorageObjects returns only key and length and getBlobPath keeps only the key, so the copy started at byte 0 and pulled the envelope into the backup. RESTORE then failed.

  The first failure masked the second: checksums.txt aborted the backup before any blob corruption
  could surface. Fixing only the empty key would have turned a loud failure into backups that report
  success and cannot be restored.

The same bug was on the BACKUP ... TO Disk(...) path (an s3/s3_plain disk on the pool's endpoint): DiskObjectStorage::copyFile -> copyFileImpl -> S3ObjectStorage::copyObjectToAnotherObjectStorage.

• Inline entry: copyFileImpl now reads the bytes with readInlineDataToString (implemented for CAS) and writes them to the destination. S3ObjectStorage throws LOGICAL_ERROR on an empty key.
• Blob: copyFileImpl gets the offset from the new IMetadataStorage::getObjectPayloadOffset and passes it as object_from_offset. S3ObjectStorage passes it to copyS3File as src_offset, so both the ranged copy and the
  fallback read only the payload.

RESTORE onto CAS is not affected: it writes each part through one CAS transaction.

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fixed BACKUP of a table on a CAS to an S3 and Disk destination on the same endpoint.

Documentation entry for user-facing changes

Adds docs/en/antalya/cas/operations/backup.md describing how BACKUP/RESTORE behave on a content-addressed disk, and links it from the CAS index and roadmap.

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [9544ebf]

[filimonov]

Two things I would change before merge, one is shape and one is a bug.

Shape: the generic API should not learn about envelopes. The PR touches 23 files outside ContentAddressed/ (IDisk, IMetadataStorage, IObjectStorage, S3ObjectStorage, copyS3File, four disk wrappers, ObjectStorageQueue). That is a rebase cost on every release, and the new offset is a defaulted argument that future upstream callers of copyObjectToAnotherObjectStorage / getBlobPath will not know about, so blobs they copy will carry the envelope silently once inline files stop failing first.

The root cause is that DataSourceDescription::operator== and sameKind compare only type, object storage type and endpoint, so a CAS disk looks identical to a plain s3 disk on the same endpoint. Fixing it there keeps everything inside code we already patch:

• In the DiskObjectStorage constructor (DiskObjectStorage.cpp:124), mark the description for a content-addressed metadata storage (suffix on description, CAS-gated). Then DiskObjectStorage::copyFile falls into IDisk::copyThroughBuffers, and all four backup writers fail sameKind and copy through buffers. Correct bytes, no generic diff. This also covers MOVE out of CAS (CAS-254).
• copyS3File::performCopy has a real upstream bug: src_offset is ignored when the single-operation CopyObject is chosen. A 3-line fix (never single-op when offset != 0) is independent of CAS and can go upstream on its own.
• If we want server-side copy for S3(...) destinations, one CAS-gated branch in BackupWriterS3::copyFileFromDisk can call copyS3File with the payload offset as the existing src_offset and a raw readObject fallback reader. No src_object_offset, no getObjectPayloadOffset, no wrapper changes.

Bug: zero-byte blob files. A wide part with an all-empty Array or all-NULL Nullable column has 0-byte arr.bin / n.bin (reproduced with the PR binary). partFileMustStayBlob keeps them blobs, every blob copy is ranged, and calculatePartSize(0) throws LOGICAL_ERROR. BACKUP skips empty files, IDisk::copyFile does not. Needs a size == 0 branch and a test.

Smaller items:

• getObjectPayloadOffset is only called on the non-CAS branches and always returns 0 there; readInlineDataToString on CAS has no caller (the description says copyFileImpl uses it, it uses getContentAddressedFileCopySource).
• The blob source is validated in the producer and again in both consumers; src_blob != copy_source.object and payload_size != object.bytes_size cannot be true by construction.
• No gtest for the ranged copy, although S3ObjectStorageConditionalOpsTest in gtest_writebuffer_s3.cpp already has the mock and a LocalObjectStorage destination (the base IObjectStorage seek path is otherwise untested).
• GCS: supportsMultiPartCopy is false there, so every blob falls back to the buffered path. Correct, but logged only at TRACE and not tested; worth a line in backend.md.
• Cost: every blob is now Create + UploadPartCopy + Complete, even a 100-byte primary.idx.