Skip to content

Antalya 26.8: Implement TRUNCATE TABLE for Iceberg Engine (REST … - #2374

Merged
zvonand merged 5 commits into
antalya-26.8from
feature/antalya-26.8/pr-2125
Sep 28, 2026
Merged

zvonand merged 5 commits into
antalya-26.8from
feature/antalya-26.8/pr-2125

Conversation

@zvonand

@zvonand zvonand commented Sep 15, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Frontport for Antalya 26.3

  • 26.1 Antalya port - Implement TRUNCATE TABLE for Iceberg Engine (REST catalog support) Feature: Support TRUNCATE TABLE for Iceberg engine #1529, It's a frontport from 26.1, contains:
    • feat(iceberg): Implement TRUNCATE TABLE for Iceberg Engine (REST catalog support) — Core implementation: metadata-only truncation generating a new overwrite snapshot with empty manifest list, committed atomically via REST catalog
    • fix(iceberg): pass new_snapshot to updateMetadata in IcebergStorageSink — Fixed silent breakage of all INSERTs on REST catalog tables (wrong JSON object passed to catalog->updateMetadata)
    • fix(iceberg): restore return false in RestCatalog::updateMetadata — Preserve retry contract; add LOG_WARNING for diagnostics
    • fix(iceberg): revert Mutations.cpp updateMetadata to pass new_snapshot — Same fix as IcebergStorageSink, applied to ALTER TABLE DELETE/UPDATE path
    • refactor(iceberg): add comment explaining Avro zigzag encoding — Reviewer-requested documentation for manual Avro OCF serialization
    • refactor(iceberg): address code review feedback on TRUNCATE implementation — Named zero arguments, helper functions, restart integration test (feature (iceberg): Implement TRUNCATE TABLE for Iceberg Engine (REST … #1655 by @il9ue) (Antalya 26.6: Implement TRUNCATE TABLE for Iceberg Engine (REST … #2125 by @zvonand).

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

Cherry-picked from #2125.


Adaptation notes for antalya-26.3

The following API changes between antalya-26.1 and antalya-26.3 required adaptation:

  • RestCatalog constructor signature updated to match antalya-26.3 calling convention in DataLakeConfiguration.h
  • DatabaseDataLakeSetting member names updated
  • Cargo.lock restored to antalya-26.3 version (our changes do not touch any Rust code)

All feature logic is identical to the merged antalya-26.1 implementation.

…next commit)

---
Original cherry-pick message follows:

Merge pull request #2125 from Altinity/feature/antalya-26.6/pr-1655

Antalya 26.6: Implement TRUNCATE TABLE for Iceberg Engine (REST …
# Conflicts:
#	src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp
#	src/Storages/ObjectStorage/DataLakes/Iceberg/MetadataGenerator.cpp
#	src/Storages/ObjectStorage/DataLakes/Iceberg/MetadataGenerator.h
Kept antalya-26.8's `SnapshotOperation` parameter, `setSnapshotTotals`
helper and `generateManifestOnlySnapshot`, and layered the PR's
`is_truncate` handling on top of them.

Adapted: `generateNextMetadata` gained `bool is_truncate` after antalya-26.8's `SnapshotOperation operation` parameter (existing callers pass `operation` positionally); the truncate call site passes `SnapshotOperation::Append` explicitly
Adapted: the PR's truncate `overwrite` operation and zeroed `total-*` counters expressed in antalya-26.8's `operation_name` / `setSnapshotTotals` shape
@zvonand zvonand added releasy Created/managed by RelEasy antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions ai-resolved Port conflict auto-resolved by Claude labels Sep 15, 2026
@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Workflow [PR], commit [50d5a07]

@zvonand zvonand mentioned this pull request Sep 15, 2026
32 of 33 tasks
@zvonand

zvonand commented Sep 28, 2026

Copy link
Copy Markdown
Member Author

@blau-ai

@blau-ai

blau-ai commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

CI triage for 50d5a07

Verdict: 3 failing checks — 0 caused by this PR. All are pre-existing/infra/flaky.

Check Classification PR-caused?
Stateless tests (amd_debug, distributed plan, s3 storage, parallel) Flaky under randomized MergeTree settings ❌ No
GrypeScanServer (-alpine) / Grype Scan Base-image CVE (infra) ❌ No
GrypeScanKeeper / Grype Scan Base-image CVE (infra) ❌ No

Everything else (builds, integration, stress, AST fuzzer, BuzzHouse, all Regression suites, etc.) is green.


1. Stateless tests — 03572_export_merge_tree_part_limits_and_table_functions — flaky, not PR-caused

The only failing test in the suite. It expects an exact file count from a byte-limited export:

---- Count files in big_destination_max_bytes, should be 5 (4 parquet, 1 commit)
-5
+4

The row-limited export in the same test (export_merge_tree_part_max_rows_per_file) stayed at 5 and passed — that split is deterministic (4194304 / 1048576 = 4 files). Only the byte-split count moved.

CI's own randomized-settings diagnosis is conclusive:

Step 1: Re-running with the same randomized settings ... Runs: 5, Failed: 5  -> reproducible
Step 2: Re-running WITHOUT randomized settings ...      Runs: 4, Failed: 0  -> passes
Step 3: Minimizing ... Culprit MergeTree settings: --index_granularity 41655

Mechanism: export_merge_tree_part_max_bytes_per_file=3500000 splits by the part's on-disk byte size, which depends on index_granularity. With the randomized large granularity (41655) the part packs into 3 parquet files (+1 commit = 4) instead of 4 (+1 commit = 5). The test comment itself notes "this should generate ~4 files" — the count was never meant to be exact under arbitrary MergeTree settings.

This test is not in this PR's diff (it came in via the earlier cherry-pick of #2146, commit 80ee409) and it carries only # Tags: no-fasttest — it does not opt out of randomized MergeTree settings, so it is exposed to this. Not a functional regression from this PR; a re-run will usually pass, but the real fix is to make the test robust.

Concrete fix (test-robustness, applies to the feature test, not this PR's code):

--- a/tests/queries/0_stateless/03572_export_merge_tree_part_limits_and_table_functions.sh
+++ b/tests/queries/0_stateless/03572_export_merge_tree_part_limits_and_table_functions.sh
 #!/usr/bin/env bash
-# Tags: no-fasttest
+# Tags: no-fasttest, no-random-merge-tree-settings
 # Tag no-fasttest: requires s3 storage
+# Tag no-random-merge-tree-settings: byte-based export file count depends on physical part layout (index_granularity)

no-random-merge-tree-settings is the established tag for exactly this (107 uses in 0_stateless). If you'd rather keep randomization on, the alternative is to assert a range instead of an exact 5 for the byte-limited case.

2 & 3. Grype scans (server -alpine and keeper) — infra, not PR-caused

Both fail on the same single High CVE in the base image:

✘ [ Fail ] '/docker vulnerabilities/CVE-2026-85091@nvd:cpe,High'
Grype Scan Completed with 1 high/critical vulnerabilities

CVE-2026-85091 is a zlib heap-buffer-overflow (zlib 1.3.1.2–1.3.2), i.e. an OS package in the Alpine/base Docker image (altinityinfra/clickhouse-server:26.8.6...altinitytest), not in anything this PR changes (the diff is Iceberg / StorageObjectStorage.cpp). Any PR building these images at this time trips it; it's a branch-wide base-image issue for the image maintainers (bump zlib / base image, or allowlist the CVE), not something fixable in this PR.


Bottom line: none of the three failures are caused by this PR. Safe to re-run the stateless suite (or apply the tag fix above to stop the export test flapping); the two Grype failures need a base-image zlib bump and are out of scope for this PR.

I did not push anything. If you'd like, I can open a small PR from a blau/* branch with the no-random-merge-tree-settings fix for 03572... — just say the word (or tell me to commit it directly to feature/antalya-26.8/pr-2125).

🤖 Automated CI triage. I can't build/run ClickHouse in this container; conclusions are from the praktika S3 report and CI logs for 50d5a07.

@zvonand
zvonand merged commit 6404943 into antalya-26.8 Sep 28, 2026
391 of 494 checks passed
@zvonand zvonand added verified Approved for release port-antalya PRs to be ported to all new Antalya releases labels Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-resolved Port conflict auto-resolved by Claude antalya antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions port-antalya PRs to be ported to all new Antalya releases releasy Created/managed by RelEasy verified Approved for release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants