Antalya 26.8: Bump to 26.8.27

[zvonand]

Changelog category (leave one):

• Not for changelog (changelog entry is not required)

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Unit tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [a12c7e0]

[zvonand]

@blau-ai analyze CI fails, fix related fails directly in this branch

[blau-ai]

CI triage for bump/antalya-26.8/26.8.27 @ 7d824c6

I analyzed the failing checks and pushed fixes for the two that are clearly PR‑caused and fixable without a build. Commit: 56241dce698 (pushed directly to this branch, as requested; Signed-off-by added so it does not add to the DCO failure).

Verdict

Failing checks split into: 2 PR‑caused code regressions (fixed here), 1 PR‑caused Iceberg read regression (flagged, needs owner), and the rest infra / scanner / pre‑existing (not fixable in this PR).

---

✅ Fixed in 56241dce698

1. std::bad_function_call crash masking ENGINE=Iceberg(...) — src/Parsers/FunctionSecretArgumentsFinder.cpp
The backport of ClickHouse#122770 (part of this bump) refactored findTableEngineSecretArguments() to dispatch through a std::function find_secrets invoked once at the end (classifyPositionalArguments(); find_secrets();). Every engine branch assigns the lambda (or returns), except the Iceberg branch, which called findIcebergTableEngineSecretArguments() directly without assigning find_secrets and without returning — so it fell through to find_secrets() on an empty std::function:

3. DB::FunctionSecretArgumentsFinder::findTableEngineSecretArguments()
4. DB::FunctionSecretArgumentsFinderAST::FunctionSecretArgumentsFinderAST(...)
5. DB::ASTFunction::hasSecretParts() const

Any CREATE TABLE ... ENGINE=Iceberg(...) crashes when the query is masked for logging. Fix: assign the lambda like every sibling branch (keeps the classifyPositionalArguments() pre‑pass).

• Caused: Integration tests test_storage_iceberg_disks/test.py::test_single_iceberg_file[*] (Code: 1001 std::bad_function_call), shard 2/8 and related.

2. Stale IcebergStripNothing gtests — .../Iceberg/tests/gtest_iceberg_strip_nothing.cpp

• IcebergStripNothing.ExcludedColumnsHaveNoStatistics
• IcebergStripNothing.StatisticsWithoutExclusionCoverEveryColumn

The 26.8.27 merge moved Iceberg data‑file column sizes from an in‑memory col->byteSize() estimate in DataFileStatistics::update() to the real on‑disk sizes reported by the output format at finalize() via addColumnSizesOnDisk() (see MultipleFileWriter::finalize()). The test still expected update() alone to populate column sizes, so getColumnSizes() was empty. Fix: populate sizes via addColumnSizesOnDisk() in the test to match the new design. (Restoring byteSize() in update() would be wrong — addColumnSizesOnDisk uses +=, so it would double‑count.)

• Caused: Unit tests (asan_ubsan) — 2/20824 failed.

I can't build/run ClickHouse in this environment, so correctness of both fixes will be confirmed by CI on this push.

---

⚠️ PR‑caused, NOT fixed (needs the Iceberg feature owner)

Stateless tests — Iceberg Parquet geo‑pruning family 04510–04514 (suite "distributed plan, selected tests" reports 4 failures; I confirmed 04513_parquet_geo_pruning_iceberg_renamed_bbox_column and 04514_parquet_geo_pruning_geostats_fallback). Both are 100% reproducible, with and without randomized settings. Symptom — a SELECT after INSERT INTO ... ENGINE=IcebergLocal(...) returns the wrong values:

=== all rows ===        === south texas ===
-1  +0                  -1  +0
-2  +0                  -2  +0
-3  +0
-4  +0

i.e. id/geometry columns read back as 0 (row‑group pruning count is still correct). These Altinity tests have been in the branch since Aug 31 and presumably passed before the bump, so this is a real read‑after‑insert regression in the Iceberg data path from the merge (the #2363 cherry‑pick notes "unresolved conflict markers" around this area). This needs someone who can build/run the Iceberg insert path — I didn't want to guess at a data‑correctness fix blind.

---

⏭️ Infra / scanner / not PR‑caused (no code fix here)

Check	Why	Action
Integration — test_s3_cluster/* (shards 1,3,4,5)	Timed out while waiting for instance 'c2.s0_0_1' to start — the whole test cluster failed to come up (environment), not a code failure	Re‑run
Keeper Stress Tests (PR)	ClickHouse binary not found at .../ci/tmp/clickhouse — binary not provided to the job	Re‑run / pipeline wiring
Parser memory check	Resolve master binary ERROR: No master clickhouse-examples artifact was found — missing master baseline	Re‑run after master artifact exists
Grype Scan (keeper / server‑alpine)	Base‑image CVEs (4 high/critical keeper, 1 server‑alpine) — inherent to deps, not changed by this PR	Triage separately; typically non‑blocking for a bump
DCO	one or more commits lack Signed-off-by (common on backport/merge chains)	Needs sign‑off / DCO override by a maintainer; my commit is signed

Note: I couldn't enumerate the 4th failure of the amd_debug, parallel stateless suite individually; it almost certainly overlaps the same geo‑pruning family.

---

Next steps: re‑run after the push to clear the infra flakes and confirm fixes #1/#2; the geo‑pruning 04510–04514 regression still needs an Iceberg owner.

[zvonand]

@blau-ai fix the failing related tests directly in this branch

[blau-ai]

Looked at the failing checks on this bump and pushed fixes for two of the three test clusters directly to bump/antalya-26.8/26.8.27 (as requested). CI on the PR validates them — I can't build/run here.

✅ Fixed (2 commits pushed)

1. test_storage_iceberg_no_spark/test_unknown_type.py (6 failures) — server crash on Iceberg write
fed75c6 (Fix Iceberg write crash on excluded columns in addColumnSizesOnDisk)
Only test_unknown_type_write is the real failure; the other 5 are the Connection refused cascade after the server aborts. Root cause: the bump combined upstream ClickHouse#121392 (addColumnSizesOnDisk(), iterates every schema field and throws if one has no on-disk size) with Altinity #2363 (the unknown-type write path, which strips not-physically-written columns and marks them via excludeColumns()). The getters all skip excluded positions; addColumnSizesOnDisk() didn't, so finalize() hit LOGICAL_ERROR: Written data file does not report the on-disk size of column placeholder and took node1 down. Fix skips excluded positions there too (no-op for normal tables).

2. test_mask_sensitive_info::test_create_table / ::test_table_functions — Iceberg secret over-masking
55745c4 (Mask Iceberg storage_type engine/function args without over-hiding)
Backport ClickHouse#122770 added a generic fail-closed pass (classifyPositionalArguments()) that hides every positional arg after the first key = value. The Altinity-only storage_type = 's3' leading named arg tripped it, so Iceberg(storage_type='s3', <url>, <key>, <secret>) was masked as ('[HIDDEN]', '[HIDDEN]', '[HIDDEN]') instead of hiding only the secret. Fix: Iceberg returns right after its own finder (same pattern BigQuery already uses); that finder skips storage_type and delegates to findS3*/findAzure*, which run their own positional-after-named fail-close, so nothing is left unmasked. (Follows up the 56241dce698 crash-fix in the same file.)

⚠️ Not fixed — needs a build (code regression, not test-side)

04512/04513/04514_parquet_geo_pruning_* (stateless, 100% deterministic)
SELECT id on the renamed-column IcebergLocal table returns 0,0,0,0 instead of 1,2,3,4, while ParquetPrunedRowGroups: 1 still matches — so the GeoParquet pruning fix is intact and the file is read (4 rows), but the id column is default-filled. That only happens when field_id→column-name resolution fails for that column, which points at the Iceberg schema-evolution / manifest schema-id read path reworked in this bump — not the geo code and not a test-fixture issue. I couldn't pin the exact line statically (couldn't pull the full clickhouse-server.log), and it's a C++ change I can't verify without a build, so I'm not guessing at it.
Suggested next step (needs local repro): git bisect the Iceberg commits in d75a682..HEAD under src/Storages/ObjectStorage/DataLakes/Iceberg/, starting with 0be270bbf68 (ClickHouse#122574 "Use less schema comparison", changes data-file schema-id resolution), 801e55472b9 (ClickHouse#114521), and the ManifestFileIterator schema-id-header change; check what underlying_format_read_schema_id the fixture file resolves to and whether getColumnMapperById returns {1:"id"}.

ℹ️ Infra / not test fixes

• Parser memory check — "No master clickhouse-examples artifact was found" (missing baseline).
• Keeper Stress Tests — "ClickHouse binary not found" (pipeline didn't provide the binary).
• Grype scans (keeper/server) — CVE scan, not a test.
• DCO — backport/merge commits lack Signed-off-by; fixing needs a history rewrite/force-push, which I won't do.

[zvonand]

@blau-ai

[blau-ai]

CI triage for a12c7e0 (bump to 26.8.27)

Verdict: no clearly PR-caused code failures. The core ClickHouse CI (praktika) is green — builds, stateless, integration, unit, stress, AST fuzzer and BuzzHouse all pass; a few stateless/stress jobs were still running at the time of writing. All the red checks fall into: a process gate (DCO), image CVE scans (Grype), two CI-infra artifact errors, and four flaky/environment-sensitive regression suites.

Breakdown: 0 PR-caused · 2 process/scan gates · 2 infra · 4 flaky/environment (+ Iceberg (2) still running).

---

Process / scan gates (not code)

DCO — fail. The branch contains merge commits and upstream robot/backport commits with no Signed-off-by trailer, which DCO rejects. Expected for a bump PR. Note your own non-merge commits (e.g. a12c7e0, 4c289ce, 6ebd500) also lack a sign-off. To go green you can either add sign-offs to your own commits (git commit --amend -s / git rebase --signoff) or have a maintainer override DCO for the merge/robot commits — the merge/backport commits can only be cleared by override.

Grype Scan (keeper 4 high/critical, server-alpine 1) — fail. These scan the built container images for CVEs in OS/base-image packages, not PR logic. A version bump pulls newer upstream images, so newly-disclosed CVEs surface here. Review the per-image Grype HTML reports linked on the checks; these are normally triaged/accepted or fixed by a base-image bump in the release process, and do not indicate a correctness problem in this diff.

CI infrastructure (not code) — safe to re-run

Parser memory check — ERROR. Failing sub-step is Resolve master binary: No master clickhouse-examples artifact was found. The baseline (master) binary it diffs memory against was not available. Re-run once the master artifact exists.

Keeper Stress Tests (PR) — ERROR. ClickHouse binary not found at .../ci/tmp/clickhouse. The pipeline must provide the binary (e.g. build or artifact step). The build artifact was not delivered to the job — a pipeline plumbing issue, not a test failure. Re-run.

Regression (testflows) — flaky / environment

oauth — fail. Failures concentrated in the access control and groups sub-features, with generic AssertionError: Oops! Assertion failed and heavy retry churn (298/1814 retries failed). That pattern is characteristic of external IdP / token-caching timing flakiness rather than a logic regression. The bump does touch Access/AccessControl.cpp and Access/Authentication.cpp, so I cannot 100% exclude a regression — but the signature favors flakiness. Re-run; if access control/groups reproduce identically, worth a closer look.

swarms — fail (4/1535). Failing scenarios: cluster discovery with wrong cluster name, node failure / check restart swarm node, and cpu overload / cpu overload all swarm nodes. These are chaos/resilience and CPU-stress tests — timing-sensitive and classically flaky. Re-run.

s3_azure_1 — fail (2 scenarios) and tiered_storage_cas — fail. Both fail with ClickHouse server is not healthy (s3_azure: disk/remote host filter, disk/syntax; tiered_storage_cas: the whole with cas feature). That is a server-health precondition failing mid-suite and cascading to the following tests — a runner/stability issue, not deterministic test logic. Given stress/fuzzer/BuzzHouse all passed in praktika, a code crash is unlikely, but this is the one bucket that could mask one. Recommended: re-run these two suites; if server is not healthy reproduces, pull clickhouse-server.log.zst from that run to rule out a crash before accepting them as infra.

---

Suggested next steps

1. DCO: add -s sign-offs to your own commits and/or request a maintainer DCO override for the merge/robot commits.
2. Parser memory check / Keeper Stress: re-run — both are missing-artifact/pipeline errors, nothing to fix in the diff.
3. Grype: review the two image reports; handle via base-image/release triage as usual.
4. Regression (oauth, swarms, s3_azure_1, tiered_storage_cas): re-run. For s3_azure_1 and tiered_storage_cas, check the server log on re-run to confirm server not healthy is environmental and not a crash.

No code change is required from the diff for any of the above. I have not pushed anything.

— @blau-ai (automated triage; evidence from the praktika result_pr.json and the testflows reports for a12c7e0)