diff --git a/website/docs/components/data-accelerators/cayenne/index.md b/website/docs/components/data-accelerators/cayenne/index.md index b96af8f07..eb33fda2b 100644 --- a/website/docs/components/data-accelerators/cayenne/index.md +++ b/website/docs/components/data-accelerators/cayenne/index.md @@ -133,7 +133,7 @@ These acceleration parameters (set under `acceleration.params`) configure the op | Parameter | Description | | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `cayenne_datalake_location` | Object-store URL prefix for the cold tier (the bottom tier of the storage cascade). Must be an `s3://` URL (e.g. `s3://bucket/prefix`) — a general-purpose S3 or S3-compatible bucket, distinct from the warm S3 Express One Zone store. When set, a background promotion stage graduates the warm local-disk tier to read-optimized, Z-order-clustered Vortex files on this store, and queries span the warm and cold tiers with per-tier pushdown. Unset (default) disables the cold tier. Requires key-based deletes (`cayenne_deletion_mode` auto-resolves to `key`; an explicit `position` is rejected at registration) and `refresh_mode: changes` or `append`. A `primary_key` is required to activate the tier — without one the tier registers but stays inactive. **v1 constraints:** local `file://` locations are not supported; partitioned and position-delete tables are not supported. | -| `cayenne_datalake_clustering_columns` | Comma-separated liquid-clustering key columns for cold files (multi-column Z-order), e.g. `tenant_id,ts`. Clustering tightens each cold file's per-column zone maps so selective queries on any clustering dimension prune at the storage layer. When unset, falls back to `cayenne_sort_columns`, then the primary key. An entry that does not exist in the schema is ignored with a warning (clustering falls back to `cayenne_sort_columns`, then the primary key). | +| `cayenne_datalake_clustering_columns` | Comma-separated liquid-clustering key columns for cold files (multi-column Z-order), e.g. `tenant_id,ts`. Clustering tightens each cold file's per-column zone maps so selective queries on any clustering dimension prune at the storage layer. When unset, falls back to `cayenne_sort_columns`; when that is also unset, cold-tier promotion clusters by the hottest columns observed in query pushdown filters (default-on adaptive layout), falling back to the primary key when no usable observed column exists. An entry that does not exist in the schema is ignored with a warning. | | `cayenne_datalake_target_file_size_mb` | Target size for cold-tier Vortex files in MB. Larger than the warm `cayenne_target_file_size_mb` because object stores favor fewer, larger objects and cold scans are range reads. Accepts `auto` or an explicit MB value. Defaults to `512`. | | `cayenne_datalake_warm_max_bytes` | The warm tier graduates to cold once its total Vortex bytes reach this threshold. `0` (default) disables the byte trigger; set alongside `cayenne_datalake_warm_max_files` to bound warm-tier size. When the tier is enabled and neither trigger is set, Cayenne applies a default byte trigger (16× `cayenne_datalake_target_file_size_mb`) so promotion is not silently disabled. | | `cayenne_datalake_warm_max_files` | The warm tier graduates to cold once its Vortex file count reaches this threshold. `0` (default) disables the file-count trigger. | @@ -597,7 +597,7 @@ The cold tier lets a table grow beyond local NVMe capacity while keeping recent, - **Promotion (write path).** A dedicated background worker graduates the warm tier once the size or file-count threshold (`cayenne_datalake_warm_max_bytes` / `cayenne_datalake_warm_max_files`) is crossed, evaluated every `cayenne_datalake_tiering_check_interval_ms`. Promotion is **incremental carry-forward**: rather than re-materializing the whole table each cycle, it classifies the existing cold manifest into *dirty* files that may host a tombstoned key — using per-PK-column min/max rectangles from each file's persisted statistics, refined by a per-file PK bloom filter, and conservative in the safe direction (a false positive costs an extra rewrite; a missed tombstone is impossible) — and *clean* files that are provably untouched. Only the warm delta plus the dirty files are re-read (all deletes applied, one version per key), **Z-order clustered** for tight multi-column zone maps, and written as read-optimized Vortex at the larger `cayenne_datalake_target_file_size_mb` under a per-promotion prefix; clean files are carried forward by manifest reference and never re-read. The new files are atomically registered while the promoted warm files are cleared in a single transaction, so promotion cost tracks the *changed* data, not total table size. - **Cross-tier scan (read path).** Queries span all tiers and push filters, projection, and limits down to each. Cold files are pruned from statistics held in the metastore, so pruning requires **no object-store round-trip on the query path**. Each cold file's PK bloom lets an upsert's keyset rebuild answer cold-tier key existence without scanning the object store. A `DELETE` after promotion correctly hides a cold-resident row. - **Physical GC.** A periodic mark-and-sweep, rooted at the manifest, reclaims cold objects orphaned by carry-forward rewrites. It runs every `cayenne_datalake_gc_interval_ms` (default 5m), which doubles as the orphan grace period: an object no longer referenced by the manifest is deleted only after it has been observed orphaned for at least one interval. -- **Clustering.** Cold files are clustered by `cayenne_datalake_clustering_columns` (multi-column Z-order / Morton order), falling back to `cayenne_sort_columns` and then the primary key. Clustering on more than one dimension prunes far better than a single-column sort for selective queries on any clustering column. +- **Clustering.** Cold files are clustered by `cayenne_datalake_clustering_columns` (multi-column Z-order / Morton order). When it is unset, clustering falls back to `cayenne_sort_columns`; when that is also unset, cold-tier promotion clusters by the hottest columns observed in query pushdown filters (default-on adaptive layout), and finally the primary key. Clustering on more than one dimension prunes far better than a single-column sort for selective queries on any clustering column. ### Requirements and v1 constraints