Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions blog/2026-06-04-tiering-service-part1.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ This three-part walkthrough aims to bring some clarity to the confusing parts of

**Part 1** builds the mental model from scratch and by the end of it you'll be able to describe, step by step, what happens between the moment a tiering timer fires and the moment a lake snapshot is committed.

**Parts 2 and 3** take that mental model and add the dials (parallelism, table kinds, freshness, multi-table behavior, scale-out) and then put it into a real production deployment (failures, pitfalls, monitoring).
**[Part 2](/blog/fluss-tiering-service-deep-dive-part2) and Part 3** take that mental model and add the dials (parallelism, table kinds, freshness, multi-table behavior, scale-out) and then put it into a real production deployment (failures, pitfalls, monitoring).

**Tiering Service Deep Dive, 3-parts:**
* **Part 1 - The Mental Model:** how one tiering round actually works, from timer fire to lake commit.
* **Part 2 - Tuning:** per-table dials, multi-table dynamics, and scaling out.
* **[Part 2 - Tuning](/blog/fluss-tiering-service-deep-dive-part2):** per-table dials, multi-table dynamics, and scaling out.
* **Part 3 - In Production** failure modes, design pitfalls, and monitoring.
<!-- truncate -->

Expand Down Expand Up @@ -126,11 +126,11 @@ Three concepts are important to understand when you're reading this:

We have one table, called `orders`, with four buckets. It's a log table (we'll talk about what that means in Part 2, when we cover table kinds. For now, just think **"an append-only stream of records"**). Freshness is configured to five minutes. The Flink tiering job is up and running.

Here's what happens:.
Here's what happens:

#### T+0s the timer fires.

Five minutes have passed since the last tiering round for `orders`. The coordinator transitions the table from WAITING to READY and pushes it onto the back of the pending queue. The act of entering the pending queue is also what increments the table's tiering epoch · to (say) 7 · so every fresh attempt is stamped at enqueue time, before any job has picked it up. Right now the queue contains just `orders`.
Five minutes have passed since the last tiering round for `orders`. The coordinator transitions the table from WAITING to READY and pushes it onto the back of the pending queue. The act of entering the pending queue is also what increments the table's tiering epoch, to (say) 7, so every fresh attempt is stamped at enqueue time, before any job has picked it up. Right now the queue contains just `orders`.

#### T+0s the Flink job sends its heartbeat.

Expand Down Expand Up @@ -172,9 +172,10 @@ The freshness timer is computed from when the round **completed**, not from when

That's it. Round complete. The lake now contains `orders` data up to the offsets that were current at T+0s. In another ~five minutes, the whole process repeats.

> **The most subtle point in this section: The "stopping offset" is decided at planning time, not at reading time.** When the enumerator asks the tablet servers "what's your latest offset?" at T+0s, those answers freeze. Any records written to Fluss after T+0s but before the readers actually start working are not part of this round · they'll be picked up in the next one. This is correct behavior, but it means the lake always lags by at least one round, even if your freshness is configured aggressively.
> **The most subtle point in this section: The "stopping offset" is decided at planning time, not at reading time.** When the enumerator asks the tablet servers "what's your latest offset?" at T+0s, those answers freeze. Any records written to Fluss after T+0s but before the readers actually start working are not part of this round; they'll be picked up in the next one. This is correct behavior, but it means the lake always lags by at least one round, even if your freshness is configured aggressively.

## Next Up
You've now got the mental model, the processes, the heartbeat conversation, the lifecycle, and what happens during one full round.
The next part takes that round and shows what changes when you start tuning the dials, and then what changes again when you stop thinking about one table at a time.


319 changes: 319 additions & 0 deletions blog/2026-06-09-tiering-service-part2.md

Large diffs are not rendered by default.

Binary file added blog/assets/tiering_service_dd_part2/banner.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added blog/assets/tiering_service_dd_part2/fig7.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.