-
Notifications
You must be signed in to change notification settings - Fork 41
Tiered hot/cold storage for historical data #119
Copy link
Copy link
Open
Labels
Stellar WaveIssues in the Stellar wave programIssues in the Stellar wave programarea:indexerEvent ingestionEvent ingestiondifficulty:advancedMulti-day projectMulti-day projecthelp wantedExtra attention is neededExtra attention is neededpoints:200Advanced tier — 200 Wave pointsAdvanced tier — 200 Wave points
Description
Metadata
Metadata
Assignees
Labels
Stellar WaveIssues in the Stellar wave programIssues in the Stellar wave programarea:indexerEvent ingestionEvent ingestiondifficulty:advancedMulti-day projectMulti-day projecthelp wantedExtra attention is neededExtra attention is neededpoints:200Advanced tier — 200 Wave pointsAdvanced tier — 200 Wave points
Background
Recent data is queried constantly; old data rarely. Tiering keeps hot data fast while moving cold data to cheaper storage, transparently to API consumers.
What to build
Move partitions older than a threshold to a cold store (object storage / cheaper table), and make queries spanning the boundary transparently union hot + cold results.
Key files
src/storage/tiering.ts(new)src/api.tsAcceptance criteria
Required: Before submitting, join the contributor Telegram so your work can be tracked and counted toward the Stellar Wave: https://t.me/+fxHXq8f1SwlkZDBk