Ravel

The write is acknowledged when S3 has it. Not before.

Ravel is a database for OpenTelemetry metrics, logs, and traces where object storage is the only durable component. No write-ahead log. No replicated ingest quorum. No StatefulSet. Kill any Ravel process at any instant, and every strictly acknowledged write is still there.

Strict acknowledgement is the default. A request is acknowledged only after every batch its points contributed to has its data object durably stored and its commit record created, and the response carries one commit token per shard those points flushed through. After a strict acknowledgement, no crash of any Ravel process loses that data.

Buffered acknowledgement is opt-in per request. It acknowledges after admission and enqueue to a shard actor, and it returns no commit token: it trades the guarantee above for write latency. A crash between that acknowledgement and the flush loses the buffered window. A flush whose own store calls exceed the flush lifetime budget is abandoned instead, dropping already-acknowledged rows with no crash at all. The consistency model is normative for both modes.

The durability protocols above are modeled in TLA+. TLC checked five finite models of the commit, catalog, lifecycle, resharding, and maintenance protocols, over one shared object-store model, under stated bounds and assumptions. Negative controls show that each invariant can fail. Every checked property traces to a Rust symbol and, for all but five recorded rows, a named test. The formal verification guide has the bounds, the assumptions, and the results.

Why it is built this way

Every self-hosted observability stack ends up storing data on object storage. Almost none of them start there. The common design buffers writes in a replicated ingest layer with local disks, then ships to object storage later. That is why running one means running a write-ahead log, PersistentVolumeClaims, replication factors, and rollout ordering.

Ravel makes the object store the first stop. An ingest shard builds an immutable columnar segment in memory, PUTs it, PUTs a commit record, and only then answers the exporter. The response carries a commit token. Pass that token back to a query and you read your own write, with no listing race.

The trade is explicit. You pay object-store latency on the write path, and you delete the entire replicated stateful layer. Ravel's job is to make that a good trade.

Is Ravel a fit

What it is for. Metrics, logs, and traces on any S3-compatible object store, with no stateful ingest layer to operate and no local disk in the durability path. A Prometheus-compatible query API, so existing Grafana dashboards work against it. Read-your-write on an object store: a strict acknowledgement hands back a commit token, and a query that carries the token back reads exactly that write. Multi-tenancy with per-tenant server-side encryption using a key management service (SSE-KMS), legal hold, and admission limits.

What it does not do.

• No downsampled or pre-aggregated rollups. A wide-range metrics query reads every raw hour it covers.
• Traces are queryable over SQL only, as the spans table. Logs are queryable over SQL, as the logs table, and over PromQL, through the reserved ravel_log_lines and ravel_log_bytes metric names (see PromQL over logs). Neither surface gets you LogQL, TraceQL, a trace-by-ID endpoint, or a Jaeger or Tempo API.
• Profiles are a reserved object-key prefix only. No ingest, no query.
• Exemplars are stored from the OpenTelemetry Protocol (OTLP) only. Remote Write and OTAP decode exemplars and then discard them.
• Distributed read fan-out is off unless --distributed-query and --fragment-key-file are both given. The PromQL lane and the SQL lane (on the Flight SQL service) are both in the published image.

Who should wait. If your dashboards range over months of high-cardinality metrics, the missing downsampled storage will cost you on every panel. If your logs workflow depends on LogQL's line-pipeline browsing style, or your traces workflow depends on TraceQL or the Jaeger UI, there is nothing here to point them at; a Grafana Prometheus datasource can already chart log volume and error rate against Ravel through PromQL, and filter the lines it counts by body text, but it cannot browse the lines themselves. Reading log lines is SQL's job. If you need write acknowledgement in single-digit milliseconds, strict mode pays an object-store round trip and buffered mode gives up the crash guarantee above. Ravel is pre-1.0, and until v1.0 ships it may break backward compatibility: the persistent formats are versioned contracts, but a version change before v1.0 retires the old one rather than keeping it readable, and the surfaces around them still move. A bulk data-object format bump (RSEG, RLOG, RSPAN) is therefore a non-rollbackable, forward-only data-migration event: the reader admits one on-object version at a time, so once data is written at a new version, a build that predates the bump cannot read it, and objects left at the retired version are wiped or re-ingested. The other persistent formats do not work this way: rebuildable fold outputs and the additive, field-number-frozen records evolve without a one-way step. The two-version reader window that makes an upgrade reversible across one version boundary opens at v1.0, so treat a format upgrade as one-way until then. See the segment, log, and span format specs for the exact posture.

What works

Ravel ingests OTLP natively and answers PromQL and SQL. Everything below is either in the published container image or behind a named cargo feature, and the matrix says which.

The matrix describes main, not a release. Its "In published image" column means the surface is compiled into the image built from main, so a surface that lands after the tag the quickstart pins is not yet in the image you get by following the quickstart. Every entry below is in the pinned 0.15.0 image; git log v0.15.0..main is what tells you whether that is still true after the next feature lands, and CHANGELOG.md records which release each surface first shipped in. Two entries missed the previous pinned tag this way: the alerts and audit SQL tables were registered, and PromQL over logs was answered, only after 0.13.0 was cut, so a reader following the README against that image got a missing table and an empty vector.

Surface	Signals	Feature gate	In published image
OTLP ingest, HTTP and gRPC	metrics, logs, traces	none	yes
Prometheus Remote Write 1.0 and 2.0 ingest	metrics	none	yes
OTAP ingest, gRPC	metrics	otap	yes
PromQL HTTP API	metrics, logs as ravel_log_lines and ravel_log_bytes	none	yes
SQL over POST /api/v1/sql	metrics as samples (scalar samples only, never native histograms), logs as logs, traces as spans, alert history as alerts, audit records as audit	sql	yes
Flight SQL	the same five tables, with the same samples limit	flight-sql	yes

No crate in the workspace declares a default feature set, so sql, flight-sql, and otap are off in any build that does not ask for them. The published ravel-server image is built with --features sql,flight-sql,otap, so all three are compiled in. POST /api/v1/sql and Flight SQL answer as soon as the server is up; Flight SQL is a gRPC service on the gRPC listener, runs ad-hoc statements, and returns unimplemented for prepared statements. OTAP ingest is registered only when the process is started with --otap (docs/otap-ingest.md). A source build gets the same surfaces by passing the same --features list to cargo.

Exactly five SQL tables are registered: samples, logs, spans, alerts, and audit. SQL is the only way to query traces, alerts, and audit records. Logs are also queryable over PromQL, as ravel_log_lines and ravel_log_bytes.

The samples table has a Float64 value column and nothing that can hold a native histogram, so a histogram sample is not a row there and SELECT count(*) FROM samples counts scalar samples only. A JSON response whose query met excluded histogram data carries a top-level warnings array saying so; query native histograms over PromQL. See the query guide.

Also live:

• A Prometheus-compatible HTTP API, so existing Grafana dashboards work. /api/v1/metadata returns real per-metric type, help, and unit for metrics whose ingest carried that metadata, and OTLP metric names get the standard Prometheus-style unit and _total suffixes at ingest (a monotonic foo with unit: "By" lands as foo_bytes_total), so the same metric matches whether it arrives over OTLP or through a collector's Prometheus exporter.
• Exemplars that link a metric sample to its trace.
• Alert rules whose every transition is written to object storage as immutable data, and readable back through the alerts SQL table.
• An analytics endpoint for change point detection and summary statistics.
• Compaction, age-based retention, and garbage collection for metrics, logs, and spans, with audit compacted and retained on its own separate schedule. Alert transitions have no maintenance path yet.
• A Kubernetes operator with a RavelCluster custom resource.
• Per-tenant typed attribute columns on the logs SQL table, so typed comparisons and aggregates need no CAST over the stringified attrs map. See the query guide.

The SQL conformance table and the PromQL conformance table in the query engine spec classify every construct as supported, intentionally rejected, or unclassified. Both are generated, not written: the PromQL table from a differential test against a real Prometheus binary, and the SQL table from the conformance suite's recorded verdict for each construct. The gaps are measured rather than claimed.

Two ingest limits are worth knowing before you point a sender at Ravel:

• Metrics must be cumulative. A delta-temporality Sum or Histogram is rejected, and the OTLP response says so. Converting delta to cumulative needs per-series state held between requests, and Ravel's compute processes hold no durable local state, so the conversion belongs in the collector: the deltatocumulative processor does it, and the ingest guide has the configuration. Senders you control can usually be set to export cumulative directly instead.
• A structured log body (an array or a map) is stored as canonical JSON text, not as a nested value. It reads back as a JSON string, so a query that wants a field inside it parses that string.

Quickstart

One command starts the whole stack from published images: MinIO for object storage, ravel-server, an OpenTelemetry Collector that feeds it your host's own metrics, and Grafana with a provisioned Ravel datasource. No Rust toolchain, no compile.

docker compose -f deploy/docker-compose/ravel.yml up -d

Open Grafana at http://127.0.0.1:3000 (admin / admin). The Ravel datasource is already wired up. The first dashboard shows your machine's metrics after a few scrape intervals.

Query the data back over the Prometheus-compatible API. Every query needs the demo bearer token, exactly as a real deployment needs a real one:

curl -s -H "Authorization: Bearer demo-token" \
  'http://127.0.0.1:4318/api/v1/query?query=system_cpu_load_average_1m'

Logs answer the same PromQL API, through the reserved ravel_log_lines and ravel_log_bytes metric names (details):

curl -s -H "Authorization: Bearer demo-token" \
  --data-urlencode 'query=sum by (job) (count_over_time(ravel_log_lines[5m]))' \
  'http://127.0.0.1:4318/api/v1/query'

The published image carries the sql feature, so POST /api/v1/sql answers by default. The registered tables are samples, logs, spans, alerts, and audit:

curl -s -X POST http://127.0.0.1:4318/api/v1/sql \
  -H "Authorization: Bearer demo-token" \
  -H "Content-Type: application/json" \
  -d '{"query":"SELECT * FROM samples LIMIT 5"}'

To watch the read-your-write path directly, run demo/walkthrough.sh while the stack is up. It ingests one export, captures its commit token, and reads that exact write back.

Stop the stack:

docker compose -f deploy/docker-compose/ravel.yml down

The getting started guide walks this same path with what each response means, how long to wait for data, and what an empty result looks like when it is expected.

Kill the server, keep the data

The GIF above is a recording of demo/kill-and-recover.sh, which demonstrates the durability claim against the running stack:

demo/kill-and-recover.sh

It ingests one export under strict acknowledgement and captures the x-ravel-commit-token from the response. It then SIGKILLs the ravel-server container, so the process cannot flush anything on its way out. It deletes that container, starts a fresh one with an empty filesystem, and reads the pre-kill sample back with min_commit_token. Nothing crosses the kill except what is in MinIO.

The script asserts every step and exits non-zero if the sample is absent or the token comes back unsatisfiable. A passing run is evidence, not a demonstration you have to watch closely. CI runs it against a live stack on every change to the quickstart.

Security of the demo stack

Every credential in deploy/docker-compose/ravel.yml is a fixed development value: the demo-token bearer token and the MinIO ravel / ravel-dev-secret pair. Every published host port binds loopback (127.0.0.1) only, so the checked-in token never fronts an ingest endpoint on your network. None of these values are for a deployment that a network can reach.

Beyond the demo stack

A real deployment points --store s3 at any S3-compatible store, exactly as the quickstart points it at MinIO, and on EC2 it can drop static keys entirely with --s3-auth instance-role. The operations guide documents every storage flag, including temporary session tokens and a rotating credentials file.

Every query byte comes from object storage, so a read cache sits in front of it. The RAM tier is on by default, bounded by --cache-max-bytes and switched off with --disable-cache; --cache-dir <path> adds a second, disposable disk tier. Neither tier holds durable state. See the caching guide.

Changing Ravel's code? make demo builds from source and runs the same round trip. It does not build the sql feature, so POST /api/v1/sql is unavailable on that path while PromQL and ingest behave the same. The development guide covers the source workflow.

How it fits together

A write is durable once its commit record is on the object store. A reader sees it once the catalog resolves that commit into a snapshot. The ingest guide covers the write path, the query guide covers the read path, and architecture is the one-page overview.

All query endpoints live under /api/v1 on the HTTP listener, which binds 127.0.0.1:4318 by default. They need Authorization: Bearer <token>, the same as ingest.

One maintenance route sits alongside them: POST /api/v1/admin/fold triggers a catalog fold for the authenticated tenant and one named signal, instead of waiting for the background fold's next tick. It takes the same bearer token the query routes take, and its response says which of four things happened: a snapshot was published, nothing_eligible was found to fold, a concurrent fold won the HEAD compare-and-swap (lost_cas), or the call was throttled. Right after a load the honest answer is nothing_eligible: an ingest hour is not foldable until the sealing window behind it has elapsed. See architecture.

Agents (MCP)

The MCP agent surface is documented in docs/guides/agents.md. It ships behind the mcp cargo feature (cargo build -p ravel-server --features mcp) and the runtime --mcp flag, both off by default: build with the feature, then pass --mcp (and --mcp-allowed-origins, required whenever the MCP listener is reachable from outside localhost) to mount POST /mcp on the query router. One tool is served today, ravel_capabilities; the other eight are catalogued and refuse calls until their bodies land.

Kubernetes

The operator runs the same ingest and query round trip on a real cluster. This needs docker, kind, and kubectl:

scripts/kind-up.sh     # cluster, images, fake S3, operator, RavelCluster
scripts/kind-demo.sh   # ingest through gateway mode, query through query mode
scripts/kind-down.sh

See the Kubernetes guide.

Container images

ravel-server, ravel-operator, and ravel-ingest-router publish to the GitHub Container Registry on every vX.Y.Z release tag, built from the root Dockerfile. Both linux/amd64 and linux/arm64 are published. Each published object is an OCI image index that carries an SBOM and full build provenance. The quickstart pins ghcr.io/nofireai/ravel-server:0.15.0. Override it with RAVEL_IMAGE.

docker pull ghcr.io/nofireai/ravel-server:latest
docker pull ghcr.io/nofireai/ravel-operator:latest
docker pull ghcr.io/nofireai/ravel-ingest-router:latest

X.Y.Z is write-once. A bad release is superseded by a new patch release, never by re-pushing the tag. latest, X, and X.Y move with the newest matching release. Pin by digest (ghcr.io/nofireai/ravel-server@sha256:...) when you need an immutable reference.

Verifying signatures

Every published index digest is signed with cosign in keyless mode. The signing certificate binds the signature to the release workflow's identity, so you can verify a pull without a pre-shared key. Releases are cut from this repository, so that is the identity in the certificate:

cosign verify \
  --certificate-identity 'https://github.com/NOFireAI/ravel/.github/workflows/publish-images.yml@refs/tags/v0.15.0' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  ghcr.io/nofireai/ravel-server:0.15.0

Replace v0.15.0 and 0.15.0 with the release you are verifying. The tag ref in --certificate-identity must be the exact tag that produced the image.

How Ravel is verified

Durability claims are cheap to write and hard to keep. These are the checks that hold Ravel to them:

• TLA+ models check the commit, catalog, lifecycle, resharding, and maintenance protocols over one shared object-store model, and every checked property observes the store itself rather than the model's own bookkeeping. Each area carries negative controls that show its invariants can fail, and reachability obligations that show a guarded state is still reached. scripts/check-tla.sh runs the smoke, negative, and traceability lanes on every pull request that touches formal/ or the paths it models. The exhaustive lane runs nightly. See the formal verification guide for the results.
• A deterministic simulation harness drives the full ingest, fold, compact, sweep, and query cycle under injected faults. It checks read-your-write, strict-ack durability, compaction equivalence, record-count conservation, and orphan-free sweeps every cycle. Any violation prints its master seed and a one-command replay. A nightly job sweeps 200 seeds.
• The PromQL evaluator is differentially tested against a pinned real Prometheus binary, and the per-construct result is published as a conformance table.
• unsafe is forbidden workspace-wide, at the compiler rather than by review.
• Property tests cover every codec and parser. Fuzz targets run on the segment and span formats.
• A fault-injection store fails operations by kind, key, and occurrence, and the failure-path tests assert its counters.
• The consistency model is normative, and its crash matrix is test-asserted.

Repository layout

• crates/ holds the types, the object store, the segment formats (RSEG for metrics, RLOG for logs, RSPAN for spans), the commit protocol, the catalog, the decoders, the ingest actors, PromQL, the query engine, and DataFusion-backed SQL.
• services/ holds ravel-server (the all, gateway, query, and maintain modes in one binary), ravel-cli (a segment, commit, and catalog inspector, and the Parquet bulk loader), ravel-ingest-router (the optional tenant-affinity front door), and the Kubernetes operator.
• docs/ holds the guides, the specs, the decision records, and the diagrams.
• deploy/ holds the quickstart compose stack, the Collector and Grafana provisioning, and the Kubernetes manifests.

Documentation

• Getting started is the recommended path from nothing to a first query.
• The documentation index lists every guide, spec, and decision record.
• Architecture is the mental model, and the consistency model is normative for acknowledgement, visibility, and crash behavior.
• Formats: RSEG for metrics, RLOG for logs, and RSPAN for spans.
• Contributing, the changelog, and the AI policy.

License

Apache 2.0. See LICENSE.