HighWater

Know when your data stops flowing.

Self-hosted data freshness & quality monitoring — the "Uptime Kuma energy" your tables, APIs and files have been waiting for.

npm i -g highwater · npx highwater demo · zero native dependencies

---

Your nightly ETL failed at 2am. Your API silently stopped updating. Someone renamed a column everyone depends on. You find out the next morning — or worse, from a stakeholder.

Uptime monitors watch whether your servers are up. HighWater watches whether your data is alive:

checks:
  - name: orders_freshness
    type: freshness          # newest row must be younger than 6h
    source: warehouse
    config:
      table: orders
      warn_after: 2h
      fail_after: 6h

That's a complete monitor. HighWater runs it on a schedule, records every observation, opens incidents, fires Slack/Discord/webhook alerts, and draws you a live dashboard.

Why HighWater

The problem	Existing answers	HighWater
"Is my data stale?"	Monte Carlo / Bigeye — enterprise SaaS, $$$	✅ free, self-hosted, one binary feel
Pipeline-internal tests	Great Expectations / Soda / dbt tests — Python, tied to your pipeline framework	✅ standalone: watches any SQLite / Postgres / MySQL / HTTP / file
DIY	Grafana + hand-written SQL alerts — hours of glue	✅ YAML in, dashboard out
Heavier is not better	Loki / OpenObserve — full observability platforms	✅ one job done well: freshness & quality

Design tenets

• Zero native dependencies — SQLite state store via Node's built-in node:sqlite. No compiled addons, no gRPC, no JVM.
• YAML in, dashboard out — a complete monitoring setup is ~20 lines of config.
• Local-first & private — results live in ~/.highwater/state.db. Your data values never leave your machine; alerts carry messages, not rows.
• Cron-friendly — highwater check single runs exit 1 on failure, so plain crontab works too.
• Demo in 30 seconds — highwater demo generates a realistic warehouse with deliberate failures so you can see incidents, not a green wall.

Quickstart

# 1. Generate a demo warehouse (120 days of realistic orders, with a stalled pipeline)
npx highwater demo

# 2. Open the dashboard
npx highwater serve --config ~/highwater-demo/config.yaml --open

You'll see two failing checks, one warning, live sparklines and open incidents — the whole story in one screen.

Monitoring your own data takes 3 steps

highwater init          # writes a commented highwater.yaml
# edit sources + checks
highwater test          # validate & dry-run everything — nothing recorded
highwater serve         # dashboard + scheduler on :4021

Run it under systemd/Docker, or keep it dead simple with cron:

*/10 * * * * highwater check --config /etc/highwater.yaml || notify-me

What it watches

Check type	Answers	Works on
freshness	Is the newest row young enough?	SQLite · Postgres · MySQL · ClickHouse (timestamp column auto-detected)
row_count	Is volume sane vs. minimum & trailing baseline?	SQLite · Postgres · MySQL · ClickHouse
schema_drift	Did anyone change the columns I depend on?	SQLite · Postgres · MySQL · ClickHouse
null_ratio	Is a column quietly filling with NULLs?	SQLite · Postgres · MySQL · ClickHouse
sql	Anything you can ask in SQL — the query is the check	SQLite · Postgres · MySQL · ClickHouse
http_json	Is an endpoint up, returning the right value, fresh enough?	any HTTP JSON API
file	Did the nightly CSV/parquet drop actually arrive?	any filesystem path

Prometheus: GET /metrics exposes per-check gauges for your Grafana. Alerting: Slack, Discord, or generic JSON webhooks — with failure thresholds (alert after N consecutive bad runs), recovery notifications, and per-check mute windows. Dashboard: live SSE updates, per-check history charts, incident timeline, keyboard-first (/ to filter).

A taste of the config

version: 1

sources:
  - name: warehouse
    type: postgres
    url: postgres://reader:${PG_PASSWORD}@db.internal:5432/analytics
  - name: api
    type: http
    url: https://api.example.com/v1
    headers: { authorization: "Bearer ${API_TOKEN}" }

checks:
  - name: orders_freshness
    type: freshness
    source: warehouse
    interval: 5m
    tags: [revenue]
    config: { table: orders, warn_after: 2h, fail_after: 6h }

  - name: dim_customer_schema
    type: schema_drift
    source: warehouse
    config:
      table: customers
      mode: superset          # additions warn, removals fail
      expect: ["id INTEGER", "email TEXT"]

  - name: api_status
    type: http_json
    source: api               # inherits base URL + auth header
    config: { url: /health, path: "data.generated_at", warn_after: 15m }

notify:
  - type: slack
    url: ${SLACK_WEBHOOK}
    on: [fail, recovery]

api_token: ${HIGHWATER_TOKEN}   # dashboard login becomes ?token=...

Full reference: docs/CONFIG.md · Architecture: docs/ARCHITECTURE.md

CLI

highwater demo [dir]        realistic demo warehouse with intentional failures
highwater init              commented starter config
highwater test              validate config + dry-run every check
highwater check             run all checks once (exit 1 on failure — cron-friendly)
highwater watch             live terminal board
highwater serve [--port]    dashboard + scheduler (default :4021)
highwater status            last known results + open incidents
highwater prune --keep 90d  retention for stored history
highwater notify-test       verify your webhook wiring

Install

npm i -g highwater          # or: brew install frankfu916/tap/highwater

Requires Node ≥ 22.5 (built-in node:sqlite). Docker:

docker run -p 4021:4021 -v ~/.highwater:/data ghcr.io/frankfu916/highwater

How it compares

	HighWater	Monte Carlo	Great Expectations	Grafana+SQL	Uptime Kuma
Self-hosted, free	✅	❌ SaaS	✅	✅	✅
Install effort	npx	sales call	pip + pipeline integration	hours	npx
Watches data freshness	✅	✅	⚠️ in-pipeline	⚠️ DIY	❌ watches servers
Config	~20 lines YAML	UI	Python code	hand-written alerts	UI
Built for	data people	enterprises	pipeline authors	ops people	ops people

Roadmap

• Prometheus /metrics endpoint
• Email (SMTP) notifications
• More sources: ClickHouse, CSV over S3/GCS
• Check templates & import from dbt schema.yml
• Multi-channel escalation (warn → page after N hours)

Hot reload: kill -HUP <pid> re-reads the config without dropping state.

See ROADMAP.md for details and CHANGELOG.md for releases.

Contributing

PRs welcome — the engine is ~1.5k lines of dependency-free TypeScript with 74 tests. Start with CONTRIBUTING.md.

License

MIT © FrankFu916

_{⌁ HighWater — because "the dashboard said green" is not a data contract.}