-
-
Notifications
You must be signed in to change notification settings - Fork 1
feat(stacks): Add Evidence — SQL+markdown BI for analytics engineers #616
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
2176457
feat(stacks): Add Evidence — SQL+markdown BI for analytics engineers
stefanko-ch 7dea190
fix(stacks): Address PR #616 review comments
stefanko-ch 4cb969e
docs(claude): Add evidence to the :latest allow-list exception
stefanko-ch 99ff5d8
fix(stacks): Address PR #616 review comment
stefanko-ch d584788
fix(stacks): Rename EVIDENCE_DOMAIN to EVIDENCE_BASE_URL + tighten test
stefanko-ch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| --- | ||
| title: "Evidence" | ||
| --- | ||
|
|
||
| ## Evidence | ||
|
|
||
|  | ||
|
|
||
| **SQL + markdown BI for analytics engineers** | ||
|
|
||
| [Evidence](https://evidence.dev) is an open-source "BI as code" framework: each page is a Markdown file with embedded SQL blocks that render to charts, tables, and inline values. Projects are plain text — version them in Git, edit them in your normal tools, and the dev server reloads on save. | ||
|
|
||
| This stack ships the Evidence `devenv` runtime preloaded with a sample project that queries the in-stack Postgres. Extend the `sources/` directory to connect ClickHouse, Trino, DuckDB, Iceberg/Lakekeeper, or any external warehouse. | ||
|
|
||
| | Setting | Value | | ||
| |---------|-------| | ||
| | Host Port | `3007` (container internal port is Evidence's default `3000`; 3000–3006 are already taken by Metabase/Uptime-Kuma/Wetty/Hoppscotch/Dagster/Wiki.js/big-AGI) | | ||
| | Suggested Subdomain | `evidence` | | ||
| | Public Access | No (Cloudflare Access via email OTP) | | ||
| | Website | [evidence.dev](https://evidence.dev) | | ||
| | Source | [GitHub](https://github.com/evidence-dev/evidence) | | ||
| | Docker image | [`evidencedev/devenv`](https://hub.docker.com/r/evidencedev/devenv) | | ||
| | Project root | `/opt/docker-server/stacks/evidence/project/` on the server (mounted at `/evidence-workspace` inside the container) | | ||
|
|
||
| ### Why Evidence | ||
|
|
||
| Most BI tools assume a GUI workflow: drag dimensions onto a canvas, save the chart as a binary artifact, hope the underlying SQL stays in sync. Evidence inverts that: SQL is the source of truth, charts are derived, the whole project diffs as plain text. That makes it a natural fit alongside the existing **code-server**, **gitea**, and **woodpecker-ci** stacks — you edit pages like you edit any other code, push to a feature branch, and review the rendered diff before merging to main. | ||
|
|
||
| Compared to the other BI tools in this stack: | ||
|
|
||
| | Tool | Best for | Auth | | ||
| |---|---|---| | ||
| | **Metabase** | Self-service exploration by non-technical users | Built-in user management | | ||
| | **Superset** | Dashboards with rich GUI editing + drilldowns | Built-in user management | | ||
| | **Evidence** | Code-first, Git-reviewed analytics pages | Cloudflare Access at the edge | | ||
|
|
||
| ### Usage | ||
|
|
||
| 1. Enable **Evidence** in the Control Plane → Spin Up. | ||
| 2. Open `https://evidence.YOUR_DOMAIN` → CF Access email OTP → landing page. | ||
| 3. Edit the sample page at `/opt/docker-server/stacks/evidence/project/pages/index.md` on the server (the project root is bind-mounted into the container at `/evidence-workspace`, so changes apply on save). | ||
| 4. Add new pages as `.md` files in `/opt/docker-server/stacks/evidence/project/pages/` — each one renders at `https://evidence.YOUR_DOMAIN/<filename>`. | ||
|
|
||
| ### Adding data sources | ||
|
|
||
| Each source lives in its own directory under `project/sources/<name>/`: | ||
|
|
||
| ``` | ||
| project/sources/ | ||
| ├── nexus_postgres/ # shipped with the stack | ||
| │ ├── connection.yaml # connection config (env-var interpolated) | ||
| │ └── database_overview.sql | ||
| └── my_clickhouse/ # operator adds this | ||
| ├── connection.yaml | ||
| └── ... | ||
| ``` | ||
|
|
||
| `connection.yaml` supports `${VAR}` interpolation against the container's environment, so the recommended pattern is: | ||
|
|
||
| 1. Add the credentials to Infisical under a folder of your choice. | ||
| 2. Reference them from `stacks/evidence/.env` (the deploy pipeline renders this from Infisical on every spin-up). | ||
| 3. Use `${VAR}` in `connection.yaml` to reference them. | ||
|
|
||
| For ClickHouse, Trino, MySQL, BigQuery, Snowflake, and others, see the [Evidence connector docs](https://docs.evidence.dev/core-concepts/data-sources/). Add the matching `@evidence-dev/<driver>` package to `stacks/evidence/project/package.json` and run `docker compose restart evidence` to pull it in. | ||
|
|
||
| ### Building a static site | ||
|
|
||
| For a production hand-off, the devenv runtime can build a static HTML export: | ||
|
|
||
| ```bash | ||
| ssh nexus 'docker exec evidence npm run sources && docker exec evidence npm run build' | ||
| ``` | ||
|
|
||
| Output lands in `/opt/docker-server/stacks/evidence/project/build/` on the server. Copy it into any of the file-store stacks (MinIO, Garage, SeaweedFS, RustFS) and serve as static HTML — or commit it to a GitHub Pages / Cloudflare Pages repo for a fully decoupled deploy. | ||
|
|
||
| ### Secrets | ||
|
|
||
| No Tofu-managed secrets specific to Evidence. The bundled sample project reads the in-stack Postgres credentials (`POSTGRES_PASSWORD`) which are already managed via the **postgres** stack and Infisical. Operator-added data sources reference whatever credentials the operator wires into `stacks/evidence/.env` — no double-managing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # ============================================================================= | ||
| # Evidence - SQL + markdown BI for analytics engineers | ||
| # ============================================================================= | ||
| # Evidence is a "BI as code" framework: write SQL queries inside | ||
| # .md files and the framework renders charts, tables, and value | ||
| # components inline. The project directory is plain markdown + | ||
| # YAML — version it in Git, edit with your normal tools. | ||
| # | ||
| # This stack ships the `devenv` runtime — the container watches the | ||
| # mounted project directory and re-renders pages on save. For a | ||
| # static production export run: | ||
| # docker exec evidence npm run sources && docker exec evidence npm run build | ||
| # and serve the `build/` output from any of the file-store stacks. | ||
| # | ||
| # Data sources are configured per-project in `project/sources/`. | ||
| # Connection strings reference env vars (Evidence supports | ||
| # ${VAR} interpolation in sources/*/connection.yaml), so the | ||
| # operator points them at the Infisical-managed Postgres / | ||
| # ClickHouse / Trino / DuckDB credentials that already exist | ||
| # elsewhere in the stack. | ||
| # | ||
| # Access: https://evidence.YOUR_DOMAIN (behind Cloudflare Access | ||
| # unless you flip `public: true` in services.yaml). | ||
| # ============================================================================= | ||
|
|
||
| services: | ||
| evidence: | ||
| # devenv:latest per CLAUDE.md exception for non-critical | ||
| # standalone tools (Evidence is a presentation layer with no | ||
| # persistent state beyond cache). The devenv image is the | ||
| # supported dev-runtime entrypoint; Evidence does not publish | ||
| # versioned devenv tags. | ||
| image: ${IMAGE_EVIDENCE:-evidencedev/devenv:latest} | ||
|
stefanko-ch marked this conversation as resolved.
|
||
| container_name: evidence | ||
| # env_file picks up the renderer-populated POSTGRES_PASSWORD / | ||
| # EVIDENCE_BASE_URL AND any operator-added variables in | ||
| # stacks/evidence/.env — Evidence sources/<name>/connection.yaml | ||
| # supports ${VAR} interpolation, so additional data-source | ||
| # credentials reach the container without having to extend the | ||
| # explicit `environment:` list below. | ||
| env_file: | ||
| - .env | ||
| restart: unless-stopped | ||
| ports: | ||
| # Host 3007 → container 3000 (Evidence's SvelteKit dev port). | ||
| # 3000-3006 are already taken by metabase/uptime-kuma/wetty/ | ||
| # hoppscotch/dagster/wikijs/big-agi. | ||
| - "3007:3000" | ||
| environment: | ||
| # Bind the dev server to all interfaces so the published port | ||
| # is reachable from the host's network namespace (and through | ||
| # the Cloudflare Tunnel). | ||
| HOST: 0.0.0.0 | ||
| PORT: "3000" | ||
| # EVIDENCE_BASE_URL (used by Evidence for absolute links + OG tags) | ||
| # is provided by env_file above — no explicit entry needed. | ||
| # Point the user-supplied sources/*/connection.yaml files at | ||
| # the in-stack Postgres for the bundled sample query. Operators | ||
| # extend the sources/ folder with additional connections. | ||
| POSTGRES_HOST: postgres | ||
| POSTGRES_PORT: "5432" | ||
| POSTGRES_DATABASE: postgres | ||
| POSTGRES_USER: nexus-postgres | ||
| POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} | ||
|
stefanko-ch marked this conversation as resolved.
|
||
| volumes: | ||
| # The sample project shipped under stacks/evidence/project/ is | ||
| # synced to /opt/docker-server/stacks/evidence/project/ by the | ||
| # deploy pipeline and mounted read-write so users can edit it | ||
| # in-place via code-server / a Git checkout. | ||
| - ./project:/evidence-workspace | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "wget -qO- http://localhost:3000/ >/dev/null 2>&1 || exit 1"] | ||
| interval: 30s | ||
| timeout: 10s | ||
| retries: 3 | ||
| start_period: 90s | ||
| networks: | ||
| - app-network | ||
|
|
||
| networks: | ||
| app-network: | ||
| external: true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| node_modules/ | ||
| .evidence/ | ||
| build/ | ||
| static/data/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| deployment: | ||
| basePath: "" | ||
|
|
||
| appearance: | ||
| default: system | ||
| switcher: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| { | ||
| "name": "nexus-evidence-project", | ||
| "version": "0.0.1", | ||
| "private": true, | ||
| "scripts": { | ||
| "build": "evidence build", | ||
| "build:strict": "evidence build:strict", | ||
| "dev": "evidence dev --host 0.0.0.0 --port 3000", | ||
| "sources": "evidence sources" | ||
| }, | ||
| "dependencies": { | ||
| "@evidence-dev/core-components": "^5.0.0", | ||
| "@evidence-dev/evidence": "^40.1.8", | ||
| "@evidence-dev/postgres": "^2.0.0" | ||
| }, | ||
| "type": "module", | ||
| "engines": { | ||
| "npm": ">=7.0.0", | ||
| "node": ">=18.0.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| --- | ||
| title: Nexus-Stack on Evidence | ||
| --- | ||
|
|
||
| Welcome to Evidence. This file is `pages/index.md` in the project mounted at | ||
| `/evidence-workspace`. Edit it from the host (or via the `code-server` / | ||
| `gitea` stacks) and the dev server reloads on save. | ||
|
|
||
| ## Postgres source | ||
|
|
||
| The bundled `sources/nexus_postgres/` reads the in-stack Postgres credentials | ||
| through the env vars that `docker-compose` populates from Infisical. The | ||
| sample query below lists the largest tables in the `public` schema — if you | ||
| have not yet loaded data, the result will be empty, which is also a healthy | ||
| signal that the connection is wired up. | ||
|
|
||
| ```sql database_overview | ||
| select * from nexus_postgres.database_overview | ||
| ``` | ||
|
|
||
| <DataTable data={database_overview} rows=25 /> | ||
|
|
||
| ## Adding more sources | ||
|
|
||
| Drop a sibling directory under `project/sources/` with its own | ||
| `connection.yaml` and Evidence will pick it up on the next `npm run sources`. | ||
| Connection strings can reference environment variables via `${VAR}` syntax, | ||
| so the recommended pattern is to add the relevant credentials to the | ||
| `stacks/evidence/.env` file (which the deploy pipeline renders from | ||
| Infisical) and reference them here. | ||
|
|
||
| For ClickHouse, Trino, DuckDB, Iceberg/Lakekeeper and other backends, see | ||
| the Evidence connector docs and add the matching `@evidence-dev/<driver>` | ||
| package to `package.json`. | ||
|
|
||
| ## Building a static export | ||
|
|
||
| For a production hand-off, run the two commands below inside the | ||
| running container: | ||
|
|
||
| ```bash | ||
| docker exec evidence npm run sources | ||
| docker exec evidence npm run build | ||
| ``` | ||
|
|
||
| The output lands in `project/build/`; copy it into any of the file-store | ||
| stacks (MinIO/Garage/SeaweedFS/RustFS) and serve it as static HTML. |
18 changes: 18 additions & 0 deletions
18
stacks/evidence/project/sources/nexus_postgres/connection.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Nexus-Stack default Postgres source. | ||
| # Credentials come from environment variables that the | ||
| # docker-compose env_file populates from Infisical. | ||
| # Add more sources by creating sibling directories under | ||
| # stacks/evidence/project/sources/<name>/ with their own | ||
| # connection.yaml that uses ${VAR} interpolation for any | ||
| # credentials. Place .sql query files alongside it; Evidence | ||
| # discovers both automatically on `npm run sources`. | ||
| name: nexus_postgres | ||
| type: postgres | ||
| options: | ||
| host: ${POSTGRES_HOST} | ||
| port: ${POSTGRES_PORT} | ||
| database: ${POSTGRES_DATABASE} | ||
| user: ${POSTGRES_USER} | ||
| password: ${POSTGRES_PASSWORD} | ||
| schema: public | ||
| ssl: false |
10 changes: 10 additions & 0 deletions
10
stacks/evidence/project/sources/nexus_postgres/database_overview.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| -- Snapshot of the public schema: table list + estimated row counts. | ||
| -- Edit or replace with queries that match the data the operator has | ||
| -- loaded into the nexus-postgres instance. | ||
| SELECT | ||
| relname AS table_name, | ||
| n_live_tup AS estimated_rows | ||
| FROM pg_stat_user_tables | ||
| ORDER BY n_live_tup DESC NULLS LAST, | ||
| relname | ||
| LIMIT 50; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.