Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions cmd/mon-server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"net/http"
"os"
"os/signal"
"strings"
"syscall"
"time"

Expand Down Expand Up @@ -450,6 +451,10 @@ func main() {
// stateful checks (failed-login threshold, security updates) on a
// 60s tick.
eng := alerts.New(st.Pool, lw.Out, sched.Out)
// Host-context enrichment for HTML alert emails; MON_PUBLIC_URL (optional)
// adds a deep link to the host page.
eng.Store = st
eng.PublicURL = strings.TrimSpace(os.Getenv("MON_PUBLIC_URL"))
go eng.Run(ctx)

// Housekeeping reaper: bounds tables that don't have a Timescale
Expand Down
4 changes: 4 additions & 0 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,6 +512,10 @@ See `docs/SIGNING.md` for the full procedure.
| `MON_TLS_KEY` | unset | Private key for direct TLS. |
| `MON_RP_ID` | `localhost` | WebAuthn RP ID — must be the bare host. |
| `MON_RP_ORIGIN` | `http://localhost:5173` | WebAuthn RP origin — full scheme+host(+port). |
| `MON_PUBLIC_URL` | unset | Public base URL (e.g. `https://mon.example.com`); adds a "View host" deep link to HTML alert emails. |
| `MON_ALLOW_INSECURE_HTTP` | unset | Set to `1` to permit plain-HTTP startup behind a TLS-terminating proxy. |
| `MON_TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For`/`X-Real-IP` headers are trusted. |
| `MON_DATA_ENCRYPTION_KEY` | unset | Base64 32-byte AES-256 key; enables TOTP-seed encryption at rest. |
| `MON_LOG_BUFFER` | `5000` | In-memory log ring buffer size. |
| `MON_INGEST_BUFFER` | `100` | Captured-ingest ring buffer size. |
| `MON_INGEST_MAX_BYTES`| `1048576` | Max captured ingest body size. |
82 changes: 82 additions & 0 deletions docs/superpowers/specs/2026-07-16-html-alert-email-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# HTML alert emails with host context — design

**Date:** 2026-07-16
**Status:** approved (design)

## Problem

Alert emails are plain text built per rule type via `fmt.Sprintf`. They are hard
to scan and identify the host only by UUID (e.g. `Host f1360f46-… sustained
ram_used_pct > 80.00`). Operators want a readable HTML email that names the host
and shows enough context to triage without opening the dashboard.

## Goals

- Render a clean, responsive HTML email for the SMTP channel.
- Resolve and show the **hostname** (not the UUID) plus host facts: IP, OS/kernel,
arch/CPU/RAM, agent version, status + last-seen, uptime.
- Keep a plain-text alternative (accessibility, non-HTML clients).
- Leave Slack/Mattermost/Discord/ntfy unchanged (they already use plain/markdown).
- No new required config. Optional `MON_PUBLIC_URL` adds a "View host" button.

## Non-goals

- Redesigning non-email channels.
- Per-rule bespoke templates — one template driven by structured fields.
- Charts/images in the email (keep it lightweight and MIME-simple).

## Design

### 1. Structured message fields (`internal/server/notify`)
Extend `notify.Message` with optional fields; `Subject`/`Body` stay as the
canonical plain-text form used by every channel:

```
Severity string // existing
FiredAt time.Time // optional
RuleName string // optional
Metric *MetricDetail // optional: Name, Current, Threshold
Host *HostContext // optional: Name, IP, OS, Kernel, Arch, CPUCores, RAMBytes, AgentVersion, Status, LastSeen, UptimeSec
HostURL string // optional deep link (only when MON_PUBLIC_URL set)
```
When `Host`/`Metric` are nil, the HTML simply omits those blocks. This keeps every
existing caller valid (they set only Subject/Body/Severity today).

### 2. Enrichment at the single choke point (`internal/server/alerts`)
All alerts pass through `Engine.fire(ctx, r, subject, body, dedup)`. Enrich there:
- `splitDedupKey(dedup)` already yields the host UUID for host-scoped rules.
- New `store` method `HostContext(ctx, hostID) (apitypes.HostContext, error)` returns
the facts from the `hosts` table (+ latest status / uptime). One query, read-only.
- Populate `Message.Host` when a host id is present; leave nil otherwise
(monitor-only / global rules render without a host block).
- `HostURL` built from `MON_PUBLIC_URL` + `/hosts/<id>` when the env var is set.
- The `Metric` block is populated for the `metric_threshold` condition type, which
already has name/value/threshold in scope; other types leave it nil.

### 3. HTML rendering (`internal/server/notify`)
- New `email_html.go`: a `html/template` (auto-escaping) producing an
email-safe, table-based, inline-styled document. Severity → accent color
(critical `#b91c1c`, warning `#e8a23a`, info `#3aa3e8`).
- `SMTP.Send` builds a `multipart/alternative` message: part 1 `text/plain`
(existing `m.Body`), part 2 `text/html` (rendered template). `buildRFC5322`
grows a multipart variant; the existing plain-only helper stays for callers
that pass no HTML.
- All dynamic values flow through `html/template`, so host/rule strings are
auto-escaped (no injection from hostnames/labels).

### 4. Error handling
- HTML render failure falls back to the plain-text-only email (never drops the
alert). Logged at warn.
- `HostContext` lookup failure → send without the host block (best-effort
enrichment must not block delivery).

## Testing
- `notify`: unit test that `SMTP` message building yields a `multipart/alternative`
with both parts and that the HTML escapes a hostile hostname.
- `notify`: template renders with `Host == nil` / `Metric == nil` (no panic, blocks omitted).
- `alerts`: `HostContext` populated maps into `Message.Host` (existing engine tests
extended or a focused test on the mapping helper).

## Rollout
- Backward compatible; no migration. Ships in the normal release.
- `MON_PUBLIC_URL` documented in OPERATIONS.md as optional (enables the deep link).
52 changes: 52 additions & 0 deletions internal/server/alerts/alerts.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"github.com/MalteKiefer/MonSys/internal/server/liveness"
"github.com/MalteKiefer/MonSys/internal/server/notify"
"github.com/MalteKiefer/MonSys/internal/server/probe"
"github.com/MalteKiefer/MonSys/internal/server/store"
)

// OnAlertFired is an optional metric-emission hook called from fire()
Expand All @@ -40,6 +41,14 @@ var OnAlertFired func(conditionType, severity string)
type Engine struct {
Pool *pgxpool.Pool

// Store enriches alert notifications with host context (HTML email). May
// be nil (tests / minimal wiring), in which case alerts still deliver
// without the host block.
Store *store.Store
// PublicURL, when set (MON_PUBLIC_URL), produces a deep link to the host
// page in the notification.
PublicURL string

LivenessOut <-chan liveness.Transition
MonitorOut <-chan probe.ResultEvent

Expand Down Expand Up @@ -531,6 +540,7 @@ func (e *Engine) fire(ctx context.Context, r ruleRow, subject, body, dedup strin
}

m := notify.Message{Subject: subject, Body: body, Severity: severity}
e.enrichHost(ctx, r.Name, dedup, &m)

delivered := []string{}
deliverErrs := map[string]string{}
Expand Down Expand Up @@ -693,6 +703,48 @@ func (e *Engine) resolve(ctx context.Context, dedup, body string) {
// "<kind>:<uuid>" and returns them as driver-friendly any values. nil means
// "store NULL"; this lets the upsert skip host_id for monitor-scoped rules
// and vice versa without a separate code path.
// enrichHost adds host context + deep link to an outgoing alert message so the
// HTML email can name the host and show its facts. Best-effort: any failure
// leaves the message plain and delivery proceeds.
func (e *Engine) enrichHost(ctx context.Context, ruleName, dedup string, m *notify.Message) {
m.RuleName = ruleName
m.FiredAt = time.Now()
if e.Store == nil {
return
}
hostArg, _ := splitDedupKey(dedup)
hid, ok := hostArg.(uuid.UUID)
if !ok {
return
}
f, err := e.Store.HostFactsFor(ctx, hid)
if err != nil {
return
}
m.Host = &notify.HostContext{
Name: f.Hostname,
IP: f.IP,
OS: f.Distro,
Kernel: f.Kernel,
Arch: f.Arch,
CPUCores: f.CPUCores,
RAMBytes: f.RAMBytes,
AgentVersion: f.AgentVersion,
Status: f.Status,
LastSeen: f.LastSeen,
UptimeSec: e.uptimeFor(hid),
}
if e.PublicURL != "" {
m.HostURL = strings.TrimRight(e.PublicURL, "/") + "/hosts/" + hid.String()
}
}

func (e *Engine) uptimeFor(id uuid.UUID) int64 {
e.stateMu.Lock()
defer e.stateMu.Unlock()
return e.hostUptime[id]
}

func splitDedupKey(dedup string) (hostArg, monitorArg any) {
idx := strings.IndexByte(dedup, ':')
if idx < 0 || idx == len(dedup)-1 {
Expand Down
Loading
Loading