Add karma as a dashboard over the alert bus - #507
Open
ChrisonSimtian wants to merge 1 commit into
Open
Conversation
Alertmanager's own UI stays the authority — its Status tab shows the routing tree and inhibition rules as Alertmanager actually parsed them, which is what you want when an alert did not arrive. karma is the nicer viewer on the same data: grouped cards, dark mode, readable on a phone. Stateless and uid 0, so no Volume and no UserNS. That last part is per-image, not house style: alertmanager needs keep-id because it runs as 65534, outside this LXC's uid map. Checked karma's image rather than copying either neighbour. Reaches alertmanager by container name over the shared network, so it does not depend on the published :9093 and keeps working if that is ever unpublished. No FILTERS_DEFAULT on purpose. karma already leads with active alerts, and pinning `@state=active` would hide SILENCED ones — the alert you most want to notice, because a forgotten silence is invisible failure rather than noise. Two env vars were wrong on the first pass and would have been accepted and ignored: the port is PORT (not LISTEN_PORT) and filters are FILTERS_DEFAULT. Verified against karma's own docs before shipping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A nicer read-only view onto Alertmanager. One stateless container, no secrets, no data dir. Deployed and verified.
Alertmanager's own UI at
:9093stays the authority — its Status tab shows the routing tree and inhibition rules as Alertmanager actually parsed them, which is what you want when an alert didn't arrive and you need to know which route caught it. karma is the better viewer on the same data: grouped cards, dark mode, readable on a phone.Where to look now
:8080:9093:3000Two decisions worth reviewing
UserNS, and that's per-image rather than house style. karma's image declaresUser: 0, so the default rootless mapping sends container-root to thepodmanuser — same assnmp_exporterandpulse. Alertmanager needskeep-idbecause it runs as 65534, outside this LXC's uid map; copying either neighbour blindly gets you acrun: setgroups: Invalid argumentcrash loop, which is how I found that out on Monitor the UniFi stack, with Alertmanager as the alert bus #501. Checked the image.FILTERS_DEFAULTon purpose. karma already leads with active alerts, and pinning@state=activewould hide silenced ones — precisely the alert you most want to notice, because a forgotten silence is invisible failure rather than noise.It reaches Alertmanager by container name over the shared quadlet network, so it doesn't depend on the published
:9093and keeps working if that's ever unpublished.Caught before shipping
Two env vars were wrong on my first pass and would have been accepted and silently ignored — karma takes
PORT(notLISTEN_PORT) andFILTERS_DEFAULT(notKARMA_DEFAULT_FILTERS). Verified against karma's configuration docs rather than assumed. Same failure family as theEnvironment=whitespace splitting on #500: config that looks right in the file and isn't what the process gets.Verified
karma.serviceactive; UI returns 200;/health→PongUpstream version 0.34.0 alertmanager=homelab/api/v2/alerts, karma wentgroups=0→groups=1, then back to0after resolving it. Alertmanager confirms 0 active alerts now.27/27 shapes valid,Apply summary — 1 applied, 0 failedNote
The smoke test routed through the
home-assistantreceiver, so there'll be a stray "karma dashboard smoke test" notification (and its resolve) in HA's notification list. Harmless, dismissable.