Skip to content

feat: add incident investigation skill for lineage-driven root cause analysis - #133

Open
alejandro-publius wants to merge 1 commit into
datahub-project:mainfrom
alejandro-publius:feat/incident-investigation-skill
Open

feat: add incident investigation skill for lineage-driven root cause analysis#133
alejandro-publius wants to merge 1 commit into
datahub-project:mainfrom
alejandro-publius:feat/incident-investigation-skill

Conversation

@alejandro-publius

Copy link
Copy Markdown

Summary

Adds datahub-incident-investigation, a sixth catalog interaction skill that takes a reported data symptom and drives it to an evidence-backed root cause, a scoped remediation, and a resolved incident.

It is deliberately not an overlap with datahub-quality. Quality detects and records — assertions, health filters, incident CRUD. Investigation explains and resolves:

  • Symptom framing — refuses to start without an observable, a magnitude, and an onset, because onset is what eliminates most candidate causes for free
  • Lineage localization — upstream and column-level traversal to build a suspect set, plus lineage path to confirm a route; an upstream that does not feed the affected column is eliminated by one command
  • Competing hypotheses — one per upstream branch across seven classes, always including "this is not an incident", tracked in a ledger where eliminations require cited evidence too
  • Necessity and sufficiency — a cause must be confined to the rows it touches and reproduce the symptom's magnitude and onset; explicit distractor taxonomy (too small, mis-dated, off-path, downstream, cosmetic, sympathetic)
  • A five-point confirmation gate — retrieved lineage path, quantitative evidence naming the blamed asset and field, magnitude stated as a number, onset aligned within one refresh interval, and at least one competitor eliminated
  • Semantic failures — the class that passes every type, volume, freshness, and null threshold while every value is wrong; caught by testing data against the documented contract rather than a threshold
  • Blast radius, remediation, verification — downstream traversal with owners, a fix at the layer that owns the defect (never one that filters the bad rows away), and two independent verification checks: symptom gone and the full existing suite still green
  • Writeback — closes the loop with updateIncidentStatus(state: RESOLVED, stage: FIXED) and addLink for the RCA report, then routes prevention work to /datahub-quality and /datahub-enrich

Every CLI and GraphQL recipe in the skill was executed against a live DataHub OSS instance (serverEnv: core, serverType: quickstart, CLI 1.7.0) before submission. Three things that run surfaced and the docs now reflect:

  • health is returned as a list of health entries, not a single object
  • properties.description and editableProperties.description are independent — an agent that reads only one will report a documented field as undocumented
  • there is no top-level incident(urn: ...) query on OSS (FieldUndefined); the working paths are the owning entity's incidents(...) connection or datahub get --urn <incident-urn>

raiseIncident, updateIncidentStatus, and addLink were all verified working on OSS, not just Cloud.

Skill structure

File Purpose
skills/datahub-incident-investigation/SKILL.md Main skill — 9-step workflow: frame → contextualize → localize → hypothesize → collect → eliminate → confirm → remediate → verify + write back
skills/datahub-incident-investigation/references/evidence-standards-reference.md Evidence taxonomy, citation rules, the necessity/sufficiency test, distractor types, the confirmation gate, semantic failure patterns
skills/datahub-incident-investigation/references/investigation-recipes-reference.md Executable CLI + GraphQL recipes for contract retrieval, lineage localization, change history, portable profiling SQL, and incident writeback
skills/datahub-incident-investigation/templates/incident-rca-report.template.md Postmortem report format
skills/datahub-incident-investigation/templates/hypothesis-ledger.template.md Live hypothesis / evidence / confirmation-gate tracking
skills/datahub-incident-investigation/README.md Skill README
commands/catalog-investigate.md Claude Code slash command wrapper

allowed-tools is scoped narrowly and read-only: Bash(datahub *), Bash(git log *), Bash(git diff *), Bash(git blame *), Read, Grep, Glob. The skill deliberately has no warehouse execution path — it emits portable ANSI profiling SQL for the user or their own approved tooling to run, and it never applies a remediation itself.

Integration changes

  • using-datahub/SKILL.md — routing table row added; new "Quality vs. Investigation" and "Lineage vs. Investigation" disambiguation blocks; Critical Rule 7 added
  • datahub-quality — "Not This Skill" row plus two key-boundary bullets separating incident state management from incident diagnosis
  • datahub-lineage — "Not This Skill" row, a note that its root-cause traversal mode is topology only, and a next-step pointer
  • datahub-search, datahub-enrich, datahub-setup — "Not This Skill" rows
  • README.md — skill section, manual-install line, repo-layout tree, commands table, "What works where" row, Contributing → where things live
  • .claude-plugin/plugin.json, .claude-plugin/marketplace.json — description strings only; no version edits

Lint: prettier and markdownlint-cli2 v0.21.0 (167 files, 0 errors) plus the pre-commit-hooks and ruff hooks all pass.

…analysis

Adds `datahub-incident-investigation`, a sixth catalog interaction skill that
takes a reported data symptom and drives it to an evidence-backed root cause,
a scoped remediation, and a resolved incident.

It complements `datahub-quality` rather than overlapping it: quality detects
and records (assertions, health, incident CRUD); investigation explains and
resolves — it traverses upstream and column-level lineage to localize the
fault, eliminates competing hypotheses on necessity and sufficiency, confirms
a cause behind a five-point evidence gate, assesses blast radius, proposes and
verifies a remediation, and closes the incident with
`updateIncidentStatus(state: RESOLVED, stage: FIXED)`.

All CLI and GraphQL recipes were executed against a live DataHub OSS instance
(`serverEnv: core`, CLI 1.7.0) before submission.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant