Lumis SDK

Deterministic-first, evidence-grounded incident recovery for data, ML, and software pipelines.

CI · Research paper · Apache-2.0 · Architecture · Configuration · Documentation index · Cookbooks · Roadmap

Important

Experimental research companion: Lumis SDK is under active development and evaluation. Its APIs, adapters, and proposed recovery workflows may change before broader production readiness. It is published so researchers, practitioners, and contributors can inspect, reproduce, test, and improve the work; it is not a claim of autonomous or production-ready remediation.

Lumis SDK is an open-source Python implementation companion to the Agentic Recovery and Incident Response reference architecture proposed in the research paper (arXiv:2608.01955). It provides reusable contracts and local reference adapters for diagnosing failures in data, machine-learning, and software-delivery pipelines while keeping models optional and consequential actions under explicit control.

Lumis SDK starts with Diagnosis-as-Code: bounded incident evidence becomes a structured, reviewable diagnosis, Markdown or JSON report, and operational-memory record. Its direction is Healing-as-Code: a guarded lifecycle for detect, triage, diagnose, plan, approve, remediate, verify, and learn.

Lumis SDK does not perform unrestricted or default production remediation. Current execution-related models are recommendation and verification contracts, not authority granted to an LLM. See research alignment and maturity for the paper-to-SDK boundary and current implementation status.

Research paper

The reference architecture was published on arXiv on 3 August 2026:

Solomon Eshun, Dennis Murage, Sharleen Muoki, Chih-Chun Chen, Stephen Adjignon, Matteo Staar, and Oliver Angélil. “Agentic Self-Healing for Data & AI Pipelines: An Affordable Vendor-Agnostic Architecture using Open-Source Software.” arXiv:2608.01955, 2026. Paper · PDF · DOI

@misc{eshun2026agentic,
  title         = {Agentic Self-Healing for Data \& AI Pipelines: An Affordable
                   Vendor-Agnostic Architecture using Open-Source Software},
  author        = {Eshun, Solomon and Murage, Dennis and Muoki, Sharleen and
                   Chen, Chih-Chun and Adjignon, Stephen and Staar, Matteo and
                   Ang{\'e}lil, Oliver},
  year          = {2026},
  eprint        = {2608.01955},
  archivePrefix = {arXiv},
  primaryClass  = {cs.ET},
  doi           = {10.48550/arXiv.2608.01955}
}

Research and implementation boundary

Artefact	Role	Repository boundary
Reference architecture	Agentic recovery and incident response lifecycle.	Technology-flexible design described by the paper.
Lumis SDK	Apache-2.0 framework and local implementation companion.	Domain contracts, application services, ports, safe reference adapters, CLI, testkit, and cookbooks.

Design principles

• Deterministic first. Known signatures and project rules run before optional model reasoning.
• Evidence grounded. Facts, evidence, hypotheses, confidence, contradictions, and missing evidence remain distinguishable.
• Model optional. The core works offline; provider integrations implement a narrow gateway port.
• Local first. SQLite and Markdown are inspectable defaults, not mandatory hosted services.
• Guarded recovery. Plans are allowlisted recommendations; approval and verification are explicit boundaries.
• Confirmed memory. Model output is never silently promoted into confirmed operational truth.
• Learn toward determinism. Repeated, verified episodes may become reviewable rule candidates; frequency alone never creates or activates a rule.
• Vendor agnostic. Domain and application packages import no observability, orchestration, cloud, or agent SDK.

Architecture

flowchart LR
    subgraph USERS[Project entry points]
        CLI[Lumis SDK CLI]
        PY[Python application]
    end
    subgraph CORE[Lumis SDK framework]
        CFG[Strict project and rule configuration]
        APP[Application services]
        DOMAIN[Domain contracts]
        PORTS[Optional provider ports]
    end
    subgraph LOCAL[Local reference adapters]
        DET[Deterministic diagnosis]
        MEM[SQLite incident memory]
        EVIDENCE[Bounded local JSON evidence]
        REPORT[Markdown and JSON reports]
        REDACT[Evidence redaction]
    end

    CLI --> CFG
    CLI --> DET
    CLI --> EVIDENCE
    CLI --> MEM
    CLI --> REPORT
    PY --> APP
    APP --> DOMAIN
    APP --> PORTS
    APP --> DET
    CFG --> DET
    EVIDENCE --> PORTS
    REDACT --> PORTS

Loading

Canonical package boundaries:

src/lumis_sdk/
├── domain/       # strict vendor-neutral models
├── application/  # use-case orchestration
├── ports/        # replaceable provider interfaces
├── adapters/     # deterministic, SQLite, evidence, report, and local adapters
├── config/       # versioned strict configuration
├── cli/          # command composition
├── security/     # redaction and evidence-safety utilities
└── testkit/      # deterministic test doubles

The proof-of-concept flat modules have been removed. New code imports the explicit domain, application, port, adapter, configuration, and security packages shown above.

Read the architecture overview, SDK reference, and configuration reference. The structured-rules API guide covers compound incident fields, evidence, fixture testing, and migration from all_contains. The evidence and JSON reports guide documents the provider contract, bounded collection behavior, report schema, and reusable testkit. The plugin SDK guide covers static manifests, metadata-only discovery, explicit loading policy, compatibility, and contract testing.

Current capabilities

Capability	Current behavior
Incident input	Local log normalization and typed vendor-neutral incident contracts.
Evidence collection	Async provider port, bounded collection service, safe failures, redaction, and a local JSON reference adapter.
Deterministic diagnosis	Legacy ordered text rules plus structured all/any/not rules with typed comparisons, required evidence, ranking, and candidate explanations.
Versioned configuration	Strict lumis.dev/v1 project and rule-set documents; unknown fields fail validation.
Reports	Deterministic Markdown or versioned JSON with facts, evidence, hypotheses, truth state, confidence, review requirement, and safety boundary.
Local memory	SQLite records, human resolutions, visible truth state, and transparent lexical search.
Model boundary	Explicit policy, budgets, schema-validated output, fake CI gateway, and deterministic fallback.
Guarded lifecycle	Context, policy, approval, verification, and audit ports with no core action executor.
Plugin SDK	Static strict manifests, lazy entry-point discovery, compatibility/support checks, default-denied sensitive authorities, and reusable factory contracts.
CLI	Initialization, diagnosis, doctor, rule validation, reports, resolution, and memory search.
Cookbooks	Synthetic data, ML regression, and software-delivery investigations with optional Agno/OpenRouter paths.

Quick start

Lumis SDK supports Python 3.11+ and uses uv.

git clone https://github.com/soloshun/lumis-sdk.git
cd lumis-sdk
uv sync --all-groups
uv run lumis --help

Install Lumis SDK

Lumis SDK is published to PyPI as lumis-sdk. Add it to a project managed by uv:

uv add "lumis-sdk>=0.1.0rc1,<0.2"

Or install it into an existing environment with pip:

pip install lumis-sdk

For a specific reproducible release, pin the version:

uv add "lumis-sdk==0.0.8"
pip install "lumis-sdk==0.0.8"

The repository's GitHub Actions workflow publishes reviewed releases through PyPI Trusted Publishing.

See the phased roadmap for the stable Python foundation, bounded model/agent contracts, intelligence and integration packages, guarded recovery protocols, TypeScript feasibility, documentation, and community plans.

Run the local deterministic example:

uv run lumis doctor \
  --config cookbook/simple-log-diagnosis/lumis/lumis.yml

uv run lumis diagnose \
  --config cookbook/simple-log-diagnosis/lumis/lumis.yml

The command reads a synthetic local log, writes a Markdown report, saves an unconfirmed incident episode to local SQLite, and prints its incident ID. It makes no network or model call.

uv run lumis report <incident-id> \
  --config cookbook/simple-log-diagnosis/lumis/lumis.yml

uv run lumis resolve <incident-id> \
  --resolution "Human-confirmed cause, action, and outcome." \
  --config cookbook/simple-log-diagnosis/lumis/lumis.yml

uv run lumis memory search "KeyError Close" \
  --config cookbook/simple-log-diagnosis/lumis/lumis.yml

Versioned project configuration

apiVersion: lumis.dev/v1
kind: Project
metadata:
  name: customer-pipeline
spec:
  environment: local
  memory:
    provider: sqlite
    path: .lumis/incidents.db
  reports:
    provider: markdown
    outputDir: .lumis/reports
  incidentSources:
    - provider: local-log
      path: logs/latest-failure.log
  evidenceProviders:
    - provider: local-json
      path: evidence/schema-diff.json
      kinds: [schema-diff]
      maxItems: 20
      maxTotalCharacters: 50000
  rules:
    files: [rules.yml]
  model:
    enabled: false

Configuration is strict: misspelled or unknown fields fail with a validation error. Relative paths resolve from the project document. Files larger than the configured safety limit are rejected. Checked stable schemas for the project, rule set, structured diagnosis rule, and JSON diagnosis report support editors and tooling.

Released v1alpha1 documents remain readable during the documented transition. Use lumis config migrate and the v1 migration guide to validate and upgrade them. Read the configuration reference, public API inventory, and compatibility policy.

CLI

lumis init
lumis doctor
lumis diagnose
lumis report
lumis resolve
lumis memory search
lumis rules validate
lumis rules test
lumis plugins list
lumis plugins doctor
lumis config migrate

doctor and validation commands do not make network calls or write incident state. Model assistance remains disabled unless application code supplies both an enabled policy and a gateway adapter.

Python API

import asyncio
from pathlib import Path

from lumis_sdk.application import DiagnosisService
from lumis_sdk.config import load_config
from lumis_sdk.domain import IncidentInput

config = load_config(Path("lumis.yml"))
service = DiagnosisService(rules=config.rules)
incident = IncidentInput(
    source_tool="local-log",
    pipeline_name=config.project,
    raw_payload={"log": "ERROR KeyError: Close"},
)
diagnosis = asyncio.run(service.diagnose(incident))

Cookbooks

• Simple local diagnosis
• Data pipeline investigation
• ML regression monitoring
• Software-delivery CI investigation
• Structured rule evaluation
• Evidence collection and JSON reporting
• Recording a human resolution

Start with a cookbook for a runnable demonstration, then use the architecture and core references above to examine the framework contracts behind it. All examples are synthetic, executable research demonstrations: they show how a consuming application can use Lumis SDK without claiming to be production control planes or autonomous remediation systems. Agent frameworks and model providers remain cookbook-only optional dependencies.

Safety

Lumis SDK treats logs, tickets, runbooks, source files, and model output as untrusted input.

• No direct shell, cloud-admin, Kubernetes-admin, or database actuation in core.
• No live model key or billable request in CI.
• No telemetry export by default.
• Bounded configuration and log reads.
• Conservative redaction before optional model use.
• Model output remains an unconfirmed hypothesis until a human or verifier confirms it.
• Execution capability requires a future RFC, allowlisted typed actions, policy, approval, audit, limits, and verification.

Read the threat model and security policy.

Development

uv sync --all-groups
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run python scripts/generate_config_schema.py --check
uv run pytest
uv build

See CONTRIBUTING.md, GOVERNANCE.md, SUPPORT.md, CHANGELOG.md, and ROADMAP.md.

Releases

Lumis SDK releases are manually dispatched through GitHub Actions and published with PyPI Trusted Publishing.

Research and standards context

Lumis SDK is informed by OpenTelemetry, OpenLineage, Prometheus, Site Reliability Engineering, ReAct, and LLM-based incident RCA research. These are design influences, not mandatory dependencies or claims of conformance.

Maintainer and license

Lumis SDK is currently maintained by Solomon Eshun and licensed under Apache License 2.0.