From 884d16cbfde6eed4dcae4266dddb0059e04b3a39 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 13 Jul 2026 19:54:01 +0000 Subject: [PATCH] feat: live tail (SSE), incident memory, demo mode, Cisco/Nokia patterns, confidence (v0.5.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Research-driven wave — competitive study showed SSE-first live viewing and incident-history recall are what make NOC tools sticky: - Live tail: GET /api/tail/ streams classified events from syslog listener sources via Server-Sent Events (zero deps, no polling, min_severity filter, keepalives, query-token auth) + Live Tail UI panel. Cursor-based fetch_new() on the syslog source: monotonic ingest counter, duplicate-free, ring-wraparound safe. - Incident memory (memory.py, AI_LOG_ANALYZER_INCIDENT_STORE): bounded local JSONL journal; action items gain 'seen Nx before (last date)' recurrence badges; /api/incidents/similar?q= token-overlap search. - Demo mode: ai-log-analyzer demo — deterministic 6-device 4-dialect synthetic incident storyline through the full pipeline; --serve starts the UI with a UDP feeder so Live Tail streams the story live. - Vendor coverage: IOS-XE, NX-OS, SR Linux patterns mapped onto canonical descriptions so stability pairing + KB work unchanged. - Per-event confidence (1.0 custom / 0.9 KB / 0.6 promoted / 0.3 unmatched). - Literal-gate extractor now handles sre common-prefix hoisting: unguarded patterns 2 -> 0. Suite 338 -> 369 tests. Version 0.4.0 -> 0.5.0. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_015XM7j9iDXYwSvjnxZYjzaq --- .env.example | 6 + CHANGELOG.md | 44 ++++++ README.md | 23 +++- pyproject.toml | 2 +- src/ai_log_analyzer/analyzer.py | 18 ++- src/ai_log_analyzer/classifier.py | 34 ++++- src/ai_log_analyzer/cli.py | 57 ++++++++ src/ai_log_analyzer/demo.py | 107 +++++++++++++++ src/ai_log_analyzer/memory.py | 157 ++++++++++++++++++++++ src/ai_log_analyzer/sources/syslog.py | 18 +++ src/ai_log_analyzer/web/app.py | 75 +++++++++++ src/ai_log_analyzer/web/static/app.js | 72 +++++++++- src/ai_log_analyzer/web/static/index.html | 34 +++++ tests/test_demo.py | 54 ++++++++ tests/test_incident_memory.py | 92 +++++++++++++ tests/test_live_tail.py | 118 ++++++++++++++++ tests/test_vendor_patterns.py | 115 ++++++++++++++++ 17 files changed, 1016 insertions(+), 10 deletions(-) create mode 100644 src/ai_log_analyzer/demo.py create mode 100644 src/ai_log_analyzer/memory.py create mode 100644 tests/test_demo.py create mode 100644 tests/test_incident_memory.py create mode 100644 tests/test_live_tail.py create mode 100644 tests/test_vendor_patterns.py diff --git a/.env.example b/.env.example index 69dc43b..6abf816 100644 --- a/.env.example +++ b/.env.example @@ -45,6 +45,12 @@ ANTHROPIC_MODEL=claude-haiku-4-5-20251001 # Unknown Patterns panel and counted as new_template_count in /api/analyze. #AI_LOG_ANALYZER_TEMPLATE_STORE=~/.cache/netlog-ai/templates.json +# ── Incident memory (optional) ──────────────────────────────────────────────── +# JSONL journal of past analysis runs. When set, action items carry a +# "seen N× before" recurrence block and /api/incidents/similar?q= searches +# the history. Local-only, bounded, never leaves the host. +#AI_LOG_ANALYZER_INCIDENT_STORE=~/.cache/netlog-ai/incidents.jsonl + # ── Custom classification rules (optional) ──────────────────────────────────── # JSON file with a list of {"pattern", "severity", "category", "description"} # objects. Loaded at startup; rules take precedence over the built-in KB. diff --git a/CHANGELOG.md b/CHANGELOG.md index 82eecd2..38bf728 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,50 @@ All notable changes to **netlog-ai** are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/); this project uses loose semantic versioning. +## [0.5.0] - 2026-07-13 + +Research-driven wave (competitive study of 2026 NOC tooling — SSE-first live +viewers, incident-history UX, Cisco/Nokia syslog references). + +### Added + +- **🔴 Live tail** — `GET /api/tail/` streams classified events + from any syslog listener source via Server-Sent Events (zero new deps, no + polling, server-side `?min_severity=` filter, keepalive frames, query-token + auth for EventSource). New Live Tail panel in the Logs tab with start/stop, + source picker, and a rolling 200-event view. Backed by a new cursor-based + `fetch_new()` on the syslog source (monotonic ingest counter — no + duplicates, wraparound-safe). +- **↻ Incident memory** (`memory.py`) — with `AI_LOG_ANALYZER_INCIDENT_STORE` + set, every run journals its action items locally (bounded JSONL); future + runs annotate items with `recurrence` ("seen 3× before, last Jul 5, on + spine-01") shown as a badge in the UI, and `/api/incidents/similar?q=` + answers free-text "have we seen this before?" via token-overlap search. +- **🎬 Demo mode** — `ai-log-analyzer demo` runs a deterministic synthetic + incident storyline (6 devices, 4 vendor dialects: flapping interface, BGP + collapse, NX-OS service crash, PSU failure, SSH brute force, SR Linux + churn, one never-seen shape) through the full pipeline and prints the + result; `--serve` starts the UI with a UDP feeder so Live Tail streams the + story in real time. Zero setup, zero LLM key needed. +- **Cisco IOS-XE / NX-OS / Nokia SR Linux classifier patterns** — + `%LINK-3-UPDOWN`, `%LINEPROTO-5-UPDOWN`, `%ETHPORT-5-IF_DOWN`, + `%SYSMGR-2-SERVICE_CRASHED`, `%VPC-2-*`, `%MODULE-2/%PLATFORM-2`, + `%DUAL-5-NBRCHANGE` (EIGRP), `%HSRP-5-STATECHANGE`, `%ENVMON`, + `%SEC_LOGIN-4/5`, `%SYS-5-CONFIG_I`, and SR Linux `bgp_mgr` session / + oper-state transitions — mapped onto the existing canonical descriptions so + stability flap-pairing, recovery filtering, and the KB all work unchanged. +- **Per-event confidence score** — every classified event now carries + `confidence`: 1.0 custom rule, 0.9 KB pattern, 0.6 raw-severity promotion, + 0.3 unmatched snippet. + +### Performance + +- Literal-gate extraction now recovers guaranteed literals from alternations + whose common prefix sre hoists into a short run (`%a|%b` → `%` + branch) — + unguarded always-scanned patterns drop from 2 to **0**. + +Suite 338 → 369 tests. + ## [0.4.0] - 2026-07-13 ### Added diff --git a/README.md b/README.md index 812266d..f29118d 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ > **Network logs in. Ranked actions out.** A local, dark-themed dashboard that classifies syslog events from any vendor (Junos, Arista EOS, FRR), builds a prioritized action list, and lets an LLM write the root-cause analysis with copy-pastable CLI fixes. -[![CI](https://github.com/gesh75/netlog-ai/actions/workflows/ci.yml/badge.svg)](https://github.com/gesh75/netlog-ai/actions/workflows/ci.yml) ![Tests](https://img.shields.io/badge/tests-338%20passing-brightgreen) ![License](https://img.shields.io/badge/license-MIT-blue) ![Python](https://img.shields.io/badge/python-3.10%2B-blue) ![Stack](https://img.shields.io/badge/stack-Flask%20%2B%20vanilla%20JS-1f6feb) +[![CI](https://github.com/gesh75/netlog-ai/actions/workflows/ci.yml/badge.svg)](https://github.com/gesh75/netlog-ai/actions/workflows/ci.yml) ![Tests](https://img.shields.io/badge/tests-369%20passing-brightgreen) ![License](https://img.shields.io/badge/license-MIT-blue) ![Python](https://img.shields.io/badge/python-3.10%2B-blue) ![Stack](https://img.shields.io/badge/stack-Flask%20%2B%20vanilla%20JS-1f6feb) > 📓 Recent changes — cross-source correlation + per-device triage (MCP tools **and** Device-tab UI) — are in [`CHANGELOG.md`](CHANGELOG.md). @@ -66,7 +66,10 @@ Tools exposed: `list_sources`, `add_source`, `fetch_logs`, `search_logs`, | 🤖 **MCP server mode** | Claude Code / Cursor / Continue can call the analyzer directly as agent tools | | 🔗 **Cross-source correlation** | **Device tab** → *Correlate Sources*: scans every registered source and tags each host `confirmed` (flagged by ≥ 2 sources) or `suspected` (1) in a sortable, severity-coded table | | 🔬 **Per-device triage** | **Device tab** → *Triage Device*: one host's verdict + 0–100 health score, severity histogram, top processes, and deduped error patterns in a single panel | -| 🔎 **Classify** | 50+ regex patterns across Junos, EOS, FRR, IOS, RFC-3164/5424 — plus your own rules via `AI_LOG_ANALYZER_CUSTOM_RULES` / `POST /api/rules` | +| 🔎 **Classify** | 70+ regex patterns across Junos, EOS, FRR, **Cisco IOS-XE, NX-OS, Nokia SR Linux**, RFC-3164/5424 — each event carries a confidence score; add your own rules via `AI_LOG_ANALYZER_CUSTOM_RULES` / `POST /api/rules` | +| 🔴 **Live tail** | Real-time SSE stream of classified events from any syslog listener source — zero polling, zero build step, server-side severity filter, its own UI panel | +| ↻ **Incident memory** | `AI_LOG_ANALYZER_INCIDENT_STORE` journals every run locally; action items get "seen 3× before (last Jul 5)" recurrence badges and `/api/incidents/similar?q=` answers "have we seen this before?" | +| 🎬 **Demo mode** | `ai-log-analyzer demo` — zero-setup synthetic incident storyline through the full pipeline; `--serve` adds a live feeder so Live Tail streams it in real time | | 🔭 **Unknown patterns** | Lines no rule matched are template-mined (Drain-style, zero deps): variables masked, shapes clustered, error-smelling templates ranked first — novel failure modes surface instead of vanishing as `info` noise. Set `AI_LOG_ANALYZER_TEMPLATE_STORE` for cross-run memory: shapes never seen in any prior run get flagged 🆕 | | 🧭 **Prioritize** | Deduped action items, ranked by severity × count, recovery events excluded | | 📶 **Fabric stability** | Per-device flap detection (interface/BGP/OSPF/LAG/VPN down↔up oscillation), event-rate bursts vs the device's own baseline, trend + heuristic 24h risk band — in `/api/analyze` and its own UI panel | @@ -113,6 +116,10 @@ ai-log-analyzer eval # heuristic, offline ai-log-analyzer eval --use-llm # blend a real LLM judge ai-log-analyzer eval --file result.json --min-score 7 # CI gate +# Zero-setup demo (synthetic incident storyline; --serve adds live streaming) +ai-log-analyzer demo +ai-log-analyzer demo --serve + # Run the full test suite pytest --cov=src --cov-report=term-missing ``` @@ -328,6 +335,8 @@ src/ai_log_analyzer/ patterns.py Drain-lite template miner for lines no rule matched stability.py Flap/burst/trend scoring + 24h risk per device judge.py LLM-as-Judge playbook quality scoring (eval CLI) + memory.py Incident journal — recurrence badges + similarity search + demo.py Synthetic incident storyline + live feeder (demo CLI) kb.py Rule-based deep-analysis KB (fallback when LLM is off) llm.py Docker Model Runner (TCP + UDS) + Anthropic Claude analyzer.py End-to-end pipeline: classify → actions → score → summary @@ -360,6 +369,8 @@ src/ai_log_analyzer/ | `GET` | `/api/sites` | List bundled site bundles | | `POST` | `/api/analyze` | Full pipeline — see request shapes below (response includes `unknown_patterns`: mined templates of unmatched lines) | | `GET` | `/api/rules` | Built-in rule count + registered custom classification rules | +| `GET` | `/api/tail/` | SSE live stream of classified events (`?min_severity=`, `?token=` on tokened deployments) | +| `GET` | `/api/incidents/similar` | Free-text search over the local incident journal (`?q=`, needs `AI_LOG_ANALYZER_INCIDENT_STORE`) | | `POST` | `/api/rules` | Add custom rules at runtime — `{"rules": [{"pattern", "severity", "category", "description"}]}` | | `POST` | `/api/optimize` | Device-level config audit + patches | | `POST` | `/api/optimize/site` | Cross-device site analysis | @@ -407,7 +418,7 @@ src/ai_log_analyzer/ ## Tested & accessible -- **338 unit + integration tests** (pytest) +- **369 unit + integration tests** (pytest) - Frontend audited across 8 review rounds: - WCAG-AA: `:focus-visible` rings, `aria-live` regions, `role=tablist/tab/tabpanel`, skip-to-main link, `prefers-reduced-motion` fallback - Responsive ≤ 1100px, PWA-ready (`theme-color`, `mobile-web-app-capable`, SVG favicon) @@ -418,9 +429,11 @@ src/ai_log_analyzer/ ## Roadmap - [ ] Multi-site comparison view (delta between two sites) -- [ ] Real-time tail mode (websocket stream of new events) +- [x] Real-time tail mode — SSE stream (`/api/tail/`) + Live Tail UI panel - [x] Slack / generic-JSON alert webhooks per severity threshold (`AI_LOG_ANALYZER_WEBHOOK_URL` — see `.env.example`) -- [ ] More vendor adapters (Nokia SR Linux, Cisco IOS-XE, Cumulus NCLU) +- [x] More vendor coverage — Cisco IOS-XE + NX-OS + Nokia SR Linux classifier patterns (Cumulus NCLU still open) +- [x] Incident memory — local journal with recurrence badges + `/api/incidents/similar` +- [x] One-command demo mode (`ai-log-analyzer demo [--serve]`) - [ ] Snapshot / replay analysis runs for regression testing - [x] Custom classification rules — JSON file (`AI_LOG_ANALYZER_CUSTOM_RULES`) + runtime `POST /api/rules`; UI editor still open - [x] Unknown-pattern template mining — Drain-style clustering of unmatched lines, surfaced in UI/API/MCP diff --git a/pyproject.toml b/pyproject.toml index d8e2784..b693bea 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "netlog-ai" -version = "0.4.0" +version = "0.5.0" description = "AI-powered network syslog analyzer with pluggable LLM providers (local Docker Model Runner / Anthropic Claude) and lab adapters." readme = "README.md" license = { text = "MIT" } diff --git a/src/ai_log_analyzer/analyzer.py b/src/ai_log_analyzer/analyzer.py index 15b659f..11b7f9a 100644 --- a/src/ai_log_analyzer/analyzer.py +++ b/src/ai_log_analyzer/analyzer.py @@ -65,9 +65,10 @@ class ActionItem: devices: list[str] sample_messages: list[str] deep_analysis: dict[str, Any] = field(default_factory=dict) + recurrence: dict[str, Any] | None = None # history from the incident store def to_dict(self) -> dict: - return { + d = { "severity": self.severity, "category": self.category, "description": self.description, @@ -76,6 +77,9 @@ def to_dict(self) -> dict: "sample_messages": self.sample_messages, "deep_analysis": self.deep_analysis, } + if self.recurrence: + d["recurrence"] = self.recurrence + return d @dataclass @@ -590,6 +594,16 @@ def analyze(events: Iterable[LogEvent], use_llm: bool = True, llm_top_n: int = 3 any_llm = summary_llm or any(a.deep_analysis.get("llm_powered") for a in action_items) top_devices = _finalize_top_devices(by_host) + generated_at = datetime.now(timezone.utc).isoformat(timespec="seconds") + + # Opt-in incident memory: stamp each action item with its recurrence + # history from prior runs, then journal this run (env-gated, local-only). + from ai_log_analyzer.memory import get_store + store = get_store() + if store is not None: + for a in action_items: + a.recurrence = store.recurrence(a.description) + store.record([a.to_dict() for a in action_items], generated_at) return AnalysisResult( score=score, grade=grade, grade_label=grade_label, @@ -598,7 +612,7 @@ def analyze(events: Iterable[LogEvent], use_llm: bool = True, llm_top_n: int = 3 classified_events=top_events, executive_summary=summary, llm_powered=any_llm, - generated_at=datetime.now(timezone.utc).isoformat(timespec="seconds"), + generated_at=generated_at, unknown_patterns=miner.to_dict(top_n=10), stability=tracker.report(), ) diff --git a/src/ai_log_analyzer/classifier.py b/src/ai_log_analyzer/classifier.py index 19aa0e1..0816889 100644 --- a/src/ai_log_analyzer/classifier.py +++ b/src/ai_log_analyzer/classifier.py @@ -28,6 +28,18 @@ (r"%bgp-3-notification|%bgp-5-adjchange.*down", "high", "routing", "BGP notification / adjacency down (FRR/IOS)"), (r"ospf.*(?:neighbor.*down|adj.*change|dead.*timer)", "high", "routing", "OSPF neighbor state change"), (r"%ospf-5-adjchg.*down|ospf.*from\s+full\s+to", "high", "routing", "OSPF adjacency loss (FRR/IOS)"), + # Cisco IOS-XE / NX-OS (%FACILITY-SEV-MNEMONIC) + Nokia SR Linux + (r"%dual-5-nbrchange.*down", "high", "routing", "EIGRP neighbor down (IOS)"), + (r"bgp_mgr.*(?:to\s+(?:idle|active|connect)\b|session.*(?:down|fail))", "high", "routing", "BGP peer down / connect failure"), + (r"%(?:link-3-updown|lineproto-5-updown).*to\s+down", "high", "interface", "Interface link down"), + (r"%ethport-5-if_down|%eth_port_channel.*down", "high", "interface", "Interface link down"), + (r"\w+_mgr.*oper.*state.*down|srlinux.*port.*down", "high", "interface", "Interface link down"), + (r"%sec_login-4-login_failed|%authpriv.*fail", "high", "security", "Authentication failure"), + (r"%sysmgr-2-service_crashed|%sysmgr.*(?:crash|terminated)", "critical", "system", "NX-OS service crash"), + (r"%vpc-2-|vpc.*peer.*(?:down|fail)", "high", "lag", "vPC peer failure (NX-OS)"), + (r"%hsrp-5-statechange|%fhrp", "high", "redundancy", "VRRP/gateway failover"), + (r"%envmon.*(?:fan|temperature|supply)|%platform.*(?:thermal|power)", "critical", "hardware", "Power supply or fan failure"), + (r"%module-2-|%platform-2-", "critical", "hardware", "Chassis alarm triggered"), (r"bfd.*(?:down|removed|session.*fail)", "high", "routing", "BFD session down"), (r"isis.*(?:adj.*down|adj.*change|lsp.*purge)", "high", "routing", "IS-IS adjacency change"), (r"license.*(?:expir|invalid|warn)|expir.*license", "high", "compliance", "License expiration warning"), @@ -43,6 +55,7 @@ (r"l3_entry|l2_entry|memory\s+block|blk:", "high", "hardware", "ASIC forwarding table error"), # ── MEDIUM ─────────────────────────────────────────────────────────── (r"snmp_trap_link_up|link\s*up|carrier.*up|if_up", "medium", "interface", "Interface link up"), + (r"%(?:link-3-updown|lineproto-5-updown).*to\s+up", "medium", "interface", "Interface link up"), (r"rpd_bgp.*establ|bgp.*established|%bgp-5-adjchange.*up", "medium", "routing", "BGP peer established"), (r"ospf.*(?:neighbor.*full|adj.*full)|%ospf-5-adjchg.*full", "medium", "routing", "OSPF neighbor established"), (r"ntp.*(?:unreachable|stratum.*16|no.*server|sync\s+lost)", "medium", "ntp", "NTP sync lost/unreachable"), @@ -61,6 +74,8 @@ (r"accepted\s+(?:publickey|password|keyboard)|sshd.*accept", "low", "auth", "SSH login accepted"), (r"session\s+(?:opened|closed)|pam_unix.*session", "low", "auth", "User session opened/closed"), (r"commit.*confirmed|config.*change|commit\b", "low", "config", "Configuration change committed"), + (r"%sys-5-config_i|%vshd-5-vshd_syslog_config_i", "low", "config", "Configuration change committed"), + (r"%sec_login-5-login_success", "low", "auth", "SSH login accepted"), (r"lldp.*(?:neighbor|add|delete|update)", "low", "discovery", "LLDP neighbor change"), (r"snmpd|agentx|snmp.*trap", "low", "monitoring", "SNMP daemon/trap activity"), (r"sshd.*(?:disconnect|close|exit)", "low", "auth", "SSH session disconnected"), @@ -136,9 +151,17 @@ def _prefer(cur: set[str] | None, new: set[str] | None) -> set[str] | None: # IN / ANY / AT / NOT_LITERAL etc. prove nothing — skip _flush() - solid = [r for r in runs if len(r) >= 3] or runs + # Prefer a solid (≥3 char) literal run; else fall back to branch-derived + # literals (sre hoists common alternation prefixes like '%' into a short + # run, which would otherwise mask the long per-alternative literals); + # else settle for the short run. + solid = [r for r in runs if len(r) >= 3] if solid: return {max(solid, key=len)} + if best and all(len(b) >= 3 for b in best): + return best + if runs: + return {max(runs, key=len)} return best @@ -272,6 +295,9 @@ class ClassifiedEvent: action: str message: str sample_message: str = field(default="") # short version for UI + # How the verdict was reached: 1.0 custom rule, 0.9 KB pattern, + # 0.6 raw-severity promotion, 0.3 unmatched (description = raw snippet). + confidence: float = field(default=0.9) def to_dict(self) -> dict: return { @@ -285,6 +311,7 @@ def to_dict(self) -> dict: "action": self.action, "message": self.message, "sample_message": self.sample_message, + "confidence": self.confidence, } @@ -324,11 +351,13 @@ def iter_classify(events: Iterable[LogEvent]) -> Iterator[ClassifiedEvent]: clean_message = strip_ansi(ev.message) combined = f"{ev.appname} {clean_message}".lower() sev, cat, desc = "info", "other", "" + confidence = 0.9 # Custom user rules take precedence over the built-in KB. for pattern, p_sev, p_cat, p_desc in _CUSTOM_RULES: if pattern.search(combined): sev, cat, desc = p_sev, p_cat, p_desc + confidence = 1.0 break if not desc: @@ -346,10 +375,12 @@ def iter_classify(events: Iterable[LogEvent]) -> Iterator[ClassifiedEvent]: if not desc: snippet = clean_message[:120].strip() or ev.appname or "General log message" desc = snippet + confidence = 0.3 # Promote raw severity if syslog says critical but classifier missed it if ev.severity_raw.lower() in ("crit", "emerg", "alert") and sev == "info": sev = "high" + confidence = 0.6 yield ClassifiedEvent( timestamp=ev.timestamp, @@ -362,6 +393,7 @@ def iter_classify(events: Iterable[LogEvent]) -> Iterator[ClassifiedEvent]: action=_action_for(sev, cat), message=clean_message[:500], sample_message=clean_message[:200], + confidence=confidence, ) diff --git a/src/ai_log_analyzer/cli.py b/src/ai_log_analyzer/cli.py index 458284a..21d9c55 100644 --- a/src/ai_log_analyzer/cli.py +++ b/src/ai_log_analyzer/cli.py @@ -48,6 +48,58 @@ def cmd_containers(_args: argparse.Namespace) -> int: return 0 +def cmd_demo(args: argparse.Namespace) -> int: + """Zero-setup demo: analyze a deterministic synthetic incident storyline. + + --serve additionally starts the web UI with a local syslog listener and a + feeder thread replaying the storyline, so the Live Tail panel streams + events in real time — no lab or devices needed. + """ + import os + + from ai_log_analyzer.adapters.file import parse_lines + from ai_log_analyzer.demo import generate_demo_lines, start_udp_feeder + + lines = generate_demo_lines() + result = analyze(parse_lines(lines), use_llm=args.llm) + + d = result.to_dict() + print(f"Demo storyline : {len(lines)} synthetic log lines, 6 devices, 4 vendor dialects") + print(f"Health score : {d['score']}/100 (grade {d['grade']} — {d['grade_label']})") + print(f"Fabric stability: {d['stability']['fabric_score']}/100") + print() + print("Executive summary:") + for b in d["executive_summary"]: + print(f" • {b}") + print() + print("Top action items:") + for a in d["action_items"][:5]: + devs = ", ".join(a["devices"][:3]) + print(f" [{a['severity'].upper():8}] {a['description']} ({a['count']}× on {devs})") + up = d["unknown_patterns"] + if up["top_templates"]: + print() + print(f"Unknown patterns ({up['template_count']} templates from {up['total_unclassified']} unmatched lines):") + for t in up["top_templates"][:3]: + flag = "⚠ " if t["severity_hint"] != "info" else " " + print(f" {flag}{t['count']:>3}× {t['template'][:90]}") + + if not args.serve: + print("\nRun `ai-log-analyzer demo --serve` to watch this stream live in the UI.") + return 0 + + port = int(os.environ.get("NETLOG_DEMO_SYSLOG_PORT", "5514")) + os.environ.setdefault("NETLOG_SOURCE_demo_TYPE", "syslog") + os.environ.setdefault("NETLOG_SOURCE_demo_URL", "udp://127.0.0.1") + os.environ.setdefault("NETLOG_SOURCE_demo_PORT", str(port)) + os.environ.setdefault("NETLOG_SOURCE_demo_BIND", "127.0.0.1") + start_udp_feeder(port) + print(f"\nDemo feeder streaming to udp://127.0.0.1:{port} — open the UI,") + print("expand '🔴 Live Tail' in the Logs tab, pick source 'demo', press Start.") + serve_main() + return 0 + + def cmd_eval(args: argparse.Namespace) -> int: """Score playbook quality with the LLM-as-Judge harness. @@ -125,6 +177,11 @@ def main() -> None: cp = sub.add_parser("containers", help="List running FRR lab containers") cp.set_defaults(func=cmd_containers) + dp = sub.add_parser("demo", help="Zero-setup demo: synthetic incident storyline through the full pipeline") + dp.add_argument("--serve", action="store_true", help="Also start the web UI with a live syslog feeder (Live Tail demo)") + dp.add_argument("--llm", action="store_true", help="Use the configured LLM (default: rule-based KB, fully offline)") + dp.set_defaults(func=cmd_demo) + ep = sub.add_parser("eval", help="Score playbook quality (LLM-as-Judge; KB self-test by default)") ep.add_argument("--file", help="Saved /api/analyze JSON result to score (default: score the built-in KB)") ep.add_argument("--use-llm", action="store_true", help="Blend in a real LLM judge (falls back to heuristic)") diff --git a/src/ai_log_analyzer/demo.py b/src/ai_log_analyzer/demo.py new file mode 100644 index 0000000..21b0563 --- /dev/null +++ b/src/ai_log_analyzer/demo.py @@ -0,0 +1,107 @@ +"""One-command demo — synthetic multi-vendor syslog with a built-in storyline. + +`ai-log-analyzer demo` needs no lab, no devices, no LLM key: it generates a +deterministic incident narrative across six devices and four vendor dialects, +runs the full analysis pipeline on it, and prints the result. With `--serve` +it also starts the web UI with a live syslog feeder, so the Live Tail panel +streams the same story in real time. + +The storyline is designed to light up every analysis surface: + * a flapping access interface on an EOS leaf → stability engine + * a BGP session collapse on a Junos spine → action items + playbooks + * an NX-OS service crash + PSU failure → critical hardware/system + * an SSH brute-force burst on the firewall → security + * SR Linux BGP session churn → vendor coverage + * a never-seen "quantum optics" appdaemon shape → unknown-pattern miner +""" +from __future__ import annotations + +import itertools +from collections.abc import Iterator + +_DEVICES = { + "spine-01": "junos", # MX — BGP collapse + "spine-02": "nxos", # Nexus — service crash + PSU + "leaf-11": "eos", # Arista — interface flapping + "leaf-12": "srl", # SR Linux — BGP churn + "edge-fw-01": "junos", # SRX — auth failures + "core-rt-01": "ios", # IOS-XE — EIGRP + config +} + + +def _ts(minute: int, second: int = 0) -> str: + return f"Jan 14 09:{minute:02d}:{second:02d}" + + +def generate_demo_lines() -> list[str]: + """Deterministic synthetic syslog — same story every run (~180 lines).""" + lines: list[str] = [] + add = lines.append + + # Baseline chatter (all healthy) + for m in range(0, 8): + add(f"{_ts(m, 5)} core-rt-01 %SYS-5-CONFIG_I: Configured from console by ops on vty0 (198.51.100.7)") + add(f"{_ts(m, 20)} spine-01 sshd[112{m}]: Accepted publickey for netops from 198.51.100.7 port 5{m}122 ssh2") + add(f"{_ts(m, 40)} leaf-11 Ebra: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet12, changed state to up") + + # Act 1 — leaf-11 interface starts flapping (minutes 8-14, 3 full cycles) + for i, m in enumerate(range(8, 14)): + state = "down" if i % 2 == 0 else "up" + add(f"{_ts(m, 10)} leaf-11 Ebra: %LINK-3-UPDOWN: Interface Ethernet49/1, changed state to {state}") + + # Act 2 — spine-01 BGP session collapses under the flap (minutes 12-18) + for m in range(12, 18): + add(f"{_ts(m, 30)} spine-01 rpd[1451]: bgp_connect_failed: peer 10.255.1.11 (External AS 65011): hold timer expired") + add(f"{_ts(m, 45)} spine-01 rpd[1451]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 10.255.1.11 state Idle") + + # Act 3 — spine-02 platform trouble (minute 15) + add(f"{_ts(15, 2)} spine-02 %SYSMGR-2-SERVICE_CRASHED: Service \"bgp\" (PID 4321) hasn't caught signal 11") + add(f"{_ts(15, 8)} spine-02 %PLATFORM-2-PS_FAIL: Power supply 1 failed or shutdown") + add(f"{_ts(15, 30)} spine-02 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/49 is down (Link failure)") + + # Act 4 — brute force against the firewall (minutes 16-19) + for m in range(16, 20): + for s in (5, 25, 45): + add(f"{_ts(m, s)} edge-fw-01 sshd[9911]: authentication failure for root from 203.0.113.66") + + # Act 5 — SR Linux BGP churn on leaf-12 (minutes 18-21) + for i, m in enumerate(range(18, 22)): + state = "idle" if i % 2 == 0 else "established" + add(f"{_ts(m, 15)} leaf-12 sr_bgp_mgr: Peer 10.255.1.1: session state changed from established to {state}") + + # Act 6 — EIGRP wobble + recovery on core (minutes 20-23) + add(f"{_ts(20, 40)} core-rt-01 %DUAL-5-NBRCHANGE: EIGRP-IPv4 100: Neighbor 10.0.12.2 (Gi0/1) is down: holding time expired") + add(f"{_ts(22, 10)} core-rt-01 %DUAL-5-NBRCHANGE: EIGRP-IPv4 100: Neighbor 10.0.12.2 (Gi0/1) is up: new adjacency") + + # Act 7 — a message shape no KB rule knows (unknown-pattern miner food) + for m in range(9, 21, 3): + add(f"{_ts(m, 55)} spine-01 appdaemon[77]: quantum optics calibration drift {m} exceeded budget on lane {m % 4}") + + # Recovery tail (minutes 22-25) + add(f"{_ts(23, 5)} leaf-11 Ebra: %LINK-3-UPDOWN: Interface Ethernet49/1, changed state to up") + add(f"{_ts(23, 30)} spine-01 rpd[1451]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 10.255.1.11 state Established") + return lines + + +def iter_feeder_lines() -> Iterator[str]: + """Endless replay of the storyline — used by `demo --serve` to keep the + Live Tail panel busy.""" + return itertools.cycle(generate_demo_lines()) + + +def start_udp_feeder(port: int, interval: float = 0.7) -> None: + """Daemon thread that pumps storyline lines at the local syslog listener.""" + import socket + import threading + import time + + def _run() -> None: + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + for line in iter_feeder_lines(): + try: + sock.sendto(f"<22>{line}".encode(), ("127.0.0.1", port)) + except OSError: + break + time.sleep(interval) + + threading.Thread(target=_run, name="demo-feeder", daemon=True).start() diff --git a/src/ai_log_analyzer/memory.py b/src/ai_log_analyzer/memory.py new file mode 100644 index 0000000..daa8b9f --- /dev/null +++ b/src/ai_log_analyzer/memory.py @@ -0,0 +1,157 @@ +"""Incident memory — "this exact issue happened before, here's when and where." + +Every analysis is ephemeral by default: close the tab and the incident is +gone. With `AI_LOG_ANALYZER_INCIDENT_STORE` set to a path, each analyze run +appends its action items to a local JSONL journal, and future runs annotate +every action item with its recurrence history: + + "recurrence": {"count": 3, "last_seen": "...", "devices": ["leaf-11"]} + +That one line changes triage: a first-ever BGP flap and the fourth BGP flap +this month on the same leaf are different incidents, and now they read +differently. A token-overlap search (`find_similar`) additionally powers the +`/api/incidents/similar` endpoint for free-text lookups ("have we seen fan +failures on spine-02?"). Zero dependencies, bounded file size, and nothing +ever leaves the host. +""" +from __future__ import annotations + +import json +import logging +import os +import re +from pathlib import Path + +logger = logging.getLogger(__name__) + +_MAX_RECORDS = 5000 # journal is rewritten (oldest dropped) past this +_TOKEN_RE = re.compile(r"[a-z0-9][a-z0-9._/-]{2,}") + + +def _tokens(text: str) -> set[str]: + return set(_TOKEN_RE.findall(text.lower())) + + +class IncidentStore: + """Append-only JSONL journal of past action items, with recurrence lookup.""" + + def __init__(self, path: str | Path) -> None: + self.path = Path(path).expanduser() + self._records: list[dict] = [] + self._load() + + def _load(self) -> None: + if not self.path.is_file(): + return + try: + for line in self.path.read_text(encoding="utf-8").splitlines(): + line = line.strip() + if not line: + continue + try: + rec = json.loads(line) + except json.JSONDecodeError: + continue + if isinstance(rec, dict) and rec.get("description"): + self._records.append(rec) + except OSError as exc: + logger.warning("incident store unreadable (%s) — starting fresh", exc) + if len(self._records) > _MAX_RECORDS: + self._records = self._records[-_MAX_RECORDS:] + + def __len__(self) -> int: + return len(self._records) + + # ── recall ──────────────────────────────────────────────────────────── + + def recurrence(self, description: str) -> dict | None: + """History of this exact issue (KB descriptions are stable strings).""" + matches = [r for r in self._records if r.get("description") == description] + if not matches: + return None + devices: set[str] = set() + for r in matches: + devices.update(r.get("devices") or []) + return { + "count": len(matches), + "last_seen": max(r.get("generated_at", "") for r in matches), + "devices": sorted(devices)[:10], + } + + def find_similar(self, query: str, top_n: int = 5) -> list[dict]: + """Free-text token-overlap search across the journal (Jaccard).""" + q = _tokens(query) + if not q: + return [] + scored: list[tuple[float, dict]] = [] + for r in self._records: + blob = f"{r.get('description', '')} {' '.join(r.get('devices') or [])} " \ + f"{' '.join(r.get('sample_messages') or [])}" + t = _tokens(blob) + if not t: + continue + overlap = len(q & t) + if overlap == 0: + continue + scored.append((overlap / len(q | t), r)) + scored.sort(key=lambda x: -x[0]) + return [ + {"score": round(s, 3), **{k: r.get(k) for k in + ("description", "severity", "category", "devices", "count", "generated_at")}} + for s, r in scored[:top_n] + ] + + # ── record ──────────────────────────────────────────────────────────── + + def record(self, action_items: list[dict], generated_at: str) -> int: + """Append this run's action items; rewrites the journal when it + outgrows the cap. Failures are logged, never raised.""" + new = [] + for a in action_items: + new.append({ + "description": a.get("description", ""), + "severity": a.get("severity", ""), + "category": a.get("category", ""), + "devices": (a.get("devices") or [])[:10], + "count": a.get("count", 0), + "sample_messages": (a.get("sample_messages") or [])[:2], + "generated_at": generated_at, + }) + if not new: + return 0 + self._records.extend(new) + try: + self.path.parent.mkdir(parents=True, exist_ok=True) + if len(self._records) > _MAX_RECORDS: + self._records = self._records[-_MAX_RECORDS:] + tmp = self.path.with_suffix(self.path.suffix + ".tmp") + tmp.write_text( + "\n".join(json.dumps(r) for r in self._records) + "\n", + encoding="utf-8") + tmp.replace(self.path) + else: + with self.path.open("a", encoding="utf-8") as fh: + for r in new: + fh.write(json.dumps(r) + "\n") + except OSError as exc: + logger.warning("could not persist incident store to %s: %s", self.path, exc) + return len(new) + + +def get_store() -> IncidentStore | None: + """Opt-in via AI_LOG_ANALYZER_INCIDENT_STORE; None when unset.""" + path = os.environ.get("AI_LOG_ANALYZER_INCIDENT_STORE", "") + return IncidentStore(path) if path else None + + +def annotate_and_record(result_dict_items: list[dict], generated_at: str) -> None: + """Stamp each action item with its recurrence history from prior runs, + then append this run to the journal. No env var → no-op.""" + store = get_store() + if store is None: + return + for a in result_dict_items: + rec = store.recurrence(a.get("description", "")) + if rec: + a["recurrence"] = rec + store.record(result_dict_items, generated_at) diff --git a/src/ai_log_analyzer/sources/syslog.py b/src/ai_log_analyzer/sources/syslog.py index b250edb..dba39be 100644 --- a/src/ai_log_analyzer/sources/syslog.py +++ b/src/ai_log_analyzer/sources/syslog.py @@ -43,6 +43,7 @@ def __init__(self, config: SourceConfig) -> None: raise SourceError(f"Unsupported syslog proto {self.proto!r}") capacity = int(config.extra.get("buffer_size", _DEFAULT_BUFFER_SIZE)) self._buffer: deque[str] = deque(maxlen=capacity) + self._total_ingested = 0 # monotonic — cursor base for fetch_new() self._lock = threading.Lock() self._sock: socket.socket | None = None self._thread: threading.Thread | None = None @@ -72,6 +73,22 @@ def fetch(self, *, since_seconds: int = 3600, limit: int = 10_000, continue yield ev + def fetch_new(self, cursor: int, limit: int = 1000) -> tuple[list[LogEvent], int]: + """Incremental read for live tailing: lines ingested since `cursor`. + + Returns (events, new_cursor). The cursor is the listener's monotonic + ingest counter — pass 0 the first time (or the previous new_cursor), + and lines that scrolled out of the ring buffer between polls are + skipped rather than duplicated. + """ + with self._lock: + total = self._total_ingested + n_new = min(total - cursor, len(self._buffer)) + lines = list(self._buffer)[-n_new:] if n_new > 0 else [] + if limit and len(lines) > limit: + lines = lines[-int(limit):] + return list(parse_lines(lines, default_host="syslog")), total + def close(self) -> None: self._stop.set() if self._sock is not None: @@ -154,6 +171,7 @@ def _ingest(self, raw: bytes) -> None: return with self._lock: self._buffer.append(line) + self._total_ingested += 1 registry.register("syslog", SyslogListenerSource.from_config) diff --git a/src/ai_log_analyzer/web/app.py b/src/ai_log_analyzer/web/app.py index c9ceb26..7024c4a 100644 --- a/src/ai_log_analyzer/web/app.py +++ b/src/ai_log_analyzer/web/app.py @@ -828,6 +828,81 @@ def api_optimize(): result["config_length"] = len(running_config) return jsonify(result) + # ── Incident memory ──────────────────────────────────────────────────── + @app.route("/api/incidents/similar", methods=["GET"]) + def api_incidents_similar(): + """Free-text search over the local incident journal. + + Requires AI_LOG_ANALYZER_INCIDENT_STORE; ?q=&top=. + """ + from ai_log_analyzer.memory import get_store + store = get_store() + if store is None: + return jsonify({"error": "incident store not configured — " + "set AI_LOG_ANALYZER_INCIDENT_STORE"}), 400 + q = (request.args.get("q") or "").strip() + if not q: + return jsonify({"error": "q parameter required"}), 400 + try: + top = max(1, min(20, int(request.args.get("top") or 5))) + except ValueError: + top = 5 + return jsonify({"query": q, "journal_size": len(store), + "matches": store.find_similar(q, top_n=top)}) + + # ── Live tail (SSE) ──────────────────────────────────────────────────── + @app.route("/api/tail/", methods=["GET"]) + def api_tail(source_id: str): + """Server-Sent Events stream of classified events from a syslog source. + + Zero-dependency real time: EventSource in the browser, chunked + response here — no websocket stack. EventSource cannot set request + headers, so on tokened deployments pass ?token=. + Optional ?min_severity=critical|high|medium|low filters server-side. + """ + if API_TOKEN and request.args.get("token", "") != API_TOKEN \ + and _token_supplied() != API_TOKEN: + abort(401) + src = source_manager.get(source_id) + if src is None: + return jsonify({"error": f"unknown source: {source_id}"}), 404 + if not hasattr(src, "fetch_new"): + return jsonify({"error": "live tail supports syslog listener sources only"}), 400 + + from ai_log_analyzer.classifier import SEV_ORDER + min_sev = (request.args.get("min_severity") or "info").lower() + max_rank = SEV_ORDER.get(min_sev, 4) + poll_s = min(5.0, max(0.2, float(request.args.get("poll") or 1.0))) + + def stream(): + import json as _json + import time as _time + from ai_log_analyzer.classifier import iter_classify + cursor = getattr(src, "_total_ingested", 0) # start at "now" + yield "event: hello\ndata: {}\n\n" + idle = 0.0 + while True: + events, cursor = src.fetch_new(cursor) + sent = False + for ce in iter_classify(iter(events)): + if SEV_ORDER.get(ce.severity, 4) <= max_rank: + yield f"data: {_json.dumps(ce.to_dict())}\n\n" + sent = True + if sent: + idle = 0.0 + else: + idle += poll_s + if idle >= 15.0: # keep proxies from timing the stream out + yield ": keepalive\n\n" + idle = 0.0 + _time.sleep(poll_s) + + return app.response_class( + stream(), + mimetype="text/event-stream", + headers={"Cache-Control": "no-store", "X-Accel-Buffering": "no"}, + ) + # ── External log sources (Kibana, Splunk, Loki, syslog, LibreNMS, ...) ──── @app.route("/api/sources", methods=["GET"]) def api_sources_list(): diff --git a/src/ai_log_analyzer/web/static/app.js b/src/ai_log_analyzer/web/static/app.js index bb99309..26afcfd 100644 --- a/src/ai_log_analyzer/web/static/app.js +++ b/src/ai_log_analyzer/web/static/app.js @@ -760,9 +760,16 @@ function render(r) { clear(aTbody); r.action_items.forEach((item) => { const devLabel = `${item.devices.length}: ${item.devices.slice(0, 3).join(", ")}`; + // Incident-memory recurrence: "this exact issue appeared N times before" + const rec = item.recurrence; + const descCell = rec && rec.count + ? el("td", {}, document.createTextNode(item.description + " "), + el("span", { className: "row-msg", + text: `↻ seen ${rec.count}× before (last ${String(rec.last_seen || "").slice(0, 10)})` })) + : el("td", { text: item.description }); const tr = el("tr", { className: "action" }, el("td", {}, sevBadge(item.severity)), - el("td", { text: item.description }), + descCell, el("td", { text: String(item.count) }), el("td", { text: devLabel }), ); @@ -2653,6 +2660,67 @@ function hideIdleStateIfAnyPanelVisible() { } // ── Init ───────────────────────────────────────────────────────────────────── +// ── Live Tail (SSE) ────────────────────────────────────────────────────────── +let _tailES = null; +let _tailCount = 0; + +async function loadTailSources() { + const sel = $("tail-source"); + if (!sel) return; + try { + const data = await fetchJSON("/api/sources"); + clear(sel); + (data.sources || []).filter((s) => s.type === "syslog").forEach((s) => { + sel.appendChild(el("option", { text: s.id })); + sel.lastChild.value = s.id; + }); + if (!sel.options.length) { + sel.appendChild(el("option", { text: "(no syslog sources registered)" })); + sel.lastChild.value = ""; + } + } catch { /* sources API unavailable — leave empty */ } +} + +function stopLiveTail() { + if (_tailES) { _tailES.close(); _tailES = null; } + const btn = $("tail-btn"); + if (btn) btn.textContent = "▶ Start Live Tail"; +} + +function toggleLiveTail() { + if (_tailES) { stopLiveTail(); return; } + const src = $("tail-source").value; + if (!src) { setStatus("Register a syslog source first (Device tab → Sources)."); return; } + const minSev = $("tail-min-sev").value; + const panel = $("live-panel"); + panel.style.display = "block"; + const idle = $("idle-state"); + if (idle) idle.style.display = "none"; + _tailCount = 0; + const tbody = $("live-table").querySelector("tbody"); + clear(tbody); + _tailES = new EventSource(`/api/tail/${encodeURIComponent(src)}?min_severity=${minSev}`); + _tailES.onmessage = (msg) => { + let e; + try { e = JSON.parse(msg.data); } catch { return; } + _tailCount += 1; + $("live-count").textContent = `(${_tailCount} events · ${src})`; + const tr = el("tr", { className: `sev-row-${e.severity || "info"}` }, + el("td", {}, sevBadge(e.severity)), + el("td", {}, el("span", { className: "row-msg", text: e.timestamp || "" })), + el("td", { text: e.hostname || "" }), + el("td", {}, el("span", { className: "row-msg", text: e.category })), + el("td", { text: e.description || "" }), + ); + tbody.insertBefore(tr, tbody.firstChild); + while (tbody.children.length > 200) tbody.removeChild(tbody.lastChild); + }; + _tailES.onerror = () => { + setStatus("Live tail connection lost — retrying automatically…"); + }; + $("tail-btn").textContent = "⏸ Stop Live Tail"; +} + document.addEventListener("DOMContentLoaded", () => { // Sidebar tabs document.querySelectorAll(".side-tab").forEach((tab) => { @@ -2674,6 +2742,8 @@ document.addEventListener("DOMContentLoaded", () => { $("site-opt-btn").addEventListener("click", runSiteOptimize); $("site-picker").addEventListener("change", showSiteMeta); $("reload-containers").addEventListener("click", loadContainers); + const _tailBtn = $("tail-btn"); + if (_tailBtn) { _tailBtn.addEventListener("click", toggleLiveTail); loadTailSources(); } // All/None bulk-toggle (previously dead handlers — the IDs existed in HTML // but no JS was wired, so clicking "All" only highlighted the button itself). const _allBtn = $("chips-all"); diff --git a/src/ai_log_analyzer/web/static/index.html b/src/ai_log_analyzer/web/static/index.html index 6c14c21..8e383e7 100644 --- a/src/ai_log_analyzer/web/static/index.html +++ b/src/ai_log_analyzer/web/static/index.html @@ -1217,6 +1217,27 @@ + +
+ 🔴 Live Tail +
+ + + + + +
+ Streams classified events from a registered syslog + listener source in real time (SSE — no polling, no build step). +
+
+
@@ -1438,6 +1459,19 @@

Action Items

+ +