Every shipped feature with its CLI command(s), the workflow that drives it in CI, the code path, the tests that gate it, and a per-feature doc link. Generated by reading the codebase; every claim resolves to a file path.
Looking for the canonical, code-derived view? See
generated-catalog.md, which is emitted bycontentops catalog regenerateand pinned drift-free in CI bycontentops catalog check. This page keeps the curated narrative; the generated page is what the pipeline guarantees stays in sync with the code.
For per-asset behaviour see asset-coverage.md.
For per-test detail see test-catalog.md.
Four PRs landed in close sequence; each row above has been refreshed, but here's the high-level so reviewers and adopters see "what's new":
- PR #237 — NVISO Detection-as-Code borrowings (4 patterns):
per-detection markdown docs (
contentops detection-docs), auto-disabled rule detection (contentops auto-disabled-rules), PR-time tuning impact preview (contentops tuning preview), URL link-rot + codespell CI checks. - PR #239 — Full gap analysis follow-through (4 picks):
pre-flight tenant diff (
contentops plan --against-tenant), dedicatedrole: testworkspace, MITRE D3FEND mapping (contentops coverage --d3fend), three operational gaps (SentinelHealth doctor probe, PR-time URL check, fork-PR doc). - PR #240 — MITRE ATT&CK Navigator layer renderer
(
contentops navigator), three coverage axes (repo + deployed + firings), score by unique-rule-name count. - PR #241 —
policy.scaffoldStrictdefault flipped from True to False. Adopters with a freshconfig/tenant.ymlno longer get META002-005 as CI-blocking errors out of the box; opt INTO strict withpolicy.scaffoldStrict: trueonce the authoring backlog drains. The operator's own internal posture is still "fail fast" via explicitscaffoldStrict: truein their tenant.yml.
V2 commands are registered in
contentops/cli/__init__.py and
defined in the contentops/cli/commands/ package.
The legacy v1 verbs (validate, deploy, delete, diff) were
removed in R4 of v1-retirement-plan.md;
see v1-to-v2-migration-guide.md
for the operator-facing verb mapping.
contentopsis the only CLI entry point. The historicalpipelinealias was removed and is not reintroduced — see CLAUDE.md invariant §2. Bothcontentops <cmd>(the console script) andpython -m contentops <cmd>work afterpip install -e .and dispatch into the same Click root group.
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Plan | contentops plan [--asset K] [--changed-since REF] [--skip-deps-check] [--against-tenant] [--role R] [--workspace W] |
validate.yml (PR gate) |
commands/apply.py (plan_cmd) → handler validate() + plan(); --against-tenant overlay via contentops/core/drift.py detect_drift() |
tests/v2/test_cli_plan_apply.py, tests/v2/test_plan_against_tenant.py |
this file | Read-only — runs each handler's validate + plan, prints the intended action per asset, exits 1 on validation errors. --against-tenant (closes G17) overlays a CREATE / UPDATE / NO-CHANGE / ORPHAN-IN-TENANT summary by also calling list_remote() against the live workspace; PR-time pre-flight diff. Off by default so fork PRs / offline runs keep working. |
| Apply | contentops apply [--asset K] [--dry-run] [--no-audit] [--changed-since REF] [--skip-deps-check] [--force-overwrite] [--role prod|integration|dev] [--workspace NAME] [--continue-on-error] |
deploy.yml (push to main, prod role) / integration-deploy.yml (PR, integration role) |
commands/apply.py (apply_cmd) → handler apply() |
tests/v2/test_cli_plan_apply.py, tests/v2/test_apply_verify_*.py, tests/v2/test_apply_continue_on_error.py |
this file | Authoritative deploy. Writes audit + state. Skips localCustomization:true envelopes unless --force-overwrite. --role / --workspace select the target Sentinel workspace from the multi-workspace config/tenant.yml (see docs/operations/multi-workspace.md). --continue-on-error treats per-rule failures as warnings (exit 0 even when some rules fail) — intended for integration PR deploys where a broken rule shouldn't block merge; prod deploys leave it off and fail loud. |
| Retry failed | contentops retry-failed [--audit-dir DIR] [--since SPEC] [--run-id ID] [--dry-run] |
retry-failed.yml |
commands/lifecycle.py (retry_failed_cmd) → contentops/audit_filter.py |
tests/v2/test_lock_unlock_retry.py, tests/v2/test_retry_failed_since.py |
this file | Re-applies only the assets a previous apply marked as failed. Default scope is the latest audit/*.jsonl. --since DURATION_OR_ISO widens to a time window across files; --run-id narrows to a single GITHUB_RUN_ID. Useful after a transient ARM 5xx burst, a Graph rate limit, or when a successful later run has masked a prior partial failure. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Drift | contentops drift [--asset K] [--write] [--no-exit-on-drift] [--report PATH] [--suppressions {honor|ignore}] [--diff] |
drift.yml (daily + PR comment) |
commands/drift.py (drift_cmd) → detect_drift() + field-diff render in contentops/core/drift.py; suppression filter in contentops/drift_suppressions.py |
tests/v2/test_drift.py, tests/v2/test_drift_roundtrip.py, tests/v2/test_drift_suppressions.py, tests/v2/test_drift_field_diff.py, tests/integration/test_collect_drift_roundtrip.py |
this file | Compare remote tenant ↔ local YAML; report new/changed/in-sync. detections/drift_suppressions.yml (with expiry dates) hides known-good portal-side tweaks; expired entries re-surface tagged [suppression-expired]; unused entries flagged [suppression-unused]. --diff prints field-level deltas for every CHANGED entry — the G2 diagnostic for "why is this rule flagged?". Optionally write changed envelopes. Exit 2 on drift (CI gate). |
| Drift PR body | contentops drift-pr-body --report PATH [--out OUT] [--labels-out OUT] |
drift.yml |
commands/drift.py (drift_pr_body_cmd) → contentops/upstream/drift_pr.py |
tests/v2/test_drift_pr_body.py |
this file | Renders a Markdown body for the auto-drift PR opened by drift.yml. |
| Drift resolve | contentops drift-resolve <id> --strategy {git|remote|merge} [--asset K] [--dry-run] |
none — local | commands/drift.py (drift_resolve_cmd) → contentops/drift_resolve.py |
tests/v2/test_drift_resolve.py |
this file | Per-rule drift reconciliation. git: local wins (no mutation). remote: write the remote envelope to local YAML. merge: editor 3-way diff (deferred — raises NotImplementedStrategy with a clear message). Useful when most drift entries should fall back to git but a specific rule was intentionally tuned in the portal (or vice versa). |
| Collect | contentops collect [--asset K] [--full] [--since ISO] [--workers N] [--rename-existing] [--role prod|integration|dev] [--workspace NAME] [--clear/--no-clear] |
collect.yml (Mondays 06:00 UTC) |
commands/collect.py (collect_cmd) |
tests/v2/test_collect_roundtrip.py, tests/v2/test_clean_cmd.py, tests/integration/test_collect_live_roundtrip.py |
docs/operations/collect.md |
Walks every drift-capable handler in parallel, writes new+changed envelopes. The "pull everything down" entry point — distinct from drift, which is gated for CI use. --clear deletes local detection YAMLs before collecting — equivalent to contentops clean --yes chained ahead; use when you want a true refresh-from-tenant rather than an additive merge. --role / --workspace pick which Sentinel workspace from config/tenant.yml to collect from. |
| Clean | contentops clean [--path PATH] [--asset K]... [--yes] |
none — local | commands/collect.py (clean_cmd) → _clean_local_detections() helper |
tests/v2/test_clean_cmd.py |
this file | Deletes detections/<asset_kind>/*.yml (and the legacy detections/sentinel/ / detections/defender/ directories), preserving detections/templates/ and detections/samples/. Prompts for confirmation unless --yes. Pairs with collect --clear. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Prune | contentops prune [--asset K] [--dry-run/--no-dry-run] [--yes] [--max-deletes N] [--include-locked] [--json] |
prune.yml (manual dispatch) |
commands/prune.py |
tests/v2/test_prune.py, tests/integration/test_prune_live.py |
docs/operations/prune.md |
Deletes remote assets that have no matching local YAML. Defaults dry-run; requires --yes + --no-dry-run to actually delete. --max-deletes is a fail-closed cap. Writes audit records. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Lint (KQL + payload) | contentops lint [--asset K] [--severity S] [--fail-on-warn] [--strict] |
lint.yml + validate.yml (PR gate, --strict) |
commands/lint.py (lint_cmd) → contentops/lint/runner.py, contentops/lint/kql.py, contentops/lint/payload.py, contentops/lint/strict.py, contentops/lint/strict_rules.py |
tests/v2/test_lint.py, tests/v2/test_remediate_payload001.py, tests/v2/test_lint_strict.py, tests/v2/test_lint_strict_take_limit.py |
this file | Pure-Python KQL static lint (KQL001-007) and payload contract (PAYLOAD001). With --strict: adds Python policy rules (KQL101 — | take / | limit forbidden in production rules) and, when the optional .NET wrapper at tools/kql_strict.dll is installed, augments with Kusto.Language parser diagnostics; otherwise prints a single advisory and ships the policy rules. Both lint.yml and validate.yml run --strict on PRs. Legacy v1 envelopes get only payload rules. |
| Coverage (MITRE ATT&CK + D3FEND) | contentops coverage [--format markdown/json/both] [--out-md PATH] [--out-json PATH] [--gaps] [--techniques-file FILE] [--d3fend] [--d3fend-file FILE] |
coverage.yml (PR comment) |
commands/coverage.py (coverage_cmd) → contentops/coverage/report.py, contentops/coverage/extract.py, contentops/coverage/gaps.py, contentops/coverage/d3fend.py |
tests/v2/test_coverage.py, tests/v2/test_coverage_extract.py, tests/v2/test_coverage_gaps.py, tests/v2/test_coverage_d3fend.py |
this file | Three modes: (1) default — ATT&CK heatmap from envelope.metadata.tactics/techniques/severity (when authored) combined with the asset-native payload location (payload.detectionAction.alertTemplate.mitreTechniques for Defender; payload.tactics/payload.techniques/payload.severity for Sentinel analytics + hunting). (2) --gaps — inverse view: techniques in a reference list NOT covered by any detection. Bundled curated list at contentops/coverage/data/mitre_attack_techniques.json; override with --techniques-file FILE. (3) --d3fend — defensive-axis companion: reads metadata.defensiveTechniques (D3-XXX IDs) and reports which D3FEND defensive techniques are implemented vs. gaps. Bundled curated list at contentops/coverage/data/d3fend_techniques.json; override with --d3fend-file FILE. Closes G14 (ATT&CK gaps) + introduces the D3FEND axis (post-PR #239). |
| Portfolio | contentops portfolio [--out-csv PATH] [--out-json PATH] [--cohort C] [--legacy/--no-legacy] [--with-telemetry] [--workspace-id ID] [--telemetry-since N] | portfolio.yml (nightly 07:00 UTC) | commands/portfolio.py (portfolio_cmd) → contentops/portfolio/report.py, contentops/workspace_kql.py | tests/v2/test_portfolio.py, tests/v2/test_workspace_kql.py | this file | Flat per-detection portfolio CSV/JSON. With --with-telemetry: F20 augmentation — adds alerts_30d, incidents_30d, closed_fp_30d, fp_rate columns sourced from the LA Query API. Closes G20. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Silent rules | contentops silent-rules [--workspace-id ID] [--role R] [--since N] [--format table|json|csv] [--out PATH] |
silent-rules.yml (Mondays 07:00 UTC + dispatch) |
commands/silent_rules.py (silent_rules_cmd) → contentops/workspace_kql.py |
tests/v2/test_workspace_kql.py |
this file | Lists every rule's SecurityAlert + SecurityIncident counts in the lookback window; rules with alerts_30d == 0 are silent-rule candidates. Closes G7. Workspace auto-derived from tenant.yml's --role (default prod); override with --workspace-id GUID. |
| Auto-disabled rules | contentops auto-disabled-rules [--workspace-id ID] [--role R] [--since N] [--format table|json|csv] [--out PATH] [--require-data] |
silent-rules.yml (same step set, runs in sequence) |
commands/auto_disabled.py (auto_disabled_rules_cmd) → contentops/workspace_kql.py auto_disabled_query() |
tests/v2/test_workspace_kql.py |
this file | NVISO Part 7 borrowing. Surfaces rules Sentinel itself disabled (consecutive query failures, schema break) — distinct from silent rules which simply produced no alerts. Unions SentinelHealth.Status in ("Disabled", "Failure") with LAQueryLogs recent-failure rows. Prerequisite: SentinelHealth diagnostic data collection must be enabled on the workspace (opt-in since ~2022) — contentops doctor --auth includes a probe (sentinel_health check) that warns when zero rows are returned. --require-data exits non-zero on empty results for CI use. |
| Tuning impact preview | contentops tuning preview [--base-ref REF] [--suppressions-path PATH] [--workspace-id ID] [--role R] [--since N] [--out PATH] [--no-workspace-query] |
tuning-impact-preview.yml (PR comment on detections/drift_suppressions.yml changes) |
commands/tuning.py (tuning_group → tuning_preview_cmd) → contentops/tuning.py + contentops/workspace_kql.py suppression_impact_query() |
tests/v2/test_tuning.py |
this file | NVISO Part 8 borrowing. When a PR adds drift suppressions, runs a 30-day retrospective KQL and reports "this suppression would have silenced N incidents and M alerts" so reviewers see the blast radius before approving. Resolves envelope id → displayName via contentops/core/discovery.py. Fork-PR safe: --no-workspace-query renders the table with - when OIDC isn't available, posts the comment anyway. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Navigator layer | contentops navigator [--repo/--no-repo] [--deployed/--no-deployed] [--firings/--no-firings] [--since N] [--workspace-id ID] [--role R] [--out PATH] [--name "..."] [--description "..."] [--fail-soft/--fail-loud] |
none — local / on-demand | commands/navigator.py (navigator_cmd) → contentops/navigator/extract.py, contentops/navigator/render.py |
tests/v2/test_navigator_extract.py, tests/v2/test_navigator_render.py |
this file | Renders a MITRE ATT&CK Navigator layer JSON aggregating three coverage axes: (1) repo envelopes (claimed coverage from metadata.techniques), (2) deployed rules (live Sentinel ARM + Defender Graph beta), (3) firings (SecurityAlert.Techniques over --since days; both Sentinel-native and Defender XDR alerts when the M365 Defender connector is enabled). Score per technique = unique-display-name count across selected axes; parent techniques auto-roll up at score 0 so the UI renders the parent tile. No SVG export — upload the JSON to https://mitre-attack.github.io/attack-navigator/ via "Open Existing Layer → Upload from local". Permissions: no new permissions required when the M365 Defender connector is installed; Sentinel Reader + existing CustomDetection.Read.All cover all three axes. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Detection docs regenerate | contentops detection-docs regenerate [--repo-root PATH] [--prune-orphans/--no-prune-orphans] |
none — local; CI gate via detection-docs check in tests |
commands/detection_docs.py → contentops/docs/render.py |
tests/v2/test_detection_docs.py |
this file | NVISO Part 4 borrowing. Renders every envelope to docs/detections/<asset>/<id>.md plus an index at docs/detections/README.md. Stdlib templating (no Jinja2) — same byte-identical-output drift-gated pattern as catalog regenerate. Source-of-truth metadata pulled from RuleMetadata (severity, MITRE tags, owner, runbookUrl, falsePositives, blindSpots, responseActions, references, lastValidatedAt, fpExpectedPerWeek). Asset-kind subdirs avoid id collisions between Sentinel and Defender variants of the same rule. |
| Detection docs check | contentops detection-docs check [--repo-root PATH] |
(run by tests/v2/test_detection_docs.py::test_committed_detection_docs_are_in_sync) |
commands/detection_docs.py |
same | this file | Byte-identical drift gate. Exit 1 if any committed file under docs/detections/ disagrees with the renderer's output, or if there are orphan .md files the generator doesn't own. Fix: run contentops detection-docs regenerate and commit the diff. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Restore | contentops restore <archive> [--out PATH] [--force] |
none — local | commands/archive.py (restore_cmd) → contentops/restore.py |
tests/v2/test_restore.py |
this file | Inverse of contentops collect. Reads a .tar.gz/.tgz archive of detections/<asset_kind>/<id>.yml files (and optional MANIFEST.json) and restores them under --out (default detections/). Refuses to overlay a non-empty target without --force. Defends against path-traversal entries. |
| Snapshot diff | contentops snapshot-diff <a.tar.gz> <b.tar.gz> [--asset K] [--format markdown|json] [--out PATH] |
none — local | commands/archive.py (snapshot_diff_cmd) → contentops/snapshot_diff.py |
tests/v2/test_snapshot_diff.py |
this file | Content-aware diff between two contentops collect archives. Indexes envelopes by (asset_kind, envelope_id) so renames don't surface as changes. Closes G23. Pairs with F10 (contentops restore). Exit 2 on changes (CI gate). |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Defender extensions probe | contentops defender-extensions-probe [--format markdown|json] [--out PATH] |
(caller's responsibility — wire via cron / scheduled workflow) | commands/diagnostics.py (defender_extensions_probe_cmd) → contentops/defender_extensions_probe.py |
tests/v2/test_defender_extensions_probe.py |
this file, docs/assets/defender_graph_extensions_deferred.md |
Probe of Microsoft Graph savedQueries / detection-tuning-rules / alert-suppression endpoints. Today returns available=false for all three (Microsoft hasn't shipped them); when one GA's, the probe surfaces it. Secondary signal — not a primary GA-discovery channel; the GA announcement comes from Microsoft. Exit 2 if any endpoint becomes available. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Defender round-trip diff | contentops defender-roundtrip-diff <envelope_id> [--path D] [--raw] |
none — local | commands/diagnostics.py (defender_roundtrip_diff_cmd) → contentops/defender_roundtrip.py |
tests/v2/test_defender_roundtrip.py, tests/v2/test_apply_verify_defender.py::test_apply_verifies_when_remote_has_server_managed_nested_fields |
this file | Diagnose a verified=False / MISMATCH from contentops apply for a Defender custom detection. Loads the local envelope, fetches the live remote, and reports which _HASHED_FIELDS paths differ under the same canonical-JSON projection compute_content_hash uses. By default applies _strip_server_fields so the diagnostic matches what apply would see; --raw skips the strip to spot new server-managed fields the stripper doesn't yet know about. Read-only — no tenant writes. Exit 2 on differences. |
| Sentinel round-trip diff | contentops sentinel-roundtrip-diff <envelope_id> [--path D] [--raw] [--role prod|integration|dev] [--workspace NAME] |
none — local | commands/diagnostics.py (sentinel_roundtrip_diff_cmd) → contentops/sentinel_roundtrip.py + contentops/utils/roundtrip_diff.py |
tests/v2/test_sentinel_roundtrip.py |
this file | Sentinel counterpart to defender-roundtrip-diff. Dispatches by asset kind (analytic / hunting / parser / watchlist) to the right handler's _HASHED_FIELDS + _strip_server_fields. data_connector is not currently supported (uses _projection() not _HASHED_FIELDS). Hunting + parser dispatch via the LA workspace path (la_resource_url); analytic + watchlist via the SecurityInsights namespace (get_resource). On multi-workspace tenants pass --role or --workspace. Read-only — no tenant writes. Exit 2 on differences. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| Disable | contentops disable RULE_ID [--reason TEXT] |
emergency-disable.yml |
commands/lifecycle.py |
tests/v2/test_disable.py, tests/v2/test_emergency_disable_workflow.py |
docs/emergency-disable-workflow.md |
Sets status: deprecated in YAML and (optionally) appends disableReason. Doesn't commit. The workflow opens a PR for review. Single-rule break-glass. |
| Explain | contentops explain RULE_ID [--path D] [--audit-dir A] [--format markdown|json] |
none — local | commands/diagnostics.py (explain_cmd) → contentops/explain.py |
tests/v2/test_explain.py |
this file | Single-command rule context: envelope summary, dependencies (from detections/dependencies.yml), state (from state/state.json), recent audit records (5 newest), and drift status (from drift_report.json if present). Markdown default; --format json for scripted consumption. Exits 1 if no rule with that id is found. |
| Lock | contentops lock RULE_ID |
lock-unlock.yml |
commands/lifecycle.py |
tests/v2/test_lock_unlock_retry.py |
this file | Adds top-level localCustomization: true to the envelope. Subsequent apply skips it without --force-overwrite. |
| Unlock | contentops unlock RULE_ID |
lock-unlock.yml |
commands/lifecycle.py |
tests/v2/test_lock_unlock_retry.py |
this file | Removes the localCustomization flag. Inverse of lock. |
| Lifecycle promote | contentops lifecycle promote RULE_ID [--max-validation-age-days N] [--force] [--dry-run] |
none — local | commands/lifecycle.py (lifecycle_group) → contentops/lifecycle.py |
tests/v2/test_lifecycle_promote.py |
this file | Promote experimental → production after gates pass. Active gates: status_is_experimental, recent_validation (G19 enforcement). Deferred gates (skipped, hooks present): live_test_pass (F2), fp_rate_threshold (F20). Surgical YAML edit (one-line status: flip). --force overrides gate failures. Closes G13 with reduced gate set. |
| Rollback | contentops rollback SHA [--asset K] [--dry-run/--no-dry-run] [--yes] |
none yet — local | commands/rollback.py (rollback_cmd) → contentops/rollback.py |
tests/v2/test_rollback.py |
this file | Replays the YAML at SHA against the tenant. Materialises detections/ at SHA into a temp tree via git ls-tree + git show, then runs each handler's validate + apply. Defaults dry-run. Audit records carry message="rollback to <sha>". Non-destructive (a rule that exists today but didn't at SHA is left alone — run prune for full reset). |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| New (scaffold) | contentops new ASSET ID [--name N] [--out PATH] [--force] |
none — local | commands/new.py → contentops/devex/scaffold.py |
tests/v2/test_devex_scaffold.py |
docs/assets/cli_new_from_template.md |
Generate a valid envelope YAML from a Pydantic-validated template. 12 supported asset kinds. |
| New from Marketplace template | contentops new --from-template GUID [--id ID] |
none — local (needs Azure credential) | commands/new.py → contentops/devex/templates_remote.py |
tests/integration/test_sentinel_analytic_scaffold_deploys.py |
docs/assets/cli_new_from_template.md |
Fetches the template from alertRuleTemplates, materialises a sentinel_analytic envelope. |
| Search templates | contentops new --search-template SUBSTRING |
none — local (needs Azure credential) | commands/new.py |
(read-only — covered indirectly) | docs/assets/cli_new_from_template.md |
Lists up to 20 alert rule templates whose ARM name or displayName contains the substring. |
| Bootstrap | contentops bootstrap --subscription S --resource-group RG --workspace WS [--location L] [--env E] [--dry-run] | promote-to-integration.yml | contentops/cli/commands/bootstrap.py | tests/v2/test_bootstrap_cli.py | this file | Idempotent first-run setup: creates RG + LA workspace, onboards Sentinel, writes config/tenant.yml. Each step skips if the resource already exists. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| State sync push | contentops state sync push [--env E] [--remote R] [--no-push] |
deploy.yml (after prod apply) |
commands/state.py (state_sync_push) → contentops/state_sync.py |
tests/v2/test_state_sync.py, tests/v2/test_workflow_state_and_telemetry.py |
this file | Stages state/state.json as an orphan commit on refs/heads/state/<env> via git plumbing (hash-object + mktree + commit-tree + update-ref), then force-pushes to remote. Working tree never disturbed. Closes G15. |
| State sync pull | contentops state sync pull [--env E] [--remote R] [--no-fetch] |
deploy.yml (before prod apply) |
commands/state.py (state_sync_pull) → contentops/state_sync.py |
tests/v2/test_state_sync.py, tests/v2/test_workflow_state_and_telemetry.py |
this file | Fetches refs/heads/state/<env>, reads state.json from it, writes to state/state.json. Tolerant of missing ref (first run on a fresh env). |
| State sync status | contentops state sync status [--env E] |
none | commands/state.py (state_sync_status) → contentops/state_sync.py |
tests/v2/test_state_sync.py |
this file | Compares the local state file's SHA against the remote ref's tree blob. Exit 1 if diverged. |
| Feature | CLI | Workflow | Code path | Tests | Doc | What it does |
|---|---|---|---|---|---|---|
| State show | contentops state show [--env E] [--asset K] [--format text/json] |
none — local | commands/state.py → contentops/state.py |
tests/v2/test_state_file.py |
this file, architecture.md |
Print the per-env state file (last apply SHA, managed assets per kind). |
| State forget | contentops state forget ENVELOPE_ID --asset K [--env E] |
none — local | commands/state.py |
tests/v2/test_state_file.py |
this file | Drop one envelope id from state — used after a manual portal cleanup. |
| Audit verify | contentops audit verify [--root PATH] |
audit-verify.yml (Mondays 04:00 UTC) |
commands/audit.py (audit_verify_cmd) → contentops/audit/writer.py |
tests/v2/test_audit.py, tests/v2/test_audit_chain.py |
audit-trail.md |
Walk every audit/*.jsonl and recompute the hash chain. Exit 1 on tamper. |
| Audit query | contentops audit query {latest | failures | by-actor | rollbacks | timeline} ... |
none — local | commands/audit.py (audit_query_group) → contentops/audit_query.py |
tests/v2/test_audit_query.py |
audit-trail.md |
Forensic + compliance queries over audit/*.jsonl without writing jq. Five subcommands; --format table|json|csv; --since window on most. Pure read; never writes the chain. |
| Doctor | contentops doctor [--auth] [--matrix] [--format text/json] [--fix] [--dry-run] |
(called from contentops test --live) |
commands/doctor.py (doctor_cmd) → contentops/devex/doctor.py |
tests/v2/test_devex_doctor.py, tests/v2/test_doctor_fix.py |
this file | Environment + config sanity checks: python version, deps, .env, auth env vars, tenant.yml parse, detections parse, git, optional token acquisition, optional per-handler list_remote() matrix. With --fix: applies the whitelisted safe autofixers (copy .env.example→.env, mkdir detections/{sentinel,defender}); never touches credentials or YAML payloads. --dry-run previews fixes. |
| Test wrapper | contentops test [--live] [-k EXPR] |
none — local | contentops/cli/commands/test_runner.py |
n/a (it runs tests) | docs/development/live-integration-tests.md |
Runs pytest with sensible defaults. With --live: doctor green check + RUN_LIVE_TESTS=1 + tests/integration/. |
The legacy v1 verbs validate, deploy, diff, and delete were
removed in R4 of the retirement plan. The verb mapping for runbook
authors is in
v1-to-v2-migration-guide.md;
the short version is: validate → plan, deploy → apply,
diff → drift, delete → prune.
The full per-asset coverage table — endpoint, RBAC, hash projection,
limitations, live-test status — is in
asset-coverage.md. Summary count: 6 asset
kinds registered (focused taxonomy after the Phase 1 reduction
from 27 broader kinds): sentinel_analytic, sentinel_hunting,
sentinel_watchlist, sentinel_parser, sentinel_data_connector,
defender_custom_detection. All 6 are write-capable; the deleted
read-only handlers (workspace manager, source controls, incidents,
incident tasks, watchlist items) can be rebuilt from git history if
needed.
Handler files live one-per-kind under
contentops/handlers/. The protocol is
documented in architecture.md.
KQL rules in contentops/lint/kql.py:
| ID | Severity | Catches |
|---|---|---|
| KQL001 | error | Unbalanced brackets (), [], {} (after stripping strings + comments). |
| KQL002 | error | Unterminated string at EOF (handles Kusto verbatim @"..." escapes). |
| KQL003 | error | Empty / comment-only query. |
| KQL004 | warning | project * (forces reading every column). |
| KQL005 | warning | Bare | take with no number. KQL101 (strict mode) catches the operator regardless of argument; KQL005 stays a warning at the heuristic tier so non-strict runs still see it. |
| KQL006 | warning | evaluate bag_unpack (expensive). |
| KQL007 | error | union * / union kind=inner * (reads every table). |
Payload rule in contentops/lint/payload.py:
| ID | Severity | Catches |
|---|---|---|
| PAYLOAD001 | error | templateVersion is set but alertRuleTemplateName is empty/missing. ARM rejects PUT with HTTP 400; this rule blocks at PR time. Added after the 2026-05-06 incident — see docs/archive/incidents/broken-analytics-2026-05-06.md. |
| PAYLOAD002 | warning | payload.displayName would produce a canonical slug longer than the 80-char cap in contentops.utils.slug.displayname_slug. When that happens the envelope id silently truncates and any cross-reference quoting the un-truncated form (hand-authored audit notes etc.) drifts. Advisory only -- the truncation is non-destructive at deploy time. Applies to sentinel_analytic / sentinel_hunting / defender_custom_detection. |
Snippet-substitution rules in contentops/lint/snippets.py (Phase 4 -- workspace-aware KQL snippet substitution):
| ID | Severity | Catches |
|---|---|---|
| KQLOVERRIDE001 | error | Placeholder must match exactly {{folder/file.yml}} (no spaces, .yml suffix required). |
| KQLOVERRIDE002 | error | Placeholder path must be relative -- no .. segments, no leading / or \. Defends against path-traversal during resolution. |
| KQLOVERRIDE003 | error | Placeholder must be the only non-whitespace token on its line (trailing // line comment tolerated). The both-missing fallback drops the entire line; a mid-line placeholder would silently delete surrounding KQL. |
| KQLOVERRIDE004 | error | Every file under overrides/**/*.yml must parse as YAML and contain a content: (string) key. |
Strict-mode policy rules in contentops/lint/strict_rules.py (gated by --strict):
| ID | Severity | Catches |
|---|---|---|
| KQL101 | error | | take N or | limit N in a deployed analytic/hunting query. Both cap the result set non-deterministically — a noisy rule looks well-behaved during tuning and genuine alerts get dropped silently in production. Fix: use top N by <field> for a deterministic bounded result, or drop the operator entirely. Pinned by tests/v2/test_lint_strict_take_limit.py. |
Lint runs against legacy v1 + v2 envelopes alike for payload rules,
but skips KQL/cost rules for v1 envelopes (legacy quality bar). See
the runner at contentops/lint/runner.py:51.
| Workflow file | Trigger | What it runs | Effect |
|---|---|---|---|
audit-verify.yml |
weekly (Mondays 04:00 UTC) + PR on audit/ + dispatch |
contentops audit verify --root . |
Fails on chain tamper. Status only. |
ci.yml |
PR + push to main | pytest unit, pip-audit, CLI smoke | Blocks merge on test failures or vulnerable deps. |
collect.yml |
weekly (Mondays 06:00 UTC) + dispatch | contentops collect --role <role> |
Opens PR if tree changed. Tree-change detection uses git status --porcelain (catches untracked files in fresh-kind directories). role input picks which workspace to snapshot. Bot commits use subject chore(collect): ... — deploy.yml skips on this subject. |
| coverage.yml | PR on detections/ or contentops/ + dispatch | contentops coverage | Posts sticky PR comment with MITRE heatmap. Workflow name: is mitre-attack-coverage — that's the required-check name in branch protection. |
| deploy.yml | push to main on detections/** + dispatch | contentops state sync pull → contentops apply --role prod (or --workspace <name> if input set) --changed-since <prev SHA> --json-report apply-report.json → contentops state sync push | The merge-to-prod path. A gate job runs first and skips the deploy when the head commit subject matches ^chore\((collect\|drift)\): or the author is github-actions[bot] — keeps merged collect/drift PRs from triggering a re-deploy of the data they just snapshot. Phase 8: dispatch input workspace lets operators pin one specific prod workspace by name (drops --role prod in favor of --workspace). Empty default = iterate every prod workspace. Uploads audit/*.jsonl and apply-report.json (90-day retention), and persists state/state.json to refs/heads/state/<env>. Concurrency-serialised; no cancel-in-progress. Subscription/RG/workspace resolved per-workspace from config/tenant.yml (no vars.AZURE_SUBSCRIPTION_ID). |
| integration-deploy.yml | PR on detections/** + dispatch | contentops apply --role integration --continue-on-error (PR: --changed-since base; dispatch: full, optional --workspace <name>) | Deploys changed detection content to the integration Sentinel workspace(s) so authoring mistakes are caught before merge to main. No-ops gracefully when no workspace with role: integration exists in config/tenant.yml. --continue-on-error keeps a broken rule from blocking a PR; failures still appear in the summary + audit. Dispatch defaults to dry-run. Phase 8: dispatch input workspace lets operators pin one specific integration workspace by name; PR-trigger runs always iterate every integration workspace. |
| drift.yml | daily 06:00 UTC + dispatch + PR on detections/** | contentops drift --write --report then contentops drift-pr-body (scheduled) / gh pr comment (PR mode) | Scheduled run opens a PR with collected drift and closes prior stale auto-drift PRs. PR-mode run posts an informational comment to the PR (environment: automation, no protection rules, no auto-PR). |
| emergency-disable.yml | dispatch (rule_id, reason, confirm="DISABLE") | contentops disable | Branch + PR. No auto-merge. Confirmation guard at line 54. |
| integration.yml | dispatch (with i-know-this-hits-prod) + PR labelled run-integration | pytest tests/integration/ with RUN_LIVE_TESTS=1, I_UNDERSTAND_THIS_IS_PRODUCTION | Live tenant calls. Production environment approval gate. |
| lint.yml | PR + push to main + nightly 08:00 UTC + dispatch | contentops lint | Status only. Errors gate merge. Warnings gating opt-in via fail_on_warn. |
| lock-unlock.yml | dispatch (action, rule_id) | contentops lock / contentops unlock | Branch + PR. No auto-merge. |
| portfolio.yml | nightly 07:00 UTC + dispatch | contentops portfolio | Uploads CSV + JSON (90-day retention). |
| production-promotion-check.yml | PR on detections/ | python scripts/detect_production_promotions.py | Sticky PR comment listing rules being promoted to production. Status only. |
| promote-to-integration.yml | dispatch (confirm="PROMOTE") | bootstrap → collect from prod → rewrite to integration → apply to integration | Defaults to dry-run. Integration environment approval gate. |
| prune.yml | dispatch (env, asset, dry_run, max_deletes, include_locked, confirm) | contentops prune | Defaults dry-run. Confirmation guard confirm == "CONFIRM". Per-env approval gate. |
| release.yml | tag v* | pip build + GitHub Release | Marks -rc / -beta tags as prerelease. |
| retry-failed.yml | dispatch (env, dry_run) | contentops retry-failed | Defaults dry-run. Per-env approval gate. |
| silent-rules.yml | weekly (Mondays 07:00 UTC) + dispatch | contentops silent-rules + contentops auto-disabled-rules | Uploads CSV + JSON telemetry reports for (a) silent rules in the lookback window and (b) rules Sentinel auto-disabled (NVISO Part 7). Read-only against the workspace; auto-derives workspace from tenant.yml --role prod. |
| spelling.yml | PR on detections/, contentops/, docs/, scripts/, .codespellrc | codespell | Catches typos in author-controlled prose before reviewers see them. Config + domain-term ignore list lives at .codespellrc. NVISO Part 3 borrowing. |
| references-check.yml | weekly (Saturdays 06:00 UTC) + dispatch | python scripts/check_references.py --format summary | HEAD-checks every URL in envelope metadata.references[] and runbookUrl. Reports broken links in the GitHub Actions step summary. PR-time variant in validate.yml checks only URLs added in the diff. NVISO Part 3 borrowing. |
| tuning-impact-preview.yml | PR on detections/drift_suppressions.yml | contentops tuning preview | Posts/updates a single PR comment with the 30-day blast-radius (incidents + alerts that would be silenced) for each new suppression entry. Fork PRs render with - instead of counts (OIDC unavailable). NVISO Part 8 borrowing. |
| validate.yml | PR on detections/, contentops/, config/ | contentops lint --strict + contentops plan | Blocks merge on lint errors, plan validation errors, dependency-graph violations. No --skip-deps-check on purpose. |
Cross-reference of CLI ↔ workflow conventions and parity is in
docs/reference/cli-workflow-matrix.md.
| Feature | What | Source files |
|---|---|---|
| Compliance frameworks | NIST CSF 2.0, ISO 27001:2022 control → detection mappings. | compliance/mappings/nist_csf.yml, compliance/mappings/iso27001.yml. |
| Dependency graph | Optional detections/dependencies.yml declares per-detection prerequisites (tables, watchlists, parsers, detections, external). Validated by contentops plan (and validate.yml). |
contentops/core/dependencies.py. |
| Tenant config | config/tenant.yml (v3 multi-workspace schema). One Entra ID tenant; 0–1 Defender XDR; 0–N Sentinel workspaces tagged role: prod\|integration\|dev. Identity (AZURE_CLIENT_ID, AZURE_TENANT_ID) lives in env vars / GitHub Variables, not the YAML; subscription is derived per-workspace from the file. Loaded by contentops/config.py; migration helper at scripts/migrate_tenant_config.py. See docs/operations/multi-workspace.md. | config/tenant.yml. |
| Script | What |
|---|
| scripts/remediate_payload001.py | Surgically remove dangling templateVersion lines from envelopes (PAYLOAD001 hotfix). Idempotent. |
| scripts/migrate_tenant_config.py | Convert a legacy single-workspace config/tenant.yml (top-level sentinel: block) to the v3 multi-workspace schema (sentinelWorkspaces: list with role:). Idempotent. Surfaced in docs/operations/multi-workspace.md. |
| scripts/check_version_bump.py | CI helper that enforces a version bump when a detection's content changes (no silent overwrites). |
| scripts/detect_production_promotions.py | PR comment generator listing detections being promoted to status: production. |
| scripts/check_references.py | HEAD-checks every URL in envelope metadata.references[] and runbookUrl. Two modes: full corpus (default; weekly cron) and --diff-base REF (PR-time, walks only URLs added in the diff). NVISO Part 3 borrowing. |