Add dmc-prepop CLI — Phase 1: SCCM extraction

[adamswbrown]

Summary

• New dmc-prepop Python CLI under dmc-prepop/. Extracts System Center inventory into CSV files for Dr Migrate (DMC) pre-population.
• Phase 1 only — SCCM module. SCOM CMDB (Phase 2) and SCOM DW (Phase 3) will land in follow-up PRs.
• Implements all 8 queries from the technical spec: servers, CPU, storage, network, software, services, roles, patches.

Architecture

• dmc_prepop/db.py — Connection abstraction. Default driver is pymssql (no ODBC install, works on Linux). --auth integrated flips to pyodbc with Windows Integrated Auth (requires pip install -e .[odbc]).
• dmc_prepop/output.py — Streaming CSV writer. UTF-8 with BOM, comma-delimited, quoted strings, csv.QUOTE_NONNUMERIC to match the spec's documented DMC import format.
• dmc_prepop/normalize.py — Hostname normaliser (lowercase, strip whitespace, strip trailing $, NetBIOS-only). Applied to every Hostname-bearing column so SCCM ↔ SCOM ↔ SCOMDW joins work without further normalisation downstream.
• dmc_prepop/sccm/queries.py — All 8 queries as Query dataclasses. Each lists its required_views so the extractor can schema-verify before execution.
• dmc_prepop/sccm/extract.py — For each query: check INFORMATION_SCHEMA.VIEWS for required views, skip with a clear log message if any are missing, otherwise run in streaming fetchmany mode and write CSV.
• dmc_prepop/coverage.py — Emits coverage_summary.json with per-query status (executed / skipped / failed), row counts, missing views, and distinct host count from the spine. Phase 4 will add cross-source gap analysis.
• dmc_prepop/cli.py — Click-based CLI. Phase 1 ships the sccm subcommand; future phases add scom / scom-dw / all.

Outputs (8 CSVs + summary)

sccm_servers.csv, sccm_cpu.csv, sccm_storage.csv, sccm_network.csv, sccm_software.csv, sccm_services.csv, sccm_roles.csv, sccm_patches.csv, coverage_summary.json.

Filenames match the spec exactly so downstream merge tooling can be built against a stable contract.

Spec adherence notes

• Query SQL is verbatim from the spec (Queries 1.1–1.8) with one safe rewrite: Query 1.8's COUNT(CASE WHEN ucs.Status = 2 THEN 1 END) is changed to SUM(CASE WHEN ucs.Status = 2 THEN 1 ELSE 0 END) — semantically equivalent for this counter pattern, more portable across SQL Server versions.
• Universal server filter (Operating_System_Name_and0 LIKE '%Server%' AND Obsolete0 = 0 AND Active0 = 1) is applied to every query that joins v_R_System.
• --timestamp-output flag implemented per spec implementation note 7 (idempotency).
• Schema verification per spec implementation note 3 — INFORMATION_SCHEMA.VIEWS checked before each query, graceful skip with logged reason.
• Per-query row counts streamed to stderr via logging; coverage summary written to JSON at end of run.

Deferred to follow-up PRs

• Phase 2 — SCOM CMDB module (Queries 2.1–2.7, including MTV_* runtime view enumeration).
• Phase 3 — SCOM DW module (Queries 3.1–3.6) with paged date-range streaming for Perf.vPerfHourly (millions of rows).
• Phase 4 — Cross-product hostname-keyed merge helper and full coverage gap analysis (SCCM-not-SCOM, SCOM-not-SCCM, SQL/IIS discovery counts).

Test plan

• pip install -e dmc-prepop succeeds
• dmc-prepop --help and dmc-prepop sccm --help render
• Run against a real (non-prod) SCCM site DB: dmc-prepop sccm --server <db> --database CM_<site> --user <reader> --password <...> --out output/
• Verify all 8 CSVs are produced (or cleanly skipped with missing_views recorded in coverage_summary.json)
• Open sccm_servers.csv in Excel and confirm UTF-8 BOM, headers, and Hostname normalisation (lowercase, NetBIOS only)
• Spot-check that virtual/physical classification populated correctly for known VMs
• Run with --timestamp-output and confirm dated subdirectory created
• Try a site missing one of the optional views (e.g. v_GS_SERVER_FEATURE) and confirm graceful skip rather than crash
• Confirm coverage_summary.json has correct executed/skipped/failed counts and per-query row totals

https://claude.ai/code/session_01JgkCjwqTFGQBra1unq3d8v

---

Generated by Claude Code