coverage-thresholds.json

{
  "_agent_pmo": "74cf183",
  "_doc": "Single source of truth for code coverage thresholds. See REPO-STANDARDS-SPEC [COVERAGE-THRESHOLDS-JSON]. NO GitHub repo variables. NO env vars. This file is read by the internal _coverage_check recipe inside make test. make test exits non-zero if measured < threshold. Thresholds ratchet UP only.",
  "default_threshold": 90,
  "projects": {
    "basilisk-checker": {
      "threshold": 93
    },
    "basilisk-cli": {
      "threshold": 95
    },
    "basilisk-db": {
      "threshold": 100
    },
    "basilisk-lsp": {
      "threshold": 88
    },
    "basilisk-parser": {
      "threshold": 100
    },
    "basilisk-resolver": {
      "threshold": 95
    },
    "basilisk-stubs": {
      "threshold": 93
    },
    "basilisk-config": {
      "threshold": 97
    },
    "vsix": {
      "threshold": 99
    },
    "nvim": {
      "threshold": 47
    }
  },
  "conformance": {
    "_doc": "DEAD AS OF 2026-08-08: this measurement CANNOT RUN. Basilisk's withdrawal from python/typing removed BasiliskTypeChecker from the suite's conformance/src/type_checker.py, whose TYPE_CHECKERS tuple now registers only mypy, pyright, zuban, pyrefly, pycroscope and ty. `--only-run` is matched by name against that tuple (conformance/src/main.py: `if options.only_run and options.only_run != type_checker.name: continue`), so `--only-run basilisk` is not an argparse error — it matches NOTHING: main.py exits 0 having graded no checker and written no results/basilisk/*.toml, and run_conformance.py's run_harness() then raises \"the real harness wrote no results ... it did not run\". The conformance jobs in .github/workflows/release.yml and the two conformance passes in scripts/test-rust.sh are COMMENTED OUT for this reason — a gate that can only fail must not block releases. Restoring the measurement would require vendoring a scorer or injecting an adapter, which [CHKARCH-CONFORMANCE] declares a BUILD FAILURE. Whether this block is deleted outright or replaced by a disclosed non-official harness is the user's decision, not an agent's. The rest of this note describes the mechanism as it worked while upstream still carried the adapter. Live PEP conformance measurement. POLICY: the number is a REGRESSION DETECTOR, never a target — see [CHKARCH-CONFORMANCE]. It samples one fixed corpus the checker was historically developed against, so it cannot tell you whether a rule analyses code; only whether today's binary agrees with yesterday's on files it has already seen. NEVER publish, quote, or market this figure. conformance/run_conformance.py clones python/typing@main FRESH on every run, then runs the suite's OWN unmodified src/main.py --only-run basilisk against the compiled binary — via the adapter the suite USED to carry — and records the exact graded commit in website/src/_data/conformance_report.json. The result is the REAL harness's OWN verdict, produced by the same code that grades pyright/mypy/pyrefly/ty/zuban/pycroscope: a file passes only when its errors_diff is empty, counting every diagnostic the binary emits (errors AND warnings). There is NO vendored calculator and NO cached-fixtures fallback — if the real harness cannot be cloned and run, the build FAILS. The binary runs in its default configuration — the pure PEP set; Basilisk's opt-in house-style rules never run (see [CHKARCH-CONFIGURATION-ONLY]). Configuring a rule off before measuring, hand-editing conformance_status.csv, or editing this block to match a run is forbidden. KNOWN CONTRADICTION: `threshold` below is a pass-percentage floor, and a floor over a corpus the code was fitted to is the incentive that produced the fitted predicates (CONFORMANCE-INTEGRITY-AUDIT §6.3). Deleting a rule that decides from source text rather than resolved symbols is REQUIRED ([CHKARCH-TEXT-MATCHED-LOGIC]) and is expected to LOWER this number — which this floor turns into a build failure. Removing the floor is the user's decision, not an agent's. Until they decide: make the deletion, report the drop and the failing gate, and stop there. Do NOT restore the code, refit the rule, or lower this value to get green.",
    "threshold": 100,
    "_fp_ceiling_doc": "Total false-positive diagnostics across the suite (diagnostics Basilisk reports on a line the suite does NOT mark # E, or outside a satisfied # E[tag] group). Measured by conformance/run_conformance.py --gate, which runs the REAL python/typing harness on the compiled binary and delegates the comparison to conformance/assert_wheel_conformance.py (run by scripts/test-rust.sh inside make test). A false positive on real code is a genuine defect worth fixing on its own merits, independent of this suite. Same contradiction as `threshold` above: close a gap by fixing the checker or by deleting logic that never analysed anything — never by silencing a rule to hold the ceiling.",
    "max_false_positives": 0
  }
}