feat(orchestration): allow quarantine_max_ratio tolerance in readiness gate by AJ-EthereaLogic-ai · Pull Request #48 · Org-EthereaLogic/DriftSentinel

Anthony Johnson II (AJ-EthereaLogic-ai) · 2026-05-05T16:28:14Z

Closes #36.

Summary

Dataset contract gains an optional quarantine_max_ratio knob (default 0.0, range [0.0, 1.0]) — backward-compatible zero-tolerance behavior preserved when the key is absent.
_validate_dataset_readiness raises only when the schema is invalid or quarantine_ratio > quarantine_max_ratio. Schema violations still fail closed regardless of the threshold.
Intake evidence records quarantine_max_ratio and tolerance_applied so audits see the gate decision; quarantine_ratio retains its current semantics. run_dataset_intake's PASS/FAIL verdict honors the same threshold so the intake artifact and the drift/benchmark gate agree.
Loader (_validate_quarantine_max_ratio) rejects out-of-range, non-numeric, boolean, and non-finite values at load time with a ConfigError naming the key.

Why

NYC TLC iter-1 demo evidence: 66,651 of 3 M rows duplicated on the 4-col business key (~2.2%) — a real-world floor, not a defect. The previous zero-tolerance gate forced operators to either pre-clean parquet (silently deleting evidence), widen the business key to 8 columns (distorting the contract), or drop the key entirely (disabling the check). The single-knob ratio resolves all three.

Plan

specs/DS-PATCH-036_quarantine_max_ratio_tolerance.md — full design, validation, and acceptance criteria.
specs/DS-TM-001_Traceability_Matrix.md — DS-FR-005 / DS-SR-002 row links DS-PATCH-036; v1.4 changelog entry.

Test plan

uv run ruff check . — All checks passed
uv run mypy src/driftsentinel tests — Success: no issues found in 60 source files
uv run pytest — 487 passed
New tests/test_orchestration.py::TestValidateDatasetReadiness covers all four ratio cases (0/0, 0/>0, 5%/3%, 5%/7%) plus the schema-violation invariant and the tolerance_applied flag-state matrix.
New tests/test_dataset_orchestration.py::TestIntakeToleranceEvidence asserts the new fields land in the written intake envelope.
New tests/test_config_loading.py cases cover the loader-boundary rejections (out-of-range, non-numeric, boolean, negative) and the absence-of-key default.
tests/test_intake.py gains a known-fraction test locking quarantine_ratio rounding semantics relied on by the new gate.

🤖 Generated with Claude Code

…s gate (closes #36) Dataset contracts gain an optional quarantine_max_ratio knob (default 0.0, range [0.0, 1.0]). _validate_dataset_readiness now raises only when the schema is invalid OR quarantine_ratio exceeds the threshold, so real-world feeds with known low-rate quirks (e.g. NYC TLC's ~2.2% true business-key duplicates) no longer force operators to pre-clean files, widen business keys, or drop the key entirely. Schema-level violations still fail closed regardless of the threshold. Intake evidence records quarantine_max_ratio and tolerance_applied so audits can see the gate decision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

codacy-production · 2026-05-05T16:29:14Z

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 53 complexity · 7 duplication

Metric Results

Complexity 53

Duplication 7

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

_{TIP This summary will be updated as you push new changes.}

+    if isinstance(raw, bool) or not isinstance(raw, (int, float)):
+        raise ConfigError(f"contract.quarantine_max_ratio must be a number in [0.0, 1.0] ({context}); got {raw!r}")
+    value = float(raw)
+    if value != value or value in (float("inf"), float("-inf")):  # NaN or inf


codecov · 2026-05-05T16:30:39Z

Codecov Report

❌ Patch coverage is 91.30435% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/driftsentinel/orchestration/runner.py	87.87%	3 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

codacy-production

Pull Request Overview

The pull request introduces the quarantine_max_ratio parameter to allow configurable tolerance for quarantine violations during readiness checks. While the core logic aligns with the requirements, the implementation is currently not up to standards according to Codacy analysis. The most significant issue is the non-idiomatic numeric validation in src/driftsentinel/config/loader.py, which is flagged by both Pylint and Semgrep. Additionally, the loader has seen a significant increase in complexity. There are also minor opportunities for code cleanup in the orchestration runner regarding redundant casting and string formatting. Addressing the static analysis findings in the config loader is required to meet quality standards.

About this PR

Codacy analysis indicates this PR is not up to standards. This is primarily driven by the new static analysis issues in the config loader and significant complexity increases in loader.py (+9) and runner.py (+7).

Test suggestions

Verify default (0.0) ratio blocks any quarantined rows (backward compatibility)
Verify ratio below threshold passes with tolerance_applied set to True
Verify ratio above threshold fails with measured ratio in error message
Verify schema violation fails even if threshold is 1.0 (strict schema invariant)
Verify loader rejects invalid types (booleans, strings) and out-of-range numbers
Verify intake evidence artifact contains the new decision fields
Verify quarantine_ratio rounding/fractional logic is deterministic for the gate

_{TIP Improve review quality by adding custom instructions}
_{TIP How was this review? Give us feedback}

codacy-production · 2026-05-05T16:31:03Z

+    if isinstance(raw, bool) or not isinstance(raw, (int, float)):
+        raise ConfigError(f"contract.quarantine_max_ratio must be a number in [0.0, 1.0] ({context}); got {raw!r}")
+    value = float(raw)
+    if value != value or value in (float("inf"), float("-inf")):  # NaN or inf


_{🔴 HIGH RISK}

Suggestion: Use math.isfinite() to check for NaN and infinity values. This replaces the manual value != value comparison and the explicit check against float('inf') with a single, clear standard library call that returns False for both NaN and infinite values.

Try running the following prompt in your IDE agent:

In src/driftsentinel/config/loader.py, refactor the NaN and infinity check on line 113 to use math.isfinite(value). Ensure the math module is imported at the top of the file.

See Issue in Codacy
See Issue in Codacy

codacy-production · 2026-05-05T16:31:03Z

            f"{dataset_label} does not satisfy the registered contract. "
-            f"Fix the {dataset_label.lower()} before running drift or benchmark. "
-            + "; ".join(details)
+            f"Fix the {dataset_label.lower()} before running drift or benchmark. " + "; ".join(details)


_{⚪ LOW RISK}

Suggestion: Simplify string concatenation by moving the joined list into the f-string.

codacy-production · 2026-05-05T16:31:03Z

+            details.append(f"quarantine_ratio={float(evaluation['quarantine_ratio']):.4f}")
+            details.append(f"quarantine_max_ratio={float(evaluation['quarantine_max_ratio']):.4f}")


_{⚪ LOW RISK}

Nitpick: Remove redundant float() casts as these values are already guaranteed to be floats.

…37 closed - #36 closed via PR #48 (commit 95b9002) - #37 closed via PR #49 (commit b4d5b82) - Reconciled project Status field for #35 (In Progress → Done) - No documentation drift fixed; no iteration carry-over - Validation: lint, typecheck, 491 tests pass; bundle SKIPPED (no Databricks auth) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-code-quality Bot found potential problems May 5, 2026

View reviewed changes

codacy-production Bot reviewed May 5, 2026

View reviewed changes

fix(config): use math predicates for quarantine ratio validation

cca9fdf

Anthony Johnson II (AJ-EthereaLogic-ai) merged commit 95b9002 into main May 5, 2026
11 checks passed

Anthony Johnson II (AJ-EthereaLogic-ai) deleted the feat/issue-36-quarantine-max-ratio-tolerance branch May 5, 2026 16:37

Navigation Menu

Uh oh!

Uh oh!

feat(orchestration): allow quarantine_max_ratio tolerance in readiness gate - #48

feat(orchestration): allow quarantine_max_ratio tolerance in readiness gate#48
Anthony Johnson II (AJ-EthereaLogic-ai) merged 2 commits into
mainfrom
feat/issue-36-quarantine-max-ratio-tolerance

Anthony Johnson II (AJ-EthereaLogic-ai) commented May 5, 2026

Uh oh!

codacy-production Bot commented May 5, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 5, 2026 •

edited

Loading

Uh oh!

codacy-production Bot left a comment

Uh oh!

codacy-production Bot May 5, 2026

Uh oh!

codacy-production Bot May 5, 2026

Uh oh!

codacy-production Bot May 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		details.append(f"quarantine_ratio={float(evaluation['quarantine_ratio']):.4f}")
		details.append(f"quarantine_max_ratio={float(evaluation['quarantine_max_ratio']):.4f}")

Uh oh!

Uh oh!

Conversation

Anthony Johnson II (AJ-EthereaLogic-ai) commented May 5, 2026

Summary

Why

Plan

Test plan

Uh oh!

codacy-production Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Up to standards ✅

Uh oh!

codecov Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codacy-production Bot left a comment

Choose a reason for hiding this comment

Pull Request Overview

About this PR

Test suggestions

Uh oh!

codacy-production Bot May 5, 2026

Choose a reason for hiding this comment

Uh oh!

codacy-production Bot May 5, 2026

Choose a reason for hiding this comment

Uh oh!

codacy-production Bot May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

codacy-production Bot commented May 5, 2026 •

edited

Loading

codecov Bot commented May 5, 2026 •

edited

Loading