Skip to content

Deepen tests (32->164) & demos (5->20), harden errors, fix 3 detection/UX bugs#3

Open
cognis-digital wants to merge 1 commit into
mainfrom
feat/deepen-tests-demos-hardening
Open

Deepen tests (32->164) & demos (5->20), harden errors, fix 3 detection/UX bugs#3
cognis-digital wants to merge 1 commit into
mainfrom
feat/deepen-tests-demos-hardening

Conversation

@cognis-digital

Copy link
Copy Markdown
Owner

Summary

This PR roughly 4x's the depth of the repo with real work: it grows the test
suite from 32 -> 164 tests, the demo suite from 5 -> 20 narrated scenarios,
hardens error handling, and fixes three genuine detection/UX bugs -- all with the
public API unchanged.

Bug fixes

  1. SSNs embedded in free text were not detected. The SSN detector used an
    anchored ^\s*\d{3}-\d{2}-\d{4}\s*$ regex, so it only matched when the SSN
    was the entire cell. An SSN buried in a clinical note ("... SSN 123-45-6789 to redact") escaped -- the exact free-text leak the tool claims to catch.
    Added a bounded search variant. The 3-2-4 dashed shape is distinct from phone
    numbers, so the search is false-positive safe.
  2. Bare 3-digit numbers > 89 were mis-flagged as "age > 89" (S3). A value
    like 200 (a count, dosage, code, or a ZIP3 prefix such as 606) was flagged
    as a patient age. Constrained the age heuristic to a plausible human age
    (90..125; the oldest verified human age is 122), eliminating the false
    positives while still catching real ages >= 90.
  3. --delimiter '\t' crashed with an uncaught TypeError. Passing the
    literal two-character token \t (a very common shell mistake) produced a
    traceback and a bare exit 1. The CLI now resolves \t/tab to a real tab,
    and analyze_csv validates the delimiter is a single character with a clear
    ValueError.

Error handling

  • analyze_csv raises a clear ValueError on an empty file (no header row) and
    rejects non-single-character delimiters.
  • The CLI validates -k/-l are positive, that -l is paired with
    --sensitive, and -- importantly -- that -k/-l are given
    --quasi-identifiers. Previously, gating on -k with no --qi silently
    PASSED
    , a dangerous false negative for a tool used as a CI gate. It now also
    catches IsADirectoryError/OSError and prints a readable message.

Tests (32 -> 164)

New focused suites: test_k_anonymity.py, test_l_diversity.py (incl. combined
sensitive tuples), test_safe_harbor.py (all 18 categories, value + header
detection, the new regression cases), test_malformed.py (empty/header-only/
ragged/BOM/unicode/duplicate-header datasets and error paths), test_cli.py
(every documented exit code + argument validation + output formats),
test_sarif_edge.py (rule catalog, DEID-K/L results, path normalization), and
test_analyze.py (pass/fail composition, smallest_classes). python -m pytest -q is green.

Demos (5 -> 20)

Fifteen new runnable, offline, exit-0 scenarios (see docs/DEMOS.md), plus new
synthetic fixtures under demos/11-14. demos/run_all.py runs all twenty end to
end under PYTHONUTF8=1.

All fixtures are synthetic. No real patient data anywhere in this repository.

Tests: 32 -> 164. New focused suites for k-anonymity math, l-diversity
math (including combined sensitive tuples), the 18 Safe Harbor rules
(value + header detection), malformed/edge-case datasets, CLI exit codes
and argument validation, SARIF 2.1.0 edge cases, and analyze_rows
pass/fail composition. python -m pytest -q is green.

Demos: 5 -> 20 narrated, offline, exit-0 scenarios covering payer/claims,
genomics technical ids, CI exit codes, JSON pipeline consumption, free-text
leaks, cell suppression, non-comma delimiters, device telemetry, the k/l
trade-off, operator errors, the full Safe Harbor catalog, before/after
remediation, clean-vs-dirty SARIF, small-cohort singletons, and multi-
attribute QIs. run_all.py and docs/DEMOS.md updated; new synthetic fixtures
added under demos/11-14.

Bug fixes (public API unchanged):
1. SSN detection missed SSNs embedded in free text (the anchored ^...$
   regex only matched whole cells). Added a bounded search so an SSN inside
   a clinical note is caught - the exact free-text leak the tool claims to
   find. The 3-2-4 dashed shape is distinct from phone numbers, so the
   search is false-positive safe.
2. Any bare 3-digit number >89 (e.g. a code, count, dosage, or ZIP3 prefix
   like 200/606) was mis-flagged as 'age > 89' (S3). Constrained the age
   heuristic to a plausible human age (90..125; oldest verified human is 122)
   to remove the false positives.
3. Passing --delimiter '\t' (a literal two-char token, a common shell
   mistake) crashed with an uncaught TypeError and exit 1 and no message.
   The CLI now resolves the \t / tab tokens to a real tab, and analyze_csv
   validates the delimiter is a single character with a clear ValueError.

Error handling:
- analyze_csv now raises a clear ValueError on an empty file (no header row)
  and rejects a non-single-character delimiter.
- The CLI validates -k/-l are positive, that -l has --sensitive, and that
  -k/-l are given quasi-identifiers (previously gating on -k with no --qi
  silently PASSED - a dangerous false negative for a CI gate). It also
  catches IsADirectoryError/OSError and prints a readable error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant