Where: faircode/manifest.py:58-61 (TargetSpec.compute, "equals"/"isin" methods).
The gap: neither method validates that its required companion field is actually set - value for "equals", values for "isin" - and the two failure modes are inconsistent with each other:
>>> from faircode.manifest import TargetSpec
>>> import pandas as pd
>>> df = pd.DataFrame({"flag": [1, 0, 1]})
>>> TargetSpec(column="flag", method="equals").compute(df).tolist() # no `value`
[0, 0, 0]
>>> TargetSpec(column="flag", method="isin").compute(df) # no `values`
TypeError: only list-like objects are allowed to be passed to isin(), you passed a `NoneType`
"equals" with a missing value silently compares every row to None and produces an all-zero label column with no error at all - the model would train on a constant target with no diagnostic anywhere. "isin" with missing values does raise, but with a raw pandas TypeError that never mentions audit.yaml or which manifest field is wrong.
Why it matters: an all-zero (or all-one) target is one of the most silent, hard-to-notice manifest mistakes possible - training/evaluation will "succeed" and produce numbers, just meaningless ones.
Suggested fix: validate the required companion field at TargetSpec construction (or manifest load time), raising a clear error for both "equals" without value and "isin" without values - matching ProtectedAttribute's existing "need disadvantaged_values or advantaged_values" validation style.
Where:
faircode/manifest.py:58-61(TargetSpec.compute,"equals"/"isin"methods).The gap: neither method validates that its required companion field is actually set -
valuefor"equals",valuesfor"isin"- and the two failure modes are inconsistent with each other:"equals"with a missingvaluesilently compares every row toNoneand produces an all-zero label column with no error at all - the model would train on a constant target with no diagnostic anywhere."isin"with missingvaluesdoes raise, but with a raw pandasTypeErrorthat never mentionsaudit.yamlor which manifest field is wrong.Why it matters: an all-zero (or all-one) target is one of the most silent, hard-to-notice manifest mistakes possible - training/evaluation will "succeed" and produce numbers, just meaningless ones.
Suggested fix: validate the required companion field at
TargetSpecconstruction (or manifest load time), raising a clear error for both"equals"withoutvalueand"isin"withoutvalues- matchingProtectedAttribute's existing "need disadvantaged_values or advantaged_values" validation style.