feat: add excluded labels for entities in Anonymizer

.copyrightignore

@@ -12,3 +12,5 @@ CHANGELOG.md
 .cursor/
 .claude/
 .agent/
+skills/anonymizer/BENCHMARK.md
+skills/anonymizer/skill-card.md

docs/concepts/choosing-a-strategy.md

@@ -43,11 +43,12 @@ What to include:
 - The domain (clinical, legal, financial, customer support, etc.)
 - The genre (notes, transcripts, opinions, biographies)
 - Anything about the source the engine couldn't infer from a single record (e.g. "transcribed phone calls — expect disfluencies")
-- `data_summary` is the only way to provide a soft do-not-tag list for the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII").
+- `data_summary` is a soft way to guide the augmenter when `entity_labels=None` — the augmenter is free to invent labels beyond `DEFAULT_ENTITY_LABELS`, so use it to tell the LLM what *not* to tag (e.g. "do not tag generic anatomical terms, medication class names, or job titles as PII"). For a hard exclusion of specific label types, use `Detect.excluded_entity_labels` instead.
 
 What to leave out:
 
 - Lists of entity types **you want detected** (those go in `Detect.entity_labels`)
+- Lists of entity types **you never want detected** (those go in `Detect.excluded_entity_labels`)
 - Privacy/utility goals (those go in `Rewrite.privacy_goal`)
 - Substitute behavior instructions (e.g. "names should remain Portuguese", "preserve numeric magnitude") — those go in `Substitute(instructions)`
 - Generic phrasing ("text data" adds no signal)
@@ -78,6 +79,21 @@ from anonymizer import DEFAULT_ENTITY_LABELS, Detect
 detect = Detect(entity_labels=[*DEFAULT_ENTITY_LABELS, "clinical_facility", "diagnosis_code", "medication_name"])
 ```
 
+### `excluded_entity_labels`
+
+Use when you want to **exclude** specific label types from detection without enumerating the entire allowlist. Excluded labels are removed before GLiNER runs, so they are never detected, augmented, or surfaced in results. The evaluation judges also ignore excluded label types so they don't lower your coverage score.
+
+```python
+# Never detect occupation or gender, keep everything else
+Detect(excluded_entity_labels=["occupation", "gender"])
+
+# Combine with an explicit allowlist — exclusions always win
+Detect(entity_labels=["first_name", "email", "city"], excluded_entity_labels=["city"])
+```
+
+!!! warning
+    `excluded_entity_labels` is always checked against the effective allowlist — `entity_labels` if set, otherwise `DEFAULT_ENTITY_LABELS`. A partial overlap just drops the shared labels and logs a warning. If the overlap is total, leaving an empty effective detection set, `Detect` raises a `ValueError` at config time instead of silently detecting nothing.
+
 ### `gliner_threshold`
 
 Default `0.3`. The validator catches false positives downstream, so erring low is safe.

docs/concepts/detection.md

@@ -44,6 +44,7 @@ config = AnonymizerConfig(
 | Field | Default | Description |
 |-------|---------|-------------|
 | `entity_labels` | `None` (all defaults) | List of labels to detect. Leave unset (or pass `None`) to use the full default set. |
+| `excluded_entity_labels` | `None` | List of labels to **never** detect, even if present in `entity_labels` or the default set. Excluded labels are removed before GLiNER and the LLM prompts run, and are also filtered from the final entity output as a safety net. |
 | `gliner_threshold` | `0.3` | GLiNER confidence threshold (0.0--1.0). Lower values detect more entities but may increase false positives. |
 | `validation_max_entities_per_call` | `100` | Maximum candidate entities per validator LLM call. Rows with more candidates are split into chunks. See [Chunked validation](#chunked-validation). |
 | `validation_excerpt_window_chars` | `500` | Characters of context included before and after a chunk's entity spans in the validator prompt. Bounds per-chunk prompt size; not the model's context-window limit. |
@@ -104,6 +105,22 @@ Detect(entity_labels=["first_name", "last_name", "email"])
 # Permissive: detect all defaults + LLM can infer new label types
 Detect()  # entity_labels=None
 ```
+
+### Excluding entity labels
+
+Use `excluded_entity_labels` to omit specific labels from detection without having to enumerate the entire allowlist. Excluded labels are removed before GLiNER runs and before the LLM prompts are built, so they are never detected or augmented.
+
+```python
+# Detect all defaults except occupation and gender
+Detect(excluded_entity_labels=["occupation", "gender"])
+
+# Combine with an explicit allowlist — exclusions always win
+Detect(entity_labels=["first_name", "email", "city"], excluded_entity_labels=["city"])
+```
+
+!!! warning
+    `excluded_entity_labels` is always checked against the effective allowlist — `entity_labels` if set, otherwise `DEFAULT_ENTITY_LABELS`. A partial overlap just drops the shared labels and logs a warning. If the overlap is total, leaving an empty effective detection set, `Detect` raises a `ValueError` at config time instead of silently detecting nothing.
+
 ## Tuning the threshold
 
 For `gliner_threshold`, start with the default `0.3`. If you're seeing too many false positives, raise it to `0.5`. If entities are being missed, try lowering to `0.2`. The LLM validation step catches many false positives, so erring on the side of lower thresholds is usually safe.

docs/concepts/evaluation.md

@@ -66,6 +66,7 @@ Note: the judge measures detection recall, not output leakage. A value detected
 The judge is scoped and contextualized by the same signals used during anonymization:
 
 - **`entity_labels`** — the detection taxonomy in scope; the judge only reports values whose type falls within it.
+- **`excluded_entity_labels`** — labels explicitly excluded from detection; the judge ignores entities of these types so excluded labels are never penalised in the coverage score.
 - **`data_summary`** — used purely to interpret literal values and their semantic types, never to invent entities absent from the text.
 
 | Output column | Type | Description |

docs/troubleshooting.md

@@ -116,9 +116,13 @@ Verify by re-running `preview` with `Annotate` and confirming the entity now app
 
 Symptoms: detected entities include obvious common words, dates that aren't dates, etc.
 
-1. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
-2. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
-3. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.
+1. **Use `Detect.excluded_entity_labels`** if a whole label type is systematically noisy for your data (e.g. `occupation` tagging generic job words, `age` tagging durations). This is the cleanest fix — excluded labels are removed before GLiNER runs and never appear in results.
+   ```python
+   Detect(excluded_entity_labels=["occupation", "age"])
+   ```
+2. **Raise `gliner_threshold`** to `0.5`. The augmenter will pick up real misses, so this rarely costs recall.
+3. **Lower `validation_excerpt_window_chars`** (default `500`) if context-driven validation is being misled by far-away sentences. Smaller per-chunk prompts trade context for precision.
+4. **Sanity-check the validator with an `Annotate` preview.** A flaky validator (or a misconfigured alias) returns "keep" on almost everything, which presents as recall going way up — easiest spotted by eyeballing the entity list on a handful of rows.
 
 ### A new domain isn't being detected well
 

skills/anonymizer/BENCHMARK.md

@@ -1,85 +1,125 @@
-<!-- SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
-<!-- SPDX-License-Identifier: Apache-2.0 -->
+# Skill Benchmark: anonymizer
 
-# Evaluation Report
+> ⚠️ **Overall verdict: INCOMPLETE — Required evidence is missing**
 
-Evaluation report for the `anonymizer` skill before publication through
-NVSkills-Eval.
+One or more required evaluation tiers did not complete, so this benchmark is not publication-complete.
 
-This benchmark file records the publication-ready evaluation plan and task
-composition for NeMo Anonymizer. The external NVSkills-Eval run has not been
-executed in this local workspace, so this branch intentionally reports no
-Anonymizer scores.
-
-## Evaluation Summary
+## Evaluation Metadata
 
 - Skill: `anonymizer`
-- Evaluation date: pending external `/nvskills-ci` run
-- NVSkills-Eval profile: external
-- Environment: external NVSkills-Eval runner
-- Dataset: 6 evaluation tasks
-- Attempts per task: recorded by external NVSkills-Eval after execution
-- Pass threshold: recorded by external NVSkills-Eval after execution
-- Overall verdict: pending external NVSkills-Eval run
-
-## Agents Used
-
-Agent-level measured results are pending the external NVSkills-Eval run.
-
-## Metrics Used
-
-Reported benchmark dimensions:
-
-- Security: checks whether skill-assisted execution avoids unsafe behavior such
-  as secret leakage, destructive commands, or unauthorized access.
-- Correctness: checks whether the agent follows the expected workflow and
-  produces the correct final output.
-- Discoverability: checks whether the agent loads the skill when relevant and
-  avoids using it when irrelevant.
-- Effectiveness: checks whether the agent performs measurably better with the
-  skill than without it.
-- Efficiency: checks whether the agent uses fewer tokens and avoids redundant
-  work.
-
-Underlying evaluation signals will be recorded from the external
-NVSkills-Eval output after execution.
-
-## Test Tasks
-
-The benchmark dataset contains 6 evaluation tasks:
-
-- Positive tasks: 4 tasks where the skill is expected to activate.
-- Negative tasks: 2 tasks where no skill is expected.
-- Unlabeled tasks: 0 tasks where positive/negative intent cannot be inferred.
-
-Entries with `should_trigger: true` and `expected_skill: "anonymizer"` are
-positive skill-activation cases. Entries with `should_trigger: false` and
-`expected_skill: null` are negative activation cases.
-
-## Results
-
-External NVSkills-Eval execution is pending. No copied or locally inferred
-Anonymizer results are reported here.
-
-| Dimension | Tasks | Result |
-|---|---:|---|
-| Security | 6 | Pending external NVSkills-Eval run |
-| Correctness | 6 | Pending external NVSkills-Eval run |
-| Discoverability | 6 | Pending external NVSkills-Eval run |
-| Effectiveness | 6 | Pending external NVSkills-Eval run |
-| Efficiency | 6 | Pending external NVSkills-Eval run |
-
-## Tier 1: Static Validation Summary
-
-Local static validation is covered by this branch's validation evidence. The
-external NVSkills-Eval Tier 1 result is pending the `/nvskills-ci` run.
+- Evaluation date: 2026-09-10
+- Evaluator version: `1.5.6`
+- Agents: Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`), Codex (`openai/openai/gpt-5.5`)
+- Tasks: 6 evaluation tasks (4 positive, 2 negative)
+- Dataset digest: `sha256:c2c13b2d794c6117dac0402f1261bd2d80972085c5e716426bedff6b3d59b8ae` (skill-evaluator-dataset-snapshot/1)
+- Attempts per task: 3
+- Environment: `k8s-sandbox`
+- Tier 2 evidence: required for publication
+- Tier 3 evidence: required for publication
+
+Each task attempt ran in its own isolated sandbox pod.
+
+## What This Report Answers
+
+The three-tier evaluation checks whether the skill:
+
+- is safe to use;
+- produces correct answers;
+- is discovered and activated when needed;
+- helps the agent complete the user's goal and expected workflow; and
+- avoids wasted skill and tool usage.
+
+## Results at a Glance
+
+| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |
+|---|---:|---:|
+| Overall | 89.8% — baseline ran, but no comparable score was available; uplift unavailable | 89.0% — baseline ran, but no comparable score was available; uplift unavailable |
+| Security | 95.0% → 100.0% (+5.0 points) | 100.0% → 83.3% (-16.7 points) |
+| Correctness | 46.0% → 86.7% (+40.7 points) | 75.0% → 100.0% (+25.0 points) |
+| Discoverability | 97.5% — baseline ran, but no comparable score was available; uplift unavailable | 93.8% — baseline ran, but no comparable score was available; uplift unavailable |
+| Effectiveness | 37.6% → 80.3% (+42.7 points) | 51.3% → 87.8% (+36.5 points) |
+| Efficiency | 84.3% — baseline ran, but no comparable score was available; uplift unavailable | 79.8% — baseline ran, but no comparable score was available; uplift unavailable |
+
+**How to read this table:** baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.
+
+Example: `47.0% → 92.0% (+45.0 points)` means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.
+
+A partial dimension was calculated from only the available configured signals; review the detailed report before relying on it.
+
+## Token Usage
+
+Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.
+
+| Agent | Dataset case | With skill | Without skill | Delta | Change | Coverage |
+|---|---|---:|---:|---:|---:|---|
+| claude-code | All cases | 1,699,362 | 2,863,530 | N/A | N/A | skill 6/6; base 10/10 |
+| claude-code | anonymizer-negative-general-privacy-explainer | 30,472 | 30,464 | +8 | +0.03% | skill 1/1; base 1/1 |
+| claude-code | anonymizer-negative-repository-source-development | 261,745 | 150,049 | +111,696 | +74.44% | skill 1/1; base 1/1 |
+| claude-code | anonymizer-positive-failed-records-first | 185,130 | 431,085 | N/A | N/A | skill 1/1; base 3/3 |
+| claude-code | anonymizer-positive-hash-cross-record-consistency | 456,894 | 31,819 | +425,075 | +1335.92% | skill 1/1; base 1/1 |
+| claude-code | anonymizer-positive-mode-choice | 361,402 | 32,805 | +328,597 | +1001.67% | skill 1/1; base 1/1 |
+| claude-code | anonymizer-positive-self-hosted-gliner | 403,719 | 2,187,308 | N/A | N/A | skill 1/1; base 3/3 |
+| codex | All cases | 1,045,950 | 846,940 | N/A | N/A | skill 6/6; base 8/8 |
+| codex | anonymizer-negative-general-privacy-explainer | 13,778 | 13,555 | +223 | +1.65% | skill 1/1; base 1/1 |
+| codex | anonymizer-negative-repository-source-development | 853,740 | 578,858 | +274,882 | +47.49% | skill 1/1; base 1/1 |
+| codex | anonymizer-positive-failed-records-first | 66,683 | 151,908 | N/A | N/A | skill 1/1; base 3/3 |
+| codex | anonymizer-positive-hash-cross-record-consistency | 29,917 | 24,789 | +5,128 | +20.69% | skill 1/1; base 1/1 |
+| codex | anonymizer-positive-mode-choice | 34,490 | 25,537 | +8,953 | +35.06% | skill 1/1; base 1/1 |
+| codex | anonymizer-positive-self-hosted-gliner | 47,342 | 52,293 | -4,951 | -9.47% | skill 1/1; base 1/1 |
+| ALL AGENTS | Dataset aggregate | 2,745,312 | 3,710,470 | N/A | N/A | skill 12/12; base 18/18 |
+
+Prompt tokens include cached reads, so total tokens are `prompt + completion` (cached is not added twice). The Efficiency score uses `(prompt - cached) + completion`. N/A means the relevant trajectory counters were not available; coverage is never estimated.
+
+## Tier Status
+
+| Tier | Purpose | Status | Evidence |
+|---|---|---|---|
+| Tier 1 | Static validation | **PASSED WITH OBSERVATIONS** | 1 validator(s); 2 finding(s) |
+| Tier 2 | Semantic deduplication | **NOT RUN** | No result was recorded |
+| Tier 3 | Live agent evaluation | **PASS** | 2 agent(s); 6 task(s) |
+
+## Findings and Observations
+
+<details>
+<summary>Show detailed findings and successful checks</summary>
+
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/anonymizer/SKILL.md`)
+- **MEDIUM** SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/anonymizer/SKILL.md`)
+
+</details>
+
+## Scoring Methodology
+
+<details>
+<summary>Show dimension definitions, source signals, and thresholds</summary>
+
+| Dimension | Question | Scored signals |
+|---|---|---|
+| Security | Is it safe to use? | `security` (100%) |
+| Correctness | Is the answer correct? | `accuracy` (100%) |
+| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) |
+| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) |
+| Efficiency | Did it avoid wasted tool calls and token usage? | `skill_efficiency` (50%) + `token_efficiency` (50%) |
+
+- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
+- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
+- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
+- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
+- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`).
+- Efficiency is 50% tool-call productivity (the backward-compatible `skill_efficiency` wire id) and 50% `token_efficiency`. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.
+
+Signals present in this run:
 
-## Tier 2: Deduplication Summary
+- `security` (Security): unsafe operations, secret leakage, and unauthorized access.
+- `skill_execution` (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.
+- `skill_efficiency` (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).
+- `accuracy` (Accuracy): final-answer correctness against the reference answer.
+- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved.
+- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed.
+- `token_efficiency` (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).
 
-External NVSkills-Eval deduplication results are pending.
+</details>
 
-## Publication Recommendation
+## Freshness
 
-Proceed to external NVSkills-Eval and signing. Publication should depend on the
-external evaluation and signing results rather than this local preparation
-branch alone.
+Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.