From 4eee7b2ba4280633d5dce44f9e1b5ad708883dd9 Mon Sep 17 00:00:00 2001 From: Adam Yang Date: Wed, 24 Jun 2026 20:55:15 +1200 Subject: [PATCH 1/5] refactor!: split structured scanning into RedactWire.Structured add-on MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Core RedactWire no longer depends on System.Text.Json — it pulled STJ transitively for exactly one path (structured scanning), forcing every string-only consumer onto STJ 10.x. Pre-1.0, so the breaking move is cheap now. - New package src/RedactWire.Structured (netstandard2.0): JSON/XML/object scanners + StructuredPiiMatch moved here, namespace RedactWire kept so detector.DetectX(...) extension methods still resolve with just an added package reference. References core + System.Text.Json. Version 0.3.0 (tracks core). - Core: removed the System.Text.Json PackageReference (only dep now is DI.Abstractions) and the 3 Redactor.DetectJson/Xml/Object static forwarders. BREAKING: structured users call Redactor.Default.DetectJson(...) or detector.DetectX. - Tests: structured tests moved to tests/RedactWire.Structured.Tests (references the add-on); core test project stays STJ-free. 281 total green (259 core + 22 structured). - Solution, release workflow (packs both), README/nuget.md/CLAUDE.md updated. No detection-behavior or moved-signature changes — packaging move only. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/release.yml | 13 +++--- CLAUDE.md | 17 +++---- README.md | 32 +++++++++----- RedactWire.slnx | 2 + .../JsonPiiScanner.cs | 0 .../ObjectPiiScanner.cs | 0 src/RedactWire.Structured/Polyfills.cs | 11 +++++ .../RedactWire.Structured.csproj | 44 +++++++++++++++++++ .../StructuredPiiMatch.cs | 0 .../XmlPiiScanner.cs | 0 src/RedactWire.Structured/nuget.md | 28 ++++++++++++ src/RedactWire/RedactWire.csproj | 1 - src/RedactWire/Redactor.cs | 14 ++---- src/RedactWire/nuget.md | 6 ++- .../JsonScanTests.cs | 0 .../ObjectScanTests.cs | 0 .../RedactWire.Structured.Tests.csproj | 27 ++++++++++++ .../XmlScanTests.cs | 0 18 files changed, 156 insertions(+), 39 deletions(-) rename src/{RedactWire/Structured => RedactWire.Structured}/JsonPiiScanner.cs (100%) rename src/{RedactWire/Structured => RedactWire.Structured}/ObjectPiiScanner.cs (100%) create mode 100644 src/RedactWire.Structured/Polyfills.cs create mode 100644 src/RedactWire.Structured/RedactWire.Structured.csproj rename src/{RedactWire/Structured => RedactWire.Structured}/StructuredPiiMatch.cs (100%) rename src/{RedactWire/Structured => RedactWire.Structured}/XmlPiiScanner.cs (100%) create mode 100644 src/RedactWire.Structured/nuget.md rename tests/{RedactWire.Tests => RedactWire.Structured.Tests}/JsonScanTests.cs (100%) rename tests/{RedactWire.Tests => RedactWire.Structured.Tests}/ObjectScanTests.cs (100%) create mode 100644 tests/RedactWire.Structured.Tests/RedactWire.Structured.Tests.csproj rename tests/{RedactWire.Tests => RedactWire.Structured.Tests}/XmlScanTests.cs (100%) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b795e37..efff53d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -39,12 +39,13 @@ jobs: - run: dotnet test -c Release --no-build - name: Pack - run: > - dotnet pack src/RedactWire/RedactWire.csproj - -c Release --no-build - -p:Version=${{ steps.ver.outputs.version }} - -p:ContinuousIntegrationBuild=true - -o artifacts + run: | + for proj in src/RedactWire/RedactWire.csproj src/RedactWire.Structured/RedactWire.Structured.csproj; do + dotnet pack "$proj" -c Release --no-build \ + -p:Version=${{ steps.ver.outputs.version }} \ + -p:ContinuousIntegrationBuild=true \ + -o artifacts + done - name: NuGet login (Trusted Publishing, OIDC) uses: NuGet/login@8d196754b4036150537f80ac539e15c2f1028841 diff --git a/CLAUDE.md b/CLAUDE.md index 365c635..9231bea 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,14 +22,15 @@ namespace). Use `Redactor.Detect/HasPii/Redact` for zero-config calls. ## Architecture -- Core library (`src/RedactWire`) multi-targets **net8.0;netstandard2.0**. Dependencies: - `Microsoft.Extensions.DependencyInjection.Abstractions` (contracts-only, both TFMs) so - the DI bootstrap ships in core; `System.Text.Json` only on netstandard2.0 (built into - net8). We deliberately keep the package count low — no separate extension NuGets. -- Structured scanning lives in `Structured/`: `DetectJson` (System.Text.Json), - `DetectXml` (System.Xml, **XXE-safe**: DtdProcessing.Prohibit + null resolver), - `DetectObject` (reflection; cycle detection, depth cap, collections, skips framework - types). All are extension methods on `PiiDetector`; string values only. +- Core library (`src/RedactWire`) targets **netstandard2.0**; its only dependency is + `Microsoft.Extensions.DependencyInjection.Abstractions` (contracts-only) so the DI + bootstrap ships in core. **No `System.Text.Json`** — keep it that way. +- Structured scanning is a separate opt-in package, **`src/RedactWire.Structured`** + (depends on core + `System.Text.Json`): `DetectJson` (System.Text.Json), `DetectXml` + (System.Xml, **XXE-safe**: DtdProcessing.Prohibit + null resolver), `DetectObject` + (reflection; cycle detection, depth cap, collections, skips framework types). All are + extension methods on `PiiDetector` in namespace `RedactWire`; string values only. The + static `Redactor` has NO structured forwarders — call `Redactor.Default.DetectJson(...)`. - Three ways to use it: 1. Static `Redactor` facade — zero bootstrap. 2. `PiiDetectorBuilder` — manual configuration. diff --git a/README.md b/README.md index 1d1171c..5441b93 100644 --- a/README.md +++ b/README.md @@ -37,9 +37,9 @@ that drives overlap resolution. - **Severity model** (`Critical > High > Medium > Low`): the primary key for overlap resolution — a higher-severity match wins over an overlapping lower-severity one, even at lower confidence. -- **Structured scanning:** detect PII inside **JSON**, **XML**, and **object graphs**, - with each hit located by JSONPath / XPath / property path. Standard libraries only - (`System.Text.Json`, `System.Xml`, reflection); XML is parsed safely (no XXE). +- **Structured scanning** (opt-in **`RedactWire.Structured`** add-on): detect PII inside + **JSON**, **XML**, and **object graphs**, each hit located by JSONPath / XPath / property + path. Kept out of core so plain-string users don't pull `System.Text.Json`. - **Redaction:** mask (length-preserving), remove, type-label, or a custom replacement. - **Honest "no PII":** a requested culture with no rule pack is flagged `Supported = false` instead of silently "passing". @@ -180,17 +180,26 @@ strings need entropy/context and are a later phase. See [`docs/rules/secrets.md` ## Structured scanning (JSON / XML / objects) -Scan structured data and get each match with its location. Available on `PiiDetector` -(and the static `Redactor`). +Add-on package — keeps `System.Text.Json` out of the core: + +```bash +dotnet add package RedactWire.Structured +``` + +Scan structured data and get each match with its location. `DetectJson/DetectXml/DetectObject` +are extension methods on `PiiDetector`, so they light up once the add-on is referenced +(use `Redactor.Default.DetectJson(...)` for the shared static detector). ```csharp -foreach (var h in Redactor.DetectJson("""{"user":{"email":"a@b.com"},"ssn":"123-45-6789"}""")) +var detector = PiiDetectorBuilder.CreateDefault().AddCulture(new CultureInfo("en-US")).Build(); + +foreach (var h in detector.DetectJson("""{"user":{"email":"a@b.com"},"ssn":"123-45-6789"}""")) Console.WriteLine($"{h.Path}: {h.Match.Type}"); // $.user.email: Email // $.ssn: SocialSecurity -Redactor.DetectXml(""); // -> /u/@email -Redactor.DetectObject(myPoco); // -> User.Contacts[0].Phone +detector.DetectXml(""); // -> /u/@email +detector.DetectObject(myPoco); // -> User.Contacts[0].Phone ``` Notes: @@ -198,8 +207,8 @@ Notes: - XML is parsed with DTD processing prohibited and no external resolver — **XXE-safe**. - Object scanning walks public properties/fields, handles collections/dictionaries, detects cycles, caps depth, and does not recurse into framework types. -- `System.Text.Json` is built into modern .NET; on `netstandard2.0`/.NET Framework it - comes as a NuGet dependency (the library multi-targets so net8+ stays clean). +- The add-on uses `System.Text.Json` (JSON) and BCL `System.Xml` / reflection (XML, objects). + This is the only reason `System.Text.Json` is split out of the core package. ## Extending @@ -284,7 +293,8 @@ yield return new RuleHit(value, start, length, 0.9, | Path | What | |---|---| -| `src/RedactWire` | the library (+ DI bootstrap) | +| `src/RedactWire` | the core library (detection / validation / redaction + DI bootstrap) | +| `src/RedactWire.Structured` | JSON/XML/object scanning add-on (`System.Text.Json`) | | `src/RedactWire.Cli` | command-line scanner / redactor | | `samples/RedactWire.Sample.Web` | ASP.NET Core Razor Pages PII tester | | `tests/RedactWire.Tests` | xUnit tests | diff --git a/RedactWire.slnx b/RedactWire.slnx index 4b22299..96dbe58 100644 --- a/RedactWire.slnx +++ b/RedactWire.slnx @@ -4,9 +4,11 @@ + + diff --git a/src/RedactWire/Structured/JsonPiiScanner.cs b/src/RedactWire.Structured/JsonPiiScanner.cs similarity index 100% rename from src/RedactWire/Structured/JsonPiiScanner.cs rename to src/RedactWire.Structured/JsonPiiScanner.cs diff --git a/src/RedactWire/Structured/ObjectPiiScanner.cs b/src/RedactWire.Structured/ObjectPiiScanner.cs similarity index 100% rename from src/RedactWire/Structured/ObjectPiiScanner.cs rename to src/RedactWire.Structured/ObjectPiiScanner.cs diff --git a/src/RedactWire.Structured/Polyfills.cs b/src/RedactWire.Structured/Polyfills.cs new file mode 100644 index 0000000..a03947d --- /dev/null +++ b/src/RedactWire.Structured/Polyfills.cs @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: Apache-2.0 +// Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ + +// netstandard2.0 lacks IsExternalInit, required by `record` positional members / `init`. +// Each assembly needs its own internal copy (core's is internal to RedactWire). +#if NETSTANDARD2_0 +namespace System.Runtime.CompilerServices +{ + internal static class IsExternalInit { } +} +#endif diff --git a/src/RedactWire.Structured/RedactWire.Structured.csproj b/src/RedactWire.Structured/RedactWire.Structured.csproj new file mode 100644 index 0000000..b206caf --- /dev/null +++ b/src/RedactWire.Structured/RedactWire.Structured.csproj @@ -0,0 +1,44 @@ + + + + netstandard2.0 + enable + enable + latest + true + true + $(NoWarn);CS1591 + + + + + RedactWire.Structured + 0.3.0 + Adam Yang + Object IT Limited + RedactWire + Structured PII scanning add-on for RedactWire — detect PII inside JSON, XML and object graphs, each hit located by JSONPath / XPath / property path. Adds a System.Text.Json dependency; install only if you need structured scanning (the core RedactWire package stays free of it). + Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ + Apache-2.0 + https://github.com/adamy/RedactWire + https://github.com/adamy/RedactWire + git + pii;pii-detection;redaction;privacy;json;xml;structured;jsonpath;xpath;data-protection;dotnet + nuget.md + true + true + true + snupkg + true + + + + + + + + + + + + diff --git a/src/RedactWire/Structured/StructuredPiiMatch.cs b/src/RedactWire.Structured/StructuredPiiMatch.cs similarity index 100% rename from src/RedactWire/Structured/StructuredPiiMatch.cs rename to src/RedactWire.Structured/StructuredPiiMatch.cs diff --git a/src/RedactWire/Structured/XmlPiiScanner.cs b/src/RedactWire.Structured/XmlPiiScanner.cs similarity index 100% rename from src/RedactWire/Structured/XmlPiiScanner.cs rename to src/RedactWire.Structured/XmlPiiScanner.cs diff --git a/src/RedactWire.Structured/nuget.md b/src/RedactWire.Structured/nuget.md new file mode 100644 index 0000000..9786642 --- /dev/null +++ b/src/RedactWire.Structured/nuget.md @@ -0,0 +1,28 @@ +# RedactWire.Structured + +Structured-data scanning add-on for [**RedactWire**](https://www.nuget.org/packages/RedactWire/) — +detect PII inside **JSON**, **XML** and **object graphs**, each hit located by JSONPath, +XPath, or property path. + +Split out of the core package so plain-string detection stays free of a +`System.Text.Json` dependency; install this only when you need structured scanning. + +```csharp +using RedactWire; // PiiDetector, Redactor +using Microsoft.Extensions.DependencyInjection; // (if using DI) + +var detector = PiiDetectorBuilder.CreateDefault().AddCulture(new CultureInfo("en-US")).Build(); + +foreach (var h in detector.DetectJson("""{"user":{"email":"a@b.com"},"ssn":"123-45-6789"}""")) + Console.WriteLine($"{h.Path}: {h.Match.Type}"); // $.user.email: Email / $.ssn: SocialSecurity + +detector.DetectXml(""); // -> /u/@email (XXE-safe) +detector.DetectObject(myPoco); // -> User.Contacts[0].Phone + +// or via the static facade's shared detector: +Redactor.Default.DetectJson(json); +``` + +`DetectJson` / `DetectXml` / `DetectObject` are extension methods on `PiiDetector`, so they +light up automatically once this package is referenced. License: **Apache-2.0**. +Docs: https://github.com/adamy/RedactWire diff --git a/src/RedactWire/RedactWire.csproj b/src/RedactWire/RedactWire.csproj index ed87183..08aa8aa 100644 --- a/src/RedactWire/RedactWire.csproj +++ b/src/RedactWire/RedactWire.csproj @@ -37,7 +37,6 @@ - diff --git a/src/RedactWire/Redactor.cs b/src/RedactWire/Redactor.cs index c04a5c1..4f23807 100644 --- a/src/RedactWire/Redactor.cs +++ b/src/RedactWire/Redactor.cs @@ -46,15 +46,7 @@ public static ValidationResult Validate(string value, PiiType type, string? subt public static ValidationResult Validate(string value, CultureInfo culture, PiiType type, string? subtype = null) => Default.Validate(value, culture, type, subtype); - /// Scan a JSON string; matches are located by JSONPath. - public static IReadOnlyList DetectJson(string json, params CultureInfo[] cultures) => - Default.DetectJson(json, cultures); - - /// Scan an XML string; matches are located by XPath. Parsed safely (no XXE). - public static IReadOnlyList DetectXml(string xml, params CultureInfo[] cultures) => - Default.DetectXml(xml, cultures); - - /// Scan an object graph; matches are located by property path. - public static IReadOnlyList DetectObject(object? graph, params CultureInfo[] cultures) => - Default.DetectObject(graph, cultures); + // Structured scanning (JSON/XML/object) moved to the RedactWire.Structured add-on + // package. With that package referenced, call Redactor.Default.DetectJson(...) etc. + // (extension methods on PiiDetector). } diff --git a/src/RedactWire/nuget.md b/src/RedactWire/nuget.md index e1cedc3..0a37eb6 100644 --- a/src/RedactWire/nuget.md +++ b/src/RedactWire/nuget.md @@ -9,8 +9,10 @@ credit card, and a national ID with a bad check digit is dropped, not guessed. postcodes), most gated by the real check-digit algorithm. - **Secret detection** — API keys & tokens (OpenAI, AWS, GitHub, Stripe, Slack, Google, SendGrid, npm), JWT, PEM private keys — via `AddSecretDetection()`. Great for AI/LLM logs. -- Structured scanning of **JSON / XML / objects**; severity-driven overlap resolution; - redaction (mask / remove / label / custom). +- Severity-driven overlap resolution; redaction (mask / remove / label / custom). +- Structured scanning of **JSON / XML / objects** is an opt-in add-on — + [`RedactWire.Structured`](https://www.nuget.org/packages/RedactWire.Structured/) — so this + core package carries no `System.Text.Json` dependency. ## Quick start diff --git a/tests/RedactWire.Tests/JsonScanTests.cs b/tests/RedactWire.Structured.Tests/JsonScanTests.cs similarity index 100% rename from tests/RedactWire.Tests/JsonScanTests.cs rename to tests/RedactWire.Structured.Tests/JsonScanTests.cs diff --git a/tests/RedactWire.Tests/ObjectScanTests.cs b/tests/RedactWire.Structured.Tests/ObjectScanTests.cs similarity index 100% rename from tests/RedactWire.Tests/ObjectScanTests.cs rename to tests/RedactWire.Structured.Tests/ObjectScanTests.cs diff --git a/tests/RedactWire.Structured.Tests/RedactWire.Structured.Tests.csproj b/tests/RedactWire.Structured.Tests/RedactWire.Structured.Tests.csproj new file mode 100644 index 0000000..2fd939d --- /dev/null +++ b/tests/RedactWire.Structured.Tests/RedactWire.Structured.Tests.csproj @@ -0,0 +1,27 @@ + + + + net8.0 + enable + enable + + false + true + + + + + + + + + + + + + + + + + + diff --git a/tests/RedactWire.Tests/XmlScanTests.cs b/tests/RedactWire.Structured.Tests/XmlScanTests.cs similarity index 100% rename from tests/RedactWire.Tests/XmlScanTests.cs rename to tests/RedactWire.Structured.Tests/XmlScanTests.cs From 68f2d4a3ff43a1ce92c55e3489ae02522ff68f60 Mon Sep 17 00:00:00 2001 From: Adam Yang Date: Sat, 27 Jun 2026 09:56:38 +1200 Subject: [PATCH 2/5] chore: bump RedactWire and RedactWire.Structured to 0.4.0 Co-Authored-By: Claude Opus 4.8 --- .github/workflows/release.yml | 4 ++-- src/RedactWire.Structured/RedactWire.Structured.csproj | 2 +- src/RedactWire/RedactWire.csproj | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index efff53d..339c45b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -5,9 +5,9 @@ on: workflow_dispatch: inputs: version: - description: 'Package version (e.g. 0.2.0)' + description: 'Package version (e.g. 0.4.0)' required: true - default: '0.2.0' + default: '0.4.0' permissions: contents: write # create the GitHub Release diff --git a/src/RedactWire.Structured/RedactWire.Structured.csproj b/src/RedactWire.Structured/RedactWire.Structured.csproj index b206caf..0fa44d3 100644 --- a/src/RedactWire.Structured/RedactWire.Structured.csproj +++ b/src/RedactWire.Structured/RedactWire.Structured.csproj @@ -13,7 +13,7 @@ RedactWire.Structured - 0.3.0 + 0.4.0 Adam Yang Object IT Limited RedactWire diff --git a/src/RedactWire/RedactWire.csproj b/src/RedactWire/RedactWire.csproj index 08aa8aa..a563ea2 100644 --- a/src/RedactWire/RedactWire.csproj +++ b/src/RedactWire/RedactWire.csproj @@ -14,7 +14,7 @@ RedactWire - 0.3.0 + 0.4.0 Adam Yang Object IT Limited RedactWire From e65123df2da3bbc33e029a4a29eef14068a54d9a Mon Sep 17 00:00:00 2001 From: Adam Yang Date: Sat, 27 Jun 2026 09:58:41 +1200 Subject: [PATCH 3/5] docs(structured): make 'requires the main RedactWire package' explicit Co-Authored-By: Claude Opus 4.8 --- src/RedactWire.Structured/nuget.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/src/RedactWire.Structured/nuget.md b/src/RedactWire.Structured/nuget.md index 9786642..2cf7e59 100644 --- a/src/RedactWire.Structured/nuget.md +++ b/src/RedactWire.Structured/nuget.md @@ -4,9 +4,20 @@ Structured-data scanning add-on for [**RedactWire**](https://www.nuget.org/packa detect PII inside **JSON**, **XML** and **object graphs**, each hit located by JSONPath, XPath, or property path. +> ⚠️ **This is an add-on, not a standalone package.** It extends the main +> [**RedactWire**](https://www.nuget.org/packages/RedactWire/) package — you build the +> detector with RedactWire's API (`PiiDetectorBuilder` / `Redactor`) and this package adds +> the `DetectJson` / `DetectXml` / `DetectObject` methods on top. RedactWire is pulled in +> automatically as a dependency, but everything except the three scan methods lives there, +> so read its docs first. + Split out of the core package so plain-string detection stays free of a `System.Text.Json` dependency; install this only when you need structured scanning. +```bash +dotnet add package RedactWire.Structured # brings in RedactWire automatically +``` + ```csharp using RedactWire; // PiiDetector, Redactor using Microsoft.Extensions.DependencyInjection; // (if using DI) From fa033a652c65852dcf717cc0c060fe3bf49278ae Mon Sep 17 00:00:00 2001 From: Adam Yang Date: Sat, 27 Jun 2026 10:14:31 +1200 Subject: [PATCH 4/5] sample: separate Structured JSON and XML pages; scope scan to one culture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - /StructuredJson and /StructuredXml pages (was one combined page), each with its own sample and a culture selector. Sample references the RedactWire.Structured add-on. - Fix noisy output: the DI detector has every built-in pack, so scanning structured data with no culture surfaced a generic 4-digit postcode/phone hit per country (67 matches on the sample JSON). Pages now pass the chosen culture to DetectJson/DetectXml -> 4. - Nav: "Structured · JSON" / "Structured · XML". Co-Authored-By: Claude Opus 4.8 --- .../Pages/Shared/_Layout.cshtml | 6 ++ .../Pages/StructuredJson.cshtml | 62 +++++++++++++++++++ .../Pages/StructuredJson.cshtml.cs | 44 +++++++++++++ .../Pages/StructuredXml.cshtml | 62 +++++++++++++++++++ .../Pages/StructuredXml.cshtml.cs | 43 +++++++++++++ .../RedactWire.Sample.Web.csproj | 1 + 6 files changed, 218 insertions(+) create mode 100644 samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml create mode 100644 samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml.cs create mode 100644 samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml create mode 100644 samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml.cs diff --git a/samples/RedactWire.Sample.Web/Pages/Shared/_Layout.cshtml b/samples/RedactWire.Sample.Web/Pages/Shared/_Layout.cshtml index 2ff9ff9..bd1ba3b 100644 --- a/samples/RedactWire.Sample.Web/Pages/Shared/_Layout.cshtml +++ b/samples/RedactWire.Sample.Web/Pages/Shared/_Layout.cshtml @@ -22,6 +22,12 @@
  • Home
    • +
  • + Structured · JSON +
    • +
  • + Structured · XML +
  • Privacy
    • diff --git a/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml b/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml new file mode 100644 index 0000000..cef19c9 --- /dev/null +++ b/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml @@ -0,0 +1,62 @@ +@* SPDX-License-Identifier: Apache-2.0 *@ +@* Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ *@ +@page +@model StructuredJsonModel +@{ + ViewData["Title"] = "RedactWire — JSON scan"; +} + +

    Structured scan — JSON

    +

    + RedactWire.Structured add-on: detector.DetectJson(...) locates + each PII hit by JSONPath. +

    + +
    +
    +
    + + +
    +
    +
    + +
    + +
    + +@if (Model.Error is not null) +{ +
    @Model.Error
    +} + +@if (Model.Results is not null) +{ +
    +

    @Model.Results.Count match(es)

    + @if (Model.Results.Count > 0) + { + + + + + + @foreach (var h in Model.Results.OrderBy(h => h.Path)) + { + + + + + + + + } + +
    Path (JSONPath)TypeValueSeverityConf
    @h.Path@(h.Match.Subtype ?? h.Match.Type.ToString())@h.Match.Value@h.Match.Severity@h.Match.Confidence.ToString("0.00")
    + } +} diff --git a/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml.cs b/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml.cs new file mode 100644 index 0000000..49937ea --- /dev/null +++ b/samples/RedactWire.Sample.Web/Pages/StructuredJson.cshtml.cs @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: Apache-2.0 +// Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ + +using System.Globalization; +using System.Text.Json; +using Microsoft.AspNetCore.Mvc; +using Microsoft.AspNetCore.Mvc.RazorPages; +using RedactWire; // PiiDetector + DetectJson extension method (RedactWire.Structured) + +namespace RedactWire.Sample.Web.Pages; + +public class StructuredJsonModel : PageModel +{ + private readonly PiiDetector _detector; // injected singleton (see Program.cs) + + public StructuredJsonModel(PiiDetector detector) => _detector = detector; + + [BindProperty] + public string? Input { get; set; } + + // Scope to one country — the DI detector has every built-in pack, so scanning against + // all of them would surface a generic 4-digit "postcode" hit per country (noise). + [BindProperty] + public string Culture { get; set; } = "en-US"; + + public static IReadOnlyList Cultures => PiiDetectorBuilder.AvailableCultures; + + public IReadOnlyList? Results { get; private set; } + public string? Error { get; private set; } + + private const string Sample = + "{\n \"user\": { \"email\": \"john@x.co.nz\", \"ssn\": \"123-45-6789\" },\n" + + " \"contacts\": [ { \"phone\": \"(415) 555-0132\" } ],\n" + + " \"card\": \"4242 4242 4242 4242\"\n}"; + + public void OnGet() => Input ??= Sample; + + public void OnPost() + { + if (string.IsNullOrWhiteSpace(Input)) return; + try { Results = _detector.DetectJson(Input, new CultureInfo(Culture)); } + catch (JsonException ex) { Error = "Invalid JSON: " + ex.Message; } + } +} diff --git a/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml b/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml new file mode 100644 index 0000000..25fa632 --- /dev/null +++ b/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml @@ -0,0 +1,62 @@ +@* SPDX-License-Identifier: Apache-2.0 *@ +@* Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ *@ +@page +@model StructuredXmlModel +@{ + ViewData["Title"] = "RedactWire — XML scan"; +} + +

    Structured scan — XML

    +

    + RedactWire.Structured add-on: detector.DetectXml(...) locates + each PII hit by XPath. Parsed XXE-safe (DTDs prohibited). +

    + +
    +
    +
    + + +
    +
    +
    + +
    + +
    + +@if (Model.Error is not null) +{ +
    @Model.Error
    +} + +@if (Model.Results is not null) +{ +
    +

    @Model.Results.Count match(es)

    + @if (Model.Results.Count > 0) + { + + + + + + @foreach (var h in Model.Results.OrderBy(h => h.Path)) + { + + + + + + + + } + +
    Path (XPath)TypeValueSeverityConf
    @h.Path@(h.Match.Subtype ?? h.Match.Type.ToString())@h.Match.Value@h.Match.Severity@h.Match.Confidence.ToString("0.00")
    + } +} diff --git a/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml.cs b/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml.cs new file mode 100644 index 0000000..f70c4b1 --- /dev/null +++ b/samples/RedactWire.Sample.Web/Pages/StructuredXml.cshtml.cs @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: Apache-2.0 +// Copyright 2026 Adam Yang, Object IT Limited, Auckland, NZ + +using System.Globalization; +using System.Xml; +using Microsoft.AspNetCore.Mvc; +using Microsoft.AspNetCore.Mvc.RazorPages; +using RedactWire; // PiiDetector + DetectXml extension method (RedactWire.Structured) + +namespace RedactWire.Sample.Web.Pages; + +public class StructuredXmlModel : PageModel +{ + private readonly PiiDetector _detector; // injected singleton (see Program.cs) + + public StructuredXmlModel(PiiDetector detector) => _detector = detector; + + [BindProperty] + public string? Input { get; set; } + + // Scope to one country — the DI detector has every built-in pack, so scanning against + // all of them would surface a generic 4-digit "postcode" hit per country (noise). + [BindProperty] + public string Culture { get; set; } = "en-US"; + + public static IReadOnlyList Cultures => PiiDetectorBuilder.AvailableCultures; + + public IReadOnlyList? Results { get; private set; } + public string? Error { get; private set; } + + private const string Sample = + "\n 123-45-6789\n" + + " (415) 555-0132\n 4242 4242 4242 4242\n"; + + public void OnGet() => Input ??= Sample; + + public void OnPost() + { + if (string.IsNullOrWhiteSpace(Input)) return; + try { Results = _detector.DetectXml(Input, new CultureInfo(Culture)); } + catch (XmlException ex) { Error = "Invalid XML: " + ex.Message; } + } +} diff --git a/samples/RedactWire.Sample.Web/RedactWire.Sample.Web.csproj b/samples/RedactWire.Sample.Web/RedactWire.Sample.Web.csproj index 922b24b..e89a401 100644 --- a/samples/RedactWire.Sample.Web/RedactWire.Sample.Web.csproj +++ b/samples/RedactWire.Sample.Web/RedactWire.Sample.Web.csproj @@ -2,6 +2,7 @@ + From dac9630e4ce6210fbfaa50e8f477c9d6d386ada8 Mon Sep 17 00:00:00 2001 From: Adam Yang Date: Sat, 27 Jun 2026 10:19:32 +1200 Subject: [PATCH 5/5] docs: add CHANGELOG with 0.4.0 breaking note; PackageReleaseNotes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Structured scanning split out of core is a breaking change — documented in CHANGELOG.md and surfaced on NuGet via PackageReleaseNotes on both packages. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 36 +++++++++++++++++++ .../RedactWire.Structured.csproj | 1 + src/RedactWire/RedactWire.csproj | 1 + 3 files changed, 38 insertions(+) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..48f6955 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,36 @@ +# Changelog + +All notable changes to RedactWire. This project is pre-1.0 — breaking changes can land in +minor versions. + +## 0.4.0 + +### ⚠️ Breaking + +- **Structured scanning (JSON / XML / object) moved out of core** into a new opt-in package + **`RedactWire.Structured`**. The core `RedactWire` package no longer depends on + `System.Text.Json` — string-only consumers carry one less dependency. +- **`Redactor.DetectJson` / `DetectXml` / `DetectObject` removed.** Reference the + `RedactWire.Structured` package and call them on a detector instead: + ```csharp + // before + Redactor.DetectJson(json); + // after (after `dotnet add package RedactWire.Structured`) + Redactor.Default.DetectJson(json); // shared static detector + detector.DetectJson(json, culture); // or your own builder/detector + ``` + `DetectJson/DetectXml/DetectObject` are extension methods on `PiiDetector` (namespace + `RedactWire`), so they light up automatically once the add-on is referenced. Method + signatures and detection behavior are unchanged — this is a packaging move only. + +### Added + +- **Secret detection** — `AddSecretDetection()` enables provider-prefixed API keys/tokens + (OpenAI, AWS access-key-id, GitHub, Stripe, Slack, Google, SendGrid, npm), JWT and PEM + private keys, as `PiiType.Secret` (provider in `Subtype`). On by default in `Redactor`. +- **Remove / replace built-in rules** — `RemoveRule(type[, subtype])`, `RemoveRules(predicate)`, + `ReplaceInvariantRule(rule)`, `ReplaceRule(culture, rule)`. `IPiiRule` now exposes `Subtype`. +- **Region-based culture resolution** — a country's languages share one pack + (`en-IN`/`hi-IN`, `en-CA`/`fr-CA`, Singapore's four languages); same language across + countries stays distinct (`zh-CN`/`zh-TW`/`zh-HK`/`zh-SG`). +- More country packs (now 50+), incl. Malaysia (MyKad) and Australia Medicare. diff --git a/src/RedactWire.Structured/RedactWire.Structured.csproj b/src/RedactWire.Structured/RedactWire.Structured.csproj index 0fa44d3..522a6dc 100644 --- a/src/RedactWire.Structured/RedactWire.Structured.csproj +++ b/src/RedactWire.Structured/RedactWire.Structured.csproj @@ -25,6 +25,7 @@ git pii;pii-detection;redaction;privacy;json;xml;structured;jsonpath;xpath;data-protection;dotnet nuget.md + 0.4.0 — New add-on holding RedactWire's structured scanning (JSON/XML/object), split out of core so string-only users don't pull System.Text.Json. Requires the RedactWire package (pulled in automatically). Call detector.DetectJson/DetectXml/DetectObject or Redactor.Default.DetectJson(...). true true true diff --git a/src/RedactWire/RedactWire.csproj b/src/RedactWire/RedactWire.csproj index a563ea2..fe2e816 100644 --- a/src/RedactWire/RedactWire.csproj +++ b/src/RedactWire/RedactWire.csproj @@ -28,6 +28,7 @@ GitHub topics can't hold these, so they live here only). --> pii;pii-detection;privacy;gdpr;ccpa;hipaa;redaction;data-masking;anonymization;tokenization;dlp;compliance;data-protection;secret-detection;secret-scanning;secrets;api-key;credentials;jwt;openai;aws;dotnet;csharp;netstandard;national-id;tax-id;passport;phone-number;creditcard;iban;ssn;sin;nino;tfn;medicare;ird;nric;mykad;aadhaar;pan;cpf;cnpj;curp;rfc;steuer-id;nir;bsn;dni;codice-fiscale;nif;pesel;personnummer;hetu;cpr;rrn;my-number;hkid;身份证;居民身份证;マイナンバー;個人番号;주민등록번호;身分證;香港身份證;ИНН;СНИЛС;บัตรประชาชน;ΑΦΜ;ΑΜΚΑ;आधार;cccd nuget.md + 0.4.0 — BREAKING: structured scanning (JSON/XML/object) moved to the new RedactWire.Structured package; core no longer depends on System.Text.Json. Redactor.DetectJson/DetectXml/DetectObject removed — reference RedactWire.Structured and call Redactor.Default.DetectJson(...) or detector.DetectJson(...). Added: secret detection (AddSecretDetection), remove/replace built-in rules, region-based culture resolution. See CHANGELOG.md. true true true