Skip to content

adamy/RedactWire

Repository files navigation

RedactWire

NuGet Downloads CI License: Apache 2.0

A lightweight, culture-aware PII detection, validation and redaction library for .NET.

RedactWire is regex-first and checksum-validated: it doesn't just match patterns, it verifies them (Luhn for cards, mod-97 for IBAN, area/group rules for US SSNs), so a random 16-digit number isn't reported as a credit card. Rules are grouped per culture, results tell you which checks ran and which passed, and every match carries a severity that drives overlap resolution.

  • Target: netstandard2.0 — works on .NET Framework 4.6.1+, .NET Core, .NET 5–8+, Mono, Unity, Xamarin.
  • Dependencies: one, contracts-only (Microsoft.Extensions.DependencyInjection.Abstractions), so the DI bootstrap ships in the box. No other runtime dependencies.
  • License: Apache 2.0.

Features

  • Invariant (country-agnostic) rules: Email, Credit card (Luhn), IPv4, IBAN (mod-97).
  • Secret detection — API keys & tokens (OpenAI, AWS, GitHub, Stripe, Slack, Google, SendGrid, npm), JWT, and PEM private keys. Opt-in via AddSecretDetection() (on by default in the static Redactor). Ideal for scrubbing AI/LLM logs.
  • Built-in packs for 50+ countries (see Coverage) — national IDs, tax numbers, phones, passports, postcodes — most gated by the real check digit/algorithm.
  • Region-based resolution: a country's languages share one pack (en-IN/hi-IN/ta-IN, en-CA/fr-CA, de-CH/fr-CH/it-CH, Singapore's four official languages), while the same language across different countries stays distinct (zh-CN/zh-TW/zh-HK/zh-SG).
  • Validation: Validate(value, [culture,] type)Valid / Invalid / Unsupported.
  • Checksum gating: a failed checksum drops the candidate — it is not reported as low-confidence noise.
  • Severity model (Critical > High > Medium > Low): the primary key for overlap resolution — a higher-severity match wins over an overlapping lower-severity one, even at lower confidence.
  • Structured scanning (opt-in RedactWire.Structured add-on): detect PII inside JSON, XML, and object graphs, each hit located by JSONPath / XPath / property path. Kept out of core so plain-string users don't pull System.Text.Json.
  • Redaction: mask (length-preserving), remove, type-label, or a custom replacement.
  • Honest "no PII": a requested culture with no rule pack is flagged Supported = false instead of silently "passing".
  • Extensible: add your own rules per culture without forking (implement IPiiRule, or just use RegexRule).

Coverage

Invariant (every culture): Email, Credit card, IPv4, IBAN.

Built-in country packs (representative culture shown; all languages of a country resolve to the same pack):

Region Packs
Americas en-US en-CA es-MX pt-BR es-AR es-CL es-CO
Europe en-GB fr-FR de-DE it-IT es-ES pt-PT nl-NL nl-BE de-CH de-AT pl-PL cs-CZ sk-SK hu-HU el-GR en-IE sv-SE nb-NO da-DK fi-FI is-IS et-EE lt-LT lv-LV sl-SI lb-LU ru-RU tr-TR
Asia-Pacific zh-CN zh-HK zh-TW zh-MO ja-JP ko-KR en-IN id-ID vi-VN th-TH en-PH en-PK bn-BD en-SG ms-MY en-AU en-NZ
Middle East / Africa ar-SA ar-EG fa-IR he-IL en-NG en-ZA

PiiDetectorBuilder.AvailableCultures returns this list at runtime. Each pack is documented under docs/rules/localized/, with verification status in docs/rules/VERIFICATION.md.

Identifiers (searchable, incl. local names): SSN · SIN · NINO · NHS · TFN · Medicare · IRD · NRIC · MyKad · Aadhaar · PAN · CPF · CNPJ · CURP · RFC · Steuer-ID · NIR · BSN · DNI/NIE · Codice Fiscale · NIF · PESEL · Personnummer · Fødselsnummer · HETU · CPR · RRN · My Number · HKID · 身份证 · 居民身份证 · 身分證 · 香港身份證 · マイナンバー · 個人番号 · 주민등록번호 · ИНН · СНИЛС · บัตรประชาชน · ΑΦΜ · ΑΜΚΑ · आधार · CCCD.

Patterns and checksums are being verified against authoritative sources (see the verification log). Treat unverified packs as best-effort until confirmed.

Install

dotnet add package RedactWire

Quick start

Three ways to use it — pick what fits your app.

1. Static facade (zero bootstrap)

using RedactWire;

PiiResult r = Redactor.Detect("My SSN is 123-45-6789");
bool hasPii  = Redactor.HasPii("Call (415) 555-0132");
string clean = Redactor.Redact("Card 4242 4242 4242 4242");

2. Builder (manual configuration)

using System.Globalization;
using RedactWire;

var detector = PiiDetectorBuilder.CreateDefault()      // invariant rules
    .AddCulture(new CultureInfo("en-US"))              // en-US pack
    .UseOverlapStrategy(OverlapStrategy.KeepHighestConfidence)
    .Build();

PiiResult result = detector.Detect("Email john@x.co.nz, SSN 123-45-6789");
foreach (var m in result.AllMatches)
    Console.WriteLine($"{m.Severity} {m.Type} '{m.Value}' conf={m.Confidence:0.00}");

3. Dependency injection (ASP.NET Core / generic host)

// Program.cs
builder.Services.AddRedactWire(b => b.AddCulture(new CultureInfo("en-US")));

// then inject PiiDetector anywhere
public class MyService(PiiDetector detector) { /* detector.Detect(...) */ }

Results

Detect returns a PiiResult:

  • InvariantCulturePiiResult of country-agnostic matches (always evaluated).
  • Cultures — one CulturePiiResult per requested culture, each with:
    • Matches, RulesEvaluated
    • Supported — was a rule pack found for this culture?
    • PassedSupported && Matches.Count == 0
  • HasPii, AllMatches (flat view).

Each PiiMatch carries Type, Value, Start, Length, Confidence, Rule, Culture, Severity.

Redaction

var r = Redactor.Detect("SSN 123-45-6789");
r.Redact();                                                  // "SSN ***********"
r.Redact(new RedactionOptions { Mode = RedactionMode.Label }); // "SSN [SocialSecurity]"
r.Redact(new RedactionOptions { Custom = m => $"<{m.Type}>" });

Validation

Check whether a string is, in full, a valid PII item of a given type — same rules and checksums as detection. Returns Valid / Invalid / Unsupported (the last kept distinct so "no rule for this type/culture" isn't mistaken for "invalid").

Redactor.Validate("123-45-6789", PiiType.SocialSecurity);            // Valid
Redactor.Validate("123-45-0000", PiiType.SocialSecurity);            // Invalid
Redactor.Validate("110101199001010015",
    new CultureInfo("zh-CN"), PiiType.NationalId);                   // Valid (GB11643)
Redactor.Validate("x", new CultureInfo("fr-FR"), PiiType.NationalId); // Unsupported

// PiiType.Custom needs a subtype:
detector.Validate("112-233-445 95", new CultureInfo("ru-RU"), PiiType.Custom, "SNILS");

The no-culture overload validates against the detector's configured cultures; pass a CultureInfo to target one. Must be a full-string match (surrounding whitespace ignored).

Secret detection

API keys, tokens and private keys — country-agnostic, so they run as invariant rules.

var d = PiiDetectorBuilder.CreateDefault().AddSecretDetection().Build();
foreach (var m in d.Detect("OPENAI_API_KEY=sk-abc...xyz").AllMatches.Where(m => m.Type == PiiType.Secret))
    Console.WriteLine(m.Subtype);   // OpenAiKey

Redactor.Redact("aws key AKIAIOSFODNN7EXAMPLE");   // "aws key ***************..." (secrets on by default)

Covers OpenAI, AWS access-key-id, GitHub, Stripe, Slack, Google, SendGrid, npm, JWT and PEM private keys (high-precision, provider-prefixed). AWS secret keys / Azure keys / connection strings need entropy/context and are a later phase. See docs/rules/secrets.md.

Structured scanning (JSON / XML / objects)

Add-on package — keeps System.Text.Json out of the core:

dotnet add package RedactWire.Structured

Scan structured data and get each match with its location. DetectJson/DetectXml/DetectObject are extension methods on PiiDetector, so they light up once the add-on is referenced (use Redactor.Default.DetectJson(...) for the shared static detector).

var detector = PiiDetectorBuilder.CreateDefault().AddCulture(new CultureInfo("en-US")).Build();

foreach (var h in detector.DetectJson("""{"user":{"email":"a@b.com"},"ssn":"123-45-6789"}"""))
    Console.WriteLine($"{h.Path}: {h.Match.Type}");
// $.user.email: Email
// $.ssn: SocialSecurity

detector.DetectXml("<u email=\"a@b.com\"/>");   // -> /u/@email
detector.DetectObject(myPoco);                  // -> User.Contacts[0].Phone

Notes:

  • Only string values are scanned (a number that lost its formatting isn't reliable PII).
  • XML is parsed with DTD processing prohibited and no external resolver — XXE-safe.
  • Object scanning walks public properties/fields, handles collections/dictionaries, detects cycles, caps depth, and does not recurse into framework types.
  • The add-on uses System.Text.Json (JSON) and BCL System.Xml / reflection (XML, objects). This is the only reason System.Text.Json is split out of the core package.

Extending

Add a custom rule without touching the library. Most needs are covered by RegexRule:

var detector = PiiDetectorBuilder.CreateDefault()
    .AddCulture(new CultureInfo("en-US"))
    .AddRule(new CultureInfo("en-US"),                 // bind to one culture
        new RegexRule("EmployeeId", PiiType.NationalId,
            @"(?<v>\bEMP\d{6}\b)",
            baseConfidence: 0.8,
            severity: PiiSeverity.Critical))           // optional; defaults from the type
    .AddInvariantRule(myGlobalRule)                    // always-on, culture-agnostic
    .Build();

For anything beyond a single regex, implement IPiiRule. A rule only reports raw RuleHits — the engine stamps the culture, the rule id, and the severity default, so there's no plumbing to get wrong:

public sealed class MyRule : IPiiRule
{
    public string Name => "MyRule";
    public PiiType Type => PiiType.NationalId;

    public IEnumerable<RuleHit> Find(string text)
    {
        // ... locate a match ...
        yield return new RuleHit(value, start, length, confidence: 0.9);
        // (pass a PiiSeverity to override the type default)
    }
}

var detector = PiiDetectorBuilder.CreateDefault()
    .AddCulture(new CultureInfo("en-US"))
    .AddCulture(new CultureInfo("en-GB"))
    .AddRule(new MyRule())   // bind to every configured culture at once
    .Build();

Remove or replace a built-in rule

PiiDetectorBuilder.CreateDefault()
    .AddCulture(new CultureInfo("en-US"))
    .AddSecretDetection()
    .RemoveRule(PiiType.PostalCode)              // drop ZIP/postcode rules (too noisy)
    .RemoveRule(PiiType.Secret, "OpenAiKey")     // drop one secret type, keep the rest
    .ReplaceInvariantRule(new RegexRule("Email", PiiType.Email, myStricterEmailPattern))
    .ReplaceRule(new CultureInfo("en-US"), myUsPhoneRule)   // override by rule Name
    .Build();

RemoveRule(type[, subtype]) removes built-in or custom rules everywhere; ReplaceRule / ReplaceInvariantRule swap a rule sharing the same Name. Call them after the Add* that loaded the rules.

Custom PII types

PiiType is an enum (enums can't be extended). For a type that isn't in the list, use PiiType.Custom and give it a real name via RuleHit.Subtype — the name flows into the match and into redaction labels:

yield return new RuleHit(value, start, length, 0.9,
    Severity: PiiSeverity.Critical, Subtype: "NhiNumber");
// match.Type == PiiType.Custom, match.Subtype == "NhiNumber"
// Redact(Label)  ->  "[NhiNumber]"

Scope & roadmap

  • Names and addresses by NER are out of the core (regex floods false positives). The Address rule is a regex heuristic; full name detection is reserved for an optional RedactWire.Ner package (local ONNX, e.g. GLiNER) — kept separate to avoid forcing a large model + its license on every consumer.
  • More country packs to come (UK, DE/FR, AU/JP/SG/IN, …).

Projects

Path What
src/RedactWire the core library (detection / validation / redaction + DI bootstrap)
src/RedactWire.Structured JSON/XML/object scanning add-on (System.Text.Json)
src/RedactWire.Cli command-line scanner / redactor
samples/RedactWire.Sample.Web ASP.NET Core Razor Pages PII tester
tests/RedactWire.Tests xUnit tests
docs/rules/ per-rule documentation (common.md, severity.md, localized/)

CLI

echo "My SSN is 123-45-6789" | dotnet run --project src/RedactWire.Cli
dotnet run --project src/RedactWire.Cli -- --redact --text "Card 4242 4242 4242 4242"

Exit code is 1 when PII is found, 0 when clean — handy in CI.

License

Apache License 2.0. See LICENSE.

About

Culture-aware PII detection, validation & redaction for .NET. Checksum-validated rules for 58 countries — SSN, Aadhaar, NRIC, MyKad, CPF, CURP, 身份证, マイナンバー, 주민등록번호, ИНН and more. JSON/XML/object scanning, severity-driven redaction.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages