diff --git a/README.md b/README.md index fb6352e60..1b916e56d 100644 --- a/README.md +++ b/README.md @@ -166,12 +166,16 @@ OpenTaint includes agent skills that turn static analysis into an end-to-end app npx skills add https://github.com/seqra/opentaint ``` -The `appsec-agent` skill orchestrates a full project assessment: build the project, run OpenTaint, discover the attack surface, add targeted rules, model missing library data flows, triage findings, and optionally generate dynamic proof-of-concept checks for confirmed vulnerabilities. +The `appsec-agent` skill is the entry point: it builds the project, runs OpenTaint, turns the project's attack surface into universal rules, models missing library data flows, triages findings, and optionally generates dynamic proof-of-concept checks for confirmed vulnerabilities. It runs in one of three modes, differing only in what the run starts from: + +- **onboarding** — the external-method frontier, taken as a trust boundary and classified once, to build the project's universal rule and model corpus +- **discovery** — the project, a diff, or an informal spec of what changed or what matters +- **enactment** — a finding set you supply, reproduced as verified rules Included skills cover the common security-analysis loop: - **Scan and triage:** `build-project`, `run-scan`, `analyze-findings`, `generate-poc` -- **Coverage expansion:** `triage-dependencies`, `discover-attack-surface`, `create-test-project`, `create-rule`, `assemble-lib-rules` +- **Coverage expansion:** `triage-dependencies`, `discover-attack-surface`, `discover-universal-boundaries`, `create-test-project`, `create-rule`, `assemble-lib-rules` - **Dataflow modeling:** `analyze-external-methods`, `create-pass-through-approximation`, `create-dataflow-approximation`, `debug-rule`, `report-analyzer-issue` --- diff --git a/skills-templates/appsec-agent/main.md.j2 b/skills-templates/appsec-agent/main.md.j2 index bc58f87ea..5009c31d7 100644 --- a/skills-templates/appsec-agent/main.md.j2 +++ b/skills-templates/appsec-agent/main.md.j2 @@ -1,6 +1,6 @@ --- name: appsec-agent -description: Run an end-to-end OpenTaint application-security analysis while owning the long project build and scans and delegating each other pipeline stage. Use when the user asks to find vulnerabilities, or scan an application for security issues +description: Run an end-to-end OpenTaint application-security analysis in one of three modes — onboarding a project's dependency frontier, discovering vulnerabilities from the project, a diff, or a spec, or enacting a supplied finding set — owning the long project build and scans and delegating each other pipeline stage. Use when the user asks to find vulnerabilities, scan an application for security issues, reproduce or validate a supplied finding set, or continue an OpenTaint run license: Apache-2.0 metadata: author: opentaint @@ -9,6 +9,10 @@ metadata: {% include "sections/preamble.md" %} +## Modes + +{% include "sections/modes.md" %} + ## Setup {% include "sections/setup.md" %} diff --git a/skills-templates/appsec-agent/sections/dispatching.md b/skills-templates/appsec-agent/sections/dispatching.md index 305ca095c..98d46d57b 100644 --- a/skills-templates/appsec-agent/sections/dispatching.md +++ b/skills-templates/appsec-agent/sections/dispatching.md @@ -3,17 +3,20 @@ Dispatch exactly one stage-orchestrator subagent for each stage invocation: ``` Invoke the Skill orchestrate-stage first, then follow its instructions precisely Inputs: - stage: + stage: ``` -For a `deep` approximation round, also pass `sinks: true`. A subagent inherits the project-root working directory, so omit `project-root`. +For a `deep` model round, also pass `sinks: true`. A subagent inherits the project-root working directory, so omit `project-root`. Stage context: -- `sources` — discover dependency sources, author their rules, and wire the joins +- `intake` — turn this mode's input into the run's families: the swept frontier in onboarding, the diff or spec in discovery, the normalized reference set in enactment +- `boundaries` — generalize each family into one universal source and one universal sink, and seed the rule units from them +- `sources` — author the seeded source units' rules and wire the joins - `approx-round` — classify and build one dropped-method frontier; use a fresh agent for each new frontier -- `sinks` — author classified sink rules and wire the joins +- `sinks` — author the seeded sink units' rules and wire the joins - `triage` — classify the latest findings and refresh the vulnerability report +- `crossref` — judge a reference set against the latest scan and refresh its coverage manifest - `poc` — reproduce confirmed findings and add the outcomes to the report - `escalation` — repair or settle a stage artifact, or report a scan-wide no-SARIF failure diff --git a/skills-templates/appsec-agent/sections/key-constraints.md b/skills-templates/appsec-agent/sections/key-constraints.md index 66d68e7d7..bec6b049b 100644 --- a/skills-templates/appsec-agent/sections/key-constraints.md +++ b/skills-templates/appsec-agent/sections/key-constraints.md @@ -1,3 +1,7 @@ -- read pipeline state through `/scripts/get_status.py`, not by hand — don't re-derive it with glob/grep/`python3 -c`/yaml scans over `.opentaint/tracking`, `results`, or the `*.yaml`, nor open finding/unit/SARIF files just to review progress. If its output doesn't settle the question, re-run it with `--full` before opening any file +- read pipeline state through `/scripts/get_status.py`, not by hand — don't re-derive it with glob/grep/`python3 -c`/yaml scans over `.opentaint/tracking`, `results`, or the `*.yaml`, nor open finding/unit/reference/SARIF files just to review progress. If its output doesn't settle the question, re-run it with `--full` before opening any file - don't author or edit stage-owned artifacts or tracking; MAIN writes only `model_commit`, `build_jdk`, and `max_memory` in `state.yaml` - keep one generated project model for the run; never hand-edit or replace it mid-analysis — fix the build and rebuild before starting a new run +- source and sink boundaries come before model work; a model never compensates for a boundary that was never authored +- run one mode per pass, and never switch `mode` mid-pass: the intake behind the current families would no longer be the one on disk +- where the tree carries a reference set, coverage is counted by unique finding identity, never by rule id or raw SARIF result count, and a result counts as a reproduction only when its trace carries the finding's own attack path +- never drop a supplied finding as unsuitable for taint analysis diff --git a/skills-templates/appsec-agent/sections/modes.md b/skills-templates/appsec-agent/sections/modes.md new file mode 100644 index 000000000..770938759 --- /dev/null +++ b/skills-templates/appsec-agent/sections/modes.md @@ -0,0 +1,46 @@ +### What each mode takes in + +- **onboarding** — no input beyond the project itself. The frontier sweep flags the dependencies that can carry untrusted data, partitions the members the project actually calls, and verdicts each one: a trust boundary, an effect, or neither. It is the widest intake there is, and its output — a classified frontier, universal rules for the stack, and the models behind them — is what makes every later pass cheap +- **discovery** — a diff, a spec, a ticket, or a sentence about what the project does and what would be bad. Ask for whatever the user has and record its path; with nothing supplied the scope is the whole project. Intake reads it, resolves it to code, and groups that code into families +- **enactment** — the supplied findings, as a manifest, SARIF, scanner report, or a directory of finding documents. Ask for the path when it isn't given. If the user only described the findings in conversation, write them to a file first and use that; the pipeline resumes from disk, not from this thread + +### Onboarding runs once + +The frontier sweep is the expensive pass, and its corpus is durable: the classification ledger, the universal rules, and the models stay on disk and apply to every scan afterwards, whichever mode ran it. So onboard a project once and then work in discovery or enactment. Bootstrap refuses a second onboarding pass over a tree that already had one and says so; a genuinely new dependency stack is a new tree. + +### The modes compose + +They are not alternatives, and picking one is not a commitment. One `.opentaint/` tree accumulates the artifacts of every pass over it, in any order and as many times as the project needs: + +- **onboarding, then anything** — the natural start. A discovery or enactment pass over an onboarded tree finds most of its boundaries already ruled and modeled, and spends its work on what its own input names +- **enactment, then discovery** — reproduce the supplied report first, then hunt with the rules it produced. Boundaries derived from real findings are exactly the ones a discovery pass would otherwise have to argue for +- **discovery, then enactment** — assess the project, then measure a report against the corpus that pass built. What the report names but the scan missed is now a rule or modeling gap you can point at +- **any of them, again on a later commit** — the tree is long-lived. A new HEAD makes the model stale, so the pass rebuilds and rescans, and every rule, model, and verdict carries over. That's how a run becomes a regression check rather than a one-off + +So `mode` in `state.yaml` is the intake of the *current pass*, not a property of the tree. Switching it is normal, needs no fresh tree, and strands nothing. + +### Read what the tree already holds + +If `.opentaint/tracking/state.yaml` exists, find out where the project stands before choosing: + +```bash +uv run /scripts/get_status.py --full +``` + +Its header prints the current `mode`, the run's levels, the tracked finding set or spec if there is one, and — once the tree has more than one pass — the `passes:` chain. The phase lines say whether that pass is finished or mid-flight. + +- mid-flight pass — resume it rather than starting a different one over the top of it +- finished pass, and the user wants more — that's a new pass, and the choice below applies again +- tell the user what's there either way, in one line: which pass, where it stands, what carried over + +### Choose this pass + +Decide from what the user brought, then confirm it with them before bootstrapping: + +- **enactment** — they supplied findings and want them reproduced, validated, converted into reusable rules, or cross-checked against OpenTaint. "Does OpenTaint catch these?", "reproduce this report", "turn these findings into rules" +- **discovery** — no finding set to measure against; the goal is what the project is vulnerable to, in the whole project or in what a diff or spec names. "Find vulnerabilities", "scan this app", "did this PR introduce anything?" +- **onboarding** — the tree has never been onboarded and the user wants the project's own rule and model corpus built before anything is measured. Recommend it on a cold start when the run is not urgent; it is the pass that makes the others accurate + +The signal for enactment is whether a finding set exists to be measured against, not the vocabulary. A user who says "audit this against last year's pentest" and has the pentest is enactment; a user who says "reproduce the bug I think is in here" and has only a hunch is discovery. + +When the user wants both — reproduce the report *and* find what it missed — say that it is two passes over one tree, recommend enactment first so the discovery pass inherits its boundaries, and run them one at a time. Never try to drive two modes in a single pass. diff --git a/skills-templates/appsec-agent/sections/preamble.md b/skills-templates/appsec-agent/sections/preamble.md index 81065ad8a..b450b9eea 100644 --- a/skills-templates/appsec-agent/sections/preamble.md +++ b/skills-templates/appsec-agent/sections/preamble.md @@ -1,5 +1,13 @@ # AppSec Agent -Orchestrate an end-to-end OpenTaint security analysis. Keep the long project build and every full-project scan in this main session; delegate each bounded source, approximation, sink, triage, and PoC stage to an `orchestrate-stage` subagent, which owns its leaf fan-out and joins. +Run an end-to-end OpenTaint security analysis. Keep the long project build and every full-project scan in this main session; delegate each bounded intake, boundary, rule, approximation, triage, and PoC stage to an `orchestrate-stage` subagent, which owns its leaf fan-out and joins. -OpenTaint is a whole-program, interprocedural, field-sensitive alias analysis SAST. The run produces confirmed vulnerabilities plus reusable project-specific rules and approximations under one self-contained `.opentaint/` directory at the project root. +OpenTaint is a whole-program, interprocedural, field-sensitive alias analysis SAST. A run produces confirmed vulnerabilities plus the project's own universal rules and the models behind them — the passThrough and dataflow approximations that carry taint through library code, which the pipeline's `approximations` phase builds. Everything lands under one self-contained `.opentaint/` directory at the project root. + +This is the only entry point. It runs in one of three modes, which differ in what the run takes as its input — and therefore in where the universal rules come from — and in nothing else: + +- **onboarding** — the external-method frontier: every dependency member the project's own code calls, taken as a trust boundary until a leaf verdicts it. Run once per project, to build the universal rule and model corpus the later passes inherit +- **discovery** — the project, a diff, or an informal spec of what changed or what matters; the code it names becomes the boundary evidence +- **enactment** — a finding set the user supplies: a report, pentest results, or source-to-sink traces, reproduced as verified rules + +Whatever the input, intake groups it into families, the boundaries stage generalizes each family into one universal source and one universal sink, and everything downstream is the same pipeline in the same order. diff --git a/skills-templates/appsec-agent/sections/setup.md b/skills-templates/appsec-agent/sections/setup.md index bbe498665..bea87168f 100644 --- a/skills-templates/appsec-agent/sections/setup.md +++ b/skills-templates/appsec-agent/sections/setup.md @@ -11,29 +11,38 @@ After installing, run `opentaint health` to confirm everything's resolved. This workflow requires two subagent levels: MAIN → stage orchestrator → leaf. Confirm the harness permits depth 2 before starting; otherwise ask the user to enable it. -### 3. Determine the language +### 3. Choose the mode and locate its input -Read the project's build files to fix the target language — Maven/Gradle → java, `go.mod` → go, and so on. Record it at bootstrap; stage orchestrators pass it to language-coupled leaves. +Pick `onboarding`, `discovery`, or `enactment` per Modes, confirm the choice with the user, and get the path its intake needs — the supplied findings for enactment, the diff or spec for discovery when there is one. Onboarding needs no input beyond the project. -### 4. Choose the workflow +### 4. Determine the language -Ask the user for both levels together: +Read the project's build files to fix the target language — Maven/Gradle → java, `go.mod` → go, and so on. Record it at bootstrap; stage orchestrators pass it to language-coupled leaves. When a repo carries build markers for more than one, ask the user which to analyze. -1. Scan level — `lite` · `normal` · `deep` +### 5. Choose the workflow + +Ask the user for the knobs this mode has, together: + +1. Scan level — `lite` · `normal` · `deep`, **discovery mode only** - lite — build + scan (expected, when there are already existing artifacts) - - normal — build + scan + custom approximations - - deep — build + scan + custom approximations + custom rules - - recommend by what's on disk: a cold start (no `.opentaint` artifacts) → deep; a prior run's artifacts already present → lite + - normal — build + scan + custom models + - deep — build + scan + custom models + custom universal rules + - recommend by what's on disk: a cold start (no `.opentaint` artifacts) → deep; a prior pass's artifacts already present → lite + - onboarding and enactment are always deep: sweeping the frontier and reproducing a finding set both need the full rule and model toolbox, so there is no level to ask for 2. Triage level — `static` · `dynamic` - static — classify findings from the model, no running app - dynamic — static + PoC per confirmed TP. This launches a few test services on the user's machine (local instances and ports), torn down at the end of the run. Make that clear in the option -### 5. Bootstrap +### 6. Bootstrap -Seed the run state and the working tree with the chosen levels and language: +Seed the run state and the working tree: ```bash -uv run /scripts/generate.py init --scan-level --triage-level --language +uv run /scripts/generate.py init --mode \ + --triage-level --language \ + [--scan-level ] [--findings ] [--spec ] ``` -It writes `state.yaml`, seeds `history.yaml`, and creates the `.opentaint/` tree. +`--scan-level` is discovery's; `--findings` is enactment's and required the first time; `--spec` is discovery's and optional. It writes `state.yaml`, appends this pass to `history.yaml`, and creates the `.opentaint/` tree — plus `tracking/reference/` in enactment mode. + +Over a tree an earlier pass already built, it prints what carried over and keeps all of it — that pass's rules, models, boundary specs, and verdicts are this pass's starting corpus, and `get_status.py` will report their phases `DONE` rather than redoing them. It refuses a second onboarding pass over an already-onboarded tree. diff --git a/skills-templates/appsec-agent/sections/state-resumption.md b/skills-templates/appsec-agent/sections/state-resumption.md index 2b3701834..6e5afea2f 100644 --- a/skills-templates/appsec-agent/sections/state-resumption.md +++ b/skills-templates/appsec-agent/sections/state-resumption.md @@ -2,26 +2,35 @@ Use this ownership map to route work and scan errors: ``` .opentaint/ - project/ MAIN build - results/ MAIN scan - rules/ sources or sinks stage - pass-through/ approximation stage - dataflow/ approximation stage - tracking/state.yaml MAIN run knobs - tracking/ stage agents, leaves, and join scripts otherwise - vulnerabilities.md triage / PoC stage - issues/ escalation stage + project/ MAIN build + results/ MAIN scan + rules/ sources or sinks stage + pass-through/ approximation stage + dataflow/ approximation stage + tracking/state.yaml MAIN run knobs + tracking/scope.yaml intake stage (onboarding, discovery) + tracking/reference/ intake stage writes, crossref stage judges (enactment) + tracking/boundaries/ boundaries stage + tracking/ stage agents, leaves, and join scripts otherwise + enactment.md crossref stage + vulnerabilities.md triage / PoC stage + issues/ escalation stage ``` -The tree is long-lived. On resume, reuse `DONE` artifacts; `get_status.py` derives the next phase from disk. Existing rules and approximations apply to every scan. +The tree is long-lived and outlives this pass. On resume, reuse `DONE` artifacts; `get_status.py` derives the next phase from disk. Existing rules and models apply to every scan, whichever pass created them. Never delete or rewrite an artifact because this pass didn't produce it — an onboarding pass's classification ledger, a discovery pass's boundary specs, and an enactment pass's reference set are all as durable as your own. -`state.yaml` shape: +`state.yaml` shape — `mode` is this pass's intake, not a property of the tree, so a later pass in another mode simply rewrites it and keeps everything else: ```yaml +mode: enactment scan_level: deep triage_level: dynamic language: java +findings: reports/pentest-2026-07.md +spec: null model_commit: 0123456789abcdef0123456789abcdef01234567 build_jdk: null max_memory: null ``` + +`findings` is the supplied set an enactment pass is measured against and `spec` is what a discovery pass was scoped by; both stay in `state.yaml` across passes in other modes, so a later pass in that mode resumes the same input. Neither is edited by hand mid-pass — pointing an in-flight pass at a different file strands the intake built from the old one. A genuinely different input is a new pass, bootstrapped with a new `--findings` or `--spec`. diff --git a/skills-templates/appsec-agent/sections/workflow.md b/skills-templates/appsec-agent/sections/workflow.md index 3b51e17ad..8aabceaa0 100644 --- a/skills-templates/appsec-agent/sections/workflow.md +++ b/skills-templates/appsec-agent/sections/workflow.md @@ -1,27 +1,40 @@ -The run is one fixed pipeline; the selected levels determine which phases are in scope. Use `uv run /scripts/get_status.py` to choose the next action: +The run is one fixed pipeline, the same in every mode; the mode decides what intake works from, and the selected levels decide which phases are in scope. Use `uv run /scripts/get_status.py` to choose the next action: ``` build → MAIN: build -discover / source_rules → stage subagent: sources +intake → stage subagent: intake +boundaries → stage subagent: boundaries +source_rules → stage subagent: sources scan → MAIN: scan approximations → stage subagent: approx-round, then MAIN: rescan; repeat sink_rules → stage subagent: sinks, then MAIN: rescan triage → stage subagent: triage poc → stage subagent: poc +crossref → stage subagent: crossref (whenever the tree carries a reference set) ``` +The spine is fixed — build, sources, scan, models, sinks, triage — and it is fixed for a reason. Both boundary sides are known before the first scan, because that scan is what proves them and names the taint frontier the model work then answers; the sink rules are authored against that frontier rather than guessed ahead of it. Never let a model stand in for a boundary that was never authored: an approximation carries taint through a carrier, it does not decide what is untrusted or what is dangerous. + +`crossref` appears whenever the tree carries a reference set — the pass that supplied it need not be this one. This pass's rescans changed what those findings reproduce, so re-judging them and refreshing `.opentaint/enactment.md` is part of finishing, not optional cleanup. + ### Build in MAIN When status reports `build`, load and follow the `build-project` skill in this main session. Run its long build command through the harness's main-session background-command facility and wait for its completion event. -Record the returned `build_jdk` in `.opentaint/tracking/state.yaml`. Record `model_commit` as the full HEAD only when no source file is uncommitted, otherwise set it to null. Build non-convergence blocks the run because no later phase can proceed without the model. +Record the returned `build_jdk` in `.opentaint/tracking/state.yaml`. Record `model_commit` as the full HEAD only when no source file is uncommitted, otherwise set it to null. Build non-convergence blocks the run because no later phase can proceed without the model. Keep one untouched model for the whole pass. ### Scan in MAIN When status reports `scan`, or a stage returns with a rescan pending, load and follow the `run-scan` skill in this main session. Start the scan with the harness's main-session background-command facility, keep the engine's self-timeout, add a 1200-second outer backstop, and wait for the process completion event. -A valid `.opentaint/results/report.sarif` means the scan completed, including exit 254 after an engine timeout. Record `max_memory: 16G` when the scan had to bump memory and reuse it on later scans. If no SARIF exists after the allowed retry/backstop, follow the repair path below for a malformed rule/approximation; otherwise dispatch `orchestrate-stage` with `stage: escalation` and the scan `setup` to write the scan-wide resource issue, then stop. +A valid `.opentaint/results/report.sarif` means the scan completed, including exit 254 after an engine timeout. Record `max_memory: 16G` when the scan had to bump memory and reuse it on later scans. If no SARIF exists after the allowed retry/backstop, follow the repair path below for a malformed rule/model; otherwise dispatch `orchestrate-stage` with `stage: escalation` and the scan `setup` to write the scan-wide resource issue, then stop. -When a scan or later stage reports a malformed approximation, unloadable created rule, ineffective join, or a created rule's false positive/negative, route the exact diagnosis and artifact path/id to the responsible stage agent per Dispatching, then scan again in MAIN. +When a scan or later stage reports a malformed model, unloadable created rule, ineffective join, or a created rule's false positive/negative, route the exact diagnosis and artifact path/id to the responsible stage agent per Dispatching, then scan again in MAIN. After every build, scan, or stage return, run `uv run /scripts/get_status.py` once to choose the next action. Use `--full` at run start, on resume, or when the brief output does not settle the question. + +### Iterating + +The pipeline loops by design, and status is what closes it. A rescan makes every cross-reference pending again; a trace stopped at an opaque carrier sends the run back to a model round; a rule blamed for a miss goes back to the stage that authored it, so status returns there before closing again. Follow status through each loop rather than declaring the run finished early — an earlier phase reading `IN_PROGRESS` after a later one ran is the loop working, not a regression. + +The run is complete when status reports `run complete`: `.opentaint/vulnerabilities.md` current, and — where the tree carries a reference set — every supplied finding either reproduced or recorded with the blocker that stopped it, in `.opentaint/enactment.md`. Report the outcome keeping raw SARIF results, validated findings, and unique vulnerability identities as separate counts. diff --git a/skills-templates/create-pass-through-approximation/references/java.md.j2 b/skills-templates/create-pass-through-approximation/references/java.md.j2 index 238e762c5..9022e8d6e 100644 --- a/skills-templates/create-pass-through-approximation/references/java.md.j2 +++ b/skills-templates/create-pass-through-approximation/references/java.md.j2 @@ -179,6 +179,7 @@ passThrough: ### 3. Common mistakes to check +- *every* method in a file still dropped, with no load error — the whole file was silently skipped: check it starts with the `language: java` header (a headerless or mis-headered file loads to nothing) - the `function` matcher doesn't match the real method — check the package, class, name, and `overrides` - a `from`/`to` points at the wrong position — it must land where the data actually is - a broken slot channel — the writer and reader name different `Class#slot#java.lang.Object` triples, or the slot isn't typed `java.lang.Object`, so the taint drops between them diff --git a/skills-templates/discover-attack-surface/main.md.j2 b/skills-templates/discover-attack-surface/main.md.j2 index 4d8142730..26d5d54ef 100644 --- a/skills-templates/discover-attack-surface/main.md.j2 +++ b/skills-templates/discover-attack-surface/main.md.j2 @@ -1,6 +1,6 @@ --- name: discover-attack-surface -description: Classify project-used dependency members and record the taint sources among them not covered by the built-in rules. Use for the source-discovery depth pass +description: Classify a slice of the external-method frontier — the dependency members the project calls — as trust boundaries or not, and record the taint sources among them that the built-in rules don't cover. Use for the frontier sweep of an onboarding pass, or wherever a scoped attack surface has to be classified license: Apache-2.0 metadata: author: opentaint diff --git a/skills-templates/discover-attack-surface/sections/preamble.md b/skills-templates/discover-attack-surface/sections/preamble.md index 85162b006..320f248af 100644 --- a/skills-templates/discover-attack-surface/sections/preamble.md +++ b/skills-templates/discover-attack-surface/sections/preamble.md @@ -1,3 +1,3 @@ # Skill: Discover Attack Surface -Work one assignment of project-used dependency members and pick out the taint sources among them — the methods where untrusted data first enters. The concrete inspection commands and value formats are language-specific — read `references/.md` per Inputs and follow its numbered steps, which key to the ones below +Work one assignment of the external-method frontier — the dependency members the project's own code calls — and pick out the trust boundaries among them: the methods where untrusted data first enters. Every member on the plan arrives as a candidate boundary and leaves with a verdict, so a member you leave unjudged is a gap in the sweep, not a default-safe. The concrete inspection commands and value formats are language-specific — read `references/.md` per Inputs and follow its numbered steps, which key to the ones below diff --git a/skills-templates/discover-universal-boundaries/main.md.j2 b/skills-templates/discover-universal-boundaries/main.md.j2 new file mode 100644 index 000000000..1ee6665b7 --- /dev/null +++ b/skills-templates/discover-universal-boundaries/main.md.j2 @@ -0,0 +1,30 @@ +--- +name: discover-universal-boundaries +description: Generalize a family of attack-surface evidence — finding traces, swept frontier members, or the code a diff or spec names — into one reusable source and one reusable sink boundary, saturated against the whole family. Use before rule authoring when reproducing supplied findings, turning a discovered surface into universal rules, consolidating case-specific rules, or replacing an incidental source such as a map access or an arbitrary method call +license: Apache-2.0 +metadata: + author: opentaint + version: "{{ version }}" +--- + +{% include "sections/preamble.md" %} + +## Inputs + +{% include "sections/input.md" %} + +## Workflow + +{% include "sections/workflow.md" %} + +## Output + +{% include "sections/output.md" %} + +## Tracking + +{% include "sections/tracking.md" %} + +## Constraints + +{% include "sections/constraints.md" %} diff --git a/skills-templates/discover-universal-boundaries/references/java.md.j2 b/skills-templates/discover-universal-boundaries/references/java.md.j2 new file mode 100644 index 000000000..a0652708b --- /dev/null +++ b/skills-templates/discover-universal-boundaries/references/java.md.j2 @@ -0,0 +1,50 @@ +# discover-universal-boundaries — Java / JVM + +## Workflow + +### 1. Reconstruct every trace + +Dependency members come from the resolved jars under `.opentaint/project/dependencies`: locate a class with `unzip -l | grep `, read its signatures with `javap -p -s -classpath `, and prefer the source jar or a decompiler when the body matters. + +Java-shaped propagation that is not ingress: a DTO or entity getter, `Map#get` after the request was already bound, a service method delegating a value it was handed, an `Optional`/stream chain over it, and framework binding of a value the endpoint already accepted. + +### 2. Propose the source + +Built-in Java source rules live under `java/lib/{generic,spring}/` within the `opentaint health --rules` root, the project's own under `.opentaint/rules/java`. A boundary they already match needs no new one, and their reach is wider than it looks: + +- `lib/spring/untrusted-data-source.yaml` taints *every* non-scalar parameter of a method annotated `@RequestMapping`/`@GetMapping`/`@PostMapping`/`@PutMapping`/`@PatchMapping`/`@DeleteMapping`, whatever the parameter's own annotation — so `@RequestBody`, `@RequestParam`, `@PathVariable`, and `@ModelAttribute` on a Spring MVC endpoint are already sourced, as is a cookie read through `WebUtils#getCookie` +- `lib/generic/servlet-untrusted-data-source.yaml` taints the `HttpServletRequest` parameter of a servlet entry method (`doGet`, `doPost`, `doPut`, `doDelete`, `doTrace`, `_jspService`) as a whole object, plus `MessageBodyReader#readFrom`, a commons-fileupload `parseRequest(...).getName()`, and `Part#getSubmittedFileName` +- `lib/spring/untrusted-path-source.yaml` and `lib/generic/seam-untrusted-data-source.yaml` carry their own shapes + +A Java family's shared source is therefore usually *outside* those shapes, and that is where to aim: a non-HTTP transport (`TextMessage#getPayload`, `ConsumerRecord#value`, a JMS or gRPC payload), a reactive `ServerRequest`/`ServerHttpRequest` accessor, a non-Spring stack's own request accessor, or a second-order re-entry where a persisted attacker-controlled record is read back. + +Two shapes carry a Java boundary: an annotation on the declaration, with the enclosing annotated class or method as its `pattern-inside` context, or an accessor on a boundary type. Take the annotated parameter or the accessor itself, never the DTO getter downstream of it — the getter would taint trusted objects just as readily, and the binding step is already propagation. + +### 3. Propose the sink + +Where the primitive effects live, with the built-in `java/lib/` coverage named: + +- SSRF — `RestTemplate`, `URL#openConnection` (`generic/ssrf-sinks.yaml`); `WebClient` and `HttpClient#send` are not built in +- command injection — `Runtime#exec`, `ProcessBuilder` (`generic/command-injection-sinks.yaml`) +- SQL — `Statement#execute*`, `JdbcTemplate`, an `EntityManager` native query (`spring/jdbc-sqli-sinks.yaml`) +- path traversal — `Files#*` reached through `Paths#get`/`Path#resolve` (`generic/path-traversal-sinks.yaml`) +- SSTI, expression, and code injection — the template engine's `process`/`merge` (`generic/template-injection-sinks.yaml`), `SpelExpressionParser` (`spring/spel-injection-sinks.yaml`), `generic/code-injection-sinks.yaml` +- deserialization — `ObjectMapper#readValue`, a SnakeYAML `load` (`generic/unsafe-deserialization-sinks.yaml`); `ObjectInputStream#readObject` is not built in +- log injection — the `Logger` call's message or argument (`generic/logging-sinks.yaml`) +- and their own files for XXE, LDAP, SMTP, reflection, unvalidated redirect, response splitting, and XSS response writes + +Check the bundled set before proposing a sink boundary: it is broad enough that a family often needs only the source side. Prefer the library boundary over the project wrapper that calls it — a reusable library sink keeps the rule useful past this run. + +### 5. Identify the precision controls + +Java's validation impostors: `@Valid`/`@Validated` with Bean Validation constraints, `@NotNull`, `@Size`, or `@Pattern` on a shape that is not the invariant, and a `parse`/`valueOf` that only proves the value's type. Record them as not-security-relevant validators rather than sanitizers. + +Real Java controls: `InetAddress#getByName` plus a private-range rejection for SSRF, `Path#normalize` plus a `startsWith` containment check for traversal, an owner-scoped repository lookup for IDOR, `MessageDigest#isEqual` or HMAC verification for callbacks, and a context-correct encoder or CR/LF stripping for output and log injection. + +A restriction usually reads as `pattern-inside` on the enclosing annotated class or method — that is what keeps a universal accessor scoped to the endpoint that actually takes untrusted input. + +### 6. Write the specification + +`candidate_patterns` entries take `method` as `owner.Class#member` with `signature` its quoted JVM descriptor, so overloads stay distinct. An annotation-carried boundary takes the annotation's own FQN as `method` with `signature: null`, as the spec format shows. The dependency identity to record on the seeded units is the Maven GAV, `group:artifact:version`. + +`create-rule`'s `references/java.md` holds the pattern shapes those units turn into. diff --git a/skills-templates/discover-universal-boundaries/sections/constraints.md b/skills-templates/discover-universal-boundaries/sections/constraints.md new file mode 100644 index 000000000..38043d2dc --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/constraints.md @@ -0,0 +1,7 @@ +- Work only the assigned evidence, your family's spec, and the reference `family` field on a split +- Universal does not mean untyped or unconstrained — generalize the boundary, then recover precision with context restrictions and sanitizers, never by re-narrowing the boundary to the evidence +- Never propose all methods, all map values, or all getters as a source, and never a `pattern-either` with one branch per evidence item +- Never mark the spec saturated on a round that widened, broke, or split anything +- Never propose a sanitizer that would suppress a trace the evidence says is real, and never record a validator as a sanitizer without the invariant it enforces +- Record a poor taint fit as an explicit pseudo-boundary; never drop the evidence item +- Don't create or recommend approximations here — note the opaque carriers and stop diff --git a/skills-templates/discover-universal-boundaries/sections/input.md b/skills-templates/discover-universal-boundaries/sections/input.md new file mode 100644 index 000000000..bfae2bbd8 --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/input.md @@ -0,0 +1,7 @@ +{% include "shared/inputs-preamble.md" %} + +{% include "shared/inputs/project-root.md" %} +{% include "shared/inputs/language.md" %} +- `family` (required) — kebab-case name of the family; also the name of its spec and of the rule units seeded from it +- `evidence` (required) — the ids assigned to this family. Reference finding ids when the pass has a reference set, whose normalized files are `.opentaint/tracking/reference/.yaml`; otherwise the members or code areas `.opentaint/tracking/scope.yaml` recorded for the family +- `findings` (optional) — path to the supplied finding manifest or report the reference set was normalized from, when there is one. Read it for detail a normalized file doesn't carry diff --git a/skills-templates/discover-universal-boundaries/sections/output.md b/skills-templates/discover-universal-boundaries/sections/output.md new file mode 100644 index 000000000..a3a31e6eb --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/output.md @@ -0,0 +1,11 @@ +Return every spec path written (one per family or subfamily) and, in a few lines each: + +- the universal source, and which more incidental candidates you rejected; +- the universal sink, and the vulnerability-class scope it stays inside; +- how many saturation rounds it took and what the last round changed; +- the assigned evidence by factorization status — `covered`, `needs-restriction`, `unfactored`; +- any family split, with the evidence that moved and — when it is a reference finding — the files you rewrote; +- the sanitizers, negative patterns, and context restrictions listed separately from the boundaries; and +- unresolved boundary evidence, as the `open_questions` entries. + +Never paste file contents. diff --git a/skills-templates/discover-universal-boundaries/sections/preamble.md b/skills-templates/discover-universal-boundaries/sections/preamble.md new file mode 100644 index 000000000..909f0eb46 --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/preamble.md @@ -0,0 +1,7 @@ +# Discover universal boundaries + +Generalize one family of known attack-surface evidence into rule-ready boundaries: a single source and a single sink that every item in the family factors through. The evidence differs with the pass that produced it — a reference finding's trace, a frontier member a sweep verdicted, a code area a diff or spec named — and the reasoning is the same for all three. The boundary vocabulary — how a member is named, what carries a usage condition, which packages realize each primitive effect — is language-specific: read `references/.md` per Inputs and follow its numbered steps, which key to the ones below. + +The goal is not the broadest syntax that happens to match every trace. It is the most primitive semantic boundary that stays inside the vulnerability class — general enough that the family needs one rule per side, specific enough that the rule still means something. Precision is recovered afterwards with context restrictions and sanitizers, listed separately, never by narrowing the boundary back down to the evidence you started from. + +A boundary is only universal once it has been saturated: widened, re-checked against the whole family, and left unchanged by a full round. diff --git a/skills-templates/discover-universal-boundaries/sections/tracking.md b/skills-templates/discover-universal-boundaries/sections/tracking.md new file mode 100644 index 000000000..4958025d8 --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/tracking.md @@ -0,0 +1,5 @@ +This skill writes the boundary spec for its family, and — only when a split moves a reference finding — the `family` field on the reference files it moved. Leave `stages.units_seeded` pending: seeding the rule units is the calling stage's step. Touch no other reference field, no scope file, no rule unit, and no rule. + +{% include "shared/tracking/boundary-spec.md" %} + +{% include "shared/tracking/reference-finding.md" %} diff --git a/skills-templates/discover-universal-boundaries/sections/workflow.md b/skills-templates/discover-universal-boundaries/sections/workflow.md new file mode 100644 index 000000000..d3364f05d --- /dev/null +++ b/skills-templates/discover-universal-boundaries/sections/workflow.md @@ -0,0 +1,84 @@ +### 1. Reconstruct every trace + +For each assigned evidence item, read what the pass recorded for it — a reference finding's normalized file, or the member or code area `scope.yaml` named — and then the project source it points at, and record: + +1. the attacker or untrusted authority; +2. the first project-visible ingress; +3. the transformations and trust-domain crossings on the way; +4. the validators, guards, and authorization decisions it passes; +5. the primitive security-relevant effect at the end; and +6. the vulnerable invariant that is absent or defeated. + +Read enough surrounding source to tell the real boundary from incidental syntax — the language reference says how to reach a dependency member's source, and which local shapes are propagation rather than ingress. A getter, collection lookup, or service method is usually propagation or context, not ingress. + +### 2. Propose the source + +Move backward from the evidence's own expressions until you reach the earliest reusable trust-boundary value the family shares. Prefer, in order when applicable: + +- request body, parameter, path, query, header, cookie, or multipart value; +- message, frame, packet, event, webhook, or callback payload; +- deserialized external object; +- persisted attacker-controlled record at a second-order re-entry boundary; +- tenant or user-controlled configuration at activation or read-back; +- environment or runtime configuration entering a security decision; or +- an explicit structural pseudo-source standing for attacker-selected identity or control state. + +The language reference names the boundaries its built-in source rules already carry — one they match needs no new boundary — and what identifies a boundary in that language. Reject a candidate that is merely a map or collection read after untrusted data has already entered, a `$METAVAR.method(...)` with no boundary type, signature, declarative marker, or enclosing entrypoint, a ubiquitous getter that would taint trusted objects just as readily, or an internal carrier that ordinary propagation or a later approximation should handle. + +Express narrow usage conditions separately, as typed patterns, `pattern-inside`, `...`, and whatever declarative marker the language offers. Never bake incidental access syntax into the source. + +### 3. Propose the sink + +Move forward from the evidence's own service calls to the most primitive operation that realizes the vulnerability, while staying specific to its class. Prefer boundaries such as: + +- network connect, request, send, or download for SSRF; +- process, script, expression, template, query, or deserialization execution for injection; +- path resolution plus filesystem read/write for traversal; +- privileged object read or disclosure for IDOR and data exposure; +- privileged object mutation or durable state commit for integrity and authorization failures; +- authentication or authorization decision for control bypass; +- redirect or token release for redirect and OAuth issues; +- signature verification or unsigned callback state commit for callback integrity; +- secret activation, credential construction, or outbound use for secret exposure; and +- logger argument consumption for log injection. + +The language reference names the packages that realize these effects, and its built-in sinks. Don't stop at a controller-to-service call when the primitive effect is analyzable deeper in the project or a reusable library sink can express it. Use a structural sink only when the vulnerability *is* the missing control at that boundary, or when deeper propagation is genuinely unavailable. + +### 4. Saturate + +A boundary proposed from a few traces is a guess until it survives the whole family. Loop until a full round changes nothing. + +Each round: + +1. Factor every assigned evidence item — not only the new ones — through the current boundaries: + + ```text + universal source -> item-specific context -> propagation -> universal sink + ``` + +2. For each item that does not factor, generalize the offending side by exactly one step toward a more primitive boundary — never by adding a second alternative that merely spells out that item's syntax. A `pattern-either` listing one branch per evidence item is the failure this skill exists to prevent. +3. Re-check the items that already factored. A widening that breaks an earlier factorization is a widening too far: back it out and split instead. +4. Challenge the widened boundary in both directions — if the sink now admits a different vulnerability class, narrow it back to the primitive effect or add a class-specific context restriction; if the source now admits trusted values, record what separates them as a context restriction rather than shrinking the boundary. +5. Record the round: what changed, and which factorization statuses moved. + +The family is saturated when a full round widened nothing, broke nothing, and left every assigned item either `covered` or `needs-restriction` with a named restriction. Stop and split — or record the item `unfactored`, with what blocked it in `open_questions` — rather than looping a fourth time on the same one. + +When the only boundary the whole family shares is arbitrary syntax, the family was wrong: split it. Each subfamily gets its own spec named `-` and its own saturation loop, and every assigned item lands in exactly one subfamily. A moved reference finding has its own file's `family` rewritten to the subfamily that now owns it; other evidence moves with the spec it is listed in, and the calling stage reconciles the family list to the specs you return. + +Keep independently triggerable paths distinct in the factorization even when they share both boundaries. + +### 5. Identify the precision controls + +List these separately from the positive boundaries — the spec records them, and they must never be folded into the boundary patterns: + +- sanitizers that actually enforce the relevant invariant; +- `pattern-not` exclusions for safe expression forms; +- `pattern-not-inside` exclusions for safe guarded regions; +- `pattern-inside` contexts that constrain a universal boundary to the intended entrypoint, type, tenant, or vulnerability family; and +- validators that are *not* security-relevant, recorded explicitly so nothing later mistakes them for sanitizers. + +Real sanitization looks like resolved-IP private-range rejection for SSRF, canonical-path containment for traversal, strict identifier ownership checks for IDOR, signature verification for callbacks, and CR/LF neutralization for log injection. Presence, length, and parsing alone normally sanitize none of these; the language reference names the usual impostors and what the real controls look like in that language. + +### 6. Write the specification + +Write one spec per family or subfamily, listing the assigned ids under `evidence`. `candidate_patterns` must be concrete enough for `create-rule` to test, in the member shape that language's rule units use — the language reference gives that shape and the dependency identity to record with it. Record opaque carriers under `approximation_candidates`, and stop there: no approximation is created or recommended until a rule-first scan proves a trace stops at one. diff --git a/skills-templates/orchestrate-stage/references/approx-round.md b/skills-templates/orchestrate-stage/references/approx-round.md index 95c1b0375..847747f20 100644 --- a/skills-templates/orchestrate-stage/references/approx-round.md +++ b/skills-templates/orchestrate-stage/references/approx-round.md @@ -37,6 +37,7 @@ It merges every batch's `skipped` / `engine_issues` into `approximations/skipped - `language` - `batch` - `methods` (optional) — only an explicitly assigned subset to redo + - dataflow step 1 → dispatch create-test-project - `language` - `type: dataflow` diff --git a/skills-templates/orchestrate-stage/references/boundaries.md.j2 b/skills-templates/orchestrate-stage/references/boundaries.md.j2 new file mode 100644 index 000000000..eaeffbb3d --- /dev/null +++ b/skills-templates/orchestrate-stage/references/boundaries.md.j2 @@ -0,0 +1,37 @@ +# Universal boundaries + +Generalize each family intake produced into one saturated universal source and one universal sink, and seed the source and sink units those boundaries imply. Every mode runs this stage: the families differ — a swept frontier, a diff or spec, a supplied finding set — but what comes out is the same universal rule material, and everything it writes feeds the ordinary rule-authoring stages. + +Read the family list from `get_status.py`, which takes it from `.opentaint/tracking/scope.yaml` in onboarding and discovery mode, and from the reference findings' own `family` field in enactment mode. + +## Discover the boundaries + +Fan out discover-universal-boundaries, one leaf per family. + +Inputs each: +- `language` +- `family` +- `evidence` — the ids the family carries: reference finding ids in enactment mode, the members or code areas `scope.yaml` recorded otherwise +- `findings` (enactment mode) — the supplied-findings path `state.yaml` names, so a leaf can read a finding beyond its normalized file + +Expect back — `.opentaint/tracking/boundaries/.yaml` with `saturation.status: saturated`, one `factorization` entry per assigned evidence item, and the controls listed separately from the positive boundaries. A leaf that splits its family writes one spec per subfamily; in enactment mode it also rewrites `family` on each reference finding it moved, and otherwise the split is recorded in the specs themselves — reconcile `scope.yaml` to the specs that came back so the family list and the specs still agree. + +{% include "shared/tracking/boundary-spec.md" %} + +A spec returning with an `unfactored` evidence item is not a failure to retry blindly — read its `open_questions`, and either re-dispatch the leaf with that item split out as its own family or accept the pseudo-boundary it proposes. + +## Seed the rule units + +For each saturated spec, write its `candidate_patterns` into the family's source and sink units, then set `stages.units_seeded: done` on the spec. The unit file name is the family, so the two sides and the spec stay tied together. + +{% include "shared/tracking/source-unit.md" %} + +{% include "shared/tracking/sink-unit.md" %} + +Copy `method`, `signature`, and `note` from the spec's `candidate_patterns` (plus `vuln_class` on the sink side), leave `rule_id: null` and the `stages` pending, and fill `dependencies` with the dependency each pattern's package comes from — empty when the boundary is a project member, as a structural pseudo-boundary usually is. Do not carry `context_restrictions`, `sanitizers`, or `negative_patterns` into the units: the boundary is the positive pattern, and the controls stay listed in the spec. + +Both sides are seeded here, and only the source side is authored before the first scan. That is deliberate: the sink boundary is decided now, on the evidence, and authored later against the frontier that scan names — never invented by a model round that found a carrier. + +## Stage gate + +`get_status.py` drives `boundaries`, naming families without a spec, unsaturated specs, evidence with no factorization, and unseeded specs. Finish when it reads `DONE`, or when the next step it reports is source rules. diff --git a/skills-templates/orchestrate-stage/references/crossref.md.j2 b/skills-templates/orchestrate-stage/references/crossref.md.j2 new file mode 100644 index 000000000..8ef28084e --- /dev/null +++ b/skills-templates/orchestrate-stage/references/crossref.md.j2 @@ -0,0 +1,45 @@ +# Cross-reference — scan results against the reference set + +Judge, per supplied finding, whether the latest scan reproduced it, and assemble the coverage manifest. This is the stage that closes an enactment pass: every rule, model, and verdict is already in place, so what the scan shows now is what the pass delivered. Nothing else may set a reference finding's `status`. + +It runs in any mode over a tree that carries a reference set an earlier enactment pass left. That pass's new rules and rescans changed what those findings reproduce — usually for the better, occasionally by losing one — so re-judge them all and refresh the manifest. The work is identical; only the pass that triggered it differs. + +It is also the stage that decides what the run still owes. A judgement here can send the pipeline back — an unmodeled carrier to an approximation round, a rule-caused miss to the stage that authored the rule — and status will report that earlier phase as current again. That is the loop working; re-enter this stage after the rescan rather than closing on stale results. + +## Match by identity, never by rule id + +For each reference finding `get_status.py` lists as pending, compare the scan's results against its recorded identity: the source the trace enters at, the propagation it goes through, the sink it reaches, and the location. A result counts as a reproduction only when its trace carries the same attack path. A rule firing somewhere, a matching vulnerability class, or a matching file is not a match. + +Record the outcome on the reference file: + +{% include "shared/tracking/reference-finding.md" %} + +- reproduced — set `status: reproduced` and put the matching SARIF result hashes in `matched_hashes` +- overmatch — the trace reproduces, but the same rule also fires on flows that are not this finding: still `reproduced`, and note the overmatch in `notes` +- stops at an opaque carrier — `status: unreproduced`, `cause: approximation`, and the exact carrier(s) in `blocked_at`. Get the carrier from `.opentaint/results/dropped-external-methods.yaml` or a localized reachability trace, not from a guess +- no source or sink fact on the expected path — `status: unreproduced`, `cause: rule`. Say in `notes` which side is missing and why, and report it to MAIN: the stage that authored the rule fixes it and the run comes back here +- reproducible only by a modeling limit the engine cannot express — `status: unreproduced`, `cause: engine`, one-line `blocker`, and `blocked_at` cleared + +Set `crossref: done` on every file you judge. A later rescan makes them pending again, which is the loop working: judge them against the new results rather than trusting the old verdict. + +Independently, note scan results that carry no reference finding — those are OpenTaint-exclusive and go to triage like any other finding. Do not count them as reproductions. + +## Close the blocked carriers + +When `get_status.py` reports traces stopping at unmodeled carriers, that work belongs to an approximation round, not here: report the carriers to MAIN, which runs the round and the rescan and re-enters this stage. Only after the rescan proves a carrier still breaks the path does it become an `engine` cause with a `blocker`. + +## Write the coverage manifest + +Once no reference finding is pending or blocked, rewrite `.opentaint/enactment.md` from the current reference files: + +- the finding-level coverage table — one row per supplied finding: id, vulnerability class, family, `status`, and the one-clause reason for an unreproduced one +- the three counts kept apart: raw SARIF results, validated findings, unique vulnerability identities +- reproduced, OpenTaint-exclusive, and reference-exclusive findings as separate sections, exclusives only when triage validated them +- the reusable artifacts the run produced: source rules, sink rules, joins, approximations +- one blocker line per non-reproduced finding, naming the rule, modeling, or engine limitation that remains + +Reflect only current state — rewrite the manifest, don't append to it. + +## Stage gate + +`get_status.py` names pending reference findings, blocked carriers, then the manifest. Finish when `crossref` is `DONE`. Report reproduced/unreproduced totals and every blocker. diff --git a/skills-templates/orchestrate-stage/references/intake.md.j2 b/skills-templates/orchestrate-stage/references/intake.md.j2 new file mode 100644 index 000000000..e78629ab9 --- /dev/null +++ b/skills-templates/orchestrate-stage/references/intake.md.j2 @@ -0,0 +1,75 @@ +# Intake — this mode's input, as the run's families + +Turn what the pass was given into the families the boundaries stage generalizes. The three modes start from different material and converge on the same handoff: a named family per group, each carrying the evidence it was grouped from. Nothing here authors a rule, proposes a boundary, or touches a model. + +Read `mode` from `get_status.py --full` and follow that mode's section. A family is the set of evidence you expect to share one universal source and one universal sink — partition by vulnerability class or by a cohesive attack surface, never by file batches or arbitrary count. + +## Onboarding — the external-method frontier + +Every dependency member the project's own code calls is a trust boundary until a leaf says otherwise. That over-approximation is the point: the sweep classifies the whole frontier once, and every later pass inherits the verdicts. + +### 1. Triage the dependencies + +Dispatch triage-dependencies when status names it. + +Expect back — `.opentaint/tracking/coverage.yaml` written with the flagged packages; status advances to the frontier partition. + +### 2. Partition the frontier + +Run: + +```bash +uv run /scripts/generate.py partition frontier +``` + +It writes balanced `.opentaint/tracking/rules/plans/lib-NNN.yaml` plans over the flagged packages' members the project actually calls, one disjoint slice per leaf. On re-entry it partitions only members no prior pass verdicted. + +### 3. Sweep it + +Fan out discover-attack-surface, one per plan. + +Inputs each: +- `language` +- `plan` + +At the join run: + +```bash +uv run /scripts/generate.py mark-safe +``` + +Expect back — each leaf records the trust boundaries it finds into its plan and writes any source unit(s); the join folds source/safe verdicts into `classification.yaml` and prunes the consumed plans. The sweep is a single fan-out pass, not a loop. + +### 4. Group the frontier into families + +Read the verdicted boundaries back from the ledger and the source units the sweep wrote, and group them into families — usually one per attack surface a set of members shares (a framework's request accessors, a client library's egress calls), not one per dependency. Write them to `.opentaint/tracking/scope.yaml`, evidence being the members themselves. + +{% include "shared/tracking/scope.md" %} + +## Discovery — the project, a diff, or a spec + +`state.yaml` names the scoping document under `spec` when there is one; with none, the scope is the whole project. + +Resolve the input to code before grouping it: read the diff or spec, then the project source it points at — the endpoints, handlers, jobs, and consumers it touches, and the calls they make into dependencies. An informal spec ("this service takes uploads from partners and renders them") is scoped the same way: name the surfaces it implies and read them. + +Then group what you read into families and write `.opentaint/tracking/scope.yaml`, evidence being the members, endpoints, or code areas each family was grouped from — concrete enough that the boundary leaf can go straight to the source. + +Fan this reading out when the scope is large: one leaf per slice, each reporting the surfaces it found. Assign the families yourself once every slice has reported, since that decision needs the whole scope. + +{% include "shared/tracking/scope.md" %} + +## Enactment — the supplied finding set + +`state.yaml` names the supplied findings under `findings` — a manifest, SARIF, report, or directory of finding documents. Write one `.opentaint/tracking/reference/.yaml` per supplied finding. + +{% include "shared/tracking/reference-finding.md" %} + +Give each finding a stable id of its own — the supplied one when it has one. Preserve separately triggerable attack paths as separate findings even when they share a sink. Never drop a finding for being a poor fit for taint analysis: an authorization, integrity, configuration, hard-coded-secret, or structural-control finding gets a reference file like any other, and becomes an explicit pseudo-boundary later. + +Then group the findings into families and set `family` on each. Enactment's families live on the reference files themselves rather than in `scope.yaml`, because the assignment has to move with the finding when a family splits. + +Fan out this normalization when the supplied set is large: one leaf per slice of the supplied report, each writing its own reference files. Assign the families yourself once every file exists, since that decision needs the whole set. + +## Stage gate + +`get_status.py` drives `intake`, naming the mode's next step — the dependency triage, the frontier plans, the scope file, or the findings still without a family. Finish when it reads `DONE`, or when the next step it reports is the boundaries stage. diff --git a/skills-templates/orchestrate-stage/references/sinks.md b/skills-templates/orchestrate-stage/references/sinks.md index 338becd36..7cfeeabfa 100644 --- a/skills-templates/orchestrate-stage/references/sinks.md +++ b/skills-templates/orchestrate-stage/references/sinks.md @@ -1,6 +1,6 @@ # Author sinks + assemble -Author the sink rules flagged into per-package units during approximation classification and join them to every relevant source. The final project rescan follows to surface findings. +Author the sink units this run seeded — the boundaries stage's sink side, plus the sinks flagged into per-package units during model classification — and join them to every relevant source. The final project rescan follows to surface findings. ## Sink lib rules diff --git a/skills-templates/orchestrate-stage/references/sources.md b/skills-templates/orchestrate-stage/references/sources.md index 961bb7003..822c7c698 100644 --- a/skills-templates/orchestrate-stage/references/sources.md +++ b/skills-templates/orchestrate-stage/references/sources.md @@ -1,36 +1,8 @@ -# Discover sources + source rules +# Author sources + assemble -Discover the project's used dependency members that are taint sources, author source rules, and wire them to the built-in sinks. The first project scan follows and names the later taint frontier. On re-entry, partition plans only members no prior run verdicted. +Author the source rules the boundaries stage seeded into per-family units, and wire them to every relevant sink. The first project scan follows: it is what proves the boundaries and names the taint frontier the later stages work from. -## Triage dependencies - -Dispatch triage-dependencies when status names it. - -Expect back — `.opentaint/tracking/coverage.yaml` written; status advances to source discovery. - -## Discover sources - -Run: - -```bash -uv run /scripts/generate.py partition discover -``` - -It writes balanced `.opentaint/tracking/rules/plans/lib-NNN.yaml` plans, one disjoint slice per leaf. Fan out discover-attack-surface, one per plan. - -Inputs each: -- `language` -- `plan` - -At the join run: - -```bash -uv run /scripts/generate.py mark-safe -``` - -Expect back — each agent records the sources it finds into its plan and writes any source unit(s); the join folds source/safe verdicts into `classification.yaml` and prunes the consumed plans. Discovery is a single fan-out pass, not a loop. - -Run `uv run /scripts/get_status.py` to confirm `discover` `DONE`. +On re-entry, work only the units `get_status.py` still lists as pending — a unit that already passes is a prior pass's result, not work to redo. ## Source lib rules @@ -66,4 +38,4 @@ Expect back — existing tag-expanded joins reused where they already cover the ## Stage gate -`get_status.py` drives `discover` then `source_rules`, naming the current sub-step and units. Finish when both are `DONE`, or when the next step it reports is the project scan. +`get_status.py` drives `source_rules`, naming the current sub-step and units. Finish when it reads `DONE`, or when the next step it reports is the project scan. diff --git a/skills-templates/orchestrate-stage/sections/workflow.md b/skills-templates/orchestrate-stage/sections/workflow.md index a2ca715e8..c69c24032 100644 --- a/skills-templates/orchestrate-stage/sections/workflow.md +++ b/skills-templates/orchestrate-stage/sections/workflow.md @@ -1,14 +1,19 @@ Match the stage keyword to its reference and read it fully: ``` +intake → /references/intake.md +boundaries → /references/boundaries.md sources → /references/sources.md approx-round → /references/approx-round.md sinks → /references/sinks.md triage → /references/triage.md poc → /references/poc.md +crossref → /references/crossref.md (a tree carrying a reference set) escalation → /references/escalation.md ``` +`intake` is the only stage whose work depends on the run's `mode`; its reference carries a section per mode. Everything after it reads the same whichever mode produced the families. + Run the bundled script to get the setup overview before proceeding to the reference's instructions: ```bash diff --git a/skills-templates/shared/scripts/_common.py b/skills-templates/shared/scripts/_common.py index 092f216dd..9c2ad0b8e 100644 --- a/skills-templates/shared/scripts/_common.py +++ b/skills-templates/shared/scripts/_common.py @@ -1,4 +1,4 @@ -"""Shared helpers for the appsec-agent orchestrator scripts. +"""Shared helpers for the OpenTaint pipeline orchestrator scripts. Not a runnable script — imported by the PEP723 entry points (get_status.py, generate.py), which carry the pyyaml dependency. Every path resolves under the fixed @@ -6,6 +6,7 @@ scripts from the project root. """ import glob +import re import subprocess from pathlib import Path @@ -20,6 +21,9 @@ SINKS_TR = RULES_TR / "sinks" JOINS_TR = RULES_TR / "joins" FINDINGS_TR = TRACKING / "findings" +SCOPE = TRACKING / "scope.yaml" # every mode: what intake scoped, as the family list +REFERENCE_TR = TRACKING / "reference" # enactment mode: the supplied findings, normalized +BOUNDARIES_TR = TRACKING / "boundaries" # every mode: per-family universal boundary specs RESULTS = ROOT / "results" DROPPED = RESULTS / "dropped-external-methods.yaml" SARIF = RESULTS / "report.sarif" @@ -96,6 +100,22 @@ def git_head(): return None +# ---- intake scope ---- + +def scope_families(): + """(name, evidence) per family the intake stage scoped, in scope.yaml order. + + One shape for every mode: the evidence items are reference finding ids in enactment mode, + and the members or code areas intake settled on in onboarding and discovery mode. The + boundaries stage generalizes one family per entry, whichever mode wrote it.""" + out = [] + for f in (load_yaml(SCOPE, {}) or {}).get("families") or []: + name = strip_quotes((f or {}).get("name", "")) if isinstance(f, dict) else "" + if name: + out.append((name, [strip_quotes(str(e)) for e in (f.get("evidence") or [])])) + return out + + # ---- approximation batch readers (shared by coverage + partition) ---- # a method counts classified once it sits in any batch classification bucket or in build.done @@ -176,6 +196,12 @@ def build_done_keys(): return keys +# ---- finding files ---- + +RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) +VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) + + def ledger_verdicted_keys(): """method+signature keys already verdicted in classification.yaml (source ∪ safe).""" doc = load_yaml(RULES_TR / "classification.yaml", {}) or {} diff --git a/skills-templates/shared/scripts/generate.py b/skills-templates/shared/scripts/generate.py index 8f98987bc..51e1d1339 100644 --- a/skills-templates/shared/scripts/generate.py +++ b/skills-templates/shared/scripts/generate.py @@ -9,8 +9,8 @@ init bootstrap the .opentaint tree + state.yaml from the workflow flags partition analyze dropped external methods -> per-root batch plans (approximations) - partition discover coverage.yaml's used members -> balanced discover plans - mark-safe discover plans' verdicts -> classification.yaml ledger (+prune plans) + partition frontier coverage.yaml's used members -> balanced frontier plans + mark-safe frontier plans' verdicts -> classification.yaml ledger (+prune plans) merge-skipped batch skipped/engine_issues -> approximations/skipped.yaml (+prune plans) findings results/report.sarif -> per-rule finding tracking files (idempotent) """ @@ -26,19 +26,18 @@ import yaml -from _common import (APPROX, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, class_of, classified_keys, - dropped_entries, dump_yaml, fqn_base, git_head, - ledger_verdicted_keys, load_yaml, member_key, package_of, - strip_quotes) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, + SOURCES_TR, RULE_RE, TRACKING, VERDICT_RE, class_of, classified_keys, + batch_files, dropped_entries, dump_yaml, fqn_base, git_head, + ledger_verdicted_keys, load_yaml, member_key, package_of, strip_quotes) ANALYZE_BUDGET = 20 # methods per approximation batch ANALYZE_MISC = 6 # roots with <= this many methods pool into one misc batch ROOT_DEPTH = 2 # library root = first 2 dotted segments -DISCOVER_TARGET, DISCOVER_BAND = 50, 15 # project-used members per discover plan (~50, loose) +FRONTIER_TARGET, FRONTIER_BAND = 50, 15 # project-used members per frontier plan (~50, loose) -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" @@ -47,33 +46,98 @@ # the durable directories a run writes into; the leaves/scripts mkdir on write, but seeding # them up front gives every stage a place to land and makes the empty tree self-describing. INIT_DIRS = [TRACKING, APPROX, SOURCES_TR, SINKS_TR, JOINS_TR, FINDINGS_TR, - RESULTS, RULES, PASS_THROUGH, DATAFLOW] + BOUNDARIES_TR, RESULTS, RULES, PASS_THROUGH, DATAFLOW] +ENACTMENT_DIRS = [REFERENCE_TR] # the normalized supplied findings STATE_DERIVED = ("model_commit", "build_jdk", "max_memory") # build/scan fill these, init preserves +def carried_over(): + """What a pass inherits from the passes before it, as (label, count) — everything durable + the tree already holds. Every mode writes into one tree, so a pass never starts empty + unless the tree is.""" + def n(paths): + return sum(1 for _ in paths) + return [("rule unit", n(SOURCES_TR.glob("*.yaml")) + n(SINKS_TR.glob("*.yaml"))), + ("created rule", n(p for p in RULES.rglob("*.yaml") if p.is_file())), + ("boundary spec", n(BOUNDARIES_TR.glob("*.yaml")) if BOUNDARIES_TR.is_dir() else 0), + ("approximation batch", len(batch_files())), + ("reference finding", n(REFERENCE_TR.glob("*.yaml")) if REFERENCE_TR.is_dir() else 0), + ("triaged finding", n(FINDINGS_TR.glob("*.yaml")))] + + +def onboarded(runs): + """Onboarding sweeps the whole external-method frontier and its corpus is durable, so it is + a one-time pass. Re-running init in onboarding mode is a resume while the tree is still in + that pass; once a later mode took over, the sweep is done and asking for it again is a + mistake worth naming.""" + return any(str((r or {}).get("type", "")).startswith("onboarding/") for r in runs) + + def cmd_init(args): - for d in INIT_DIRS: - d.mkdir(parents=True, exist_ok=True) state_path = TRACKING / "state.yaml" prior = load_yaml(state_path, {}) or {} - resume = bool(prior) - state = {"scan_level": args.scan_level, "triage_level": args.triage_level, + enactment = args.mode == "enactment" + # the finding set is a property of the tree, not of one pass — an enactment pass that follows + # an earlier one inherits it, so --findings is required only the first time + findings = args.findings or prior.get("findings") + if enactment and not findings: + raise SystemExit("init --mode enactment requires --findings ") + if args.mode == "discovery" and not args.scan_level: + raise SystemExit("init --mode discovery requires --scan-level") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if args.mode == "onboarding" and onboarded(runs) and prior.get("mode") != "onboarding": + raise SystemExit("this tree is already onboarded — its frontier sweep, universal rules, " + "and models are on disk and every later pass builds on them; run " + "discovery or enactment instead") + # onboarding sweeps the frontier and enactment reproduces a supplied finding set: both always + # need the full rule + approximation toolbox, so their level is fixed rather than asked for. + scan_level = args.scan_level if args.mode == "discovery" else "deep" + + for d in INIT_DIRS + (ENACTMENT_DIRS if enactment else []): + d.mkdir(parents=True, exist_ok=True) + # `mode` is this pass's intake, not a permanent property of the tree: the modes compose, in + # any order and repeatedly across commits, and every artifact below is shared between them. + state = {"mode": args.mode, "scan_level": scan_level, "triage_level": args.triage_level, "language": args.language or prior.get("language")} + if findings: # kept across other modes, so a later enactment pass resumes it + state["findings"] = findings + spec = args.spec or prior.get("spec") + if spec: + state["spec"] = spec for k in STATE_DERIVED: # never clobber what build/scan already learned state[k] = prior.get(k) state_path.write_text(dump_yaml(state), encoding="utf-8") - # history: append one run entry on a fresh init, never on resume (the derived knobs survived) + # history: one entry per pass. Re-running init with the same knobs on the same commit is a + # resume of the current pass, not a new one — a different mode, level, or commit starts one. hist_path = TRACKING / "history.yaml" - runs = (load_yaml(hist_path, {}) or {}).get("runs") or [] - if not resume: - runs.append({"commit": git_head(), "type": f"{args.scan_level}/{args.triage_level}"}) + entry = {"commit": git_head(), + "type": f"{args.mode}/{scan_level}/{args.triage_level}"} + new_pass = not runs or runs[-1] != entry + if new_pass: + runs.append(entry) hist_path.write_text(dump_yaml({"runs": runs}), encoding="utf-8") - mode = "resumed (derived knobs preserved)" if resume else "fresh" - print(f"init {mode}: scan_level={state['scan_level']} triage_level={state['triage_level']} " - f"language={state['language']}") - print(f"seeded {len(INIT_DIRS)} directories under .opentaint/") + prior_mode = prior.get("mode") + if not prior: + how = "fresh tree" + elif prior_mode and prior_mode != args.mode: + how = f"new {args.mode} pass over the existing {prior_mode} tree" + elif new_pass: + how = f"new {args.mode} pass (pass {len(runs)})" + else: + how = "resumed (derived knobs preserved)" + print(f"init {how}: mode={args.mode} scan_level={scan_level} " + f"triage_level={state['triage_level']} language={state['language']}") + if findings: + print(f"findings={findings}") + if spec: + print(f"spec={spec}") + if prior: + kept = ", ".join(f"{c} {label}{'' if c == 1 else 's'}" for label, c in carried_over() if c) + print(f"carried over: {kept or 'nothing yet'}") + print(f"seeded {len(INIT_DIRS) + (len(ENACTMENT_DIRS) if enactment else 0)} " + "directories under .opentaint/") print("next: uv run scripts/get_status.py --full") return 0 @@ -163,7 +227,7 @@ def write_plans(plans, out_dir, prefix_id): if isinstance(x, dict) else x) for p, v in sorted(scopes.items())} path = out_dir / f"{pid}.yaml" - # source: null is the unprocessed sentinel — a discover agent overwrites it with the + # source: null is the unprocessed sentinel — a frontier agent overwrites it with the # list of sources it found (an empty list when it finds none). mark-safe folds only # plans whose sentinel was replaced, so an un-returned plan is never marked safe. path.write_text(dump_yaml({"id": pid, "scopes": norm, "source": None}), encoding="utf-8") @@ -227,7 +291,7 @@ def cmd_analyze(args): return 0 -# ---- partition discover ---- +# ---- partition frontier ---- def yaml_modules(model_yaml): # each module in project.yaml as (packages, moduleClasses); only `packages` says which of a @@ -299,8 +363,8 @@ def pending_packages(): return tuple(p for p in (cov.get("packages") or []) if isinstance(p, str) and p) -def cmd_discover(args): - regen_plans(DISCOVER_PLANS) +def cmd_frontier(args): + regen_plans(FRONTIER_PLANS) packages = pending_packages() if not packages: print("nothing to plan — no pending package in coverage.yaml", file=sys.stderr) @@ -314,26 +378,26 @@ def cmd_discover(args): if not todo: print("nothing to plan — every used member already verdicted", file=sys.stderr) return 0 - cap = DISCOVER_TARGET + DISCOVER_BAND - plans = pack(atomize(todo, cap), DISCOVER_TARGET, cap) + cap = FRONTIER_TARGET + FRONTIER_BAND + plans = pack(atomize(todo, cap), FRONTIER_TARGET, cap) rows = {f: [{"method": f, "signature": s} for s in sorted(sigs[f])] for f in sigs} plans = [{pkg: [r for f in members for r in rows[f]] for pkg, members in plan.items()} for plan in plans] - for p in write_plans(plans, DISCOVER_PLANS, "lib"): + for p in write_plans(plans, FRONTIER_PLANS, "lib"): print(p) return 0 def cmd_partition(args): - return cmd_analyze(args) if args.kind == "analyze" else cmd_discover(args) + return cmd_analyze(args) if args.kind == "analyze" else cmd_frontier(args) -# ---- mark-safe (discover join) ---- +# ---- mark-safe (frontier join) ---- def cmd_mark_safe(args): - plans = sorted(glob.glob(str(DISCOVER_PLANS / "lib-*.yaml"))) + plans = sorted(glob.glob(str(FRONTIER_PLANS / "lib-*.yaml"))) if not plans: - print("no discover plans to reconcile", file=sys.stderr) + print("no frontier plans to reconcile", file=sys.stderr) return 0 ledger = RULES_TR / "classification.yaml" doc = load_yaml(ledger, {}) or {} @@ -343,7 +407,7 @@ def cmd_mark_safe(args): for p in plans: pdoc = load_yaml(p, {}) or {} raw = pdoc.get("source") - if raw is None: # sentinel intact — no discover agent returned for it + if raw is None: # sentinel intact — no frontier agent returned for it unprocessed.append(p) continue members = {member_key(m) for v in (pdoc.get("scopes") or {}).values() for m in v} @@ -353,7 +417,7 @@ def cmd_mark_safe(args): processed.append(p) print(f"{Path(p).name}: {len(srcs)} sources, {len(members - srcs)} safe") if not processed: - print("no processed discover plans (every plan still carries source: null) — " + print("no processed frontier plans (every plan still carries source: null) — " "fan out discover-attack-surface first", file=sys.stderr) return 0 safe -= source @@ -363,7 +427,7 @@ def cmd_mark_safe(args): if not args.keep: for p in processed: Path(p).unlink() - print(f"pruned {len(processed)} reconciled discover plan(s)") + print(f"pruned {len(processed)} reconciled frontier plan(s)") if unprocessed: print(f"left {len(unprocessed)} unprocessed plan(s) (source: null) for re-dispatch: " + ", ".join(Path(p).name for p in unprocessed)) @@ -420,10 +484,8 @@ def cmd_merge_skipped(args): _FP_PREFERENCE = ("vulnerabilitySourceSinkHash", "vulnerabilityWithTraceHash") -RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) HASHES_RE = re.compile(r'^sarif_hashes:\s*\[(.*)\]\s*$', re.M) HASHES_BLOCK_RE = re.compile(r'^sarif_hashes:\s*\n((?:[ \t]+-[^\n]*\n?)+)', re.M) -VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) def docker_name(seed, taken): @@ -561,17 +623,27 @@ def main(): sub = ap.add_subparsers(dest="cmd", required=True) i = sub.add_parser("init", help="bootstrap the .opentaint tree + state.yaml from workflow flags") - i.add_argument("--scan-level", required=True, choices=["lite", "normal", "deep"]) + i.add_argument("--mode", required=True, choices=["onboarding", "discovery", "enactment"], + help="onboarding: sweep the external-method frontier once. discovery: work " + "from the project, a diff, or an informal spec. enactment: reproduce a " + "supplied finding set") + i.add_argument("--scan-level", choices=["lite", "normal", "deep"], + help="discovery mode only; onboarding and enactment are always deep") i.add_argument("--triage-level", required=True, choices=["static", "dynamic"]) i.add_argument("--language", default=None, help="target language, determined by the orchestrator") + i.add_argument("--findings", default=None, + help="enactment mode: path to the supplied finding manifest/report/directory") + i.add_argument("--spec", default=None, + help="discovery mode: path to the diff, spec, or note that scopes the pass; " + "omit to scope the pass to the whole project") i.set_defaults(func=cmd_init) p = sub.add_parser("partition", help="split classification work into per-agent plans") - p.add_argument("kind", choices=["analyze", "discover"]) + p.add_argument("kind", choices=["analyze", "frontier"]) p.set_defaults(func=cmd_partition) - m = sub.add_parser("mark-safe", help="merge discover plans into classification.yaml") - m.add_argument("--keep", action="store_true", help="keep the reconciled discover plans") + m = sub.add_parser("mark-safe", help="merge frontier plans into classification.yaml") + m.add_argument("--keep", action="store_true", help="keep the reconciled frontier plans") m.set_defaults(func=cmd_mark_safe) s = sub.add_parser("merge-skipped", help="rebuild approximations/skipped.yaml from batches") @@ -581,6 +653,7 @@ def main(): f = sub.add_parser("findings", help="seed per-rule finding files from results/report.sarif") f.set_defaults(func=cmd_findings) + args = ap.parse_args() return args.func(args) diff --git a/skills-templates/shared/scripts/get_status.py b/skills-templates/shared/scripts/get_status.py index aca3b06b6..e8411bb26 100644 --- a/skills-templates/shared/scripts/get_status.py +++ b/skills-templates/shared/scripts/get_status.py @@ -22,19 +22,21 @@ import sys from pathlib import Path -from _common import (APPROX, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, ROOT, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, build_done_keys, classified_keys, - dropped_entries, git_head, load_yaml, member_key, - modeled_entries, skipped_keys) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, ROOT, RULES, RULES_TR, SARIF, SCOPE, SINKS_TR, + SOURCES_TR, TRACKING, build_done_keys, classified_keys, dropped_entries, + git_head, load_yaml, member_key, modeled_entries, scope_families, + skipped_keys, strip_quotes) STATE = load_yaml(TRACKING / "state.yaml", {}) or {} +MODE = STATE.get("mode") or "discovery" SCAN_LEVEL = STATE.get("scan_level") TRIAGE_LEVEL = STATE.get("triage_level") -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" VULN = ROOT / "vulnerabilities.md" +ENACTMENT = ROOT / "enactment.md" GLOBAL_CAP = 10 @@ -49,6 +51,12 @@ def load_units(d): if Path(d).is_dir() else [] +def load_docs(d): + """(path, doc) for every tracking file in a directory — reference, boundary, control.""" + return [(p, load_yaml(p, {}) or {}) for p in sorted(Path(d).glob("*.yaml"))] \ + if Path(d).is_dir() else [] + + def load_joins(): return [(p.stem, load_yaml(p, {}) or {}) for p in sorted(JOINS_TR.glob("*.yaml"))] \ if JOINS_TR.is_dir() else [] @@ -157,10 +165,13 @@ def ph_build(): "dispatch build-project"], None -def ph_discover(): +def ph_frontier(): + """Onboarding's intake: the external-method frontier, swept as one trust boundary. Every + dependency member the project's own code calls is a candidate boundary until a leaf verdicts + it, and the ledger is what makes that verdict durable for every later pass.""" if not (TRACKING / "coverage.yaml").is_file(): return False, ["dispatch triage-dependencies"], None - leftover = sorted(glob.glob(str(DISCOVER_PLANS / "*.yaml"))) + leftover = sorted(glob.glob(str(FRONTIER_PLANS / "*.yaml"))) units = load_units(SOURCES_TR) ledger = load_yaml(RULES_TR / "classification.yaml", {}) or {} if leftover: @@ -169,7 +180,7 @@ def ph_discover(): tasks.append("then run `scripts/generate.py mark-safe` to reconcile the plans") return False, tasks, None if not ledger and not units: - return False, ["run `scripts/generate.py partition discover` to plan the used members"], None + return False, ["run `scripts/generate.py partition frontier` to plan the called members"], None return True, [], None @@ -280,18 +291,141 @@ def ph_poc(): return True, [], None +# ---- intake: one contract, three mode-specific derivations ---- + +def ph_reference_set(): + """Enactment's intake: the supplied findings, normalized and grouped. The `family` field on + each reference file is the assignment — it moves with the finding when a family splits, so + the families are read back from the files rather than from a separate list.""" + docs = load_docs(REFERENCE_TR) + if not docs: + src = STATE.get("findings") or "state.yaml findings unset" + return False, [f"normalize the supplied findings ({src}) into " + ".opentaint/tracking/reference/.yaml"], None + missing = sorted(p.stem for p, d in docs if not strip_quotes(d.get("family", ""))) + if missing: + return False, ["reference findings not assigned to a boundary family:"] \ + + [f" {m}" for m in missing], None + return True, [], None + + +def ph_scope(): + """Onboarding's and discovery's intake join: the families their evidence groups into, written + to scope.yaml. Enactment carries the same information on its reference files instead.""" + fams = scope_families() + if not fams: + src = STATE.get("spec") or "the whole project" + what = ("the frontier the sweep verdicted as sources and effects" if MODE == "onboarding" + else f"the code {src} names") + return False, [f"group {what} into families and write {SCOPE}"], None + empty = [n for n, ev in fams if not ev] + if empty: + return False, ["families in scope.yaml with no evidence recorded:"] \ + + [f" {n}" for n in empty], None + return True, [], None + + +def ph_intake(): + if MODE == "enactment": + return ph_reference_set() + if MODE == "onboarding": + done, tasks, note = ph_frontier() + if not done: + return done, tasks, note + return ph_scope() + + +def families(): + """(family, evidence ids) for this pass, whichever mode scoped it — reference finding ids in + enactment mode, the members or areas intake recorded otherwise. The boundaries stage + generalizes one universal source and sink per entry.""" + if MODE == "enactment": + out = {} + for path, doc in load_docs(REFERENCE_TR): + fam = strip_quotes(doc.get("family", "")) + if fam: + out.setdefault(fam, []).append(path.stem) + return sorted(out.items()) + return scope_families() + + +def ph_boundaries(): + specs = {p.stem: d for p, d in load_docs(BOUNDARIES_TR)} + fams = families() + missing = [f for f, _ in fams if f not in specs] + if missing: + return False, ["dispatch discover-universal-boundaries, one per family:"] \ + + [f" {f}" for f in missing], None + # a split rewrites the family on the evidence it moves, so every spec here owns its evidence + unsaturated = [f for f, _ in fams + if str((specs[f].get("saturation") or {}).get("status", "")).strip() + != "saturated"] + if unsaturated: + return False, ["boundary specs not saturated:"] + [f" {f}" for f in unsaturated], None + unfactored = sorted(f"{f}: {e}" for f, evidence in fams for e in evidence + if e not in (specs[f].get("factorization") or {})) + if unfactored: + return False, ["evidence with no factorization in its spec:"] \ + + [f" {r}" for r in unfactored], None + unseeded = [f for f, _ in fams if (specs[f].get("stages") or {}).get("units_seeded") != "done"] + if unseeded: + return False, ["seed the source and sink units from these specs' candidate_patterns:"] \ + + [f" {f}" for f in unseeded], None + return True, [], None + + +def ph_crossref(): + if not SARIF.is_file(): + return False, ["dispatch run-scan"], None + docs = load_docs(REFERENCE_TR) + scanned = SARIF.stat().st_mtime + pend = [p for p, d in docs + if str(d.get("crossref", "pending")).strip() != "done" or p.stat().st_mtime < scanned] + if pend: + return False, [f"cross-reference the scan against {len(pend)} reference finding(s):"] \ + + [f" {p}" for p in pend], None + blocked = sorted({str(m) for _, d in docs for m in (d.get("blocked_at") or [])}) + if blocked: + return False, ["expected traces stop at unmodeled carriers:"] + [f" {m}" for m in blocked] \ + + ["model them in an approximation round, rescan, then cross-reference again"], None + rep = sum(1 for _, d in docs if str(d.get("status", "")).strip() == "reproduced") + stale = newest_mtime([p for p, _ in docs]) > (ENACTMENT.stat().st_mtime + if ENACTMENT.is_file() else 0) + if not ENACTMENT.is_file() or stale: + return False, [f"rewrite .opentaint/enactment.md coverage manifest " + f"({rep}/{len(docs)} reproduced)"], None + return True, [], None + + +def has_reference_set(): + """A reference set outlives the pass that created it. Any later pass that rescans changes what + it reproduces, so the cross-reference stays in scope — otherwise an onboarding or discovery + pass would leave a coverage manifest that silently describes an older scan.""" + return REFERENCE_TR.is_dir() and any(REFERENCE_TR.glob("*.yaml")) + + +# One pipeline, whatever the mode brought to it. Intake and its universal boundaries differ — +# the swept frontier, the diff or spec, the supplied findings — but from the boundary specs on, +# every mode runs the same stages in the same order: sources before the scan that proves them, +# approximations against the frontier that scan names, sinks after it, then triage. Only the +# closing phases are conditional, on the triage level, the controls knob, and whether the tree +# carries a reference set to re-judge. PHASES = [ ("build", ph_build, lambda: True), - ("discover", ph_discover, lambda: SCAN_LEVEL == "deep"), + ("intake", ph_intake, lambda: SCAN_LEVEL == "deep"), + ("boundaries", ph_boundaries, lambda: SCAN_LEVEL == "deep"), ("source_rules", ph_source_rules, lambda: SCAN_LEVEL == "deep"), ("scan", ph_scan, lambda: True), ("approximations", ph_approximations, lambda: SCAN_LEVEL in ("normal", "deep")), ("sink_rules", ph_sink_rules, lambda: SCAN_LEVEL == "deep"), ("triage", ph_triage, lambda: True), ("poc", ph_poc, lambda: TRIAGE_LEVEL == "dynamic"), + ("crossref", ph_crossref, has_reference_set), ] + + # ---- caps ---- def free_gb(): @@ -329,8 +463,16 @@ def evaluate(): def cmd_full(): commit = short(STATE.get("model_commit")) or "none" - print(f"scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} language={STATE.get('language')} " - f"commit={commit} cap={GLOBAL_CAP} (heavy {heavy_cap()})") + print(f"mode={MODE} scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} " + f"language={STATE.get('language')} commit={commit} " + f"cap={GLOBAL_CAP} (heavy {heavy_cap()})") + if STATE.get("findings"): # printed in every mode — the set stays tracked + print(f"findings={STATE.get('findings')}") + if STATE.get("spec"): + print(f"spec={STATE.get('spec')}") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if len(runs) > 1: + print("passes: " + " -> ".join(str(r.get("type", "?")) for r in runs)) rows, current = evaluate() # a phase downstream of the current stage that vacuously satisfies its own check is not # actually done — its producing stage hasn't run — so it reads PENDING, never DONE. diff --git a/skills-templates/shared/tracking/boundary-spec.md b/skills-templates/shared/tracking/boundary-spec.md new file mode 100644 index 000000000..003ba8012 --- /dev/null +++ b/skills-templates/shared/tracking/boundary-spec.md @@ -0,0 +1,28 @@ +`.opentaint/tracking/boundaries/.yaml` — one boundary family: the single universal source and single universal sink every piece of the family's evidence factors through, plus the controls that recover precision. `evidence` lists what the family was grouped from — reference finding ids in enactment mode, the members or code areas `scope.yaml` recorded otherwise. `candidate_patterns` are concrete enough for `create-rule` to test, and seed the family's source and sink units verbatim. `factorization` carries one entry per evidence item, `status` one of `covered` (factors through both boundaries as-is), `needs-restriction` (factors only under a named `context_restrictions` entry), or `unfactored` (does not factor — explain in `open_questions` and split or add a pseudo-boundary). `saturation` records the widen-and-recheck rounds and only reads `saturated` once a full round changed neither the boundaries nor any factorization. `approximation_candidates` are opaque carriers noted for later — never acted on before a scan proves the trace stops there. Keep it clear from comments + +```yaml +family: ssrf +evidence: [DSC-014, DSC-021] +source: + semantic_boundary: external request value entering a controller + candidate_patterns: + - { method: org.springframework.web.bind.annotation.RequestBody, signature: null, note: annotated controller parameter } + context_restrictions: [] +sink: + semantic_boundary: outbound HTTP request with a caller-supplied URL + candidate_patterns: + - { method: org.springframework.web.client.RestTemplate#getForObject, signature: "(Ljava/lang/String;Ljava/lang/Class;[Ljava/lang/Object;)Ljava/lang/Object;", vuln_class: ssrf, note: egress with attacker URL } + context_restrictions: [] +sanitizers: [] +negative_patterns: [] +factorization: + DSC-014: { source: request body callbackUrl, sink: RestTemplate#getForObject, status: covered } + DSC-021: { source: request param targetUrl, sink: RestTemplate#getForObject, status: needs-restriction, restriction: admin-only controller } +approximation_candidates: [] +open_questions: [] +saturation: + rounds: 3 + status: saturated +stages: + units_seeded: done +``` diff --git a/skills-templates/shared/tracking/reference-finding.md b/skills-templates/shared/tracking/reference-finding.md new file mode 100644 index 000000000..330f224e1 --- /dev/null +++ b/skills-templates/shared/tracking/reference-finding.md @@ -0,0 +1,21 @@ +`.opentaint/tracking/reference/.yaml` — one supplied finding, normalized to a stable identity and carried through boundary discovery and the cross-reference. The file is named for the finding's own id. `family` ties it to its boundary spec and is rewritten when a family splits. `status` is `pending` until the cross-reference judges it, then `reproduced` or `unreproduced`; `cause` explains an `unreproduced` one so the pipeline knows who owns it — `rule` (a boundary, restriction, or sanitizer is wrong), `approximation` (an opaque carrier breaks the path), or `engine` (a modeling limit, paired with `blocker`). `blocked_at` lists the carriers still to model and is cleared once they are modeled or judged terminal. `matched_hashes` are the SARIF result hashes whose trace carries this finding's identity — never a rule-id match alone. Keep it clear from comments + +```yaml +id: DSC-014 +vuln_class: ssrf +family: ssrf +source: request body field `callbackUrl` on POST /api/webhook/register +propagation: WebhookReqVO -> WebhookDO -> WebhookService#dispatch +sink: RestTemplate#getForObject in WebhookService#dispatch +expected_location: yudao-module-infra/.../WebhookService.java:88 +guards: URL parsed with new URI(...), no private-range rejection +status: reproduced +cause: null +blocker: null +blocked_at: [] +matched_hashes: [a1b2c3d4e5f6a7b8] +crossref: done +notes: > + crossref: join ssrf-webhook-ext fired at WebhookService#dispatch:88 with the trace entering at + the registration body — same attack path as the reference finding +``` diff --git a/skills-templates/shared/tracking/scope.md b/skills-templates/shared/tracking/scope.md new file mode 100644 index 000000000..4aff24f08 --- /dev/null +++ b/skills-templates/shared/tracking/scope.md @@ -0,0 +1,14 @@ +`.opentaint/tracking/scope.yaml` — what this pass's intake scoped, and the families the boundaries stage generalizes one at a time. `scope` is one line on what the pass was given, so a later reader knows what the family list came from. Each family carries the `evidence` it was grouped from: the frontier members a sweep verdicted, or the members, endpoints, and code areas a diff or spec resolved to. Enactment mode writes no scope file — its families live on the reference findings, which carry the same assignment on the file that moves with them. Keep it clear from comments + +```yaml +mode: discovery +scope: docs/2026-07-webhooks.md — partner webhook registration and delivery +families: + - name: ssrf-webhook + evidence: + - com.acme.webhook.WebhookController#register + - com.acme.webhook.WebhookService#dispatch + - name: upload-path + evidence: + - com.acme.upload.UploadController#store +``` diff --git a/skills-templates/triage-dependencies/sections/workflow.md b/skills-templates/triage-dependencies/sections/workflow.md index 89bfc69d2..d5b05cdf9 100644 --- a/skills-templates/triage-dependencies/sections/workflow.md +++ b/skills-templates/triage-dependencies/sections/workflow.md @@ -1,6 +1,6 @@ ### 1. List the dependencies -Read `.opentaint/project/project.yaml` — the `dependencies` list under each per-language projects entry is every third-party dependency the model resolved. Resolve each to the library it is. Most of a large project's dependencies are transitive infrastructure +Read `.opentaint/project/project.yaml` — the `dependencies:` list under each per-language projects entry (e.g. `javaProjects:`) is every third-party dependency the model resolved. Resolve each to the library it is. Most of a large project's dependencies are transitive infrastructure ### 2. Mark each library diff --git a/skills/appsec-agent/SKILL.md b/skills/appsec-agent/SKILL.md index 72143a40b..bb9fc4731 100644 --- a/skills/appsec-agent/SKILL.md +++ b/skills/appsec-agent/SKILL.md @@ -1,6 +1,6 @@ --- name: appsec-agent -description: Run an end-to-end OpenTaint application-security analysis while owning the long project build and scans and delegating each other pipeline stage. Use when the user asks to find vulnerabilities, or scan an application for security issues +description: Run an end-to-end OpenTaint application-security analysis in one of three modes — onboarding a project's dependency frontier, discovering vulnerabilities from the project, a diff, or a spec, or enacting a supplied finding set — owning the long project build and scans and delegating each other pipeline stage. Use when the user asks to find vulnerabilities, scan an application for security issues, reproduce or validate a supplied finding set, or continue an OpenTaint run license: Apache-2.0 metadata: author: opentaint @@ -9,9 +9,66 @@ metadata: # AppSec Agent -Orchestrate an end-to-end OpenTaint security analysis. Keep the long project build and every full-project scan in this main session; delegate each bounded source, approximation, sink, triage, and PoC stage to an `orchestrate-stage` subagent, which owns its leaf fan-out and joins. +Run an end-to-end OpenTaint security analysis. Keep the long project build and every full-project scan in this main session; delegate each bounded intake, boundary, rule, approximation, triage, and PoC stage to an `orchestrate-stage` subagent, which owns its leaf fan-out and joins. -OpenTaint is a whole-program, interprocedural, field-sensitive alias analysis SAST. The run produces confirmed vulnerabilities plus reusable project-specific rules and approximations under one self-contained `.opentaint/` directory at the project root. +OpenTaint is a whole-program, interprocedural, field-sensitive alias analysis SAST. A run produces confirmed vulnerabilities plus the project's own universal rules and the models behind them — the passThrough and dataflow approximations that carry taint through library code, which the pipeline's `approximations` phase builds. Everything lands under one self-contained `.opentaint/` directory at the project root. + +This is the only entry point. It runs in one of three modes, which differ in what the run takes as its input — and therefore in where the universal rules come from — and in nothing else: + +- **onboarding** — the external-method frontier: every dependency member the project's own code calls, taken as a trust boundary until a leaf verdicts it. Run once per project, to build the universal rule and model corpus the later passes inherit +- **discovery** — the project, a diff, or an informal spec of what changed or what matters; the code it names becomes the boundary evidence +- **enactment** — a finding set the user supplies: a report, pentest results, or source-to-sink traces, reproduced as verified rules + +Whatever the input, intake groups it into families, the boundaries stage generalizes each family into one universal source and one universal sink, and everything downstream is the same pipeline in the same order. + +## Modes + +### What each mode takes in + +- **onboarding** — no input beyond the project itself. The frontier sweep flags the dependencies that can carry untrusted data, partitions the members the project actually calls, and verdicts each one: a trust boundary, an effect, or neither. It is the widest intake there is, and its output — a classified frontier, universal rules for the stack, and the models behind them — is what makes every later pass cheap +- **discovery** — a diff, a spec, a ticket, or a sentence about what the project does and what would be bad. Ask for whatever the user has and record its path; with nothing supplied the scope is the whole project. Intake reads it, resolves it to code, and groups that code into families +- **enactment** — the supplied findings, as a manifest, SARIF, scanner report, or a directory of finding documents. Ask for the path when it isn't given. If the user only described the findings in conversation, write them to a file first and use that; the pipeline resumes from disk, not from this thread + +### Onboarding runs once + +The frontier sweep is the expensive pass, and its corpus is durable: the classification ledger, the universal rules, and the models stay on disk and apply to every scan afterwards, whichever mode ran it. So onboard a project once and then work in discovery or enactment. Bootstrap refuses a second onboarding pass over a tree that already had one and says so; a genuinely new dependency stack is a new tree. + +### The modes compose + +They are not alternatives, and picking one is not a commitment. One `.opentaint/` tree accumulates the artifacts of every pass over it, in any order and as many times as the project needs: + +- **onboarding, then anything** — the natural start. A discovery or enactment pass over an onboarded tree finds most of its boundaries already ruled and modeled, and spends its work on what its own input names +- **enactment, then discovery** — reproduce the supplied report first, then hunt with the rules it produced. Boundaries derived from real findings are exactly the ones a discovery pass would otherwise have to argue for +- **discovery, then enactment** — assess the project, then measure a report against the corpus that pass built. What the report names but the scan missed is now a rule or modeling gap you can point at +- **any of them, again on a later commit** — the tree is long-lived. A new HEAD makes the model stale, so the pass rebuilds and rescans, and every rule, model, and verdict carries over. That's how a run becomes a regression check rather than a one-off + +So `mode` in `state.yaml` is the intake of the *current pass*, not a property of the tree. Switching it is normal, needs no fresh tree, and strands nothing. + +### Read what the tree already holds + +If `.opentaint/tracking/state.yaml` exists, find out where the project stands before choosing: + +```bash +uv run /scripts/get_status.py --full +``` + +Its header prints the current `mode`, the run's levels, the tracked finding set or spec if there is one, and — once the tree has more than one pass — the `passes:` chain. The phase lines say whether that pass is finished or mid-flight. + +- mid-flight pass — resume it rather than starting a different one over the top of it +- finished pass, and the user wants more — that's a new pass, and the choice below applies again +- tell the user what's there either way, in one line: which pass, where it stands, what carried over + +### Choose this pass + +Decide from what the user brought, then confirm it with them before bootstrapping: + +- **enactment** — they supplied findings and want them reproduced, validated, converted into reusable rules, or cross-checked against OpenTaint. "Does OpenTaint catch these?", "reproduce this report", "turn these findings into rules" +- **discovery** — no finding set to measure against; the goal is what the project is vulnerable to, in the whole project or in what a diff or spec names. "Find vulnerabilities", "scan this app", "did this PR introduce anything?" +- **onboarding** — the tree has never been onboarded and the user wants the project's own rule and model corpus built before anything is measured. Recommend it on a cold start when the run is not urgent; it is the pass that makes the others accurate + +The signal for enactment is whether a finding set exists to be measured against, not the vocabulary. A user who says "audit this against last year's pentest" and has the pentest is enactment; a user who says "reproduce the bug I think is in here" and has only a hunch is discovery. + +When the user wants both — reproduce the report *and* find what it missed — say that it is two passes over one tree, recommend enactment first so the discovery pass inherits its boundaries, and run them one at a time. Never try to drive two modes in a single pass. ## Setup @@ -28,63 +85,85 @@ After installing, run `opentaint health` to confirm everything's resolved. This workflow requires two subagent levels: MAIN → stage orchestrator → leaf. Confirm the harness permits depth 2 before starting; otherwise ask the user to enable it. -### 3. Determine the language +### 3. Choose the mode and locate its input + +Pick `onboarding`, `discovery`, or `enactment` per Modes, confirm the choice with the user, and get the path its intake needs — the supplied findings for enactment, the diff or spec for discovery when there is one. Onboarding needs no input beyond the project. -Read the project's build files to fix the target language — Maven/Gradle → java, `go.mod` → go, and so on. Record it at bootstrap; stage orchestrators pass it to language-coupled leaves. +### 4. Determine the language -### 4. Choose the workflow +Read the project's build files to fix the target language — Maven/Gradle → java, `go.mod` → go, and so on. Record it at bootstrap; stage orchestrators pass it to language-coupled leaves. When a repo carries build markers for more than one, ask the user which to analyze. -Ask the user for both levels together: +### 5. Choose the workflow -1. Scan level — `lite` · `normal` · `deep` +Ask the user for the knobs this mode has, together: + +1. Scan level — `lite` · `normal` · `deep`, **discovery mode only** - lite — build + scan (expected, when there are already existing artifacts) - - normal — build + scan + custom approximations - - deep — build + scan + custom approximations + custom rules - - recommend by what's on disk: a cold start (no `.opentaint` artifacts) → deep; a prior run's artifacts already present → lite + - normal — build + scan + custom models + - deep — build + scan + custom models + custom universal rules + - recommend by what's on disk: a cold start (no `.opentaint` artifacts) → deep; a prior pass's artifacts already present → lite + - onboarding and enactment are always deep: sweeping the frontier and reproducing a finding set both need the full rule and model toolbox, so there is no level to ask for 2. Triage level — `static` · `dynamic` - static — classify findings from the model, no running app - dynamic — static + PoC per confirmed TP. This launches a few test services on the user's machine (local instances and ports), torn down at the end of the run. Make that clear in the option -### 5. Bootstrap +### 6. Bootstrap -Seed the run state and the working tree with the chosen levels and language: +Seed the run state and the working tree: ```bash -uv run /scripts/generate.py init --scan-level --triage-level --language +uv run /scripts/generate.py init --mode \ + --triage-level --language \ + [--scan-level ] [--findings ] [--spec ] ``` -It writes `state.yaml`, seeds `history.yaml`, and creates the `.opentaint/` tree. +`--scan-level` is discovery's; `--findings` is enactment's and required the first time; `--spec` is discovery's and optional. It writes `state.yaml`, appends this pass to `history.yaml`, and creates the `.opentaint/` tree — plus `tracking/reference/` in enactment mode. + +Over a tree an earlier pass already built, it prints what carried over and keeps all of it — that pass's rules, models, boundary specs, and verdicts are this pass's starting corpus, and `get_status.py` will report their phases `DONE` rather than redoing them. It refuses a second onboarding pass over an already-onboarded tree. ## Workflow -The run is one fixed pipeline; the selected levels determine which phases are in scope. Use `uv run /scripts/get_status.py` to choose the next action: +The run is one fixed pipeline, the same in every mode; the mode decides what intake works from, and the selected levels decide which phases are in scope. Use `uv run /scripts/get_status.py` to choose the next action: ``` build → MAIN: build -discover / source_rules → stage subagent: sources +intake → stage subagent: intake +boundaries → stage subagent: boundaries +source_rules → stage subagent: sources scan → MAIN: scan approximations → stage subagent: approx-round, then MAIN: rescan; repeat sink_rules → stage subagent: sinks, then MAIN: rescan triage → stage subagent: triage poc → stage subagent: poc +crossref → stage subagent: crossref (whenever the tree carries a reference set) ``` +The spine is fixed — build, sources, scan, models, sinks, triage — and it is fixed for a reason. Both boundary sides are known before the first scan, because that scan is what proves them and names the taint frontier the model work then answers; the sink rules are authored against that frontier rather than guessed ahead of it. Never let a model stand in for a boundary that was never authored: an approximation carries taint through a carrier, it does not decide what is untrusted or what is dangerous. + +`crossref` appears whenever the tree carries a reference set — the pass that supplied it need not be this one. This pass's rescans changed what those findings reproduce, so re-judging them and refreshing `.opentaint/enactment.md` is part of finishing, not optional cleanup. + ### Build in MAIN When status reports `build`, load and follow the `build-project` skill in this main session. Run its long build command through the harness's main-session background-command facility and wait for its completion event. -Record the returned `build_jdk` in `.opentaint/tracking/state.yaml`. Record `model_commit` as the full HEAD only when no source file is uncommitted, otherwise set it to null. Build non-convergence blocks the run because no later phase can proceed without the model. +Record the returned `build_jdk` in `.opentaint/tracking/state.yaml`. Record `model_commit` as the full HEAD only when no source file is uncommitted, otherwise set it to null. Build non-convergence blocks the run because no later phase can proceed without the model. Keep one untouched model for the whole pass. ### Scan in MAIN When status reports `scan`, or a stage returns with a rescan pending, load and follow the `run-scan` skill in this main session. Start the scan with the harness's main-session background-command facility, keep the engine's self-timeout, add a 1200-second outer backstop, and wait for the process completion event. -A valid `.opentaint/results/report.sarif` means the scan completed, including exit 254 after an engine timeout. Record `max_memory: 16G` when the scan had to bump memory and reuse it on later scans. If no SARIF exists after the allowed retry/backstop, follow the repair path below for a malformed rule/approximation; otherwise dispatch `orchestrate-stage` with `stage: escalation` and the scan `setup` to write the scan-wide resource issue, then stop. +A valid `.opentaint/results/report.sarif` means the scan completed, including exit 254 after an engine timeout. Record `max_memory: 16G` when the scan had to bump memory and reuse it on later scans. If no SARIF exists after the allowed retry/backstop, follow the repair path below for a malformed rule/model; otherwise dispatch `orchestrate-stage` with `stage: escalation` and the scan `setup` to write the scan-wide resource issue, then stop. -When a scan or later stage reports a malformed approximation, unloadable created rule, ineffective join, or a created rule's false positive/negative, route the exact diagnosis and artifact path/id to the responsible stage agent per Dispatching, then scan again in MAIN. +When a scan or later stage reports a malformed model, unloadable created rule, ineffective join, or a created rule's false positive/negative, route the exact diagnosis and artifact path/id to the responsible stage agent per Dispatching, then scan again in MAIN. After every build, scan, or stage return, run `uv run /scripts/get_status.py` once to choose the next action. Use `--full` at run start, on resume, or when the brief output does not settle the question. +### Iterating + +The pipeline loops by design, and status is what closes it. A rescan makes every cross-reference pending again; a trace stopped at an opaque carrier sends the run back to a model round; a rule blamed for a miss goes back to the stage that authored it, so status returns there before closing again. Follow status through each loop rather than declaring the run finished early — an earlier phase reading `IN_PROGRESS` after a later one ran is the loop working, not a regression. + +The run is complete when status reports `run complete`: `.opentaint/vulnerabilities.md` current, and — where the tree carries a reference set — every supplied finding either reproduced or recorded with the blocker that stopped it, in `.opentaint/enactment.md`. Report the outcome keeping raw SARIF results, validated findings, and unique vulnerability identities as separate counts. + ## Dispatching Dispatch exactly one stage-orchestrator subagent for each stage invocation: @@ -92,17 +171,20 @@ Dispatch exactly one stage-orchestrator subagent for each stage invocation: ``` Invoke the Skill orchestrate-stage first, then follow its instructions precisely Inputs: - stage: + stage: ``` -For a `deep` approximation round, also pass `sinks: true`. A subagent inherits the project-root working directory, so omit `project-root`. +For a `deep` model round, also pass `sinks: true`. A subagent inherits the project-root working directory, so omit `project-root`. Stage context: -- `sources` — discover dependency sources, author their rules, and wire the joins +- `intake` — turn this mode's input into the run's families: the swept frontier in onboarding, the diff or spec in discovery, the normalized reference set in enactment +- `boundaries` — generalize each family into one universal source and one universal sink, and seed the rule units from them +- `sources` — author the seeded source units' rules and wire the joins - `approx-round` — classify and build one dropped-method frontier; use a fresh agent for each new frontier -- `sinks` — author classified sink rules and wire the joins +- `sinks` — author the seeded sink units' rules and wire the joins - `triage` — classify the latest findings and refresh the vulnerability report +- `crossref` — judge a reference set against the latest scan and refresh its coverage manifest - `poc` — reproduce confirmed findings and add the outcomes to the report - `escalation` — repair or settle a stage artifact, or report a scan-wide no-SARIF failure @@ -116,32 +198,45 @@ Use this ownership map to route work and scan errors: ``` .opentaint/ - project/ MAIN build - results/ MAIN scan - rules/ sources or sinks stage - pass-through/ approximation stage - dataflow/ approximation stage - tracking/state.yaml MAIN run knobs - tracking/ stage agents, leaves, and join scripts otherwise - vulnerabilities.md triage / PoC stage - issues/ escalation stage + project/ MAIN build + results/ MAIN scan + rules/ sources or sinks stage + pass-through/ approximation stage + dataflow/ approximation stage + tracking/state.yaml MAIN run knobs + tracking/scope.yaml intake stage (onboarding, discovery) + tracking/reference/ intake stage writes, crossref stage judges (enactment) + tracking/boundaries/ boundaries stage + tracking/ stage agents, leaves, and join scripts otherwise + enactment.md crossref stage + vulnerabilities.md triage / PoC stage + issues/ escalation stage ``` -The tree is long-lived. On resume, reuse `DONE` artifacts; `get_status.py` derives the next phase from disk. Existing rules and approximations apply to every scan. +The tree is long-lived and outlives this pass. On resume, reuse `DONE` artifacts; `get_status.py` derives the next phase from disk. Existing rules and models apply to every scan, whichever pass created them. Never delete or rewrite an artifact because this pass didn't produce it — an onboarding pass's classification ledger, a discovery pass's boundary specs, and an enactment pass's reference set are all as durable as your own. -`state.yaml` shape: +`state.yaml` shape — `mode` is this pass's intake, not a property of the tree, so a later pass in another mode simply rewrites it and keeps everything else: ```yaml +mode: enactment scan_level: deep triage_level: dynamic language: java +findings: reports/pentest-2026-07.md +spec: null model_commit: 0123456789abcdef0123456789abcdef01234567 build_jdk: null max_memory: null ``` +`findings` is the supplied set an enactment pass is measured against and `spec` is what a discovery pass was scoped by; both stay in `state.yaml` across passes in other modes, so a later pass in that mode resumes the same input. Neither is edited by hand mid-pass — pointing an in-flight pass at a different file strands the intake built from the old one. A genuinely different input is a new pass, bootstrapped with a new `--findings` or `--spec`. + ## Key constraints -- read pipeline state through `/scripts/get_status.py`, not by hand — don't re-derive it with glob/grep/`python3 -c`/yaml scans over `.opentaint/tracking`, `results`, or the `*.yaml`, nor open finding/unit/SARIF files just to review progress. If its output doesn't settle the question, re-run it with `--full` before opening any file +- read pipeline state through `/scripts/get_status.py`, not by hand — don't re-derive it with glob/grep/`python3 -c`/yaml scans over `.opentaint/tracking`, `results`, or the `*.yaml`, nor open finding/unit/reference/SARIF files just to review progress. If its output doesn't settle the question, re-run it with `--full` before opening any file - don't author or edit stage-owned artifacts or tracking; MAIN writes only `model_commit`, `build_jdk`, and `max_memory` in `state.yaml` - keep one generated project model for the run; never hand-edit or replace it mid-analysis — fix the build and rebuild before starting a new run +- source and sink boundaries come before model work; a model never compensates for a boundary that was never authored +- run one mode per pass, and never switch `mode` mid-pass: the intake behind the current families would no longer be the one on disk +- where the tree carries a reference set, coverage is counted by unique finding identity, never by rule id or raw SARIF result count, and a result counts as a reproduction only when its trace carries the finding's own attack path +- never drop a supplied finding as unsuitable for taint analysis diff --git a/skills/appsec-agent/scripts/_common.py b/skills/appsec-agent/scripts/_common.py index 092f216dd..9c2ad0b8e 100644 --- a/skills/appsec-agent/scripts/_common.py +++ b/skills/appsec-agent/scripts/_common.py @@ -1,4 +1,4 @@ -"""Shared helpers for the appsec-agent orchestrator scripts. +"""Shared helpers for the OpenTaint pipeline orchestrator scripts. Not a runnable script — imported by the PEP723 entry points (get_status.py, generate.py), which carry the pyyaml dependency. Every path resolves under the fixed @@ -6,6 +6,7 @@ scripts from the project root. """ import glob +import re import subprocess from pathlib import Path @@ -20,6 +21,9 @@ SINKS_TR = RULES_TR / "sinks" JOINS_TR = RULES_TR / "joins" FINDINGS_TR = TRACKING / "findings" +SCOPE = TRACKING / "scope.yaml" # every mode: what intake scoped, as the family list +REFERENCE_TR = TRACKING / "reference" # enactment mode: the supplied findings, normalized +BOUNDARIES_TR = TRACKING / "boundaries" # every mode: per-family universal boundary specs RESULTS = ROOT / "results" DROPPED = RESULTS / "dropped-external-methods.yaml" SARIF = RESULTS / "report.sarif" @@ -96,6 +100,22 @@ def git_head(): return None +# ---- intake scope ---- + +def scope_families(): + """(name, evidence) per family the intake stage scoped, in scope.yaml order. + + One shape for every mode: the evidence items are reference finding ids in enactment mode, + and the members or code areas intake settled on in onboarding and discovery mode. The + boundaries stage generalizes one family per entry, whichever mode wrote it.""" + out = [] + for f in (load_yaml(SCOPE, {}) or {}).get("families") or []: + name = strip_quotes((f or {}).get("name", "")) if isinstance(f, dict) else "" + if name: + out.append((name, [strip_quotes(str(e)) for e in (f.get("evidence") or [])])) + return out + + # ---- approximation batch readers (shared by coverage + partition) ---- # a method counts classified once it sits in any batch classification bucket or in build.done @@ -176,6 +196,12 @@ def build_done_keys(): return keys +# ---- finding files ---- + +RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) +VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) + + def ledger_verdicted_keys(): """method+signature keys already verdicted in classification.yaml (source ∪ safe).""" doc = load_yaml(RULES_TR / "classification.yaml", {}) or {} diff --git a/skills/appsec-agent/scripts/generate.py b/skills/appsec-agent/scripts/generate.py index 8f98987bc..51e1d1339 100644 --- a/skills/appsec-agent/scripts/generate.py +++ b/skills/appsec-agent/scripts/generate.py @@ -9,8 +9,8 @@ init bootstrap the .opentaint tree + state.yaml from the workflow flags partition analyze dropped external methods -> per-root batch plans (approximations) - partition discover coverage.yaml's used members -> balanced discover plans - mark-safe discover plans' verdicts -> classification.yaml ledger (+prune plans) + partition frontier coverage.yaml's used members -> balanced frontier plans + mark-safe frontier plans' verdicts -> classification.yaml ledger (+prune plans) merge-skipped batch skipped/engine_issues -> approximations/skipped.yaml (+prune plans) findings results/report.sarif -> per-rule finding tracking files (idempotent) """ @@ -26,19 +26,18 @@ import yaml -from _common import (APPROX, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, class_of, classified_keys, - dropped_entries, dump_yaml, fqn_base, git_head, - ledger_verdicted_keys, load_yaml, member_key, package_of, - strip_quotes) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, + SOURCES_TR, RULE_RE, TRACKING, VERDICT_RE, class_of, classified_keys, + batch_files, dropped_entries, dump_yaml, fqn_base, git_head, + ledger_verdicted_keys, load_yaml, member_key, package_of, strip_quotes) ANALYZE_BUDGET = 20 # methods per approximation batch ANALYZE_MISC = 6 # roots with <= this many methods pool into one misc batch ROOT_DEPTH = 2 # library root = first 2 dotted segments -DISCOVER_TARGET, DISCOVER_BAND = 50, 15 # project-used members per discover plan (~50, loose) +FRONTIER_TARGET, FRONTIER_BAND = 50, 15 # project-used members per frontier plan (~50, loose) -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" @@ -47,33 +46,98 @@ # the durable directories a run writes into; the leaves/scripts mkdir on write, but seeding # them up front gives every stage a place to land and makes the empty tree self-describing. INIT_DIRS = [TRACKING, APPROX, SOURCES_TR, SINKS_TR, JOINS_TR, FINDINGS_TR, - RESULTS, RULES, PASS_THROUGH, DATAFLOW] + BOUNDARIES_TR, RESULTS, RULES, PASS_THROUGH, DATAFLOW] +ENACTMENT_DIRS = [REFERENCE_TR] # the normalized supplied findings STATE_DERIVED = ("model_commit", "build_jdk", "max_memory") # build/scan fill these, init preserves +def carried_over(): + """What a pass inherits from the passes before it, as (label, count) — everything durable + the tree already holds. Every mode writes into one tree, so a pass never starts empty + unless the tree is.""" + def n(paths): + return sum(1 for _ in paths) + return [("rule unit", n(SOURCES_TR.glob("*.yaml")) + n(SINKS_TR.glob("*.yaml"))), + ("created rule", n(p for p in RULES.rglob("*.yaml") if p.is_file())), + ("boundary spec", n(BOUNDARIES_TR.glob("*.yaml")) if BOUNDARIES_TR.is_dir() else 0), + ("approximation batch", len(batch_files())), + ("reference finding", n(REFERENCE_TR.glob("*.yaml")) if REFERENCE_TR.is_dir() else 0), + ("triaged finding", n(FINDINGS_TR.glob("*.yaml")))] + + +def onboarded(runs): + """Onboarding sweeps the whole external-method frontier and its corpus is durable, so it is + a one-time pass. Re-running init in onboarding mode is a resume while the tree is still in + that pass; once a later mode took over, the sweep is done and asking for it again is a + mistake worth naming.""" + return any(str((r or {}).get("type", "")).startswith("onboarding/") for r in runs) + + def cmd_init(args): - for d in INIT_DIRS: - d.mkdir(parents=True, exist_ok=True) state_path = TRACKING / "state.yaml" prior = load_yaml(state_path, {}) or {} - resume = bool(prior) - state = {"scan_level": args.scan_level, "triage_level": args.triage_level, + enactment = args.mode == "enactment" + # the finding set is a property of the tree, not of one pass — an enactment pass that follows + # an earlier one inherits it, so --findings is required only the first time + findings = args.findings or prior.get("findings") + if enactment and not findings: + raise SystemExit("init --mode enactment requires --findings ") + if args.mode == "discovery" and not args.scan_level: + raise SystemExit("init --mode discovery requires --scan-level") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if args.mode == "onboarding" and onboarded(runs) and prior.get("mode") != "onboarding": + raise SystemExit("this tree is already onboarded — its frontier sweep, universal rules, " + "and models are on disk and every later pass builds on them; run " + "discovery or enactment instead") + # onboarding sweeps the frontier and enactment reproduces a supplied finding set: both always + # need the full rule + approximation toolbox, so their level is fixed rather than asked for. + scan_level = args.scan_level if args.mode == "discovery" else "deep" + + for d in INIT_DIRS + (ENACTMENT_DIRS if enactment else []): + d.mkdir(parents=True, exist_ok=True) + # `mode` is this pass's intake, not a permanent property of the tree: the modes compose, in + # any order and repeatedly across commits, and every artifact below is shared between them. + state = {"mode": args.mode, "scan_level": scan_level, "triage_level": args.triage_level, "language": args.language or prior.get("language")} + if findings: # kept across other modes, so a later enactment pass resumes it + state["findings"] = findings + spec = args.spec or prior.get("spec") + if spec: + state["spec"] = spec for k in STATE_DERIVED: # never clobber what build/scan already learned state[k] = prior.get(k) state_path.write_text(dump_yaml(state), encoding="utf-8") - # history: append one run entry on a fresh init, never on resume (the derived knobs survived) + # history: one entry per pass. Re-running init with the same knobs on the same commit is a + # resume of the current pass, not a new one — a different mode, level, or commit starts one. hist_path = TRACKING / "history.yaml" - runs = (load_yaml(hist_path, {}) or {}).get("runs") or [] - if not resume: - runs.append({"commit": git_head(), "type": f"{args.scan_level}/{args.triage_level}"}) + entry = {"commit": git_head(), + "type": f"{args.mode}/{scan_level}/{args.triage_level}"} + new_pass = not runs or runs[-1] != entry + if new_pass: + runs.append(entry) hist_path.write_text(dump_yaml({"runs": runs}), encoding="utf-8") - mode = "resumed (derived knobs preserved)" if resume else "fresh" - print(f"init {mode}: scan_level={state['scan_level']} triage_level={state['triage_level']} " - f"language={state['language']}") - print(f"seeded {len(INIT_DIRS)} directories under .opentaint/") + prior_mode = prior.get("mode") + if not prior: + how = "fresh tree" + elif prior_mode and prior_mode != args.mode: + how = f"new {args.mode} pass over the existing {prior_mode} tree" + elif new_pass: + how = f"new {args.mode} pass (pass {len(runs)})" + else: + how = "resumed (derived knobs preserved)" + print(f"init {how}: mode={args.mode} scan_level={scan_level} " + f"triage_level={state['triage_level']} language={state['language']}") + if findings: + print(f"findings={findings}") + if spec: + print(f"spec={spec}") + if prior: + kept = ", ".join(f"{c} {label}{'' if c == 1 else 's'}" for label, c in carried_over() if c) + print(f"carried over: {kept or 'nothing yet'}") + print(f"seeded {len(INIT_DIRS) + (len(ENACTMENT_DIRS) if enactment else 0)} " + "directories under .opentaint/") print("next: uv run scripts/get_status.py --full") return 0 @@ -163,7 +227,7 @@ def write_plans(plans, out_dir, prefix_id): if isinstance(x, dict) else x) for p, v in sorted(scopes.items())} path = out_dir / f"{pid}.yaml" - # source: null is the unprocessed sentinel — a discover agent overwrites it with the + # source: null is the unprocessed sentinel — a frontier agent overwrites it with the # list of sources it found (an empty list when it finds none). mark-safe folds only # plans whose sentinel was replaced, so an un-returned plan is never marked safe. path.write_text(dump_yaml({"id": pid, "scopes": norm, "source": None}), encoding="utf-8") @@ -227,7 +291,7 @@ def cmd_analyze(args): return 0 -# ---- partition discover ---- +# ---- partition frontier ---- def yaml_modules(model_yaml): # each module in project.yaml as (packages, moduleClasses); only `packages` says which of a @@ -299,8 +363,8 @@ def pending_packages(): return tuple(p for p in (cov.get("packages") or []) if isinstance(p, str) and p) -def cmd_discover(args): - regen_plans(DISCOVER_PLANS) +def cmd_frontier(args): + regen_plans(FRONTIER_PLANS) packages = pending_packages() if not packages: print("nothing to plan — no pending package in coverage.yaml", file=sys.stderr) @@ -314,26 +378,26 @@ def cmd_discover(args): if not todo: print("nothing to plan — every used member already verdicted", file=sys.stderr) return 0 - cap = DISCOVER_TARGET + DISCOVER_BAND - plans = pack(atomize(todo, cap), DISCOVER_TARGET, cap) + cap = FRONTIER_TARGET + FRONTIER_BAND + plans = pack(atomize(todo, cap), FRONTIER_TARGET, cap) rows = {f: [{"method": f, "signature": s} for s in sorted(sigs[f])] for f in sigs} plans = [{pkg: [r for f in members for r in rows[f]] for pkg, members in plan.items()} for plan in plans] - for p in write_plans(plans, DISCOVER_PLANS, "lib"): + for p in write_plans(plans, FRONTIER_PLANS, "lib"): print(p) return 0 def cmd_partition(args): - return cmd_analyze(args) if args.kind == "analyze" else cmd_discover(args) + return cmd_analyze(args) if args.kind == "analyze" else cmd_frontier(args) -# ---- mark-safe (discover join) ---- +# ---- mark-safe (frontier join) ---- def cmd_mark_safe(args): - plans = sorted(glob.glob(str(DISCOVER_PLANS / "lib-*.yaml"))) + plans = sorted(glob.glob(str(FRONTIER_PLANS / "lib-*.yaml"))) if not plans: - print("no discover plans to reconcile", file=sys.stderr) + print("no frontier plans to reconcile", file=sys.stderr) return 0 ledger = RULES_TR / "classification.yaml" doc = load_yaml(ledger, {}) or {} @@ -343,7 +407,7 @@ def cmd_mark_safe(args): for p in plans: pdoc = load_yaml(p, {}) or {} raw = pdoc.get("source") - if raw is None: # sentinel intact — no discover agent returned for it + if raw is None: # sentinel intact — no frontier agent returned for it unprocessed.append(p) continue members = {member_key(m) for v in (pdoc.get("scopes") or {}).values() for m in v} @@ -353,7 +417,7 @@ def cmd_mark_safe(args): processed.append(p) print(f"{Path(p).name}: {len(srcs)} sources, {len(members - srcs)} safe") if not processed: - print("no processed discover plans (every plan still carries source: null) — " + print("no processed frontier plans (every plan still carries source: null) — " "fan out discover-attack-surface first", file=sys.stderr) return 0 safe -= source @@ -363,7 +427,7 @@ def cmd_mark_safe(args): if not args.keep: for p in processed: Path(p).unlink() - print(f"pruned {len(processed)} reconciled discover plan(s)") + print(f"pruned {len(processed)} reconciled frontier plan(s)") if unprocessed: print(f"left {len(unprocessed)} unprocessed plan(s) (source: null) for re-dispatch: " + ", ".join(Path(p).name for p in unprocessed)) @@ -420,10 +484,8 @@ def cmd_merge_skipped(args): _FP_PREFERENCE = ("vulnerabilitySourceSinkHash", "vulnerabilityWithTraceHash") -RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) HASHES_RE = re.compile(r'^sarif_hashes:\s*\[(.*)\]\s*$', re.M) HASHES_BLOCK_RE = re.compile(r'^sarif_hashes:\s*\n((?:[ \t]+-[^\n]*\n?)+)', re.M) -VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) def docker_name(seed, taken): @@ -561,17 +623,27 @@ def main(): sub = ap.add_subparsers(dest="cmd", required=True) i = sub.add_parser("init", help="bootstrap the .opentaint tree + state.yaml from workflow flags") - i.add_argument("--scan-level", required=True, choices=["lite", "normal", "deep"]) + i.add_argument("--mode", required=True, choices=["onboarding", "discovery", "enactment"], + help="onboarding: sweep the external-method frontier once. discovery: work " + "from the project, a diff, or an informal spec. enactment: reproduce a " + "supplied finding set") + i.add_argument("--scan-level", choices=["lite", "normal", "deep"], + help="discovery mode only; onboarding and enactment are always deep") i.add_argument("--triage-level", required=True, choices=["static", "dynamic"]) i.add_argument("--language", default=None, help="target language, determined by the orchestrator") + i.add_argument("--findings", default=None, + help="enactment mode: path to the supplied finding manifest/report/directory") + i.add_argument("--spec", default=None, + help="discovery mode: path to the diff, spec, or note that scopes the pass; " + "omit to scope the pass to the whole project") i.set_defaults(func=cmd_init) p = sub.add_parser("partition", help="split classification work into per-agent plans") - p.add_argument("kind", choices=["analyze", "discover"]) + p.add_argument("kind", choices=["analyze", "frontier"]) p.set_defaults(func=cmd_partition) - m = sub.add_parser("mark-safe", help="merge discover plans into classification.yaml") - m.add_argument("--keep", action="store_true", help="keep the reconciled discover plans") + m = sub.add_parser("mark-safe", help="merge frontier plans into classification.yaml") + m.add_argument("--keep", action="store_true", help="keep the reconciled frontier plans") m.set_defaults(func=cmd_mark_safe) s = sub.add_parser("merge-skipped", help="rebuild approximations/skipped.yaml from batches") @@ -581,6 +653,7 @@ def main(): f = sub.add_parser("findings", help="seed per-rule finding files from results/report.sarif") f.set_defaults(func=cmd_findings) + args = ap.parse_args() return args.func(args) diff --git a/skills/appsec-agent/scripts/get_status.py b/skills/appsec-agent/scripts/get_status.py index aca3b06b6..e8411bb26 100644 --- a/skills/appsec-agent/scripts/get_status.py +++ b/skills/appsec-agent/scripts/get_status.py @@ -22,19 +22,21 @@ import sys from pathlib import Path -from _common import (APPROX, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, ROOT, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, build_done_keys, classified_keys, - dropped_entries, git_head, load_yaml, member_key, - modeled_entries, skipped_keys) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, ROOT, RULES, RULES_TR, SARIF, SCOPE, SINKS_TR, + SOURCES_TR, TRACKING, build_done_keys, classified_keys, dropped_entries, + git_head, load_yaml, member_key, modeled_entries, scope_families, + skipped_keys, strip_quotes) STATE = load_yaml(TRACKING / "state.yaml", {}) or {} +MODE = STATE.get("mode") or "discovery" SCAN_LEVEL = STATE.get("scan_level") TRIAGE_LEVEL = STATE.get("triage_level") -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" VULN = ROOT / "vulnerabilities.md" +ENACTMENT = ROOT / "enactment.md" GLOBAL_CAP = 10 @@ -49,6 +51,12 @@ def load_units(d): if Path(d).is_dir() else [] +def load_docs(d): + """(path, doc) for every tracking file in a directory — reference, boundary, control.""" + return [(p, load_yaml(p, {}) or {}) for p in sorted(Path(d).glob("*.yaml"))] \ + if Path(d).is_dir() else [] + + def load_joins(): return [(p.stem, load_yaml(p, {}) or {}) for p in sorted(JOINS_TR.glob("*.yaml"))] \ if JOINS_TR.is_dir() else [] @@ -157,10 +165,13 @@ def ph_build(): "dispatch build-project"], None -def ph_discover(): +def ph_frontier(): + """Onboarding's intake: the external-method frontier, swept as one trust boundary. Every + dependency member the project's own code calls is a candidate boundary until a leaf verdicts + it, and the ledger is what makes that verdict durable for every later pass.""" if not (TRACKING / "coverage.yaml").is_file(): return False, ["dispatch triage-dependencies"], None - leftover = sorted(glob.glob(str(DISCOVER_PLANS / "*.yaml"))) + leftover = sorted(glob.glob(str(FRONTIER_PLANS / "*.yaml"))) units = load_units(SOURCES_TR) ledger = load_yaml(RULES_TR / "classification.yaml", {}) or {} if leftover: @@ -169,7 +180,7 @@ def ph_discover(): tasks.append("then run `scripts/generate.py mark-safe` to reconcile the plans") return False, tasks, None if not ledger and not units: - return False, ["run `scripts/generate.py partition discover` to plan the used members"], None + return False, ["run `scripts/generate.py partition frontier` to plan the called members"], None return True, [], None @@ -280,18 +291,141 @@ def ph_poc(): return True, [], None +# ---- intake: one contract, three mode-specific derivations ---- + +def ph_reference_set(): + """Enactment's intake: the supplied findings, normalized and grouped. The `family` field on + each reference file is the assignment — it moves with the finding when a family splits, so + the families are read back from the files rather than from a separate list.""" + docs = load_docs(REFERENCE_TR) + if not docs: + src = STATE.get("findings") or "state.yaml findings unset" + return False, [f"normalize the supplied findings ({src}) into " + ".opentaint/tracking/reference/.yaml"], None + missing = sorted(p.stem for p, d in docs if not strip_quotes(d.get("family", ""))) + if missing: + return False, ["reference findings not assigned to a boundary family:"] \ + + [f" {m}" for m in missing], None + return True, [], None + + +def ph_scope(): + """Onboarding's and discovery's intake join: the families their evidence groups into, written + to scope.yaml. Enactment carries the same information on its reference files instead.""" + fams = scope_families() + if not fams: + src = STATE.get("spec") or "the whole project" + what = ("the frontier the sweep verdicted as sources and effects" if MODE == "onboarding" + else f"the code {src} names") + return False, [f"group {what} into families and write {SCOPE}"], None + empty = [n for n, ev in fams if not ev] + if empty: + return False, ["families in scope.yaml with no evidence recorded:"] \ + + [f" {n}" for n in empty], None + return True, [], None + + +def ph_intake(): + if MODE == "enactment": + return ph_reference_set() + if MODE == "onboarding": + done, tasks, note = ph_frontier() + if not done: + return done, tasks, note + return ph_scope() + + +def families(): + """(family, evidence ids) for this pass, whichever mode scoped it — reference finding ids in + enactment mode, the members or areas intake recorded otherwise. The boundaries stage + generalizes one universal source and sink per entry.""" + if MODE == "enactment": + out = {} + for path, doc in load_docs(REFERENCE_TR): + fam = strip_quotes(doc.get("family", "")) + if fam: + out.setdefault(fam, []).append(path.stem) + return sorted(out.items()) + return scope_families() + + +def ph_boundaries(): + specs = {p.stem: d for p, d in load_docs(BOUNDARIES_TR)} + fams = families() + missing = [f for f, _ in fams if f not in specs] + if missing: + return False, ["dispatch discover-universal-boundaries, one per family:"] \ + + [f" {f}" for f in missing], None + # a split rewrites the family on the evidence it moves, so every spec here owns its evidence + unsaturated = [f for f, _ in fams + if str((specs[f].get("saturation") or {}).get("status", "")).strip() + != "saturated"] + if unsaturated: + return False, ["boundary specs not saturated:"] + [f" {f}" for f in unsaturated], None + unfactored = sorted(f"{f}: {e}" for f, evidence in fams for e in evidence + if e not in (specs[f].get("factorization") or {})) + if unfactored: + return False, ["evidence with no factorization in its spec:"] \ + + [f" {r}" for r in unfactored], None + unseeded = [f for f, _ in fams if (specs[f].get("stages") or {}).get("units_seeded") != "done"] + if unseeded: + return False, ["seed the source and sink units from these specs' candidate_patterns:"] \ + + [f" {f}" for f in unseeded], None + return True, [], None + + +def ph_crossref(): + if not SARIF.is_file(): + return False, ["dispatch run-scan"], None + docs = load_docs(REFERENCE_TR) + scanned = SARIF.stat().st_mtime + pend = [p for p, d in docs + if str(d.get("crossref", "pending")).strip() != "done" or p.stat().st_mtime < scanned] + if pend: + return False, [f"cross-reference the scan against {len(pend)} reference finding(s):"] \ + + [f" {p}" for p in pend], None + blocked = sorted({str(m) for _, d in docs for m in (d.get("blocked_at") or [])}) + if blocked: + return False, ["expected traces stop at unmodeled carriers:"] + [f" {m}" for m in blocked] \ + + ["model them in an approximation round, rescan, then cross-reference again"], None + rep = sum(1 for _, d in docs if str(d.get("status", "")).strip() == "reproduced") + stale = newest_mtime([p for p, _ in docs]) > (ENACTMENT.stat().st_mtime + if ENACTMENT.is_file() else 0) + if not ENACTMENT.is_file() or stale: + return False, [f"rewrite .opentaint/enactment.md coverage manifest " + f"({rep}/{len(docs)} reproduced)"], None + return True, [], None + + +def has_reference_set(): + """A reference set outlives the pass that created it. Any later pass that rescans changes what + it reproduces, so the cross-reference stays in scope — otherwise an onboarding or discovery + pass would leave a coverage manifest that silently describes an older scan.""" + return REFERENCE_TR.is_dir() and any(REFERENCE_TR.glob("*.yaml")) + + +# One pipeline, whatever the mode brought to it. Intake and its universal boundaries differ — +# the swept frontier, the diff or spec, the supplied findings — but from the boundary specs on, +# every mode runs the same stages in the same order: sources before the scan that proves them, +# approximations against the frontier that scan names, sinks after it, then triage. Only the +# closing phases are conditional, on the triage level, the controls knob, and whether the tree +# carries a reference set to re-judge. PHASES = [ ("build", ph_build, lambda: True), - ("discover", ph_discover, lambda: SCAN_LEVEL == "deep"), + ("intake", ph_intake, lambda: SCAN_LEVEL == "deep"), + ("boundaries", ph_boundaries, lambda: SCAN_LEVEL == "deep"), ("source_rules", ph_source_rules, lambda: SCAN_LEVEL == "deep"), ("scan", ph_scan, lambda: True), ("approximations", ph_approximations, lambda: SCAN_LEVEL in ("normal", "deep")), ("sink_rules", ph_sink_rules, lambda: SCAN_LEVEL == "deep"), ("triage", ph_triage, lambda: True), ("poc", ph_poc, lambda: TRIAGE_LEVEL == "dynamic"), + ("crossref", ph_crossref, has_reference_set), ] + + # ---- caps ---- def free_gb(): @@ -329,8 +463,16 @@ def evaluate(): def cmd_full(): commit = short(STATE.get("model_commit")) or "none" - print(f"scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} language={STATE.get('language')} " - f"commit={commit} cap={GLOBAL_CAP} (heavy {heavy_cap()})") + print(f"mode={MODE} scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} " + f"language={STATE.get('language')} commit={commit} " + f"cap={GLOBAL_CAP} (heavy {heavy_cap()})") + if STATE.get("findings"): # printed in every mode — the set stays tracked + print(f"findings={STATE.get('findings')}") + if STATE.get("spec"): + print(f"spec={STATE.get('spec')}") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if len(runs) > 1: + print("passes: " + " -> ".join(str(r.get("type", "?")) for r in runs)) rows, current = evaluate() # a phase downstream of the current stage that vacuously satisfies its own check is not # actually done — its producing stage hasn't run — so it reads PENDING, never DONE. diff --git a/skills/create-pass-through-approximation/references/java.md b/skills/create-pass-through-approximation/references/java.md index 238e762c5..9022e8d6e 100644 --- a/skills/create-pass-through-approximation/references/java.md +++ b/skills/create-pass-through-approximation/references/java.md @@ -179,6 +179,7 @@ passThrough: ### 3. Common mistakes to check +- *every* method in a file still dropped, with no load error — the whole file was silently skipped: check it starts with the `language: java` header (a headerless or mis-headered file loads to nothing) - the `function` matcher doesn't match the real method — check the package, class, name, and `overrides` - a `from`/`to` points at the wrong position — it must land where the data actually is - a broken slot channel — the writer and reader name different `Class#slot#java.lang.Object` triples, or the slot isn't typed `java.lang.Object`, so the taint drops between them diff --git a/skills/discover-attack-surface/SKILL.md b/skills/discover-attack-surface/SKILL.md index 84af32d47..73d9ded34 100644 --- a/skills/discover-attack-surface/SKILL.md +++ b/skills/discover-attack-surface/SKILL.md @@ -1,6 +1,6 @@ --- name: discover-attack-surface -description: Classify project-used dependency members and record the taint sources among them not covered by the built-in rules. Use for the source-discovery depth pass +description: Classify a slice of the external-method frontier — the dependency members the project calls — as trust boundaries or not, and record the taint sources among them that the built-in rules don't cover. Use for the frontier sweep of an onboarding pass, or wherever a scoped attack surface has to be classified license: Apache-2.0 metadata: author: opentaint @@ -9,7 +9,7 @@ metadata: # Skill: Discover Attack Surface -Work one assignment of project-used dependency members and pick out the taint sources among them — the methods where untrusted data first enters. The concrete inspection commands and value formats are language-specific — read `references/.md` per Inputs and follow its numbered steps, which key to the ones below +Work one assignment of the external-method frontier — the dependency members the project's own code calls — and pick out the trust boundaries among them: the methods where untrusted data first enters. Every member on the plan arrives as a candidate boundary and leaves with a verdict, so a member you leave unjudged is a gap in the sweep, not a default-safe. The concrete inspection commands and value formats are language-specific — read `references/.md` per Inputs and follow its numbered steps, which key to the ones below ## Inputs diff --git a/skills/discover-universal-boundaries/SKILL.md b/skills/discover-universal-boundaries/SKILL.md new file mode 100644 index 000000000..73ff6f582 --- /dev/null +++ b/skills/discover-universal-boundaries/SKILL.md @@ -0,0 +1,192 @@ +--- +name: discover-universal-boundaries +description: Generalize a family of attack-surface evidence — finding traces, swept frontier members, or the code a diff or spec names — into one reusable source and one reusable sink boundary, saturated against the whole family. Use before rule authoring when reproducing supplied findings, turning a discovered surface into universal rules, consolidating case-specific rules, or replacing an incidental source such as a map access or an arbitrary method call +license: Apache-2.0 +metadata: + author: opentaint + version: "0.3.0" +--- + +# Discover universal boundaries + +Generalize one family of known attack-surface evidence into rule-ready boundaries: a single source and a single sink that every item in the family factors through. The evidence differs with the pass that produced it — a reference finding's trace, a frontier member a sweep verdicted, a code area a diff or spec named — and the reasoning is the same for all three. The boundary vocabulary — how a member is named, what carries a usage condition, which packages realize each primitive effect — is language-specific: read `references/.md` per Inputs and follow its numbered steps, which key to the ones below. + +The goal is not the broadest syntax that happens to match every trace. It is the most primitive semantic boundary that stays inside the vulnerability class — general enough that the family needs one rule per side, specific enough that the rule still means something. Precision is recovered afterwards with context restrictions and sanitizers, listed separately, never by narrowing the boundary back down to the evidence you started from. + +A boundary is only universal once it has been saturated: widened, re-checked against the whole family, and left unchanged by a full round. + +## Inputs + +Provided by the caller, fall back to the default value when omitted. Ask back only when a required input is missing and has no sensible default + +- `project-root` (optional) — root of the target project. Opentaint keeps all analysis artifacts under the fixed `/.opentaint/` directory, so every `.opentaint/...` path below resolves there. Default: current directory +- `language` (required) — target language for this project and language-specific instructions +- `family` (required) — kebab-case name of the family; also the name of its spec and of the rule units seeded from it +- `evidence` (required) — the ids assigned to this family. Reference finding ids when the pass has a reference set, whose normalized files are `.opentaint/tracking/reference/.yaml`; otherwise the members or code areas `.opentaint/tracking/scope.yaml` recorded for the family +- `findings` (optional) — path to the supplied finding manifest or report the reference set was normalized from, when there is one. Read it for detail a normalized file doesn't carry + +## Workflow + +### 1. Reconstruct every trace + +For each assigned evidence item, read what the pass recorded for it — a reference finding's normalized file, or the member or code area `scope.yaml` named — and then the project source it points at, and record: + +1. the attacker or untrusted authority; +2. the first project-visible ingress; +3. the transformations and trust-domain crossings on the way; +4. the validators, guards, and authorization decisions it passes; +5. the primitive security-relevant effect at the end; and +6. the vulnerable invariant that is absent or defeated. + +Read enough surrounding source to tell the real boundary from incidental syntax — the language reference says how to reach a dependency member's source, and which local shapes are propagation rather than ingress. A getter, collection lookup, or service method is usually propagation or context, not ingress. + +### 2. Propose the source + +Move backward from the evidence's own expressions until you reach the earliest reusable trust-boundary value the family shares. Prefer, in order when applicable: + +- request body, parameter, path, query, header, cookie, or multipart value; +- message, frame, packet, event, webhook, or callback payload; +- deserialized external object; +- persisted attacker-controlled record at a second-order re-entry boundary; +- tenant or user-controlled configuration at activation or read-back; +- environment or runtime configuration entering a security decision; or +- an explicit structural pseudo-source standing for attacker-selected identity or control state. + +The language reference names the boundaries its built-in source rules already carry — one they match needs no new boundary — and what identifies a boundary in that language. Reject a candidate that is merely a map or collection read after untrusted data has already entered, a `$METAVAR.method(...)` with no boundary type, signature, declarative marker, or enclosing entrypoint, a ubiquitous getter that would taint trusted objects just as readily, or an internal carrier that ordinary propagation or a later approximation should handle. + +Express narrow usage conditions separately, as typed patterns, `pattern-inside`, `...`, and whatever declarative marker the language offers. Never bake incidental access syntax into the source. + +### 3. Propose the sink + +Move forward from the evidence's own service calls to the most primitive operation that realizes the vulnerability, while staying specific to its class. Prefer boundaries such as: + +- network connect, request, send, or download for SSRF; +- process, script, expression, template, query, or deserialization execution for injection; +- path resolution plus filesystem read/write for traversal; +- privileged object read or disclosure for IDOR and data exposure; +- privileged object mutation or durable state commit for integrity and authorization failures; +- authentication or authorization decision for control bypass; +- redirect or token release for redirect and OAuth issues; +- signature verification or unsigned callback state commit for callback integrity; +- secret activation, credential construction, or outbound use for secret exposure; and +- logger argument consumption for log injection. + +The language reference names the packages that realize these effects, and its built-in sinks. Don't stop at a controller-to-service call when the primitive effect is analyzable deeper in the project or a reusable library sink can express it. Use a structural sink only when the vulnerability *is* the missing control at that boundary, or when deeper propagation is genuinely unavailable. + +### 4. Saturate + +A boundary proposed from a few traces is a guess until it survives the whole family. Loop until a full round changes nothing. + +Each round: + +1. Factor every assigned evidence item — not only the new ones — through the current boundaries: + + ```text + universal source -> item-specific context -> propagation -> universal sink + ``` + +2. For each item that does not factor, generalize the offending side by exactly one step toward a more primitive boundary — never by adding a second alternative that merely spells out that item's syntax. A `pattern-either` listing one branch per evidence item is the failure this skill exists to prevent. +3. Re-check the items that already factored. A widening that breaks an earlier factorization is a widening too far: back it out and split instead. +4. Challenge the widened boundary in both directions — if the sink now admits a different vulnerability class, narrow it back to the primitive effect or add a class-specific context restriction; if the source now admits trusted values, record what separates them as a context restriction rather than shrinking the boundary. +5. Record the round: what changed, and which factorization statuses moved. + +The family is saturated when a full round widened nothing, broke nothing, and left every assigned item either `covered` or `needs-restriction` with a named restriction. Stop and split — or record the item `unfactored`, with what blocked it in `open_questions` — rather than looping a fourth time on the same one. + +When the only boundary the whole family shares is arbitrary syntax, the family was wrong: split it. Each subfamily gets its own spec named `-` and its own saturation loop, and every assigned item lands in exactly one subfamily. A moved reference finding has its own file's `family` rewritten to the subfamily that now owns it; other evidence moves with the spec it is listed in, and the calling stage reconciles the family list to the specs you return. + +Keep independently triggerable paths distinct in the factorization even when they share both boundaries. + +### 5. Identify the precision controls + +List these separately from the positive boundaries — the spec records them, and they must never be folded into the boundary patterns: + +- sanitizers that actually enforce the relevant invariant; +- `pattern-not` exclusions for safe expression forms; +- `pattern-not-inside` exclusions for safe guarded regions; +- `pattern-inside` contexts that constrain a universal boundary to the intended entrypoint, type, tenant, or vulnerability family; and +- validators that are *not* security-relevant, recorded explicitly so nothing later mistakes them for sanitizers. + +Real sanitization looks like resolved-IP private-range rejection for SSRF, canonical-path containment for traversal, strict identifier ownership checks for IDOR, signature verification for callbacks, and CR/LF neutralization for log injection. Presence, length, and parsing alone normally sanitize none of these; the language reference names the usual impostors and what the real controls look like in that language. + +### 6. Write the specification + +Write one spec per family or subfamily, listing the assigned ids under `evidence`. `candidate_patterns` must be concrete enough for `create-rule` to test, in the member shape that language's rule units use — the language reference gives that shape and the dependency identity to record with it. Record opaque carriers under `approximation_candidates`, and stop there: no approximation is created or recommended until a rule-first scan proves a trace stops at one. + +## Output + +Return every spec path written (one per family or subfamily) and, in a few lines each: + +- the universal source, and which more incidental candidates you rejected; +- the universal sink, and the vulnerability-class scope it stays inside; +- how many saturation rounds it took and what the last round changed; +- the assigned evidence by factorization status — `covered`, `needs-restriction`, `unfactored`; +- any family split, with the evidence that moved and — when it is a reference finding — the files you rewrote; +- the sanitizers, negative patterns, and context restrictions listed separately from the boundaries; and +- unresolved boundary evidence, as the `open_questions` entries. + +Never paste file contents. + +## Tracking + +This skill writes the boundary spec for its family, and — only when a split moves a reference finding — the `family` field on the reference files it moved. Leave `stages.units_seeded` pending: seeding the rule units is the calling stage's step. Touch no other reference field, no scope file, no rule unit, and no rule. + +`.opentaint/tracking/boundaries/.yaml` — one boundary family: the single universal source and single universal sink every piece of the family's evidence factors through, plus the controls that recover precision. `evidence` lists what the family was grouped from — reference finding ids in enactment mode, the members or code areas `scope.yaml` recorded otherwise. `candidate_patterns` are concrete enough for `create-rule` to test, and seed the family's source and sink units verbatim. `factorization` carries one entry per evidence item, `status` one of `covered` (factors through both boundaries as-is), `needs-restriction` (factors only under a named `context_restrictions` entry), or `unfactored` (does not factor — explain in `open_questions` and split or add a pseudo-boundary). `saturation` records the widen-and-recheck rounds and only reads `saturated` once a full round changed neither the boundaries nor any factorization. `approximation_candidates` are opaque carriers noted for later — never acted on before a scan proves the trace stops there. Keep it clear from comments + +```yaml +family: ssrf +evidence: [DSC-014, DSC-021] +source: + semantic_boundary: external request value entering a controller + candidate_patterns: + - { method: org.springframework.web.bind.annotation.RequestBody, signature: null, note: annotated controller parameter } + context_restrictions: [] +sink: + semantic_boundary: outbound HTTP request with a caller-supplied URL + candidate_patterns: + - { method: org.springframework.web.client.RestTemplate#getForObject, signature: "(Ljava/lang/String;Ljava/lang/Class;[Ljava/lang/Object;)Ljava/lang/Object;", vuln_class: ssrf, note: egress with attacker URL } + context_restrictions: [] +sanitizers: [] +negative_patterns: [] +factorization: + DSC-014: { source: request body callbackUrl, sink: RestTemplate#getForObject, status: covered } + DSC-021: { source: request param targetUrl, sink: RestTemplate#getForObject, status: needs-restriction, restriction: admin-only controller } +approximation_candidates: [] +open_questions: [] +saturation: + rounds: 3 + status: saturated +stages: + units_seeded: done +``` + +`.opentaint/tracking/reference/.yaml` — one supplied finding, normalized to a stable identity and carried through boundary discovery and the cross-reference. The file is named for the finding's own id. `family` ties it to its boundary spec and is rewritten when a family splits. `status` is `pending` until the cross-reference judges it, then `reproduced` or `unreproduced`; `cause` explains an `unreproduced` one so the pipeline knows who owns it — `rule` (a boundary, restriction, or sanitizer is wrong), `approximation` (an opaque carrier breaks the path), or `engine` (a modeling limit, paired with `blocker`). `blocked_at` lists the carriers still to model and is cleared once they are modeled or judged terminal. `matched_hashes` are the SARIF result hashes whose trace carries this finding's identity — never a rule-id match alone. Keep it clear from comments + +```yaml +id: DSC-014 +vuln_class: ssrf +family: ssrf +source: request body field `callbackUrl` on POST /api/webhook/register +propagation: WebhookReqVO -> WebhookDO -> WebhookService#dispatch +sink: RestTemplate#getForObject in WebhookService#dispatch +expected_location: yudao-module-infra/.../WebhookService.java:88 +guards: URL parsed with new URI(...), no private-range rejection +status: reproduced +cause: null +blocker: null +blocked_at: [] +matched_hashes: [a1b2c3d4e5f6a7b8] +crossref: done +notes: > + crossref: join ssrf-webhook-ext fired at WebhookService#dispatch:88 with the trace entering at + the registration body — same attack path as the reference finding +``` + +## Constraints + +- Work only the assigned evidence, your family's spec, and the reference `family` field on a split +- Universal does not mean untyped or unconstrained — generalize the boundary, then recover precision with context restrictions and sanitizers, never by re-narrowing the boundary to the evidence +- Never propose all methods, all map values, or all getters as a source, and never a `pattern-either` with one branch per evidence item +- Never mark the spec saturated on a round that widened, broke, or split anything +- Never propose a sanitizer that would suppress a trace the evidence says is real, and never record a validator as a sanitizer without the invariant it enforces +- Record a poor taint fit as an explicit pseudo-boundary; never drop the evidence item +- Don't create or recommend approximations here — note the opaque carriers and stop diff --git a/skills/discover-universal-boundaries/references/java.md b/skills/discover-universal-boundaries/references/java.md new file mode 100644 index 000000000..a0652708b --- /dev/null +++ b/skills/discover-universal-boundaries/references/java.md @@ -0,0 +1,50 @@ +# discover-universal-boundaries — Java / JVM + +## Workflow + +### 1. Reconstruct every trace + +Dependency members come from the resolved jars under `.opentaint/project/dependencies`: locate a class with `unzip -l | grep `, read its signatures with `javap -p -s -classpath `, and prefer the source jar or a decompiler when the body matters. + +Java-shaped propagation that is not ingress: a DTO or entity getter, `Map#get` after the request was already bound, a service method delegating a value it was handed, an `Optional`/stream chain over it, and framework binding of a value the endpoint already accepted. + +### 2. Propose the source + +Built-in Java source rules live under `java/lib/{generic,spring}/` within the `opentaint health --rules` root, the project's own under `.opentaint/rules/java`. A boundary they already match needs no new one, and their reach is wider than it looks: + +- `lib/spring/untrusted-data-source.yaml` taints *every* non-scalar parameter of a method annotated `@RequestMapping`/`@GetMapping`/`@PostMapping`/`@PutMapping`/`@PatchMapping`/`@DeleteMapping`, whatever the parameter's own annotation — so `@RequestBody`, `@RequestParam`, `@PathVariable`, and `@ModelAttribute` on a Spring MVC endpoint are already sourced, as is a cookie read through `WebUtils#getCookie` +- `lib/generic/servlet-untrusted-data-source.yaml` taints the `HttpServletRequest` parameter of a servlet entry method (`doGet`, `doPost`, `doPut`, `doDelete`, `doTrace`, `_jspService`) as a whole object, plus `MessageBodyReader#readFrom`, a commons-fileupload `parseRequest(...).getName()`, and `Part#getSubmittedFileName` +- `lib/spring/untrusted-path-source.yaml` and `lib/generic/seam-untrusted-data-source.yaml` carry their own shapes + +A Java family's shared source is therefore usually *outside* those shapes, and that is where to aim: a non-HTTP transport (`TextMessage#getPayload`, `ConsumerRecord#value`, a JMS or gRPC payload), a reactive `ServerRequest`/`ServerHttpRequest` accessor, a non-Spring stack's own request accessor, or a second-order re-entry where a persisted attacker-controlled record is read back. + +Two shapes carry a Java boundary: an annotation on the declaration, with the enclosing annotated class or method as its `pattern-inside` context, or an accessor on a boundary type. Take the annotated parameter or the accessor itself, never the DTO getter downstream of it — the getter would taint trusted objects just as readily, and the binding step is already propagation. + +### 3. Propose the sink + +Where the primitive effects live, with the built-in `java/lib/` coverage named: + +- SSRF — `RestTemplate`, `URL#openConnection` (`generic/ssrf-sinks.yaml`); `WebClient` and `HttpClient#send` are not built in +- command injection — `Runtime#exec`, `ProcessBuilder` (`generic/command-injection-sinks.yaml`) +- SQL — `Statement#execute*`, `JdbcTemplate`, an `EntityManager` native query (`spring/jdbc-sqli-sinks.yaml`) +- path traversal — `Files#*` reached through `Paths#get`/`Path#resolve` (`generic/path-traversal-sinks.yaml`) +- SSTI, expression, and code injection — the template engine's `process`/`merge` (`generic/template-injection-sinks.yaml`), `SpelExpressionParser` (`spring/spel-injection-sinks.yaml`), `generic/code-injection-sinks.yaml` +- deserialization — `ObjectMapper#readValue`, a SnakeYAML `load` (`generic/unsafe-deserialization-sinks.yaml`); `ObjectInputStream#readObject` is not built in +- log injection — the `Logger` call's message or argument (`generic/logging-sinks.yaml`) +- and their own files for XXE, LDAP, SMTP, reflection, unvalidated redirect, response splitting, and XSS response writes + +Check the bundled set before proposing a sink boundary: it is broad enough that a family often needs only the source side. Prefer the library boundary over the project wrapper that calls it — a reusable library sink keeps the rule useful past this run. + +### 5. Identify the precision controls + +Java's validation impostors: `@Valid`/`@Validated` with Bean Validation constraints, `@NotNull`, `@Size`, or `@Pattern` on a shape that is not the invariant, and a `parse`/`valueOf` that only proves the value's type. Record them as not-security-relevant validators rather than sanitizers. + +Real Java controls: `InetAddress#getByName` plus a private-range rejection for SSRF, `Path#normalize` plus a `startsWith` containment check for traversal, an owner-scoped repository lookup for IDOR, `MessageDigest#isEqual` or HMAC verification for callbacks, and a context-correct encoder or CR/LF stripping for output and log injection. + +A restriction usually reads as `pattern-inside` on the enclosing annotated class or method — that is what keeps a universal accessor scoped to the endpoint that actually takes untrusted input. + +### 6. Write the specification + +`candidate_patterns` entries take `method` as `owner.Class#member` with `signature` its quoted JVM descriptor, so overloads stay distinct. An annotation-carried boundary takes the annotation's own FQN as `method` with `signature: null`, as the spec format shows. The dependency identity to record on the seeded units is the Maven GAV, `group:artifact:version`. + +`create-rule`'s `references/java.md` holds the pattern shapes those units turn into. diff --git a/skills/orchestrate-stage/SKILL.md b/skills/orchestrate-stage/SKILL.md index e46ab21f9..6a700174b 100644 --- a/skills/orchestrate-stage/SKILL.md +++ b/skills/orchestrate-stage/SKILL.md @@ -16,14 +16,19 @@ Complete one bounded stage of the OpenTaint security workflow. Dispatch its inde Match the stage keyword to its reference and read it fully: ``` +intake → /references/intake.md +boundaries → /references/boundaries.md sources → /references/sources.md approx-round → /references/approx-round.md sinks → /references/sinks.md triage → /references/triage.md poc → /references/poc.md +crossref → /references/crossref.md (a tree carrying a reference set) escalation → /references/escalation.md ``` +`intake` is the only stage whose work depends on the run's `mode`; its reference carries a section per mode. Everything after it reads the same whichever mode produced the families. + Run the bundled script to get the setup overview before proceeding to the reference's instructions: ```bash diff --git a/skills/orchestrate-stage/references/approx-round.md b/skills/orchestrate-stage/references/approx-round.md index 95c1b0375..847747f20 100644 --- a/skills/orchestrate-stage/references/approx-round.md +++ b/skills/orchestrate-stage/references/approx-round.md @@ -37,6 +37,7 @@ It merges every batch's `skipped` / `engine_issues` into `approximations/skipped - `language` - `batch` - `methods` (optional) — only an explicitly assigned subset to redo + - dataflow step 1 → dispatch create-test-project - `language` - `type: dataflow` diff --git a/skills/orchestrate-stage/references/boundaries.md b/skills/orchestrate-stage/references/boundaries.md new file mode 100644 index 000000000..82f7542a9 --- /dev/null +++ b/skills/orchestrate-stage/references/boundaries.md @@ -0,0 +1,84 @@ +# Universal boundaries + +Generalize each family intake produced into one saturated universal source and one universal sink, and seed the source and sink units those boundaries imply. Every mode runs this stage: the families differ — a swept frontier, a diff or spec, a supplied finding set — but what comes out is the same universal rule material, and everything it writes feeds the ordinary rule-authoring stages. + +Read the family list from `get_status.py`, which takes it from `.opentaint/tracking/scope.yaml` in onboarding and discovery mode, and from the reference findings' own `family` field in enactment mode. + +## Discover the boundaries + +Fan out discover-universal-boundaries, one leaf per family. + +Inputs each: +- `language` +- `family` +- `evidence` — the ids the family carries: reference finding ids in enactment mode, the members or code areas `scope.yaml` recorded otherwise +- `findings` (enactment mode) — the supplied-findings path `state.yaml` names, so a leaf can read a finding beyond its normalized file + +Expect back — `.opentaint/tracking/boundaries/.yaml` with `saturation.status: saturated`, one `factorization` entry per assigned evidence item, and the controls listed separately from the positive boundaries. A leaf that splits its family writes one spec per subfamily; in enactment mode it also rewrites `family` on each reference finding it moved, and otherwise the split is recorded in the specs themselves — reconcile `scope.yaml` to the specs that came back so the family list and the specs still agree. + +`.opentaint/tracking/boundaries/.yaml` — one boundary family: the single universal source and single universal sink every piece of the family's evidence factors through, plus the controls that recover precision. `evidence` lists what the family was grouped from — reference finding ids in enactment mode, the members or code areas `scope.yaml` recorded otherwise. `candidate_patterns` are concrete enough for `create-rule` to test, and seed the family's source and sink units verbatim. `factorization` carries one entry per evidence item, `status` one of `covered` (factors through both boundaries as-is), `needs-restriction` (factors only under a named `context_restrictions` entry), or `unfactored` (does not factor — explain in `open_questions` and split or add a pseudo-boundary). `saturation` records the widen-and-recheck rounds and only reads `saturated` once a full round changed neither the boundaries nor any factorization. `approximation_candidates` are opaque carriers noted for later — never acted on before a scan proves the trace stops there. Keep it clear from comments + +```yaml +family: ssrf +evidence: [DSC-014, DSC-021] +source: + semantic_boundary: external request value entering a controller + candidate_patterns: + - { method: org.springframework.web.bind.annotation.RequestBody, signature: null, note: annotated controller parameter } + context_restrictions: [] +sink: + semantic_boundary: outbound HTTP request with a caller-supplied URL + candidate_patterns: + - { method: org.springframework.web.client.RestTemplate#getForObject, signature: "(Ljava/lang/String;Ljava/lang/Class;[Ljava/lang/Object;)Ljava/lang/Object;", vuln_class: ssrf, note: egress with attacker URL } + context_restrictions: [] +sanitizers: [] +negative_patterns: [] +factorization: + DSC-014: { source: request body callbackUrl, sink: RestTemplate#getForObject, status: covered } + DSC-021: { source: request param targetUrl, sink: RestTemplate#getForObject, status: needs-restriction, restriction: admin-only controller } +approximation_candidates: [] +open_questions: [] +saturation: + rounds: 3 + status: saturated +stages: + units_seeded: done +``` + +A spec returning with an `unfactored` evidence item is not a failure to retry blindly — read its `open_questions`, and either re-dispatch the leaf with that item split out as its own family or accept the pseudo-boundary it proposes. + +## Seed the rule units + +For each saturated spec, write its `candidate_patterns` into the family's source and sink units, then set `stages.units_seeded: done` on the spec. The unit file name is the family, so the two sides and the spec stay tied together. + +`.opentaint/tracking/rules/sources/.yaml` — one source unit per package (a dependency can span several packages, each its own unit), the file named for that package with `.` → `-`. `dependencies` names the dependency the package comes from, `sources` each an entry point `{ method, signature, note, rule_id }` (`signature` the member's JVM descriptor, always quoted so array types `[…` stay valid YAML in a flow mapping), `stages` tracks the unit through rule authoring, and a `blocker` string is added under it when the unit can't be made to pass. Keep it clear from comments + +```yaml +dependencies: + - org.springframework:spring-websocket:6.1.0 +sources: + - { method: org.springframework.web.socket.TextMessage#getPayload, signature: "()Ljava/lang/String;", note: untrusted WebSocket frame data, rule_id: null } +stages: + test_project: pending + tests_passing: pending +``` + +`.opentaint/tracking/rules/sinks/.yaml` — one sink unit per package (a dependency can span several packages, each its own unit), the file named for that package with `.` → `-`. `dependencies` names the dependency the package comes from, `sinks` each a dangerous operation reached by the taint frontier `{ method, signature, vuln_class, note, rule_id }` — `signature` the member's JVM descriptor so overloads stay distinct, always quoted (array types contain `[`, which is invalid unquoted in a flow mapping), `vuln_class` per entry since one package can host several, `note` a few words on the danger, the tainted argument left unpinned. `stages` tracks the unit through rule authoring. Keep it clear from comments + +```yaml +dependencies: + - cn.hutool:hutool-core:5.8.20 +sinks: + - { method: cn.hutool.core.io.FileUtil#writeBytes, signature: "([BLjava/lang/String;)Ljava/io/File;", vuln_class: path-traversal, note: writes data to an untrusted path, rule_id: null } +stages: + test_project: pending + tests_passing: pending +``` + +Copy `method`, `signature`, and `note` from the spec's `candidate_patterns` (plus `vuln_class` on the sink side), leave `rule_id: null` and the `stages` pending, and fill `dependencies` with the dependency each pattern's package comes from — empty when the boundary is a project member, as a structural pseudo-boundary usually is. Do not carry `context_restrictions`, `sanitizers`, or `negative_patterns` into the units: the boundary is the positive pattern, and the controls stay listed in the spec. + +Both sides are seeded here, and only the source side is authored before the first scan. That is deliberate: the sink boundary is decided now, on the evidence, and authored later against the frontier that scan names — never invented by a model round that found a carrier. + +## Stage gate + +`get_status.py` drives `boundaries`, naming families without a spec, unsaturated specs, evidence with no factorization, and unseeded specs. Finish when it reads `DONE`, or when the next step it reports is source rules. diff --git a/skills/orchestrate-stage/references/crossref.md b/skills/orchestrate-stage/references/crossref.md new file mode 100644 index 000000000..961383877 --- /dev/null +++ b/skills/orchestrate-stage/references/crossref.md @@ -0,0 +1,65 @@ +# Cross-reference — scan results against the reference set + +Judge, per supplied finding, whether the latest scan reproduced it, and assemble the coverage manifest. This is the stage that closes an enactment pass: every rule, model, and verdict is already in place, so what the scan shows now is what the pass delivered. Nothing else may set a reference finding's `status`. + +It runs in any mode over a tree that carries a reference set an earlier enactment pass left. That pass's new rules and rescans changed what those findings reproduce — usually for the better, occasionally by losing one — so re-judge them all and refresh the manifest. The work is identical; only the pass that triggered it differs. + +It is also the stage that decides what the run still owes. A judgement here can send the pipeline back — an unmodeled carrier to an approximation round, a rule-caused miss to the stage that authored the rule — and status will report that earlier phase as current again. That is the loop working; re-enter this stage after the rescan rather than closing on stale results. + +## Match by identity, never by rule id + +For each reference finding `get_status.py` lists as pending, compare the scan's results against its recorded identity: the source the trace enters at, the propagation it goes through, the sink it reaches, and the location. A result counts as a reproduction only when its trace carries the same attack path. A rule firing somewhere, a matching vulnerability class, or a matching file is not a match. + +Record the outcome on the reference file: + +`.opentaint/tracking/reference/.yaml` — one supplied finding, normalized to a stable identity and carried through boundary discovery and the cross-reference. The file is named for the finding's own id. `family` ties it to its boundary spec and is rewritten when a family splits. `status` is `pending` until the cross-reference judges it, then `reproduced` or `unreproduced`; `cause` explains an `unreproduced` one so the pipeline knows who owns it — `rule` (a boundary, restriction, or sanitizer is wrong), `approximation` (an opaque carrier breaks the path), or `engine` (a modeling limit, paired with `blocker`). `blocked_at` lists the carriers still to model and is cleared once they are modeled or judged terminal. `matched_hashes` are the SARIF result hashes whose trace carries this finding's identity — never a rule-id match alone. Keep it clear from comments + +```yaml +id: DSC-014 +vuln_class: ssrf +family: ssrf +source: request body field `callbackUrl` on POST /api/webhook/register +propagation: WebhookReqVO -> WebhookDO -> WebhookService#dispatch +sink: RestTemplate#getForObject in WebhookService#dispatch +expected_location: yudao-module-infra/.../WebhookService.java:88 +guards: URL parsed with new URI(...), no private-range rejection +status: reproduced +cause: null +blocker: null +blocked_at: [] +matched_hashes: [a1b2c3d4e5f6a7b8] +crossref: done +notes: > + crossref: join ssrf-webhook-ext fired at WebhookService#dispatch:88 with the trace entering at + the registration body — same attack path as the reference finding +``` + +- reproduced — set `status: reproduced` and put the matching SARIF result hashes in `matched_hashes` +- overmatch — the trace reproduces, but the same rule also fires on flows that are not this finding: still `reproduced`, and note the overmatch in `notes` +- stops at an opaque carrier — `status: unreproduced`, `cause: approximation`, and the exact carrier(s) in `blocked_at`. Get the carrier from `.opentaint/results/dropped-external-methods.yaml` or a localized reachability trace, not from a guess +- no source or sink fact on the expected path — `status: unreproduced`, `cause: rule`. Say in `notes` which side is missing and why, and report it to MAIN: the stage that authored the rule fixes it and the run comes back here +- reproducible only by a modeling limit the engine cannot express — `status: unreproduced`, `cause: engine`, one-line `blocker`, and `blocked_at` cleared + +Set `crossref: done` on every file you judge. A later rescan makes them pending again, which is the loop working: judge them against the new results rather than trusting the old verdict. + +Independently, note scan results that carry no reference finding — those are OpenTaint-exclusive and go to triage like any other finding. Do not count them as reproductions. + +## Close the blocked carriers + +When `get_status.py` reports traces stopping at unmodeled carriers, that work belongs to an approximation round, not here: report the carriers to MAIN, which runs the round and the rescan and re-enters this stage. Only after the rescan proves a carrier still breaks the path does it become an `engine` cause with a `blocker`. + +## Write the coverage manifest + +Once no reference finding is pending or blocked, rewrite `.opentaint/enactment.md` from the current reference files: + +- the finding-level coverage table — one row per supplied finding: id, vulnerability class, family, `status`, and the one-clause reason for an unreproduced one +- the three counts kept apart: raw SARIF results, validated findings, unique vulnerability identities +- reproduced, OpenTaint-exclusive, and reference-exclusive findings as separate sections, exclusives only when triage validated them +- the reusable artifacts the run produced: source rules, sink rules, joins, approximations +- one blocker line per non-reproduced finding, naming the rule, modeling, or engine limitation that remains + +Reflect only current state — rewrite the manifest, don't append to it. + +## Stage gate + +`get_status.py` names pending reference findings, blocked carriers, then the manifest. Finish when `crossref` is `DONE`. Report reproduced/unreproduced totals and every blocker. diff --git a/skills/orchestrate-stage/references/intake.md b/skills/orchestrate-stage/references/intake.md new file mode 100644 index 000000000..baafc8bec --- /dev/null +++ b/skills/orchestrate-stage/references/intake.md @@ -0,0 +1,121 @@ +# Intake — this mode's input, as the run's families + +Turn what the pass was given into the families the boundaries stage generalizes. The three modes start from different material and converge on the same handoff: a named family per group, each carrying the evidence it was grouped from. Nothing here authors a rule, proposes a boundary, or touches a model. + +Read `mode` from `get_status.py --full` and follow that mode's section. A family is the set of evidence you expect to share one universal source and one universal sink — partition by vulnerability class or by a cohesive attack surface, never by file batches or arbitrary count. + +## Onboarding — the external-method frontier + +Every dependency member the project's own code calls is a trust boundary until a leaf says otherwise. That over-approximation is the point: the sweep classifies the whole frontier once, and every later pass inherits the verdicts. + +### 1. Triage the dependencies + +Dispatch triage-dependencies when status names it. + +Expect back — `.opentaint/tracking/coverage.yaml` written with the flagged packages; status advances to the frontier partition. + +### 2. Partition the frontier + +Run: + +```bash +uv run /scripts/generate.py partition frontier +``` + +It writes balanced `.opentaint/tracking/rules/plans/lib-NNN.yaml` plans over the flagged packages' members the project actually calls, one disjoint slice per leaf. On re-entry it partitions only members no prior pass verdicted. + +### 3. Sweep it + +Fan out discover-attack-surface, one per plan. + +Inputs each: +- `language` +- `plan` + +At the join run: + +```bash +uv run /scripts/generate.py mark-safe +``` + +Expect back — each leaf records the trust boundaries it finds into its plan and writes any source unit(s); the join folds source/safe verdicts into `classification.yaml` and prunes the consumed plans. The sweep is a single fan-out pass, not a loop. + +### 4. Group the frontier into families + +Read the verdicted boundaries back from the ledger and the source units the sweep wrote, and group them into families — usually one per attack surface a set of members shares (a framework's request accessors, a client library's egress calls), not one per dependency. Write them to `.opentaint/tracking/scope.yaml`, evidence being the members themselves. + +`.opentaint/tracking/scope.yaml` — what this pass's intake scoped, and the families the boundaries stage generalizes one at a time. `scope` is one line on what the pass was given, so a later reader knows what the family list came from. Each family carries the `evidence` it was grouped from: the frontier members a sweep verdicted, or the members, endpoints, and code areas a diff or spec resolved to. Enactment mode writes no scope file — its families live on the reference findings, which carry the same assignment on the file that moves with them. Keep it clear from comments + +```yaml +mode: discovery +scope: docs/2026-07-webhooks.md — partner webhook registration and delivery +families: + - name: ssrf-webhook + evidence: + - com.acme.webhook.WebhookController#register + - com.acme.webhook.WebhookService#dispatch + - name: upload-path + evidence: + - com.acme.upload.UploadController#store +``` + +## Discovery — the project, a diff, or a spec + +`state.yaml` names the scoping document under `spec` when there is one; with none, the scope is the whole project. + +Resolve the input to code before grouping it: read the diff or spec, then the project source it points at — the endpoints, handlers, jobs, and consumers it touches, and the calls they make into dependencies. An informal spec ("this service takes uploads from partners and renders them") is scoped the same way: name the surfaces it implies and read them. + +Then group what you read into families and write `.opentaint/tracking/scope.yaml`, evidence being the members, endpoints, or code areas each family was grouped from — concrete enough that the boundary leaf can go straight to the source. + +Fan this reading out when the scope is large: one leaf per slice, each reporting the surfaces it found. Assign the families yourself once every slice has reported, since that decision needs the whole scope. + +`.opentaint/tracking/scope.yaml` — what this pass's intake scoped, and the families the boundaries stage generalizes one at a time. `scope` is one line on what the pass was given, so a later reader knows what the family list came from. Each family carries the `evidence` it was grouped from: the frontier members a sweep verdicted, or the members, endpoints, and code areas a diff or spec resolved to. Enactment mode writes no scope file — its families live on the reference findings, which carry the same assignment on the file that moves with them. Keep it clear from comments + +```yaml +mode: discovery +scope: docs/2026-07-webhooks.md — partner webhook registration and delivery +families: + - name: ssrf-webhook + evidence: + - com.acme.webhook.WebhookController#register + - com.acme.webhook.WebhookService#dispatch + - name: upload-path + evidence: + - com.acme.upload.UploadController#store +``` + +## Enactment — the supplied finding set + +`state.yaml` names the supplied findings under `findings` — a manifest, SARIF, report, or directory of finding documents. Write one `.opentaint/tracking/reference/.yaml` per supplied finding. + +`.opentaint/tracking/reference/.yaml` — one supplied finding, normalized to a stable identity and carried through boundary discovery and the cross-reference. The file is named for the finding's own id. `family` ties it to its boundary spec and is rewritten when a family splits. `status` is `pending` until the cross-reference judges it, then `reproduced` or `unreproduced`; `cause` explains an `unreproduced` one so the pipeline knows who owns it — `rule` (a boundary, restriction, or sanitizer is wrong), `approximation` (an opaque carrier breaks the path), or `engine` (a modeling limit, paired with `blocker`). `blocked_at` lists the carriers still to model and is cleared once they are modeled or judged terminal. `matched_hashes` are the SARIF result hashes whose trace carries this finding's identity — never a rule-id match alone. Keep it clear from comments + +```yaml +id: DSC-014 +vuln_class: ssrf +family: ssrf +source: request body field `callbackUrl` on POST /api/webhook/register +propagation: WebhookReqVO -> WebhookDO -> WebhookService#dispatch +sink: RestTemplate#getForObject in WebhookService#dispatch +expected_location: yudao-module-infra/.../WebhookService.java:88 +guards: URL parsed with new URI(...), no private-range rejection +status: reproduced +cause: null +blocker: null +blocked_at: [] +matched_hashes: [a1b2c3d4e5f6a7b8] +crossref: done +notes: > + crossref: join ssrf-webhook-ext fired at WebhookService#dispatch:88 with the trace entering at + the registration body — same attack path as the reference finding +``` + +Give each finding a stable id of its own — the supplied one when it has one. Preserve separately triggerable attack paths as separate findings even when they share a sink. Never drop a finding for being a poor fit for taint analysis: an authorization, integrity, configuration, hard-coded-secret, or structural-control finding gets a reference file like any other, and becomes an explicit pseudo-boundary later. + +Then group the findings into families and set `family` on each. Enactment's families live on the reference files themselves rather than in `scope.yaml`, because the assignment has to move with the finding when a family splits. + +Fan out this normalization when the supplied set is large: one leaf per slice of the supplied report, each writing its own reference files. Assign the families yourself once every file exists, since that decision needs the whole set. + +## Stage gate + +`get_status.py` drives `intake`, naming the mode's next step — the dependency triage, the frontier plans, the scope file, or the findings still without a family. Finish when it reads `DONE`, or when the next step it reports is the boundaries stage. diff --git a/skills/orchestrate-stage/references/sinks.md b/skills/orchestrate-stage/references/sinks.md index 338becd36..7cfeeabfa 100644 --- a/skills/orchestrate-stage/references/sinks.md +++ b/skills/orchestrate-stage/references/sinks.md @@ -1,6 +1,6 @@ # Author sinks + assemble -Author the sink rules flagged into per-package units during approximation classification and join them to every relevant source. The final project rescan follows to surface findings. +Author the sink units this run seeded — the boundaries stage's sink side, plus the sinks flagged into per-package units during model classification — and join them to every relevant source. The final project rescan follows to surface findings. ## Sink lib rules diff --git a/skills/orchestrate-stage/references/sources.md b/skills/orchestrate-stage/references/sources.md index 961bb7003..822c7c698 100644 --- a/skills/orchestrate-stage/references/sources.md +++ b/skills/orchestrate-stage/references/sources.md @@ -1,36 +1,8 @@ -# Discover sources + source rules +# Author sources + assemble -Discover the project's used dependency members that are taint sources, author source rules, and wire them to the built-in sinks. The first project scan follows and names the later taint frontier. On re-entry, partition plans only members no prior run verdicted. +Author the source rules the boundaries stage seeded into per-family units, and wire them to every relevant sink. The first project scan follows: it is what proves the boundaries and names the taint frontier the later stages work from. -## Triage dependencies - -Dispatch triage-dependencies when status names it. - -Expect back — `.opentaint/tracking/coverage.yaml` written; status advances to source discovery. - -## Discover sources - -Run: - -```bash -uv run /scripts/generate.py partition discover -``` - -It writes balanced `.opentaint/tracking/rules/plans/lib-NNN.yaml` plans, one disjoint slice per leaf. Fan out discover-attack-surface, one per plan. - -Inputs each: -- `language` -- `plan` - -At the join run: - -```bash -uv run /scripts/generate.py mark-safe -``` - -Expect back — each agent records the sources it finds into its plan and writes any source unit(s); the join folds source/safe verdicts into `classification.yaml` and prunes the consumed plans. Discovery is a single fan-out pass, not a loop. - -Run `uv run /scripts/get_status.py` to confirm `discover` `DONE`. +On re-entry, work only the units `get_status.py` still lists as pending — a unit that already passes is a prior pass's result, not work to redo. ## Source lib rules @@ -66,4 +38,4 @@ Expect back — existing tag-expanded joins reused where they already cover the ## Stage gate -`get_status.py` drives `discover` then `source_rules`, naming the current sub-step and units. Finish when both are `DONE`, or when the next step it reports is the project scan. +`get_status.py` drives `source_rules`, naming the current sub-step and units. Finish when it reads `DONE`, or when the next step it reports is the project scan. diff --git a/skills/orchestrate-stage/scripts/_common.py b/skills/orchestrate-stage/scripts/_common.py index 092f216dd..9c2ad0b8e 100644 --- a/skills/orchestrate-stage/scripts/_common.py +++ b/skills/orchestrate-stage/scripts/_common.py @@ -1,4 +1,4 @@ -"""Shared helpers for the appsec-agent orchestrator scripts. +"""Shared helpers for the OpenTaint pipeline orchestrator scripts. Not a runnable script — imported by the PEP723 entry points (get_status.py, generate.py), which carry the pyyaml dependency. Every path resolves under the fixed @@ -6,6 +6,7 @@ scripts from the project root. """ import glob +import re import subprocess from pathlib import Path @@ -20,6 +21,9 @@ SINKS_TR = RULES_TR / "sinks" JOINS_TR = RULES_TR / "joins" FINDINGS_TR = TRACKING / "findings" +SCOPE = TRACKING / "scope.yaml" # every mode: what intake scoped, as the family list +REFERENCE_TR = TRACKING / "reference" # enactment mode: the supplied findings, normalized +BOUNDARIES_TR = TRACKING / "boundaries" # every mode: per-family universal boundary specs RESULTS = ROOT / "results" DROPPED = RESULTS / "dropped-external-methods.yaml" SARIF = RESULTS / "report.sarif" @@ -96,6 +100,22 @@ def git_head(): return None +# ---- intake scope ---- + +def scope_families(): + """(name, evidence) per family the intake stage scoped, in scope.yaml order. + + One shape for every mode: the evidence items are reference finding ids in enactment mode, + and the members or code areas intake settled on in onboarding and discovery mode. The + boundaries stage generalizes one family per entry, whichever mode wrote it.""" + out = [] + for f in (load_yaml(SCOPE, {}) or {}).get("families") or []: + name = strip_quotes((f or {}).get("name", "")) if isinstance(f, dict) else "" + if name: + out.append((name, [strip_quotes(str(e)) for e in (f.get("evidence") or [])])) + return out + + # ---- approximation batch readers (shared by coverage + partition) ---- # a method counts classified once it sits in any batch classification bucket or in build.done @@ -176,6 +196,12 @@ def build_done_keys(): return keys +# ---- finding files ---- + +RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) +VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) + + def ledger_verdicted_keys(): """method+signature keys already verdicted in classification.yaml (source ∪ safe).""" doc = load_yaml(RULES_TR / "classification.yaml", {}) or {} diff --git a/skills/orchestrate-stage/scripts/generate.py b/skills/orchestrate-stage/scripts/generate.py index 8f98987bc..51e1d1339 100644 --- a/skills/orchestrate-stage/scripts/generate.py +++ b/skills/orchestrate-stage/scripts/generate.py @@ -9,8 +9,8 @@ init bootstrap the .opentaint tree + state.yaml from the workflow flags partition analyze dropped external methods -> per-root batch plans (approximations) - partition discover coverage.yaml's used members -> balanced discover plans - mark-safe discover plans' verdicts -> classification.yaml ledger (+prune plans) + partition frontier coverage.yaml's used members -> balanced frontier plans + mark-safe frontier plans' verdicts -> classification.yaml ledger (+prune plans) merge-skipped batch skipped/engine_issues -> approximations/skipped.yaml (+prune plans) findings results/report.sarif -> per-rule finding tracking files (idempotent) """ @@ -26,19 +26,18 @@ import yaml -from _common import (APPROX, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, class_of, classified_keys, - dropped_entries, dump_yaml, fqn_base, git_head, - ledger_verdicted_keys, load_yaml, member_key, package_of, - strip_quotes) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, DROPPED, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, RESULTS, RULES, RULES_TR, SARIF, SINKS_TR, + SOURCES_TR, RULE_RE, TRACKING, VERDICT_RE, class_of, classified_keys, + batch_files, dropped_entries, dump_yaml, fqn_base, git_head, + ledger_verdicted_keys, load_yaml, member_key, package_of, strip_quotes) ANALYZE_BUDGET = 20 # methods per approximation batch ANALYZE_MISC = 6 # roots with <= this many methods pool into one misc batch ROOT_DEPTH = 2 # library root = first 2 dotted segments -DISCOVER_TARGET, DISCOVER_BAND = 50, 15 # project-used members per discover plan (~50, loose) +FRONTIER_TARGET, FRONTIER_BAND = 50, 15 # project-used members per frontier plan (~50, loose) -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" @@ -47,33 +46,98 @@ # the durable directories a run writes into; the leaves/scripts mkdir on write, but seeding # them up front gives every stage a place to land and makes the empty tree self-describing. INIT_DIRS = [TRACKING, APPROX, SOURCES_TR, SINKS_TR, JOINS_TR, FINDINGS_TR, - RESULTS, RULES, PASS_THROUGH, DATAFLOW] + BOUNDARIES_TR, RESULTS, RULES, PASS_THROUGH, DATAFLOW] +ENACTMENT_DIRS = [REFERENCE_TR] # the normalized supplied findings STATE_DERIVED = ("model_commit", "build_jdk", "max_memory") # build/scan fill these, init preserves +def carried_over(): + """What a pass inherits from the passes before it, as (label, count) — everything durable + the tree already holds. Every mode writes into one tree, so a pass never starts empty + unless the tree is.""" + def n(paths): + return sum(1 for _ in paths) + return [("rule unit", n(SOURCES_TR.glob("*.yaml")) + n(SINKS_TR.glob("*.yaml"))), + ("created rule", n(p for p in RULES.rglob("*.yaml") if p.is_file())), + ("boundary spec", n(BOUNDARIES_TR.glob("*.yaml")) if BOUNDARIES_TR.is_dir() else 0), + ("approximation batch", len(batch_files())), + ("reference finding", n(REFERENCE_TR.glob("*.yaml")) if REFERENCE_TR.is_dir() else 0), + ("triaged finding", n(FINDINGS_TR.glob("*.yaml")))] + + +def onboarded(runs): + """Onboarding sweeps the whole external-method frontier and its corpus is durable, so it is + a one-time pass. Re-running init in onboarding mode is a resume while the tree is still in + that pass; once a later mode took over, the sweep is done and asking for it again is a + mistake worth naming.""" + return any(str((r or {}).get("type", "")).startswith("onboarding/") for r in runs) + + def cmd_init(args): - for d in INIT_DIRS: - d.mkdir(parents=True, exist_ok=True) state_path = TRACKING / "state.yaml" prior = load_yaml(state_path, {}) or {} - resume = bool(prior) - state = {"scan_level": args.scan_level, "triage_level": args.triage_level, + enactment = args.mode == "enactment" + # the finding set is a property of the tree, not of one pass — an enactment pass that follows + # an earlier one inherits it, so --findings is required only the first time + findings = args.findings or prior.get("findings") + if enactment and not findings: + raise SystemExit("init --mode enactment requires --findings ") + if args.mode == "discovery" and not args.scan_level: + raise SystemExit("init --mode discovery requires --scan-level") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if args.mode == "onboarding" and onboarded(runs) and prior.get("mode") != "onboarding": + raise SystemExit("this tree is already onboarded — its frontier sweep, universal rules, " + "and models are on disk and every later pass builds on them; run " + "discovery or enactment instead") + # onboarding sweeps the frontier and enactment reproduces a supplied finding set: both always + # need the full rule + approximation toolbox, so their level is fixed rather than asked for. + scan_level = args.scan_level if args.mode == "discovery" else "deep" + + for d in INIT_DIRS + (ENACTMENT_DIRS if enactment else []): + d.mkdir(parents=True, exist_ok=True) + # `mode` is this pass's intake, not a permanent property of the tree: the modes compose, in + # any order and repeatedly across commits, and every artifact below is shared between them. + state = {"mode": args.mode, "scan_level": scan_level, "triage_level": args.triage_level, "language": args.language or prior.get("language")} + if findings: # kept across other modes, so a later enactment pass resumes it + state["findings"] = findings + spec = args.spec or prior.get("spec") + if spec: + state["spec"] = spec for k in STATE_DERIVED: # never clobber what build/scan already learned state[k] = prior.get(k) state_path.write_text(dump_yaml(state), encoding="utf-8") - # history: append one run entry on a fresh init, never on resume (the derived knobs survived) + # history: one entry per pass. Re-running init with the same knobs on the same commit is a + # resume of the current pass, not a new one — a different mode, level, or commit starts one. hist_path = TRACKING / "history.yaml" - runs = (load_yaml(hist_path, {}) or {}).get("runs") or [] - if not resume: - runs.append({"commit": git_head(), "type": f"{args.scan_level}/{args.triage_level}"}) + entry = {"commit": git_head(), + "type": f"{args.mode}/{scan_level}/{args.triage_level}"} + new_pass = not runs or runs[-1] != entry + if new_pass: + runs.append(entry) hist_path.write_text(dump_yaml({"runs": runs}), encoding="utf-8") - mode = "resumed (derived knobs preserved)" if resume else "fresh" - print(f"init {mode}: scan_level={state['scan_level']} triage_level={state['triage_level']} " - f"language={state['language']}") - print(f"seeded {len(INIT_DIRS)} directories under .opentaint/") + prior_mode = prior.get("mode") + if not prior: + how = "fresh tree" + elif prior_mode and prior_mode != args.mode: + how = f"new {args.mode} pass over the existing {prior_mode} tree" + elif new_pass: + how = f"new {args.mode} pass (pass {len(runs)})" + else: + how = "resumed (derived knobs preserved)" + print(f"init {how}: mode={args.mode} scan_level={scan_level} " + f"triage_level={state['triage_level']} language={state['language']}") + if findings: + print(f"findings={findings}") + if spec: + print(f"spec={spec}") + if prior: + kept = ", ".join(f"{c} {label}{'' if c == 1 else 's'}" for label, c in carried_over() if c) + print(f"carried over: {kept or 'nothing yet'}") + print(f"seeded {len(INIT_DIRS) + (len(ENACTMENT_DIRS) if enactment else 0)} " + "directories under .opentaint/") print("next: uv run scripts/get_status.py --full") return 0 @@ -163,7 +227,7 @@ def write_plans(plans, out_dir, prefix_id): if isinstance(x, dict) else x) for p, v in sorted(scopes.items())} path = out_dir / f"{pid}.yaml" - # source: null is the unprocessed sentinel — a discover agent overwrites it with the + # source: null is the unprocessed sentinel — a frontier agent overwrites it with the # list of sources it found (an empty list when it finds none). mark-safe folds only # plans whose sentinel was replaced, so an un-returned plan is never marked safe. path.write_text(dump_yaml({"id": pid, "scopes": norm, "source": None}), encoding="utf-8") @@ -227,7 +291,7 @@ def cmd_analyze(args): return 0 -# ---- partition discover ---- +# ---- partition frontier ---- def yaml_modules(model_yaml): # each module in project.yaml as (packages, moduleClasses); only `packages` says which of a @@ -299,8 +363,8 @@ def pending_packages(): return tuple(p for p in (cov.get("packages") or []) if isinstance(p, str) and p) -def cmd_discover(args): - regen_plans(DISCOVER_PLANS) +def cmd_frontier(args): + regen_plans(FRONTIER_PLANS) packages = pending_packages() if not packages: print("nothing to plan — no pending package in coverage.yaml", file=sys.stderr) @@ -314,26 +378,26 @@ def cmd_discover(args): if not todo: print("nothing to plan — every used member already verdicted", file=sys.stderr) return 0 - cap = DISCOVER_TARGET + DISCOVER_BAND - plans = pack(atomize(todo, cap), DISCOVER_TARGET, cap) + cap = FRONTIER_TARGET + FRONTIER_BAND + plans = pack(atomize(todo, cap), FRONTIER_TARGET, cap) rows = {f: [{"method": f, "signature": s} for s in sorted(sigs[f])] for f in sigs} plans = [{pkg: [r for f in members for r in rows[f]] for pkg, members in plan.items()} for plan in plans] - for p in write_plans(plans, DISCOVER_PLANS, "lib"): + for p in write_plans(plans, FRONTIER_PLANS, "lib"): print(p) return 0 def cmd_partition(args): - return cmd_analyze(args) if args.kind == "analyze" else cmd_discover(args) + return cmd_analyze(args) if args.kind == "analyze" else cmd_frontier(args) -# ---- mark-safe (discover join) ---- +# ---- mark-safe (frontier join) ---- def cmd_mark_safe(args): - plans = sorted(glob.glob(str(DISCOVER_PLANS / "lib-*.yaml"))) + plans = sorted(glob.glob(str(FRONTIER_PLANS / "lib-*.yaml"))) if not plans: - print("no discover plans to reconcile", file=sys.stderr) + print("no frontier plans to reconcile", file=sys.stderr) return 0 ledger = RULES_TR / "classification.yaml" doc = load_yaml(ledger, {}) or {} @@ -343,7 +407,7 @@ def cmd_mark_safe(args): for p in plans: pdoc = load_yaml(p, {}) or {} raw = pdoc.get("source") - if raw is None: # sentinel intact — no discover agent returned for it + if raw is None: # sentinel intact — no frontier agent returned for it unprocessed.append(p) continue members = {member_key(m) for v in (pdoc.get("scopes") or {}).values() for m in v} @@ -353,7 +417,7 @@ def cmd_mark_safe(args): processed.append(p) print(f"{Path(p).name}: {len(srcs)} sources, {len(members - srcs)} safe") if not processed: - print("no processed discover plans (every plan still carries source: null) — " + print("no processed frontier plans (every plan still carries source: null) — " "fan out discover-attack-surface first", file=sys.stderr) return 0 safe -= source @@ -363,7 +427,7 @@ def cmd_mark_safe(args): if not args.keep: for p in processed: Path(p).unlink() - print(f"pruned {len(processed)} reconciled discover plan(s)") + print(f"pruned {len(processed)} reconciled frontier plan(s)") if unprocessed: print(f"left {len(unprocessed)} unprocessed plan(s) (source: null) for re-dispatch: " + ", ".join(Path(p).name for p in unprocessed)) @@ -420,10 +484,8 @@ def cmd_merge_skipped(args): _FP_PREFERENCE = ("vulnerabilitySourceSinkHash", "vulnerabilityWithTraceHash") -RULE_RE = re.compile(r'^rule_id:\s*(.+?)\s*$', re.M) HASHES_RE = re.compile(r'^sarif_hashes:\s*\[(.*)\]\s*$', re.M) HASHES_BLOCK_RE = re.compile(r'^sarif_hashes:\s*\n((?:[ \t]+-[^\n]*\n?)+)', re.M) -VERDICT_RE = re.compile(r'^verdict:\s*(.+?)\s*$', re.M) def docker_name(seed, taken): @@ -561,17 +623,27 @@ def main(): sub = ap.add_subparsers(dest="cmd", required=True) i = sub.add_parser("init", help="bootstrap the .opentaint tree + state.yaml from workflow flags") - i.add_argument("--scan-level", required=True, choices=["lite", "normal", "deep"]) + i.add_argument("--mode", required=True, choices=["onboarding", "discovery", "enactment"], + help="onboarding: sweep the external-method frontier once. discovery: work " + "from the project, a diff, or an informal spec. enactment: reproduce a " + "supplied finding set") + i.add_argument("--scan-level", choices=["lite", "normal", "deep"], + help="discovery mode only; onboarding and enactment are always deep") i.add_argument("--triage-level", required=True, choices=["static", "dynamic"]) i.add_argument("--language", default=None, help="target language, determined by the orchestrator") + i.add_argument("--findings", default=None, + help="enactment mode: path to the supplied finding manifest/report/directory") + i.add_argument("--spec", default=None, + help="discovery mode: path to the diff, spec, or note that scopes the pass; " + "omit to scope the pass to the whole project") i.set_defaults(func=cmd_init) p = sub.add_parser("partition", help="split classification work into per-agent plans") - p.add_argument("kind", choices=["analyze", "discover"]) + p.add_argument("kind", choices=["analyze", "frontier"]) p.set_defaults(func=cmd_partition) - m = sub.add_parser("mark-safe", help="merge discover plans into classification.yaml") - m.add_argument("--keep", action="store_true", help="keep the reconciled discover plans") + m = sub.add_parser("mark-safe", help="merge frontier plans into classification.yaml") + m.add_argument("--keep", action="store_true", help="keep the reconciled frontier plans") m.set_defaults(func=cmd_mark_safe) s = sub.add_parser("merge-skipped", help="rebuild approximations/skipped.yaml from batches") @@ -581,6 +653,7 @@ def main(): f = sub.add_parser("findings", help="seed per-rule finding files from results/report.sarif") f.set_defaults(func=cmd_findings) + args = ap.parse_args() return args.func(args) diff --git a/skills/orchestrate-stage/scripts/get_status.py b/skills/orchestrate-stage/scripts/get_status.py index aca3b06b6..e8411bb26 100644 --- a/skills/orchestrate-stage/scripts/get_status.py +++ b/skills/orchestrate-stage/scripts/get_status.py @@ -22,19 +22,21 @@ import sys from pathlib import Path -from _common import (APPROX, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, - PASS_THROUGH, ROOT, RULES, RULES_TR, SARIF, SINKS_TR, - SOURCES_TR, TRACKING, build_done_keys, classified_keys, - dropped_entries, git_head, load_yaml, member_key, - modeled_entries, skipped_keys) +from _common import (APPROX, BOUNDARIES_TR, DATAFLOW, FINDINGS_TR, JOINS_TR, MODEL, + PASS_THROUGH, REFERENCE_TR, ROOT, RULES, RULES_TR, SARIF, SCOPE, SINKS_TR, + SOURCES_TR, TRACKING, build_done_keys, classified_keys, dropped_entries, + git_head, load_yaml, member_key, modeled_entries, scope_families, + skipped_keys, strip_quotes) STATE = load_yaml(TRACKING / "state.yaml", {}) or {} +MODE = STATE.get("mode") or "discovery" SCAN_LEVEL = STATE.get("scan_level") TRIAGE_LEVEL = STATE.get("triage_level") -DISCOVER_PLANS = RULES_TR / "plans" +FRONTIER_PLANS = RULES_TR / "plans" APPROX_PLANS = APPROX / "plans" VULN = ROOT / "vulnerabilities.md" +ENACTMENT = ROOT / "enactment.md" GLOBAL_CAP = 10 @@ -49,6 +51,12 @@ def load_units(d): if Path(d).is_dir() else [] +def load_docs(d): + """(path, doc) for every tracking file in a directory — reference, boundary, control.""" + return [(p, load_yaml(p, {}) or {}) for p in sorted(Path(d).glob("*.yaml"))] \ + if Path(d).is_dir() else [] + + def load_joins(): return [(p.stem, load_yaml(p, {}) or {}) for p in sorted(JOINS_TR.glob("*.yaml"))] \ if JOINS_TR.is_dir() else [] @@ -157,10 +165,13 @@ def ph_build(): "dispatch build-project"], None -def ph_discover(): +def ph_frontier(): + """Onboarding's intake: the external-method frontier, swept as one trust boundary. Every + dependency member the project's own code calls is a candidate boundary until a leaf verdicts + it, and the ledger is what makes that verdict durable for every later pass.""" if not (TRACKING / "coverage.yaml").is_file(): return False, ["dispatch triage-dependencies"], None - leftover = sorted(glob.glob(str(DISCOVER_PLANS / "*.yaml"))) + leftover = sorted(glob.glob(str(FRONTIER_PLANS / "*.yaml"))) units = load_units(SOURCES_TR) ledger = load_yaml(RULES_TR / "classification.yaml", {}) or {} if leftover: @@ -169,7 +180,7 @@ def ph_discover(): tasks.append("then run `scripts/generate.py mark-safe` to reconcile the plans") return False, tasks, None if not ledger and not units: - return False, ["run `scripts/generate.py partition discover` to plan the used members"], None + return False, ["run `scripts/generate.py partition frontier` to plan the called members"], None return True, [], None @@ -280,18 +291,141 @@ def ph_poc(): return True, [], None +# ---- intake: one contract, three mode-specific derivations ---- + +def ph_reference_set(): + """Enactment's intake: the supplied findings, normalized and grouped. The `family` field on + each reference file is the assignment — it moves with the finding when a family splits, so + the families are read back from the files rather than from a separate list.""" + docs = load_docs(REFERENCE_TR) + if not docs: + src = STATE.get("findings") or "state.yaml findings unset" + return False, [f"normalize the supplied findings ({src}) into " + ".opentaint/tracking/reference/.yaml"], None + missing = sorted(p.stem for p, d in docs if not strip_quotes(d.get("family", ""))) + if missing: + return False, ["reference findings not assigned to a boundary family:"] \ + + [f" {m}" for m in missing], None + return True, [], None + + +def ph_scope(): + """Onboarding's and discovery's intake join: the families their evidence groups into, written + to scope.yaml. Enactment carries the same information on its reference files instead.""" + fams = scope_families() + if not fams: + src = STATE.get("spec") or "the whole project" + what = ("the frontier the sweep verdicted as sources and effects" if MODE == "onboarding" + else f"the code {src} names") + return False, [f"group {what} into families and write {SCOPE}"], None + empty = [n for n, ev in fams if not ev] + if empty: + return False, ["families in scope.yaml with no evidence recorded:"] \ + + [f" {n}" for n in empty], None + return True, [], None + + +def ph_intake(): + if MODE == "enactment": + return ph_reference_set() + if MODE == "onboarding": + done, tasks, note = ph_frontier() + if not done: + return done, tasks, note + return ph_scope() + + +def families(): + """(family, evidence ids) for this pass, whichever mode scoped it — reference finding ids in + enactment mode, the members or areas intake recorded otherwise. The boundaries stage + generalizes one universal source and sink per entry.""" + if MODE == "enactment": + out = {} + for path, doc in load_docs(REFERENCE_TR): + fam = strip_quotes(doc.get("family", "")) + if fam: + out.setdefault(fam, []).append(path.stem) + return sorted(out.items()) + return scope_families() + + +def ph_boundaries(): + specs = {p.stem: d for p, d in load_docs(BOUNDARIES_TR)} + fams = families() + missing = [f for f, _ in fams if f not in specs] + if missing: + return False, ["dispatch discover-universal-boundaries, one per family:"] \ + + [f" {f}" for f in missing], None + # a split rewrites the family on the evidence it moves, so every spec here owns its evidence + unsaturated = [f for f, _ in fams + if str((specs[f].get("saturation") or {}).get("status", "")).strip() + != "saturated"] + if unsaturated: + return False, ["boundary specs not saturated:"] + [f" {f}" for f in unsaturated], None + unfactored = sorted(f"{f}: {e}" for f, evidence in fams for e in evidence + if e not in (specs[f].get("factorization") or {})) + if unfactored: + return False, ["evidence with no factorization in its spec:"] \ + + [f" {r}" for r in unfactored], None + unseeded = [f for f, _ in fams if (specs[f].get("stages") or {}).get("units_seeded") != "done"] + if unseeded: + return False, ["seed the source and sink units from these specs' candidate_patterns:"] \ + + [f" {f}" for f in unseeded], None + return True, [], None + + +def ph_crossref(): + if not SARIF.is_file(): + return False, ["dispatch run-scan"], None + docs = load_docs(REFERENCE_TR) + scanned = SARIF.stat().st_mtime + pend = [p for p, d in docs + if str(d.get("crossref", "pending")).strip() != "done" or p.stat().st_mtime < scanned] + if pend: + return False, [f"cross-reference the scan against {len(pend)} reference finding(s):"] \ + + [f" {p}" for p in pend], None + blocked = sorted({str(m) for _, d in docs for m in (d.get("blocked_at") or [])}) + if blocked: + return False, ["expected traces stop at unmodeled carriers:"] + [f" {m}" for m in blocked] \ + + ["model them in an approximation round, rescan, then cross-reference again"], None + rep = sum(1 for _, d in docs if str(d.get("status", "")).strip() == "reproduced") + stale = newest_mtime([p for p, _ in docs]) > (ENACTMENT.stat().st_mtime + if ENACTMENT.is_file() else 0) + if not ENACTMENT.is_file() or stale: + return False, [f"rewrite .opentaint/enactment.md coverage manifest " + f"({rep}/{len(docs)} reproduced)"], None + return True, [], None + + +def has_reference_set(): + """A reference set outlives the pass that created it. Any later pass that rescans changes what + it reproduces, so the cross-reference stays in scope — otherwise an onboarding or discovery + pass would leave a coverage manifest that silently describes an older scan.""" + return REFERENCE_TR.is_dir() and any(REFERENCE_TR.glob("*.yaml")) + + +# One pipeline, whatever the mode brought to it. Intake and its universal boundaries differ — +# the swept frontier, the diff or spec, the supplied findings — but from the boundary specs on, +# every mode runs the same stages in the same order: sources before the scan that proves them, +# approximations against the frontier that scan names, sinks after it, then triage. Only the +# closing phases are conditional, on the triage level, the controls knob, and whether the tree +# carries a reference set to re-judge. PHASES = [ ("build", ph_build, lambda: True), - ("discover", ph_discover, lambda: SCAN_LEVEL == "deep"), + ("intake", ph_intake, lambda: SCAN_LEVEL == "deep"), + ("boundaries", ph_boundaries, lambda: SCAN_LEVEL == "deep"), ("source_rules", ph_source_rules, lambda: SCAN_LEVEL == "deep"), ("scan", ph_scan, lambda: True), ("approximations", ph_approximations, lambda: SCAN_LEVEL in ("normal", "deep")), ("sink_rules", ph_sink_rules, lambda: SCAN_LEVEL == "deep"), ("triage", ph_triage, lambda: True), ("poc", ph_poc, lambda: TRIAGE_LEVEL == "dynamic"), + ("crossref", ph_crossref, has_reference_set), ] + + # ---- caps ---- def free_gb(): @@ -329,8 +463,16 @@ def evaluate(): def cmd_full(): commit = short(STATE.get("model_commit")) or "none" - print(f"scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} language={STATE.get('language')} " - f"commit={commit} cap={GLOBAL_CAP} (heavy {heavy_cap()})") + print(f"mode={MODE} scan={SCAN_LEVEL} triage={TRIAGE_LEVEL} " + f"language={STATE.get('language')} commit={commit} " + f"cap={GLOBAL_CAP} (heavy {heavy_cap()})") + if STATE.get("findings"): # printed in every mode — the set stays tracked + print(f"findings={STATE.get('findings')}") + if STATE.get("spec"): + print(f"spec={STATE.get('spec')}") + runs = (load_yaml(TRACKING / "history.yaml", {}) or {}).get("runs") or [] + if len(runs) > 1: + print("passes: " + " -> ".join(str(r.get("type", "?")) for r in runs)) rows, current = evaluate() # a phase downstream of the current stage that vacuously satisfies its own check is not # actually done — its producing stage hasn't run — so it reads PENDING, never DONE. diff --git a/skills/triage-dependencies/SKILL.md b/skills/triage-dependencies/SKILL.md index ea8aab241..346a665f9 100644 --- a/skills/triage-dependencies/SKILL.md +++ b/skills/triage-dependencies/SKILL.md @@ -21,7 +21,7 @@ Provided by the caller, fall back to the default value when omitted. Ask back on ### 1. List the dependencies -Read `.opentaint/project/project.yaml` — the `dependencies` list under each per-language projects entry is every third-party dependency the model resolved. Resolve each to the library it is. Most of a large project's dependencies are transitive infrastructure +Read `.opentaint/project/project.yaml` — the `dependencies:` list under each per-language projects entry (e.g. `javaProjects:`) is every third-party dependency the model resolved. Resolve each to the library it is. Most of a large project's dependencies are transitive infrastructure ### 2. Mark each library