Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion skills-templates/assemble-lib-rules/main.md.j2
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: assemble-lib-rules
description: Write the per-vuln-class security join rules that merge the created source/sink lib rules with the built-ins. Use to wire lib rules into project-level joins
description: Expand rule-family tags and write only the per-vulnerability security joins still needed to connect created source/sink library rules with the built-ins. Use to wire lib rules into project-level joins
license: Apache-2.0
metadata:
author: opentaint
Expand Down
1 change: 1 addition & 0 deletions skills-templates/assemble-lib-rules/sections/gotchas.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
- Ref the existing lib rules (built-in + created), never re-declare a source or sink
- A tag is an open family, not shorthand for one rule — use it only when every active same-language member should fan out through the join
- Keep produced joins comment-free
6 changes: 3 additions & 3 deletions skills-templates/assemble-lib-rules/sections/output.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
### Artifacts

- one join file per (vuln class, sink rule) under `.opentaint/rules/<lang>/security/<class>-<sink>-lib-ext.yaml`, each refing all relevant sources + its one sink
- `.opentaint/tracking/rules/joins/<class>.yaml` — one per vuln class, recording every join produced (per Tracking)
- zero or more extension join files under `.opentaint/rules/<lang>/security/`, only for concrete source-to-sink combinations not already covered through an existing rule/tag join
- `.opentaint/tracking/rules/joins/<class>.yaml` — one per vuln class, recording the concrete components covered by existing and created joins (per Tracking)

### Summary

- one line per join: class, sink, source count, and which ends are new
- one line per vulnerability class: concrete source/sink counts, reused tag-expanded joins, and any extension join created
2 changes: 1 addition & 1 deletion skills-templates/assemble-lib-rules/sections/preamble.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Skill: Assemble Lib Rules

Source and sink library rules are authored per package but never paired across them. Write the security joins that pair them — one per vuln class, each merging the created source/sink rules with the built-ins, mirroring the built-in security rules. The joins carry no test project, the main scan verifies them.
Source and sink library rules expose open role families through tags. Related custom sources may share a project-specific tag, and a custom join may consume that family with one `tag:` ref. Expand the existing built-in and custom joins first: a created rule that reused a consumed family tag may already be wired without another file. Add only the security joins needed for uncovered source-to-sink combinations, using tags for deliberate family expansion and exact rule refs for isolated components. The joins carry no test project; the main scan verifies them.
2 changes: 1 addition & 1 deletion skills-templates/assemble-lib-rules/sections/tracking.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
This skill writes the joins tracking, one file per vuln class, setting each file's `stages.written: done`. The main scan verifies the joins, don't touch `verified`.
This skill writes the joins tracking, one file per vuln class, setting each file's `stages.written: done`. Record concrete source/sink refs after expanding tags so deterministic status checks don't need to reimplement rule loading. The main scan verifies the joins; don't touch `verified`.

{% include "shared/tracking/joins.md" %}
46 changes: 20 additions & 26 deletions skills-templates/assemble-lib-rules/sections/workflow.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,29 @@
### 1. Read the created lib rules and the built-ins
### 1. Inventory components and expand existing joins

Read every source unit under `.opentaint/tracking/rules/sources/` and sink unit under `.opentaint/tracking/rules/sinks/` — the `rule_id`s already recorded, the sinks carrying their `vuln_class` — and the built-in source/sink lib rules:
Read every source unit under `.opentaint/tracking/rules/sources/` and sink unit under `.opentaint/tracking/rules/sinks/` — the concrete `rule_id`s already recorded and each sink's `vuln_class` — plus every created library/security rule. Locate and read the built-in library/security rules through:

```bash
opentaint health --rules
```

Collect every source rule (built-in + created) and every sink rule grouped by vuln class. A rule is built-in when its ref resolves in the loaded built-in ruleset, created otherwise — that membership, not a stored tag, tells the two apart.
For each source and sink component, record its concrete ref, tags, exposed metavariable, language, and relevant vulnerability class. A `tag:` ref is a language-scoped union of every active rule carrying that exact tag, whether the rules are built-in or custom; an explicit `rule:` ref selects only one rule. Several custom sources may therefore share one project-specific tag and be consumed together by a custom join. Expand both forms in every existing security join and enumerate the concrete source-to-sink pairs its `on` clauses cover. A created rule that reused a tag already consumed by a built-in or custom join is wired by that join automatically.

When join files already exist from a prior run, reuse them as the baseline: re-assemble to fold in any new lib rule — a new source widens the `on` of every join for its vuln class, a new sink adds a join — and leave a join whose wiring no new rule touches as-is. Don't rewrite joins that already hold.
When custom join files already exist, use them as part of this baseline. Don't rewrite a join whose expansion already covers the intended pairs.

### 2. Write one security join per (vuln class, sink rule)
### 2. Add only uncovered combinations

A join references exactly ONE sink rule — several sinks can't merge into one join. So a vuln class with more than one relevant sink becomes several joins: one per sink rule, each refing all the relevant sources on the left. Sources are many, the sink is always one.
For each vulnerability class, compare all relevant source × sink combinations with the expanded pairs from step 1. Write a new `mode: join` rule only for combinations still uncovered:

For each vuln class, and within it each sink rule that needs new wiring, write a join under `.opentaint/rules/<lang>/security/<class>-<sink>-lib-ext.yaml` with `mode: join`, refing the relevant sources + that one sink, wiring only the new-end combinations in `on`:
- Prefer `tag:` when every active member of that role family should participate. Tagged source and sink sides expand as a Cartesian product, so one join can deliberately cover several components.
- Use an exact `rule:` ref when tag expansion would pull in an unrelated component or duplicate a pair already covered elsewhere.
- A ref contains exactly one of `tag` or `rule`, every `as` alias is unique, and a join cannot reference another join.
- Read the component before writing `on`: custom components expose `$UNTRUSTED`, while a built-in may expose another name.

- a created sink ← from every relevant source (built-in + created)
- a built-in sink ← from created sources only (a built-in source → built-in sink pair is already covered by the built-in join)

Two rules here:

- Unique id — use `id: <class>-<sink>-lib-ext`, never the bare class name; a custom join named `ssrf`/`xxe`/`path-traversal` collides silently with the built-in join of that id and is dropped with no error (only the scan's rule statistics reveal it)
- Right metavariable each side — the source side is always `$UNTRUSTED`. The sink side is `$UNTRUSTED` for a custom rules, but a built-in sink may bind another name — read it from how that sink is wired in the built-in security rules and use it: `source.$UNTRUSTED -> sink.$<its-metavar>`
Use a unique extension id such as `<class>-lib-ext`; never reuse the built-in class id because duplicate ids are dropped. Preserve the built-in rule's user-facing metadata for that vulnerability class.

```yaml
rules:
- id: ssrf-webclient-ssrf-sink-lib-ext
- id: ssrf-lib-ext
severity: ERROR
message: Untrusted data reaches an SSRF sink
metadata:
Expand All @@ -36,21 +33,18 @@ rules:
mode: join
join:
refs:
- rule: java/lib/generic/servlet-untrusted-data-source.yaml#java-servlet-untrusted-data-source
as: servlet-source
- rule: java/lib/spring/webflux-request-source.yaml#webflux-request-source
as: webflux-source
- rule: java/lib/spring/webclient-ssrf-sink.yaml#webclient-ssrf-sink
- tag: orders-untrusted-data-source
as: orders-source
- tag: ssrf-sink
as: sink
on:
- 'servlet-source.$UNTRUSTED -> sink.$UNTRUSTED'
- 'webflux-source.$UNTRUSTED -> sink.$UNTRUSTED'
- 'orders-source.$UNTRUSTED -> sink.$UNTRUSTED'
```

The same class's built-in sink is a second file (e.g. `ssrf-java-ssrf-sink-lib-ext.yaml`), refing only the created sources → that built-in sink.
This example intentionally adds every custom source whose `tags` list contains `orders-untrusted-data-source` to every active Java SSRF sink carrying `ssrf-sink`. If only one source should connect, replace its `tag:` ref with that component's exact `rule:` ref.

### 3. Verify every rule is wired, then stop
### 3. Record concrete coverage and verify

With the joins written, confirm no orphan before returning — a source or sink not merged into a join. Cross-check the `rule_id`s in the source and sink units against the joins: every created source rule must appear as a source end in the `on` of at least one join, and every created sink rule must be the sink of a join, and each created sink's join must ref all its relevant sources (built-in + created) on the left. Add a join (per step 2) for anything still unwired.
Update each class's joins tracking with concrete rule refs, even when the covering join uses tags: list every created source covered under `sources`, and one `joins` entry per concrete sink with the `rule_id` of the existing or newly created security join that covers it. Several sink entries may name the same tag-expanded join. Don't create a redundant join file merely to produce tracking state.

Set `stages.written: done` (per Tracking) and return per Output.
Expand all joins again and confirm there is no orphan or missing pair: every created source reaches every relevant sink class, every created sink is reachable from all relevant sources, and no pair is represented twice. Set `stages.written: done` (per Tracking) and return per Output.
33 changes: 32 additions & 1 deletion skills-templates/create-rule/references/java.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ Built-in source/sink lib rules live under `java/lib/generic/` (framework-neutral

### 2. Author the library rules

Every custom library source or sink declares its role family under top-level `tags`. Tags are exact, language-scoped strings used only to expand `join.refs`; they don't change matching or activate a library rule. Follow the built-in vocabulary: source families use `<framework>-untrusted-data-source`, and sink families use the security rule's exact `<vulnerability>-sink` tag (for example `servlet-untrusted-data-source`, `spring-untrusted-data-source`, `sqli-sink`, `ssrf-sink`, or `path-traversal-sink`). Read the built-in library and security rules before choosing.

A tag is an open extension point: adding `sqli-sink` opts the rule into every Java join that references that tag. Reuse it only when that fan-out is correct. Otherwise use a stable project-specific family such as `<unit>-untrusted-data-source` or `<unit>-<vulnerability>-sink`; the assembly stage will add the required join. Multiple custom sources can declare the same project-specific source tag, and a custom join can select the whole family with one `tag:` ref. Use an explicit `rule:` ref when only one component should be selected.

Reference a built-in:

```yaml
Expand All @@ -27,6 +31,8 @@ rules:
- id: my-custom-source
options:
lib: true
tags:
- servlet-untrusted-data-source
severity: NOTE
message: Custom untrusted data source
languages: [java]
Expand All @@ -42,13 +48,37 @@ rules:
- pattern: doPost
```

To make several custom sources a reusable family, repeat the same exact list-form tag on every source rule:

```yaml
tags:
- orders-untrusted-data-source
```

A custom security join can then consume all active Java source rules in that family:

```yaml
join:
refs:
- tag: orders-untrusted-data-source
as: orders-source
- tag: sqli-sink
as: sink
on:
- 'orders-source.$UNTRUSTED -> sink.$UNTRUSTED'
```

Keep the shared tag project-specific unless these sources are intentionally members of a built-in family. Tag expansion is language-scoped and includes every active matching rule, including other custom rules.

Custom sink library rule (`.opentaint/rules/java/lib/generic/my-sink.yaml`):

```yaml
rules:
- id: my-custom-sink
options:
lib: true
tags:
- sqli-sink
severity: NOTE
message: Custom dangerous operation
languages: [java]
Expand Down Expand Up @@ -90,7 +120,7 @@ rules:
- 'src.$UNTRUSTED -> sink.$UNTRUSTED'
```

The marker rules resolve from the sub-project's `test-rules` root, your lib rules from `.opentaint/rules` — `test rule run` is passed both. Metavariable names must match across `refs` and `on`.
The marker rules resolve from the sub-project's `test-rules` root, your lib rules from `.opentaint/rules` — `test rule run` is passed both. Metavariable names must match across `refs` and `on`. Keep these test refs explicit: replacing `rule:` with `tag:` would pull every active family member into the test and stop it isolating the new component.

### 4. Test until success

Expand All @@ -103,6 +133,7 @@ The concrete operators per verdict:
## Constraints

- Custom library rules go under `java/lib/generic/` or `java/lib/spring/`, mirroring the built-in layout — never directly under `java/lib/`
- Tags are exact strings, not display labels. Match the built-in security join's spelling, including distinctions such as `sqli-sink` versus the vulnerability class `sql-injection`
- For a simple structural pattern (no dataflow), omit `mode` — it uses the default mode
- Don't unpack or grep the analyzer JAR

Expand Down
1 change: 1 addition & 0 deletions skills-templates/create-rule/sections/constraints.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{% include "shared/engine/facts.md" %}

- Library rules MUST have `options.lib: true` and `severity: NOTE`
- Every custom source/sink library rule MUST declare at least one deliberate role-family `tags` value. Reuse an existing tag only when open expansion into all of that tag's consumers is intended
- The test joins (`mode: join`) MUST have `metadata.cwe` and `metadata.short-description`
- Metavariable names must match across `refs` and `on` clauses, or the join won't connect. Bind the tainted value to one consistent metavariable in every lib source/sink rule
- The `rule` path in `refs` is relative to its ruleset root — a marker ref resolves under the test project's marker rules, a lib ref under the scanned rules tree
Expand Down
2 changes: 1 addition & 1 deletion skills-templates/create-rule/sections/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

### Summary

- the lib rule ids (created or referenced), a one-line test summary for the side, and the exact `test rule run` command used
- the lib rule ids (created or referenced), each rule's family tags, a one-line test summary for the side, and the exact `test rule run` command used
- if blocked at step 5: `stages.tests_passing` left pending, and the cause —
- a dropped library method on the failing sample's flow → the methods that need a model, to be approximated before a re-dispatch
- nothing dropped and no clear rule cause → non-convergence, for the orchestrator to intervene
6 changes: 4 additions & 2 deletions skills-templates/create-rule/sections/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ In fix mode, don't author from the unit: go straight to that one flagged rule, a

### 2. Author the library rules

Derive each rule's pattern from the unit's fully-qualified names, recorded signatures, and annotations. Bind the tainted value to one consistent metavariable in every rule so the security joins assembled later reference one name. The rule forms — a built-in `refs`, a custom source rule, a custom sink rule, and where custom rules go — are in the language reference.
Derive each rule's pattern from the unit's fully-qualified names, recorded signatures, and annotations. Bind the tainted value to one consistent metavariable in every rule so the security joins assembled later reference one name.

Give every custom library rule its source/sink family `tags`. Reuse a built-in role tag only when this rule should intentionally extend every join that consumes that family; otherwise create a stable project-specific role tag. Give related custom sources the same project-specific tag when custom joins should consume them as one family. Tags don't alter matching, and a fix must preserve them unless the rule's family membership is itself wrong. The rule forms, tag selection, and where custom rules go are in the language reference.

### 3. Write the test joins

A library rule emits nothing on its own — to exercise it, wire it to the generic taint marker in a throwaway test join. Write one join for the side into the test project's marker rules, referencing the generic marker on one end and each new lib rule on the other, so a positive sample's tainted value flows marker-to-rule (a sink side) or rule-to-marker (a source side). These joins live only in the test project, never in the scanned rules tree, so the main scan never loads them. The join form, its naming, and where it goes are in the language reference.
A library rule emits nothing on its own — to exercise it, wire it to the generic taint marker in a throwaway test join. Write one join for the side into the test project's marker rules, referencing the generic marker on one end and each new lib rule on the other, so a positive sample's tainted value flows marker-to-rule (a sink side) or rule-to-marker (a source side). Use exact `rule:` refs here, not the library rule's open family tag, so the test isolates only the component under test. These joins live only in the test project, never in the scanned rules tree, so the main scan never loads them. The join form, its naming, and where it goes are in the language reference.

### 4. Test until success

Expand Down
4 changes: 2 additions & 2 deletions skills-templates/orchestrate-stage/references/sinks.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Inputs:
- `unit`
- `fix-target` (optional) — only the scan-flagged rule correction explicitly assigned by the task

Expect back — each sink's `rule_id` set and `stages.tests_passing: done`.
Expect back — each sink's `rule_id` set, every custom rule carrying its deliberate sink-family tag, and `stages.tests_passing: done`.

## Assemble the full joins

Expand All @@ -32,7 +32,7 @@ Once no sink unit is pending, status names unwired sink rules. Dispatch assemble
Inputs:
- `language`

Expect back — the created-sink joins added to the joins tally, each refing all relevant sources. Then delete the sink units' `test-compiled/` models.
Expect back — existing tag-expanded joins reused where they already cover the created sinks, extension joins added only for uncovered combinations, and the concrete coverage recorded in the joins tally. Then delete the sink units' `test-compiled/` models.

## Stage gate

Expand Down
4 changes: 2 additions & 2 deletions skills-templates/orchestrate-stage/references/sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Inputs:
- `unit`
- `fix-target` (optional) — only the scan-flagged rule correction explicitly assigned by the task

Expect back — each source's `rule_id` set and `stages.tests_passing: done`.
Expect back — each source's `rule_id` set, every custom rule carrying its deliberate source-family tag, and `stages.tests_passing: done`.

## Assemble source joins

Expand All @@ -62,7 +62,7 @@ Once no source unit is pending, status names unwired created sources. Dispatch a
Inputs:
- `language`

Expect back — the joins tally written: one join per built-in sink, each refing all created sources for that vulnerability class. Then delete the source units' `test-compiled/` models.
Expect back — existing tag-expanded joins reused where they already cover the created sources, extension joins added only for uncovered combinations, and the concrete coverage recorded in the joins tally. Then delete the source units' `test-compiled/` models.

## Stage gate

Expand Down
Loading
Loading