[Initiative]: Evaluate AI-generated threat model drafts as a starting point for security self-assessments

[realshuting]

Name

Generated threat-model drafts for security self-assessments

Short description

Evaluate whether AI-generated threat model drafts (alpha-omega-security/threat-model) improve the quality and consistency of project security self-assessments, and whether to recommend the approach in the self-assessment guide.

Responsible group

TAG Security and Compliance

Does the initiative belong to a subproject?

Yes

Subproject name

security-assessments

Primary contact

@realshuting (Shuting Zhao)

Additional contacts

@JustinCappos, @Santoshkumarpuppala

Initiative description

Security self-assessments submitted by projects are uneven in quality. The TAG self-assessment template defines what sections to write, but not how to write them well, and projects without security expertise often struggle to fill it in.

The agent skills at alpha-omega-security/threat-model generate a draft threat model / security contract from a project's code and docs. Compared to writing from a blank template, the approach adds:

• Draft content generated from the code, so maintainers correct a draft instead of authoring from scratch
• Every claim tagged with an evidence level (documented / maintainer-confirmed / inferred), giving reviewers a ready-made list of what to challenge
• Coverage stated as a count (e.g. "38 of 97 entry points covered"), so uncovered areas are visible instead of silent
• Explicitly disclaimed guarantees listed as a first-class section, instead of simply being absent

Known gap: the tool does not identify, for each claimed security guarantee, the code that actually rejects a violating request. That still needs a maintainer who knows the codebase. Example: the authorization discussion on #2264 (Meshery self-assessment), where the assessment stated authorization "is enforced" but the enforcement code had to be located by the maintainers during review. Any recommendation of the tool should pair the generated draft with one line per guarantee from the maintainers: the code that refuses a violating request, and what the refusal looks like.

Scope:

1. Collect experience from recent assessment reviews (kcp, Meshery, Higress)
2. Trial the generated-draft approach with 1-2 projects that are starting or stalled on their self-assessment
3. Decide whether to recommend the tool in the self-assessment guide as an optional starting point
4. Independently of the tool, consider adding its quality properties (evidence tags, coverage count, enforcement point per guarantee) to the self-assessment guide and reviewer guidance

Timeline: evaluation over the next 2-3 months, aligned with in-flight assessment reviews.

Origin: Slack discussion.

Deliverable(s) or exit criteria

• Written summary of the trial results: what the generated drafts got right, and what reviewers/maintainers still had to add
• Recorded decision: recommend / do not recommend the tool in the self-assessment guide
• If recommended: PR to the self-assessment guide adding it as an optional starting point, including the required maintainer additions (enforcement point per guarantee)
• If the quality properties prove useful independently of the tool: PR updating the self-assessment guide and reviewer guidance with them

Tracking document for meeting and progress

https://notes.cncf.io/PxSKxLceQyKwMfbZJFdvRg?both

[Santoshkumarpuppala]

Happy to be on this. Having gone through the schema rather than just the output, I think the "known gap" above can be stated more precisely, and it changes where the fix belongs.

What the schema mandates. In schema.json, a claimed guarantee (properties_provided[]) requires property, violation_symptom, severity_tier and provenance. A disclaimed one (properties_not_provided[]) requires property, reason and a false_friend boolean — which is a genuinely good idea, since a property a reader would assume is upheld is more dangerous than one nobody expected. trust_boundaries[] takes an optional reachability_precondition.

What no field asks for. Nothing in the schema records, for a claimed property, the code that refuses a request violating it. violation_symptom is what a violation looks like; reachability_precondition is how you reach the boundary. The word "enforce" does not appear anywhere in references/principles.md.

And that looks deliberate rather than an oversight. The principles file says the model is "not a vulnerability assessment, audit, or pentest" and that it captures "the unwritten assumptions" rather than restating what the source already says. A threat model describes the contract; the enforcement point is an implementation fact. Asking the generator to emit it would push it toward auditing the code, which is the thing the design explicitly refuses. I don't think that should change.

So the gap is real but it is not the tool's to close — which argues for scope item 4 doing the work, and doing it whether or not the tool is recommended. Evidence tags, a coverage count and one enforcement line per claimed boundary are properties of a good self-assessment. The first two the generator gives you; the third is the layer the TAG guide adds on top.

Two data points that the enforcement line is the right thing to require, both from reviews in scope 1:

On Meshery, the assessment said authorization "is enforced in conjunction with a remote provider" — unfalsifiable as written. The replacement in the open PR names the enforcement point: per-operation in server/models/remote_provider.go, 91 gated operations, not the middleware chain. I checked that against meshery/meshery at master rather than take it on trust — ErrOperationNotAvaibale appears exactly 91 times. One line, verifiable in about a minute, where the previous sentence could not be checked at all.

On Higress, after the review on #2268 the maintainers named the enforcement point for the critical boundaries themselves and pushed it to their own repo. Same shape, no prompting needed once the question was asked.

One caution against my own suggestion: only ask for it where a boundary is actually claimed and load-bearing. On Meshery the authorization claim mattered because the server holds cluster-admin-equivalent RBAC, so that check is the only thing between an authenticated principal and any managed cluster. A guide that demands an enforcement line for every guarantee will get boilerplate for the ones nothing depends on.

What I can pick up: the scope-1 write-up across kcp, Meshery and Higress — what the generated draft would have covered, what the reviewer had to ask for, and what came back. Happy to take a scope-2 trial project too; remote and async, September through November.

[Santoshkumarpuppala]

Update, since this initiative is about that tool. I ran the deterministic tiers against open PR #13 (the one adding the authorization-scope dimension): 304 tests pass, golden model validates clean at 43 checks. Two findings bear on the case above.

The G2 coverage check joins the whole §1.7 table into one string and substring-matches one token per dimension. Delete all three authorization-scope rows from the golden zlib model, put the word "authorization" in an unrelated row's conditions, and it still returns OK: 43 passed, 0 error(s), 0 warning(s). Remove that one phrase and it fails correctly — so the check bites, it just cannot tell the dimension being present from the word being present.

Second, three of the four enum values PR #13 adds have no test coverage: removing "authorization", "credential" or "principal" leaves all 304 tests green, while removing "authorization-scope" fails 68. That last one is the control.

Both filed as alpha-omega-security/threat-model#8. Neither argues against recommending the approach — but "coverage stated as a count, so uncovered areas are visible instead of silent" is the line the first finding touches, and worth knowing before it goes into the guide.

Scope-1 write-up offer from my earlier comment still stands.