fix(ai-agent): complete the ATR category mapping and repair its target reference - #459
fix(ai-agent): complete the ATR category mapping and repair its target reference#459eeee2345 wants to merge 1 commit into
Conversation
…t reference Two defects in the artifact merged in gemaraproj#428, found while building the follow-on catalogs. The target reference does not resolve. `target-reference.reference-id` and the matching entry in `metadata.mapping-references` both say `ATR-AI-AGENT-CAP`, but the capability catalog's `metadata.id` is `ATR-AI-AGENT-CAPABILITY-CATALOG`. Per the `#MappingReference` doc comment, when `url` is absent the id is the referenced artifact's `metadata.id`, so as merged this mapping points at nothing. Both occurrences now use the real id. The mapping is a category short. ATR's `rules/` directory on main carries ten detection categories; this document maps nine. `model-security` was added upstream after gemaraproj#428 merged and never got an entry, so the artifact silently under-reports its own coverage. Added, with the honest caveat: only the behaviour-extraction rules in that category relate to a capability this catalog models, and the rationale says so rather than reaching for an adjacent one. The capabilities the rest of the category needs are a separate proposal. Also drops the word "nine" from the description. Not changed to "ten": a count in prose goes stale the next time upstream adds a category, which is exactly how this one became wrong. Verified: TestSchemaValidation passes, and every capability id referenced by the mapping now resolves against the capability catalog. Signed-off-by: Adam Lin <adam@agentthreatrule.org>
|
@jpower432 — the red X on this PR is not from this branch, and the detail is probably worth having. CI fails on The timeline points at the release rather than at any branch:
Those two are the only CI runs since the release, and both failed. The dependabot one is the useful data point: it bumps GitHub Actions dependencies and cannot touch a CUE schema, so the compatibility gate has been failing for anything that runs it since Reading it as a schema question rather than a test question: Nothing here needs anything from this PR; flagging it so the failure is not read as coming from these two lines of YAML. Happy to open a separate issue with the reproduction if that is more useful than a comment on an unrelated PR. |
|
Thanks @eeee2345. Seeing the failure on other PRs as well. I will get a fix up to unblock this. |
|
@eeee2345 A rebase should fix the failing CI |
2 participants
@jpower432 — two defects in the artifact from #428, found while building the follow-on we discussed there. Split out as a fix so it does not sit behind the design conversation on the catalogs.
The target reference does not resolve.
target-reference.reference-idand its entry inmetadata.mapping-referencesboth sayATR-AI-AGENT-CAP, but the capability catalog'smetadata.idisATR-AI-AGENT-CAPABILITY-CATALOG.#MappingReferencesays that whenurlis absent the id is the referenced artifact'smetadata.id, so as merged this document points at nothing. Both occurrences fixed.The mapping is a category short. ATR's
rules/on main has ten detection categories; this maps nine.model-securitywas added upstream after #428 merged and never got an entry, so the artifact under-reports its own coverage without saying so.Added, with the caveat stated rather than papered over: of that category's rules, only behaviour extraction relates to a capability this catalog models. The other two — poisoned fine-tuning data, and unsafe deserialization when a model artifact is loaded — are model supply chain, and there is no capability here for either. The rationale says that instead of reaching for an adjacent capability to make the row look complete. Proposing the missing capabilities is a separate change.
Also drops "nine" from the description. Not changed to "ten" — a count in prose goes stale the next time upstream adds a category, which is precisely how this one became wrong.
Verified:
go test -run TestSchemaValidationpasses, and every capability id the mapping references now resolves against the capability catalog.Unrelated, but worth knowing since you will see it locally:
TestNoBreakingChangescurrently fails on#EvaluationLogand#ControlEvaluationagainst released v1.5.0. It fails on a cleanmaintoo, so it is not from this branch.