Skip to content

docs(evaluation): list the actual DETECTORS identifiers - #51

Open
joeymussalli wants to merge 1 commit into
trustabl:mainfrom
joeymussalli:docs/detector-identifiers
Open

docs(evaluation): list the actual DETECTORS identifiers#51
joeymussalli wants to merge 1 commit into
trustabl:mainfrom
joeymussalli:docs/detector-identifiers

Conversation

@joeymussalli

Copy link
Copy Markdown

EVALUATION.md names ten ecosystems in prose and then tells readers, forty lines later, that a scan can be narrowed with DETECTORS. It never says what to write. The root README gives three examples and an ellipsis -- claude_sdk,openai_sdk,google_adk,... -- so anyone wanting to restrict a scan to, say, LangGraph has no documented token and no way to find one short of reading the scanner's source.

That gap has a sharp edge. An unrecognised value is forwarded to the scanner verbatim, so narrowing to a name that does not exist produces a scan with nothing to detect -- which reads as a clean repository rather than as a misconfiguration. This is the same trap EVALUATION.md already warns about under "An empty result is not a pass", arrived at from a different direction.

Added the full identifier table next to the ecosystem list where a reader is already looking, plus two clarifications that are not obvious from the names: langchain covers LangChain and LangGraph, which is why ten ecosystems do not map to ten identifiers; and openshell is a cross-cutting detector for shell-invocation tools rather than an SDK.

Identifiers cross-checked against two sources: the scanner's own documented --detectors list, and the SDK-named top-level directories of the trustabl-rules repository, which agree.

Documentation only; no code changes.

EVALUATION.md names ten ecosystems in prose and then tells readers, forty lines
later, that a scan can be narrowed with DETECTORS. It never says what to write.
The root README gives three examples and an ellipsis --
`claude_sdk,openai_sdk,google_adk,...` -- so anyone wanting to restrict a scan
to, say, LangGraph has no documented token and no way to find one short of
reading the scanner's source.

That gap has a sharp edge. An unrecognised value is forwarded to the scanner
verbatim, so narrowing to a name that does not exist produces a scan with
nothing to detect -- which reads as a clean repository rather than as a
misconfiguration. This is the same trap EVALUATION.md already warns about under
"An empty result is not a pass", arrived at from a different direction.

Added the full identifier table next to the ecosystem list where a reader is
already looking, plus two clarifications that are not obvious from the names:
`langchain` covers LangChain *and* LangGraph, which is why ten ecosystems do not
map to ten identifiers; and `openshell` is a cross-cutting detector for
shell-invocation tools rather than an SDK.

Identifiers cross-checked against two sources: the scanner's own documented
--detectors list, and the SDK-named top-level directories of the trustabl-rules
repository, which agree.

Documentation only; no code changes.
@joeymussalli
joeymussalli force-pushed the docs/detector-identifiers branch from 5887eef to 7e0083e Compare August 24, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant