docs(evaluation): list the actual DETECTORS identifiers - #51
Open
joeymussalli wants to merge 1 commit into
Open
Conversation
EVALUATION.md names ten ecosystems in prose and then tells readers, forty lines later, that a scan can be narrowed with DETECTORS. It never says what to write. The root README gives three examples and an ellipsis -- `claude_sdk,openai_sdk,google_adk,...` -- so anyone wanting to restrict a scan to, say, LangGraph has no documented token and no way to find one short of reading the scanner's source. That gap has a sharp edge. An unrecognised value is forwarded to the scanner verbatim, so narrowing to a name that does not exist produces a scan with nothing to detect -- which reads as a clean repository rather than as a misconfiguration. This is the same trap EVALUATION.md already warns about under "An empty result is not a pass", arrived at from a different direction. Added the full identifier table next to the ecosystem list where a reader is already looking, plus two clarifications that are not obvious from the names: `langchain` covers LangChain *and* LangGraph, which is why ten ecosystems do not map to ten identifiers; and `openshell` is a cross-cutting detector for shell-invocation tools rather than an SDK. Identifiers cross-checked against two sources: the scanner's own documented --detectors list, and the SDK-named top-level directories of the trustabl-rules repository, which agree. Documentation only; no code changes.
joeymussalli
force-pushed
the
docs/detector-identifiers
branch
from
August 24, 2026 20:05
5887eef to
7e0083e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
EVALUATION.md names ten ecosystems in prose and then tells readers, forty lines later, that a scan can be narrowed with DETECTORS. It never says what to write. The root README gives three examples and an ellipsis --
claude_sdk,openai_sdk,google_adk,...-- so anyone wanting to restrict a scan to, say, LangGraph has no documented token and no way to find one short of reading the scanner's source.That gap has a sharp edge. An unrecognised value is forwarded to the scanner verbatim, so narrowing to a name that does not exist produces a scan with nothing to detect -- which reads as a clean repository rather than as a misconfiguration. This is the same trap EVALUATION.md already warns about under "An empty result is not a pass", arrived at from a different direction.
Added the full identifier table next to the ecosystem list where a reader is already looking, plus two clarifications that are not obvious from the names:
langchaincovers LangChain and LangGraph, which is why ten ecosystems do not map to ten identifiers; andopenshellis a cross-cutting detector for shell-invocation tools rather than an SDK.Identifiers cross-checked against two sources: the scanner's own documented --detectors list, and the SDK-named top-level directories of the trustabl-rules repository, which agree.
Documentation only; no code changes.