From 5275ee7679c1fab2b9beda6c113fc784a81c6fd5 Mon Sep 17 00:00:00 2001 From: bradAGI <46579244+bradAGI@users.noreply.github.com> Date: Mon, 24 Aug 2026 14:46:18 -0400 Subject: [PATCH] feat(langchain): add LC-021, LC-022 TypeScript description quality rules MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The TypeScript counterpart to LC-018/019. LC-010 only checks that a description exists, so a tool described "TODO: describe this tool." or "Gets data." passes today while giving the model no selection signal. LangChain.js has no docstring to fall back on, so the description field is the entire account of the tool the model sees, and the Zod schema does not compensate — it constrains the arguments once the model has chosen this tool, never whether choosing it was right. Mis-selection scales badly here because an agent is routinely handed a dozen or more tools at once, and each wrong pick spends an iteration against the maxIterations bound LC-111 checks for. --- langchain/tool_definition.yaml | 60 ++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/langchain/tool_definition.yaml b/langchain/tool_definition.yaml index 16cde34..6ca4b59 100644 --- a/langchain/tool_definition.yaml +++ b/langchain/tool_definition.yaml @@ -77,3 +77,63 @@ rules: Pass a one-sentence description in the tool() config (or the DynamicStructuredTool options) naming what the tool does, the inputs it expects, and what it returns. Write it for the model, not a human reader. + + - id: LC-021 + title: TypeScript LangChain tool description is a placeholder + severity: low + confidence: 0.85 + language: typescript + applies_to: + - langchain_tool + scope: tool + match: + has_description_text: + - todo + - tbd + - fixme + - placeholder + - no description + - does stuff + explanation: > + The tool sets a description, so it passes LC-010, but the string is a + placeholder rather than real content. LangChain.js has no docstring to + fall back on — the description field is the entire account of the tool the + model sees — so "TODO: describe this tool" leaves the model guessing from + the tool name. The Zod schema does not compensate: it constrains the shape + of the arguments once the model has chosen this tool, never whether + choosing it was right. Mis-selection scales badly here because a + LangChain agent is routinely handed a dozen or more tools at once, and + each wrong pick consumes an iteration against the executor's maxIterations + budget, so a run can exhaust its steps and return nothing useful. + fix: > + Replace the placeholder with a real description covering what the tool + does, what it returns, and when the model should call it rather than a + neighboring tool. The string is passed to the model verbatim, so write it + for the model rather than a human maintainer. + + - id: LC-022 + title: TypeScript LangChain tool description is too short to guide model selection + severity: low + confidence: 0.8 + language: typescript + applies_to: + - langchain_tool + scope: tool + match: + all: + - has_docstring: true + - description_length_lt: 40 + explanation: > + A description under 40 characters is rarely enough to convey what a tool + does, what it returns, and when to call it rather than a similarly named + neighbor. With no docstring fallback in LangChain.js, a stub like "Gets + data." is the whole prompt-side account of the tool, so scope and + preconditions are left to guesswork. The cost compounds in the executor + loop: each mis-selected call spends an iteration against the + maxIterations bound LC-111 checks for, so a thin description can burn the + run's step budget rather than the run producing an answer. + fix: > + Expand the description to at least a full sentence covering inputs, + outputs, and the situation in which this tool should be used over the + alternatives. Where two tools are easy to confuse, say in each which one + the other case belongs to.