feat(openai_sdk): add OAI-027, OAI-028 TypeScript description quality rules - #92
Open
bradAGI wants to merge 1 commit into
Open
feat(openai_sdk): add OAI-027, OAI-028 TypeScript description quality rules#92bradAGI wants to merge 1 commit into
bradAGI wants to merge 1 commit into
Conversation
… rules The TypeScript counterpart to OAI-025/026. OAI-022 only checks that a description exists, so a tool described "TODO: describe this tool." or "Gets data." passes today while leaving the tool in exactly the state OAI-022 exists to prevent. The Zod parameters schema does not compensate: it constrains the shape of the arguments once the model has decided to call this tool, never whether that decision was right, so a placeholder yields a well-formed call to the wrong tool. With handoffs the reach is wider — an agent picks between its own tools and its peers' from these strings, so the stub can route the conversation to the wrong agent rather than merely the wrong function.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The TypeScript counterpart to OAI-025/026 (#87). OAI-022 only checks that a description exists, so a tool described
"TODO: describe this tool."or"Gets data."passes today — while sitting in exactly the state OAI-022 exists to prevent. OAI-022's own text calls the description "the model's primary routing signal alongside the tool name," which is precisely why a placeholder in that slot matters.Two points, both aimed at mitigations people assume already cover this:
parametersschema doesn't compensate. It constrains the shape of the arguments once the model has decided to call this tool, never whether that decision was right. A placeholder description yields a well-formed call to the wrong tool — validation passes, the run is still wrong.OAI-028 adds the cost that's hardest to trace: each mis-selection spends a turn against the agent's max-turns budget, and nothing in a trace attributes the wasted turn to the description that caused it — the run just looks slow or confused.
Verification — engine built at
main:Fire (one
tool({...})described"TODO: describe this tool.", one"Gets data."):OAI-027, OAI-028, OAI-202Silent (full description naming when to prefer the neighboring tool):
OAI-202(OAI-202 is the pre-existing missing-CLAUDE.md repo rule.)
OAI-028 pairs
description_length_lt: 40withhas_docstring: trueso it doesn't double-report against OAI-022 on an empty description, same as CSDK-018.No new predicates, so no
schema_versionbump.Merge note: this and #87 both append to
openai_sdk/tool_definition.yaml, so whichever lands second needs a trivial rebase — the rule blocks are independent and the resolution is "keep both." Happy to rebase on request.