Skip to content

Proper naming of TS functions could potentially save a lot of wasted tokens #7

@genesiscz

Description

@genesiscz

Proper naming of TS functions could potentially save a lot of wasted tokens

Summary

The toolscript tool lists available servers and their tools using snake_case naming conventions (e.g., azure-devops, search_workitem), but the actual TypeScript API uses camelCase names (e.g., azureDevops, searchWorkitem). This naming mismatch causes confusing TypeScript type errors and requires multiple failed attempts to discover the correct function names.

Also, the hook suggestion uses {server}__{tool} in snake_case naming convention, which is confusing and not helpful.

Consider using Skill("toolscript") to access these MCP tools:
azure-devops__search_code, azure-devops__search_wiki, azure-devops__search_workitem

TLDR:

  • LLM has to do a lot of guessing in order to correctly call a tool
  • Would be the best if we wasted a bit of tokens on the start in order to make sure the calls are correct on the first try
  • It would even probably make sense to create more alias of the same tool in order to let the LLM hit the correct name most of the time (search_workitem, searchWorkItem)

Problem Description

When using the toolscript tool to search Azure DevOps work items, the following sequence occurred:

What the tool listed:

Server: azure-devops
Tool: search_workitem

What actually works in TypeScript:

tools.azureDevops.searchWorkitem({ searchText: "..." });

Failed Attempts

  1. First attempt - Used the listed naming exactly:
 tools.azure_devops__search_workitem({...})
 // ❌ Error: Property 'azure_devops__search_workitem' does not exist
  1. Second attempt - Converted to snake_case namespace
  2. Third attempt - Used camelCase for server but snake_case for tool
  3. Fourth attempt - Tried camelCase with snake_case parameters
  4. Fifth attempt - Finally correct

Root Cause

The toolscript list-tools and toolscript list-servers commands display tool and server names in their "human-readable" forms (snake_case), while the TypeScript runtime uses camelCase property names. This disconnect forces users to:

  1. Guess at the correct naming convention
  2. Trial-and-error through multiple TypeScript compilation errors
  3. Spend unnecessary time debugging what should be straightforward

Additional Issue: Hook Suggestion

The SessionStart hook also suggested incorrect naming:

Consider using Skill("toolscript") to access these MCP tools:
azure-devops__search_code, azure-devops__search_wiki, azure-devops__search_workitem

These suggestions use a double-underscore format (azure-devops__search_workitem) which doesn't match the TypeScript API either.

Proposed Solution

Update the toolscript list-tools and toolscript list-servers output to include both formats:

Best approach: Show full name (servers -> camelCase) or (tools -> <server>.<tool>) with param types directly

toolscript list-servers:

Name        	TypeScript Name
Azure DevOps   azureDevops

toolscript list-tools:

\`tools.<server>.<tool>\`   - Description (params)
azureDevops.searchWorkitem  - Get Azure DevOps Work Item search results for a given search text ({ searchText: string })
azureDevops.searchCode      - Search Azure DevOps Repositories for a given search text ({ searchText: string })

Impact

  • Tokens wasted: LLMs could do much better if they didn't have to guess the correct naming convention and failing iteratively.

Suggested PR Changes

  1. Update list-tools command to display TypeScript-compatible names
  2. Update list-servers command to display TypeScript-compatible names (camelCase)
  3. Update documentation to clarify the naming convention or emphasize to use --types
  4. Consider updating hook suggestions to use correct camelCase format or even not use the hook at all - unfortunately, it's not possible to disable the hook now and it's taking a bit of time to submit the prompt due to this.
  5. Narrow down the suggested command for AI - toolscript list-tools should be enough.

Example Usage (After Fix)

# Clear, unambiguous naming
toolscript list-tools azureDevops
# Returns: azureDevops.searchWorkitem({searchText: string}), azureDevops.searchCode({searchText: string}), azureDevops.searchWiki({searchText: string})...

# Will correctly infer the correct function name and parameters without trial-and-error:
toolscript exec 'import {tools} from "toolscript"; const result = await tools.azureDevops.searchWorkitem({searchText: "pdf"})'

Complete Error Transcript

Execution Log of Opus 4.5 (thinking) - very long, would be just better to do list-tools which gives you all tools with params at once:

The Toolscript skill was invoked, and I need to use it to search Azure DevOps.

The user already provided the arguments they want to use with the `azure-devops__search_workitem` tool. Looking at the Toolscript documentation, I should use either:

- `toolscript search "what you need" --output`, or
- `toolscript exec 'import { tools } from "toolscript"; console.log(await tools.server.toolName({ param: "value" }))'`

Before making any calls, I should verify what MCP tools are available. From the system reminder, I see:

- `azure-devops__search_workitem`
- `azure-devops__search_wiki`
- `azure-devops__search_code`

So I should use Toolscript to call `azure-devops__search_workitem` with the provided query parameters.

However, per the MCP CLI instructions, I should run `mcp-cli info` first to confirm the schema before making calls.

Since I want to search for specific work items related to PDF/downloads, I’ll construct a Toolscript command that queries Azure DevOps for work items related to PDF and download issues, using the `azure-devops__search_workitem` tool with appropriate search parameters.

⏺ Bash(`toolscript exec 'import { tools } from "toolscript"; const result = await tools.azure_devops__search_workitem({query: "pdf OR soubory OR stáhnout", state: "Active,Resolved" });console.log(JSON.stringify(result, null, 2));'`)

Result:

	Error: Exit code 1
	TS2339 [ERROR]: Property 'azure_devops__search_workitem' does not exist on type '{
	braveSearch: { braveWebSearch(...): Promise<unknown>; ... };
	... 17 more ...;
	playwright: { ...; };
	}'.
	at file:///var/folders/vx/cd3sbys11nnfl2rj71r3_55r0000gn/T/cb37cb3b8b1bc655.ts:1:62

	error: Type checking failed.
	info: The program failed type-checking, but it still might work correctly.


It seems the tool name format is different than expected. Next, I should list the available servers/tools to discover the correct tool names.

⏺ Bash(toolscript list-servers)
...
... Huge chunk of servers
...
∴ Thinking…
  Good, now I see the azure-devops server is available. Let me list the tools available
  on that
  server.
  ⏺ Now let me list the Azure DevOps tools:
  ⏺ Bash(toolscript list-tools azure-devops)
∴ Thinking…
  Perfect! I can see the search_workitem tool is available. Now let me use it to search for PDF/file download related work items.

  ⏺ Now let me search for PDF-related work items:

⏺ Bash(toolscript exec 'import {tools} from "toolscript"; const result = await
  tools.azure_devops.search_workitem({search_text: "pdf OR soubory OR stáhnout"});
  console.log(JSON.stringify(result, null, 2))')
  ⎿  Error: Exit code 1
  TS2551 [ERROR]: Property 'azure_devops' does not exist on type '{ braveSearch: {
  braveWebSearch(params: BraveSearchBraveWebSearchParams): Promise<unknown>;
  braveLocalSearch(params: BraveSearchBraveLocalSearchParams): Promise<...>; }; ... 17
  more
  ...; playwright: { ...; }; }'. Did you mean 'azureDevops'?
  at
  file:///var/folders/vx/cd3sbys11nnfl2rj71r3_55r0000gn/T/693c13928c674aba.ts:1:62
  'azureDevops' is declared here.
  azureDevops: {
  ^
  at http://localhost:64541/runtime/tools.ts?_t=1768346994319:11786:3
  error: Type checking failed.
  info: The program failed type-checking, but it still might work correctly.
∴ Thinking…
  Ah, it should be azureDevops not azure_devops. Let me fix that.
  ⏺ Bash(toolscript exec 'import {tools} from "toolscript"; const result = await
  tools.azureDevops.search_workitem({search_text: "pdf OR soubory OR stáhnout"});
  console.log(JSON.stringify(result, null, 2))')
  ⎿  Error: Exit code 1
...
...
...
 info: The program failed type-checking, but it still might work correctly.
∴ Thinking…
 It should be searchWorkitem in camelCase. Let me try that.

⏺ Bash(toolscript exec 'import {tools} from "toolscript"; const result = await
  tools.azureDevops.searchWorkitem({search_text: "pdf OR soubory OR stáhnout"});
  console.log(JSON.stringify(result, null, 2))')
  ⎿  Error: Exit code 1
  TS2561 [ERROR]: Object literal may only specify known properties, but 'search_text'
  does not
  exist in type 'AzureDevopsSearchWorkitemParams'. Did you mean to write
  'searchText'?
  at
  file:///var/folders/vx/cd3sbys11nnfl2rj71r3_55r0000gn/T/9727aba42f607eeb.ts:1:90
  error: Type checking failed.
  info: The program failed type-checking, but it still might work correctly.

Eventually it figured that out, but it took about 30 000 tokens to do so.

Execution Log (prettier)

User Request: Search Azure DevOps for PDF/file download related work items. Use toolscript.

→ Step 1: User invokes toolscript skill with intent to search Azure DevOps

→ Step 2: Agent attempts to discover available tools
  Command: toolscript list-servers
  Result: Shows "azure-devops" as available server name

→ Step 3: Agent lists tools for that server
  Command: toolscript list-tools azure-devops
  Result: Shows "search_workitem" as available tool name

→ Step 4: First attempt using listed names with double-underscore
  Command: toolscript exec 'import {tools} from "toolscript"; const result = await tools.azure_devops__search_workitem({query: "pdf OR soubory OR stáhnout", state: "Active,Resolved"}); console.log(JSON.stringify(result, null, 2))'
  Error: TS2339 [ERROR]: Property 'azure_devops__search_workitem' does not exist
  Root Cause: Listed name format doesn't match API

→ Step 5: Second attempt converting to snake_case namespace
  Command: toolscript exec 'import {tools} from "toolscript"; const result = await tools.azure_devops.search_workitem({search_text: "pdf OR soubory OR stáhnout"}); console.log(JSON.stringify(result, null, 2))'
  Error: TS2551 [ERROR]: Property 'azure_devops' does not exist
  Suggestion: Did you mean 'azureDevops'?
  Root Cause: Tool lists "azure-devops" (kebab-case) but API expects "azureDevops" (camelCase)

→ Step 6: Third attempt using suggested camelCase server name
  Command: toolscript exec 'import {tools} from "toolscript"; const result = await tools.azureDevops.search_workitem({search_text: "pdf OR soubory OR stáhnout"}); console.log(JSON.stringify(result, null, 2))'
  Error: TS2551 [ERROR]: Property 'search_workitem' does not exist
  Suggestion: Did you mean 'searchWorkitem'?
  Root Cause: Tool lists "search_workitem" (snake_case) but API expects "searchWorkitem" (camelCase)

→ Step 7: Fourth attempt using camelCase for both server and tool
  Command: toolscript exec 'import {tools} from "toolscript"; const result = await tools.azureDevops.searchWorkitem({search_text: "pdf OR soubory OR stáhnout"}); console.log(JSON.stringify(result, null, 2))'
  Error: TS2561 [ERROR]: Object literal may only specify known properties, but 'search_text' does not exist
  Suggestion: Did you mean 'searchText'?
  Root Cause: Parameter names also use camelCase in API

→ Step 8: Fifth attempt with fully camelCased API call
  Command: toolscript exec 'import {tools} from "toolscript"; const result = await tools.azureDevops.searchWorkitem({searchText: "pdf OR soubory OR stáhnout"}); console.log(JSON.stringify(result, null, 2))'
  Result: ✅ SUCCESS!

Total Attempts: 5
Failed Attempts: 4
Tokens Wasted: ~30 000 tokens of trial-and-error

Error Details

Attempt Expected Listed Got Error
1 azureDevops.searchWorkitem() azure-devops.search_workitem azure_devops__search_workitem Property doesn't exist
2 azureDevops.searchWorkitem() azure-devops.search_workitem azure_devops.search_workitem Server not found
3 azureDevops.searchWorkitem() azure-devops.search_workitem azureDevops.search_workitem Tool not found
4 azureDevops.searchWorkitem() azure-devops.search_workitem azureDevops.searchWorkitem() w/ search_text Param not found
5 azureDevops.searchWorkitem() azure-devops.search_workitem azureDevops.searchWorkitem() w/ searchText ✅ Works

Would be happy to contribute a PR to fix this if the maintainers agree with the approach.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions