Skip to content

feat: add MCP server for coding agents (Claude Code, Cursor, Windsurf) - #11

Open
rajarshidattapy wants to merge 1 commit into
Dhravya:mainfrom
rajarshidattapy:main
Open

feat: add MCP server for coding agents (Claude Code, Cursor, Windsurf)#11
rajarshidattapy wants to merge 1 commit into
Dhravya:mainfrom
rajarshidattapy:main

Conversation

@rajarshidattapy

Copy link
Copy Markdown

Closes #1

What this adds

An MCP (Model Context Protocol) server at mcp/index.ts that exposes webpull's crawling engine as tools callable by any MCP-compatible coding agent.

Why it belongs here

webpull's primary use case is feeding docs to LLMs. The CLI forces a manual two-step — pull first, then point the agent at the directory. This collapses it into a single agent-native call:

# Before
$ webpull https://docs.anthropic.com
$ claude "read ./docs.anthropic.com and explain streaming"

# After (agent does it all)
> "pull the Anthropic docs and show me all streaming examples"

Tools exposed

Tool Description
pull_docs Crawl a URL using webpull, save as local markdown
search_docs Full-text search across pulled docs
read_doc Read a specific file by path
list_docs List all pulled sites and file counts
run_code Execute JS/TS/Python/Bash in a sandboxed subprocess

Setup (one command)

claude mcp add webpull bun run /path/to/mcp/index.ts

Implementation notes

  • Zero new dependencies — hand-rolled stdio JSON-RPC loop, no @modelcontextprotocol/sdk needed. Runs on Bun, same runtime webpull already requires.
  • Single file — everything lives in mcp/index.ts. No new packages, no build step.
  • Shells out to the installed webpull CLI — doesn't import from src/, so the MCP server stays decoupled from internal refactors.
  • Docs persist across sessions — stored in ~/.webpull-docs/<hostname>/, only pulled once per site.
  • run_code is scoped to developer machines — inherits the user's env, not a multi-tenant sandbox. Documented clearly.

Files changed

  • mcp/index.ts — MCP server (new file)
  • mcp/README.md — setup instructions for Claude Code, Cursor, Windsurf
  • package.json — adds webpull-mcp bin entry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

# Bug: Crawler fallback escapes scope path and crawls entire hostname

1 participant