Skip to content

AGENTS.md

Latest commit

 

History

History
185 lines (142 loc) · 5.42 KB

AGENTS.md

File metadata and controls

185 lines (142 loc) · 5.42 KB
Raw

DocForge — Agent Guide

You are an AI agent connecting to DocForge, a document ingest platform for RAG pipelines. Read this file first, self-register, then use MCP or REST to convert documents into clean Markdown and deterministic chunks.

Quick start (60 seconds)

  1. Discover — GET /api/v1/agent-manifest (JSON: URLs, tools, chunking options, docs)
  2. Register — POST /api/v1/agents/register with { "name": "your-agent-name" }
  3. Save — api_key and id from the response (shown once only)
  4. Connect MCP — use cursor_config from the register response (stdio docforge-mcp)
  5. Ingest — call ingest_document (MCP) or POST /api/v1/ingest (REST)

Machine-readable manifest

GET /api/v1/agent-manifest

Returns JSON with selfRegistration, mcp, rest, chunking, documentation links, and a ready-to-paste Cursor MCP config.

Also available at:

GET /.well-known/docforge.json

Self-registration (REST)

POST /api/v1/agents/register
Content-Type: application/json

{
  "name": "my-rag-ingest-bot",
  "description": "Production document pipeline for vector store",
  "capabilities": ["ingest", "chunk", "convert"]
}

Response (201):

{
  "id": "my-rag-ingest-bot-a1b2c3d4",
  "name": "my-rag-ingest-bot",
  "api_key": "df_…",
  "mcp_command": "docforge-mcp",
  "mcp_env": {
    "DOCFORGE_API_URL": "http://127.0.0.1:8787",
    "DOCFORGE_API_KEY": "df_…"
  },
  "cursor_config": { "mcpServers": { "docforge": { "command": "docforge-mcp", "env": { … } } } },
  "message": "Store api_key securely — it cannot be retrieved again."
}

Self-registration (MCP)

  1. Ensure docforge-mcp is on PATH (pip install -e . in the DocForge repo).
  2. Connect MCP with command docforge-mcp and env DOCFORGE_API_URL.
  3. Call tool register_agent with { "name": "string", "description?": "string" }.
  4. Save api_key from the JSON result.
  5. Set env DOCFORGE_API_KEY in MCP config for authenticated calls.

Authenticated operations

Send header on REST calls (optional but recommended for tracking):

X-DocForge-Key: df_YOUR_KEY

MCP tools

Tool Auth Purpose
get_agent_manifest None Fetch manifest JSON
register_agent None Self-register; returns api_key once
list_agents None List agents (no secrets)
health_check None API health + version
convert_document Optional key Local file → Markdown
chunk_markdown Optional key Markdown → deterministic chunks
ingest_document Optional key Full pipeline: file → MD + chunks
agent_whoami Required key Your registered profile

REST endpoints

Method Path Auth Purpose
GET /api/v1/agent-manifest — Machine-readable manifest
GET /api/v1/health — Health check
POST /api/v1/agents/register — Self-register
GET /api/v1/agents — List agents
GET /api/v1/agents/me Key Your profile
POST /api/v1/ingest Optional File → Markdown + chunks
POST /api/v1/convert Optional File → Markdown
POST /api/v1/chunk Optional Markdown → chunks

Supported input formats

Extension Format
.pdf PDF (text extraction via PyMuPDF)
.html, .htm HTML → Markdown
.docx Word → Markdown
.txt, .md Plain text / Markdown

Chunking strategies

strategy Behavior
headings Split on Markdown H1–H6; each chunk includes heading_path
size Fixed token windows with overlap; fully deterministic

Both strategies produce stable chunk id values (SHA256-derived) for the same input.

Chunk JSON shape

{
  "id": "a1b2c3d4e5f67890",
  "index": 0,
  "content": "…",
  "token_estimate": 142,
  "char_count": 580,
  "heading_path": ["Introduction", "Section A"],
  "metadata": { "strategy": "headings", "section_index": 0 }
}

Cursor MCP configuration

After registration, merge cursor_config from the register response:

{
  "mcpServers": {
    "docforge": {
      "command": "docforge-mcp",
      "env": {
        "DOCFORGE_API_URL": "http://127.0.0.1:8787",
        "DOCFORGE_API_KEY": "df_REPLACE_AFTER_REGISTER"
      }
    }
  }
}

Security rules for agents

  1. Never log or commit api_key to git, issues, or chat transcripts.
  2. Never re-register if you already have a key — register once per agent identity.
  3. Store the key in MCP env or host secret store only.
  4. Chunk output may contain document content — handle per your data classification policy.

Documentation map

Resource URL (relative to deployment origin)
This guide /AGENTS.md
FAQ (AEO) /docs/faq.md or /FAQ.md
LLM index /llms.txt
MCP reference /docs/agents/mcp.md
REST reference /docs/agents/rest.md
Chunking guide /docs/agents/chunking.md
Cursor skill /docs/skill.md
JSON manifest /api/v1/agent-manifest
OpenAPI /api/docs

Human UI

Humans can register agents at {origin}/ → Agents tab. Agents should use API/MCP directly.

Local development

git clone https://github.com/bdeeps/docforge.git
./scripts/dev.sh

API + UI: http://127.0.0.1:8787

Repository

Source: https://github.com/bdeeps/docforge