Knowledge base data lookups: typed conditions and full-replacement updates

README.md

@@ -36,7 +36,7 @@ Open-source toolkit for **Pipefy** developers: a Model Context Protocol (MCP) se
 
 | Component | Package / path | Purpose |
 |-----------|----------------|---------|
-| **MCP server** | `pipefy-mcp-server` | Exposes **172** tools to MCP clients (Cursor, Claude Desktop, Claude Code, and others). |
+| **MCP server** | `pipefy-mcp-server` | Exposes **176** tools to MCP clients (Cursor, Claude Desktop, Claude Code, and others). |
 | **CLI** | `pipefy-cli` | Terminal commands aligned with MCP capabilities; see [`docs/parity.md`](docs/parity.md). |
 | **SDK** | `pipefy` | Vendor GraphQL client, services, and models shared by MCP and CLI. |
 | **Skills** | [`skills/`](skills/) | Markdown playbooks (Anthropic Skills format) for common Pipefy workflows. |
@@ -141,7 +141,7 @@ Deprecation and semver (post-1.0): [`docs/DEPRECATION.md`](docs/DEPRECATION.md).
 
 ## MCP server
 
-The server registers **172 tools** across thirteen domains. Canonical names: `PIPEFY_TOOL_NAMES` in [`packages/mcp/src/pipefy_mcp/tools/registry.py`](packages/mcp/src/pipefy_mcp/tools/registry.py).
+The server registers **176 tools** across thirteen domains. Canonical names: `PIPEFY_TOOL_NAMES` in [`packages/mcp/src/pipefy_mcp/tools/registry.py`](packages/mcp/src/pipefy_mcp/tools/registry.py).
 
 Tool descriptions and `Args:` blocks come from Python docstrings (what MCP clients show to models). Per-area reference docs cover parameters, edge cases, and cross-cutting behavior.
 
@@ -155,7 +155,7 @@ Tool descriptions and `Args:` blocks come from Python docstrings (what MCP clien
 | **Reports** | 17 | Pipe and organization reports, async exports. | [docs](docs/mcp/tools/reports.md) |
 | **Automations & AI** | 23 | Automations, AI automations, AI agents, validators. | [docs](docs/mcp/tools/automations-and-ai.md) |
 | **LLM providers** | 5 | Discovery reads for the organization's LLM providers (custom + Pipefy-managed), vendor model lists, owner defaults, dependencies, and a read-access probe. | [docs](docs/mcp/tools/llm-providers.md) |
-| **Knowledge bases** | 10 | Pipe-scoped AI knowledge bases: list all items, plain text and document CRUD (one-shot PDF upload), and a read-access probe. Attach sources to agents/behaviors via `dataSourceIds`. | [docs](docs/mcp/tools/knowledge-bases.md) |
+| **Knowledge bases** | 14 | Pipe-scoped AI knowledge bases: list all items, plain text / document (one-shot PDF upload) / data lookup CRUD, and a read-access probe. Attach sources to agents/behaviors via `dataSourceIds`. | [docs](docs/mcp/tools/knowledge-bases.md) |
 | **iPaaS** | 4 | Lazy discovery, invocation, and app-connection setup for a pipe's iPaaS (Advanced Automations) workspace (`get_ipaas_tools`, `call_ipaas_tool`, plus the connection meta-tools). | [docs](docs/mcp/tools/ipaas.md) |
 | **Observability** | 11 | Logs, usage, credits, execution metrics, job exports. | [docs](docs/mcp/tools/observability.md) |
 | **Members, email & webhooks** | 11 | Membership, inbox email, webhooks. | [docs](docs/mcp/tools/members-email-webhooks.md) |

docs/mcp/README.md

@@ -13,7 +13,7 @@ Material in this tree describes **`pipefy-mcp-server`**: the MCP process, tool b
 | [`tools/reports.md`](tools/reports.md) | Pipe and organization reports, async exports |
 | [`tools/automations-and-ai.md`](tools/automations-and-ai.md) | Traditional automations, AI automations, AI agents, validators |
 | [`tools/llm-providers.md`](tools/llm-providers.md) | LLM provider discovery: custom + system providers, vendor models, defaults, dependencies, access probe |
-| [`tools/knowledge-bases.md`](tools/knowledge-bases.md) | Pipe-scoped AI knowledge bases: list, plain text and document (PDF) CRUD, read-access probe |
+| [`tools/knowledge-bases.md`](tools/knowledge-bases.md) | Pipe-scoped AI knowledge bases: list, plain text, document (PDF), and data lookup CRUD, read-access probe |
 | [`tools/observability.md`](tools/observability.md) | Logs, usage, credits, execution metrics, job exports |
 | [`tools/members-email-webhooks.md`](tools/members-email-webhooks.md) | Membership, inbox email, webhooks |
 | [`tools/organization.md`](tools/organization.md) | Organization metadata |

docs/mcp/tools/knowledge-bases.md

@@ -1,14 +1,14 @@
 # Knowledge bases
 
-Pipe-scoped AI knowledge bases: list every item on a pipe, full CRUD for plain-text and document (PDF) sources, and a read-access probe. **10 tools.**
+Pipe-scoped AI knowledge bases: list every item on a pipe, full CRUD for plain-text, document (PDF), and data lookup sources, and a read-access probe. **14 tools.**
 
 Knowledge bases are the data sources an AI agent draws on. Each item's `id` is what you attach to an agent or behavior via `dataSourceIds` (see [Automations & AI](automations-and-ai.md)): use `get_ai_knowledge_bases` to discover the IDs, then `validate_ai_agent_behaviors(data_source_ids=[...])` to check membership before writing the agent.
 
 ---
 
 | Tool | Read-only | Role |
 |------|-----------|------|
-| `get_ai_knowledge_bases` | Yes | Lists every knowledge base item on a pipe (plain texts, documents, data lookups) as one flat list — no pagination. Each item carries `id`, `type` (e.g. `knowledge_base_plain_texts`), `name`, `description`, `updatedAt`. |
+| `get_ai_knowledge_bases` | Yes | Lists every knowledge base item on a pipe (plain texts, documents, data lookups) as one flat list — no pagination. Each item carries `id`, `type` (`knowledge_base_plain_texts`, `knowledge_base_documents`, or `data_lookups` — the data lookup discriminator is not prefixed), `name`, `description`, `updatedAt`. |
 | `get_ai_knowledge_base_plain_text` | Yes | Fetches one plain text by `id`, including its full `content`. |
 | `create_ai_knowledge_base_plain_text` | No | Creates a plain text. `name`, `content` (1-3500 chars), and `description` (1-900 chars) are all required. |
 | `update_ai_knowledge_base_plain_text` | No | Partial update by `plain_text_id`: pass any of `name` / `content` / `description` (at least one); omitted fields keep their stored value. |
@@ -17,11 +17,15 @@ Knowledge bases are the data sources an AI agent draws on. Each item's `id` is w
 | `create_ai_knowledge_base_document` | No | Creates a document from a local PDF in one shot (presigned URL, S3 PUT, create mutation). `name`, `file_path`, and `description` (1-900 chars) are required. |
 | `update_ai_knowledge_base_document` | No | Metadata-only partial update by `document_id`: pass `name` and/or `description` (at least one). The PDF file cannot be replaced. |
 | `delete_ai_knowledge_base_document` | No | Deletes a document permanently. Two-step: preview with `confirm=false` (default), execute with `confirm=true`. |
+| `get_ai_knowledge_base_data_lookup` | Yes | Fetches one data lookup by `id`. The payload never includes `conditions` (see below). |
+| `create_ai_knowledge_base_data_lookup` | No | Creates a data lookup: a data source that searches cards in a source pipe by conditions and returns selected fields. `name`, `description` (1-900 chars), `source_repo_id`, `output_fields`, and `conditions` are required. |
+| `update_ai_knowledge_base_data_lookup` | No | Replaces a data lookup's full definition: `source_repo_id`, `output_fields`, and `conditions` are required on every call; omitting `search_query` clears it. Only `name`/`description` are partial. |
+| `delete_ai_knowledge_base_data_lookup` | No | Deletes a data lookup permanently. Two-step: preview with `confirm=false` (default), execute with `confirm=true`. |
 | `validate_knowledge_base_access` | Yes | Probes whether the current credential can read a pipe's knowledge bases, classifying failures into structured problems instead of opaque errors. |
 
 ## Identifiers: pipe UUID, not numeric ID
 
-Every knowledge base operation is scoped by the pipe **UUID** (`pipe_uuid`), not the numeric pipe ID — this follows the Pipefy GraphQL API. `get_pipe` returns the `uuid` field; `get_ai_knowledge_bases` returns each item's `id` (a data-source UUID) for the plain-text-by-id operations and for `dataSourceIds`.
+Every knowledge base operation is scoped by the pipe **UUID** (`pipe_uuid`), not the numeric pipe ID — this follows the Pipefy GraphQL API. `get_pipe` returns the `uuid` field; `get_ai_knowledge_bases` returns each item's `id` (a data-source UUID) for the by-id operations and for `dataSourceIds`. The one deliberate exception is a data lookup's `source_repo_id`, which is the **numeric** ID of the source pipe (see below).
 
 ## Plain-text limits (enforced client-side)
 
@@ -49,11 +53,30 @@ Failures are tagged with the step that failed — `file_read`, `presigned_url`,
 
 **Update is metadata-only.** `update_ai_knowledge_base_document` changes `name` / `description`; it cannot replace the PDF. `content` returned by `get_ai_knowledge_base_document` is the stored document URL, not the extracted text.
 
+## Data lookups (typed conditions, full-replacement updates)
+
+A data lookup lets an agent search cards in a **source pipe** by conditions and return the values of selected fields. Its definition has four parts: `source_repo_id` (the source pipe), `output_fields` (which field values matching records return), `conditions` (how records are filtered), and an optional `search_query`.
+
+The toolkit validates the definition client-side because the API accepts shapes that only fail later, when an agent actually runs the lookup:
+
+- **`source_repo_id` must be the numeric pipe ID**, not a UUID. The API accepts a UUID at create time, but the lookup then breaks when an agent runs it — long after the write succeeded.
+- **`output_fields`**: 1-30 field IDs — start-form/phase field slugs (e.g. `customer_email`) plus static fields such as `id`, `title`, `current_phase`, `assignees`, `created_by`, `created_at`.
+- **`conditions`**: at least one object, each with `field` (a field ID as above) and `operator` (an opaque backend string, e.g. `"eq"`, `"contains"`). A condition is one of two shapes:
+  - **Static** — `value` is required and must be a string (the backend silently drops non-scalar values and the runtime rejects non-strings): `{"field": "title", "operator": "contains", "value": "urgent"}`.
+  - **AI-filled** — `usingFillWithAi: true` with `inputName`, `inputType` (opaque, e.g. `"text"`, `"number"`), and `inputDescription`; `value` must be omitted (the AI asks the user for it at runtime): `{"field": "customer_email", "operator": "eq", "usingFillWithAi": true, "inputName": "Customer email", "inputType": "text", "inputDescription": "The customer's email address"}`.
+
+  Condition keys accept camelCase (as above) or snake_case; unknown keys are rejected so a typo cannot silently drop a constraint. A condition may also carry the optional passthrough keys `attribute` and `fieldUuid`, used by backend-specific search modes (`field` is still required — the backend rejects a condition without one).
+- **`search_query`** is a backend-defined search mode marker, not free text; leave it unset unless you know the backend value you need.
+
+**Reads never return `conditions`.** The API stores them but does not expose them on `get_ai_knowledge_base_data_lookup` (or on the create/update responses), so keep your lookup definition — including conditions — as the client-side source of truth.
+
+**Updates replace the whole definition.** The backend rewrites the stored definition with exactly what the update carries: `source_repo_id`, `output_fields`, and `conditions` are required on every `update_ai_knowledge_base_data_lookup` call (pass the complete condition set, not a delta), and omitting `search_query` clears it. Only `name`/`description` behave partially (omitted keeps the stored value).
+
 ## Probe semantics and the write gate
 
-- A green `validate_knowledge_base_access` proves **read access only** (`read_ai_agents` on the pipe) — never the `manage_ai_agents` entitlement that plain-text and document create / update / delete require.
+- A green `validate_knowledge_base_access` proves **read access only** (`read_ai_agents` on the pipe) — never the `manage_ai_agents` entitlement that every knowledge base create / update / delete (plain text, document, data lookup) requires.
 - An **empty knowledge base list** is a valid green result (`knowledge_base_count: 0`), not a failure.
-- The **CLI gates writes** on the probe: `pipefy kb plain-text create` / `update` and `pipefy kb document create` / `update` run the read-access probe first and fail with the classified problem if it is denied, before attempting the mutation.
+- The **CLI gates writes** on the probe: `pipefy kb plain-text create` / `update`, `pipefy kb document create` / `update`, and `pipefy kb data-lookup create` / `update` run the read-access probe first and fail with the classified problem if it is denied, before attempting the mutation.
 - The **MCP tools stay explicit-validate-first**: create / update do not auto-probe. Call `validate_knowledge_base_access` yourself before writing.
 - **Deletes require confirmation**: the MCP tool needs `confirm=true`; the CLI needs `--yes` (or an interactive prompt).
 

docs/parity.md

@@ -2,7 +2,7 @@
 
 This matrix is the source of truth for **MCP tool ↔ `pipefy` CLI** coverage. Update it whenever MCP tools or CLI commands are added, renamed, or removed.
 
-**Registry source:** `PIPEFY_TOOL_NAMES` in `packages/mcp/src/pipefy_mcp/tools/registry.py` (must stay in sync with this table: **172** tools).
+**Registry source:** `PIPEFY_TOOL_NAMES` in `packages/mcp/src/pipefy_mcp/tools/registry.py` (must stay in sync with this table: **176** tools).
 
 **Later CLI coverage:** areas such as attachments, field conditions, email, audit export, traditional automations, exports/usage, introspection, and raw GraphQL appear as **shipped** below when the matching Typer commands exist in `packages/cli`.
 
@@ -30,6 +30,7 @@ For **database records**, `find_records` result nodes may use **`fields`** while
 | `clone_pipe` | `pipefy pipe clone` | shipped | optional `--org`. |
 | `create_ai_agent` | `pipefy agent create` | shipped | AI Agents domain; post-v0.1 CLI unless explicitly rescoped. |
 | `create_ai_automation` | `pipefy ai-automation create` | shipped | AI Automations domain; post-v0.1 CLI unless explicitly rescoped. |
+| `create_ai_knowledge_base_data_lookup` | `pipefy kb data-lookup create` | shipped | Knowledge bases; pipe-scoped (`--pipe-uuid`, `--name`, `--description` <=900, `--source-repo-id` numeric pipe ID, `--output-fields` JSON array 1-30, `--conditions` JSON array, optional `--search-query`). Conditions are typed client-side (static needs a string value; AI-filled needs the input trio). CLI gates on the read-access probe. |
 | `create_ai_knowledge_base_document` | `pipefy kb document create` | shipped | Knowledge bases; one-shot PDF upload (presigned URL, S3 PUT, create mutation). Pipe-scoped (`--pipe-uuid`, `--file`, `--name`, `--description` <=900, all required). `.pdf` and 20 MiB cap enforced client-side; step-tagged errors (`file_read`/`presigned_url`/`s3_upload`/`kb_create`). CLI gates on the read-access probe. Indexing is asynchronous. |
 | `create_ai_knowledge_base_plain_text` | `pipefy kb plain-text create` | shipped | Knowledge bases; pipe-scoped (`--pipe-uuid`, `--name`, `--content` <=3500, `--description` <=900, all required). CLI gates on the read-access probe; limits fail fast client-side. |
 | `create_automation` | `pipefy automation create` | shipped | (`--pipe`, `--name`, `--trigger-id`, `--action-id`, optional `--extra` JSON). |
@@ -55,6 +56,7 @@ For **database records**, `find_records` result nodes may use **`fields`** while
 | `create_webhook` | `pipefy webhook create` | shipped | — |
 | `delete_ai_agent` | `pipefy agent delete` | shipped | AI Agents domain. |
 | `delete_ai_automation` | `pipefy ai-automation delete` | shipped | AI Automations domain. |
+| `delete_ai_knowledge_base_data_lookup` | `pipefy kb data-lookup delete` | shipped | Knowledge bases; pipe-scoped (`--id`, `--pipe-uuid`). MCP requires `confirm=true`; CLI requires `--yes` (or interactive prompt). |
 | `delete_ai_knowledge_base_document` | `pipefy kb document delete` | shipped | Knowledge bases; pipe-scoped (`--id`, `--pipe-uuid`). MCP requires `confirm=true`; CLI requires `--yes` (or interactive prompt). |
 | `delete_ai_knowledge_base_plain_text` | `pipefy kb plain-text delete` | shipped | Knowledge bases; pipe-scoped (`--id`, `--pipe-uuid`). MCP requires `confirm=true`; CLI requires `--yes` (or interactive prompt). |
 | `delete_automation` | `pipefy automation delete` | shipped | destructive: `--yes` or confirm. |
@@ -95,6 +97,7 @@ For **database records**, `find_records` result nodes may use **`fields`** while
 | `get_ai_automation` | `pipefy ai-automation get` | shipped | AI Automations domain. |
 | `get_ai_automations` | `pipefy ai-automation list` | shipped | AI Automations domain. |
 | `get_ai_credit_usage` | `pipefy usage credits` | shipped | (`--organization`, `--period`). |
+| `get_ai_knowledge_base_data_lookup` | `pipefy kb data-lookup get` | shipped | Knowledge bases; pipe-scoped data lookup (`--id`, `--pipe-uuid`); the payload never includes `conditions` (the API does not expose them on reads). |
 | `get_ai_knowledge_base_document` | `pipefy kb document get` | shipped | Knowledge bases; pipe-scoped document metadata (`--id`, `--pipe-uuid`); `content` is the stored document URL, not the extracted text. |
 | `get_ai_knowledge_base_plain_text` | `pipefy kb plain-text get` | shipped | Knowledge bases; pipe-scoped plain text with content (`--id`, `--pipe-uuid`). |
 | `get_ai_knowledge_bases` | `pipefy kb list` | shipped | Knowledge bases; all items on a pipe (plain text, documents, data lookups) with `id` for `dataSourceIds` (`--pipe-uuid`; no pagination). |
@@ -168,6 +171,7 @@ For **database records**, `find_records` result nodes may use **`fields`** while
 | `unpublish_sub_portal` | `pipefy portal sub-portal unpublish` | shipped | internal_api `updateSubPortalElement(subPortalUuid: null)`; sets `subPortals[].published` to false. |
 | `update_ai_agent` | `pipefy agent update` | shipped | AI Agents domain. |
 | `update_ai_automation` | `pipefy ai-automation update` | shipped | AI Automations domain. |
+| `update_ai_knowledge_base_data_lookup` | `pipefy kb data-lookup update` | shipped | Knowledge bases; full-replacement update (`--id`, `--pipe-uuid`, required `--source-repo-id`/`--output-fields`/`--conditions` every call; omitted `--search-query` clears it; only `--name`/`--description` are partial). CLI gates on the read-access probe. |
 | `update_ai_knowledge_base_document` | `pipefy kb document update` | shipped | Knowledge bases; metadata-only partial update (`--id`, `--pipe-uuid`, optional `--name`/`--description`, at least one); no file replacement. CLI gates on the read-access probe. |
 | `update_ai_knowledge_base_plain_text` | `pipefy kb plain-text update` | shipped | Knowledge bases; partial update (`--id`, `--pipe-uuid`, optional `--name`/`--content`/`--description`, at least one). CLI gates on the read-access probe; limits fail fast client-side. |
 | `update_automation` | `pipefy automation update` | shipped | (`--extra` JSON). |
@@ -209,6 +213,6 @@ for n in m.body:
             print(len(v.args[0].elts))"
 ```
 
-Expect **172** tool names in `PIPEFY_TOOL_NAMES` and **172** data rows in the parity table (excluding the header rows).
+Expect **176** tool names in `PIPEFY_TOOL_NAMES` and **176** data rows in the parity table (excluding the header rows).
 
 When adding or removing an MCP tool, update **this file** and `PIPEFY_TOOL_NAMES` in the same change set.