diff --git a/docs/extensions/overview.md b/docs/extensions/overview.md index bd69756..4f09ff4 100644 --- a/docs/extensions/overview.md +++ b/docs/extensions/overview.md @@ -149,9 +149,11 @@ NeatContext ships two first-party extensions authored through **exactly the same contract** you use — each is a single self-contained `server.cjs` with a manifest, no NeatContext internals: -- **PagerDuty** — `oauth2_pkce` connection, three read-only incident tools. +- **PagerDuty** — `oauth2_pkce` connection, three read-only incident tools + (usage guide: [Using the PagerDuty Extension](../guides/pagerduty-extension.md)). - **Datadog** — `api_key` connection (API key + application key + optional site), - one log-search tool. + one log-search tool + (usage guide: [Using the Datadog Extension](../guides/datadog-extension.md)). On the Extensions page, click the **folder icon** on either card to open its folder and read the source. They are the best starting templates. diff --git a/docs/features/using-extensions.md b/docs/features/using-extensions.md index cb2ab75..3fe7856 100644 --- a/docs/features/using-extensions.md +++ b/docs/features/using-extensions.md @@ -19,8 +19,11 @@ Two first-party connectors come **bundled** (marked *Built-in*; they can be disabled but not removed): - **PagerDuty** — read-only incident tools (incident details, related incidents, - past incidents), connected with OAuth. + past incidents), connected with OAuth. Step-by-step guide: + [Using the PagerDuty Extension](../guides/pagerduty-extension.md). - **Datadog** — read-only log search, connected with your Datadog API keys. + Step-by-step guide: + [Using the Datadog Extension](../guides/datadog-extension.md). They are also reference implementations: click the **folder icon** on either card to open its source. diff --git a/docs/guides/datadog-extension.md b/docs/guides/datadog-extension.md new file mode 100644 index 0000000..1b9d824 --- /dev/null +++ b/docs/guides/datadog-extension.md @@ -0,0 +1,144 @@ +--- +sidebar_position: 3 +--- + +# Using the Datadog Extension + +NeatContext ships a **built-in Datadog connector** with one read-only tool: +**log search**. The model can pull matching log events — timestamp, status, +service, host, message, tags — while investigating an incident, using the same +query syntax you'd type into the Datadog Log Explorer. It connects with your +**Datadog API and application keys**, and it can only *read* logs: it never +writes anything to your Datadog account. + +This guide walks through getting the keys, connecting, and searching logs from +chat, step by step. + +## Prerequisites + +- **NeatContext** installed, with a **tool-calling-capable** model configured + and active (see [Getting Started](../getting-started.md)). +- A **Datadog account** with logs in it, and permission to create an + application key. + +## Step 1 — Get your two keys from Datadog + +Datadog authenticates API access with a pair of keys; the extension needs both: + +1. **API key** — in Datadog, go to **Organization Settings → API Keys** + (avatar menu, bottom left). Use an existing key or create one. +2. **Application key** — go to **Organization Settings → Application Keys** + and create one. The application key acts *as you*, with your permissions — + it (or your user) must have the **`logs_read_data`** permission. If your + organization scopes application keys, include at least `logs_read_data`. + +Also note which **Datadog site** your organization is on — it's the domain you +open Datadog at: + +| You log in at | Your site is | +|---|---| +| `app.datadoghq.com` | `datadoghq.com` (the default) | +| `app.datadoghq.eu` | `datadoghq.eu` | +| `us3.datadoghq.com` | `us3.datadoghq.com` | +| `us5.datadoghq.com` | `us5.datadoghq.com` | +| `ap1.datadoghq.com` | `ap1.datadoghq.com` | + +## Step 2 — Connect on the Extensions page + +Open the **Extensions** page from the top bar. Datadog ships bundled, so its +card is already there, marked *Built-in* — nothing to add or install. Make sure +it is **enabled**, then fill the inline form: + +![The Datadog card with its inline API-key form](/img/features/extension-datadog-card.png) + +1. **API key** — paste the API key from Step 1. +2. **Application key** — paste the application key from Step 1. +3. **Site** — leave empty if you're on `datadoghq.com`; otherwise enter your + site from the table above (e.g. `datadoghq.eu`). +4. Click **Connect**. + +The keys are **encrypted with your OS secure storage**, stay on your machine, +and are handed to the extension only at the moment a tool runs. + +:::caution[Wrong site looks like wrong keys] +If the site doesn't match where your organization lives, Datadog rejects the +keys as if they were invalid. When a connection that "should work" is denied, +check the site first. +::: + +## Step 3 — Search logs from chat + +In any chat, ask a question the logs can answer. The model calls +`neatcontext_datadog_search_logs` with a Datadog **log search query** — the +same syntax as the Log Explorer search bar — and you'll see the call as an +**activity step** above the answer. + +Prompts to try: + +```text +Search Datadog for errors from the payments service in the last 15 minutes. +What's the dominant failure? +``` + +```text +Show me checkout-api logs with @http.status_code:500 over the last 2 hours, +oldest first. When did they start? +``` + +```text +Pull the last 50 error logs on host web-7 and summarize the distinct error +messages. +``` + +You can steer the query precisely — everything in your prompt maps onto the +tool's inputs: + +| You say | The tool uses | +|---|---| +| "errors from the payments service" | `query: "service:payments status:error"` — any Log Explorer query works, including facets like `@http.status_code:500`. `*` means all logs. | +| "in the last 2 hours" / "between 09:00 and 09:30 UTC" | `from`/`to` — relative (`now-15m`, `now-2h`) or ISO-8601 timestamps. **Default: the last 15 minutes.** | +| "the last 50 logs" | `limit` — 1 to 100 events per call (default 20). | +| "oldest first" | `sort` — newest first by default. | +| "only the `main` index" | `indexes` — defaults to all log indexes. | +| "get the next page" | `cursor` — each result carries a `next_cursor` for pagination, so the model can keep fetching. | + +Each returned event carries its **timestamp, status, service, host, message** +(long messages are truncated at 2,000 characters), and **tags** — enough to +spot the pattern and pivot the investigation. + +As always in NeatContext, the log evidence lands inside *your* context: the +**domain profile** and the tab's **knowledge folders** (runbooks, TSGs, +postmortems) shape what the model checks first and which actions it recommends. +The [Incident Analysis walkthrough](./incident-analysis.md) shows that +combination end-to-end. + +## Troubleshooting + +- **The answer says Datadog isn't connected**, with a **Connect Datadog** + button in the chat — you asked before connecting. Click the button (or use + the Extensions page) and ask again. +- **"Datadog denied access (status 401/403)"** — the keys were rejected. In + order of likelihood: the **site** doesn't match your organization (see the + caution above), the **application key** lacks the `logs_read_data` + permission, or a key was revoked. Fix in Datadog, then **Disconnect** and + reconnect with the corrected values. +- **"Datadog rate-limited the log search"** — you hit Datadog's API rate + limit. Wait a moment and ask again; narrower queries and smaller limits help. +- **No logs come back, but no error** — the query matched nothing in the time + range. Remember the default window is only the **last 15 minutes**; say + "in the last 24 hours" (or give explicit times) to widen it. + +## Rotating or removing keys + +Click **Disconnect** on the Datadog card to delete the stored keys from your +machine. To rotate keys, disconnect and reconnect with the new pair. + +## Next + +- Pair log search with incident tools: + [Using the PagerDuty Extension](./pagerduty-extension.md). +- See profiles + knowledge + tools working together: + [Incident Analysis walkthrough](./incident-analysis.md). +- Curious how it works? The extension is a reference implementation — click the + **folder icon** on its card to read the source, and see + [API-Key Extensions](../extensions/api-key-extensions.md) to build your own. diff --git a/docs/guides/pagerduty-extension.md b/docs/guides/pagerduty-extension.md new file mode 100644 index 0000000..c204242 --- /dev/null +++ b/docs/guides/pagerduty-extension.md @@ -0,0 +1,133 @@ +--- +sidebar_position: 2 +--- + +# Using the PagerDuty Extension + +NeatContext ships a **built-in PagerDuty connector**: three read-only incident +tools the model can call while you investigate — incident details, related +incidents, and similar past incidents. It connects with **OAuth** (you sign in +through your browser; no API keys to create), and it can only *read*: it never +acknowledges, resolves, or modifies anything in your PagerDuty account. + +This guide walks through connecting it and using it in chat, step by step. + +## Prerequisites + +- **NeatContext** installed, with a **tool-calling-capable** model configured + and active (see [Getting Started](../getting-started.md)). +- A **PagerDuty account** you can sign into in your browser. Any user that can + view incidents is enough — the connection asks only for **read** access. + +## Step 1 — Find the PagerDuty card + +Open the **Extensions** page from the top bar. PagerDuty ships bundled, so its +card is already there, marked *Built-in* — nothing to add or install. Make sure +it is **enabled**. + +The card lists the three tools it gives the model (hover a tool for its +description): + +| Tool | What it does | +|---|---| +| `neatcontext_pagerduty_get_incident` | Fetch one incident's details: title, status, urgency, priority, service, assignments, plus its recent **log entries** (the incident timeline). | +| `neatcontext_pagerduty_get_related_incidents` | Incidents related to a given one **right now** — correlated by machine-learning grouping, service dependencies, or a shared likely cause. | +| `neatcontext_pagerduty_get_past_incidents` | **Past** incidents similar to a given one, ranked by similarity — earlier occurrences that look like duplicates or recurrences. | + +## Step 2 — Connect through your browser + +1. Click **Connect** on the PagerDuty card. + + ![The PagerDuty card with its Connect button](/img/features/extension-pagerduty-card.png) + +2. Your **system browser** opens on PagerDuty's sign-in page. Sign in (if you + aren't already) and **authorize** the requested read access. +3. PagerDuty redirects back to NeatContext, which catches the redirect + locally and stores the tokens. The card now shows the connection as active. + +That's the whole setup. The tokens are **encrypted with your OS secure +storage**, never leave your machine, and are **refreshed automatically** — you +won't be asked to sign in again during normal use. + +:::info[The browser flow uses a localhost redirect] +The sign-in round trip finishes on a temporary local address +(`http://localhost:48973/...`) that exists only for the seconds the flow is in +progress. If a firewall prompt appears, allow it — nothing is listening outside +that moment. +::: + +## Step 3 — Ask about an incident + +In any chat, reference an incident in whichever form you have at hand — the +tools accept all three: + +- the **incident URL** you'd copy from your browser or a Slack alert, e.g. + `https://your-subdomain.pagerduty.com/incidents/Q3XYZ12ABC45DE` +- the **incident ID** (`Q3XYZ12ABC45DE`) +- the **incident number** (`4211`) + +Some prompts to try: + +```text +Summarize https://acme.pagerduty.com/incidents/Q3XYZ12ABC45DE — +who's assigned, and what does the timeline show so far? +``` + +```text +Are there incidents related to Q3XYZ12ABC45DE right now? +``` + +```text +Has incident 4211 happened before? Find similar past incidents. +``` + +Watch the **activity steps** above the answer — you'll see each PagerDuty tool +call as it happens; tool use is never invisible. And because this is +NeatContext, the answer is shaped by the **domain profile** and grounded in the +**knowledge folders** attached to the tab: pair the PagerDuty tools with your +team's runbooks and the model triages the incident *your team's* way (see the +[Incident Analysis walkthrough](./incident-analysis.md) for that pattern +end-to-end). + +### What each tool accepts + +You normally never spell these out — the model fills them in — but knowing the +knobs helps you phrase requests: + +- **Incident details** can include or skip the incident's log entries + ("give me the details without the timeline"), and fetch more of them + ("include the last 50 log entries" — up to 100). +- **Past incidents** returns the 5 most similar by default; ask for more + ("find the 20 most similar past incidents") up to 999. + +## Troubleshooting + +- **The answer says PagerDuty isn't connected**, with a **Connect PagerDuty** + button in the chat — you asked an incident question before connecting. Click + the button (or connect on the Extensions page) and ask again. +- **It was working, now it asks to reconnect** — the stored token was revoked + or could not be refreshed (e.g. your PagerDuty session was removed by an + admin). Click **Connect** again; the browser round trip repairs it. +- **"PagerDuty is connected, but the OAuth token does not have permission…"** — + your PagerDuty user can sign in but can't read that incident (or that + account's data). Check with your PagerDuty admin that your user can view the + incident in the PagerDuty web UI; if you can see it there, you can read it + here. +- **"Could not find a PagerDuty incident ID, number, or incident URL"** — the + reference didn't parse. Paste the full incident URL; that form is always + unambiguous. + +## Disconnecting + +Click **Disconnect** on the PagerDuty card to delete the stored tokens from +your machine. The card returns to its unconnected state; connect again anytime. + +## Next + +- Pair incident tools with team runbooks: + [Incident Analysis walkthrough](./incident-analysis.md). +- Add log search alongside it: + [Using the Datadog Extension](./datadog-extension.md). +- Curious how it works? The extension is a reference implementation — click the + **folder icon** on its card to read the source, and see + [OAuth Extensions](../extensions/oauth-extensions.md) to build your own.