Skip to content

Commit ca7d8b3

Browse files
placerdaCopilot
andcommitted
Expand Foundry observability guidance
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent be20c44 commit ca7d8b3

3 files changed

Lines changed: 46 additions & 0 deletions

File tree

docs/tutorial-end-to-end.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -525,6 +525,22 @@ Use this loop in the video:
525525
| Eval context | From a Foundry eval run, inspect row-level explanations and, when available, the trace attached to the interaction. | The repo keeps the exact target, dataset, gate, and evidence together. |
526526
| Trace learning | Export or curate traces that represent real issues. | `agentops eval promote-traces` turns reviewed traces into regression candidates. |
527527

528+
For the screen recording, make the Foundry side visible before opening AgentOps
529+
Cockpit:
530+
531+
| Panel | Show | Say |
532+
|---|---|---|
533+
| Project overview / connected resources | Foundry project plus attached App Insights. | "This is where runtime telemetry is connected." |
534+
| Agent or endpoint Traces | One Trace ID, span tree, input/output, metadata, latency, model/tool call, and conversation context if present. | "This is the single interaction drilldown." |
535+
| Foundry Evaluations | The managed eval run and row-level scoring. | "This is the quality evidence for the candidate." |
536+
| Operate overview | Aggregate health, errors, latency, usage, and Ask AI when available. | "This is the production operations view." |
537+
| Application Insights Logs | KQL for the same operation or trace. | "This is the raw Azure Monitor investigation path." |
538+
| Red Teaming / safety | Scan entry point or linked scan result. | "This is the managed safety review path." |
539+
540+
Then open AgentOps Doctor/Cockpit to show the complement: repo-side gates,
541+
workflow state, evidence, findings, and links back to those official Foundry and
542+
Azure Monitor surfaces.
543+
528544
If runtime discovery does not find a connected App Insights resource, or your
529545
identity cannot read it, set the connection string in the AgentOps local env:
530546

docs/tutorial-hosted-agent-quickstart.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -381,6 +381,20 @@ union traces, requests, dependencies
381381
| order by timestamp desc
382382
```
383383

384+
If you are demonstrating a real Foundry Hosted Agent instead of the local
385+
FastAPI sample, spend a minute in the Foundry observability panels too:
386+
387+
| Foundry / Azure surface | What to show | Why it matters |
388+
|---|---|---|
389+
| Hosted agent / endpoint page | The deployed endpoint or agent reference that `agentops.yaml` calls. | Connects the repo target to the runtime being observed. |
390+
| Agent Traces | A recent request, Trace ID, spans, input/output, metadata, latency, model call, tool calls, and conversation context when present. | Shows the richer Foundry-managed runtime trace that the local sample cannot emit. |
391+
| Operate overview | Aggregate latency, failures, usage, and Ask AI when available. | Shows service health beyond one request. |
392+
| Application Insights Logs | KQL for the same operation ID or trace ID. | Gives the raw Azure Monitor drilldown path. |
393+
394+
The transition is the same as the prompt-agent tutorial: Foundry and Azure
395+
Monitor own live observability; AgentOps checks whether those signals are wired
396+
into eval gates, Doctor findings, Cockpit, and release evidence.
397+
384398
Those attributes are tutorial conventions, not special Foundry fields. A
385399
deployed Foundry Hosted Agent uses the same App Insights backend and Foundry
386400
trace surface, but its runtime spans include richer agent, tool, model, and

docs/tutorial-prompt-agent-quickstart.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,22 @@ This is the Foundry side of Operate. AgentOps does not replace it; later Doctor,
128128
Cockpit, and release evidence check whether the repo can point reviewers back to
129129
these official runtime signals.
130130

131+
For a recording, make this a short Foundry observability tour before you move
132+
back to the terminal:
133+
134+
| Foundry / Azure surface | What to show | Why it matters |
135+
|---|---|---|
136+
| Project overview / connected resources | The Foundry project and attached App Insights resource. | Confirms telemetry has an official place to land. |
137+
| Agent Traces | The latest playground run, Trace ID, spans, input/output, metadata, latency, token/model details, and any Conversation ID. | Shows runtime behavior for one concrete interaction. |
138+
| Foundry Evaluations | The managed eval run and row-level details after CI runs. | Shows quality scoring where Foundry executes the prompt-agent eval. |
139+
| Operate overview | Aggregate latency, failures, usage, and Ask AI when available. | Shows service-level health, not just one trace. |
140+
| Application Insights Logs | The same operation from the Azure Monitor side when you need raw KQL drilldown. | Gives engineers the low-level query path behind the dashboard. |
141+
| Red Teaming / safety scans | The managed scan entry point, if your project has it enabled. | Frames safety work as Foundry-managed evidence that AgentOps later links into release review. |
142+
143+
The transition line for the tutorial is: Foundry tells you what happened in the
144+
runtime; AgentOps checks whether the repo, CI, and release evidence are wired so
145+
reviewers can act on those signals.
146+
131147
## 2. Create a clean workspace and install AgentOps
132148

133149
```powershell

0 commit comments

Comments
 (0)