Skip to content

docs: reframe sandbox as team-shared in tutorials #184

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
placerda merged 1 commit into from
May 28, 2026
Merged

docs: reframe sandbox as team-shared in tutorials #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions docs/tutorial-end-to-end.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ review.
.azure/
├── config.json # defaultEnvironment: sandbox
├── .gitignore # excludes */.env
├── sandbox/.env # author's safe space (private Foundry project or local app)
├── sandbox/.env # team authoring / experimentation space (Foundry project for prompts, or local/shared app for hosted agents)
├── dev/.env # team-shared dev project / endpoint (PR + deploy gate)
├── qa/.env # qa project / endpoint
└── prod/.env # production project / endpoint
Expand All @@ -54,10 +54,13 @@ the agent URL (`AGENTOPS_AGENT_ENDPOINT`) differs per environment because the
hosted endpoint itself is the per-environment artifact.

> **Why a separate sandbox?** When authors save in the Foundry playground,
> the platform auto-increments the version every save. If everyone saves to
> the same project, dev fills up with experimental versions, making
> traceability and rollback messy. Sandbox is the personal/team author
> space; dev is the gated, shared promotion target.
> the platform auto-increments the version every save. If experimentation
> happens in the same project that CI promotes from, dev fills up with
> half-baked versions and traceability and rollback become messy. Sandbox
> is the team's authoring and experimentation space (one project works
> for most teams; split per-stream or per-developer only if save
> collisions become a real problem); dev is the gated promotion target
> CI writes to.

## The cross-environment identity story (versioning callout)

Expand Down Expand Up @@ -834,8 +837,8 @@ You are ready for a release review when:
(prompt SHA + git SHA). For hosted agents, the git commit SHA is the
identity recorded in `results.json` and evidence.
- `.azure/` separates sandbox from dev (and qa / prod if you provisioned
them); the sandbox is the author's safe space and dev is the shared
promotion target.
them); the sandbox is the team's authoring and experimentation space
and dev is the shared promotion target.
- CI uses the expected runner for the target (cloud Foundry eval for
prompt agents in CI, local runner for hosted endpoints).
- Eval results or Microsoft Foundry eval metadata are attached to the
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorial-hosted-agent-quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ applies. For this tutorial you will work with two of them:

| Environment | What it is in this tutorial | Purpose |
|---|---|---|
| **sandbox** | The local FastAPI endpoint on your machine (`http://127.0.0.1:8000`). For a more realistic setup, this can also be a dedicated Foundry Hosted Agent or ACA revision per developer. | Author-side experimentation. Iterate, regress, fix, and validate with `agentops eval run` locally. No shared blast radius. |
| **sandbox** | The local FastAPI endpoint on your machine (`http://127.0.0.1:8000`). For a more realistic setup, this can also be a Foundry Hosted Agent or ACA revision shared by the team (or per-stream/per-developer if your team prefers that). | Author-side experimentation. Iterate, regress, fix, and validate with `agentops eval run` locally. No shared-with-CI blast radius. |
| **dev** | A deployed Foundry Hosted Agent, Azure Container Apps revision, AKS service, or any HTTPS endpoint reachable from CI. | Team-shared environment. The PR workflow evaluates this URL to verify it is still healthy. Deploy workflows (or your existing CI) update it on merge. |

Each environment maps to its own `.azure/<env>/.env` file with its own
Expand Down
39 changes: 21 additions & 18 deletions docs/tutorial-prompt-agent-quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ This path validates the Foundry-native multi-environment route:
Doctor blocking, release evidence, threshold enforcement, and Cockpit.

The toolkit benefit is the **release loop across environments**. You will
author the prompt in a personal **sandbox** Foundry project where saves are
private experiments, then let CI prove the prompt is safe to merge by
staging it as a candidate in a shared **dev** Foundry project, evaluating
that exact candidate, running Doctor against the result, and — only when
both pass — promoting the deploy.
author the prompt in a **sandbox** Foundry project where saves are
experimentation only and never trigger CI, then let CI prove the prompt
is safe to merge by staging it as a candidate in the team's **dev**
Foundry project, evaluating that exact candidate, running Doctor against
the result, and — only when both pass — promoting the deploy.

Pay special attention to Doctor in this tutorial: it does not only report
whether thresholds passed, it also catches slow regressions (for example,
Expand Down Expand Up @@ -52,7 +52,7 @@ permission prompts.
| Check | Why it matters |
|---|---|
| Azure CLI is installed and `az login` succeeds with the tenant that owns the Foundry projects. | AgentOps, Foundry SDK calls, and CI setup all need the same Azure identity context. |
| You can create **two** Foundry projects in the same Azure subscription (or have two existing projects you can use). | The tutorial uses a personal sandbox project plus a shared dev project; the PR gate stages candidates in dev. |
| You can create **two** Foundry projects in the same Azure subscription (or have two existing projects you can use). | The tutorial uses a sandbox project for authoring and experimentation plus a shared dev project for the PR gate; the PR workflow stages candidates in dev. |
| You can publish a prompt agent in each Foundry project. | The tutorial seeds the same `travel-agent:1` baseline in both projects so the deploy workflow has a known template to look up. |
| You can create or attach Application Insights for at least the dev Foundry project. | Foundry Traces, the Operate dashboard, Doctor, and Cockpit need telemetry to tell the observability story. Sandbox observability is optional. |
| You can push to the tutorial GitHub repository and run GitHub Actions. | The PR gate only runs after the repo is pushed. |
Expand All @@ -67,11 +67,11 @@ Before the hands-on steps, hold this picture in your head:

```
sandbox Foundry project dev Foundry project
(personal author iteration) (shared environment, PR gate target,
where merge deploys land)
(authoring + experimentation; (shared environment, PR gate target,
used by you or the team) where merge deploys land)
│ │
│ travel-agent:1 (seed) │ travel-agent:1 (seed, same instructions)
│ travel-agent:2,3,4,... (your saves) │ travel-agent:2,3,... (created by CI per PR / deploy)
│ travel-agent:2,3,4,... (free saves) │ travel-agent:2,3,... (created by CI per PR / deploy)
│ │
└──── git is the source of truth ─────────►│
.agentops/prompts/travel-agent.md
Expand Down Expand Up @@ -155,17 +155,19 @@ meet, not a place where AgentOps imposes a particular skill stack.
You need two Foundry projects in the same Azure subscription. Use these
names so the rest of the tutorial reads naturally:

- `travel-agent-sandbox` — your personal author iteration space. Saves
here never affect CI.
- `travel-agent-sandbox` — the authoring and experimentation space. Saves
here never trigger CI. One project is fine whether you are solo or
working with a small team; everyone with access can iterate here.
- `travel-agent-dev` — the first shared environment. The PR gate stages
candidates here, and the dev deploy workflow lands here.

> **Team tip.** This tutorial assumes a solo walkthrough, so one sandbox
> project is enough. For a team, give each developer their own sandbox
> project (for example, `travel-agent-paulo-sandbox`,
> `travel-agent-mariana-sandbox`) so simultaneous saves do not collide,
> and keep one shared `travel-agent-dev`. AgentOps does not care how many
> sandbox projects exist; only the dev/qa/prod chain is what CI promotes
> **Team scaling.** A single sandbox project works fine for a solo
> walkthrough and for small teams. If you grow to the point that
> simultaneous saves collide, or different feature streams need to
> experiment in isolation, you can split into per-stream sandboxes
> (`travel-agent-checkout-sandbox`, `travel-agent-search-sandbox`, etc.)
> or per-developer sandboxes. AgentOps does not care how many sandbox
> projects exist; only the dev / qa / prod chain is what CI promotes
> through.

### Path A — Foundry portal (always available)
Expand Down Expand Up @@ -205,7 +207,8 @@ skill propose the changes before applying them:
I want to set up two Azure AI Foundry projects in the same subscription
for an AgentOps tutorial:

1. travel-agent-sandbox - my personal author iteration space.
1. travel-agent-sandbox - the authoring and experimentation space
(used by me, or shared with my team for iteration).
2. travel-agent-dev - shared dev environment used by CI as the PR gate
target and the dev deploy target.

Expand Down
Loading