Skip to content

Add Store-ready A365 S2S exporter sample - #276

Closed
Nikhil Navakiran (nikhilNava) wants to merge 16 commits into
microsoft:mainfrom
nikhilNava:nikhilc/add-a365-s2s-sample
Closed

Add Store-ready A365 S2S exporter sample#276
Nikhil Navakiran (nikhilNava) wants to merge 16 commits into
microsoft:mainfrom
nikhilNava:nikhilc/add-a365-s2s-sample

Conversation

@nikhilNava

@nikhilNava Nikhil Navakiran (nikhilNava) commented Sep 25, 2026

Copy link
Copy Markdown
Collaborator

What

Replaces #217 with a writable fork-based branch and adds a Store-ready Agent 365 S2S observability sample under samples/a365/s2s/.

Changes

  • Merged current main without rebasing the original work.
  • Demonstrates InvokeAgentScope, ApplyGuardrailScope, InferenceScope, ExecuteToolScope, and OutputScope.
  • Populates the Microsoft Learn Store-publishing contract for agent, Blueprint, tenant, human caller, channel, conversation, endpoint, model, messages, and tool attributes.
  • Intentionally omits microsoft.agent.user.id and microsoft.agent.user.email for S2S.
  • Uses one agent app instance client ID for token acquisition, telemetry identity, and the export URL; rejects tenant or agent mismatches before MSAL acquisition.
  • Preserves token caching with a 60-second refresh buffer and idempotent exporter logging.
  • Removes the sample-local .env.example, pyproject.toml, and uv.lock; setup is documented in the sample README.
  • Adds network-free span-contract and resolver/logging regression tests.

Validation

  • 35 passed across the S2S contract tests plus guardrail and invoke-agent scope suites.
  • Python compilation and git diff --check passed.
  • Final code review found no remaining Critical or Important issues.

Supersedes #217 because its organization-owned head branch could not be updated by the authenticated PR author account.

nikhilc-microsoft and others added 15 commits June 24, 2026 12:08
@nikhilc-microsoft
Add A365 S2S exporter sample
b4da17a 
Add a self-contained sample under samples/a365_s2s/ demonstrating A365
telemetry export using the S2S (service-to-service) flow:

- Custom a365_token_resolver mirroring the SDK FIC flow (app -> instance
  token exchange via MSAL) minus the agentic-user step, plus
  a365_use_s2s_endpoint=True.
- Manual A365 scope classes (InvokeAgent/Inference/ExecuteTool), so the
  sample needs no LLM.
- uv-managed project (pyproject.toml), .env via python-dotenv, and DEBUG
  logging so developers can see the export HTTP status and correlation id.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Address PR feedback for A365 S2S sample
389aa74 
- Use request_tenant_id for the MSAL authority and fail fast when it
  diverges from the configured credential tenant, so tokens are acquired
  for the same tenant they are cached under.
- Guard _configure_export_logging against duplicate StreamHandlers and
  disable propagation to avoid double-logging on repeated runs.
- Move the sample under samples/a365/s2s and update README references.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 516a9ff0-9d67-4aae-87af-e7e6395a66e1
@nikhilc-microsoft
Fix broad-exception pylint pragma placement for pinned pylint
906b7fc 
The inline pragma was not recognized by the CI-pinned pylint (3.2.7)
because it followed the vestigial ruff noqa in the same comment. Ruff is
not configured in this repo, so drop the noqa and lead the comment with
the pylint disable pragma.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 516a9ff0-9d67-4aae-87af-e7e6395a66e1
@nikhilc-microsoft
Cap openai<2.45.0 to fix openai-agents integration test failures
a120b05 
openai 2.45.0 made InputTokensDetails.cache_write_tokens a required
field. openai-agents (pulled transitively via the agent-framework and
openai-agents extras) still constructs InputTokensDetails(cached_tokens=0)
in agents/usage.py, which now raises pydantic ValidationError and breaks
the tests/a365/integration/openai suite.

Constrain openai to >=2.36.0,<2.45.0 in both extras. This satisfies
openai-agents 0.18.0 (openai<3,>=2.36.0) and agent-framework-openai
1.10.0 (openai<3,>=1.99.0), resolving to openai 2.44.0. Verified that
InputTokensDetails(cached_tokens=0) succeeds on 2.44.0 and fails on 2.45.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 516a9ff0-9d67-4aae-87af-e7e6395a66e1
@nikhilc-microsoft
Document A365 S2S store validation design
d6e627e 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Plan A365 S2S store validation update
ed4e0b3 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Ignore local git worktrees
d21df75 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Merge main into A365 S2S sample
8f4a1a6 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Restore A365 S2S sample README entry
bc339c4 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Fix A365 S2S sample setup guidance
d9d35f2 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Remove dotenv from A365 S2S sample
512bf94 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Use shell-neutral S2S sample path
bcd0fa4 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Remove Task 1 S2S README catalog row
55914d7 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Add store-ready telemetry to A365 S2S sample
8812f4a 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilc-microsoft
Remove implementation planning artifacts
9442404 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 25, 2026 21:04
@nikhilNava Nikhil Navakiran (nikhilNava) mentioned this pull request Sep 25, 2026
Merged
@github-actions

github-actions Bot commented Sep 25, 2026 •
edited
Loading

Copy link
Copy Markdown

Performance comparison

Threshold: regressions >15.0% on gating scenarios fail the build. Higher ops/s is better; positive Δ means the PR is slower.

Scenario Gating Baseline (ops/s) Candidate (ops/s) Δ % Status
azure_monitor_log yes 17,073.6 16,411.2 +4.04% ✅
azure_monitor_span yes 156,445.6 160,462.1 -2.50% ✅
otel_log no 19,304.1 18,929.0 +1.98% ✅
otel_span no 41,972.7 41,382.2 +1.43% ✅

Copilot AI reviewed Sep 25, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Add bounded MSAL request timeouts and correct the attempt-number examples in the documentation.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds a Store-ready Agent 365 S2S observability exporter sample with MSAL authentication, manual telemetry scopes, documentation, and regression tests.

Changes:

  • Implements S2S token resolution, validation, caching, and exporter logging.
  • Demonstrates Store-contract telemetry across A365 scopes.
  • Adds setup documentation, catalog discoverability, and contract tests.
File Summary
tests/​a365/​test_s2s_sample.py Span contract and resolver regression tests
samples/​a365/​s2s/​s2s_exporter.py S2S exporter and telemetry generation
samples/​a365/​s2s/​README.md Setup and Store-validation guidance
README.md Adds the sample to the examples table

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread samples/a365/s2s/s2s_exporter.py
Comment on lines +183 to +187
app = msal.ConfidentialClientApplication(
client_id=client_id,
client_credential=client_secret,
authority=authority,
)
@nikhilc-microsoft
Fix S2S sample validation
f942536 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nikhilNava Nikhil Navakiran (nikhilNava) added the skip-changelog Not required for changes to test files, github actions, etc. label Sep 25, 2026
@nikhilNava

Nikhil Navakiran (nikhilNava) commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Closing this replacement PR. The completed S2S sample changes have been moved to the original PR #217 as requested.

@nikhilNava

Nikhil Navakiran (nikhilNava) commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #217, which now contains the completed changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@hectorhdzg Hector Hernandez (hectorhdzg) Awaiting requested review from hectorhdzg hectorhdzg is a code owner
@JacksonWeber Jackson Weber (JacksonWeber) Awaiting requested review from JacksonWeber JacksonWeber is a code owner
@lzchen Leighton Chen (lzchen) Awaiting requested review from lzchen lzchen is a code owner
@MSNev Nev (MSNev) Awaiting requested review from MSNev MSNev is a code owner
@rads-1996 Radhika Gupta (rads-1996) Awaiting requested review from rads-1996 rads-1996 is a code owner
@jeremydvoss Jeremy Voss (jeremydvoss) Awaiting requested review from jeremydvoss jeremydvoss is a code owner

Assignees

No one assigned

Labels

skip-changelog Not required for changes to test files, github actions, etc.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nikhilNava @nikhilc-microsoft