feat: dogfood APM on microsoft/apm -- .apm/ primitives, compile, --check gate#842
Draft
sergio-sisternes-epam wants to merge 10 commits intomicrosoft:mainfrom
Draft
feat: dogfood APM on microsoft/apm -- .apm/ primitives, compile, --check gate#842sergio-sisternes-epam wants to merge 10 commits intomicrosoft:mainfrom
sergio-sisternes-epam wants to merge 10 commits intomicrosoft:mainfrom
Conversation
…istic sort base_dir Two foundational fixes required before apm compile --check can enforce a strict stdout contract and round-trip determinism: 1. context_optimizer.py: replace bare print() with logger.debug(). Timing output is opt-in via --verbose/DEBUG level and must never leak to default stdout. 2. template_builder.py: build_conditional_sections() now takes a required base_dir parameter. Previously Path.cwd() was used in the sort key and relative-path display, making compile output depend on the user's current working directory. Sort order must be deterministic regardless of where apm is invoked from. Updated the single caller in agents_compiler.py to pass self.base_dir. Added unit tests covering logger routing (caplog vs capsys) and deterministic sort behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The apm compile --check flag (landing in a later commit) will run in Tier 1 CI and read user-authored markdown from .apm/ primitives. _resolve_path previously accepted absolute paths unguarded and performed naive base_path/path joins with no traversal or symlink-escape checks. This is the first path_security import into the compilation/ subsystem. New contract (3-gate fail-closed): 1. Absolute paths are rejected outright (return None). 2. validate_path_segments rejects '..' at parse time; './' is allowed since it is legitimate in markdown links (allow_current_dir=True). 3. ensure_path_within resolves symlinks and asserts containment after the join; the resolved path is returned on success. PathTraversalError / OSError / ValueError all map to None, which the caller surfaces as 'Referenced file not found' via the existing validate_link_targets flow. Three other pre-existing path-traversal gaps in the compilation subsystem (apm.yml output_path, applyTo patterns, full path_security integration across compile) are explicitly deferred to a labelled security follow-up issue filed pre-merge. Supply-chain-security-expert review: 0 blockers, 0 majors, 3 minors (all non-exploitable edge cases filed under the follow-up). Tests: TestResolvePathSecurity covers absolute rejection, traversal at depth, current-directory allowance, symlink escape, and an integration check via validate_link_targets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…instructions (microsoft#792) Closes microsoft#792. Instructions with empty or missing applyTo frontmatter were previously silently dropped by both the distributed compiler and the single-file template builder. They now aggregate into .github/copilot-instructions.md for the vscode-family targets (vscode, copilot, agents, opencode, codex, all, minimal). Design (approved by python-architect pre-implementation): - New AgentsCompiler._compile_copilot_instructions() sibling emitter called between _compile_agents_md and _compile_claude_md in the target-routing block. Returns Optional[CompilationResult]; None when no root-scoped instructions exist so no empty file is written and _merge_results stays clean. - New should_compile_copilot_instructions() predicate in target_detection so the gate can diverge from should_compile_agents_md later if needed. - New build_root_sections() helper in template_builder mirrors build_conditional_sections but without pattern headers; filters empty-applyTo instructions, deterministic sort by portable_relpath(path, base_dir). - Hardcoded path .github/copilot-instructions.md (GitHub Copilot convention); no new CompilationConfig field needed. - Prefixed stat key 'copilot_instructions_written' to avoid _merge_results collisions. Also standardises the generated-file header across all three emitters. New constant GENERATED_HEADER in compilation/constants.py: Replaces the inconsistent AGENTS.md ('from distributed .apm/ primitives'), CLAUDE.md ('Generated by APM CLI') and template-builder variants. Updates the target-description strings in target_detection so apm init / status output reflects the new file. Tests: new tests/unit/compilation/test_copilot_instructions.py with 12 cases covering mixed fixture, empty case, deterministic sort, round-trip stability, header presence, source attribution, dry-run mode, target gating, and an integration test asserting both AGENTS.md and copilot-instructions.md are produced with the correct content split. Existing tests updated for the header standardisation. Full suite: 4804 passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a CI-friendly --check mode to `apm compile` that compares the expected compiler output against on-disk generated files and exits: 0 – all outputs match, 1 – drift or stale files, 2 – unrecoverable. Implementation highlights: - New CATEGORY_DRIFT diagnostic level with drift() / drift_count - AgentsCompiler.preview_all_outputs() dry-runs the full pipeline and returns Dict[Path, str] without writing to disk - _run_check() / _render_drift_report() in the CLI compare previews to disk, detect stale well-known outputs, and emit a concise report - --check implies --local-only and is mutually exclusive with --validate, --watch, --dry-run, --single-agents, --clean - Remove 'No applyTo pattern specified' validation warning – root-scoped instructions are now first-class primitives Co-authored-by: Claude <noreply@anthropic.com>
Creates the first APM manifest for microsoft/apm itself and ports all
26 existing agent primitives from .github/** into .apm/:
.apm/instructions/ (9) - 8 copied from .github/instructions/ plus
new contributing.instructions.md aggregating
.github/copilot-instructions.md as a
root-scoped (applyTo-less) instruction.
.apm/agents/ (10) - byte-identical copies of .github/agents/.
.apm/skills/ (8) - byte-identical copies of .github/skills/.
This commit adds sources only; it does NOT regenerate .github/** outputs.
apm compile --check now reports drift for every .github/** file, which
the next commit resolves by running `apm compile` for real.
Refs microsoft#695, microsoft#792.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removes legacy .github/instructions/ and .github/agents/ source trees (ported to .apm/ in the previous commit) and regenerates all agent-tool outputs from .apm/ primitives: - Root AGENTS.md and CLAUDE.md (aggregated) - Distributed AGENTS.md and CLAUDE.md in .github/, docs/src/, src/apm_cli/, src/apm_cli/integration/, tests/ - .github/copilot-instructions.md (root-scoped instructions) Adds compilation.exclude patterns to apm.yml to scope discovery to first-party primitives (excludes tests/, templates/, packages/, build/, docs/node_modules/). Marks generated outputs as linguist-generated in .gitattributes so GitHub's diff views collapse them by default and language stats exclude them. Removes AGENTS.md from .gitignore since it is now a committed compile artifact. Closes microsoft#695, closes microsoft#792. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Runs 'apm compile --check' on every PR and merge_group event to ensure AGENTS.md, CLAUDE.md, and .github/copilot-instructions.md stay in sync with .apm/ primitives. Read-only check: exit 0 when outputs match, exit 1 on drift. The error message directs contributors to run 'apm compile' locally and commit the regenerated outputs. Part of the microsoft#695 / microsoft#792 dogfooding PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- CONTRIBUTING.md: new "Recompiling agent outputs" section explaining .apm/ is source of truth and the compile workflow - docs/.../cli-commands.md: document apm compile --check flag and exit-code contract - docs/.../manifest-schema.md: document compilation.exclude patterns - CHANGELOG.md: entries under [Unreleased] for --check, copilot- instructions compile target, root-scoped instructions, dogfood switch, and link_resolver containment fix README callout proposal written to session files for user approval (per doc-sync rule 2). Part of the microsoft#695 / microsoft#792 dogfooding PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
APM uses APM to manage its own agent primitives. The new section points readers at the .apm/ source tree, the apm compile workflow, and the CI gate -- linking to CONTRIBUTING.md for the full procedure. Part of the microsoft#695 / microsoft#792 dogfooding PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sergio-sisternes-epam
force-pushed
the
feat/792-dogfood-apm
branch
2 times, most recently
from
e4d1001 to
1ccc60d
Compare
Compiled outputs fall into two groups:
- GitHub-hosted consumers (Copilot in PR/chat, Agentic Workflows, Cloud Agents)
read AGENTS.md and .github/** directly from the repo -- no build step runs
on their side, so we MUST commit these files.
- Claude Code runs exclusively on a developer's machine. Contributors can
regenerate CLAUDE.md locally via apm compile.
This commit implements that split:
- .gitignore: gitignore CLAUDE.md / **/CLAUDE.md (with exceptions for template
fixtures, docs-site pages, and test fixtures).
- Untrack the 5 previously-tracked CLAUDE.md files.
- .gitattributes: drop CLAUDE.md linguist-generated entries (no longer tracked);
keep AGENTS.md and copilot-instructions.md markers.
- .github/workflows/ci.yml: scope the drift gate to 'apm compile -t copilot
--check' so CI only asserts sync on the outputs we actually commit.
- CONTRIBUTING.md / README.md / cli-commands.md: document the policy
explicitly ('pre-built for GitHub Copilot; other platforms run apm compile
on checkout') including a note on how local -t copilot --check interacts
with a full-target local compile.
- CHANGELOG.md: clarify the Unreleased dogfood entry.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sergio-sisternes-epam
force-pushed
the
feat/792-dogfood-apm
branch
from
1ccc60d to
13dcdbc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #695, closes #792. Supersedes closed #754.
Dogfoods APM on
microsoft/apmitself:.github/instructions/**+.github/agents/**into a.apm/source tree (9 instructions, 10 agents, 8 skills).apm.ymlat the repo root (name: apm-cli,target: all) drives compilation.apm compilenow regenerates every agent-tool output: rootAGENTS.md+CLAUDE.md, distributedAGENTS.md+CLAUDE.mdunder.github/,docs/src/,src/apm_cli/,src/apm_cli/integration/,tests/, and the new.github/copilot-instructions.md.apm compile --checkin Tier 1 so hand-edits to generated files or forgotten recompiles fail the build with a clear remediation hint.What's new
feat(compile):.github/copilot-instructions.mdemitter (apm compile should emit .github/copilot-instructions.md (dogfood gap) #792). Root-scoped instructions (noapplyTo) now compile into a single aggregated file that Copilot reads natively. Root-scoping is a first-class primitive -- theNo 'applyTo'warning was a bug masquerading as a lint.feat(compile):apm compile --checkflag. Read-only drift verification. Exit codes0match /1drift /2unrecoverable error. Drift report to stderr (stdout reserved for future--json), silent on success. Distinguishes content drift (apm compile) from stale files (apm compile --clean).fix(compilation):link_resolver._resolve_pathcontainment. Three independent gates (is_absolute,validate_path_segments,ensure_path_within) fail closed on traversal, symlink escape, and absolute paths. Supply-chain-security review: 0 blockers.fix(compilation): route timing output through logger and use deterministicbase_dirfor sort stability across platforms.How the dogfood works
.apm/**.apm compilelocally; commit the regenerated outputs in the same PR.apm compile --check-- drift fails the build and tells the contributor exactly which command to run.See
CONTRIBUTING.md-- Recompiling agent outputs and the new## Dogfoodingsection in the README.Commit structure
fbaa46979389ablink_resolver._resolve_path8372876.github/copilot-instructions.md(#792)7f8f95d--checkflag1ce108fapm.yml+ populate.apm/with repo primitives74b772bapm compile6d09d4eapm compile --checkgate to Tier 18f6aded--checkand the.apm/workflow125c48b## DogfoodingsectionEach commit is atomic and green.
Validation
uv run pytest tests/unit tests/test_console.py).uv run apm compile --checkexits 0 on HEAD.AGENTS.md/CLAUDE.mdpairs +.github/copilot-instructions.md)..gitattributesmarks all generated outputslinguist-generated=trueso GitHub diffs collapse them and language stats ignore them.Reviews run
Before opening this PR the following reviews were run and all findings addressed:
--checkcontract, exit codes, stderr, symbols).compilation.excludeinapm.yml.Follow-ups (filed, out of scope here)
.chatmode.mdprimitive type #840 -- deprecate and remove legacy.chatmode.mdprimitive type. No primitive in this repo uses it; the code paths and patterns add noise.link_resolver._resolve_pathinput guards (empty / whitespace-only strings) and expand test coverage for NUL bytes, backslash traversal, andfile://URIs.Known limitations (documented, not blocking)
--checkstale detection uses a hardcoded_WELL_KNOWN_OUTPUTSlist. DistributedAGENTS.md/CLAUDE.mdin subdirs need aTODO(stale-distributed-agents)follow-up once the distributed compiler exposes a "possible outputs" query. Content drift is fully covered; stale-only detection is the gap.verbose_detailin the drift report routes through Rich stdout (pre-existing infra constraint), not stderr. Accepted by cli-logging-expert.Breaking changes
None for consumers of APM.
For
microsoft/apmcontributors:.github/instructions/**and.github/agents/**no longer exist as source directories. Edit under.apm/instead, thenapm compile. CONTRIBUTING.md covers the workflow.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com