ci: add pin-sync CI + regression guards for MCP install-script drift#63
Merged
Conversation
The template->install-script pin drift that broke directory_tree (#34) shipped because the sync test was never wired to run — the repo had no CI at all. - Add .github/workflows/agent-environment-tests.yml to run the pin-sync tests on every PR touching the template, install script, or tests. - Add tests/test_filesystem_pin.py: minimum-CalVer floor so the filesystem pin can't regress below the upstream fix (2025.12.18), which the sync test alone would miss when template and script are bumped down together. (from #57) - Close the reverse-drift blind spot in test_package_sync: the docstring claimed to catch stale packages left in the script, but only checked the template->script direction. Now asserts script-template is empty too. - Document the pin policy in services/agent-environment/README.md. (from #57) Supersedes the remainder of #57 (its version bumps already landed via 60ae87d). Co-Authored-By: Abhishek Enaguthi <enaguthiabhishek@gmail.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Credit to @Abhishek21g in #57
Why
The template→install-script pin drift that broke
directory_tree(#34) shipped to Docker images because the repo has no CI —tests/test_mcp_config_sync.pyexisted and does catch version mismatches, but nothing ever ran it. A perfect sync test prevents nothing if it isn't wired up.This closes that gap and hardens the tests, superseding the remainder of #57 (its version bumps already landed via
60ae87d; template↔script are fully in sync on main today, verified both directions).Changes
.github/workflows/agent-environment-tests.yml— runs the pin-sync tests on every PR (and push to main) that touches the template, install script, or tests. This is the piece that actually prevents recurrence.tests/test_filesystem_pin.py(from fix(agent-environment): sync MCP install script with template pins (#34) #57) — a minimum-CalVer floor for the filesystem pin. The sync test only enforcesscript == template; if both get bumped back to the broken@2025.11.25the bug returns silently. This guard rejects anything below the upstream fix (2025.12.18). Cleaned up the two Greptile nits (redundant assertion removed, upstream issue ref aligned to#3110).tests/test_mcp_config_sync.py— closed a reverse-drift blind spot: the docstring claimed to catch stale packages left in the script, but the code only checked template→script. Now also assertsscript − templateis empty (bloat / stale removals). Verified it fires with a negative test.README.md(from fix(agent-environment): sync MCP install script with template pins (#34) #57) — documents the pin policy.Verification
Both new guards confirmed to fire on regression (stale-package injection; pin rolled back to 2025.11.25).
Follow-up
#57 can be closed — its install-script sync already merged, and the useful remainder (test + docs) is carried here rebased and cleaned.
🤖 Generated with Claude Code
Greptile Summary
This PR wires up CI for the
agent-environmentservice and hardens the existing pin-sync tests to close the gap that let thedirectory_treeregression (#34) reach Docker images undetected..github/workflows/agent-environment-tests.yml: New workflow runspytest tests/on every PR or push tomainthat touches the template, install script, tests, or the workflow file itself — path filters are tight and correct.test_filesystem_pin.py: Adds a CalVer floor guard (>= 2025.12.18) so that even a perfectly-synced template+script pair can't silently regress to the broken@2025.11.25pin.test_mcp_config_sync.py: Closes the reverse-drift blind spot by also assertingscript − template = ∅, preventing stale packages from accumulating in the install script after they're removed from the template.Confidence Score: 5/5
Safe to merge — adds CI and test-only files with no changes to production code paths.
All three changed files are either a new GitHub Actions workflow or test code. The reverse-direction sync check is logically correct, and no production code is modified.
No files require special attention.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD PR[PR or push to main] -->|paths filter matches| TRIGGER[Workflow triggered] TRIGGER --> CHECKOUT[checkout v4] CHECKOUT --> PYSETUP[setup-python 3.12] PYSETUP --> INSTALL[pip install pytest] INSTALL --> RUN[pytest tests] RUN --> T1[test_mcp_config_sync] RUN --> T2[test_filesystem_pin] T1 --> FWD[Forward: template pkgs present in script] FWD -->|missing| FAIL1[AssertionError: missing packages] FWD -->|ok| REV[Reverse: script pkgs present in template] REV -->|extra| FAIL2[AssertionError: stale packages in script] REV -->|ok| PASS1[Sync OK] T2 --> READPIN[Read filesystem pin from template json] READPIN --> PARSEPIN[Parse CalVer tuple] PARSEPIN -->|below 2025.12.18| FAIL3[AssertionError: pin too old] PARSEPIN -->|meets minimum| PASS2[Pin OK]Reviews (2): Last reviewed commit: "Update README.md" | Re-trigger Greptile