SDLC_HARNESS - #23
SDLC_HARNESS#23aryansingh0012 wants to merge 17 commits into
Conversation
|
looks empty |
Signed-off-by: Aryan Singh <aryan.s@ltts.com>
|
@aryansingh0012 lets discuss next steps in our ai meeting to do not double the work. I started to build artifacts from our current meta model in eclipse-score/docs-as-code#826 |
FScholPer
left a comment
There was a problem hiding this comment.
Reviewed the full diff (29 files, ~1.7k lines) against main. The stage-artifact
model, the loopback log and the evidence recording are useful and they do not collide with
context-discipline: this package produces the lifecycle trail, ours selects context.
Four concrete defects below, each with a suggested change where a mechanical fix exists.
1. CI is red for a mechanical reason — the inlined Python in
apm-marketplace-check.yml raises IndentationError: unexpected indent at
repository = os.environ["MCP_REPO"], so the remote-MCP smoke test never runs. Suggestion inline.
2. mcp.yml advertises a tool that does not exist — trace_need is declared, but
serve.py::call_tool has no branch for it, so calling it returns Unknown tool: trace_need.
Since the sphinx-needs integration was removed in b336a87, I'd drop the declaration (and
the "sphinx-needs traceability" wording in description) rather than re-add the tool. For the
needs trace itself there is now a projection in metamodel-flow/context-discipline
(requirements_projection.py → RequirementsSource, typed nodes with derived_from,
satisfied_by, covers, partially_verifies, fully_verifies), so this is a good seam to
consume instead of re-implementing.
3. The review stage is unreachable — STAGE_REQUIREMENTS["review"] requires
harness/run.json, but the only file any tool writes is
harness/implementation-evidence.json (implementation_tracking.py::EVIDENCE_FILENAME).
check_stage_completeness(stage="review") can therefore never report complete. Suggestions
inline for both the code and the instructions file.
4. No tests — the package ships without any (b56e638, cf4d123 removed them) and
scripts/validate.sh only checks that files exist. For a server whose purpose is gating,
I'd want at least: path-traversal rejection in issue_directory, evidence replacement by
task_id, the status allowlist, and one check_stage_completeness case per stage.
One design question, not a review blocker — better suited to the AI meeting: STAGE_REQUIREMENTS
hardcodes a stage model, and requirement_ids in record_implementation_evidence are
unvalidated free strings. Both exist as data in the metamodel already (process workflows with
input/output/responsible/approved_by; needs with their link types). Deriving the gates
and validating the IDs against that data would remove the duplication we talked about.
| revision = os.environ["MCP_REF"] | ||
| repository = os.environ["MCP_REPO"] | ||
| for server in config["servers"].values(): | ||
| args = server.get("args", []) | ||
| for index, argument in enumerate(args): | ||
| if argument.startswith("git+https://github.com/eclipse-score/mcp-servers#"): | ||
| if argument.startswith("git+https://github.com/") and "#" in argument: | ||
| subdirectory = argument.split("#", 1)[1] | ||
| args[index] = argument.replace( | ||
| "git+https://github.com/eclipse-score/mcp-servers#", | ||
| f"git+https://github.com/eclipse-score/mcp-servers@{revision}#", | ||
| 1, | ||
| argument, | ||
| f"git+https://github.com/{repository}@{revision}#{subdirectory}", | ||
| ) |
There was a problem hiding this comment.
repository = ... is indented one level too deep and the rewrite loop below it is mis-indented as well, so the heredoc fails to parse before it can rewrite .vscode/mcp.json. argument.replace(argument, ...) is also a whole-string replace, so building the new value directly is clearer.
| revision = os.environ["MCP_REF"] | |
| repository = os.environ["MCP_REPO"] | |
| for server in config["servers"].values(): | |
| args = server.get("args", []) | |
| for index, argument in enumerate(args): | |
| if argument.startswith("git+https://github.com/eclipse-score/mcp-servers#"): | |
| if argument.startswith("git+https://github.com/") and "#" in argument: | |
| subdirectory = argument.split("#", 1)[1] | |
| args[index] = argument.replace( | |
| "git+https://github.com/eclipse-score/mcp-servers#", | |
| f"git+https://github.com/eclipse-score/mcp-servers@{revision}#", | |
| 1, | |
| argument, | |
| f"git+https://github.com/{repository}@{revision}#{subdirectory}", | |
| ) | |
| revision = os.environ["MCP_REF"] | |
| repository = os.environ["MCP_REPO"] | |
| prefix = "git+https://github.com/" | |
| for server in config["servers"].values(): | |
| args = server.get("args", []) | |
| for index, argument in enumerate(args): | |
| if argument.startswith(prefix) and "#" in argument: | |
| subdirectory = argument.split("#", 1)[1] | |
| args[index] = ( | |
| f"{prefix}{repository}@{revision}#{subdirectory}" | |
| ) |
| - name: trace_need | ||
| description: Return a sphinx-needs trace chain, including linked needs and reverse references. | ||
| input_schema: | ||
| type: object | ||
| properties: | ||
| need_id: | ||
| type: string | ||
| needs_json_path: | ||
| type: string | ||
| required: [need_id, needs_json_path] |
There was a problem hiding this comment.
trace_need has no handler in serve.py::call_tool, so an agent that follows tools/list gets Unknown tool: trace_need. Dropping the declaration keeps the manifest honest after the sphinx-needs removal in b336a87.
| - name: trace_need | |
| description: Return a sphinx-needs trace chain, including linked needs and reverse references. | |
| input_schema: | |
| type: object | |
| properties: | |
| need_id: | |
| type: string | |
| needs_json_path: | |
| type: string | |
| required: [need_id, needs_json_path] |
|
|
||
| name: sdlc-harness | ||
| version: 0.1.0 | ||
| description: MCP server for SDLC lifecycle management and sphinx-needs traceability |
There was a problem hiding this comment.
The sphinx-needs part was removed from the implementation, so the description promises something the server no longer does.
| description: MCP server for SDLC lifecycle management and sphinx-needs traceability | |
| description: MCP server for SDLC lifecycle stages and implementation evidence |
| "03-architecture.md", | ||
| "04-plan.yaml", | ||
| "05-tasks/", | ||
| "harness/run.json", |
There was a problem hiding this comment.
No tool writes harness/run.json — record_implementation_evidence writes harness/implementation-evidence.json (EVIDENCE_FILENAME). As it stands, check_stage_completeness(issue_id, "review") always reports the review stage as incomplete.
| "harness/run.json", | |
| "harness/implementation-evidence.json", |
| 2. Explain system boundaries and significant technical choices in `03-architecture.md`. | ||
| 3. Convert the architecture into ordered work in `04-plan.yaml` and `05-tasks/`. | ||
| 4. Remove draft/review markers only after the artifacts are consistent and reviewed. | ||
| 5. Store actual implementation evidence in `harness/run.json` before review. |
There was a problem hiding this comment.
Same filename mismatch in the guidance: the agent is told to store evidence in harness/run.json, but the tool it is supposed to call writes harness/implementation-evidence.json.
| 5. Store actual implementation evidence in `harness/run.json` before review. | |
| 5. Store actual implementation evidence with `record_implementation_evidence` (written to `harness/implementation-evidence.json`) before review. |
SDLC_HARNESS