Skip to content

SDLC_HARNESS - #23

Draft
aryansingh0012 wants to merge 17 commits into
eclipse-score:mainfrom
aryansingh0012:feat/add-sdlc-harness
Draft

aryansingh0012 wants to merge 17 commits into
eclipse-score:mainfrom
aryansingh0012:feat/add-sdlc-harness

Conversation

@aryansingh0012

@aryansingh0012 aryansingh0012 commented Sep 10, 2026

Copy link
Copy Markdown

SDLC_HARNESS

@FScholPer

Copy link
Copy Markdown
Contributor

looks empty

@FScholPer

Copy link
Copy Markdown
Contributor

@aryansingh0012 lets discuss next steps in our ai meeting to do not double the work. I started to build artifacts from our current meta model in eclipse-score/docs-as-code#826

@aryansingh0012
aryansingh0012 marked this pull request as draft September 15, 2026 13:47
@aryansingh0012 aryansingh0012 changed the title Merge pull request #1 from eclipse-score/main SDLC_HARNESS Sep 15, 2026

@FScholPer FScholPer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff (29 files, ~1.7k lines) against main. The stage-artifact
model, the loopback log and the evidence recording are useful and they do not collide with
context-discipline: this package produces the lifecycle trail, ours selects context.
Four concrete defects below, each with a suggested change where a mechanical fix exists.

1. CI is red for a mechanical reason — the inlined Python in
apm-marketplace-check.yml raises IndentationError: unexpected indent at
repository = os.environ["MCP_REPO"], so the remote-MCP smoke test never runs. Suggestion inline.

2. mcp.yml advertises a tool that does not existtrace_need is declared, but
serve.py::call_tool has no branch for it, so calling it returns Unknown tool: trace_need.
Since the sphinx-needs integration was removed in b336a87, I'd drop the declaration (and
the "sphinx-needs traceability" wording in description) rather than re-add the tool. For the
needs trace itself there is now a projection in metamodel-flow/context-discipline
(requirements_projection.pyRequirementsSource, typed nodes with derived_from,
satisfied_by, covers, partially_verifies, fully_verifies), so this is a good seam to
consume instead of re-implementing.

3. The review stage is unreachableSTAGE_REQUIREMENTS["review"] requires
harness/run.json, but the only file any tool writes is
harness/implementation-evidence.json (implementation_tracking.py::EVIDENCE_FILENAME).
check_stage_completeness(stage="review") can therefore never report complete. Suggestions
inline for both the code and the instructions file.

4. No tests — the package ships without any (b56e638, cf4d123 removed them) and
scripts/validate.sh only checks that files exist. For a server whose purpose is gating,
I'd want at least: path-traversal rejection in issue_directory, evidence replacement by
task_id, the status allowlist, and one check_stage_completeness case per stage.

One design question, not a review blocker — better suited to the AI meeting: STAGE_REQUIREMENTS
hardcodes a stage model, and requirement_ids in record_implementation_evidence are
unvalidated free strings. Both exist as data in the metamodel already (process workflows with
input/output/responsible/approved_by; needs with their link types). Deriving the gates
and validating the IDs against that data would remove the duplication we talked about.

Comment on lines 73 to 83
revision = os.environ["MCP_REF"]
repository = os.environ["MCP_REPO"]
for server in config["servers"].values():
args = server.get("args", [])
for index, argument in enumerate(args):
if argument.startswith("git+https://github.com/eclipse-score/mcp-servers#"):
if argument.startswith("git+https://github.com/") and "#" in argument:
subdirectory = argument.split("#", 1)[1]
args[index] = argument.replace(
"git+https://github.com/eclipse-score/mcp-servers#",
f"git+https://github.com/eclipse-score/mcp-servers@{revision}#",
1,
argument,
f"git+https://github.com/{repository}@{revision}#{subdirectory}",
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repository = ... is indented one level too deep and the rewrite loop below it is mis-indented as well, so the heredoc fails to parse before it can rewrite .vscode/mcp.json. argument.replace(argument, ...) is also a whole-string replace, so building the new value directly is clearer.

Suggested change
revision = os.environ["MCP_REF"]
repository = os.environ["MCP_REPO"]
for server in config["servers"].values():
args = server.get("args", [])
for index, argument in enumerate(args):
if argument.startswith("git+https://github.com/eclipse-score/mcp-servers#"):
if argument.startswith("git+https://github.com/") and "#" in argument:
subdirectory = argument.split("#", 1)[1]
args[index] = argument.replace(
"git+https://github.com/eclipse-score/mcp-servers#",
f"git+https://github.com/eclipse-score/mcp-servers@{revision}#",
1,
argument,
f"git+https://github.com/{repository}@{revision}#{subdirectory}",
)
revision = os.environ["MCP_REF"]
repository = os.environ["MCP_REPO"]
prefix = "git+https://github.com/"
for server in config["servers"].values():
args = server.get("args", [])
for index, argument in enumerate(args):
if argument.startswith(prefix) and "#" in argument:
subdirectory = argument.split("#", 1)[1]
args[index] = (
f"{prefix}{repository}@{revision}#{subdirectory}"
)

Comment on lines +76 to +85
- name: trace_need
description: Return a sphinx-needs trace chain, including linked needs and reverse references.
input_schema:
type: object
properties:
need_id:
type: string
needs_json_path:
type: string
required: [need_id, needs_json_path]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trace_need has no handler in serve.py::call_tool, so an agent that follows tools/list gets Unknown tool: trace_need. Dropping the declaration keeps the manifest honest after the sphinx-needs removal in b336a87.

Suggested change
- name: trace_need
description: Return a sphinx-needs trace chain, including linked needs and reverse references.
input_schema:
type: object
properties:
need_id:
type: string
needs_json_path:
type: string
required: [need_id, needs_json_path]


name: sdlc-harness
version: 0.1.0
description: MCP server for SDLC lifecycle management and sphinx-needs traceability

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sphinx-needs part was removed from the implementation, so the description promises something the server no longer does.

Suggested change
description: MCP server for SDLC lifecycle management and sphinx-needs traceability
description: MCP server for SDLC lifecycle stages and implementation evidence

"03-architecture.md",
"04-plan.yaml",
"05-tasks/",
"harness/run.json",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No tool writes harness/run.jsonrecord_implementation_evidence writes harness/implementation-evidence.json (EVIDENCE_FILENAME). As it stands, check_stage_completeness(issue_id, "review") always reports the review stage as incomplete.

Suggested change
"harness/run.json",
"harness/implementation-evidence.json",

2. Explain system boundaries and significant technical choices in `03-architecture.md`.
3. Convert the architecture into ordered work in `04-plan.yaml` and `05-tasks/`.
4. Remove draft/review markers only after the artifacts are consistent and reviewed.
5. Store actual implementation evidence in `harness/run.json` before review.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same filename mismatch in the guidance: the agent is told to store evidence in harness/run.json, but the tool it is supposed to call writes harness/implementation-evidence.json.

Suggested change
5. Store actual implementation evidence in `harness/run.json` before review.
5. Store actual implementation evidence with `record_implementation_evidence` (written to `harness/implementation-evidence.json`) before review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants