Skip to content

validate_blueprint_contract detects file-creation via magic prose phrases in description (affected_files drift) #167

@azalio

Description

@azalio

Summary

validate_blueprint_contract distinguishes "subtask creates a new file" from "subtask modifies an existing file" by string-matching magic phrases in the free-text subtask description. New-file subtasks raise an affected_files drift warning unless the prose happens to contain one of creates new / new file / introduces / adds new. This is brittle and couples a structural property (create vs modify) to natural-language wording.

mapify_version: 3.10.0

Observed

During /map-plan Step 5.6, four subtasks that legitimately create new modules were flagged:

ST-002: affected_files drift — none of ['python/pipeline/dynamic_k8s.py',
'python/test_dynamic_k8s.py'] exist under <repo>; verify the decomposer didn't
hallucinate file names. If this subtask CREATES the files from scratch, mark that
in the subtask description (phrases: 'creates new', 'new file', 'introduces',
'adds new') to silence this warning.

(same for ST-004 verification.py, ST-006 probe_planning.py, ST-007 probe_execution.py/probe_judge.py)

The only documented way to silence it was to inject a magic phrase, so I had to prepend "Creates new file(s). " to each description purely to satisfy the matcher.

Impact

  • False positives: a create-subtask phrased as "scaffold module X" / "implement new analyzer Y" still warns even though it clearly creates files.
  • False negatives risk: a modify subtask whose description happens to say "handles the new field" could be mis-read as a create and skip the expected-present check.
  • Pollutes subtask descriptions with boilerplate written for the parser, not the reader.

Expected

Represent create-vs-modify structurally, not via prose. E.g.:

  • A per-file op on affected_files: [{"path": "...", "op": "create"}, {"path": "...", "op": "modify"}], or
  • two arrays creates_files: [...] / modifies_files: [...].

Then the validator checks create paths as expected-absent and modify paths as expected-present, with no dependence on description wording. (The task-decomposer prompt already knows which files are new, so it can emit the structured field directly.)

Environment

  • mapify 3.10.0, validate_blueprint_contract via .map/scripts/map_step_runner.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions