fix: harden commit-semantic follow-up

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## v1.0.1 - 2026-03-23
+
+### Changed
+- Reworked the commit history pipeline around `commit-extract` monthly JSONL output and the domain-based `commit-semantic` 5-stage flow.
+- Switched `commit-semantic` to LLM-only discover/classify semantics with explicit runtime provenance in `summary.json`.
+- Updated `repo-structure` hotspot ingestion to read the new JSONL-based upstream artifacts.
+
+### Fixed
+- Hardened LLM classification trust boundaries by rejecting unknown domains and malformed IDs.
+- Made SHA-to-file-path parsing delimiter-safe to avoid silent mis-parsing of hex-like file names.
+- Aligned semantic hotspot consumption with the actual `domains-aggregated.jsonl` schema.
+
 ## v1.0.0
 
 Stable release. 108 tests passing across all safety boundaries.

CLAUDE.md

@@ -37,8 +37,8 @@ Semantic Harness is a **Claude Code skill repository** for extracting structured
 # Install
 pip install -e ".[test]"
 
-# Tests (46 passing)
-pytest tests/test_system.py -q
+# Tests
+pytest tests -q
 
 # Skill commands (via Claude Code)
 /semantic-fact-pipeline    # discover → review → refine → baseline

README.md

@@ -85,31 +85,36 @@ Requires: semantic assets from capability 2.
 
 ---
 
-### 4. commit-semantic — Git History → Domain Cases
+### 4. commit — Git History Analysis
 
-Extracts structured semantic cases from git commit history. Produces deduplicated, pattern-aggregated case libraries for few-shot samples, rule extraction, and training data.
+The commit capability currently has two linked stages:
 
-**Pipeline:**
-```
-/commit-semantic-pipeline 最近 50 个 commit
-/commit-semantic-pipeline HEAD~100..HEAD，排除 config 目录
-/commit-semantic-pipeline 最近一个月，增量模式
-```
+**A. `commit-extract`**
+- Reads git history
+- Produces structured monthly JSONL artifacts in `data/commit-extract/YYYY-MM.jsonl`
+- Output includes `sections`, `rules_invariants`, and commit-level metadata
 
-**Individual steps:**
+**B. `commit-semantic`**
+- Consumes `data/commit-extract/*.jsonl`
+- Runs a 5-stage pipeline: `discover → ingest → aggregate → distill → export`
+- Produces domain-oriented outputs in `data/commit-semantic/`
+
+**Commands:**
 ```
-/commit-semantic-collect   # git history → semantic_case_inputs/
-/commit-semantic-generate  # semantic_case_inputs/ → semantic_cases/
-/commit-semantic-export    # semantic_cases/ → exports/ (dedup + patterns)
+/commit-extract run
+/commit-semantic run
 ```
 
-**Python API:**
-```python
-from src.commit_semantic.pipeline import run_pipeline
-run_pipeline(repo_path=".", commit_range="HEAD~50..HEAD", executor=my_llm)
+**Key outputs:**
+```
+data/commit-extract/YYYY-MM.jsonl
+data/commit-semantic/domains.json
+data/commit-semantic/domains-aggregated.jsonl
+data/commit-semantic/canonical-demands.jsonl
+data/commit-semantic/summary.json
 ```
 
-→ Details: `README-commit-semantic.md`, `docs/commit-semantic/user-guide.md`
+→ Details: `skills/commit-extract/SKILL.md`, `skills/commit-semantic/SKILL.md`
 
 ---
 
@@ -134,10 +139,11 @@ skills/                    # skill definitions (SKILL.md per skill)
   semantic-fact-pipeline/  # capability 1 pipeline
   semantic-pipeline/       # capability 2 pipeline
   demand-pipeline/         # capability 3 pipeline
-  commit-semantic-pipeline/# capability 4 pipeline
+  commit-extract/          # commit history extraction
+  commit-semantic/         # commit domain aggregation pipeline
   semantic-extract/        # capability 5: commit + rules extraction
   semantic-*/              # fact + semantic individual skills
-  commit-semantic-*/       # git history individual skills
+  commit-semantic-*/       # legacy / transitional git-history docs and helpers
 
 src/                       # Python runtime
   semantic/                # semantic layer implementation

docs/commit-semantic/skills-reference.md

@@ -1,6 +1,9 @@
 # commit-semantic Skills 参考
 
-本文档描述三个 Claude Code skill 的接口与行为。这些 skill 在 Claude Code 对话框中通过 `/` 命令调用，由 Claude 解析自然语言参数后执行。
+本文档描述旧版 `commit-semantic-collect / generate / export` 三段式 skill 接口。
+
+> 注意：当前推荐入口已经切换为：`/commit-extract run` → `/commit-semantic run`。
+> 本文档保留作为历史接口参考，不是当前主路径说明。
 
 ---
 

skills/commit-semantic/run.py

@@ -557,6 +557,11 @@ def _apply_classify_responses(
         classified = 0
         classified_ids: set[int] = set()
         staged_units = [dict(unit) for unit in current_units]
+        allowed_domains = {
+            domain.get("domain")
+            for domain in self._load_domains()
+            if domain.get("domain")
+        }
         for response in llm_responses:
             mapping = parse_llm_classifications(response)
             if not mapping:
@@ -570,7 +575,14 @@ def _apply_classify_responses(
                 print("  ! LLM classification batch returned invalid output")
                 return False
             for id_str, domain in mapping.items():
-                idx = int(id_str)
+                if not isinstance(domain, str) or domain not in allowed_domains:
+                    print(f"  ! LLM classification returned unknown domain: {domain}")
+                    return False
+                try:
+                    idx = int(id_str)
+                except (TypeError, ValueError):
+                    print(f"  ! LLM classification returned malformed id: {id_str}")
+                    return False
                 if 0 <= idx < len(indices) and indices[idx] < len(staged_units):
                     staged_units[indices[idx]]["domain"] = domain
                     classified += 1

skills/repo_structure/run.py

@@ -320,13 +320,12 @@ def _aggregate_hotspots(
 
         for domain in aggregated_domains[:10]:
             domain_name = domain.get("domain") or domain.get("domain_id") or "unknown"
-            commit_count = domain.get("commit_count") or len(domain.get("commit_shas", []) or [])
-            file_list = domain.get("file_paths") or domain.get("files") or []
+            distinct_commits = domain.get("distinct_commits", 0)
             hotspots.append({
                 "fact_id": str(uuid.uuid4()),
                 "fact_type": "hotspot_signal",
                 "domain": "semantic_domain",
-                "statement": f"Domain '{domain_name}' is a semantic hotspot across {commit_count} commits",
+                "statement": f"Domain '{domain_name}' is a semantic hotspot across {distinct_commits} commits",
                 "confidence": "confirmed",
                 "status": "active",
                 "repo_snapshot_commit": head,
@@ -339,8 +338,7 @@ def _aggregate_hotspots(
                     "stable_ref": f"domain:{domain_name}",
                     "rationale": "From commit-semantic aggregated domain output",
                 }],
-                "commit_count": commit_count,
-                "files": sorted(str(path) for path in file_list) if isinstance(file_list, list) else [],
+                "distinct_commits": distinct_commits,
             })
 
         return hotspots

src/commit_semantic/domain_utils.py

@@ -149,8 +149,9 @@ def build_sha_file_map(repo_path: str, shas: list[str]) -> tuple[dict[str, list[
     if not shas:
         return {}, True
 
+    sha_marker = "__SEMANTIC_HARNESS_SHA__="
     result = subprocess.run(
-        ["git", "log", "--name-only", "--format=%H", "--stdin", "--no-walk"],
+        ["git", "log", "--name-only", f"--format={sha_marker}%H", "--stdin", "--no-walk"],
         input="\n".join(shas),
         capture_output=True, text=True, cwd=repo_path,
     )
@@ -160,12 +161,12 @@ def build_sha_file_map(repo_path: str, shas: list[str]) -> tuple[dict[str, list[
 
     sha_map: dict[str, list[str]] = {}
     current_sha = None
-    for line in result.stdout.splitlines():
-        line = line.strip()
+    for raw_line in result.stdout.splitlines():
+        line = raw_line.strip()
         if not line:
             continue
-        if len(line) == 40 and all(c in "0123456789abcdef" for c in line):
-            current_sha = line
+        if line.startswith(sha_marker):
+            current_sha = line[len(sha_marker):]
             sha_map[current_sha] = []
         elif current_sha:
             sha_map[current_sha].append(line)

tests/e2e/test_commit_semantic.py

@@ -1495,6 +1495,88 @@ def test_ambiguous_non_path_signals_require_orchestration(self, tmp_path):
         assert state.metadata["needs_llm_classify"] == 1
         assert not (semantic_dir / "units" / "all.jsonl").exists()
 
+    def test_classify_rejects_unknown_domain(self, tmp_path):
+        mod = load_commit_semantic_module()
+        semantic_dir = tmp_path / "data" / "commit-semantic"
+        semantic_dir.mkdir(parents=True)
+        save_json(
+            {
+                "domains": [
+                    {
+                        "domain": "auth",
+                        "description": "Authentication and sessions",
+                        "paths": ["src/auth/"],
+                        "keywords": ["auth", "login"],
+                    }
+                ]
+            },
+            str(semantic_dir / "domains.json"),
+        )
+        mod.SEMANTIC_OUTPUT = semantic_dir
+
+        from src.harness_state import HarnessState
+        runner = mod.CommitSemanticRunner()
+        state = HarnessState(
+            stage="init",
+            metadata={
+                "completed_stages": [],
+                "artifacts_written": [],
+                "status": "ok",
+                "classify_unit_indices": [0],
+            },
+        )
+        units = [{"summary": "Add login"}]
+
+        ok = runner._apply_classify_responses(
+            [json.dumps([{"id": "0", "domain": "unknown"}])],
+            state,
+            units=units,
+        )
+
+        assert ok is False
+        assert "domain" not in units[0]
+
+    def test_classify_rejects_malformed_id(self, tmp_path):
+        mod = load_commit_semantic_module()
+        semantic_dir = tmp_path / "data" / "commit-semantic"
+        semantic_dir.mkdir(parents=True)
+        save_json(
+            {
+                "domains": [
+                    {
+                        "domain": "auth",
+                        "description": "Authentication and sessions",
+                        "paths": ["src/auth/"],
+                        "keywords": ["auth", "login"],
+                    }
+                ]
+            },
+            str(semantic_dir / "domains.json"),
+        )
+        mod.SEMANTIC_OUTPUT = semantic_dir
+
+        from src.harness_state import HarnessState
+        runner = mod.CommitSemanticRunner()
+        state = HarnessState(
+            stage="init",
+            metadata={
+                "completed_stages": [],
+                "artifacts_written": [],
+                "status": "ok",
+                "classify_unit_indices": [0],
+            },
+        )
+        units = [{"summary": "Add login"}]
+
+        ok = runner._apply_classify_responses(
+            [json.dumps([{"id": "not-an-int", "domain": "auth"}])],
+            state,
+            units=units,
+        )
+
+        assert ok is False
+        assert "domain" not in units[0]
+
 
 if __name__ == "__main__":
     pytest.main([__file__, "-v"])

tests/test_commit_semantic_domain.py

@@ -74,6 +74,24 @@ def test_git_failure(self, tmp_path):
         assert ok is False
         assert sha_map == {"abc123": []}
 
+    def test_40_char_hex_file_path_is_not_treated_as_commit_header(self, monkeypatch):
+        """Hex-looking file paths stay attached to the current commit."""
+        repo = str(Path(__file__).parent.parent)
+        sha = "a" * 40
+        hex_path = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"
+
+        class Completed:
+            returncode = 0
+            stdout = f"__SEMANTIC_HARNESS_SHA__={sha}\n{hex_path}\nsrc/normal.py\n"
+            stderr = ""
+
+        monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: Completed())
+
+        sha_map, ok = build_sha_file_map(repo, [sha])
+
+        assert ok is True
+        assert sha_map == {sha: [hex_path, "src/normal.py"]}
+
 
 # ---------------------------------------------------------------
 # T12-T14: assign_domain_by_path

tests/test_repo_structure.py

@@ -162,7 +162,7 @@ def test_hotspot_consumes_commit_semantic(self, tmp_path, monkeypatch):
             json.dumps({"sha": "abc", "file_paths": ["src/hermes/registry.py", "src/hermes/registry.py"]}) + "\n"
         )
         (tmp_path / "data/commit-semantic/domains-aggregated.jsonl").write_text(
-            json.dumps({"domain": "registry", "commit_count": 2, "file_paths": ["src/hermes/registry.py"]}) + "\n"
+            json.dumps({"domain": "registry", "distinct_commits": 2, "count": 3}) + "\n"
         )
 
         from src.harness_state import HarnessState
@@ -181,6 +181,12 @@ def test_hotspot_consumes_commit_semantic(self, tmp_path, monkeypatch):
         assert "metadata" in data
         assert "facts" in data
 
+        semantic_fact = next(fact for fact in data["facts"] if fact["domain"] == "semantic_domain")
+        assert semantic_fact["distinct_commits"] == 2
+        assert semantic_fact["statement"] == "Domain 'registry' is a semantic hotspot across 2 commits"
+        assert "commit_count" not in semantic_fact
+        assert "files" not in semantic_fact
+
 
 class TestExtract:
     def test_extract_produces_codebase_map(self, tmp_path, monkeypatch):
@@ -336,7 +342,7 @@ def test_full_pipeline_produces_baseline(self, tmp_path, monkeypatch):
 
         # commit-semantic aggregated domains
         (tmp_path / "data/commit-semantic/domains-aggregated.jsonl").write_text(
-            json.dumps({"domain": "registry", "commit_count": 2, "file_paths": ["src/hermes/registry.py"]}) + "\n"
+            json.dumps({"domain": "registry", "distinct_commits": 2, "count": 3}) + "\n"
         )
 
         from src.harness_state import HarnessState