feat(semantic): Layer 2 — rules-based semantic viewer - #36
Merged
Merged
Conversation
Add the Layer 2 "Semantic viewer (rules-centric)" from AGENTS.md §10 as a self-contained `semantic` module. Turns markdown into a `DocumentModel` and a `SemanticPanel` using deterministic rules only (no LLM): markdown -> parser::BlockTree -> analyzer(rules) -> DocumentModel -> SemanticPanel - parser: SourceRange/LineIndex/BlockTree via pulldown-cmark into_offset_iter - model: DocumentType, Classified<T>, BlockClass, DocumentModel (§4.2) - analyzer: doctype (frontmatter/filename/heading heuristics), block_class, code intent, table semantics (status column), task extraction - panel: outline / TODO / risk / open-questions side panel, each entry with a SourceRangeLink back to the source (not yet UI IR, per roadmap) Deliverables per AGENTS.md Layer 2: DocumentModel, rules doctype inference, SourceRangeLink. Interference note: kept entirely additive under `src/semantic/` (only a single `mod semantic;` line added to main.rs; no Cargo.toml/dep changes) so it does not collide with the parallel Layer 1 workspace migration / SourceRange parser in `layer1-sourcerange-parser`. On integration the `semantic::parser` foundation is expected to be replaced by `mdpeek-core::parser::BlockTree`. 124 unit tests cover parsing, classification and panel building; clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QcBxPCWg3P2QwyqMGpTyfp
takeshiD
marked this pull request as ready for review
July 5, 2026 00:07
Layer 1 (#12–#20) merged to main: workspace split into fine-grained crates (mdpeek-gfm/parser/render-html/render-term/watcher/server) and a SourceRange- aware BlockTree in mdpeek-parser. Integrate Layer 2 on top of that foundation: - Move the Layer 2 semantic analyser out of the `mdpeek` binary (src/semantic/) into a new workspace crate `crates/mdpeek-analyzer`, matching Layer 1's fine-grained crate layout. - Delete the duplicate SourceRange parser (src/semantic/parser.rs); the crate now consumes `mdpeek_parser::BlockTree` (this was the planned integration point noted in the original Layer 2 PR). - Adapt to mdpeek-parser's API: BlockId(u64) content-stable ids, CodeBlock{language}, Item{task}, MetadataBlock as a block, no per-block byte_range (table analysis now slices source by SourceRange line numbers), frontmatter() accessor. - Add a `links` module: mdpeek-parser folds inline links into text, so Layer 2 re-parses once to recover hyperlinks with source ranges for DocumentModel. - Resolve src/main.rs conflict by taking main's version (Layer 2 no longer lives in the binary). mdpeek-analyzer: 40 tests. Full workspace: all tests pass, clippy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QcBxPCWg3P2QwyqMGpTyfp
Add a startup-loaded config switch for choosing between rules-based and
LLM-based generation, per AGENTS.md 論点 F/K.
- mdpeek-analyzer: new `generation` module with `GenerationStrategy`
(RulesFirst/LlmFirst) + `GenerationConfig` policy and a `should_use_llm()`
decision function the Layer 3 generator will consult per node:
* llm disabled -> never (strictly rules)
* llm_first + enabled -> always prefer the LLM
* rules_first -> escalate only below `confidence_threshold` (0.6)
- binary `[llm]` config section: `enabled`, `strategy`, `confidence_threshold`,
loaded at startup and mapped via `Config::generation_config()`.
- main: resolve the policy at startup; serve mode logs the effective policy
(term mode left untouched so piped output stays clean).
- config.example.toml: document the `[llm]` section.
Verified at runtime: `strategy = "llm_first"` and `"rules_first"` (and the
disabled default) are picked up from config.toml and reflected in the startup
log. Workspace tests pass, clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QcBxPCWg3P2QwyqMGpTyfp
…ODO closers - Add docs/sample-design-doc.md exercising the rules-detectable structures (outline, task list, status table, mermaid/toml/http code fences, risks / open-questions sections, inline TODO, links). - Add `cargo run -p mdpeek-analyzer --example analyze -- <file>` to print the rules-stage DocumentModel / SemanticPanel + code-block and table analyses. - panel: strip trailing comment closers so `<!-- TODO: x -->` yields "x". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QcBxPCWg3P2QwyqMGpTyfp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Layer 2 — Semantic viewer (rules-centric)
Implements Layer 2 from AGENTS.md §10, built on top of Layer 1's
mdpeek-parser.Turns markdown into a
DocumentModel+SemanticPanelwith deterministic rules only (no LLM):Lives in a new workspace crate
crates/mdpeek-analyzer, matching Layer 1's fine-grained crate layout.What's here
model(§4.2) —DocumentType,Classified<T>,BlockClass,DocumentModel,OutlineEntry,Task.analyzer—doctype(frontmatter → filename → heading-set → content heuristics, confidence-scored),block_class(section propagation),code(intent sniffing: shell/json/yaml/toml/sql/http/diagram/diff/lang),table(columns + status-column detection, by re-reading the source lines),tasks(checkbox extraction).links—mdpeek-parserfolds inline links into block text, so Layer 2 re-parses once to recover hyperlinks with source ranges forDocumentModel.links.panel— outline / TODO (task items + inline TODO·FIXME·XXX·HACK) / risk / open-questions side panel, every entry with aSourceRangeLinkback to source. Not yet UI IR (that's Layer 3), per the roadmap.Layer 2 deliverables (AGENTS.md)
DocumentModelSourceRangeLinkIntegration with Layer 1 (merged)
Layer 1 (#12–#20, now on
main) split the project into fine-grained crates and added a SourceRange-awareBlockTreeinmdpeek-parser. This branch mergesmainand integrates accordingly:mdpeek_parser::BlockTree(this was the planned integration point).mdpeek-parser's API:BlockId(u64)content-stable ids,CodeBlock{language},Item{task},MetadataBlockas a block, no per-blockbyte_range(table analysis slices source bySourceRangeline numbers),frontmatter()accessor.src/main.rsmerge conflict resolved by takingmain's version — Layer 2 no longer lives in the binary.Net diff vs
mainis just the newmdpeek-analyzercrate + its workspace registration.Verification
cargo test --workspace— all pass (mdpeek-analyzer40; Layer 1 crates unchanged & green).cargo clippy --workspace --all-targets— clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01QcBxPCWg3P2QwyqMGpTyfp