diff --git a/CHANGELOG.md b/CHANGELOG.md index df195908..03102ff2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,7 +30,7 @@ once it reaches a published 0.1.0 release. - [S-137] Every invocation form lvm2 documents now reaches usage as its own alternative, keeping the prose sentence above it as its option group instead of inventing a heading from the form's own usage line (`mandible lvcreate`). - [S-139] A single-dash long option carrying an interior uppercase letter keeps its whole name when a sibling row in the same table shares its lowercase prefix, so `mandible mksquashfs` shows `-noI` and its nine siblings instead of ten rows reading `-n`. - [S-143, S-144] lowdown's man-page-like `--help` rendering now parses its bullet-marked commands and options instead of rendering verbatim, recovering all 30 subcommands and 9 flags of `mandible nix`, including a `/`-joined alias and a multi-word value name (#138). -- [S-147] A flag documented once per invocation form now keeps every form's own value name instead of one form's name beside another's choices, so `mandible lvcreate`'s `--type` shows `linear, striped, raid10, snapshot, thin` beside its `raid1, mirror` choices. +- [S-147] A flag documented once per invocation form now renders one placeholder and one unioned choice list instead of splitting across a value name and a separate choices line, so `mandible lvcreate`'s `--type` shows no placeholder beside `values: linear, striped, raid1, mirror, raid, raid10, snapshot, thin-pool, cache-pool, thin, vdo, cache, writecache`. - [S-145] A single-dash long option in a table with no double-dash row anywhere keeps its whole name and case now, so `mandible mksquashfs` and `mandible sqfstar` show `-pf`, `-ef`, `-Xhelp`, `-Xstrategy`, `-Xhc` and `-Xbcj` instead of splitting each one. - [S-145] A tab-separated value name on a single-dash long option survives the same repair now, so `mandible mksquashfs` and `mandible sqfstar` keep `-mem`, `-comp` and `-mkfs-time` with their placeholders instead of losing them. - [S-146] A heading with no indent step to its own rows, and a bare label above a nested option block, now name a group instead of leaving every row underneath ungrouped, so `mandible mksquashfs` shows its ten option headings and its five compressor names. @@ -52,6 +52,7 @@ once it reaches a published 0.1.0 release. - [S-163] A `+/-name` alternation row's own expansion no longer collides with an ordinary row documenting the same spelling, and a `+word` row's value column is now borrowed onto its `-word` sibling when the ordinary repair can't recover a bare one, so `mandible Xvfb` keeps one `-render` with its four choices, one `+render`, and the same `name` value on both `+extension` and `-extension`; its `+word` spellings also render in the long column now, beside `-render`, instead of the short column at column 0. - [S-168] A colon-introduced list of bare-name choices sitting directly under a placeholder row — or a `+word`/`-word` pair sharing one — now becomes that placeholder's own choices instead of folding into the row's own description, so `mandible Xvfb`'s `+extension`/`-extension` pair shows its 19 run-time-toggleable extension names as choices on both halves. - [S-145] A single-dash-long table with no column padding at all now recovers a row's genuine bracketed value one space after its name, so `mandible Xvfb`'s `-render` and `-deferglyphs` keep their own choices instead of losing them outright. +- [S-167] A `Usage:` line whose leading word carries a bracketed optional-abbreviation suffix (`v[ersion]`, `g[dbserver]`) now reaches the tree as its own subcommand, named and shown by its full word with the short prefix kept as an alias, and probed for its own flags, instead of folding into the usage block (`mandible lldb-server`). ## [0.7.0] - 2026-09-05 diff --git a/corpus/README.md b/corpus/README.md index 286926b7..1e94e6e8 100644 --- a/corpus/README.md +++ b/corpus/README.md @@ -571,21 +571,21 @@ produces no root, the same reasoning `must_not_describe` uses. ### A value name after the real app's own root refill: `must_value_names_after_root_refill` `must_value_name` is satisfied by *any* entity carrying the spelling, on -the raw, unrefilled tree — which is exactly why it cannot see `lvcreate`'s -own defect (docs/shapes.md S-147). `--type` reaches that tree once per -invocation form (`linear`, `striped`, the choices `raid1`/`mirror`, ...), -each form still its own entity, each already naming its own value: the -positive claim passes trivially before any fold has even happened. The -loss only exists once the same node is folded, and every running session -folds it: `mandible::background::Warmer::submit_root_fill` always merges -the already-extracted root against a fresh reprobe as soon as the TUI -opens, and `Runner::fill_node`'s own contract ("`existing` is always -included as a candidate") is what pools every same-spelling entity into -one bucket for `mandible_core::merge::merge_entity_bucket` to fold. +the raw, unrefilled tree — which is exactly why it cannot see a +same-spelling merge bucket's own defect (docs/shapes.md S-147). A flag +whose several invocation forms each name their own value reaches the raw +tree once per form, each form still its own entity, each already naming +its own value: the positive claim passes trivially before any fold has +even happened. The loss only exists once the same node is folded, and +every running session folds it: `mandible::background::Warmer::submit_root_fill` +always merges the already-extracted root against a fresh reprobe as soon +as the TUI opens, and `Runner::fill_node`'s own contract ("`existing` is +always included as a candidate") is what pools every same-spelling entity +into one bucket for `mandible_core::merge::merge_entity_bucket` to fold. ```toml [contract.must_value_names_after_root_refill] -"--type" = ["linear", "striped", "raid10", "snapshot", "thin"] +"-help" = ["topic"] ``` `cargo xtask corpus` simulates that exact refill — `merge_nodes` over two @@ -599,6 +599,39 @@ listed substring, naming what was expected and what survived instead. A fixture that produces no root fails this exactly as it fails `must_value_name`. +S-147's own follow-up ruling (2026-09-07 "queue", docs/design.md §16) +narrowed what this field can still claim about `lvcreate`'s own `--type`: +a merge bucket whose forms disagree about a *literal* value now unions +every one of them into `choices` instead, rendering one placeholder +(``, or none, never a comma list of literals) beside the union — +see `must_choices_after_root_refill` below for that half. This field +still states its own claim correctly for a bucket disagreeing about a +genuine, capitalised placeholder name. + +### Choices after the real app's own root refill: `must_choices_after_root_refill` + +The `choices` twin of `must_value_names_after_root_refill`, for the same +S-147 follow-up: a same-spelling bucket whose forms each name a different +*literal* value (`linear`, `striped`, `raid10`, ...) unions every one, +plus any `choices` a bracket-row form already carried, into one list on +the refilled entity. `must_attach_choices` alone cannot see this: it +walks the raw, unrefilled tree, where `.find()` sees only the first +invocation form's own entity and its usually-empty `choices`. + +```toml +[contract.must_choices_after_root_refill] +"--type" = ["linear", "striped", "raid1", "mirror", "raid", "raid10", "snapshot", "thin-pool", "cache-pool", "thin", "vdo", "cache", "writecache"] +``` + +`cargo xtask corpus` simulates the same refill +`must_value_names_after_root_refill` does and checks the named flag's +*merged* `choices` for every listed value (matched the way +`must_attach_choices` matches: exact name, no substring). `cargo xtask +corpus` fails when the flag is absent from the refilled tree, or when +the merged choices are missing any listed value, naming what was +expected and what the merged choices actually held. A fixture that +produces no root fails this exactly as it fails `must_attach_choices`. + ### What `--bless` does and does not assert: `verdict_scope` `--bless` freezes the *entire* tree into `expected.snap` — node summaries, diff --git a/corpus/fail2ban-client/1.0.2/meta.toml b/corpus/fail2ban-client/1.0.2/meta.toml index 7f1a66e3..d609acf7 100644 --- a/corpus/fail2ban-client/1.0.2/meta.toml +++ b/corpus/fail2ban-client/1.0.2/meta.toml @@ -18,4 +18,4 @@ min_subcommands = 14 [xfail] broken = true -reason = "1. Fixed. A description continuation beginning with a dash used to end the Command: bare block early and read as a flag row; bare_block_end now recognizes it as a wrapped-prose continuation (S-027's own test) instead, so no --with-time flag reaches the tree. 2. The Command: section holds 95 rows of multi-word command patterns (set loglevel , get banip [|--with-time], reload [--restart] [--unban] ) under ten centered ALL-CAPS group labels. bare_block_end's baseline is fixed (docs/shapes.md S-149), so the block now reads every row cleanly, but emit_subcommands still drops any row whose name field is not a single command-name-shaped word: a bare-word first line recovers (start, restart, stop, banned, status, ping, echo, help, version, flushlogs, 10 nodes), while every set, get, add, unban and reload row is dropped rather than fabricated, since none of them ever has a bare-word-only form. The command-pattern-table detector still reads 18 tools and 224 findings fleet-wide, 85 of them here, above the five-tool bar. The name rule (one command node per distinct leading word, each row's own pattern appended to its usage list) is not yet shipped." +reason = "1. Fixed. A description continuation beginning with a dash used to end the Command: bare block early and read as a flag row; bare_block_end now recognizes it as a wrapped-prose continuation (S-027's own test) instead, so no --with-time flag reaches the tree. 2. The Command: section holds 95 rows of multi-word command patterns (set loglevel , get banip [|--with-time], reload [--restart] [--unban] ) under ten centered ALL-CAPS group labels. bare_block_end's baseline is fixed (docs/shapes.md S-149), so the block now reads every row cleanly, but emit_subcommands still drops any row whose name field is not a single command-name-shaped word: a bare-word first line recovers (start, restart, stop, banned, status, ping, echo, help, version, flushlogs, 10 nodes), while every set, get, add, unban and reload row is dropped rather than fabricated, since none of them ever has a bare-word-only form. The command-pattern-table detector still reads 18 tools and 224 findings fleet-wide, 85 of them here, above the five-tool bar. The name rule (one command node per distinct leading word, each row's own pattern appended to its usage list) is not shipped: round 10 built it behind a column-identity guard, which held here and gained the four named targets, and a full-PATH sweep-diff then caught six fabricated nodes on three other tools (options on xauth, gprofng on gprofng, attach/logs/respawn/rm on claude). That is S-141's third refusal; docs/shapes.md S-141 carries all three with the names each one invented." diff --git a/corpus/lldb-server/18.1.3/expected.snap b/corpus/lldb-server/18.1.3/expected.snap new file mode 100644 index 00000000..120cc941 --- /dev/null +++ b/corpus/lldb-server/18.1.3/expected.snap @@ -0,0 +1,32 @@ +name: lldb-server +description: Invoke subcommand for additional help +provenance: + sources: + - help-text + confidence: 0.15 +children_filled: true +subcommands: +- name: version + aliases: + - v + provenance: + sources: + - help-text + invocation_attested: true + abbrev_probe_attested: true +- name: gdbserver + aliases: + - g + provenance: + sources: + - help-text + invocation_attested: true + abbrev_probe_attested: true +- name: platform + aliases: + - p + provenance: + sources: + - help-text + invocation_attested: true + abbrev_probe_attested: true diff --git a/corpus/lldb-server/18.1.3/help.stderr.txt b/corpus/lldb-server/18.1.3/help.stderr.txt new file mode 100644 index 00000000..bd8997d4 --- /dev/null +++ b/corpus/lldb-server/18.1.3/help.stderr.txt @@ -0,0 +1,5 @@ +Usage: + lldb-server v[ersion] + lldb-server g[dbserver] [options] + lldb-server p[latform] [options] +Invoke subcommand for additional help diff --git a/corpus/lldb-server/18.1.3/help.txt b/corpus/lldb-server/18.1.3/help.txt new file mode 100644 index 00000000..e69de29b diff --git a/corpus/lldb-server/18.1.3/meta.toml b/corpus/lldb-server/18.1.3/meta.toml new file mode 100644 index 00000000..bba818a8 --- /dev/null +++ b/corpus/lldb-server/18.1.3/meta.toml @@ -0,0 +1,34 @@ +# S-167: a bare `Usage:` heading's rows repeat the tool's own name plus +# one subcommand word abbreviated with a bracket suffix. `version`, +# `gdbserver` and `platform` recover as subcommand nodes, never usage +# forms, each shown by its full word (never the bracketed source +# spelling) with the row's short prefix kept as an alias. Each node is +# invocation_attested and abbrev_probe_attested (docs/design.md §16, +# 2026-09-13), so a live `mandible` run probes ` --help` for its own +# flags; this frozen fixture captures only the root parse, with no +# subprocess, so it cannot show that fill. The help text sits entirely on +# stderr; stdout is empty. + +[bless] +provenance = "agent" + +[tool] +name = "lldb-server" +version = "18.1.3" +platform = "ubuntu-24.04 (aarch64)" +captured_with = "mandible 0.7.0" + +[[capture]] +argv = ["lldb-server", "--help"] +stdout = "help.txt" +stderr = "help.stderr.txt" +exit_code = 0 + +[contract] +expected_framework = "generic" +min_subcommands = 3 + +[contract.must_display_name] +version = "version" +gdbserver = "gdbserver" +platform = "platform" diff --git a/corpus/lvcreate/2.03.16/expected.snap b/corpus/lvcreate/2.03.16/expected.snap index 0edb97d1..7165d0de 100644 --- a/corpus/lvcreate/2.03.16/expected.snap +++ b/corpus/lvcreate/2.03.16/expected.snap @@ -133,6 +133,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: raid + value_kind: Required + group: Create a raid LV (a specific raid level must be used, e.g. raid1). + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -315,6 +323,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: thin-pool + value_kind: Required + group: Create a thin pool. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -405,6 +421,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: cache-pool + value_kind: Required + group: Create a cache pool. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -536,6 +560,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: thin + value_kind: Required + group: Create a thin LV that is a snapshot of an external origin LV. + provenance: + sources: + - help-text-synopsis - spellings: - -T - --thin @@ -543,6 +575,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: vdo + value_kind: Required + group: Create a LV that returns VDO when used. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -621,6 +661,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: cache + value_kind: Required + group: Create a new LV, then attach the specified cachepool which converts the new LV to type cache. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -720,6 +768,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: cache + value_kind: Required + group: Create a new LV, then attach the specified cachevol which converts the new LV to type cache. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -794,6 +850,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: cache + value_kind: Required + group: Create a new LV, then attach a cachevol created from the specified cache device, which converts the new LV to type cache. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -876,6 +940,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: writecache + value_kind: Required + group: Create a new LV, then attach the specified cachevol which converts the new LV to type writecache. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents @@ -911,6 +983,14 @@ flags: provenance: sources: - help-text +- spellings: + - --type + value_name: writecache + value_kind: Required + group: Create a new LV, then attach a cachevol created from the specified cache device, which converts the new LV to type writecache. + provenance: + sources: + - help-text-synopsis - spellings: - -l - --extents diff --git a/corpus/lvcreate/2.03.16/meta.toml b/corpus/lvcreate/2.03.16/meta.toml index 83b305d7..1a0666f0 100644 --- a/corpus/lvcreate/2.03.16/meta.toml +++ b/corpus/lvcreate/2.03.16/meta.toml @@ -38,13 +38,17 @@ must_not_contain_positionals = ["COMMON_OPTIONS"] [contract.must_flag_group] "--mirrorlog" = "Create a raid1 or mirror LV." -# docs/shapes.md S-147: seven invocation forms all reach the tree spelled -# `--type`, each naming (or, for the raid1/mirror form's own choices, not -# naming) a different literal value. `must_value_name` alone cannot state -# this claim — it passes vacuously off the raw, unrefilled tree, where the -# forms are still separate entities and each already names its own value. -# The claim only means something once the same background root-refill the -# running app always performs (`Warmer::submit_root_fill`) folds them into -# one bucket, which is what this field simulates. -[contract.must_value_names_after_root_refill] -"--type" = ["linear", "striped", "raid10", "snapshot", "thin"] +# docs/shapes.md S-147 follow-up (ruled 2026-09-07 "queue"): thirteen +# invocation forms all reach the tree spelled `--type`, each naming a +# different literal value, either directly (`--type raid`) or via a +# bracket row's own trailing choice list (`raid1|mirror`). +# `must_attach_choices` alone cannot state this claim — it walks the raw, +# unrefilled tree, where `.find()` sees only the first form's own, +# usually-empty `choices`. The claim only means something once the same +# background root-refill the running app always performs +# (`Warmer::submit_root_fill`) folds every form into one bucket, which is +# what `must_choices_after_root_refill` simulates. No form ever names +# `--type`'s value with a capitalised token, so the merged flag carries no +# placeholder at all rather than a fabricated ``. +[contract.must_choices_after_root_refill] +"--type" = ["linear", "striped", "raid1", "mirror", "raid", "raid10", "snapshot", "thin-pool", "cache-pool", "thin", "vdo", "cache", "writecache"] diff --git a/docs/design.md b/docs/design.md index 65c2710d..8329540b 100644 --- a/docs/design.md +++ b/docs/design.md @@ -959,6 +959,17 @@ allowlist below. heading evidence strong enough to probe — the two bits are never conflated, and this gate reads only `heading_attested`. + A third, narrower bit, `abbrev_probe_attested`, admits one closed case + (§16, docs/shapes.md S-167). It marks a node the usage-optional- + abbreviation recognizer produced. That recognizer matches a `Usage:` + line whose leading word carries a bracketed optional-abbreviation + suffix, for example `lldb-server`'s `g[dbserver]`. This gate now admits + a node when `heading_attested` is true or `abbrev_probe_attested` is + true. No other recognizer ever sets this bit. `invocation_attested` + never implies it; a headingless-invocation-table node still stays + declined. Rule 0's thirteen-program list is checked first, and it wins + unconditionally before this gate runs at all. + 1. **Never invoke a bare binary.** An argv is never empty. Running an arbitrary binary with no arguments is how you launch a REPL, block on stdin, start a daemon, or trigger a tool whose no-argument default is an @@ -1287,7 +1298,13 @@ into structured entities. qualify; a bare word list under no heading does not. 7. A candidate name must match `^[a-z][a-z0-9_.-]*$` with no whitespace, and every emitted name must occur literally in the tool's own raw text - (the existence oracle, §13.1). + (the existence oracle, §13.1). One narrow reconstruction is permitted + (§16). A subcommand name may also satisfy this rule when it is a raw + token with its bracket characters deleted and nothing else changed. No + character may be added, removed beyond the brackets, reordered, or + changed in case. `g[dbserver]` yields `gdbserver` this way and no other + token yields it. This is the only reconstruction rule 7 permits for a + subcommand name. 8. Two evidence classes short of a heading are tracked separately. `invocation_attested` marks a row that repeats the tool's own name, or a table whose row shape is unambiguous even without a heading. @@ -2908,6 +2925,64 @@ Ships anyway: the fix moved 11 tools once measured tree-wide (`rustc`, subcommand-count losses on a full-`PATH` sweep-diff of 2269 tools, alongside S-157's own sweep. +**An S-167 node shows its full word, never the bracketed spelling +(2026-09-13).** Shown `mandible lldb-server` with its three commands +rendered `v[ersion]`, `g[dbserver]`, `p[latform]`, the maintainer rejected +keeping the source spelling as the display name. The rule: the node's name +and displayed form are both the full word (`gdbserver`), and the row's own +short prefix (`g`) is kept as `CommandNode::aliases` instead, the IR's +existing alias slot (§4.5). `display_name` is no longer set for this +shape. A new, narrower attestation bit, `CommandNode::abbrev_probe_attested` +/ `NodeHints::abbrev_probe_attested`, is set only by this recognizer and +admits the node to §6 rule 0's probe gate on its own, without touching +`heading_attested`'s own meaning or admitting `invocation_attested` in +general. A live `mandible` run now probes each child with its own full +word (`lldb-server gdbserver --help`) and fills its own flags; rule 0's +thirteen-program list is checked first and wins unconditionally, proved by +`mandible-extract/tests/exec_policy.rs`'s +`abbrev_probe_attested_word_is_probed_even_though_not_heading_attested` and +`rule_0_still_refuses_an_abbrev_probe_attested_word_naming_a_never_probe_tool`. +Measured on this box: `lldb-server 'g[dbserver]' --help` answers +byte-identical to `lldb-server gdbserver --help`, since lldb-server +matches a subcommand by prefix. The bracketed form is not refused here, +but the full word is still the right argv, since it is the word a user +would type and the bracket characters have no business in argv. Fixture: +`corpus/lldb-server/18.1.3`, a frozen-bytes capture with no subprocess, so +it shows the repaired root parse but not the probe-filled children a live +run produces. Docs/shapes.md S-167. + +**Rule 7 gets one narrow reconstruction for S-167 (2026-09-13).** S-167's +own node names (`gdbserver`, `version`, `platform`, unchanged since the +shape first shipped) never occur as a contiguous substring of +`lldb-server`'s raw text, only `g[dbserver]` etc. do, confirmed directly: +`lldb-server --help | grep -c gdbserver` is 0 on this box. A full-`PATH` +sweep found `existence_fabrication_tools` at 56 on the pre-S-167 baseline +and 58 once S-167 shipped, the two lldb-server binaries each newly +reporting 3 fabrications. This predates the display fix directly above; +naming these nodes by the full word, not the gate widening, is what rule 7 +never accounted for. The ruling: a +subcommand name also satisfies rule 7 when it is a raw token with its +bracket characters deleted and nothing else changed, no character added, +none reordered, no case changed. `mandible-extract::help_text:: +reconstruct_abbrev_word` computes it, re-exported for +`xtask/src/existence.rs::tool_name_prefixed_row_words`, which inserts the +reconstructed name alongside its existing modifier-stripped candidate at +the same already-attested command-list position; the position requirement +itself is untouched, widened only to also recognize that position under +the tool's own full-path spelling (`/usr/bin/lldb-server`), the real +shape a live probe actually captures and the same spelling S-167's own row +parser already tolerated, which the position check had not. A break-it +check disabled the reconstruction and confirmed four tests turn red: +`tool_name_prefixed_row_words_attests_the_bracket_deleted_reconstruction`, +`tool_name_prefixed_row_words_attests_the_reconstruction_under_a_full_ +path_spelling`, `detect_does_not_flag_lldb_servers_real_abbreviated_ +subcommands`, and `detect_still_flags_a_name_that_is_not_a_bracket_ +deletion_of_anything`, the last of which also pins the negative case, a +genuinely fabricated name sitting in the same row is still caught. +Re-measured with `xtask coverage --tools lldb-server,lldb-server-18` +against the real installed binaries: `existence_fabrication_tools` reads +0 and `exist` reads 0 for both. Docs/shapes.md S-167. + ### Deferred, with the reason each is not simply undone **Sub-case (b) of the `-h` fallback is unmeasured and must stay that way until diff --git a/docs/shapes.md b/docs/shapes.md index 0e5a5a2e..6d63dddb 100644 --- a/docs/shapes.md +++ b/docs/shapes.md @@ -2596,23 +2596,47 @@ entry's `tools` field and nothing else. It does not get a new entry. set loglevel sets logging level to . - tools: fail2ban-client, busctl, hostnamectl, localectl, networkctl, resolvectl, timedatectl -- handling: Open. Two prototypes were built and refused: reading each token of - a row's own name field separately fabricated nine command rows on - fail2ban-client (`logtarget`, `persistent`, `of`, `list`, `files`, `filter`, - `for`, `back`, `failures`), each a word cut out of a wrapped description. The - `bare_block_end` baseline blocker underneath is fixed now (S-149): every row - of fail2ban-client's `Command:` table is read, and its 10 single-word - commands recover cleanly with no fabrication. The name rule itself, one node - per distinct leading word with each row's own pattern appended to that - node's `usage` list, is not shipped: `emit_subcommands` still drops any row - whose name field is not a single command-name-shaped word or a bare-word - plus an ALL-CAPS-only placeholder run, so `set`, `get`, `add`, `unban` and - `reload` recover no node at all. `command-pattern-table` - (`xtask/src/command_pattern_table.rs`) stays as the instrument. +- handling: Open, refused three times. The name rule, one node per distinct + leading word with each row's own pattern appended to that node's `usage` + list, is NOT shipped: `emit_subcommands` still drops any row whose name + field is not a single command-name-shaped word or a bare-word plus an + ALL-CAPS-only placeholder run, so on fail2ban-client `set`, `get`, `add`, + `unban` and `reload` recover no node at all. `command-pattern-table` + (`xtask/src/command_pattern_table.rs`) stays as the instrument, reported and + not gated. + The three refusals, each with the names it invented, so a fourth attempt has + the whole negative list: + 1. Round 8 read each token of a row's own name field as its own command and + fabricated nine nodes on fail2ban-client: `logtarget`, `persistent`, + `of`, `list`, `files`, `filter`, `for`, `back`, `failures`. Every one is a + word cut out of a WRAPPED DESCRIPTION, not out of a name field. + 2. Round 9 fixed the block boundary underneath (S-149) and deliberately did + not retry the name rule. Every row of fail2ban-client's `Command:` table + is read now, and its 10 single-word commands recover cleanly with no + fabrication. + 3. Round 10 built the rule behind a column-identity guard: a candidate row + is admitted only when its name field begins at the block's own name + column, on the reasoning that a wrapped description continuation begins + at the description column by construction. The guard held on + fail2ban-client, which gained exactly the four named targets (`add`, + `get`, `reload`, `set`), and it FAILED elsewhere, because a prose line + and a repeated program name can both begin at the name column. A + full-`PATH` sweep-diff named six fabricated nodes on three tools: + `options` on xauth, cut from the prose line `options are:`, which is a + heading and not a command; `gprofng` on gprofng, the tool's own name cut + from rows like `gprofng collect app`; and `attach`, `logs`, `respawn`, + `rm` on claude, the first two cut from prose ("attach to an existing", + "logs to a specific file path") and the last two matching no row at all. + Column identity is a necessary condition and not a sufficient one. + The bar for a fourth attempt: a guard that separates a table ROW from a + prose line at the same indent, checked on xauth, gprofng and claude by name + before any count is reported. - fleet: 18 tools/224 findings on a full-`PATH` sweep, 2026-09-06. fail2ban-client holds 85 of them and 84 were read by hand and are genuine. Above the five-tool bar. Not shipped; the fixture's `min_subcommands = 14` - floor documents the gap, currently 10. + floor documents the gap, currently 10. Round 10's refused prototype would + have taken fail2ban-client to 14, which is why a count alone can never + admit this rule: the same build fabricated six nodes on three other tools. ### S-142: usage label glued to the program name, wrapped mid-bracket at column zero @@ -2801,17 +2825,26 @@ entry's `tools` field and nothing else. It does not get a new entry. - tools: lvcreate; a merge-step fix, so any tool whose node reaches `mandible_core::merge::merge_entity_bucket` with a same-spelling, same-value-kind bucket naming different literal values is covered -- handling: Fixed. `lvcreate` reaches this bucket once per invocation - form, each form naming its own value for `--type`. `merge_entity_bucket` - picked one form's `value_name` by authority, so the rendered row showed - `--type linear` beside the `raid1`/`mirror` form's own `choices`, - dropping `striped`, `raid10`, `snapshot` and `thin` outright. It now - unions every distinct value name across the bucket, in first-appearance - order, joined the way `choices` already joins for display. Maintainer- - absent default, recorded in docs/design.md §16. `must_value_name` passes +- handling: Fixed, twice. `lvcreate` reaches this bucket once per + invocation form, each form naming its own value for `--type`. + `merge_entity_bucket` picked one form's `value_name` by authority, so + the rendered row showed `--type linear` beside the `raid1`/`mirror` + form's own `choices`, dropping `striped`, `raid10`, `snapshot` and + `thin` outright. A first fix unioned every distinct value name into + `value_name` itself; ruled confusing (docs/design.md §16) since one + flag then showed two lists. The bucket now renders one placeholder and + one unioned `choices` list: every + literal lowercase value, from a form's own `value_name` and from any + `choices` it already carried, joins `choices`; a capitalised token + stays the placeholder name; no placeholder is fabricated when every + form named a literal. `mandible_extract`'s stanza-head recovery also + now reads a leading flag's own bare usage-form value (`--type raid`, + not just the bracketed `[ --type x ]` rows), so `--type` now carries + thirteen values, not five. `must_value_name`/`must_attach_choices` pass vacuously on the raw, unrefilled tree; `must_value_names_after_root_refill` - (`corpus/README.md`) simulates the real app's own root refill and is the - field that actually states the claim. + and `must_choices_after_root_refill` (`corpus/README.md`) simulate the + real app's own root refill and are the fields that actually state the + claim. - fleet: `same-spelling-fold-loss` (`xtask/src/detector/same_spelling_fold_loss.rs`), widened to also flag two same-identity entities that both take a value but name it @@ -3100,6 +3133,7 @@ entry's `tools` field and nothing else. It does not get a new entry. `tclobjnew-bpfcc`, reads 1 tool (grub-mkimage, 2 findings). Below the five-tool bar. Not fixed this round; `corpus/grub-mkimage/2.12` stays `[xfail]` for `--format`'s own description. 2026-09-12. + ### S-157: a bare-word value placeholder after a single-dash-long spelling - id: S-157 @@ -3428,6 +3462,7 @@ entry's `tools` field and nothing else. It does not get a new entry. tree's flag spellings) than the narrow structural cause this fix closes (no blank line anywhere, no recognized usage line); left as a future finding, not chased here. + ### S-166: header-declared three-column option table, env-variable column - id: S-166 @@ -3453,6 +3488,52 @@ entry's `tools` field and nothing else. It does not get a new entry. seed's own captures reads 42 tools / 42 findings, the whole `qemu-*-static` set, 2026-09-12. +### S-167: a usage form's leading word carries a bracketed abbreviation suffix + +- id: S-167 +- looks like: | + Usage: + lldb-server v[ersion] + lldb-server g[dbserver] [options] + lldb-server p[latform] [options] +- tools: lldb-server, lldb-server-18, gcc-ar, unsquashfs, sqfscat, bridge +- handling: Fixed, revised. A usage form whose leading word + after the program name is a command word with a bracketed + optional-abbreviation suffix names a subcommand: the node is named by the + whole word with the brackets removed (`gdbserver`), and that same whole + word is the displayed name — the maintainer rejected showing the + bracketed source spelling (`g[dbserver]`) in the tree. The row's own short + prefix (`g`) is kept as an alias instead of a display spelling. The nodes + are `invocation_attested`, never `heading_attested`, but now also carry a + third, narrower attestation bit, `abbrev_probe_attested`, admitted only + for a node this exact recognizer produced (design §6 rule 0). That bit + alone now clears rule 0's probe gate, so `mandible lldb-server`'s three + children are probed with their own full word (`lldb-server gdbserver + --help`) and fill their own flags; rule 0's thirteen-program list still + wins first and unconditionally, proved by the `exec_policy` shim suite. + Confirmed directly: `lldb-server 'g[dbserver]' --help` also answers, + byte-identical to `lldb-server gdbserver --help`, because lldb-server + matches its subcommand by prefix — the bracketed form is not refused + here, but the full word is still the right argv, since it is the word a + user would type and the bracket characters have no business in argv. + The emitted name is the source token minus its bracket characters and + nothing else changed, which is why design §7 Tier B rule 7's existence + oracle needed a narrow amendment (§16): `gdbserver` is not a contiguous + substring of the raw text, only `g[dbserver]` is, so the oracle now also + attests a subcommand name reached this way. +- fleet: `usage-optional-word-table` + (`xtask/src/usage_optional_word_table.rs`) named 10 tools/17 findings as a + raw shape before the round. What moved on a full-`PATH` sweep of 2323 + tools, 2026-09-13: 6 subcommands gained across 2 tools, `lldb-server` + 0 to 3 and `lldb-server-18` 0 to 3, with names `gdbserver`, `platform` and + `version`. Zero flag losses, zero flag gains, zero subcommand losses, and + all nine named controls byte-identical. The same-day display/alias/probe + amendment above changed no node counts (names, not display forms, gate + fleet-wide sweeps), so this count stands unchanged. Fixture: + `corpus/lldb-server/18.1.3`, whose `must_display_name` pins all three + repaired names; the fixture replays frozen bytes with no subprocess, so + it cannot show the probe-filled children a live run produces. + ### S-168: a colon-introduced choice list under a placeholder pair - id: S-168 diff --git a/mandible-core/src/entity.rs b/mandible-core/src/entity.rs index 9c69daa5..729d1ed4 100644 --- a/mandible-core/src/entity.rs +++ b/mandible-core/src/entity.rs @@ -188,6 +188,25 @@ pub struct Choice { pub description: Option, } +/// Whether `value_name` is one clean literal enumerated value (`"raid1"`, +/// `"thin-pool"`), not a placeholder (`"Number"`) or notation glued onto +/// a neighboring flag (`blkid`'s `"[--match-tag"`, a restated value spec +/// `"y|n"`). docs/design.md §16's S-147 follow-up: a capitalised token +/// stays out of a merged `choices` list. One word of lowercase ASCII +/// letters, digits and hyphens only; any other punctuation is notation, +/// not an enumerated member. Shared by `merge::merge_entity_bucket` and +/// `mandible_extract`'s stanza-head recovery, so both agree on one rule. +pub fn is_literal_choice_value(value_name: &str) -> bool { + !value_name.is_empty() + && value_name + .chars() + .next() + .is_some_and(|c| c.is_ascii_lowercase()) + && value_name + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-') +} + impl Choice { /// A choice with no documented description — the common case. pub fn bare(name: impl Into) -> Choice { diff --git a/mandible-core/src/lib.rs b/mandible-core/src/lib.rs index dd4358a6..54c9c1cf 100644 --- a/mandible-core/src/lib.rs +++ b/mandible-core/src/lib.rs @@ -24,7 +24,7 @@ mod provenance; mod snapshot; mod text; -pub use entity::{Choice, Dashes, Entity, EntityKind, Spelling}; +pub use entity::{is_literal_choice_value, Choice, Dashes, Entity, EntityKind, Spelling}; pub use merge::{ merge_entity_lists, merge_nodes, merge_subcommand_lists, pair_aliases, MergeError, }; diff --git a/mandible-core/src/merge.rs b/mandible-core/src/merge.rs index 58716d61..6f22d1ce 100644 --- a/mandible-core/src/merge.rs +++ b/mandible-core/src/merge.rs @@ -11,7 +11,7 @@ //! order wins": attempt order is a cost ordering (spec §7); conflict //! resolution is authority (spec §4.4). -use crate::entity::{Dashes, Entity, EntityKind, Spelling}; +use crate::entity::{is_literal_choice_value, Choice, Dashes, Entity, EntityKind, Spelling}; use crate::node::CommandNode; use crate::provenance::{Axis, Provenance}; use crate::text::Text; @@ -181,6 +181,11 @@ pub fn merge_nodes(mut candidates: Vec) -> Result) -> Result (EntityKind, String) { (e.kind, key) } +/// Resolve a bucket's merged `value_name` and `choices` together +/// (docs/design.md §16's S-147 follow-up). Split out of +/// [`merge_entity_bucket`] to stay under `clippy::too_many_lines`. +/// +/// Every distinct `value_name` the bucket's forms carry decides which of +/// three shapes applies: zero or one distinct name is ordinary (no +/// disagreement); every name literal (`is_literal_choice_value`) is +/// `--type`'s own shape, one placeholder plus one unioned `choices`; a +/// mix of a real placeholder with something else (`tar`'s `--rsh-command +/// COMMAND` beside its own default `/usr/bin/rsh`) keeps the +/// union-into-`value_name` behavior S-147 already shipped fleet-wide. +fn resolve_value_name_and_choices(bucket: &[Entity]) -> (Option, Vec) { + let distinct_value_names: Vec<&str> = { + let mut v: Vec<&str> = Vec::new(); + for f in bucket { + if let Some(name) = f.value_name.as_deref() { + if !v.contains(&name) { + v.push(name); + } + } + } + v + }; + let has_any_choices = bucket.iter().any(|f| !f.choices.is_empty()); + let all_literal_disagreement = distinct_value_names.len() > 1 + && distinct_value_names + .iter() + .all(|n| is_literal_choice_value(n)); + let single_literal_beside_choices = distinct_value_names.len() == 1 + && has_any_choices + && is_literal_choice_value(distinct_value_names[0]); + if all_literal_disagreement || single_literal_beside_choices { + // One placeholder, one unioned choice list, first-appearance + // order, joined the way `choices` already joins for display + // (spec §9.2, "values: raid1, mirror"). Every name here is known + // literal, so `value_name` stays `None` — never fabricated. + let mut choices: Vec = Vec::new(); + let mut choice_names: Vec<&str> = Vec::new(); + for f in bucket { + if let Some(name) = f.value_name.as_deref() { + if is_literal_choice_value(name) && !choice_names.contains(&name) { + choice_names.push(name); + choices.push(Choice::bare(name.to_string())); + } + } + for c in &f.choices { + if !choice_names.contains(&c.name.as_str()) { + choice_names.push(&c.name); + choices.push(c.clone()); + } + } + } + (None, choices) + } else if distinct_value_names.len() > 1 { + // A mixed disagreement (at least one non-literal name): the + // union-into-`value_name` behavior S-147 shipped fleet-wide, + // unchanged. `choices` still resolves by authority, as it always + // has outside the all-literal shape above. + ( + Some(distinct_value_names.join(", ")), + pick_vec( + bucket.iter().map(|f| (&f.provenance, &f.choices)), + Axis::Prose, + ), + ) + } else { + // The ordinary case: at most one distinct name, so there is + // nothing to disagree about. Exactly the pre-S147 resolution. + ( + pick_option( + bucket + .iter() + .map(|f| (&f.provenance, f.value_name.as_ref())), + Axis::Structural, + ), + pick_vec( + bucket.iter().map(|f| (&f.provenance, &f.choices)), + Axis::Prose, + ), + ) + } +} + fn merge_entity_bucket(mut bucket: Vec) -> Entity { if bucket.len() == 1 { return bucket.pop().expect("len checked"); @@ -357,15 +446,7 @@ fn merge_entity_bucket(mut bucket: Vec) -> Entity { // no other source can have seen the same flag spelled the other way. let negatable = bucket.iter().any(|f| f.negatable()); let single_dash = bucket.iter().any(|f| f.single_dash()); - // Union, not pick-one (docs/shapes.md S-147). `lvcreate` reaches this - // bucket once per invocation form, each naming a different literal - // value for `--type` (`linear`, `striped`, `raid10`, ...): a - // highest-authority single winner rendered one form's value name - // beside another form's `choices`, silently dropping every other - // form's name. Every distinct name across the whole bucket survives, - // in first-appearance order, joined the same way `choices` already - // joins for display (spec §9.2, "values: raid1, mirror"). - let value_name = union_value_names(bucket.iter().map(|f| f.value_name.as_deref())); + let (value_name, choices) = resolve_value_name_and_choices(&bucket); let value_kind = bucket .iter() .map(|f| f.value_kind) @@ -375,10 +456,6 @@ fn merge_entity_bucket(mut bucket: Vec) -> Entity { crate::node::ValueKind::Required => 2, }) .unwrap_or(crate::node::ValueKind::None); - let choices = pick_vec( - bucket.iter().map(|f| (&f.provenance, &f.choices)), - Axis::Prose, - ); let repeatable = bucket.iter().any(|f| f.repeatable); let required = bucket.iter().any(|f| f.required); let deprecated = pick_option( @@ -551,26 +628,6 @@ where best.map(|(_, v)| v.clone()).unwrap_or_default() } -/// Union every distinct non-empty `value_name` across one identity bucket, -/// in first-appearance order, joined by `", "` (spec §9.2's own join for -/// `choices`). Unlike [`pick_option`], authority plays no part here: two -/// invocation forms of one flag can each name a real, different literal -/// value, and a single winner would silently drop the rest (docs/shapes.md -/// S-147). `None` when the bucket names none at all. -fn union_value_names<'a, I: IntoIterator>>(names: I) -> Option { - let mut seen: Vec<&str> = Vec::new(); - for name in names.into_iter().flatten() { - if !seen.contains(&name) { - seen.push(name); - } - } - if seen.is_empty() { - None - } else { - Some(seen.join(", ")) - } -} - /// Unify flags that arrived as separate short/long rows from the same /// source, per spec §4.4: sources legitimately emit a flag's short and long /// forms as distinct items (e.g. `gh __complete pr -` returns `--repo` and @@ -1274,14 +1331,16 @@ mod tests { ); } - /// docs/shapes.md S-147: `lvcreate` reaches the merge bucket once per - /// invocation form, and three of its forms each name a different - /// literal value for `--type`. A single-winner pick rendered one - /// form's value name (`linear`) while a later fold attached another - /// form's `choices` (`raid1`, `mirror`) to the same row, silently - /// dropping `striped`. Every distinct name must survive. + /// docs/shapes.md S-147 follow-up: `lvcreate` + /// reaches the merge bucket once per invocation form, and three of its + /// forms each name a different literal value for `--type`. A + /// single-winner pick used to render one form's value name (`linear`) + /// while a later fold attached another form's `choices` (`raid1`, + /// `mirror`) to the same row, splitting one flag across two lists. + /// Every literal value now joins one `choices` list, and no + /// placeholder is fabricated when every form named a literal. #[test] - fn merge_unions_value_names_from_several_invocation_forms() { + fn merge_unions_literal_value_names_into_choices_from_several_invocation_forms() { fn type_flag(value_name: &str) -> Entity { let mut e = Entity::flag_long("type", Provenance::single(Source::HelpText)); e.value_kind = ValueKind::Required; @@ -1294,14 +1353,40 @@ mod tests { type_flag("raid10"), ]; let merged = merge_entity_bucket(bucket); + assert_eq!(merged.value_name, None); assert_eq!( - merged.value_name.as_deref(), - Some("linear, striped, raid10") + merged + .choices + .iter() + .map(|c| c.name.as_str()) + .collect::>(), + vec!["linear", "striped", "raid10"] ); } - /// The ordinary case — every form names the same value — must not - /// start repeating itself. + /// A bucket that disagrees about its value name but is NOT the + /// all-literal S-147 follow-up shape (`tar`'s own `--rsh-command + /// COMMAND` beside its own default `/usr/bin/rsh`) must keep the + /// union-into-`value_name` behavior S-147 already shipped fleet-wide. + /// A control-tool row moving on this fix would mean it reached + /// further than `--type`. + #[test] + fn merge_unions_a_mixed_disagreement_into_value_name_unchanged_from_s147() { + fn valued(value_name: &str) -> Entity { + let mut e = Entity::flag_long("rsh-command", Provenance::single(Source::HelpText)); + e.value_kind = ValueKind::Required; + e.value_name = Some(value_name.to_string()); + e + } + let bucket = vec![valued("COMMAND"), valued("/usr/bin/rsh")]; + let merged = merge_entity_bucket(bucket); + assert_eq!(merged.value_name.as_deref(), Some("COMMAND, /usr/bin/rsh")); + assert!(merged.choices.is_empty()); + } + + /// The ordinary case — every form names the same capitalised + /// placeholder — must not start repeating itself, and must not be + /// misread as a literal choice value. #[test] fn merge_does_not_repeat_a_value_name_every_form_agrees_on() { fn type_flag() -> Entity { @@ -1312,5 +1397,36 @@ mod tests { } let merged = merge_entity_bucket(vec![type_flag(), type_flag()]); assert_eq!(merged.value_name.as_deref(), Some("TYPE")); + assert!(merged.choices.is_empty()); + } + + /// A bucket that mixes a real bracket-form `choices` list with other + /// forms' own literal `value_name`s must union both into one list, + /// per docs/shapes.md S-147's follow-up ruling. + #[test] + fn merge_unions_a_bracket_choices_list_with_other_forms_literal_value_names() { + fn valued(value_name: &str) -> Entity { + let mut e = Entity::flag_long("type", Provenance::single(Source::HelpText)); + e.value_kind = ValueKind::Required; + e.value_name = Some(value_name.to_string()); + e + } + fn choice_row(names: &[&str]) -> Entity { + let mut e = Entity::flag_long("type", Provenance::single(Source::HelpText)); + e.value_kind = ValueKind::Required; + e.choices = names.iter().map(|n| Choice::bare(*n)).collect(); + e + } + let bucket = vec![valued("linear"), choice_row(&["raid1", "mirror"])]; + let merged = merge_entity_bucket(bucket); + assert_eq!(merged.value_name, None); + assert_eq!( + merged + .choices + .iter() + .map(|c| c.name.as_str()) + .collect::>(), + vec!["linear", "raid1", "mirror"] + ); } } diff --git a/mandible-core/src/node.rs b/mandible-core/src/node.rs index 7c1af53f..6df38567 100644 --- a/mandible-core/src/node.rs +++ b/mandible-core/src/node.rs @@ -120,6 +120,16 @@ pub struct CommandNode { /// never flagged as a fabricated phantom subcommand merely for not /// being probe-eligible. pub invocation_attested: bool, + /// True only for a node the S-167 recognizer produced (docs/shapes.md + /// S-167): a `Usage:` line whose leading word after the program name + /// carries a bracketed optional-abbreviation suffix (`g[dbserver]`). + /// A third, separate attestation reason from `heading_attested` and + /// `invocation_attested` (spec §6 rule 0): the word is trusted enough + /// to become argv for a ` --help` probe, though it did not come + /// from a recognized command heading. Never set for any other + /// recognizer; `heading_attested` keeps meaning what its own doc + /// comment says. + pub abbrev_probe_attested: bool, /// The binary this node was discovered as, when it was found by the /// `-` PATH convention rather than documented by its /// parent's own help text (spec §5.4) — e.g. `Some("cargo-clippy")` on @@ -262,6 +272,7 @@ impl CommandNode { provenance, heading_attested: false, invocation_attested: false, + abbrev_probe_attested: false, discovered_binary: None, confession: None, same_as_ancestor: false, diff --git a/mandible-core/src/snapshot.rs b/mandible-core/src/snapshot.rs index b2429297..1e609d3a 100644 --- a/mandible-core/src/snapshot.rs +++ b/mandible-core/src/snapshot.rs @@ -467,6 +467,12 @@ pub struct NodeSnapshot { /// See [`crate::CommandNode::invocation_attested`]. #[serde(skip_serializing_if = "is_false")] pub invocation_attested: bool, + /// True only for a node the S-167 recognizer produced (docs/shapes.md + /// S-167): a usage form's leading word carrying a bracketed + /// optional-abbreviation suffix. See + /// [`crate::CommandNode::abbrev_probe_attested`]. + #[serde(skip_serializing_if = "is_false")] + pub abbrev_probe_attested: bool, /// The binary this node was discovered as under the `-` /// PATH convention (spec §5.4), when it was. Omitted for every node an /// extraction tier produced — which is every node a fixture replays, @@ -564,6 +570,7 @@ impl From<&CommandNode> for NodeSnapshot { children_filled: n.children_filled, heading_attested: n.heading_attested, invocation_attested: n.invocation_attested, + abbrev_probe_attested: n.abbrev_probe_attested, discovered_binary: n.discovered_binary.clone(), unparsed: n.unparsed.iter().map(|t| t.as_str().to_string()).collect(), // The order-preservation this whole module exists to protect: diff --git a/mandible-extract/src/completion_script/mod.rs b/mandible-extract/src/completion_script/mod.rs index 882c6b95..8cb59504 100644 --- a/mandible-extract/src/completion_script/mod.rs +++ b/mandible-extract/src/completion_script/mod.rs @@ -722,6 +722,7 @@ _mytool "$@" &["mytool".to_string(), "sub".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!(matches!(result, Err(ExtractError::PathNotFound))); @@ -786,6 +787,7 @@ _mytool "$@" &["mytool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("the transcript covers the exact `completion zsh` argv this tier sends"); @@ -817,6 +819,7 @@ _mytool "$@" &["mytool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!( @@ -902,6 +905,7 @@ _mytool() { &["mytool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!( @@ -945,6 +949,7 @@ _mytool() { &["mytool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("evidence is present, so the tier must extract as it always did"); diff --git a/mandible-extract/src/help_text/mod.rs b/mandible-extract/src/help_text/mod.rs index 265b559c..de8dd7c6 100644 --- a/mandible-extract/src/help_text/mod.rs +++ b/mandible-extract/src/help_text/mod.rs @@ -47,6 +47,13 @@ pub use sections::{ /// docs/shapes.md S-059. pub use sections::strip_optional_modifier_suffix; +/// Re-exported for `xtask/src/existence.rs`: an S-167 usage-abbreviation +/// token (`g[dbserver]`) reconstructs to the emitted node's name by +/// deleting only its bracket characters, the one reconstruction +/// docs/design.md §7 Tier B rule 7 permits (§16); the oracle must agree +/// or `lldb-server`'s two real subcommands report as invented. +pub use sections::reconstruct_abbrev_word; + /// Re-exported for `xtask/src/existence.rs`'s positional-operand check: the /// oracle must agree on which lines are a synopsis before it can attest an /// operand's position. Includes the fprintf-idiom and unlabelled-synopsis @@ -61,6 +68,13 @@ pub use sections::{ starts_with_tool_name, starts_with_usage_prefix, }; +/// Re-exported for `xtask/src/existence.rs`'s S-167 reconstruction +/// (docs/design.md §16): the real row spells the tool's own name as a +/// full path (`/usr/bin/lldb-server`), the same spelling difference +/// `sections::usage_optional_word`'s own row parser already tolerates, so +/// the oracle must recognize the same row it is checking against. +pub use sections::starts_with_tool_name_spelled_differently; + /// Re-exported for `xtask/src/existence.rs`: LVM's bare invocation line /// (`vgextend VG PV ...`, no bracket notation) opens a usage block only on /// the following line reading as a bracket flag row: the oracle must open @@ -396,9 +410,9 @@ impl ExtractionTier for HelpTextTier { /// stream is whichever looks like help, stdout on a tie (spec §7 Tier B, /// M-8). See docs/shapes.md S-066. /// -/// Gated on provenance: no probe is sent at all for a non-empty `words` -/// unless [`NodeHints::heading_attested`] is true (spec §6 rule 0) — a -/// non-attested node returns [`ExtractError::Other`] instead. The man-page +/// Gated on provenance: no probe is sent for a non-empty `words` unless +/// [`NodeHints::heading_attested`] or [`NodeHints::abbrev_probe_attested`] +/// is true (spec §6 rule 0), else [`ExtractError::Other`]. The man-page /// fallback never fires for the root (six root-level binaries stay /// verbatim, S-066); the `-h` response is validated with /// [`looks_like_help_output`] (D1.3.1) before being trusted. @@ -414,7 +428,11 @@ fn probe_help_text_reporting_flag( // name the user typed, never a word any parser invented — so this // never blocks the ordinary ` --help` root probe, only a deeper // path whose last word did not come from a recognized heading. - if !words.is_empty() && !hints.heading_attested { + // + // `abbrev_probe_attested` admits ONLY a node the S-167 recognizer + // produced (docs/design.md §16); it never widens `heading_attested`'s + // own meaning and never admits `invocation_attested` in general. + if !words.is_empty() && !hints.heading_attested && !hints.abbrev_probe_attested { return Err(ExtractError::Other(format!( "refusing to probe `{} --help`: {:?} is not heading_attested, so it may be a \ fabricated subcommand rather than a real one (spec §6 rule 0)", @@ -685,7 +703,10 @@ fn raw_probe_streams( words: &[String], hints: NodeHints, ) -> Result { - if !words.is_empty() && !hints.heading_attested { + // Same gate as `probe_help_text_reporting_flag`, including the S-167 + // widening (docs/design.md §16): `abbrev_probe_attested` admits only + // that recognizer's own nodes. + if !words.is_empty() && !hints.heading_attested && !hints.abbrev_probe_attested { return Ok(RawProbeOutcome::NotAttested); } @@ -834,6 +855,7 @@ fn not_attested_fallback( &[], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) { if !root_streams.is_empty() { @@ -985,6 +1007,7 @@ mod tests { /// invented word, so `heading_attested: true` is honest throughout. const ATTESTED: NodeHints = NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; fn fixture(name: &str) -> String { @@ -1033,6 +1056,7 @@ mod tests { &["pkill".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("`pkill --help` is the one permitted shape and must be shown"); @@ -1046,6 +1070,7 @@ mod tests { &["pkill".to_string(), "something".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect_err("a positional path must still be refused"); @@ -1092,6 +1117,7 @@ mod tests { &["shimtool".to_string(), "ghost".to_string()], NodeHints { heading_attested: false, + abbrev_probe_attested: false, }, ) .expect("a not-attested refusal must resolve to Ok with an explanation, not Err"); diff --git a/mandible-extract/src/help_text/sections/mod.rs b/mandible-extract/src/help_text/sections/mod.rs index acb5ab0d..9b40eb0d 100644 --- a/mandible-extract/src/help_text/sections/mod.rs +++ b/mandible-extract/src/help_text/sections/mod.rs @@ -27,8 +27,8 @@ use super::grammar::{ }; use super::profile::{heading_matches_markers, FrameworkProfile}; use mandible_core::{ - is_command_name_shaped, strip_escapes, Choice, CommandNode, Dashes, Entity, EntityKind, - Provenance, Source, Spelling, Text, ValueKind, + is_command_name_shaped, is_literal_choice_value, strip_escapes, Choice, CommandNode, Dashes, + Entity, EntityKind, Provenance, Source, Spelling, Text, ValueKind, }; mod backfill; @@ -46,6 +46,7 @@ mod spelling; #[cfg(test)] mod test_support; mod usage; +mod usage_optional_word; use backfill::*; use bullets::*; @@ -62,6 +63,8 @@ use spelling::*; #[cfg(test)] use test_support::*; use usage::*; +pub use usage_optional_word::reconstruct_abbrev_word; +use usage_optional_word::scan_usage_optional_word_table; /// Hard cap on distinct entries (subcommands, flags, or choices) accepted /// from a single probe's output. Real `--help` output never remotely @@ -104,6 +107,13 @@ pub struct ParsedHelp { /// regex, or was dropped for lack of an owning heading/flag. Surfaced /// so `extract_node` can mark the node's provenance as a guess. pub saw_unattributable_content: bool, + /// Flags recovered by `recover_stanza_head_leading_flag_value` + /// specifically, mirrored here (also pushed into `flags`) so the + /// usage-block scan's own dedup can tell "a leading-value recovery of + /// this shape already named a different literal" apart from any other + /// reason a spelling might already be present in `flags`, and relax + /// only for that one shape. See docs/shapes.md S-147. + leading_value_recoveries: Vec, /// Names already accepted into `subcommands`, tracked alongside it so /// [`ParsedHelp::try_push_subcommand`] can reject duplicates in O(1) /// instead of an O(n) scan of `subcommands` per candidate (which would @@ -157,6 +167,37 @@ fn is_ignorable_heading(heading: &str) -> bool { || lower.contains("are equivalent") } +/// True when `candidate` names a literal value distinct from every one of +/// `existing`'s entries — meant to be called only with `existing` set to +/// [`ParsedHelp::leading_value_recoveries`], never the whole document's +/// flags, so this only ever overrides a duplicate this same leading-value +/// shape produced. `lvchange`'s `-M|--persistent` is the case: a stanza +/// head names it `y`, an unrelated paren-alternation row names it `n`; +/// without this the row scan's own dedup would drop `n` outright, +/// replacing a value that used to render rather than letting both reach +/// `merge_entity_bucket`'s union. Refused when the matching recovery is a +/// plain boolean or carries an abbreviation-bracket spelling — narrower +/// scoping alone does not rule out a same-shape false positive from +/// another stanza head. See docs/shapes.md S-147. +fn usage_flag_names_a_new_literal_value(candidate: &Entity, existing: &[Entity]) -> bool { + let Some(name) = candidate.value_name.as_deref() else { + return false; + }; + if !is_literal_choice_value(name) { + return false; + } + let mut same_spelling = existing + .iter() + .filter(|f| flag_spelling_already_present(candidate, std::slice::from_ref(f))) + .peekable(); + same_spelling.peek().is_some() + && same_spelling.all(|f| { + f.value_name.as_deref() != Some(name) + && f.value_kind != ValueKind::None + && !f.spellings.iter().any(|s| s.abbrev.is_some()) + }) +} + /// True when `heading` positively names a section whose rows describe CLI /// flags. Used only to leave an otherwise-contained examples/reporting /// region at the *same* indentation — same-indent text inside a worked @@ -1036,6 +1077,43 @@ struct BodyInput<'a> { bnf_row_lines: &'a std::collections::HashSet, } +/// Recover the flag a stanza head line names, under the group its own +/// description sentence gave it. A head naming exactly one flag is +/// [`recover_stanza_head_flag`]'s shape; a head naming a leading flag's +/// own literal value plus further required flags (`lvcreate --type raid +/// -L|--size Size[m|UNIT] VG`) is not, so the fallback runs only when the +/// first recognizer stays silent, never both, since either would +/// otherwise recover the same leading flag twice. See docs/shapes.md +/// S-147. +fn recover_stanza_head_flags( + heading: &str, + tool_name: Option<&str>, + stanza_label: Option<&str>, + st: &mut BodyScan, +) { + if let Some(mut flag) = recover_stanza_head_flag(heading, tool_name) { + if let Some(label) = stanza_label { + flag.group = Some(label.to_string()); + } + if st.result.flags.len() < MAX_RECOVERED_ENTRIES { + st.result.flags.push(flag); + } + } else if let Some(mut flag) = recover_stanza_head_leading_flag_value(heading, tool_name) { + if let Some(label) = stanza_label { + flag.group = Some(label.to_string()); + } + if st.result.flags.len() < MAX_RECOVERED_ENTRIES { + // Tracked separately from `result.flags` so the usage-block + // dedup (`usage_flag_names_a_new_literal_value`) can tell "a + // leading-value recovery of this exact shape already named a + // different literal" apart from any other reason a spelling + // might already be present, and relax only for the former. + st.result.leading_value_recoveries.push(flag.clone()); + st.result.flags.push(flag); + } + } +} + /// A heading with content indented beneath it. Each recognized section /// shape gets first refusal in turn, and the bare-word block is the /// fallback. See docs/shapes.md S-013, S-019 and S-020. @@ -1090,14 +1168,7 @@ fn emit_heading_block( st.result.usage.push(heading.clone()); } if !st.in_ignorable_section { - if let Some(mut flag) = recover_stanza_head_flag(heading, tool_name) { - if let Some(label) = stanza_label.clone() { - flag.group = Some(label); - } - if st.result.flags.len() < MAX_RECOVERED_ENTRIES { - st.result.flags.push(flag); - } - } + recover_stanza_head_flags(heading, tool_name, stanza_label.as_deref(), st); } // A headed command table whose first row sits on the heading's @@ -1852,7 +1923,21 @@ fn parse_body( None }; let usage_start = labelled_usage_start.or(unlabelled_synopsis_start); - if let Some(start) = usage_start { + // A bare `Usage:` heading (nothing else on that line) whose following + // rows each repeat the tool's own name plus one command word with an + // optional-abbreviation suffix (`lldb-server`'s `v[ersion]`) names + // subcommands, not usage forms. Tried before the ordinary usage scan + // so it never gets a chance to fold these rows into `result.usage` + // instead. See docs/shapes.md S-167. + let optional_word_table = labelled_usage_start.and_then(|start| { + tool_name.and_then(|name| scan_usage_optional_word_table(&lines, start, name)) + }); + if let Some((end, nodes)) = optional_word_table { + i = end; + for node in nodes { + result.try_push_subcommand(node); + } + } else if let Some(start) = usage_start { let scan = scan_usage_section( &lines, start, @@ -1931,13 +2016,20 @@ fn parse_body( if result.flags.len() >= MAX_RECOVERED_ENTRIES { break; } - if !flag_spelling_already_present(&flag, &result.flags) { + if !flag_spelling_already_present(&flag, &result.flags) + || usage_flag_names_a_new_literal_value(&flag, &result.leading_value_recoveries) + { result.flags.push(flag); } // else: this spelling already names a flag the block scan // recovered, so the usage-derived, always-undescribed // duplicate is not added. "Let the described version win" // taken literally: the existing entry is never touched. + // `usage_flag_names_a_new_literal_value` is the one exception + // (S-147): a real, distinct literal value still gets added + // for `merge_entity_bucket` to union later, rather than + // silently replaced by whichever entity happened to land + // first. } } diff --git a/mandible-extract/src/help_text/sections/usage.rs b/mandible-extract/src/help_text/sections/usage.rs index 26f8e1ef..d976378f 100644 --- a/mandible-extract/src/help_text/sections/usage.rs +++ b/mandible-extract/src/help_text/sections/usage.rs @@ -117,6 +117,49 @@ pub(super) fn recover_stanza_head_flag(heading: &str, tool_name: Option<&str>) - Some(flag) } +/// Recover a stanza head's own *leading* flag when it takes an immediate +/// bare literal value and further required flags follow on the same line +/// — `lvcreate --type raid -L|--size Size[m|UNIT] VG`, where +/// [`looks_like_stanza_head_flag`] refuses because `-L|--size` reads as a +/// second flag token. Recovers just the leading flag and its literal +/// value; the line's other flags are documented elsewhere. Gated to +/// [`is_literal_choice_value`] so a positional operand (`VG`, `Size[m| +/// UNIT]`) or glued docopt notation (`blkid`'s `[--match-tag`) is never +/// mistaken for one. `merge::merge_entity_bucket` folds every literal +/// value this and [`recover_stanza_head_flag`]'s bracket-row siblings +/// recover into one `choices` list. See docs/shapes.md S-089, S-147. +pub(super) fn recover_stanza_head_leading_flag_value( + heading: &str, + tool_name: Option<&str>, +) -> Option { + let name = tool_name?; + if is_ignorable_heading(heading) || !starts_with_tool_name(heading, name) { + return None; + } + let rest = heading.strip_prefix(name)?.trim_start(); + let mut words = rest.split_whitespace(); + let flag_tok = words.next()?; + if !is_bare_flag_token(flag_tok) { + return None; + } + let value_tok = words.next()?; + if !is_literal_choice_value(value_tok) { + return None; + } + let spec = parse_flag_spec(flag_tok); + if spec.spellings.is_empty() { + return None; + } + let mut flag = Entity::new( + EntityKind::Flag, + Provenance::single(Source::HelpTextSynopsis), + ); + flag.spellings = spec.spellings; + flag.value_name = Some(value_tok.to_string()); + flag.value_kind = ValueKind::Required; + Some(flag) +} + /// Pull placeholder tokens (``, bare `UPPERCASE` words not preceded /// by `-`) out of usage lines as positionals. Best-effort: usage-line /// grammar is genuinely varied (docopt-style `[OPTIONS]`, ``, diff --git a/mandible-extract/src/help_text/sections/usage_optional_word.rs b/mandible-extract/src/help_text/sections/usage_optional_word.rs new file mode 100644 index 00000000..74a3d90c --- /dev/null +++ b/mandible-extract/src/help_text/sections/usage_optional_word.rs @@ -0,0 +1,206 @@ +//! F11 / docs/shapes.md S-167: a `Usage:` line's leading word is a +//! subcommand spelled with an optional abbreviation suffix — +//! `lldb-server`'s `v[ersion]`, `g[dbserver]`, `p[latform]`. Distinct from +//! S-020's modifier table (`ar`'s `r[ab][f][u]`): there the bracket groups +//! name separate modifier LETTERS glued onto one command letter; here one +//! bracket group spells the rest of ONE whole command word. The node's +//! name and displayed form are that whole word (`gdbserver`); the row's +//! own short prefix (`g`) is kept as an alias, never as `display_name` +//! (docs/design.md §16). Never confused with S-020's own code path. + +use super::heading::{starts_with_tool_name, starts_with_tool_name_spelled_differently}; +use mandible_core::{is_command_name_shaped, CommandNode, Provenance, Source}; + +/// `token` reads as one command word with an optional-abbreviation suffix +/// only when it carries exactly one bracket group, opened right after a +/// single leading lowercase letter, holding nothing but lowercase letters, +/// and closing at the token's own end. `ar`'s `r[ab][f][u]` fails this (a +/// second group follows the first), so the two shapes never collide. +/// +/// Returns `(whole_word, short_alias)`: the full command word with the +/// brackets removed (`gdbserver`), and the bare leading letter the row +/// spelled as its abbreviation prefix (`g`) — the node's own alias, never +/// its displayed name (docs/design.md §16). +fn optional_abbrev_word(token: &str) -> Option<(String, String)> { + let mut chars = token.chars(); + let lead = chars.next()?; + if !lead.is_ascii_lowercase() { + return None; + } + let rest = &token[lead.len_utf8()..]; + let inner = rest.strip_prefix('[')?.strip_suffix(']')?; + if inner.is_empty() || inner.contains(['[', ']']) { + return None; + } + if !inner.chars().all(|c| c.is_ascii_lowercase()) { + return None; + } + let whole = format!("{lead}{inner}"); + if !is_command_name_shaped(&whole) { + return None; + } + Some((whole, lead.to_string())) +} + +/// The existence oracle's own reconstruction (docs/design.md §7 Tier B +/// rule 7, §16): `token` read back to the emitted name, or `None` when it +/// is not this exact shape. Only the bracket characters are removed. +pub fn reconstruct_abbrev_word(token: &str) -> Option { + optional_abbrev_word(token).map(|(whole, _)| whole) +} + +/// One recognized row: the tool's own name (in whatever spelling it printed +/// itself under), then a word matched by [`optional_abbrev_word`], then +/// zero or more further tokens that must each be a single bracketed +/// lowercase word (`[options]`) — anything else and the whole row is +/// refused rather than partially accepted. +fn parse_row(line: &str, tool_name: &str) -> Option<(String, String)> { + let t = line.trim(); + let is_own_name = starts_with_tool_name(t, tool_name) + || starts_with_tool_name_spelled_differently(t, tool_name); + if !is_own_name { + return None; + } + let mut words = t.split_whitespace(); + words.next()?; // the tool's own name, already confirmed above + let first = words.next()?; + let (name, alias) = optional_abbrev_word(first)?; + for trailing in words { + let inner = trailing + .strip_prefix('[') + .and_then(|s| s.strip_suffix(']'))?; + if inner.is_empty() || !inner.chars().all(|c| c.is_ascii_lowercase()) { + return None; + } + } + Some((name, alias)) +} + +/// The fewest recognized rows before this shape is trusted at all: one row +/// alone is too cheap a coincidence to act on. `lldb-server` documents +/// three. See docs/design.md §16 — below AGENTS.md §3.1's five-tool bar, +/// shipped as a recorded exception the way S-103/S-104/S-143 were. +const MIN_ROWS: usize = 2; + +/// Scan the labelled `Usage:` block starting at `heading_idx` (the bare +/// `Usage:` line itself, with nothing else on it) for this shape. +/// `Some((end, nodes))` only when every line from `heading_idx + 1` parses +/// as a row up to the first line that does not, and at least [`MIN_ROWS`] +/// did — a mixed or under-populated run is refused whole, never partially +/// accepted, so an ordinary usage synopsis is never swallowed by a guess. +/// `end` is the index of the first line NOT consumed. +pub(super) fn scan_usage_optional_word_table( + lines: &[&str], + heading_idx: usize, + tool_name: &str, +) -> Option<(usize, Vec)> { + if !lines[heading_idx].trim().eq_ignore_ascii_case("usage:") { + return None; + } + let mut i = heading_idx + 1; + let mut rows: Vec<(String, String)> = Vec::new(); + while let Some(&line) = lines.get(i) { + if line.trim().is_empty() { + break; + } + let Some(row) = parse_row(line, tool_name) else { + break; + }; + rows.push(row); + i += 1; + } + if rows.len() < MIN_ROWS { + return None; + } + let nodes = rows + .into_iter() + .map(|(name, alias)| { + let mut node = CommandNode::new(name.clone(), Provenance::single(Source::HelpText)); + // Invocation-attested, never heading-attested (§7 Tier B rule + // 8), but this recognizer's own third bit admits a probe of + // the full word anyway (§6 rule 0, docs/design.md §16). The + // row's short prefix is kept as an alias, never as a display + // spelling. + node.invocation_attested = true; + node.heading_attested = false; + node.children_filled = false; + node.abbrev_probe_attested = true; + if alias != name { + node.aliases.push(alias); + } + node + }) + .collect(); + Some((i, nodes)) +} + +#[cfg(test)] +mod tests { + use super::*; + + const LLDB_SERVER: &str = "Usage:\n lldb-server v[ersion]\n lldb-server g[dbserver] [options]\n lldb-server p[latform] [options]\nInvoke subcommand for additional help\n"; + + #[test] + fn recovers_three_named_nodes_with_the_full_word_as_name_and_the_prefix_as_alias() { + let lines: Vec<&str> = LLDB_SERVER.lines().collect(); + let (end, nodes) = scan_usage_optional_word_table(&lines, 0, "lldb-server").unwrap(); + assert_eq!(end, 4, "stops at the trailing prose sentence"); + assert_eq!(nodes.len(), 3); + assert_eq!(nodes[0].name, "version"); + assert_eq!(nodes[0].display_name, None, "no bracketed display form"); + assert_eq!(nodes[0].aliases, vec!["v".to_string()]); + assert!(nodes[0].invocation_attested); + assert!(!nodes[0].heading_attested); + assert!(nodes[0].abbrev_probe_attested); + assert_eq!(nodes[1].name, "gdbserver"); + assert_eq!(nodes[1].display_name, None); + assert_eq!(nodes[1].aliases, vec!["g".to_string()]); + assert!(nodes[1].abbrev_probe_attested); + assert_eq!(nodes[2].name, "platform"); + assert_eq!(nodes[2].display_name, None); + assert_eq!(nodes[2].aliases, vec!["p".to_string()]); + assert!(nodes[2].abbrev_probe_attested); + } + + #[test] + fn matches_the_resolved_full_path_spelling_too() { + let raw = "Usage:\n /usr/bin/lldb-server v[ersion]\n /usr/bin/lldb-server g[dbserver] [options]\n /usr/bin/lldb-server p[latform] [options]\n"; + let lines: Vec<&str> = raw.lines().collect(); + let (_, nodes) = scan_usage_optional_word_table(&lines, 0, "lldb-server").unwrap(); + assert_eq!(nodes.len(), 3); + } + + #[test] + fn refuses_ars_modifier_shape_a_second_bracket_group_follows() { + assert_eq!(optional_abbrev_word("r[ab][f][u]"), None); + } + + #[test] + fn accepts_a_single_bracket_group_that_spells_a_whole_word() { + assert_eq!( + optional_abbrev_word("v[ersion]"), + Some(("version".to_string(), "v".to_string())) + ); + } + + #[test] + fn refuses_a_single_row_below_the_min_rows_floor() { + let raw = "Usage:\n lldb-server v[ersion]\n"; + let lines: Vec<&str> = raw.lines().collect(); + assert!(scan_usage_optional_word_table(&lines, 0, "lldb-server").is_none()); + } + + #[test] + fn refuses_a_mixed_run_whole_rather_than_partially() { + let raw = "Usage:\n lldb-server v[ersion]\n lldb-server [options] \n"; + let lines: Vec<&str> = raw.lines().collect(); + assert!(scan_usage_optional_word_table(&lines, 0, "lldb-server").is_none()); + } + + #[test] + fn refuses_a_labelled_usage_line_carrying_its_own_synopsis() { + let raw = "Usage: foo [OPTIONS]\n foo v[ersion]\n foo g[dbserver]\n"; + let lines: Vec<&str> = raw.lines().collect(); + assert!(scan_usage_optional_word_table(&lines, 0, "foo").is_none()); + } +} diff --git a/mandible-extract/src/native/mod.rs b/mandible-extract/src/native/mod.rs index 8fccf9d0..145d4f33 100644 --- a/mandible-extract/src/native/mod.rs +++ b/mandible-extract/src/native/mod.rs @@ -914,6 +914,7 @@ mod tests { &["cobratool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("detect having succeeded, extract_node must too"); @@ -961,6 +962,7 @@ mod tests { &["cobratool".to_string(), "stop".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("detect having succeeded, extract_node must too"); diff --git a/mandible-extract/src/overrides/mod.rs b/mandible-extract/src/overrides/mod.rs index ee34b63c..eee3bcbc 100644 --- a/mandible-extract/src/overrides/mod.rs +++ b/mandible-extract/src/overrides/mod.rs @@ -350,6 +350,7 @@ mod tests { &["mytool".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("root override should resolve"); @@ -377,6 +378,7 @@ mod tests { &["mytool".to_string(), "some-subcommand".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!(matches!(result, Err(ExtractError::PathNotFound))); diff --git a/mandible-extract/src/runner.rs b/mandible-extract/src/runner.rs index f90c5baa..494fc11b 100644 --- a/mandible-extract/src/runner.rs +++ b/mandible-extract/src/runner.rs @@ -179,6 +179,7 @@ impl Runner { // `NodeHints::heading_attested`. let root_hints = NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; for tier in &self.tiers { @@ -239,6 +240,8 @@ impl Runner { // for this hint. Read before `existing` moves into `candidates`. let hints = NodeHints { heading_attested: existing.heading_attested, + // Same reasoning: mirrors the node's own S-167 bit (spec §6 rule 0). + abbrev_probe_attested: existing.abbrev_probe_attested, }; let mut candidates = vec![existing]; let mut statuses = Vec::new(); diff --git a/mandible-extract/src/tier.rs b/mandible-extract/src/tier.rs index 4202aa61..7a9892ed 100644 --- a/mandible-extract/src/tier.rs +++ b/mandible-extract/src/tier.rs @@ -19,6 +19,14 @@ pub struct NodeHints { /// heuristic. A non-attested node is never probed, in any shape; the /// tier declines with a per-node error instead. Spec §6 rule 0, §5.3. pub heading_attested: bool, + /// True only for a node the S-167 recognizer produced: a `Usage:` line + /// whose leading word carries a bracketed optional-abbreviation suffix + /// (docs/shapes.md S-167). A third, separate attestation reason from + /// `heading_attested`, which must keep meaning "came from a recognized + /// command heading" and stay honest. Mirrors + /// [`mandible_core::CommandNode::abbrev_probe_attested`]; never set from + /// `invocation_attested` in general (spec §6 rule 0). + pub abbrev_probe_attested: bool, } /// One source of `CommandNode` data: a known-spec catalog, `--help` grammar diff --git a/mandible-extract/tests/completion_script_gate.rs b/mandible-extract/tests/completion_script_gate.rs index 4907c419..892def42 100644 --- a/mandible-extract/tests/completion_script_gate.rs +++ b/mandible-extract/tests/completion_script_gate.rs @@ -102,6 +102,7 @@ fn a_tool_with_no_completion_evidence_never_receives_a_completion_argv() { &["widgetd".to_string()], mandible_extract::NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!(result.is_err(), "expected a refusal, got {result:?}"); @@ -142,6 +143,7 @@ fn a_tool_that_advertises_the_command_is_still_probed_and_still_extracts() { &["widgetctl".to_string()], mandible_extract::NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("evidence is present, so extraction must proceed exactly as before"); diff --git a/mandible-extract/tests/corpus_snapshot_format.rs b/mandible-extract/tests/corpus_snapshot_format.rs index f55b4873..3df92a1b 100644 --- a/mandible-extract/tests/corpus_snapshot_format.rs +++ b/mandible-extract/tests/corpus_snapshot_format.rs @@ -32,6 +32,7 @@ use std::sync::Arc; /// production. const ATTESTED: NodeHints = NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; fn fixture(name: &str) -> String { diff --git a/mandible-extract/tests/exec_policy.rs b/mandible-extract/tests/exec_policy.rs index 5a68f361..f41b5a61 100644 --- a/mandible-extract/tests/exec_policy.rs +++ b/mandible-extract/tests/exec_policy.rs @@ -489,6 +489,7 @@ exit 1 let path = ["manthing".to_string(), "sub".to_string()]; let attested = NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; let (raw, flag) = mandible_extract::help_text::raw_help(&tool, &path, attested) @@ -553,6 +554,7 @@ exit 1 &["manthing".to_string(), "sub".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("the shim always answers one of the two probes it's asked for"); @@ -602,6 +604,7 @@ fn never_probe_named_shim_never_receives_the_dash_h_fallback_even_when_man_shape &["pkill".to_string(), "sub".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); @@ -658,6 +661,7 @@ fn non_attested_subcommand_word_is_never_probed_at_all() { &["unattested".to_string(), "sub".to_string()], NodeHints { heading_attested: false, + abbrev_probe_attested: false, }, ); @@ -710,6 +714,7 @@ exit 1 &["attested".to_string(), "sub".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("an attested word's --help probe must still run and succeed"); @@ -722,6 +727,100 @@ exit 1 assert!(long_flags.contains(&"amend"), "{long_flags:?}"); } +/// Spec §6 rule 0's S-167 widening (docs/design.md §16): a node that is +/// `abbrev_probe_attested` but NOT `heading_attested` must still be +/// probed with ` --help`, proving the new bit is admitted on its own, +/// not merely tolerated alongside `heading_attested`. Mirrors +/// `attested_subcommand_word_is_still_probed_with_dash_dash_help` above, +/// with the two hints swapped. +#[test] +fn abbrev_probe_attested_word_is_probed_even_though_not_heading_attested() { + let dir = tempfile::tempdir().unwrap(); + let script = r#"#!/bin/sh +if [ "$1" = "gdbserver" ] && [ "$2" = "--help" ]; then + touch "$0.help_ran" + echo "Usage: lldb-server gdbserver [options]" + echo "" + echo "Options:" + echo " --port Bind to this port" + exit 0 +fi +echo "unexpected argv: $@" >&2 +exit 1 +"#; + let shim = write_named_shim(dir.path(), "lldb-server", script); + + let tier = HelpTextTier::default(); + let tool = ResolvedTool { + name: "lldb-server".to_string(), + path: Some(shim.clone()), + version: None, + }; + let node = tier + .extract_node( + &tool, + &["lldb-server".to_string(), "gdbserver".to_string()], + NodeHints { + heading_attested: false, + abbrev_probe_attested: true, + }, + ) + .expect("an abbrev_probe_attested word's --help probe must still run and succeed"); + + assert!( + dir.path().join("lldb-server.help_ran").exists(), + "the --help probe never ran for an abbrev_probe_attested word" + ); + let long_flags: Vec<&str> = node.flags().filter_map(|f| f.long()).collect(); + assert!(long_flags.contains(&"port"), "{long_flags:?}"); +} + +/// Rule 0's thirteen-program list still wins unconditionally over the +/// S-167 widening: a shim named like a never-probe tool must be refused +/// even when the deeper word carries `abbrev_probe_attested: true`, the +/// same "before any spawn" guarantee +/// `never_probe_named_shim_never_receives_the_dash_h_fallback_even_when_man_shaped` +/// proves for `heading_attested`. `run_inert`'s chokepoint refuses the +/// non-`["--help"]` argv before this tier's gate is even consulted, so this +/// also proves the new bit cannot be used to route around the list. +#[test] +fn rule_0_still_refuses_an_abbrev_probe_attested_word_naming_a_never_probe_tool() { + let dir = tempfile::tempdir().unwrap(); + let shim = write_named_shim( + dir.path(), + "pkill", + "#!/bin/sh\ntouch \"$0.ran\"\necho ran\n", + ); + + let tier = HelpTextTier::default(); + let tool = ResolvedTool { + name: "pkill".to_string(), + path: Some(shim.clone()), + version: None, + }; + let result = tier.extract_node( + &tool, + &["pkill".to_string(), "sub".to_string()], + NodeHints { + heading_attested: false, + abbrev_probe_attested: true, + }, + ); + + assert!( + matches!( + result, + Err(ExtractError::Exec(ExecError::RefusedUnsafeTool { .. })) + ), + "expected the never-probe list to refuse the subcommand `--help` probe outright \ + regardless of abbrev_probe_attested, got {result:?}" + ); + assert!( + !dir.path().join("pkill.ran").exists(), + "the never-probe shim was executed at all — refusal did not happen before spawn" + ); +} + /// Mirrors `non_attested_subcommand_word_is_never_probed_at_all`, but for /// spec §7 Tier B's headingless-invocation-table recognizer specifically — /// `mandible-extract/src/help_text/sections/scan.rs`'s @@ -768,6 +867,7 @@ exit 1 &["btrfslike".to_string()], NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ) .expect("root probe must succeed"); @@ -795,6 +895,7 @@ exit 1 &["btrfslike".to_string(), "device".to_string()], NodeHints { heading_attested: device.heading_attested, + abbrev_probe_attested: false, }, ); assert!( @@ -882,6 +983,7 @@ fn attestation_gate_is_load_bearing_probe_would_have_fired_without_it() { fn attested() -> NodeHints { NodeHints { heading_attested: true, + abbrev_probe_attested: false, } } diff --git a/mandible-extract/tests/native_cobra_gate.rs b/mandible-extract/tests/native_cobra_gate.rs index c235f41a..95c54b00 100644 --- a/mandible-extract/tests/native_cobra_gate.rs +++ b/mandible-extract/tests/native_cobra_gate.rs @@ -107,6 +107,7 @@ fn extract_node_errors_rather_than_probing_when_never_detected() { &["gadget".to_string()], mandible_extract::NodeHints { heading_attested: true, + abbrev_probe_attested: false, }, ); assert!( diff --git a/mandible-tui/tests/detail_pane_width_probe.rs b/mandible-tui/tests/detail_pane_width_probe.rs index 4d6bb2cf..77a27a91 100644 --- a/mandible-tui/tests/detail_pane_width_probe.rs +++ b/mandible-tui/tests/detail_pane_width_probe.rs @@ -35,6 +35,7 @@ fn systemctl_node() -> mandible_core::CommandNode { }; const ATTESTED: mandible_extract::NodeHints = mandible_extract::NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; mandible_extract::ExtractionTier::extract_node( &tier, diff --git a/mandible-tui/tests/detail_sections.rs b/mandible-tui/tests/detail_sections.rs index f6337427..4e83ddfd 100644 --- a/mandible-tui/tests/detail_sections.rs +++ b/mandible-tui/tests/detail_sections.rs @@ -960,6 +960,7 @@ fn a_real_environment_section_reaches_the_screen_through_the_real_pipeline() { }; const ATTESTED: mandible_extract::NodeHints = mandible_extract::NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; let node = mandible_extract::ExtractionTier::extract_node( &tier, diff --git a/mandible/src/app_runner.rs b/mandible/src/app_runner.rs index c8a49a55..8ad401d6 100644 --- a/mandible/src/app_runner.rs +++ b/mandible/src/app_runner.rs @@ -513,6 +513,8 @@ fn apply_effect( let hints = mandible_extract::NodeHints { heading_attested: mandible_core::resolve(&app.root, &path) .is_some_and(|n| n.heading_attested), + abbrev_probe_attested: mandible_core::resolve(&app.root, &path) + .is_some_and(|n| n.abbrev_probe_attested), }; // Through the same redirect the parse itself went through // (spec §5.4), for exactly the reason above: under a diff --git a/mandible/src/report.rs b/mandible/src/report.rs index d9a84b4e..b8cd1561 100644 --- a/mandible/src/report.rs +++ b/mandible/src/report.rs @@ -129,6 +129,7 @@ fn raw_help( // because none was needed. let hints = NodeHints { heading_attested: true, + abbrev_probe_attested: false, }; help_text::raw_help(resolved, &root_path, hints) } diff --git a/xtask/src/audit/signatures.rs b/xtask/src/audit/signatures.rs index 6fc28bda..ba587d98 100644 --- a/xtask/src/audit/signatures.rs +++ b/xtask/src/audit/signatures.rs @@ -113,21 +113,24 @@ pub(super) fn is_bare_stub(node: &CommandNode) -> bool { } /// True for a bare stub ([`is_bare_stub`]) that is also not -/// [`CommandNode::heading_attested`] — its name came from a native/cobra -/// artifact rather than a recognized `--help` heading or headingless -/// invocation table (spec §7 Tier B). Provable from the single extraction -/// pass: `help_text::raw_help` refuses to probe any node whose -/// `heading_attested` bit is false, so unlike an ordinary un-recursed +/// [`CommandNode::heading_attested`] and not +/// [`CommandNode::abbrev_probe_attested`] — its name came from a +/// native/cobra artifact rather than a recognized `--help` heading, a +/// headingless invocation table, or the S-167 usage-abbreviation +/// recognizer (spec §7 Tier B, §6 rule 0, docs/design.md §16). Provable +/// from the single extraction pass: `help_text::raw_help` refuses to +/// probe any node with neither bit set, so unlike an ordinary un-recursed /// subcommand, this one structurally cannot ever be probed. /// /// A headingless-table node still counts here even though it's /// existence-attested (`invocation_attested`) — it exempts a node from /// being counted as *fabricated* ([`crate::status::structure_sanity`]) but -/// does not make it any less permanently un-probed. +/// does not make it any less permanently un-probed. An `abbrev_probe_attested` +/// node is the opposite case: it IS probed, so it must not count here. /// /// Fixture: `corpus/git-lfs/*/help.txt`. pub(super) fn is_attestation_gated_stub(node: &CommandNode) -> bool { - is_bare_stub(node) && !node.heading_attested + is_bare_stub(node) && !node.heading_attested && !node.abbrev_probe_attested } /// Count of [`is_attestation_gated_stub`] matches across `node` and every diff --git a/xtask/src/corpus/contract.rs b/xtask/src/corpus/contract.rs index 8c7478a3..a8b09022 100644 --- a/xtask/src/corpus/contract.rs +++ b/xtask/src/corpus/contract.rs @@ -232,6 +232,7 @@ fn new_field_weakened_lines(label: &str, b: &ContractMeta, n: &ContractMeta) -> } } lines.extend(super::refill_contract::weakened_lines(label, b, n)); + lines.extend(super::refill_contract::choices_weakened_lines(label, b, n)); for spec in b.must_contain_positionals.iter() { let Some(base) = spec.strip_suffix("...") else { continue; diff --git a/xtask/src/corpus/mod.rs b/xtask/src/corpus/mod.rs index f5434004..da8a9e82 100644 --- a/xtask/src/corpus/mod.rs +++ b/xtask/src/corpus/mod.rs @@ -351,6 +351,20 @@ pub(crate) struct ContractMeta { /// docs/shapes.md S-147. #[serde(default)] must_value_names_after_root_refill: std::collections::BTreeMap>, + /// Every value a root flag's `choices` must carry after this + /// fixture's root is refilled the same way + /// `must_value_names_after_root_refill` simulates. Keyed by the + /// flag's own spelling. `must_attach_choices` alone cannot state this + /// claim for a same-spelling merge bucket: it walks the raw, + /// unrefilled tree, where `--type`'s several invocation forms are + /// still separate entities and `.find()` sees only the first one's + /// own (possibly empty) `choices`. S-147 follow-up + /// (docs/design.md §16): a bucket whose forms disagree about + /// a value name unions every literal one into `choices`, and this is + /// the only field that can see the union actually happened. See + /// `refill_contract.rs`. + #[serde(default)] + must_choices_after_root_refill: std::collections::BTreeMap>, /// Which dimensions of this fixture's tree a human actually verified /// before blessing it — machine-readable replacement for the /// "SCOPE OF REVIEW" prose comment (`git show c9bfe76`). Not itself a diff --git a/xtask/src/corpus/refill_contract.rs b/xtask/src/corpus/refill_contract.rs index cc9fdd33..934237cc 100644 --- a/xtask/src/corpus/refill_contract.rs +++ b/xtask/src/corpus/refill_contract.rs @@ -60,21 +60,9 @@ pub(crate) fn check_must_value_names_after_root_refill( if contract.must_value_names_after_root_refill.is_empty() { return failures; } - let Some(root) = root else { - failures.push(ContractFailure( - "must_value_names_after_root_refill: no root produced".into(), - )); + let Some(refilled) = refill(root, &mut failures, "must_value_names_after_root_refill") else { return failures; }; - let refilled = match mandible_core::merge_nodes(vec![root.clone(), root.clone()]) { - Ok(node) => node, - Err(e) => { - failures.push(ContractFailure(format!( - "must_value_names_after_root_refill: simulated refill failed: {e}" - ))); - return failures; - } - }; for (flag_spec, expected_substrings) in &contract.must_value_names_after_root_refill { match refilled .flags() @@ -101,3 +89,106 @@ pub(crate) fn check_must_value_names_after_root_refill( } failures } + +/// `must_choices_after_root_refill`'s own `CONTRACT WEAKENED` lines, the +/// twin of [`weakened_lines`] above for the `choices` half of S-147's +/// follow-up ruling. +pub(crate) fn choices_weakened_lines( + label: &str, + b: &ContractMeta, + n: &ContractMeta, +) -> Vec { + let mut lines = Vec::new(); + for (flag, base_names) in &b.must_choices_after_root_refill { + match n.must_choices_after_root_refill.get(flag) { + None => lines.push(format!( + "CONTRACT WEAKENED: {label} must_choices_after_root_refill[{flag:?}] \ + (assertion removed)" + )), + Some(now_names) => { + for name in base_names { + if !now_names.contains(name) { + lines.push(format!( + "CONTRACT WEAKENED: {label} \ + must_choices_after_root_refill[{flag:?}] ({name:?} dropped)" + )); + } + } + } + } + } + lines +} + +/// The `choices` twin of [`check_must_value_names_after_root_refill`]: +/// asserts every named literal survives in the refilled flag's `choices` +/// list, not its `value_name`. S-147's own follow-up ruling +/// (docs/design.md §16) moves a same-spelling bucket's disagreeing literal values +/// out of `value_name` and into one unioned `choices` list, and +/// `must_attach_choices` alone cannot see that union: it walks the raw, +/// unrefilled tree, where `.find()` sees only the first invocation +/// form's own, usually-empty `choices`. +pub(crate) fn check_must_choices_after_root_refill( + contract: &ContractMeta, + root: Option<&CommandNode>, +) -> Vec { + let mut failures = Vec::new(); + if contract.must_choices_after_root_refill.is_empty() { + return failures; + } + let Some(refilled) = refill(root, &mut failures, "must_choices_after_root_refill") else { + return failures; + }; + for (flag_spec, expected_choices) in &contract.must_choices_after_root_refill { + match refilled + .flags() + .find(|f| entity_matches_flag_spec(f, flag_spec)) + { + None => failures.push(ContractFailure(format!( + "must_choices_after_root_refill[{flag_spec:?}]: flag not present after refill" + ))), + Some(entity) => { + let missing: Vec<&str> = expected_choices + .iter() + .filter(|c| !entity.choices.iter().any(|ch| &ch.name == *c)) + .map(|s| s.as_str()) + .collect(); + if !missing.is_empty() { + failures.push(ContractFailure(format!( + "must_choices_after_root_refill[{flag_spec:?}]: expected the merged \ + choices to include {missing:?}, got {:?}", + entity + .choices + .iter() + .map(|c| c.name.as_str()) + .collect::>() + ))); + } + } + } + } + failures +} + +/// Shared "no root"/"simulated refill failed" handling for both refill +/// checks above, keyed by the caller's own contract-field name so each +/// failure still names the field it belongs to. +fn refill( + root: Option<&CommandNode>, + failures: &mut Vec, + field: &str, +) -> Option { + let Some(root) = root else { + failures.push(ContractFailure(format!("{field}: no root produced"))); + return None; + }; + match mandible_core::merge_nodes(vec![root.clone(), root.clone()]) { + Ok(node) => Some(node), + Err(e) => { + failures.push(ContractFailure(format!( + "{field}: simulated refill failed: {e}" + ))); + None + } + } +} diff --git a/xtask/src/corpus/runner.rs b/xtask/src/corpus/runner.rs index 2aa47c6d..e430e3cb 100644 --- a/xtask/src/corpus/runner.rs +++ b/xtask/src/corpus/runner.rs @@ -271,6 +271,12 @@ fn run_fixtures( root.as_ref(), ), ); + contract_failures.extend( + super::refill_contract::check_must_choices_after_root_refill( + &fixture.meta.contract, + root.as_ref(), + ), + ); let is_xfail = fixture.meta.xfail.as_ref().is_some_and(|x| x.broken); // A missing `expected.snap` is legal only for a fixture still diff --git a/xtask/src/coverage/aggregate.rs b/xtask/src/coverage/aggregate.rs index c40dcbb6..5dd43d57 100644 --- a/xtask/src/coverage/aggregate.rs +++ b/xtask/src/coverage/aggregate.rs @@ -211,6 +211,13 @@ pub struct Aggregate { pub centered_label_baseline_tools: usize, /// Real rows lost to that shape, fleet-wide — one per finding. pub centered_label_baseline_flags: usize, + /// Tools with at least one [`crate::usage_optional_word_table`] + /// finding — a bare `Usage:` block's row abbreviating a subcommand + /// with a bracket suffix (`v[ersion]`), missing from the tree (atlas + /// S-167). + pub usage_optional_word_tools: usize, + /// Abbreviated words lost to that shape, fleet-wide — one per finding. + pub usage_optional_word_flags: usize, } /// Compute aggregate stats over `rows`. @@ -292,6 +299,11 @@ pub(super) fn compute_aggregate(rows: &[Row]) -> Aggregate { let wrapped_command_flags: usize = rows.iter().map(|r| r.wrapped_command_count).sum(); let command_pattern_tools = rows.iter().filter(|r| r.command_pattern_count > 0).count(); let command_pattern_flags: usize = rows.iter().map(|r| r.command_pattern_count).sum(); + let usage_optional_word_tools = rows + .iter() + .filter(|r| r.usage_optional_word_count > 0) + .count(); + let usage_optional_word_flags: usize = rows.iter().map(|r| r.usage_optional_word_count).sum(); let centered_label_baseline_tools = rows .iter() .filter(|r| r.centered_label_baseline_count > 0) @@ -344,6 +356,8 @@ pub(super) fn compute_aggregate(rows: &[Row]) -> Aggregate { wrapped_command_flags, command_pattern_tools, command_pattern_flags, + usage_optional_word_tools, + usage_optional_word_flags, centered_label_baseline_tools, centered_label_baseline_flags, } @@ -408,7 +422,7 @@ pub(super) fn detection_rate_pct(aggregate: &Aggregate) -> f64 { /// `coverage-scoreboard.txt`). pub(super) fn aggregate_footer_line(aggregate: &Aggregate) -> String { format!( - "# aggregate: pct_flags_with_text={:.2} no_tier_count={} suspicious_count={} verbatim_count={} incomplete_count={} man_shaped_count={} zero_flag_ok_count={} misattribution_suspect_tools={} misattribution_column_aligned_tools={} existence_fabrication_tools={} bundle_collapse_tools={} bundle_destroyed_flags={} alternation_defect_tools={} alternation_defect_flags={} command_table_tools={} single_dash_split_tools={} single_dash_split_flags={} repeated_char_tools={} repeated_char_flags={} wrapped_prose_tools={} wrapped_prose_flags={} tail_operand_tools={} tail_operand_flags={} vim_family={} ragged_command_tools={} ragged_command_flags={} wrapped_command_tools={} wrapped_command_flags={} command_pattern_tools={} command_pattern_flags={} centered_label_baseline_tools={} centered_label_baseline_flags={} total={} described_flags={:.4} describable_flags={:.4} total_flags={}\n", + "# aggregate: pct_flags_with_text={:.2} no_tier_count={} suspicious_count={} verbatim_count={} incomplete_count={} man_shaped_count={} zero_flag_ok_count={} misattribution_suspect_tools={} misattribution_column_aligned_tools={} existence_fabrication_tools={} bundle_collapse_tools={} bundle_destroyed_flags={} alternation_defect_tools={} alternation_defect_flags={} command_table_tools={} single_dash_split_tools={} single_dash_split_flags={} repeated_char_tools={} repeated_char_flags={} wrapped_prose_tools={} wrapped_prose_flags={} tail_operand_tools={} tail_operand_flags={} vim_family={} ragged_command_tools={} ragged_command_flags={} wrapped_command_tools={} wrapped_command_flags={} command_pattern_tools={} command_pattern_flags={} usage_optional_word_tools={} usage_optional_word_flags={} centered_label_baseline_tools={} centered_label_baseline_flags={} total={} described_flags={:.4} describable_flags={:.4} total_flags={}\n", aggregate.pct_flags_with_text, aggregate.no_tier_count, aggregate.suspicious_count, @@ -439,6 +453,8 @@ pub(super) fn aggregate_footer_line(aggregate: &Aggregate) -> String { aggregate.wrapped_command_flags, aggregate.command_pattern_tools, aggregate.command_pattern_flags, + aggregate.usage_optional_word_tools, + aggregate.usage_optional_word_flags, aggregate.centered_label_baseline_tools, aggregate.centered_label_baseline_flags, aggregate.total, @@ -611,6 +627,11 @@ pub fn parse_aggregate_footer(scoreboard: &str) -> Option { // no such key. let mut centered_label_baseline_tools = 0usize; let mut centered_label_baseline_flags = 0usize; + // Same reasoning again, brand new field (round 10): a scoreboard + // written before the usage-optional-word-table detector existed + // carries no such key. + let mut usage_optional_word_tools = 0usize; + let mut usage_optional_word_flags = 0usize; for field in line.trim_start_matches("# aggregate:").split_whitespace() { let (key, value) = field.split_once('=')?; match key { @@ -659,6 +680,12 @@ pub fn parse_aggregate_footer(scoreboard: &str) -> Option { "wrapped_command_flags" => wrapped_command_flags = value.parse::().ok()?, "command_pattern_tools" => command_pattern_tools = value.parse::().ok()?, "command_pattern_flags" => command_pattern_flags = value.parse::().ok()?, + "usage_optional_word_tools" => { + usage_optional_word_tools = value.parse::().ok()? + } + "usage_optional_word_flags" => { + usage_optional_word_flags = value.parse::().ok()? + } "centered_label_baseline_tools" => { centered_label_baseline_tools = value.parse::().ok()? } @@ -709,6 +736,8 @@ pub fn parse_aggregate_footer(scoreboard: &str) -> Option { wrapped_command_flags, command_pattern_tools, command_pattern_flags, + usage_optional_word_tools, + usage_optional_word_flags, centered_label_baseline_tools, centered_label_baseline_flags, }) diff --git a/xtask/src/coverage/mod.rs b/xtask/src/coverage/mod.rs index b4fb95ee..8591e94b 100644 --- a/xtask/src/coverage/mod.rs +++ b/xtask/src/coverage/mod.rs @@ -213,6 +213,8 @@ fn row(tool: &str, flags: usize, pct_flags_with_text: Option, status: &'sta wrapped_command_samples: Vec::new(), command_pattern_count: 0, command_pattern_samples: Vec::new(), + usage_optional_word_count: 0, + usage_optional_word_samples: Vec::new(), centered_label_baseline_count: 0, centered_label_baseline_samples: Vec::new(), status, diff --git a/xtask/src/coverage/render_markdown.rs b/xtask/src/coverage/render_markdown.rs index a205d88e..edabd51c 100644 --- a/xtask/src/coverage/render_markdown.rs +++ b/xtask/src/coverage/render_markdown.rs @@ -147,6 +147,16 @@ fn command_pattern_sample_section_markdown(rows: &[Row]) -> String { ) } +/// Markdown twin of [`usage_optional_word_sample_lines_text`]. +fn usage_optional_word_sample_section_markdown(rows: &[Row]) -> String { + sample_section_markdown( + rows.iter() + .flat_map(|r| r.usage_optional_word_samples.iter()), + "\n**Usage-optional-word-table findings** (sample — see \ + `xtask/src/usage_optional_word_table.rs`):\n\n| sample |\n|---|\n", + ) +} + /// The one-paragraph-per-family prose block for the four command-table /// detectors (S-104/S-103/S-141/S-149), factored out of /// [`render_markdown`] to stay under its own line ceiling (AGENTS.md §2). @@ -389,6 +399,7 @@ pub(super) fn render_markdown(rows: &[Row], aggregate: &Aggregate) -> String { out.push_str(&ragged_command_sample_section_markdown(rows)); out.push_str(&wrapped_command_sample_section_markdown(rows)); out.push_str(&command_pattern_sample_section_markdown(rows)); + out.push_str(&usage_optional_word_sample_section_markdown(rows)); out.push_str(¢ered_label_baseline_sample_section_markdown(rows)); // The same machine-readable footer the text format carries, wrapped in // an HTML comment so it stays invisible when rendered but parseable by diff --git a/xtask/src/coverage/render_text.rs b/xtask/src/coverage/render_text.rs index 6cfb447b..ae1d69a8 100644 --- a/xtask/src/coverage/render_text.rs +++ b/xtask/src/coverage/render_text.rs @@ -224,6 +224,14 @@ pub(super) struct Row { /// A few of this row's own findings, pre-formatted, mirroring /// [`Self::wrapped_command_samples`]. pub(super) command_pattern_samples: Vec, + /// [`crate::usage_optional_word_table`]'s own measurement: count of + /// this tool's bare-`Usage:`-block rows abbreviating a subcommand word + /// with a bracket suffix, whose word never reached the tree (atlas + /// S-167). + pub(super) usage_optional_word_count: usize, + /// A few of this row's own findings, pre-formatted, mirroring + /// [`Self::command_pattern_samples`]. + pub(super) usage_optional_word_samples: Vec, /// [`crate::centered_label_baseline::detect_tree`]'s own measurement: /// count of this tool's shallower rows, right after a centered ALL-CAPS /// group label, whose leading word never reached the tree as a @@ -381,6 +389,7 @@ pub(super) fn render_text(rows: &[Row], aggregate: &Aggregate) -> String { out.push_str(&ragged_command_sample_lines_text(rows)); out.push_str(&wrapped_command_sample_lines_text(rows)); out.push_str(&command_pattern_sample_lines_text(rows)); + out.push_str(&usage_optional_word_sample_lines_text(rows)); out.push_str(¢ered_label_baseline_sample_lines_text(rows)); out.push_str(&fingerprint_lines(rows)); out @@ -653,6 +662,17 @@ fn command_pattern_sample_lines_text(rows: &[Row]) -> String { ) } +/// Twin of [`single_dash_sample_lines_text`] for +/// [`crate::usage_optional_word_table`]. +fn usage_optional_word_sample_lines_text(rows: &[Row]) -> String { + sample_lines_text( + rows.iter() + .flat_map(|r| r.usage_optional_word_samples.iter()), + "# usage-optional-word-table findings (sample — judge the false-positive rate \ + yourself):\n", + ) +} + /// Twin of [`single_dash_sample_lines_text`] for /// [`crate::centered_label_baseline`]. fn centered_label_baseline_sample_lines_text(rows: &[Row]) -> String { diff --git a/xtask/src/coverage/score.rs b/xtask/src/coverage/score.rs index cba33bae..71f640b7 100644 --- a/xtask/src/coverage/score.rs +++ b/xtask/src/coverage/score.rs @@ -172,18 +172,14 @@ pub(super) fn score_one(tool: &str) -> Row { } _ => (0, Vec::new()), }; - // Fourth read of the same already-fetched capture, still zero probes - // — `crate::commandtable`'s shape is visible in the text the sweep - // already has, exactly like the three detectors above. + // Fourth read of the same capture, zero probes — `crate::commandtable`. let command_table_count = match (probe.root_help_text(), result.root.as_ref()) { (Some(raw), Some(root)) if !raw.trim().is_empty() => { crate::commandtable::detect(&raw, root).missing.len() } _ => 0, }; - // The two remaining families of the three that share the `short && - // !long && value_name` fingerprint, read off the same capture on the - // same pass and costing the same zero additional subprocess spawns — + // The other two of the three `short && !long && value_name` families — // see `split_family_counts`. let ( single_dash_split_count, @@ -200,10 +196,7 @@ pub(super) fn score_one(tool: &str) -> Row { wrapped_command_count, wrapped_command_samples, ) = ragged_family_detector_counts(probe.root_help_text(), result.root.as_ref()); - let (command_pattern_count, command_pattern_samples) = - command_pattern_counts(probe.root_help_text(), result.root.as_ref()); - let (centered_label_baseline_count, centered_label_baseline_samples) = - crate::centered_label_baseline::score_counts(probe.root_help_text(), result.root.as_ref()); + let pattern_word = command_table_family_counts(probe.root_help_text(), result.root.as_ref()); Row { tool: tool.to_string(), tiers: tiers_label, @@ -240,10 +233,12 @@ pub(super) fn score_one(tool: &str) -> Row { ragged_command_samples, wrapped_command_count, wrapped_command_samples, - command_pattern_count, - command_pattern_samples, - centered_label_baseline_count, - centered_label_baseline_samples, + command_pattern_count: pattern_word.command_pattern_count, + command_pattern_samples: pattern_word.command_pattern_samples, + usage_optional_word_count: pattern_word.usage_optional_word_count, + usage_optional_word_samples: pattern_word.usage_optional_word_samples, + centered_label_baseline_count: pattern_word.centered_label_baseline_count, + centered_label_baseline_samples: pattern_word.centered_label_baseline_samples, status: status.label, fingerprint: build_fingerprint(result.root.as_ref()), } @@ -786,25 +781,68 @@ fn format_command_pattern_sample(finding: &command_pattern_table::Finding) -> St ) } -/// [`command_pattern_table::detect`], run over one tool's already-captured -/// text and tree — same zero-additional-probe reasoning as -/// [`ragged_family_detector_counts`], kept as its own function since this -/// family has no sibling to share a tuple return with. -fn command_pattern_counts(raw: Option, root: Option<&CommandNode>) -> (usize, Vec) { +/// One usage-optional-word-table finding, rendered as a single +/// audit-section line. +fn format_usage_optional_word_sample( + finding: &crate::usage_optional_word_table::Finding, +) -> String { + format!("{:?} (name {:?}) missing", finding.display, finding.name) +} + +/// [`command_table_family_counts`]'s return: one `(count, samples)` pair +/// per command-table-shaped family it bundles, named rather than +/// positional so its call site in [`score_one`] stays a single short +/// binding instead of a multi-line tuple destructure (AGENTS.md §2's line +/// ceiling). +#[derive(Default)] +struct CommandTableFamilyCounts { + command_pattern_count: usize, + command_pattern_samples: Vec, + usage_optional_word_count: usize, + usage_optional_word_samples: Vec, + centered_label_baseline_count: usize, + centered_label_baseline_samples: Vec, +} + +/// [`command_pattern_table::detect`], [`crate::usage_optional_word_table::detect`] +/// and [`crate::centered_label_baseline::score_counts`], run over one +/// tool's already-captured text and tree — same zero-additional-probe +/// reasoning as [`ragged_family_detector_counts`], bundled into one struct +/// return for the same line-count reason as that function. +fn command_table_family_counts( + raw: Option, + root: Option<&CommandNode>, +) -> CommandTableFamilyCounts { let (Some(raw), Some(root)) = (raw, root) else { - return (0, Vec::new()); + return CommandTableFamilyCounts::default(); }; if raw.trim().is_empty() { - return (0, Vec::new()); + return CommandTableFamilyCounts::default(); } - let report = command_pattern_table::detect(&raw, root); - let samples = report + let pattern_report = command_pattern_table::detect(&raw, root); + let pattern_samples = pattern_report .findings .iter() .take(FAMILY_DETECTOR_SAMPLES_PER_ROW) .map(format_command_pattern_sample) .collect(); - (report.finding_count(), samples) + let word_report = crate::usage_optional_word_table::detect(&raw, root); + let word_samples = word_report + .findings + .iter() + .take(FAMILY_DETECTOR_SAMPLES_PER_ROW) + .map(format_usage_optional_word_sample) + .collect(); + let (centered_label_baseline_count, centered_label_baseline_samples) = + crate::centered_label_baseline::score_counts(Some(raw), Some(root)); + CommandTableFamilyCounts { + command_pattern_count: pattern_report.finding_count(), + command_pattern_samples: pattern_samples, + usage_optional_word_count: word_report.finding_count(), + usage_optional_word_samples: word_samples, + centered_label_baseline_count, + centered_label_baseline_samples, + } } /// One centered-label-baseline finding, rendered as a single audit-section diff --git a/xtask/src/detector/detectors_misc.rs b/xtask/src/detector/detectors_misc.rs index 84c4540b..191a6b40 100644 --- a/xtask/src/detector/detectors_misc.rs +++ b/xtask/src/detector/detectors_misc.rs @@ -570,3 +570,44 @@ impl Detector for CommandPatternTable { crate::command_pattern_table::self_checks() } } + +/// `usage-optional-word-table` (`crate::usage_optional_word_table`, atlas +/// S-167): a bare `Usage:` block's own rows repeat the tool's name plus +/// one command word abbreviated with a bracket suffix (`lldb-server`'s +/// `v[ersion]`), which reads as ordinary usage text unless the tree +/// already carries each word as a subcommand with its source spelling. +pub(crate) struct UsageOptionalWordTable; + +impl Detector for UsageOptionalWordTable { + fn name(&self) -> &'static str { + "usage-optional-word-table" + } + fn family(&self) -> Option<&'static str> { + // No label in the closed `DEFECT_FAMILIES` set names this specific + // shape; docs/design.md §13.1e rule 6 makes that a legitimate + // `NotEvaluable` answer rather than a forced nearest match. + None + } + fn describes(&self) -> &'static str { + "a bare `Usage:` block whose rows repeat the tool's own name plus one command word \ + abbreviated with a single bracket suffix (`v[ersion]`), missing from the tree as a \ + subcommand" + } + fn hits(&self, evidence: &ToolEvidence<'_>) -> Vec { + crate::usage_optional_word_table::detect(evidence.raw, evidence.root) + .findings + .iter() + .map(|f| format!("{:?} (name {:?}) missing", f.display, f.name)) + .collect() + } + fn scope(&self) -> Scope { + Scope { + claim: "a bare `Usage:` heading whose rows are ` []`, \ + optionally followed by a lone bracket group, only", + known_exclusions: &[], + } + } + fn self_checks(&self) -> Vec { + crate::usage_optional_word_table::self_checks() + } +} diff --git a/xtask/src/detector/mod.rs b/xtask/src/detector/mod.rs index d262c96c..13bc17b0 100644 --- a/xtask/src/detector/mod.rs +++ b/xtask/src/detector/mod.rs @@ -791,6 +791,7 @@ pub fn registry() -> Vec> { Box::new(headingless_table_in_root_description::HeadinglessTableInRootDescription), Box::new(header_declared_env_column::HeaderDeclaredEnvColumn), Box::new(choice_list_under_placeholder::ChoiceListUnderPlaceholder), + Box::new(UsageOptionalWordTable), ] } diff --git a/xtask/src/existence.rs b/xtask/src/existence.rs index 2d4f3391..f1dc0de6 100644 --- a/xtask/src/existence.rs +++ b/xtask/src/existence.rs @@ -21,6 +21,7 @@ //! tests). Spec: §13.1, §13.1b. use mandible_core::{is_command_name_shaped, CommandNode, Entity, Provenance, Source}; +use std::borrow::Cow; use std::collections::HashSet; /// Whether `flag_char` may not immediately follow (or precede) a candidate @@ -164,9 +165,12 @@ fn list_row_words(raw: &str) -> HashSet<&str> { /// following the tool's own name at the start of a line /// ([`tool_name_prefixed_row_words`]). A subcommand name occurring at none /// of these is what this module calls fabricated. -fn attested_name_positions<'a>(raw: &'a str, root_name: &str) -> HashSet<&'a str> { - let mut set = line_start_words(raw); - set.extend(list_row_words(raw)); +fn attested_name_positions<'a>(raw: &'a str, root_name: &str) -> HashSet> { + let mut set: HashSet> = line_start_words(raw) + .into_iter() + .map(Cow::Borrowed) + .collect(); + set.extend(list_row_words(raw).into_iter().map(Cow::Borrowed)); set.extend(tool_name_prefixed_row_words(raw, root_name)); set } @@ -185,24 +189,54 @@ fn attested_name_positions<'a>(raw: &'a str, root_name: &str) -> HashSet<&'a str /// Widening-only: can only reduce reports, never hide a real fabrication. /// /// Fixture: `corpus/btrfs/*/help.txt`. Spec §7 Tier B. -fn tool_name_prefixed_row_words<'a>(raw: &'a str, root_name: &str) -> HashSet<&'a str> { +/// +/// Also admits the S-167 reconstruction (docs/design.md §7 Tier B rule 7, +/// §16): a token this position holds may additionally attest the name +/// [`mandible_extract::help_text::reconstruct_abbrev_word`] recovers from +/// it (`g[dbserver]` attests `gdbserver`), on top of whatever +/// [`mandible_extract::help_text::strip_optional_modifier_suffix`] already +/// attests for S-020's unrelated shape. Additive only, and only at this +/// same real command-list position — it can only widen what a position +/// already attested, never invent a new position. +/// +/// The row's own name may also be spelled differently than `root_name` +/// (a resolved full path, `/usr/bin/lldb-server`) — the real shape S-167's +/// own recognizer already tolerates +/// ([`mandible_extract::help_text::starts_with_tool_name_spelled_differently`]), +/// so the oracle must recognize the same row it is checking. +fn tool_name_prefixed_row_words<'a>(raw: &'a str, root_name: &str) -> HashSet> { + use mandible_extract::help_text::{ + starts_with_tool_name, starts_with_tool_name_spelled_differently, + }; + let mut out = HashSet::new(); if root_name.is_empty() { return out; } for line in raw.lines() { let trimmed = line.trim_start(); - let Some(rest) = trimmed.strip_prefix(root_name) else { + let Some(first_token) = trimmed.split_whitespace().next() else { continue; }; - if !(rest.is_empty() || rest.starts_with(char::is_whitespace)) { + let is_own_name = starts_with_tool_name(trimmed, root_name) + || starts_with_tool_name_spelled_differently(trimmed, root_name); + if !is_own_name { continue; } + // Sliced from `trimmed` itself (not rebuilt), so `rest` stays + // borrowed at `'a` regardless of which spelling the row's own + // first token used. + let rest = &trimmed[first_token.len()..]; for token in rest.split_whitespace().take(2) { - let bare = token.trim_end_matches([':', ',', ';']); - let bare = mandible_extract::help_text::strip_optional_modifier_suffix(bare); + let punct_trimmed = token.trim_end_matches([':', ',', ';']); + if let Some(reconstructed) = + mandible_extract::help_text::reconstruct_abbrev_word(punct_trimmed) + { + out.insert(Cow::Owned(reconstructed)); + } + let bare = mandible_extract::help_text::strip_optional_modifier_suffix(punct_trimmed); if is_command_name_shaped(bare) { - out.insert(bare); + out.insert(Cow::Borrowed(bare)); } else { break; } @@ -694,7 +728,7 @@ fn walk( node: &CommandNode, path: &str, raw: &str, - attested: &HashSet<&str>, + attested: &HashSet>, operands: &HashSet<&str>, out: &mut Vec, ) { @@ -1254,6 +1288,79 @@ mod tests { ); } + // --- S-167 reconstruction (docs/design.md §7 Tier B rule 7, §16) --- + + const LLDB_SERVER_USAGE_TEXT: &str = "Usage:\n lldb-server v[ersion]\n lldb-server g[dbserver] [options]\n lldb-server p[latform] [options]\nInvoke subcommand for additional help\n"; + + /// The regression this whole amendment exists for: `gdbserver` never + /// occurs as a contiguous substring of `lldb-server`'s own raw text + /// (only `g[dbserver]` does), yet it is the emitted node's real name, + /// not an invention. The reconstruction must attest it. + #[test] + fn tool_name_prefixed_row_words_attests_the_bracket_deleted_reconstruction() { + let words = tool_name_prefixed_row_words(LLDB_SERVER_USAGE_TEXT, "lldb-server"); + assert!(words.contains("gdbserver")); + assert!(words.contains("version")); + assert!(words.contains("platform")); + } + + /// The real captured shape: `lldb-server --help` spells its own rows + /// with the resolved full path (`/usr/bin/lldb-server`), never the + /// bare name, so the position check must tolerate that spelling too + /// or the reconstruction above never even runs against a real probe. + #[test] + fn tool_name_prefixed_row_words_attests_the_reconstruction_under_a_full_path_spelling() { + let raw = "Usage:\n /usr/bin/lldb-server v[ersion]\n /usr/bin/lldb-server g[dbserver] [options]\n /usr/bin/lldb-server p[latform] [options]\nInvoke subcommand for additional help\n"; + let words = tool_name_prefixed_row_words(raw, "lldb-server"); + assert!(words.contains("gdbserver"), "{words:?}"); + assert!(words.contains("version"), "{words:?}"); + assert!(words.contains("platform"), "{words:?}"); + } + + /// End-to-end: a tree shaped exactly as the S-167 recognizer builds it + /// (full-word names, no fabricated brackets) reports zero fabrications + /// against lldb-server's own real bytes. + #[test] + fn detect_does_not_flag_lldb_servers_real_abbreviated_subcommands() { + let mut root = help_text_node("lldb-server"); + root.subcommands.push(help_text_node("version")); + root.subcommands.push(help_text_node("gdbserver")); + root.subcommands.push(help_text_node("platform")); + let report = detect(LLDB_SERVER_USAGE_TEXT, &root); + assert_eq!( + report.fabrication_count(), + 0, + "a real S-167 subcommand must not be reported as invented: {:?}", + report + .fabrications + .iter() + .map(|f| &f.name) + .collect::>() + ); + } + + /// Break-it check (AGENTS.md §3.4): a name that is NOT a bracket- + /// deletion of any token in the raw text must still be caught, proving + /// the reconstruction widens existence by exactly one narrow shape and + /// does not blunt the oracle generally. + #[test] + fn detect_still_flags_a_name_that_is_not_a_bracket_deletion_of_anything() { + let mut root = help_text_node("lldb-server"); + root.subcommands.push(help_text_node("gdbserver")); + root.subcommands.push(help_text_node("totallyfabricated")); + let report = detect(LLDB_SERVER_USAGE_TEXT, &root); + let names: Vec<&str> = report + .fabrications + .iter() + .map(|f| f.name.as_str()) + .collect(); + assert_eq!( + names, + vec!["totallyfabricated"], + "the genuine invention must still be flagged, and the real one must not be: {names:?}" + ); + } + #[test] fn line_start_words_excludes_a_mid_line_word() { let raw = " -k, --keep-old-files don't replace existing files when extracting,\n treat them as errors\n"; diff --git a/xtask/src/main.rs b/xtask/src/main.rs index f80533eb..3f0a9a1a 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -37,6 +37,7 @@ mod status; mod tail_operand; mod transition; mod usage_only_value_name; +mod usage_optional_word_table; mod wrapped_command_continuation; mod wrapped_prose; diff --git a/xtask/src/usage_optional_word_table.rs b/xtask/src/usage_optional_word_table.rs new file mode 100644 index 00000000..14d65320 --- /dev/null +++ b/xtask/src/usage_optional_word_table.rs @@ -0,0 +1,239 @@ +//! The `usage-optional-word-table` detector (atlas S-167): a `Usage:` +//! block's own rows repeat the tool's name plus one command word +//! abbreviated with a bracket suffix (`lldb-server`'s `v[ersion]`, +//! `g[dbserver]`, `p[latform]`), which a parser reading the block as an +//! ordinary usage synopsis folds into `usage` instead of recovering as +//! subcommands. Independent of `mandible-extract`'s own recognizer +//! (`help_text::sections::usage_optional_word`): this module re-derives +//! the raw shape from the captured bytes alone and checks the tree for +//! absence, the same separation every other detector in this crate keeps. + +use mandible_core::CommandNode; + +/// `token` reads as one command word abbreviated by a single bracket +/// group opened right after one leading lowercase letter (`v[ersion]`). +/// Mirrors `mandible-extract`'s own `optional_abbrev_word` shape test, +/// kept as an independent copy since that crate cannot import this one. +fn looks_like_optional_abbrev_word(token: &str) -> bool { + let mut chars = token.chars(); + let Some(lead) = chars.next() else { + return false; + }; + if !lead.is_ascii_lowercase() { + return false; + } + let rest = &token[lead.len_utf8()..]; + let Some(inner) = rest.strip_prefix('[').and_then(|s| s.strip_suffix(']')) else { + return false; + }; + !inner.is_empty() && inner.chars().all(|c| c.is_ascii_lowercase()) +} + +/// The recovered word (`v[ersion]` -> `version`) for a token +/// [`looks_like_optional_abbrev_word`] admits. +fn abbrev_word_name(token: &str) -> String { + let mut chars = token.chars(); + let lead = chars.next().expect("checked non-empty by the caller"); + let rest = &token[lead.len_utf8()..]; + let inner = rest + .strip_prefix('[') + .and_then(|s| s.strip_suffix(']')) + .expect("checked by looks_like_optional_abbrev_word"); + format!("{lead}{inner}") +} + +/// True when `first` is `name` itself or names `name` under a different +/// spelling (a full path, or `name`'s own dotted stem) — the same +/// tolerance `mandible-extract`'s `starts_with_tool_name_spelled_differently` +/// keeps, reimplemented narrowly here. +fn names_the_tool(first: &str, name: &str) -> bool { + let basename = first.rsplit('/').next().unwrap_or(first); + basename == name || basename == name.split('.').next().unwrap_or(name) +} + +/// One raw row this detector's own grammar recognizes, whose recovered +/// name is missing from the tree. +pub struct Finding { + pub name: String, + pub display: String, +} + +pub struct Report { + pub findings: Vec, +} + +impl Report { + pub fn finding_count(&self) -> usize { + self.findings.len() + } +} + +const MIN_ROWS: usize = 2; + +/// Whether `root`'s own subcommands already carry a node named by the +/// row's full word (`name`), with the row's own short prefix (`display`'s +/// leading letter, e.g. `g` for `g[dbserver]`) among that node's +/// `aliases` (docs/design.md §16: the repaired shape shows the full word, +/// never the bracketed spelling, and keeps the prefix as an alias +/// instead of a `display_name`). +fn tree_attests(root: &CommandNode, name: &str, display: &str) -> bool { + let Some(prefix) = display.chars().next() else { + return false; + }; + let prefix = prefix.to_string(); + root.subcommands + .iter() + .any(|c| c.name == name && c.aliases.contains(&prefix)) +} + +pub fn detect(raw: &str, root: &CommandNode) -> Report { + let lines: Vec<&str> = raw.lines().collect(); + let Some(heading_idx) = lines + .iter() + .position(|l| l.trim().eq_ignore_ascii_case("usage:")) + else { + return Report { + findings: Vec::new(), + }; + }; + let mut rows: Vec<(String, String)> = Vec::new(); + let mut i = heading_idx + 1; + while let Some(&line) = lines.get(i) { + let t = line.trim(); + if t.is_empty() { + break; + } + let mut words = t.split_whitespace(); + let Some(first) = words.next() else { + break; + }; + if !names_the_tool(first, &root.name) { + break; + } + let Some(word) = words.next() else { + break; + }; + if !looks_like_optional_abbrev_word(word) { + break; + } + if !words.all(|w| w.starts_with('[') && w.ends_with(']')) { + break; + } + rows.push((abbrev_word_name(word), word.to_string())); + i += 1; + } + if rows.len() < MIN_ROWS { + return Report { + findings: Vec::new(), + }; + } + let findings = rows + .into_iter() + .filter(|(name, display)| !tree_attests(root, name, display)) + .map(|(name, display)| Finding { name, display }) + .collect(); + Report { findings } +} + +// ---------------------------------------------------------------------- +// Self-checks +// ---------------------------------------------------------------------- + +use crate::detector::{Expect, SelfCheck}; +use mandible_core::{Provenance, Source}; + +pub(crate) const LLDB_SERVER_USAGE: &str = "Usage:\n lldb-server v[ersion]\n lldb-server g[dbserver] [options]\n lldb-server p[latform] [options]\nInvoke subcommand for additional help\n"; + +fn node(name: &str) -> CommandNode { + CommandNode::new(name, Provenance::single(Source::HelpText)) +} + +pub(crate) fn self_checks() -> Vec { + vec![ + SelfCheck { + name: "lldb-server's own bytes, a bare tree missing all three subcommands", + why: "the defect itself: a tool whose only structure is a bare `Usage:` block \ + reads as ordinary usage text, not subcommands, unless the tree already \ + carries the three abbreviated words", + expect: Expect::Fires(3), + raw: LLDB_SERVER_USAGE.to_string(), + root: node("lldb-server"), + }, + SelfCheck { + name: "a correctly repaired tree", + why: "once every row's own full word reaches the tree with the row's short \ + prefix kept as an alias, the detector has nothing left to report", + expect: Expect::Silent, + raw: LLDB_SERVER_USAGE.to_string(), + root: { + let mut root = node("lldb-server"); + let mut version = node("version"); + version.aliases.push("v".to_string()); + let mut gdbserver = node("gdbserver"); + gdbserver.aliases.push("g".to_string()); + let mut platform = node("platform"); + platform.aliases.push("p".to_string()); + root.subcommands = vec![version, gdbserver, platform]; + root + }, + }, + SelfCheck { + name: "an ordinary usage line with real flags, not this shape", + why: "a genuine usage synopsis (`Usage: foo [OPTIONS] `) carries content on \ + the `Usage:` line itself and must never be misread as this bare-heading \ + shape", + expect: Expect::Silent, + raw: "Usage: foo [OPTIONS] \n".to_string(), + root: node("foo"), + }, + SelfCheck { + name: "ar's modifier-table shape, never mistaken for this one", + why: "`r[ab][f][u]` opens a SECOND bracket group right after the first closes, \ + which this detector's own single-bracket-group grammar refuses; ar's shape \ + reaches the tree through the unrelated `commands:`-heading path, never a \ + bare `Usage:` block", + expect: Expect::Silent, + raw: "Usage:\n ar r[ab][f][u]\n ar m[ab]\n".to_string(), + root: node("ar"), + }, + SelfCheck { + name: "a single row is too cheap a coincidence to act on", + why: "one abbreviated word alone could be a coincidence; the floor is two, the \ + same floor the parser itself uses", + expect: Expect::Silent, + raw: "Usage:\n foo v[ersion]\n".to_string(), + root: node("foo"), + }, + ] +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn fires_on_lldb_servers_own_bytes_against_a_bare_tree() { + let report = detect(LLDB_SERVER_USAGE, &node("lldb-server")); + assert_eq!(report.finding_count(), 3); + } + + #[test] + fn every_self_check_holds() { + for case in self_checks() { + let expected = case.expect.expected_hits(); + let report = detect(&case.raw, &case.root); + assert_eq!( + report.finding_count(), + expected, + "{}: expected {} finding(s), got {:?}", + case.name, + expected, + report + .findings + .iter() + .map(|f| &f.display) + .collect::>() + ); + } + } +}