tokenizer: an escaped backtick opens a nested substitution (#732)

[berrym]

Closes #732.

The rule

POSIX 2.6.3: inside `...` a backslash keeps its literal meaning except before $, a backtick or another backslash, where it escapes that byte. An embedded \` is therefore the delimiter of a substitution one level down, and removing those backslashes is what lets the body parse as one.

Two independent defects, which is why the spellings failed differently

1. The unquoted scanner had no escape handling at all, so the span ended at the first \` and the rest of the line mis-parsed:

$ lush -c 'echo `echo \`printf 4\``'
error[E1004]: unterminated backtick command substitution
                          # bash, zsh, dash: 4

The double-quoted reader (src/tokenizer.c) and lush_dequote_span both already consumed an escaped byte correctly. Only this path did not — which is also why the quoted spelling produced the milder symptom.

2. The body was parsed verbatim, so a surviving \` reached the sub-parse as a literal backtick and the nested command was echoed as its own source:

$ lush -c 'echo "[`echo \`printf 4\``]"'
[`printf 4`]                # bash, zsh, dash: [4]

lush_backtick_unescape removes exactly the three escapes POSIX names and leaves every other \X untouched, so a body like `printf 'a\tb'` is unaffected.

bash, zsh and dash all evaluate these forms, and lush's own model agrees that one level of quoting is removed on the way into a substitution — mode-invariant, gates nothing.

Two pre-existing defects found while probing (filed, not folded in)

Both verified identical against a build of the parent commit:

• an unquoted backtick substitution preceded by literal text is emitted literally #740 — an unquoted backtick preceded by literal text is emitted literally: echo xprintf 4y → xprintf 4y. No nesting needed; $( ) and the quoted form both work. This is why the unquoted reproduction in the test uses no leading literal.
• an unmatched bracket glob expands to nothing instead of staying literal #741 — an unmatched bracket glob expands to nothing rather than staying literal: echo [4] prints an empty line where bash and dash print [4]. The word disappears, so a command silently loses an argument.

Worth noting the interaction: with nesting involved, the input in #740 used to be a parse error, and now degrades to that same pre-existing literal-text behavior. That is progress in the same direction, not a regression.

Verification

• tests/integration/test_nested_backticks.c — 16 checks: both spellings, an assignment, three levels of nesting, the exact escape set, and the single-level / $( ) forms that must not change. Two checks pin that an unterminated backtick is still diagnosed, since the scanner now skips a byte and must not be able to skip past the end.
• 5 of 16 fail against the parent build.
• Full suite 194/194. ASan 65/65; the new test 1/1 under ASan.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!