Fix --posix to disable the GNU regular expression escapes by LoukasPap · Pull Request #550 · uutils/sed

LoukasPap · 2026-09-05T13:05:25Z

Fixes #542

Under --posix, GNU sed treats the GNU regular expression escapes as the literal character that follows the backslash. These are the operators \?, \+ and \|, the character classes \w, \W, \s and \S, and the empty-width matches \b, \B, \<, \>, \` and \'.

parse_regex_for_mode() now takes a POSIX parameter and emits the literal character for these escapes. The check comes before parse_char_escape(), otherwise \b is decoded as a backspace.

BRE and ERE

The two modes need the opposite spelling of the same literal:

In BRE the character is emitted bare, because bre_to_ere() escapes the ERE metacharacters afterwards. Emitting \? would make bre_to_ere() turn it back into the operator.
In ERE \?, \+ and \| keep their backslash, since a backslash is already what makes them literals there. Emitting them bare would make them operators.

The remaining escapes are letters and punctuation with no meaning of their own, so they are emitted bare in both modes.

codecov · 2026-09-05T13:08:49Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.89%. Comparing base (96afb29) to head (4d29500).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #550      +/-   ##
==========================================
+ Coverage   94.85%   94.89%   +0.04%     
==========================================
  Files          14       14              
  Lines        7230     7294      +64     
  Branches      432      434       +2     
==========================================
+ Hits         6858     6922      +64     
  Misses        371      371              
  Partials        1        1

Flag	Coverage Δ
macos_latest	`96.42% <100.00%> (+0.03%)`	⬆️
ubuntu_latest	`96.36% <100.00%> (+0.03%)`	⬆️
windows_latest	`0.00% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

🟡 Changes recommended

The new integration tests don’t yet cover all GNU escapes called out by #542 (notably \+ and \|), reducing end-to-end regression protection for the claimed fix.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the sed script regex parser so that when --posix is enabled, GNU regex escape extensions (e.g., \?, \+, \|, \w, \s, etc.) are treated as literals rather than GNU-specific operators/classes/anchors, aligning behavior with GNU sed and fixing #542.

Changes:

Thread context.posix into regex parsing (parse_regex_for_mode) and handle GNU escape extensions as literals under POSIX mode (with special handling for ERE metacharacters ? + |).
Add end-to-end regression tests validating --posix disables key GNU regex escapes, plus unit tests covering BRE vs ERE emission rules.

File summaries

File	Description
tests/by-util/test_sed.rs	Adds integration tests for `--posix` behavior around GNU regex escape extensions.
src/sed/delimited_parser.rs	Implements `--posix` parsing behavior for GNU regex escapes and adds targeted unit tests.
src/sed/compiler.rs	Passes `context.posix` into regex parsing for addresses and substitution patterns.

Review details

Files reviewed: 3/3 changed files
Comments generated: 2
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

LoukasPap · 2026-09-05T13:28:37Z

@sylvestre While working on this I noticed that \b, \<, \>, \` and \' are also broken without --posix, where they never match:

$ printf 'cat catalog\n' | sed 's/\<cat\>/X/g'
X catalog
$ printf 'cat catalog\n' | ./target/debug/sed 's/\<cat\>/X/g'
cat catalog

The fix looks small, and differs per escape. All three engines support \<, \>, \b, \A and \z natively, so no lookarounds are needed:

\< and \> already reach Regex::new() unchanged. They break only because NEEDS_RE in fast_regex.rs does not list them, so the pattern is taken for a literal string and remove_escapes() drops the backslash, leaving <cat>. Widening \\[AzBb] to \\[AzBb<>] is enough.
\b is decoded as a backspace by parse_char_escape(), so it has to be intercepted before that call and emitted unchanged.
\` and \' have no equivalent in the Rust engines and need translating to \A and \z.

This is the separate problem from this PR, so I left it out. Do you want to add it in this PR or open a separate issue for it?

Copilot

🟢 Approval recommended

The change is narrowly scoped, threads --posix consistently through the compiler, and includes both unit and integration tests covering the corrected behavior.

Review details

Files reviewed: 3/3 changed files
Comments generated: 0 new
Review effort level: Lite

Copilot

Copilot review overview

🟡 Changes recommended

POSIX escape handling remains incorrect inside bracket expressions and when ?, +, or | is the delimiter.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

POSIX escapes are not literalized inside bracket expressions · New

sylvestre · 2026-09-21T12:19:17Z

                    continue;
                }
+                if posix && is_gnu_regex_escape(line.current()) {
+                    // ? + | must keep their backslash to remain literals.


in ERE the fallback below already emits \? as is, could you just skip this branch for ? + | there instead of pushing the backslash again?

sylvestre · 2026-09-21T12:19:31Z

+
+#[test]
+fn test_posix_keeps_posix_regex_constructs() {
+    // Groups, quantifiers, operators and character escapes are POSIX,


\x61 isn't POSIX, does GNU sed really keep it under --posix? please double check :)

Verified, sed keeps it under --posix. But the test name doesn't capture the case so I'll move it in a new one including \o141 and \d097.

Copilot

Copilot review overview

🔵 Needs a closer look

Two moderate regex parsing issues remain in delimiter and bracket-expression handling.

Review effort: Lite
Findings: None

Resolved since last review (1)

POSIX escapes are not literalized inside bracket expressions

github-actions · 2026-09-24T12:28:37Z

GNU sed testsuite comparison:

Test results comparison:
  Current:   TOTAL: 65 / PASSED: 15 / FAILED: 42 / SKIPPED: 8
  Reference: TOTAL: 65 / PASSED: 14 / FAILED: 43 / SKIPPED: 8

Changes from main branch:
  TOTAL: +0
  PASSED: +1
  FAILED: -1

Test improvements (1):
  + xemacs

codspeed · 2026-10-05T21:23:46Z

Merging this PR will not alter performance

⚠️

1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

access_log_append

✅ 11 untouched benchmarks

_{Comparing LoukasPap:issue542 (4d29500) with main (96afb29)}

Copilot

Copilot review overview

🟡 Changes recommended

Moderate unresolved regex and bracket parsing issues remain.

Review effort: Lite
Findings: 2

Open (2)

Handle escaped '[' as a literal bracket class member · New
Preserve \b literally inside bracket expressions · New

github-actions · 2026-10-08T14:00:14Z

GNU sed testsuite comparison:

Test results comparison:
  Current:   TOTAL: 65 / PASSED: 21 / FAILED: 36 / SKIPPED: 8
  Reference: TOTAL: 65 / PASSED: 18 / FAILED: 39 / SKIPPED: 8

Changes from main branch:
  TOTAL: +0
  PASSED: +3
  FAILED: -3

Test improvements (3):
  + posix-char-class
  + word-delim
  + xemacs

Copilot

🟢 Approval recommended

No unresolved review issues were identified.

1 open finding

Handle escaped '[' as a literal bracket class member

1 resolved since last review

Preserve \b literally inside bracket expressions

🧠 Review effort: Lite

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

sylvestre · 2026-10-10T14:10:44Z

                break;
            }
-            if let Some(decoded) = parse_char_escape(line) {
+            let decoded = if posix { None } else { parse_char_escape(line) };


under --posix [\n] now holds \ and n, are you sure GNU sed doesn't still turn it into a newline there?
printf 'a\nb\n' | sed --posix 'N;s/[\n]/X/' please check and add a test :)

Just verified it holds \ and n under --posix so the newline survives. I'll include a test.

sylvestre · 2026-10-10T14:10:50Z

+                // Outside the character escapes a \ is an ordinary member of the set,
+                // so the next character is left for the loop to handle.  This is what
+                // makes `[a\]bc]` end at the first `]`, as GNU sed does.
+                result.extend_from_slice(br"\\");


this also changes [\/] (escaped delimiter in a class): it now holds \ and /, GNU keeps only /
echo 'a\b/c' | sed 's/[\/]/X/g' should give a\bXc
could you please handle the delimiter before this and add a test? common idiom for paths

GNU keeps both members here. Inside a bracket expression the backslash doesn’t quote the delimiter, it’s an ordinary member of the set:

$ echo 'a\b/c' | sed 's/[\/]/X/g' aXbXc $ echo 'a\b/c' | ./target/debug/sed 's/[\/]/X/g' aXbXc

- Add a POSIX parameter to `parse_regex_for_mode()` that emits the literal character for `\?`, `\+`, `\|`, `\w`, `\W`, `\s`, `\S`, `\b`, `\B`, `\<`, `\>`, `` \` `` and `\'` - Emit the character bare in BRE, where `bre_to_ere()` escapes it afterwards, and keep the backslash on `\?`, `\+` and `\|` in ERE - Add unit and integration tests

…or ? + |

A backslash is only special inside a bracket expression when it starts a character escape. Elsewhere it is an ordinary member of the set and does not quote the character that follows, so `[\w]` matches a backslash or a w, and `[a\]bc]` ends at the first `]`. This matches GNU sed in both basic and extended mode.

A '[' inside a bracket expression that does not open a POSIX class, collating symbol or equivalence is an ordinary member of the set. It was emitted together with the character that followed it, so the ']' in `[\[]` was swallowed and the expression was reported as unterminated. sed has no \b character escape, so remove it from parse_char_escape. A backslash in `[\b]` is an ordinary member of the set, '\b' in a regular expression is a word boundary, and in text and transliteration strings it stands for a bare 'b'. Disable the remaining GNU character escapes inside bracket expressions under --posix. They stay available outside them, as GNU sed does, so `[\t]` matches a backslash or a t while `\t` still matches a tab.

github-actions · 2026-10-11T13:20:57Z

GNU sed testsuite comparison:

Test results comparison:
  Current:   TOTAL: 65 / PASSED: 21 / FAILED: 36 / SKIPPED: 8
  Reference: TOTAL: 65 / PASSED: 18 / FAILED: 39 / SKIPPED: 8

Changes from main branch:
  TOTAL: +0
  PASSED: +3
  FAILED: -3

Test improvements (3):
  + posix-char-class
  + word-delim
  + xemacs

Copilot

🔵 Needs a closer look

Preserve \b backspace handling for non-regex syntax while applying POSIX behavior only to regex parsing.

0 open findings

1 resolved since last review

Handle escaped '[' as a literal bracket class member

🧠 Review effort: Lite

Copilot AI lite review requested due to automatic review settings September 5, 2026 13:05

Copilot started reviewing on behalf of LoukasPap September 5, 2026 13:06 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Comment thread src/sed/delimited_parser.rs Outdated

Comment thread tests/by-util/test_sed.rs

Copilot AI review requested due to automatic review settings September 5, 2026 13:40

LoukasPap force-pushed the issue542 branch from b195503 to f7a7703 Compare September 5, 2026 13:40

Copilot started reviewing on behalf of LoukasPap September 5, 2026 13:41 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 21, 2026 12:01

LoukasPap force-pushed the issue542 branch from f7a7703 to 70766ac Compare September 21, 2026 12:01

Copilot started reviewing on behalf of LoukasPap September 21, 2026 12:05 View session

Copilot AI reviewed Sep 21, 2026

View reviewed changes

Comment thread src/sed/delimited_parser.rs Outdated

sylvestre reviewed Sep 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 24, 2026 12:23

LoukasPap force-pushed the issue542 branch from 70766ac to bd3be2b Compare September 24, 2026 12:23

Copilot started reviewing on behalf of LoukasPap September 24, 2026 12:24 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

LoukasPap requested a review from sylvestre September 24, 2026 12:45

LoukasPap force-pushed the issue542 branch from bd3be2b to e177f87 Compare October 5, 2026 21:14

Copilot AI lite review requested due to automatic review settings October 5, 2026 21:14

Copilot AI reviewed Oct 5, 2026

View reviewed changes

Comment thread src/sed/delimited_parser.rs

Comment thread src/sed/delimited_parser.rs

Copilot started reviewing on behalf of LoukasPap October 5, 2026 22:18 View session

Copilot AI lite review requested due to automatic review settings October 8, 2026 13:58

LoukasPap force-pushed the issue542 branch from e177f87 to 1250999 Compare October 8, 2026 13:58

Copilot started reviewing on behalf of LoukasPap October 8, 2026 13:59 View session

Copilot AI reviewed Oct 8, 2026

View reviewed changes

sylvestre reviewed Oct 10, 2026

View reviewed changes

LoukasPap added 4 commits October 11, 2026 16:08

Split test case to separate POSIX and GNU operators and skip branch f…

d26d8d5

…or ? + |

Copilot AI lite review requested due to automatic review settings October 11, 2026 13:16

LoukasPap force-pushed the issue542 branch from 1250999 to 4d29500 Compare October 11, 2026 13:16

Copilot started reviewing on behalf of LoukasPap October 11, 2026 13:18 View session

Copilot AI reviewed Oct 11, 2026

View reviewed changes

Conversation

LoukasPap commented Sep 5, 2026

BRE and ERE

Uh oh!

codecov Bot commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

LoukasPap commented Sep 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

sylvestre Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

sylvestre Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

LoukasPap Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

github-actions Bot commented Sep 24, 2026

Uh oh!

codspeed Bot commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

sylvestre Oct 10, 2026

Choose a reason for hiding this comment

Uh oh!

LoukasPap Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sylvestre Oct 10, 2026

Choose a reason for hiding this comment

Uh oh!

LoukasPap Oct 11, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Reviewers

Assignees

codecov Bot commented Sep 5, 2026 •

edited

Loading

codspeed Bot commented Oct 5, 2026 •

edited

Loading

LoukasPap Oct 11, 2026 •

edited

Loading