Skip to content

uucore, expr: centralize regex in uucore - #14531

Open
wtcpython wants to merge 2 commits into
uutils:mainfrom
wtcpython:uucore-regex
Open

wtcpython wants to merge 2 commits into
uutils:mainfrom
wtcpython:uucore-regex

Conversation

@wtcpython

Copy link
Copy Markdown
Contributor

No description provided.

Comment thread Cargo.toml Outdated
num-prime = "0.5.0"
num-traits = "0.2.19"
fancy-regex = { version = "0.19.1", default-features = false, features = [
fancy-regex = { version = "0.19.2", default-features = false, features = [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we could move the feature selection to src/uucore/Cargo.toml

@codspeed

codspeed Bot commented Sep 13, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚡ 23 improved benchmarks
❌ 1 regressed benchmark
✅ 349 untouched benchmarks
⏩ 50 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation hostname_ip_lookup[100000] 177.6 µs 213.6 µs -16.88%
Memory factor_multiple_u64s[2] 88.8 KB 60.1 KB +47.72%
Memory hostname_basic 91 KB 62.3 KB +46.02%
Memory seq_custom_separator 93.1 KB 64.5 KB +44.5%
Memory wc_lines_extreme_line_lengths[(100000, 200)] 99.7 KB 71 KB +40.39%
Memory nl_many_lines[100000] 100.9 KB 72.2 KB +39.72%
Memory unexpand_many_lines[100000] 101.7 KB 73 KB +39.29%
Simulation complex_relative_date 397.2 µs 285.2 µs +39.27%
Memory expand_custom_tabstops[50000] 101.7 KB 73 KB +39.27%
Memory cut_characters_long_lines 103.2 KB 74.6 KB +38.47%
Memory fold_custom_width[50000] 110.7 KB 82 KB +34.97%
Simulation hostname_basic 427.1 µs 318 µs +34.29%
Memory split_numeric_suffix 115.2 KB 86.6 KB +33.13%
Memory df_deep_directory 115.4 KB 86.7 KB +33.07%
Memory join_custom_separator 120.7 KB 92.1 KB +31.15%
Memory cksum_multiple_files 125.9 KB 97.2 KB +29.51%
Memory rm_recursive_tree 133.2 KB 104.5 KB +27.44%
Memory dd_copy_separate_blocks 183.2 KB 154.5 KB +18.57%
Simulation df_deep_directory 830.3 µs 724.4 µs +14.62%
Memory file_tz_abbreviations 227.9 KB 199.2 KB +14.4%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing wtcpython:uucore-regex (09138e9) with main (a3b3a24)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@xtqqczze

This comment was marked as outdated.

@github-actions

github-actions Bot commented Sep 13, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)

@oech3

This comment was marked as resolved.

@wtcpython
wtcpython force-pushed the uucore-regex branch 2 times, most recently from 846bee8 to 20880e0 Compare September 14, 2026 00:19
@wtcpython wtcpython changed the title uucore, expr: centralize regex in uucore and adopt fancy-regex leftmost-longest uucore, expr: centralize regex in uucore Sep 14, 2026
@wtcpython
wtcpython marked this pull request as ready for review September 14, 2026 00:23
Copilot AI lite review requested due to automatic review settings September 14, 2026 00:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI review requested due to automatic review settings September 15, 2026 14:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sylvestre

Copy link
Copy Markdown
Contributor

@codspeedbot fix this regression

@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown

@codspeedbot fix this regression

Fixed in 3d05af4.

What the report was actually showing

The regression was real, not noise: every memory benchmark in the tree gained exactly the same amount — +16 allocations, +4,752 bytes allocated, +4,187 bytes peak — including utilities that never compile a regex. Only the cheapest benchmarks crossed the reporting threshold, which is why the list looks arbitrary. I reproduced it locally with the CodSpeed CLI:

hostname_basic (simulation) instructions
main (94811cc) 318,823
this branch 335,034 (+5.1%)

Cause

uucore/locales/errors/en-US.ftl is parsed on the first lookup that misses every ordinary bundle. That is meant to be an error path only, but a binary that cannot resolve its own strings misses on every id it asks for — and a bench binary calling uumain directly is exactly that. So every benchmark was parsing the whole error resource: 106k of the 319k instructions hostname_basic measures, plus 16 allocations that are never freed.

Growing that shared file therefore charges every benchmark. The eight new regex-error-* messages are ~430 bytes of Fluent, and that is what the 3–4% you saw is: hostname paying for strings it can never reach.

Fix

The ids the error resource defines are now generated at build time and checked before the bundle is built, so a lookup that cannot be in there costs a matches! instead of a parse. The regex centralization itself is untouched.

hostname_basic (simulation) instructions
main (94811cc) 318,823
this branch before the fix 335,034
this branch with the fix 212,632

The regression is gone, and every benchmark drops the error-resource parse it was never supposed to pay — expect improvements across the board in the next report rather than a flat comparison.

Validation: uucore unit tests, the build-script tests, and the expr (164), cksum (388) and printf (143) integration tests pass; cargo fmt and cargo clippy -p uucore --all-targets are clean. expr still reports its regex diagnostics in English and in French (expr: Unmatched ( or \( / expr: Parenthèse ouvrante ( ou \( non appariée), and the expr benchmarks are unchanged (1,448,685 vs 1,448,761 on main).

One caveat on the original report: its base was 94811cc rather than your merge base, and several benchmarks were compared across different CPU models. That did not affect this diagnosis — the memory numbers are byte-identical on both base runs — but it is worth reading the "Environment Differences" section before trusting any sub-10% simulation move on this repo.

Summary
  • Warming up the performance wizard...
  • Investigate the reported regression — Every util's bench gained exactly +16 allocations / +4.2 KB peak and ~5% instructions, including utils that use no regex — a fixed startup cost, not noise.
  • Reproduce locally and locate the extra allocations — Reproduced with the CodSpeed CLI: hostname_basic 318,823 -> 335,034 instructions. The cost is a full Fluent parse of uucore/locales/errors/en-US.ftl, which every bench binary triggers because it calls uumain directly and cannot resolve its own utility strings.
  • Fix the regression — The error-only strings are now consulted only for the ids they define (list generated at build time), so a lookup that misses every bundle no longer parses the resource.
  • Validate the fix and push — hostname_basic 212,632 instructions (-33% vs main), expr benches unchanged; uucore, expr, cksum and printf tests plus fmt/clippy pass. Pushed as 3d05af4.
  • Handling comment — Comment handling finished successfully

Was this response helpful? React with 👍 or 👎 to help us improve CodSpeedBot.

Copilot AI review requested due to automatic review settings September 16, 2026 14:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@SuperMuel

Copy link
Copy Markdown

@sylvestre The agent successfully pushed wtcpython@3d05af4 to the PR’s source branch, but GitHub did not synchronize the PR with the new commit. I’m investigating 👀

@wtcpython
wtcpython force-pushed the uucore-regex branch 2 times, most recently from 6ad71f6 to 73b9fbc Compare September 18, 2026 11:56
Copilot AI review requested due to automatic review settings September 18, 2026 11:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@wtcpython

Copy link
Copy Markdown
Contributor Author

@sylvestre ready

wtcpython and others added 2 commits September 19, 2026 08:35
The error-only resource is parsed on the first lookup that misses every
ordinary bundle. That is not only an error path: a binary that cannot
resolve its own strings misses on every id it asks for, and a bench binary
calling `uumain` directly is exactly that. Every benchmark in the tree
therefore parsed the whole resource -- 106k of the 319k instructions
hostname_basic measures, and 16 allocations it never frees.

Growing the resource consequently charged every benchmark, whether or not
the utility can reach the new strings. The eight regex messages added here
moved hostname_basic from 318,823 to 335,034 instructions (+5.1%) and every
utility's peak memory by 4.2 KB, in utilities that never compile a regex.

Generate the ids the resource defines at build time and check that before
building the bundle, so a lookup that cannot be in there costs a match
instead of a parse:

    hostname_basic         instructions
    main (94811cc)               318823
    this branch                  335034
    this branch, this commit     212632

Measured with
`codspeed run --mode simulation -- cargo codspeed run -p uu_hostname`.
The expr benchmarks are unchanged (1,448,685 against 1,448,761 on main),
and expr still reports its regex diagnostics, in English and in French.
Copilot AI review requested due to automatic review settings September 19, 2026 00:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants