fix(jq): carry the fold register into its UPDATE/EXTRACT nested pipes (#3145) - #3151
Merged
newhoggy merged 2 commits intoSep 18, 2026
Merged
Conversation
CoverageTotal: 93.95% ⚪ 0 pp vs Comparing No per-file coverage changes vs 🔇 0 ignored region(s), 175 tolerated region(s)
Patch coveragePatch: 100% (55/55 new lines covered)
|
CoverageTotal: 94.04% ⚪ 0 pp vs Comparing No per-file coverage changes vs 🔇 0 ignored region(s), 174 tolerated region(s)
Patch coveragePatch: 100% (55/55 new lines covered)
|
…#3145) #3133 gave a pipe stage's `Frame` the register's value, so a pipe nested under `try`/`if`/`,` could seed itself with it. A fold's UPDATE/EXTRACT takes its own route -- `FoldRegister::resolve` passes the register to `resolve_seq` explicitly, under a frame built by `unknown()`/`clone()` -- so the same nesting was still register-less there: `try ($v | .b)` could not re-establish `$v`, raised the resolver's own refusal, and `try` swallowed it. `del(foreach .a as $v (.; try ($v | .b); .))` on `{"a":{"b":1}}` echoed the document where jq writes `{"a":{}}`. The fold's update frame now carries `self.value`, gated on `self.trackable && !tr` in jq mode. The `self.trackable` half is load-bearing and was found by the fuzz, not the sweep: an untracked `FoldRegister`'s `value` is not a register at all (`enter`'s untracked arm seeds it from the ambient), and handing that to a nested pipe fabricated three writes in 18,000 programs -- none of which the sweep rows or the unit matrix reached. Not reached: a fold whose INIT is untracked *after a literal stage* re-seeds its register from the ambient, so the marker is unrecognised there (the pre-existing `literal-then-fold-untracked-init` / `carried-register-passthrough` class) and a `try` around it turns that refusal into a no-op write -- recorded in limitations.md and pinned.
… fuzz knob (#3145 review) The register reached a fold body's nested pipe but not a *sibling* branch of the same multi-output body (those go through `resolve_node`, whose non-pipe arms this frame does not reach -- #2046's documented scope limit), so an UPDATE that used to refuse wholesale could half-succeed: `(foreach .a as {a:$v} ?// {c:$v} (0; ($v[0]?, $v))) = 9` wrote `.a.c`, a key jq never names, because only the second output refused and drove a `?//` retry jq does not perform; `del(foreach .a as $v (.; (($v | .b?), 1); try ($v | .b?)))` turned a loud refusal into a silent no-op write. A body that cannot fan out (`fans_out`) cannot split that way, so the register is carried only there, and the frame stops carrying it on the `Expr::Pipe` arm at all, where the explicit argument already wins and the copy was never read. The fuzz gains `--fold-p`: the changed code is reachable only from a fold, and at the stock 0.2 weight three 6000-program runs drew too few folds to see this class -- the review found it immediately at `--fold-p 1.0`. Both weightings are now at zero fabricate / zero mismatch (3x6000 stock, 3x4000 fold-only including the review's own seed), and both fabricating shapes are pinned in the sweep.
newhoggy
force-pushed
the
issue-3145-jq-fold-body-nested-try-has-no-register
branch
from
September 18, 2026 09:57
5d6a55c to
6b21207
Compare
20 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3145
Refs #3133, #3120, #2046, #1440
Summary
#3133 gave a pipe stage's
Framethe register's value, so a pipe nested undertry/if/,could seed itself with it. A fold's UPDATE/EXTRACT takes its own route —FoldRegister::resolvepasses the register toresolve_seqexplicitly, under a frame built byunknown()/clone()— so the same nesting was still register-less there:try ($v | .b)could not re-establish$v, raised the resolver's own refusal, andtryswallowed it.Fix: the fold's update frame carries
self.value, gated onself.trackable && !tr && !fans_out(expr)in jq mode, and only on the non-Pipearm (thePipearm passes the register explicitly, whichresolve_seq_sinkprefers).The
self.trackablehalf is load-bearing, and the fuzz is what found it. My first gate was!tralone. An untrackedFoldRegister'svalueis not a register at all (enter's untracked arm seeds it from the ambient), and handing that to a nested pipe let markers re-establish where jq refuses: three fabricated writes across 18,000 fuzzed programs (e.g.(foreach first(.a) as {a:{b:$v0}} ?// {c:$v0} ([1]; ($v0[0]?, $v0); ($v0 | getpath([]) | .b?))) = 9, which jq refuses and the wrong gate wrote through). Neither the sweep rows nor the unit matrix reached any of them — the narrowed gate is at zero fabricate / zero mismatch on the same seeds.Not reached: a fold whose INIT is untracked after a literal stage re-seeds its register from the ambient, so the marker is unrecognised there (the pre-existing
literal-then-fold-untracked-init/carried-register-passthroughclass), and atryaround it turns that refusal into a no-op write:del(.a as $y | .a | 5 | foreach range(1) as $i (0; .; try ($y | .b)))is{"a":{}}in jq and echoes here. Recorded inlimitations.mdand pinned in the unit test so a later fix flips it visibly.Review round (
/code-review 3151, standard effort)Three findings, all fixed. The register reached a fold body's nested pipe but not a sibling branch of the same multi-output body (those go through
resolve_node, whose non-pipe arms this frame does not reach — #2046's scope limit), so an UPDATE that refused wholesale could half-succeed:(foreach .a as {a:$v} ?// {c:$v} (0; ($v[0]?, $v))) = 9wrote.a.c, a key jq never names, because only the second output refused and drove a?//retry jq does not perform; anddel(foreach .a as $v (.; (($v | .b?), 1); try ($v | .b?)))turned a loud refusal into a silent no-op write. The gate now declines a body that can fan out, and the frame no longer carries the register on theExpr::Pipearm at all (third finding: that copy was allocated per fold step and never read).The review found this by re-running the PR's own fuzz fold-only — the changed code is reachable only from a fold, and at the stock
FOLD_P = 0.2my 3×6000 runs drew ~3,600 fold programs, under this class's ~1-in-12,000 rate. The script gains a--fold-pflag so that weighting is a flag rather than an edit, and both fabricating shapes are now sweep rows.Acceptance oracle
scripts/jq-bind-origin-oracle-sweep.sh: 6 newfold-*rows;agree=305 fabricate=0 mismatch=0 refuse-only=38 refuse-only-NEW=0.scripts/jq-bind-origin-fuzz.pyagainst a merge-base build: 3 seeds × 6000 stock and 3 seeds × 4000 at--fold-p 1.0(including the review's seed 1001), allfabricate=0 mismatch=0. Earlier gates scored 1 fabricate + 5 mismatch (noself.trackable) and a wrong write at--fold-p 1.0(nofans_out).Test plan
cargo build --features clicargo test --features cli,simd,regex,serde— 8,958 passed, 0 failed (rerun after the review fixes)cargo clippy --all-targets --all-features -- -D warningscargo clippy --all-targets --features std,simd,serde,cli,regex,bench-runner,large-tests,mmap-tests -- -D warningscargo fmt --all -- --check/usr/bin/jq1.7.1 (unit matrix, sweep), including the merge-base binary to show the rows discriminate--baselinetest_fold_body_nested_pipe_keeps_the_register_3145(UPDATE, EXTRACT,if, comma, the sibling-copy controls, and the untouched literal-stage residual)cargo llvm-cov --features cli,simd,regex,serde --workspace— every instrumented added line ineval.rscovered/code-review 3151(standard effort): three findings, all fixed; full verification, sweep and both fuzz weightings rerun after