Skip to content

jq: large-integer +/- arithmetic can round to a different f64 than real jq even when both operands are non-negative #2906

Description

@newhoggy

Severity: Low

Summary

Found while broadly differential-fuzzing #2631's fix (large-i64 +/-/* results past
2^53). Even with #2631's a as f64 op b as f64 naive-cast approach correctly routing a
past-2^53 exact integer result through jq_bare_float_display, a residual ~5-6% of
random large-integer +/- combinations still print a different double than real jq —
even with both operands non-negative, so this is unrelated to the separately-documented
"unary minus destroys number literal" behavior.

Repro

$ /usr/bin/jq -n '869389897822472004 + 944331'
869389897823416300
$ succinctly jq -n '869389897822472004 + 944331'
869389897823416400

Both sides agree the exact integer sum is 869389897823416335; the divergence is in
which double each implementation's addition produces, not in a shortest-round-trip
tie-break once the double is known (#2542 is unrelated; last digits are even on
both sides so its tie-break mechanism cannot fire here either).

Investigated with Python: neither float(a) + float(b) (each operand cast to f64
independently, then added — what succinctly does) nor float(a + b) (the exact integer
sum cast to f64 once — what a naive "jq must use full-precision decNumber arithmetic and
round once" theory would predict) matches jq's actual answer in every case. Empirically,
across a 400-case-per-operator random sample:

op naive (f64(a) op f64(b)) only exact (f64(a op b)) only both agree neither matches jq
+ 113 11 262 14
- 100 13 269 18
* 107 15 261 17

"naive" is the dominant model (~90% combined with the "both agree" cases), which is why
#2631's fix (adopting it) is a strict improvement over main, where every past-2^53
case was wrong (100% divergence, since the exact integer was kept verbatim). But the
remaining ~4% "neither" bucket shows real jq's actual arithmetic is doing something more
subtle than either simple model — almost certainly related to jq 1.7's decNumber-backed
literal preservation interacting with plain f64 arithmetic in a mixed, not-fully-naive
way (e.g. converting a literal's exact decimal digits and the other side's already-computed
double back into decNumber's own arbitrary-precision decimal arithmetic before rounding
once to f64 for the final answer only when at least one operand is still a "live" literal
at the time of the operation) — but this hasn't been traced to jq's actual source, and no
model tried so far explains 100% of cases.

Why this is out of scope for #2631

#2631 was specifically about succinctly's own fast path bypassing jq_bare_float_display
entirely for an exact, non-overflowing i64 result — a bug in whether the shared
shortest-round-trip formatter was reached at all, not in what floating-point value reaches
it. This issue is about the floating-point value(s) submitted to that formatter sometimes
being computed for a different double than real jq's own arithmetic would produce, which
is a deeper, more fundamental gap in matching jq's own arbitrary-precision number model —
likely requiring a real decimal (decNumber-equivalent) arithmetic implementation for +/-
(and possibly *//) to close fully, not just a formatting-path fix.

Suggested fix direction

Reproduce jq's own jv_number/jvp_number_value/decNumber-literal handling for binary
arithmetic directly from jq's C source (not guessed from black-box behavior) to determine
the actual per-operation precision model, then decide whether replicating it is worth the
cost of an arbitrary-precision decimal dependency versus documenting the residual gap in
docs/compliance/jq/limitations.md per
ADR-0018.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FableSuitable for a Fable-class model to implement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions