Formatting output changed in minor version increment

[MilesCranmer]

In 2.5.0, the formatter changed how it wraps a short-form (assignment) method definition whose full line exceeds the margin. 2.4.0 wraps the right-hand-side call arguments; 2.5.0 wraps the left-hand-side signature arguments instead. The two outputs are mutually idempotent, so a codebase formatted with one version is reported as unformatted by the other.

This surfaced as a CI break for us: our format check pins JuliaFormatter@2 (floating), so when 2.5.0 was published, format(".") started reporting files across the repo as needing changes even though nothing in the repo had changed. Pinning back to 2.4.0 resolves it.

Minimal reproducer:

.JuliaFormatter.toml

style = "blue"
pipe_to_function_call = false

x.jl

check_constraints(ex::Union{AbstractExpression,AbstractExpressionNode}, options::AbstractOptions)::Bool = check_constraints(ex, options, options.maxsize)

Run using JuliaFormatter; format(".").

2.4.0 (wraps the call on the RHS):

check_constraints(ex::Union{AbstractExpression,AbstractExpressionNode}, options::AbstractOptions)::Bool = check_constraints(
    ex, options, options.maxsize
)

2.5.0-2.5.2 (wraps the signature on the LHS):

check_constraints(
    ex::Union{AbstractExpression,AbstractExpressionNode}, options::AbstractOptions
)::Bool = check_constraints(ex, options, options.maxsize)

Julia 1.12.6, blue style. 2.4.0 is unaffected; the change first appears in 2.5.0. I tested it on 2.5.2 and it is still there.

Is this an intentional style change? If so, it would help to call it out in the changelog (or have it be opt-in via the config before a v3 is released), since it silently invalidates previously-formatted code whose CI floats on @2.

Many thanks,
Miles

[penelopeysm]

Hi @MilesCranmer, thanks for opening this issue and the helpful repro!

What I'm about to say here is not formalised anywhere, but I'll be writing it up on the docs soon, since 2.5.0 caused some differences in formatting output.

A priori, I think it's kind of expected that formatting should change between versions of JuliaFormatter. If it doesn't, that means it's pointless to release a new version. There are possible arguments against that, but I think that that's generally in line with the way JuliaFormatter has been managed. The question then is what kind of formatting changes are considered good / in line with semver, vs which ones are considered bad / breaking / etc.

Now the extent to which semver should guarantee formatted output, or equivalently the extent to which formatted output is considered "API" of JuliaFormatter, is really quite blurry.

IMO, the only API surface that JuliaFormatter currently guarantees is that:

(1) it obeys all the specifications in the style guide and/or the configuration. That means if you specify margin=92 and a new version of JuliaFormatter formats something above 92 cols, then that's either a bug (if unintentional) or a breaking change (if intentional).

(2) everything in the test suite passes (modulo intentional improvements).

For (1), I don't believe that the Blue style guide, or any configuration option, specifies whether the LHS or RHS of a binary operator is to be broken onto a new line. (Although please do tell me if I missed it somewhere!) I think there is a good argument that breaking the RHS is more aesthetically pleasing and natural — in fact this afternoon I was working on #1012 which is very related. However, I don't think that 'maintain aesthetically pleasing output' is guaranteed by semver of JuliaFormatter.

For (2), this is not good enough imo because the test suite does not (and cannot) contain enough tests to be representative of all Julia code. I've been trying to put together a framework for testing against repos to try to find more cases where formatting differs. For now this is manually invoked (see e.g. #1006 (comment)), but I want to add it to CI, and I'm very happy to add SymbolicRegression as one of the target repos if you want.

TLDR I can try to figure out how to get the old behaviour back, but the motivation for this is only because it's more aesthetically pleasing, and not because I necessarily think it's a bug or a breaking change as per semver. I'll also make a note on the readme and/or changelog that 2.5.0 will have caused some differences in formatting. Finally, I think it's possible that the semver guarantees may change in the future, but that's a long-term thing. I hope that's a sufficiently helpful stance 🙂

Broadly speaking, I would even suggest that pinning the formatter version, and only specifically upgrading it when you want to, is the correct path. I think that this is the saner choice in general because even if there are improvements in the formatted output, it's not ideal to lump formatting upgrades together with other PRs. The formatter upgrades should be their own thing (which then also makes it more compatible with .git-blame-ignore-revs).

[MilesCranmer]

Thanks for explaining. No worries, I simply didn't realise that formatting stability wasn't guaranteed across minor versions.

FWIW, I think it might be nice to have something like rustfmt’s “style editions”, where a given style edition has a hard stability guarantee that newer rustfmt versions cannot modify successfully formatted output of previous versions (even for undocumented style choices). This makes it so that I can pull in bug fixes and performance improvements without worrying about CI breakages due to style churn.

[penelopeysm]

I looked into this, and #1193 would fix this, but at the cost of making formatting worse for other cases (it disables line breaks and causes lines to go over margin) -- for example this is exactly the inverse bug of #911. I think on balance it's better to respect the margin. Now, in principle, this could be very carefully coded with a ton of special-cased stuff so as to keep #911 fixed and also leave this unchanged from v2.4, but honestly, JuliaFormatter's codebase is already complex enough, and introducing more code like that is just going to create technical debt and potentially more bugs. So I'm sorry to disappoint but this is now a feature, not a bug.