Add FCW for invalid C variadic arguments

[theemathas]

View all comments

Fixes #61275 by adding an FCW for invalid C-variadic arguments: invalid_c_variadic_arguments.

Tracking issue for the FCW: #162483

For the purposes of this FCW, a valid C-variadic argument must either implement VaArgSafe, or be a thin reference.

cc @RalfJung

---

Background

C-variadic functions are functions defined either in Rust or externally via FFI that can accept any number of arguments. However, due to C ABI weirdness, only certain types can be passed as C-variadic arguments.

Previously, we had a check that would cause us to attempt to emit a hard error on commonly mistakenly used C-variadic argument types. In particular, this affected types that are not "directly supported" as a variadic argument, but would be automatically coerced to a supported type in C/C++. For instance, when passing a short, a C/C++ compiler will automatically promote this to int and so on the ABI level, an int gets passed. We do not do such coercions in Rust, so passing an i16 can lead to fatal bugs due to the wrong ABI being used.

In #61275, it was found that this hard error didn't prevent such footguns from occurring when the C-variadic function was called with generic arguments. It was then also later found that this error had a bug that caused it to depend on the details of the type inference algorithm.

The current behavior of this hard error is as follows:

• The check runs during the process of doing type inference. We compute the type of the argument, based on the information so far. If we don't yet know the concrete type, we stop and don't emit an error.
• If the argument is a function item type (as opposed to a function pointer type), we emit an error.
• If the argument is of type f32, i8, i16, u8, u16, or bool, and the type doesn't implement VaArgSafe in the current target, we emit an error.
• Otherwise, we don't emit an error. (Notably, passing a random type like String doesn't trigger this error.)

In #155697, we stabilized the ability to define C-variadic functions in Rust. With it, we also stabilized the VaArgSafe trait. This trait is implemented for types that are supported as variadic arguments. Thus, we now have the ability, in stable Rust, to describe the type requirements for being supported as a C-variadic argument.

Therefore, this PR adds an FCW that would warn against C-variadic arguments that are not VaArgSafe. In generic contexts, users can add a T: VaArgSafe bound to satisfy this lint. This FCW runs after type inference is done, but before monomorphization.

There's a caveat though: There's likely much code in the wild that passes a reference as a C-variadic argument. However, references do not implement VaArgSafe yet. Thus, we don't yet lint when a thin reference is passed as a C-variadic argument.

In the future, I expect that it would be possible to turn this into a hard error by, in the type inference/checking algorithm, adding a trait obligation that requires C-variadic arguments to implement VaArgSafe. (This is similar to what happens when one calls a fn<T: VaArgSafe>(T).) This can technically break code that wasn't previously linted, due to lifetime-dependent where bounds, and maybe due to the effect that the trait bound has on subsequent type inference. I expect this to be extremely unlikely though.

[rustbot]

r? @dingxiangfei2009

rustbot has assigned @dingxiangfei2009.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: compiler
• compiler expanded to 75 candidates
• Random selection from 19 candidates

[RalfJung]

Ah, people changed the syntax of this macro again and now one cannot easily tell whether this will be reported in dependencies or not. :/

That's a side-effect of #141936. @WaffleLapkin why is report_in_depds an optional field? The default is far from obvious. (When I introduced FutureReleaseErrorDontReportInDeps many people were surprised that FCW do not report-in-deps by default. That's why I introduced this name that makes it so obvious. IMO it is a step backwards that now we again have syntax where this is not obvious.)

View changes since the review

[theemathas]

Based on this, it seems that the field being optional is intentional:

rust/compiler/rustc_lint_defs/src/lib.rs

Lines 324 to 335 in 745de6e

	/// If set to `true`, this will make future incompatibility warnings show up in cargo's
	/// reports.
	///
	/// When a future incompatibility warning is first inroduced, set this to `false`
	/// (or, rather, don't override the default). This allows crate developers an opportunity
	/// to fix the warning before blasting all dependents with a warning they can't fix
	/// (dependents have to wait for a new release of the affected crate to be published).
	///
	/// After a lint has been in this state for a while, consider setting this to true, so it
	/// warns for everyone. It is a good signal that it is ready if you can determine that all
	/// or most affected crates on crates.io have been updated.
	pub report_in_deps: bool,

[RalfJung]

I think that's a bad choice. It certainly could have warranted a bit more discussion, given that this effectively reverted changes I made previously (#116049), in terms of what is and is not explicit in the API.

I guess people weren't aware of the prior discussion and didn't realize the downsides of the new API choice. Time to make another PR to make report_in_depds mandatory I guess... except I don't know how to make it mandatory just for FutureReleaseError; we don't need it mandatory for edition errors as those "obviously" are not reported in dependencies. That's the downside of the new structure...

[WaffleLapkin]

I was indeed not aware of the previous discussion, ugh =_=

I think the justification from #141936, "It gets especially unruly if you want to add non-FutureReleaseError* warnings which are included in the reports." was targeted at EditionAndFutureReleaseError which I was working with at the time, in the process of stabilizing never.

Looking at the current structure, I'd say we can put report_in_deps in ReleaseFcw. That adds the assumption that we only want to report warnings in dependencies if we plan to change something in a future release, but I guess that's fine (and is at the very least currently true).

I'll make a PR for this.

[RalfJung]

Looking at the current structure, I'd say we can put report_in_deps in ReleaseFcw. That adds the assumption that we only want to report warnings in dependencies if we plan to change something in a future release, but I guess that's fine (and is at the very least currently true).

I was assuming you'd not want that since it seems to partially revert your PR #141936, by coupling report_in_deps with the reason again. But it sounds great to me so if you can also live with it, all good. :)

I'll make a PR for this.

❤️

[traviscross]

Presumably we'd then add ReleaseFcw to EditionAndFutureReleaseError and to EditionAndFutureReleaseSemanticsChange? (Currently these only include EditionFcw.)

If helpful to factoring, note that lang has been following the policy of setting report_in_deps = true exactly when we make an FCW deny-by-default (and otherwise setting report_in_deps = false). Possibly, after cleaning up any lingering exceptions, it could be OK to lean on that.

[RalfJung]

#159700 is a recent example of a Warn + report_in_deps lint. It also shows up in a lot of dependency trees so maybe that was not a good call and it should have been Warn-only for a while like normal FCWs...

[beetrees]

It would be good to do a crater check run with the lint level set to deny to see if there are any commonly-used types that don't currently implement VaArgSafe.

[RalfJung]

Deny runs aren't actually that great as the error will be ignored in build-deps. Ideally we crater a version of this where we make this a hard error. Sadly that requires more patching of the code. :/

[rustbot]

miri is developed in its own repository. If the Miri part of this change can be broken out, consider making this change to rust-lang/miri instead. However, if Miri needs adjusting for rustc changes, just ignore this message.

cc @rust-lang/miri

[RalfJung]

@rust-lang/lang Nominating for team discussion. :) See the PR description for a summary. We ask for your feedback on which of these three options you would prefer (or whether you'd prefer something entirely different):

1. Trigger the lint on all !VaArgSafe types. Note that even the standard library triggers the lint then, though so far only one place where that happens was discovered.
2. Trigger the lint on all !VaArgSafe types, except thin references.
3. Trigger the lint on all !VaArgSafe types, and make thin references VaArgSafe.

My personal preference is option 3. That standard library code looks entirely reasonable, there is no reason to change it. We define &mut as being ABI-compatible with *mut, so it is odd to make a distinction between them here. That said, making the type VaArgSafe makes it possible to get a reference out of anext_arg, which is obviously very unsafe and the lifetime is entirely unconstrained -- so maybe there is a point to be made for being asymmetric wrt. what we accept on the caller vs callee side.

[rust-bors]

💔 Test for ec4987a failed: CI. Failed jobs:

• try - dist-aarch64-linux-quick (web logs, enhanced plaintext logs)
• try - dist-x86_64-linux-quick (web logs, enhanced plaintext logs)

[theemathas]

Based on the failure in the first attempt at a crater run, the nix crate trips this lint at two places:

• https://docs.rs/crate/nix/latest/source/src/sys/prctl.rs#225 (argument of type NonZeroUsize)
• https://docs.rs/crate/nix/latest/source/src/unistd.rs#1240 (argument of type AtFlags, a repr(transparent) struct around a c_int)

[rust-log-analyzer]

The job test-tidy failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

fmt: checked 7258 files
tidy check
tidy [rustdoc_json (src)]: `rustdoc-json-types` modified, checking format version
tidy: Skipping binary file check, read-only filesystem
tidy [style (compiler)]: /checkout/compiler/rustc_lint/src/builtin.rs:3295: TODO is used for tasks that should be done before merging a PR;
If you want to leave a message in the codebase use FIXME
 >> TODO: Remove this before merging. This is for crater only.
tidy [style (compiler)]: FAIL
removing old virtual environment
creating virtual environment at '/checkout/obj/build/venv' using 'python3.12' and 'venv'
Requirement already satisfied: pip in ./build/venv/lib/python3.12/site-packages (24.0)
Collecting pip
  Downloading pip-26.2.1-py3-none-any.whl.metadata (4.6 kB)
---
Running eslint on rustdoc JS files
info: ES-Check: checking 7 files...
info: ✓ ES-Check passed! All files are ES10 compatible.
typechecking javascript files
tidy: The following check failed: style (compiler)
Command `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools-bin/rust-tidy --root-path=/checkout --cargo-path=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo --output-dir=/checkout/obj/build --concurrency=4 --npm-path=/node/bin/yarn --ci=true --extra-checks=py,cpp,js,spellcheck` failed with exit code 1
Created at: src/bootstrap/src/core/build_steps/tool.rs:1627:23
Executed at: src/bootstrap/src/core/build_steps/test.rs:1747:29

Command has failed. Rerun with -v to see more details.
Bootstrap failed while executing `test src/tools/tidy tidyselftest --extra-checks=py,cpp,js,spellcheck`
Currently active steps:
test::Tidy {  } at src/bootstrap/src/core/build_steps/test.rs:1665
Build completed unsuccessfully in 0:02:44
  local time: Wed Sep  9 11:26:30 UTC 2026

[theemathas]

Based on checking locally, it does seem that passing RUSTFLAGS=--cap-lints=forbid in fact only affects the current crate, and not its dependencies. (At the very least, it doesn't affect crates.io dependencies.) So, the currently-queued crater run won't work.

I've pushed a new commit for the crater run. It emits a hard error only if --cfg crater_hack is set. I've tested this by compiling a crate that depends on perf-event-open-sys@=3.0.0, using the command RUSTFLAGS='--cfg crater_hack --force-warn invalid-c-variadic-arguments' cargo +stage1 check, and I did in fact get the hard error. Let's do a crater run using this.

@craterbot abort

@bors try

[craterbot]

🗑️ Experiment pr-162478 deleted!

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[rust-bors]

☀️ Try build successful (CI)
Build commit: 40527f3 (40527f3e444fde1deb361751bdef9b9e11d1b971)
Base parent: eca445e (eca445e5ae4a6679cc27d3a09106ce245e13a5a6)

[RalfJung]

I have removed the t-lang nomination -- I suspect having an idea of just how bad this crater run will be, and which types we could allow to make it less bad, will be very useful for the discussion.

[theemathas]

@craterbot run start=eca445e5ae4a6679cc27d3a09106ce245e13a5a6 end=40527f3e444fde1deb361751bdef9b9e11d1b971+rustflags="--cfg crater_hack --force-warn invalid-c-variadic-arguments" mode=check-only

[craterbot]

👌 Experiment pr-162478 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[Jules-Bertholet]

I'm against making this a FCW; I think it should just be a normal warn-by-default lint (perhaps deny for certain cases). There are perfectly legitimate reasons to pass non-VaArgSafe types as variadic arguments (e.g. repr(C)structs). I think it's good to be conservative when adding a new feature, so I'm fine with VaArgSafe being as restrictive as it currently is for the situations where it's currently required. But a breaking change should require much stronger justification.

[theemathas]

There are perfectly legitimate reasons to pass non-VaArgSafe types as generics (e.g. repr(C) structs).

@Jules-Bertholet Is there a legitimate reason to then proceed to pass a value of that type as a variadic argument?

[Jules-Bertholet]

If some weird C API you are linking against requires it, then that's what you have to do. (Sorry for typo, see edited message)

[theemathas]

@Jules-Bertholet I believe that passing a repr(C) struct (or a struct defined in C) as a variadic argument is UB both in C and in rust. Or if it's not UB when you pass it, it will be UB when you try to read it.

[Jules-Bertholet]

As far as I am aware, no such UB exists in the C standard (1, 2).

[beetrees]

I believe that passing a repr(C) struct (or a struct defined in C) as a variadic argument is UB both in C and in rust. Or if it's not UB when you pass it, it will be UB when you try to read it.

In C, all types that can be passed as function arguments (except for types like short and float that get promoted) can be passed as varargs.

There are perfectly legitimate reasons to pass non-VaArgSafe types as variadic arguments (e.g. repr(C)structs).

I think the solution there would be to allow such types to implement VaArgSafe (of course, this requires a bunch of design work). Hopefully the crater run should help determine whether this needs to occur before adding this FCW - the FCW could also temporarily ignore all #[repr(C)] structs etc. while that design work occurs.