Skip to content

Dont internalize extern definitions and add proper section info - #317

Merged
tamird merged 1 commit into
aya-rs:mainfrom
altugbozkurt07:ksyms
Sep 17, 2026
Merged

tamird merged 1 commit into
aya-rs:mainfrom
altugbozkurt07:ksyms

Conversation

@altugbozkurt07

@altugbozkurt07 altugbozkurt07 commented Oct 29, 2025

Copy link
Copy Markdown
Contributor

As per our discussion with @alessandrod , i added a small fix to make sure extern declarations preserve linkage/calling-convention and add the proper section info to hint the llvm backend for btf datasec generation for ksyms.


This change is Reviewable

@vadorovsky vadorovsky left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work and thanks for adding the compiler test! Just couple of nits and after that I think it's good to go. 🙂

@vadorovsky reviewed all commit messages.
Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions


-- commits line 2 at r1:
nit: s/Dont/Don't/


src/llvm/mod.rs line 270 at r1 (raw file):

            if num_blocks == 0 {
                unsafe { LLVMSetSection(value, c".ksyms".as_ptr()) };
                info!(

In DISanitizer (https://github.com/aya-rs/bpf-linker/blob/main/src/llvm/di.rs) we seem to use mostly trace! log level for fixups, let's use it here too instead of info!.


src/llvm/mod.rs line 277 at r1 (raw file):

            }
        }
        if unsafe { !LLVMIsAGlobalVariable(value).is_null() } {

As clippy suggests, you can merge these two if stamements into one.

@alessandrod

Copy link
Copy Markdown
Collaborator

let's not merge this until we have kfuncs working e2e in aya since this might not be enough

@domwst

domwst commented Aug 1, 2026

Copy link
Copy Markdown

Hi! I've recently tried to use kfuncs from a Rust ebpf program via C shims and this PR did come up very handy as bpf-linker did indeed internalize the __ksym extern declarations and threw away its signature, which did cause libbpf fail on BTF resolution. With these changes I got everything working. So, this PR does already have value in it even without e2e support of kfuncs in aya (as far as I understand it's at least blocked by the Rust compiler). Just wanted to throw it out here.

For reference, here is relevant bit from linker dump:

Before

declare !dbg !7273 internal fastcc i32 @bpf_io_uring_submit_sqes() unnamed_addr #11 section ".ksyms"

After

declare !dbg !7290 i32 @bpf_io_uring_submit_sqes(ptr noundef, i32 noundef) local_unnamed_addr #11 section ".ksyms"

@TheEnthralled

Copy link
Copy Markdown

Hey y'all, hope y'all are doing well.

I was wondering what the status of this change is. The last CI run is from February and its logs have expired. The full test suite passes locally with this commit cherry-picked onto current main (with llvm-23), including the new extern_linkage.rs case. Other than the nits, additional test cases, and failed CI builds. what is blocking it from being merged?

As @domwst mentioned, this change does fix the issues concerning calling conventions as arguments to functions declared extern "C" are now preserved.

I compiled the following code against Rust nightly, kernel v6.18, and linked it with bpf-linker excluding/including the change in this PR.

// Case 1: a kfunc call with an argument. Shows whether the linker keeps the
// declaration's parameters, and therefore the call-site argument setup.
#![no_std]
#![no_main]

#[repr(C)]
pub struct cgroup {
    _opaque: [u8; 0],
}

unsafe extern "C" {
    // Real kfunc: struct cgroup *bpf_cgroup_from_id(u64 cgid);
    fn bpf_cgroup_from_id(cgid: u64) -> *mut cgroup;
}

#[unsafe(no_mangle)]
#[unsafe(link_section = "classifier")]
pub extern "C" fn probe(_ctx: *mut core::ffi::c_void) -> i32 {
    unsafe { bpf_cgroup_from_id(7).is_null() as i32 }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    loop {}
}

#[unsafe(link_section = "license")]
#[unsafe(no_mangle)]
static LICENSE: [u8; 4] = *b"GPL\0";

I generated the LLVM-IR by running

RUSTFLAGS="-Cdebuginfo=2 -Clink-arg=--btf -Clink-arg=--emit=llvm-ir" \
  cargo +nightly build --release --target bpfel-unknown-none -Z build-std=core

LLVM-IR diff:

11c11
<   %2 = tail call fastcc noundef ptr @bpf_cgroup_from_id() #2, !dbg !1139
---
>   %2 = tail call noundef ptr @bpf_cgroup_from_id(i64 noundef 7) #2, !dbg !1139
22c22
< declare internal fastcc noundef ptr @bpf_cgroup_from_id() unnamed_addr #0
---
> declare noundef ptr @bpf_cgroup_from_id(i64 noundef) unnamed_addr #0

The object dump diff also indicates that the BPF argument register r1 is now set before the function call.

<        0:	85 10 00 00 ff ff ff ff	call -0x1
<        1:	bf 01 00 00 00 00 00 00	r1 = r0
<        2:	b7 00 00 00 01 00 00 00	r0 = 0x1
<        3:	15 01 01 00 00 00 00 00	if r1 == 0x0 goto +0x1 <probe+0x28>
<        4:	b7 00 00 00 00 00 00 00	r0 = 0x0
<        5:	95 00 00 00 00 00 00 00	exit
---
>        0:	b7 01 00 00 07 00 00 00	r1 = 0x7
>        1:	85 10 00 00 ff ff ff ff	call -0x1
>        2:	bf 01 00 00 00 00 00 00	r1 = r0
>        3:	b7 00 00 00 01 00 00 00	r0 = 0x1
>        4:	15 01 01 00 00 00 00 00	if r1 == 0x0 goto +0x1 <probe+0x30>
>        5:	b7 00 00 00 00 00 00 00	r0 = 0x0
>        6:	95 00 00 00 00 00 00 00	exit

This change also ensures the linker acts correctly when we link in raw LLVM IR bitcode. Using a C shim shim.c

#define __ksym __attribute__((section(".ksyms")))
extern void bpf_rcu_read_lock(void) __ksym;
extern void bpf_rcu_read_unlock(void) __ksym;
int shim_rcu_lock(void)   { bpf_rcu_read_lock();   return 0; }
int shim_rcu_unlock(void) { bpf_rcu_read_unlock(); return 0; }

and the following Rust program

#![no_std]
#![no_main]

unsafe extern "C" {
    fn shim_rcu_lock() -> i32;
    fn shim_rcu_unlock() -> i32;
}

#[unsafe(no_mangle)]
#[unsafe(link_section = "classifier")]
pub extern "C" fn probe(_ctx: *mut core::ffi::c_void) -> i32 {
    unsafe { shim_rcu_lock() + shim_rcu_unlock() }
}

#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    loop {}
}

#[unsafe(link_section = "license")]
#[unsafe(no_mangle)]
static LICENSE: [u8; 4] = *b"GPL\0";

we get the following when compiling with RUSTFLAGS="-Cdebuginfo=2 -Clink-arg=--btf -Clink-arg=$PWD/shim.bc -Clink-arg=--emit=llvm-ir" \ cargo +nightly build --release --target bpfel-unknown-none -Z build-std=core

declare !dbg !2270 internal fastcc void @bpf_rcu_read_lock() unnamed_addr #2 section ".ksyms"
declare !dbg !2273 internal fastcc void @bpf_rcu_read_unlock() unnamed_addr #2 section ".ksyms"

and no output when running RUSTFLAGS="-Cdebuginfo=2 -Clink-arg=--btf -Clink-arg=$PWD/shim.bc" \ cargo +nightly build --release --target bpfel-unknown-none -Z build-std=core bpftool btf dump file target/bpfel-unknown-none/release/probe-shim | grep -E "FUNC 'bpf_rcu|DATASEC '\.ksyms"

With the fix we see

declare !dbg !2270 dso_local void @bpf_rcu_read_lock() local_unnamed_addr #2 section ".ksyms"
declare !dbg !2273 dso_local void @bpf_rcu_read_unlock() local_unnamed_addr #2 section ".ksyms"

and

[7] FUNC 'bpf_rcu_read_lock' type_id=6 linkage=extern
[9] FUNC 'bpf_rcu_read_unlock' type_id=8 linkage=extern
[27] DATASEC '.ksyms' size=0 vlen=2
	type_id=7 offset=0 size=0 (FUNC 'bpf_rcu_read_lock')
	type_id=9 offset=0 size=0 (FUNC 'bpf_rcu_read_unlock')

respectively.

This change does not get us full E2E support for kfunc calling through Rust+aya as rustc still doesn't emit the requisite debug info for foreign functions. However, ksyms support was recently merged into Aya (aya-rs/aya#1372) and there are open PRs that improve on kfunc support (aya-rs/aya#1495).

However, these changes still depend on this bpf-linker change as even if the proper logic to resolve kfunc calls is added to aya (the loader) exclusively, loading the program would still fail as the linker has failed to provide the correct signatures and function preludes for the kfuncs that require a non-zero number of arguments.

While these changes are being worked on, this change would still unblock kfunc calling through Rust+aya if C shims are also utilized.

If any help is needed with getting this change ready to be merged I am more than willing to assist.

@tamird

tamird commented Sep 17, 2026

Copy link
Copy Markdown
Member

@codex review

@tamird
tamird requested review from vadorovsky and a balanced review from Copilot September 17, 2026 02:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The stated calling-convention preservation behavior lacks a regression case using an explicit calling convention.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Preserves external declarations during LLVM optimization and debug-info sanitization, enabling correct BTF .ksyms generation.

Changes:

  • Skip internalization and debug-linkage rewriting for declarations.
  • Add linkage and BTF regression tests.
  • Add Bazel fixtures for external C bitcode.
File summaries
File Description
src/llvm/mod.rs Preserves declaration linkage.
src/llvm/di.rs Preserves declaration debug linkage.
tests/extern_linkage.rs Tests external and weak linkage.
tests/c/extern.c Provides a .ksyms fixture.
tests/btf/assembly/extern.rs Verifies external-function BTF output.
tests/BUILD.bazel Registers the new tests and fixture.
Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/extern_linkage.rs Outdated
@tamird
tamird force-pushed the ksyms branch 2 times, most recently from 5e63764 to d7a04ca Compare September 17, 2026 03:37
Internalizing unresolved functions lets LLVM treat them as local
definitions and drop arguments from calls. Leave function and global
declarations unchanged, including their visibility and weak linkage.

Exclude declarations from debug-info linkage rewriting as well. Marking
a C kfunc declaration as a local definition removes its BTF FUNC record
and .ksyms entry, preventing the loader from resolving it.

Use one compiler fixture with optimized-IR and BTF revisions to check
strong and weak declarations and external-function BTF metadata. Detect
ELF output before dumping BTF so both revisions can share the source.

Supply declaration debug info through fixed LLVM IR: Clang 18 omits it
at -O0, while newer Clang bitcode can be unreadable by older supported
LLVM.

Correct the existing export test's conditional directives so its cdylib
and bin revisions select their intended crate types. Share their
identical assertions using the common CHECK prefix.

Co-authored-by: altug bozkurt <altug.bozkurt09@gmail.com>

@vadorovsky vadorovsky left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One (old) comment, after that let's ship it.

@vadorovsky partially reviewed 7 files, made 2 comments, and resolved 1 discussion.
Reviewable status: 4 of 8 files reviewed, 3 unresolved discussions (waiting on altugbozkurt07 and tamird).


src/llvm/mod.rs line 270 at r1 (raw file):

Previously, vadorovsky (Michal Rostecki) wrote…

In DISanitizer (https://github.com/aya-rs/bpf-linker/blob/main/src/llvm/di.rs) we seem to use mostly trace! log level for fixups, let's use it here too instead of info!.

@tamird This is still not addressed. 🙂 info! is too noisy for this log IMO.

@tamird
tamird requested review from tamird and vadorovsky September 17, 2026 10:10

@tamird tamird left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tamird made 1 comment.
Reviewable status: 4 of 8 files reviewed, 2 unresolved discussions (waiting on altugbozkurt07, tamird, and vadorovsky).


src/llvm/mod.rs line 270 at r1 (raw file):

Previously, vadorovsky (Michal Rostecki) wrote…

@tamird This is still not addressed. 🙂 info! is too noisy for this log IMO.

there's no info here...?

@tamird
tamird merged commit 3b3ae92 into aya-rs:main Sep 17, 2026
27 of 28 checks passed
@tamird
tamird deployed to nightly-promotion September 17, 2026 10:12 — with GitHub Actions Active
@vadorovsky

Copy link
Copy Markdown
Member

src/llvm/mod.rs line 270 at r1 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

there's no info here...?

OK, yeah, it's just reviewable hallucinating

info.png

@swananan

Copy link
Copy Markdown

I just realized that we can use Rust with C shims as a workaround for kfuncs or CO-RE. Could we document this, perhaps in the aya-book? I can open a PR to do that.

@tamird

tamird commented Sep 17, 2026

Copy link
Copy Markdown
Member

Sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants