Windows / cranelift: misaligned pointer dereference

[enizor]

frawk is built using --no-default-features (to prevent the known jemalloc issue), in debug mode with rust 1.70 (that landed the alignment check in debug),

C:\[...]\Documents\frawk>target\debug\frawk -B cranelift  "{print $1}" Cargo.toml
thread 'main' panicked at 'misaligned pointer dereference: address must be a multiple of 0x10 but is 0x419779fffe', src\codegen\intrinsics.rs:953:19
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library\std\src\panicking.rs:578
   1: core::panicking::panic_fmt
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library\core\src\panicking.rs:67
   2: core::panicking::panic_misaligned_pointer_dereference
             at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library\core\src\panicking.rs:174
   3: frawk::codegen::intrinsics::load_var_int
             at .\src\codegen\intrinsics.rs:953
   4: <unknown>

When running in release mode, UB happens and nothing is printed

C:\[...]\Documents\frawk>target\release\frawk -B cranelift  "{print $1}" Cargo.toml

C:\[...]\Documents\frawk>

Using the interpreter backend works as expected:

C:\[...]\Documents\frawk>target\debug\frawk -B interp  "{print $1}" Cargo.toml
[package]
name
version
authors
edition
[...]

I don't really know how to debug further, I'd love to help if you got some directions for me.

[ezrosent]

That's a useful check!

I just re-ran the frawk test suite on 1.72 (nightly) and it passed on macos. A couple guesses:

• Maybe windows has types with a higher alignment requirement (I know allocators generally align to 0x10 on windows; not surprised its assumed somewhere), and frawk is assuming all pointers are aligned to 8 bytes by mistake. I did a quick scan for obvious idioms along these lines and didn't find anything (plenty of code assuming pointers are aligned to 8 bytes, but that's not being violated here). If possible, could you take a look at the alignment of the Runtime type in src\codegen\intrinsics.rs? If it's 16, then that's definitely a clue that the runtime is being passed incorrectly here.

• One area that could be an issue here is the ABI we use. Some parts of the cranelift backend hard-code the SystemV ABI which is almost certainly the wrong thing on windows. That could cause some stack slots to be misaligned/wrong when we call back into a Rust function. I can try and do some digging.

I believe I can get a windows repro soon-ish, it'll just take a little time.

[enizor]

• I checked (using static_assertions) that the Runtime type actually has an alignment of 16
• I tried your last commit and it changed the panic:

C:\[...]\Documents\frawk>target\debug\frawk -B cranelift  "{print $1}" Cargo.toml
thread 'main' panicked at 'assertion failed: !map.is_null()', src\codegen\intrinsics.rs:1398:1

[ezrosent]

Something odd is definitely going on here. I was able to repro !map.is_null() after b356074, but 3017ce1 doesn't fix it, and I still get the "misaligned pointer dereference" failure on the add_zero_prints_float unit test. The failure is a different function (line 933) but the same sort of pattern: the Runtime argument appears misaligned.

However the error indicates that the underlying Runtime isn't even aligned to 8 bytes, which suggests that hand-rolled pointer arithmetic assuming 8-byte alignment everywhere isn't the issue.

I'll have to think about this more... I have very little Windows debugging experience, and I may be missing something obvious here. I can try asking around though.

[ezrosent]

Just a quick update: I believe I am misaligning stack slots in cranelift. I need to add more padding there. It's definitely a bug; not sure if it's the bug yet.

[ghuls]

I got an experimental version of frawk (with cranelift backend) working on Windows (or at least under WINE in linux).

The main issue is that functions that return U128 cause problems.

• Cranelift expects SystemV ABI, but on Windows extern "C" functions use the Windows FastCall convention (which is different for functions that that return i128/u128 (Str related functions)):
  https://github.com/bytecodealliance/wasmtime/blob/922ba7402744d7a91148eb0e004eccec4fdd6456/cranelift/fuzzgen/src/lib.rs#L241-L245
• Changing extern "C" functions that return U128 (Str) to extern "sysv64", seems to fix it.

Here are some timings:

• frawk on linux:
  • with jemalloc
  • with mimalloc (new)
• fraw on Windows (linux with WINE):
  • with default memory allocator
  • with mimalloc (as jemalloc is not supported on Windows)

# frawk on linux (with jemalloc) with interpreter backend.
$ time target/release/frawk_jemalloc -B interp 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m2.957s
user    0m2.826s
sys     0m0.243s

# frawk on linux (with jemalloc) with cranelift backend.
$ time target/release/frawk_jemalloc -B cranelift 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m1.953s
user    0m1.832s
sys     0m0.201s


# frawk on linux (with mimalloc) with interpreter backend.
$ time target/release/frawk_mimalloc -B interp 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m2.739s
user    0m2.652s
sys     0m0.183s

# frawk on linux (with mimalloc) with cranelift backend.
$ time target/release/frawk_mimalloc -B cranelift 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m1.950s
user    0m1.804s
sys     0m0.228s


# Windows executable of frawk run on linux with WINE (with standard allocator) with interpreter backend.
$ time wine target/x86_64-pc-windows-gnu/release/frawk_standard_alloc.exe -B interp 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m13.652s
user    0m13.659s
sys     0m0.257s

# Windows executable of frawk run on linux with WINE (with standard allocator) with cranelift backend.
$ time wine target/x86_64-pc-windows-gnu/release/frawk_standard_alloc.exe -B cranelift 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m12.256s
user    0m12.244s
sys     0m0.276s


# Windows executable of frawk run on linux with WINE (with mimalloc) with interpreter backend.
$ time wine target/x86_64-pc-windows-gnu/release/frawk_mimalloc.exe -B interp 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m3.528s
user    0m3.508s
sys     0m0.223s


# Windows executable of frawk run on linux with WINE (with mimalloc) with cranelift backend.
$ time wine target/x86_64-pc-windows-gnu/release/frawk_mimalloc.exe -B cranelift 'BEGIN { for (i = 0; i < 5000000; i++) { if (i == 1) { continue; } for (j = -1; j < 5;) { j += 1; if (j == 2) { continue; } print i,j; } } }' > /tmp/a

real    0m2.585s
user    0m2.567s
sys     0m0.204s

The two compiled Windows binaries (standard allocator and with mimalloc) are attached, if you want to test:
frawk_windows.zip