Stack pointer overflow detection #1580
Description
Apologies if this has already been addressed in some way that I've not found yet, but so far everything I'm seeing seems to suggest using stack protectors and instrumentation that are pretty heavy changes for something that machine ABIs normally just get "for free."
Normally, on OS with memory protection, functions that allocate more than a page worth of stack need to poke guard pages at page-sized offsets so that if they're using too much stack, they'll hit an end-of-stack page and page fault. Frames with less than a page worth of locals don't really have to do that because the calling conventions always have to store some kind of information in the vicinity of the stack pointer to return from the call, so there is always an SP-adjacent memory access when calling a function. (This is probably glossing over details.)
WebAssembly doesn't store return addresses or stack pointers in addressable memory though, which means that even with page protection, there is no guarantee of any memory accesses happening near the SP at all. It also manipulates the stack pointer with arithmetic ops and loads/stores rather than dedicated stack-adjust ops and/or stack size annotations on functions, which means runtime-level instrumentation has limited options for detecting SP overflows.
I'm not sure if it's even required that __stack_pointer is updated when the stack is pushed? Is it valid to access negative offsets from the SP in leaf functions? But even if that is assumed, there is another problem: Dynamic alloca, especially inadvertently calling it with a negative size, which just arithmetic overflows and offsets the SP in the wrong direction into valid memory.
Not sure how to best address these problems, especially with how they interact with the threads and memory control proposals, but I think at minimum it would make sense to have dedicated ops for stack growth, including detecting that the SP is "growing" in the intended direction and not wrapping around the end of the memory space. That would allow for both page-based and range-based runtime-level stack pointer growth validation strategies.
That wouldn't really even require specifying a particular way of storing a stack pointer, just a way of validating that the range of memory addresses that a stack pointer value is being expanding over are valid, and it'd be fairly easy to be backward-compatible by just polyfilling it to add/sub instructions.