possible lack of posix compliance

[tungstengmd]

https://www.diva-portal.org/smash/get/diva2:1985766/FULLTEXT01.pdf
i just saw this by way of the #ksh channel on libera.chat, thought this should maybe be looked at

[dnewhall]

Reading that paper, I do like that ksh was usually one of the highest performing shells (granted, arguably because we cheat in a couple places).

I looked into each of the tests that failed and re-ran them manually. A bunch of the tests don't actually test POSIX behavior (and instead deal with extensions and implementation-defined behavior), a good number appear to be simply incorrect (or at least vaguely defined per POSIX), some others just plain worked, and some had unexpected-but-not-wrong output or error codes.

Someone should probably double-check my work. I have a writeup for every failing test ...but listing them all would take too much space here. Here's the tl/dr: of the 34 smoosh tests that the paper lists as failing, only 8 can be considered "real" failures of POSIX behavior (AFAICT):

• shell/builtin.times.ioerror
• shell/builtin.trap.exit3
• shell/builtin.trap.subshell.loud
• shell/builtin.trap.subshell.loud2
• shell/semantics.subshell.background.traps
• shell/semantics.traps.inherit
• shell/sh.monitor.fg
• shell/sh.ps1.override

(Tests are located here: https://github.com/mgree/smoosh/tree/master/tests/shell)

shell/builtin.times.ioerror:
Tests that an unhandled SIGPIPE errors out. I think the behavior it's testing is standard, but my SIGPIPE knowledge is rusty. On my Mac, bash had the same result as ksh, but on Linux it errors (forgot to grab version numbers, of course...). dash errors on both. Pretty sure this is related to our subshells "cheating".

shell/builtin.trap.exit3:
Tests that an exit in a subshell in a trap returns the correct value. bash and dash both print nothng and return 0. We print nothing but we return 3. Again, probably related to us "cheating" at subshells.

shell/builtin.trap.subshell.loud:
Tests that exit with no return code in a subshell in a trap handler works as it's supposed to. We don't print the correct message, but neither bash nor dash print the correct value either.

shell/builtin.trap.subshell.loud2:
Tests more exit, subshell, and return value behavior. We don't output the correct output at all and return 0 instead of 1.

shell/semantics.subshell.background.traps:
Tests the trap for SIGINT and SIGQUIT with background tasks. We pass the SIGINT part, but fail the SIGQUIT part of the test.

shell/semantics.traps.inherit:
Tests traps in subshells. ksh, bash, and dash all fail in different ways. We don't handle the SIGINT correctly, apparently.

shell/sh.monitor.fg:
Test should fail for allowed-but-undefined behavior, but instead the test hangs when it gets to jobs -l.

shell/sh.ps1.override:
Tests that the prompt is included in the output in -i mode. While not spelled-out directly, the POSIX spec. seems to imply that it should. bash and dash both pass this test.