CLI: (some?) test failures not reported

[ithinkihaveacat]

The tap-spec CLI fails to report some errors:

$ cat foo # from https://testanything.org/tap-specification.html
1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Summarized correctly # TODO Not written yet

$ node_modules/.bin/tap-spec < foo
    ✔ Input file opened
    ✔ Read the rest of the file

(I would expect the output to mention the "not ok" tests.)

Including a comment at the start of the TAP input seems to "fix" things:

$ cat bar
# qqqqqq
1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Summarized correctly # TODO Not written yet

$ node_modules/.bin/tap-spec < bar

  qqqqqq

    ✔ Input file opened
    ✔ Read the rest of the file



  Failed Tests: There were 2 failures
[...]

Version: 5.0.0

[scottcorgan]

I think this might be related to scottcorgan/tap-dot#14

[ithinkihaveacat]

It turns out that the # qqqqqq workaround breaks exit status. So, either you get output on test failure, or exit status, but not both:

$ ( echo -e "1..1\nnot ok 1 - aaa" | tap-spec ) || echo FAILED
# outputs "FAILED", but no test output
$ ( echo -e "# dummy\n1..1\nnot ok 1 - aaa" | tap-spec ) || echo FAILED
# no FAILED, but test output

[JumpLink]

Are there efforts to fix that? I Use Github Actions with tab-spec and my tests are not marked as failed due to the bug. Maybe this can be fixed by upgrade tap-out.