release-rehearsal.test.sh reads its pass condition out of a fixed /tmp path, so a run that could not write reports as a refusal

[vladimirrott]

tests/release/release-rehearsal.test.sh proves that the rehearsal script
refuses a publishing mode. It writes the refusal to a fixed path under /tmp
and then greps that path for the pass condition:

$ git rev-parse --short=8 HEAD
61b3a878
$ sed -n '38,42p' tests/release/release-rehearsal.test.sh
if "$rehearsal" --publish >/tmp/sysknife-rehearsal-publish.out 2>&1; then
    printf 'FAIL: rehearsal accepted a publishing mode\n' >&2
    exit 1
fi
grep -Fq 'never publishes' /tmp/sysknife-rehearsal-publish.out

When the redirect fails, bash never runs the command, and the failure is
indistinguishable from the refusal this test is written to observe. The if
takes the false branch, which is the passing branch, and line 42 then reads a
file this run did not write.

Why it matters

Measured, replaying the shape at :38-42 against a file the process cannot
write:

$ f=$(mktemp /tmp/sk-demo-XXXX.out); printf 'rehearsal never publishes\n' > "$f"; chmod 444 "$f"; if echo 'this run really ran' > "$f" 2>&1; then echo "if-branch: TRUE (test would FAIL loudly)"; else echo "if-branch: FALSE (test treats this as the expected refusal)"; fi; grep -Fq 'never publishes' "$f" && echo "line 42 assertion: SATISFIED"; cat "$f"; rm -f "$f"
/bin/bash: line 1: /tmp/sk-demo-wtD2.out: Permission denied
if-branch: FALSE (test treats this as the expected refusal)
line 42 assertion: SATISFIED
rehearsal never publishes

scripts/release_rehearsal.sh was never invoked, and the contract test reported
that it correctly refuses to publish. That is a guard answering a question it
could not ask, which is the shape this repo files issues about in its own CI.

Two ways to arrive there, neither exotic. Another account on a shared runner or
a shared developer box owns /tmp/sysknife-rehearsal-publish.out, because the
name is fixed and predictable. Or /tmp is full or read-only, in which case
every gate that writes there fails in a way that reports as a pass here and as
an error elsewhere.

The file is also left behind. The test has no trap:

$ grep -c trap tests/release/release-rehearsal.test.sh
0

This is the only line in tests/release/ that reads a fixed /tmp path back as
a pass condition:

$ grep -rn 'grep .*/tmp/' tests/release/
tests/release/release-rehearsal.test.sh:42:grep -Fq 'never publishes' /tmp/sysknife-rehearsal-publish.out

Six of the twenty-two scripts in that directory already use mktemp, so the
pattern to copy is in the same directory.

Scope

• Give the test a private temporary directory (mktemp -d) and a
  trap 'rm -rf "$tmp"' EXIT, and put the publish output inside it.
• Separate "the command ran and refused" from "the command never ran". Capture
  the status explicitly rather than reading it off an if whose false branch
  also covers a redirect failure. Assert that the output file is non-empty
  before grepping it.
• Leave the assertion itself alone. never publishes is the right string and
  the refusal is the right behaviour; only the plumbing that observes it moves.

Do not fix this by adding || exit 1 to the redirect. That closes the one arrival
path you thought of and leaves the shape intact for the next one.

Tests first

Write the negative case before the fix, because the point of the change is that
a specific wrong state now fails.

1. Pin the fixed path: make the destination unwritable for the running user and
   run the test as it stands. It passes. Paste that.
2. Apply the fix. The same setup now fails, and the message names the write
   rather than the missing string.
3. Then prove the positive case still holds: an ordinary run passes, and the
   temporary directory is gone afterwards.

Step 1 is the one that matters. A version of this change that only adds mktemp
and a trap is tidier and proves nothing, because the run that could not write
still reads as a refusal.

Difficulty

easy. One file, about six lines, and the hard part is the test in step 1
rather than the fix. No Rust, no VM, no daemon, no credentials.

Getting started

CONTRIBUTING.md
has the build and test commands. No CLA and no copyright waiver. The project is MIT.

[vladimirrott]

@mikevillari this one is yours if you want it, and it replaces the offer I
withdrew on #420 a few minutes ago. That withdrawal was my filing error rather than
anything to do with you.

Why this one. #399 was a step that reported success over a set it never counted,
and the fix you wrote separated "the producer failed" from "the producer returned
nothing" so the two could not read the same. This is that distinction in a test
rather than a workflow: tests/release/release-rehearsal.test.sh:38 treats "the
rehearsal script refused to publish" and "bash could not open the output file"
as the same outcome, because both land in the if's false branch. The
demonstration is in the body and it runs in one line.

The part I would hold you to is step 1 under "Tests first". A change that only
swaps in mktemp -d and a trap is tidier and proves nothing, and the way to
show it proves something is to make the destination unwritable and watch the
current test pass. That negative case is the whole PR.

Say the word and I will label and assign it. No is a complete answer.

[vladimirrott]

@mikevillari withdrawing this offer, and the reason is a collision on my side rather than anything to do with you.

I offered it to you on 14 September at 16:28Z. @armutlutost opened #435 against it the following afternoon, and that PR is in review now, so there is nothing left here to pick up. You had not replied, which costs you nothing at all, but an offer left standing would have cost you an evening if you came back to it this week.

I have put a better-matched one in front of you on #442 instead.