Skip to content

Hold the release route to its three version numbers agreeing - #370

Merged
iderex merged 1 commit into
masterfrom
ci/the-route-refuses-a-release-whose-versions-disagree
Sep 6, 2026
Merged

iderex merged 1 commit into
masterfrom
ci/the-route-refuses-a-release-whose-versions-disagree

Conversation

@iderex

@iderex iderex commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Refs #88, the fifth of its five conditions: that the version in the manifest, the assembly and the changelog agree, asserted by a check rather than by reading.

Finishes: #88

What the condition was waiting on

The decision of 2026-09-05 on #88 names the published release, read by its tag, as the changelog side of the agreement, keeps docs/changelog.md as prose, and rules out the changelog: field of build.yaml because a reader taking a version out of that sentence would refuse a correct manifest. It asks that the check assert the three numbers at release time, before anything is published, and refuse the release otherwise.

The route already does both halves of that, and it has since before the question was asked:

git show origin/master:.github/workflows/publish.yaml | grep -n 'Tag ${tag} carries version\|but the assembly is stamped'
183:            echo "::error::Tag ${tag} carries version ${numeric} but build.yaml declares ${version}. Bump build.yaml, or tag the version that is in it."
333:            echo "::error::build.yaml declares version ${MANIFEST_VERSION} but the assembly is stamped ${assembly}. Derive the assembly version from build.yaml so the two cannot drift, or fix whichever one is wrong."

What was missing is the other rule this board keeps: nothing held either refusal in place. Both are shell inside a workflow file, so a compile says nothing about them and neither did the suite. Deleting either step left every check here green and published a release whose catalogue entry, archive and assembly disagreed about which version an operator installed - which is the same shape as the failure #88's body is written against, one artefact further in.

git show origin/master:Jellyfin.Plugin.MetadataSync.Tests/ReleaseRouteTests.cs | grep -c 'AssemblyVersion\|Check the tag and build.yaml agree'
0

What this adds

ReleaseVersionAgreementTests reads publish.yaml and asks, of each of the two steps, whether it carries a refusal: an if testing an inequality between two expansions with a non-zero exit inside the branch it guards. It identifies the two values compared by the text of the assignments they come from, so the gate's pair has to be the manifest read and the tag, and the build's pair the gate's approved number and the project's AssemblyVersion.

Then the join, which is the part that makes three numbers agree rather than two pairs: the number the build holds the assembly to is ${{ needs.gate.outputs.version }}, the gate's version output is produced by the step that carried the tag refusal, and that step hands forward one of the two values it compared.

ReleaseRouteTests gains the reachability half. The gate declares no dependency of its own, so a tag starts it whatever waits on it; a gate that no path to the release passes through refuses a version disagreement into a log while the same run publishes. The new leg walks the release's transitive needs and asks whether the gate is in it.

Proof that it bites

Five near-misses live in the suite as fixtures, each one thing away from a positive control that is read as a refusal: an error annotation with no exit beside it, an exit 0, an exit commented out, the comparison written as equality, and a step holding the assembly to a literal of its own. The first is the one this is mostly written for - a workflow command printing an error leaves the step's exit status at zero, so the job succeeds and the disagreement reaches the catalogue with a note about itself in a log, and in a diff it reads as the same check.

Fixtures prove the reading reports an absence. They do not prove it reports the absence in this route, so the same five were also done to publish.yaml itself, one at a time, with the file restored between each:

VSTEST_CONNECTION_TIMEOUT=600 DOTNET_CLI_UI_LANGUAGE=en dotnet test ... -f net9.0 --no-build \
  --filter "FullyQualifiedName~ReleaseRouteTests|FullyQualifiedName~ReleaseVersionAgreementTests"

the gate's exit 1 deleted, the annotation kept   -> 2 red: TheTagIsRefusedWhenTheManifestDeclaresAnotherVersion,
                                                          TheNumberTheAssemblyIsHeldToIsTheNumberTheTagWasHeldTo
the assembly-version step deleted                -> 3 red: TheReadingReachesBothStepsOfTheRoute,
                                                          TheAssemblyIsRefusedWhenItIsStampedWithAnotherVersion,
                                                          TheNumberTheAssemblyIsHeldToIsTheNumberTheTagWasHeldTo
MANIFEST_VERSION replaced by a literal           -> 1 red: TheNumberTheAssemblyIsHeldToIsTheNumberTheTagWasHeldTo
the gate's != written as =                       -> 2 red: TheTagIsRefusedWhenTheManifestDeclaresAnotherVersion,
                                                          TheNumberTheAssemblyIsHeldToIsTheNumberTheTagWasHeldTo
both `needs: gate` lines deleted                 -> 1 red: NothingIsPublishedWithoutTheGate
restored                                         -> Failed: 0, Passed: 18

The suite

dotnet build Jellyfin.Plugin.MetadataSync.sln --nologo
VSTEST_CONNECTION_TIMEOUT=600 DOTNET_CLI_UI_LANGUAGE=en dotnet test Jellyfin.Plugin.MetadataSync.Tests/Jellyfin.Plugin.MetadataSync.Tests.csproj --nologo --no-build -f net9.0
Failed: 0, Passed: 1708, Skipped: 0, Total: 1708
... -f net10.0
Failed: 0, Passed: 1708, Skipped: 0, Total: 1708

The means

C#, in the existing xunit suite, reading the workflow file the test project already copies beside its binary. It is the means ReleaseRouteTests uses for the same subject, it adds no language, runtime or dependency, and the three rules survive it: the property is refusable, the fixtures prove the refusal bites, and every number above carries the command that produced it. The alternative - a leg on a runner beside channel-freshness.yml - was considered and is the wrong shape here: what is judged is a file in this tree, not a body over the network, and a check that only runs on a schedule would report the deletion of a release guard after the release rather than before the merge.

What this does not say

Nothing about a run. This reads a workflow file, so what it judges is the shape of a refusal and never whether a runner executed one; no tag has been pushed against this branch and the publish route has not run. It also identifies the two values compared by the text of the assignments behind them, which is a weaker reading than the shape: a step fetching the manifest version by a route mentioning none of those words would be read as comparing something else, and the repair is to name the new route in the test rather than to loosen the reading. Both bounds are written at the class as well as here.

The shipped bytes are still not read at release time. The build compares the AssemblyVersion MSBuild property before it compiles, and nothing opens the archive afterwards to ask what the DLL inside it carries. That gap is narrow because Directory.Build.props derives the version from build.yaml rather than restating it, so the two cannot drift silently, and the suite's StampedVersionIsTheVersionTheManifestDeclares reads a built assembly. It is a gap all the same and it is not what this change is about.

This change is 903 added lines against the advisory 400. It is one property over one route - the reader, its five near-misses and their positive control - and dividing it would produce two halves that only make sense together, which is the thing the cap exists to prevent rather than to cause. No source file outside the test project is touched.

No second reader looked at this. The runs and readings above are the evidence in place of one.

The route already refuses each half of the agreement the fifth condition of #88
asks for: the gate refuses a tag whose numeric part is not build.yaml's
`version`, and the build refuses an assembly stamped with a number the gate did
not approve. Nothing in this tree held either of them in place, so deleting
either step left the suite green and published a release whose catalogue entry,
archive and assembly disagreed about which version an operator installed.

ReleaseVersionAgreementTests reads the route for the shape of each refusal - an
inequality between two values with a non-zero exit inside it - identifies the
two values by the assignments they come from, and asserts the join between the
two steps: the number the build holds the assembly to is the gate's output, and
that output is the number the gate compared against the tag. Two pairs are not
a three-way agreement unless the value passed between them is the value that
was compared.

The near-miss it is mostly written for is an error annotation with no exit
beside it. A workflow command printing an error leaves the step's exit status
at zero, so the job succeeds, the disagreement reaches the catalogue with a
note about itself in a log, and in a diff it reads as the same check.

ReleaseRouteTests gains the reachability half. The gate declares no dependency
of its own, so a tag starts it whatever waits on it, and a gate that no path to
the release passes through refuses a version disagreement into a log while the
same run publishes.

Signed-off-by: Nils Lehnen <30603423+iderex@users.noreply.github.com>
@iderex iderex added enhancement New feature or request ci labels Sep 6, 2026
@iderex iderex self-assigned this Sep 6, 2026
Comment thread Jellyfin.Plugin.MetadataSync.Tests/ReleaseVersionAgreementTests.cs Dismissed
Comment on lines +616 to +653
foreach (var raw in workflow.Split('\n'))
{
var line = raw.TrimEnd('\r').TrimEnd();
var indent = line.Length - line.TrimStart(' ').Length;

if (line.Length == 0)
{
if (inJob)
{
lines.Add(string.Empty);
}

continue;
}

if (indent == 0)
{
inJobs = string.Equals(line, "jobs:", StringComparison.Ordinal);
inJob = false;
continue;
}

if (!inJobs)
{
continue;
}

if (indent == 2)
{
inJob = string.Equals(line.TrimStart(' '), job + ":", StringComparison.Ordinal);
continue;
}

if (inJob)
{
lines.Add(line);
}
}
Comment on lines +686 to +702
foreach (var raw in run.Split('\n'))
{
var line = raw.TrimEnd('\r').Trim();
if (line.Length == 0 || line.StartsWith('#'))
{
continue;
}

if (line.EndsWith('\\'))
{
pending += line[..^1].TrimEnd() + " ";
continue;
}

joined.Add(pending + line);
pending = string.Empty;
}

if (line.EndsWith('\\'))
{
pending += line[..^1].TrimEnd() + " ";
@iderex
iderex merged commit 1c877cc into master Sep 6, 2026
14 of 16 checks passed
@iderex
iderex deleted the ci/the-route-refuses-a-release-whose-versions-disagree branch September 6, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants