Serve CI fixtures from localhost so no gating step is live - #7
Conversation
Closes the exception Copilot found on #4. tests/good-links.html was scanned with fail-on-broken and no continue-on-error, so one gating step depended on github.com, python.org, jupyter.org and docs.python.org all staying reachable and redirect-free. A redirect appearing on any of them turns CI red with nothing wrong in the action. A genuine 200 over genuine HTTP was the reason that step had to stay: every other deterministic fixture either skips the request (ignore-patterns) or fails transport (.invalid), so none of them exercises a successful response. The workflow now starts a small server on 127.0.0.1 and asserts four paths against it end to end -- a clean page exits 0 under fail-on-broken, a 301 is followed and counted as one redirect, a 404 is reported, and the same 404 is silenced by silent-codes. That last pair is what the httpstat.us fixture was for, before the service stopped answering and took the coverage with it. The live scans stay, because real DNS, TLS and cross-host redirects are the one thing a local server cannot cover, but all three are now informational and none can gate. Verified by running the generated serve step under the composite shell and the real checker against the real fixtures: clean 0 broken / 0 redirects, moved 0 / 1, missing 1 / 0, and missing with silent-codes 404 back to 0 / 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🔗 Link Check Results🚨 1 broken link(s) and 3 redirect(s) were found. Build Details:
Link Check Summary
Broken Linkstests/broken-links.html - 1 broken link(s): tests/broken-links.html - 3 redirect(s): AI-Powered Suggestionstests/broken-links.html - AI Suggestions: Next Steps:
📝 This comment was automatically generated by the AI-Powered Link Checker Action. |
There was a problem hiding this comment.
Pull request overview
This PR makes the CI link-check assertions deterministic by serving HTTP fixtures from a local 127.0.0.1 server, removing the last gating dependency on third-party websites while keeping live scans as informational smoke checks.
Changes:
- Update CI to start a localhost HTTP server and assert clean/redirect/missing/silent-codes behavior end-to-end against it.
- Convert previously gating live scans (e.g.,
good-links.html) into informational steps viacontinue-on-error. - Update docs and changelog to reflect that gating coverage is now offline/deterministic.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
.github/workflows/ci.yml |
Adds a localhost fixture server plus new deterministic CI steps and assertions; makes live scans non-gating. |
tests/README.md |
Updates test-fixture guidance to reflect the new offline gating strategy and removes the prior caveat. |
CHANGELOG.md |
Notes the CI change to locally served fixtures and the remaining informational live scans. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The handler answered 200 to any path except /missing and /moved, so "A clean page passes end to end" only proved that the action requested something and got 200 -- not that it requested the URLs the fixture names. A typo in a generated fixture, or a regression that mangled the URL before the request, would have passed green. Routes are now exhaustive and anything unregistered is a 500 rather than a 404, so that silent-codes: '404' cannot suppress it. With a 404 default a typo in the missing fixture would still have been silenced and that step would still have passed. Verified by injecting a typo into each of the three fixtures in turn: clean goes from 0 broken to 1, redirect from 1 redirect to 0, and missing-with-silent-codes from 0 broken to 1. All three assertions go red where previously the first two would and the third would not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
🔗 Link Check Results🚨 1 broken link(s) and 3 redirect(s) were found. Build Details:
Link Check Summary
Broken Linkstests/broken-links.html - 1 broken link(s): tests/broken-links.html - 3 redirect(s): AI-Powered Suggestionstests/broken-links.html - AI Suggestions: Next Steps:
📝 This comment was automatically generated by the AI-Powered Link Checker Action. |
The last item from the test-fixture work, and the exception Copilot caught on #4.
The problem
tests/good-links.htmlwas scanned withfail-on-broken: 'true'and nocontinue-on-error, so it gated — and its four targets are real sites. A redirect appearing on any ofgithub.com,python.org,jupyter.orgordocs.python.org, or an outage while CI happened to run, turns the build red with nothing wrong in the action. #4 could only document that rather than remove it, sotests/README.mdhad to carry a caveat instead of a rule.That step had to stay because a genuine 200 over genuine HTTP was the one thing no deterministic fixture provided. Every other one either skips the request entirely (
ignore-patterns) or fails transport (.invalid), so none of them exercises what happens when a server actually answers.The fix
The workflow starts a small server on
127.0.0.1and asserts four paths against it end to end:/okfail-on-broken: 'true';broken-link-count == 0/moved→/okredirect-count == 1,broken-link-count == 0/missingbroken-link-count == 1/missingwithsilent-codes: '404'broken-link-count == 0, exits 0 underfail-on-broken: 'true'The last pair is what the
httpstat.usfixture was for, before the service stopped answering and took the coverage with it. It is now assertable without depending on anyone.Every gating step in
test-actionis now offline:good-links.htmlandbroken-links.htmlkeep their scans, because real DNS, TLS and cross-host redirects are the one thing even a local server cannot cover — but all three live steps now carrycontinue-on-errorand none can turn CI red.Verification
I ran the generated serve step under the real composite shell (
bash --noprofile --norc -e -o pipefail) and then the reallink_checker.pyagainst the real generated fixtures:--silent-codes 404→ 0 broken, 0 redirectsAll four match the assertions the workflow makes. I also confirmed the Python heredoc dedents correctly out of the YAML block scalar and compiles, and that the server comes up and is polled for readiness rather than raced against.
tests/README.mddrops the caveat and states the rule plainly. No change to the action itself —action.ymlandlink_checker.pyare untouched.