feat(mkdocs-deploy): add notebook exclude globs and continue-on-error#11
Merged
Conversation
The notebook execution step walked notebooks-path recursively and executed every uncached notebook with check=True, so the first notebook that could not run in a clean CI runner (live network/cloud data, missing optional extra, intentionally flaky) aborted the whole docs deploy with no way to skip it. Add two optional, backward-compatible inputs: - notebooks-exclude: newline-/comma-separated glob patterns matched against each notebook's path relative to notebooks-path. Matches are skipped and rendered without fresh execution. A leading "**/" also matches at the root level. Mirrors pytest's --ignore-glob. - notebooks-continue-on-error: when true, a non-excluded notebook that fails to execute is downgraded to a warning and the remaining notebooks still run; the failing notebook is not cached so it retries next run. Both are wired into the "Prepare notebook outputs" step env and honoured by prep_notebooks.py, which logs what it skips. Default behaviour is unchanged when neither input is set. Add test jobs covering exclude globs (nested + root-level globstar) and continue-on-error (with a negative control), and document both inputs.
Document that exclude patterns use fnmatch semantics where "*" matches across "/" (so "dask/*.ipynb" is recursive and a bare "*.ipynb" matches the whole tree), and that a leading "**/" matches at any depth including the root. This prevents accidental over-exclusion from shell-glob intuition.
Use fnmatchcase instead of fnmatch so notebook exclude patterns are matched case-sensitively and deterministically regardless of runner OS. Plain fnmatch applies os.path.normcase, which made matching case-insensitive on Windows runners while remaining case-sensitive on Linux.
The negative control previously asserted only that the strict run's outcome was "failure", so any unrelated failure (setup, fixture copy) would satisfy it. Assert instead that prep_notebooks.py started (cache root created) and aborted on the failing notebook before reaching the good one (good notebook neither executed nor cached), proving the failure was the notebook-execution abort.
The exclude parser splits on both newlines and commas, but CI only exercised the newline form. Add a third failing notebook excluded via a comma-separated pattern on a shared line, so both split paths are validated.
_matches no longer uses Path.match; describe the fnmatchcase-based matching and the leading "**/" zero-directory special case accurately.
actions/cache can restore a previously-cached hello.ipynb across runs, so asserting the good notebook is absent from .jupyter_cache was non-deterministic and failed in CI. Drop the cache-presence checks and rely on the deterministic signal: the strict run failed and left the good notebook un-executed in the working tree (it sorts after the failing notebook, so a strict walk aborts before reaching it).
Under notebooks-continue-on-error, emit a single top-level ::warning:: with the tolerated-failure count after the walk, so a swallowed failure is visible in the run's annotations instead of only as a plain line inside the collapsed "Preparing notebook outputs" group.
Add .github/pull_request_template.md and .github/ISSUE_TEMPLATE so new PRs and issues follow a consistent structure.
The exclude input had a YAML example but continue-on-error was only mentioned inline. Add a combined example showing both inputs together.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds two optional, backward-compatible inputs to the
mkdocs-deploycomposite action so a single notebook thatcannot run in a clean CI runner no longer aborts the whole docs deploy.
Previously the notebook-execution step walked
notebooks-pathrecursively and executed every uncached notebookwith
check=True, so the first notebook that failed (live network/cloud data, a missing optional extra, or anintentionally-flaky notebook) aborted the entire deploy with no way to skip it. This blocked repos like
pyramidsfrom adoptingnotebooks-path.notebooks-exclude— newline-/comma-separated glob patterns matched case-sensitively against eachnotebook's path relative to
notebooks-path. Matching notebooks are skipped (rendered without freshexecution). A leading
**/also matches at the root. Mirrors pytest's--ignore-glob.notebooks-continue-on-error— whentrue, a non-excluded notebook that fails to execute is downgraded toa
::warning::and the remaining notebooks still run; the failing notebook is not cached so it retries nextrun. Default
falsepreserves the previous fail-fast behaviour.Both are wired into the "Prepare notebook outputs" step env and honoured by
prep_notebooks.py. Defaultbehaviour is unchanged when neither input is set. No new runtime dependencies.
Issues
Type of change
Check relevant points.
How Has This Been Tested?
Two new jobs were added to
.github/workflows/test-mkdocs-deploy.yml, each routing pushes/deploys to the localfake remote (
setup-fake-remote) per the repo's test-isolation rule. To reproduce, push the branch and run theTest mkdocs-deployworkflow (uv package manager,groups: docs). The exclude/continue logic was also validatedlocally by simulating the walk with notebook execution monkeypatched.
root-level
**/globstar, and a comma-separated pattern; asserts the good notebook executed + cached while theexcluded ones were neither executed nor cached.
positive run (
notebooks-continue-on-error: 'true'→ deploy succeeds, good notebook executed + cached, failingone tolerated and not cached).
All pipelines pass on the head commit:
Test mkdocs-deploy,Test python-setup/pip,Test python-setup/pixi.Checklist:
(
mkdocs-deploy/vX), not apyproject.tomlversion.History.rst.change.