Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/actions/generate-docs/action.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
name: Generate documentation
description: Generate all documentation
inputs:
build:
description: Just subcommand for `build-docs` to run
default: all

runs:
using: composite
Expand All @@ -18,4 +22,4 @@ runs:
enable-cache: true
- name: Build documentation
shell: bash
run: just build-docs
run: just build-docs::${{ inputs.build }}
10 changes: 10 additions & 0 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-tags: true
Comment thread
alexdewar marked this conversation as resolved.
fetch-depth: 0
- name: Set user name and email for git
run: |
# Needed so we can apply patches
git config --global user.name "GitHub Actions"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
- uses: ./.github/actions/generate-docs
with:
build: all_with_old
- name: Setup pages
uses: actions/configure-pages@v5
- name: Upload artifact
Expand Down
1 change: 1 addition & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# For some reason these spurious links appear in some cargo doc-generated files
^file://.+/MUSE2/book/api/index\.html$
^file://.+/MUSE2/book/README\.html$

# Some of these links give a 404, even though they're generated in the API docs. No idea why.
^https://docs.rs/erased-serde/
Expand Down
19 changes: 17 additions & 2 deletions build-docs.just
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Make rustdoc warnings fatal
export RUSTDOCFLAGS := "-D warnings"

# Build all documentation
all: cli-help file-format examples book api
# Build all documentation, except old docs
all: cli-help file-format examples versions book api

# Build all documentation, including old docs
all_with_old: all old
Comment thread
alexdewar marked this conversation as resolved.

# Build book
book:
Expand Down Expand Up @@ -33,3 +36,15 @@ file-format *ARGS:
examples:
@echo Building docs for examples
@uv run docs/generate_example_docs.py

# Build TOC for old versions
versions:
@echo Building TOC for old versions of documentation
@uv run docs/generate_versions_docs.py
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The versions target invokes uv run docs/generate_versions_docs.py, which declares an unpinned jinja2 dependency in its # /// script dependencies header; uv will fetch and execute the latest jinja2 from PyPI at docs-build time, creating a supply-chain risk. If an attacker compromises the jinja2 package or the PyPI registry, they could run arbitrary code in the docs build environment (e.g. in CI with access to repository tokens or the deployed docs). To mitigate this, pin jinja2 (and any other script dependencies) to immutable versions via a lockfile or explicit version specifiers, or vendor these dependencies so the build does not perform on-the-fly installs from the public registry.

Copilot uses AI. Check for mistakes.

# Build documentation for previous releases
old:
@# Clean output dir
@rm -rf book/release

@uv run docs/build_old_docs.py
1 change: 1 addition & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Generated documentation files
command_line_help.md
examples.md
versions.md
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@
- [Release notes](release_notes/README.md)
- [MUSE2 v2.0.0 (October 14, 2025)](release_notes/v2.0.0.md)
- [Next unreleased version](release_notes/upcoming.md)
- [Other versions of documentation](versions.md)
86 changes: 86 additions & 0 deletions docs/build_old_docs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
#
# A script to generate documentation for previous releases of MUSE2.

import os
import shutil
import subprocess as sp
from pathlib import Path
from tempfile import TemporaryDirectory

from release import get_releases

DOCS_SITE_ROOT = "https://energysystemsmodellinglab.github.io/MUSE2"
REPO_ROOT = Path(__file__).parent.parent.absolute()
DOCS_DIR = REPO_ROOT / "docs"


def clone_repo_to(dest: Path):
"""Clone this repo somewhere else."""
print("Making a copy of repo")
sp.run(("git", "clone", REPO_ROOT, dest), check=True, capture_output=True)

# Add a symlink to cargo cache dir
try:
os.symlink(REPO_ROOT / "target", dest / "target")
except (NotImplementedError, OSError):
# Only newer versions of Windows support symlinks and these require the user to have
# additional privileges (or to be in developer mode)
print(
"WARN: Could not create symlink to cache directory; cache will not be stored"
)


def apply_patches_for_release(release: str, repo_path: Path) -> None:
"""Apply patches (if any) for the given release."""
patches_dir = DOCS_DIR / "release" / "patches" / release
for patch_path in sorted(patches_dir.glob("*.patch")):
sp.run(("git", "-C", str(repo_path), "am", str(patch_path)), check=True)

Comment on lines +34 to +39
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git am requires a committer identity; on machines/CI environments without user.name/user.email configured, applying patches will fail with “Please tell me who you are”. Since this script is intended to run in a temp clone, consider setting a local identity for that clone (e.g. git -C <repo> config user.name/user.email, or passing -c user.name=... -c user.email=... to git am) so just build-docs::all_with_old works in a clean environment.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do this explicitly on the CI


def build_docs_for_release(release: str, repo_path: Path, outdir: Path) -> None:
"""Build documentation for a given release."""
print(f"Building docs for {release}")

# Check out release
sp.run(
("git", "-C", str(repo_path), "checkout", release),
check=True,
capture_output=True,
)

# Apply patches, if any
apply_patches_for_release(release, repo_path)

# Build docs
sp.run(("just", f"{repo_path!s}/build-docs"), check=True)

# Patch versions.html to redirect to main versions page
with (repo_path / "book" / "versions.html").open("w", encoding="utf-8") as f:
f.write(f"""<head>
<meta http-equiv="Refresh" content="0; URL={DOCS_SITE_ROOT}/versions.html" />
</head>""")

# Move to output directory
release_outdir = outdir / release
print(f"Copying to {release_outdir}")
shutil.move((repo_path / "book"), release_outdir)


def build_old_docs() -> None:
"""Build documentation for previous releases."""
outdir = REPO_ROOT / "book" / "release"
outdir.mkdir(parents=True, exist_ok=True)

# Clone this repo to a temporary directory
with TemporaryDirectory() as tmpdir:
repo_path = Path(tmpdir)
clone_repo_to(repo_path)

# Generate documentation for each previous release
for release in get_releases():
build_docs_for_release(release, repo_path, outdir)


if __name__ == "__main__":
build_old_docs()
31 changes: 31 additions & 0 deletions docs/generate_versions_docs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env python3
# /// script
# dependencies = [
# "jinja2",
# ]
# ///
#
# A script to generate the versions.md file, listing links to old versions of documentation.

from pathlib import Path

from jinja2 import Environment, FileSystemLoader
from release import get_releases

DOCS_DIR = Path(__file__).parent.absolute()


def generate_versions_md() -> None:
"""Write the versions.md file."""
path = DOCS_DIR / "versions.md"
print(f"Writing {path}")
env = Environment(loader=FileSystemLoader(DOCS_DIR / "templates"))
template = env.get_template("versions.md.jinja")
out = template.render(releases=get_releases())

with path.open("w", encoding="utf-8") as f:
f.write(out)


if __name__ == "__main__":
generate_versions_md()
23 changes: 23 additions & 0 deletions docs/release/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""Common functionality for working with different versions."""

import re
import subprocess as sp


def is_release_tag(tag: str) -> bool:
"""Whether the git tag indicates a version.

We don't include pre-releases.
"""
return re.match(r"^v[0-9]+\.[0-9]+\.[0-9]+$", tag) is not None


def get_releases() -> list[str]:
"""Get all release tags for this repo, sorted by semantic version."""
ret = sp.run(
("git", "tag", "-l", "--sort=-version:refname"),
Comment thread
alexdewar marked this conversation as resolved.
capture_output=True,
check=True,
encoding="utf-8",
)
return [tag for tag in ret.stdout.splitlines() if is_release_tag(tag)]
Comment thread
alexdewar marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
From 3129c56aea05dd2b7e88c76bb2050fded0529243 Mon Sep 17 00:00:00 2001
From: Aurash Karimi <a.karimi@imperial.ac.uk>
Date: Wed, 19 Nov 2025 09:48:11 +0000
Subject: [PATCH] remove unrecognised parameter

---
book.toml | 1 -
1 file changed, 1 deletion(-)

diff --git a/book.toml b/book.toml
index 41fba09c..84d181ee 100644
--- a/book.toml
+++ b/book.toml
@@ -1,7 +1,6 @@
[book]
authors = ["Alex Dewar"]
language = "en"
-multilingual = false
src = "docs"
title = "MUSE2"

--
2.53.0
8 changes: 8 additions & 0 deletions docs/templates/versions.md.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Other versions of documentation

The MUSE2 documentation for different releases is available below.

- [Current development version](./README.md)
{%- for release in releases %}
- [{{ release }}](release/{{ release }}/index.html)
Copy link

Copilot AI Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The links are relative to the current page (e.g. release/v2.0.0/index.html). If this versions page ever appears inside an old release build (e.g. under release/v2.1.0/), those links will resolve to release/v2.1.0/release/v2.0.0/... and be broken. Consider rendering absolute links (e.g. https://…/MUSE2/release/<tag>/index.html) or passing a base_url into the template that can differ between main docs and old-release docs builds.

Suggested change
- [{{ release }}](release/{{ release }}/index.html)
- [{{ release }}]({{ base_url | default('') }}release/{{ release }}/index.html)

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually these versions.md files won't actually end up being reachable from anywhere, so it doesn't matter

{%- endfor %}
Comment on lines +6 to +8
Copy link

Copilot AI Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Jinja whitespace control in {%- for release in releases %} will strip the newline before the loop, which is likely to merge the “Current development version” bullet with the first generated release bullet into a single line. Use {% for release in releases %} (and similarly for endfor) or move the - to the end tag ({% for ... -%}) if you only intended to trim trailing whitespace.

Suggested change
{%- for release in releases %}
- [{{ release }}](release/{{ release }}/index.html)
{%- endfor %}
{% for release in releases %}
- [{{ release }}](release/{{ release }}/index.html)
{% endfor %}

Copilot uses AI. Check for mistakes.
Loading