Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions copier.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,11 @@ python_version:
type: str
default: "3.10"
help: Minimum supported Python version

codescene_project_id:
type: str
default: ""
help: >-
CodeScene project id for the coverage gate (leave blank until this repo
is onboarded to CodeScene). When set, the value is surfaced in the CI
deferred-gate note; token-less repos stay green regardless.
53 changes: 27 additions & 26 deletions template/.github/workflows/ci.yml.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ jobs:
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
with:
persist-credentials: false
# Full history so a future `cs-coverage check` can reach the
# pull request's merge base (see the deferred gate note below).
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
Expand All @@ -32,7 +35,7 @@ jobs:

{% if use_rust %}
- name: Set up Rust
uses: leynos/shared-actions/.github/actions/setup-rust@455d9ed03477c0026da96c2541ca26569a74acac
uses: leynos/shared-actions/.github/actions/setup-rust@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
with:
toolchain: stable

Expand Down Expand Up @@ -82,36 +85,34 @@ jobs:
- name: Audit dependencies
run: make audit

# Coverage runs on pull requests only. Pushes to main upload their
# own coverage (and advance the ratchet baseline) via
# coverage-main.yml, so generating it here as well would duplicate
# the work and the CodeScene upload for the same commit.
- name: Test and Measure Coverage
uses: leynos/shared-actions/.github/actions/generate-coverage@296dc4aa07acf3a7f60a54fb6bccb5672a597479
if: ${{ "{{" }} github.event_name == 'pull_request' {{ "}}" }}
uses: leynos/shared-actions/.github/actions/generate-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
with:
output-path: coverage.xml
format: cobertura
artefact-name-suffix: {{ package_name | lower | replace("_", "-") }}
with-ratchet: 'true'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fix the ratchet pin before enabling it

With with-ratchet enabled here, generated repos rely on the pinned generate-coverage action to restore the main-branch baseline before PR checks. I checked the pinned action at 927edd45ae77be4251a8a18ca9eb5613a2e32cbd: it restores with key: ratchet-baseline-${{ runner.os }}-${{ github.run_id }} and restore-keys: ratchet-baseline-${{ runner.os }}-, but saves with key: ratchet-baseline-${{ runner.os }}. Because GitHub cache restore keys are prefix matches, the saved cache does not start with the restore prefix, so the baseline is never restored and coverage drops compare against the freshly-created 0 baseline. Please bump/fix the action pin or leave the ratchet disabled until the cache keys align.

Useful? React with 👍 / 👎.

{% if use_rust %}
cargo-manifest: rust_extension/Cargo.toml
{% endif %}

- name: Install CodeScene Coverage CLI
if: env.CS_ACCESS_TOKEN != ''
run: |
curl -fsSL --output install-cs-coverage-tool.sh \
https://downloads.codescene.io/enterprise/cli/install-cs-coverage-tool.sh
if [ -n "${CODESCENE_CLI_SHA256:-}" ]; then
echo "${CODESCENE_CLI_SHA256} install-cs-coverage-tool.sh" | sha256sum -c -
else
echo "::notice::CODESCENE_CLI_SHA256 is not set; skipping installer checksum verification"
fi
bash install-cs-coverage-tool.sh -y

- name: Upload coverage to CodeScene
if: env.CS_ACCESS_TOKEN != ''
run: |
if [ ! -f coverage.xml ]; then
echo "coverage.xml not found!"
exit 1
fi
cs-coverage upload \
--format "cobertura" \
--metric "line-coverage" \
coverage.xml
# The changed-line `cs-coverage check` gate is deliberately not wired
# in yet: CodeScene rejects `cs-coverage check` for any project whose
# coverage-gates configuration is absent ("Lacks the gates
# configuration."), which is a CodeScene-side, per-project setting
# rather than a CI defect. Once this repo is onboarded to CodeScene
{% if codescene_project_id -%}
# (project {{ codescene_project_id }}) and coverage-main.yml has
# uploaded to main at least once, add a `mode: check` step guarded on
# `CS_ACCESS_TOKEN != ''` with
# `project-url: https://api.codescene.io/v2/projects/{{ codescene_project_id }}`.
{%- else -%}
# — set the `codescene_project_id` copier answer — and coverage-main.yml
# has uploaded to main at least once, add a `mode: check` step guarded
# on `CS_ACCESS_TOKEN != ''` with the project's
# `project-url: https://api.codescene.io/v2/projects/<id>`.
{%- endif %}
72 changes: 72 additions & 0 deletions template/.github/workflows/coverage-main.yml.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Coverage (main)

# CodeScene accepts `cs-coverage upload` only for analysed branches, so
# main-branch coverage is uploaded here on push; pull requests generate
# coverage (and, in a future fast-follow, run the changed-line
# `cs-coverage check` gate) in ci.yml instead. This workflow also
# advances the coverage ratchet baseline: caches saved on main are
# readable by every pull-request run, so the baseline written here is the
# one PR ratchet checks compare against.
#
# `workflow_dispatch` is mandatory, not a convenience: merges performed by
# the automerge workflow's GITHUB_TOKEN do not fire push-event workflows,
# so automerged changes to main only receive coverage via a manual
# dispatch.

on:
push:
branches: [main]
workflow_dispatch:

jobs:
coverage-upload:
runs-on: ubuntu-latest
permissions:
contents: read
env:
CS_ACCESS_TOKEN: ${{ "{{" }} secrets.CS_ACCESS_TOKEN || '' {{ "}}" }}
CODESCENE_CLI_SHA256: ${{ "{{" }} vars.CODESCENE_CLI_SHA256 || '' {{ "}}" }}
{% if use_rust %}
CARGO_TERM_COLOR: always
BUILD_PROFILE: debug
{% endif %}
steps:
- name: Check out repository
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4
with:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: '3.13'

- name: Install uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5

{% if use_rust %}
- name: Set up Rust
uses: leynos/shared-actions/.github/actions/setup-rust@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
with:
toolchain: stable

{% endif %}
- name: Generate coverage
uses: leynos/shared-actions/.github/actions/generate-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
with:
output-path: coverage.xml
format: cobertura
artefact-name-suffix: {{ package_name | lower | replace("_", "-") }}
with-ratchet: 'true'
{% if use_rust %}
cargo-manifest: rust_extension/Cargo.toml
{% endif %}

- name: Upload coverage data to CodeScene
if: env.CS_ACCESS_TOKEN != ''
uses: leynos/shared-actions/.github/actions/upload-codescene-coverage@927edd45ae77be4251a8a18ca9eb5613a2e32cbd
with:
format: cobertura
path: coverage.xml
access-token: ${{ "{{" }} env.CS_ACCESS_TOKEN {{ "}}" }}
installer-checksum: ${{ "{{" }} env.CODESCENE_CLI_SHA256 {{ "}}" }}
134 changes: 133 additions & 1 deletion tests/helpers/ci_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,15 +71,22 @@ def assert_ci_coverage_action_contract(
assert (
coverage_step.get("uses")
== "leynos/shared-actions/.github/actions/generate-coverage"
"@296dc4aa07acf3a7f60a54fb6bccb5672a597479"
"@927edd45ae77be4251a8a18ca9eb5613a2e32cbd"
), "expected CI to use the pinned shared coverage action"
assert coverage_step.get("if") == "${{ github.event_name == 'pull_request' }}", (
"expected CI coverage generation to be guarded to pull requests so "
"coverage-main.yml owns the push-to-main upload"
)
coverage_inputs = require_mapping(coverage_step, "with", "coverage step")
assert coverage_inputs.get("output-path") == "coverage.xml", (
"expected CI coverage output path to match the act assertion"
)
assert coverage_inputs.get("format") == "cobertura", (
"expected CI coverage format to match the CodeScene upload"
)
assert coverage_inputs.get("with-ratchet") == "true", (
"expected CI coverage to advance the ratchet baseline"
)
assert coverage_inputs.get("artefact-name-suffix") == package_name.replace(
"_", "-"
), "expected package-specific coverage artefact name suffix"
Expand All @@ -93,6 +100,127 @@ def assert_ci_coverage_action_contract(
)


def assert_coverage_main_workflow_contract(
*, coverage_main_workflow: str, package_name: str, use_rust: bool
) -> None:
"""Assert the generated push-to-main coverage upload workflow contract.

Parameters
----------
coverage_main_workflow : str
UTF-8 text of the generated ``coverage-main.yml`` workflow.
package_name : str
Generated Python import package name used to derive the coverage
artefact suffix.
use_rust : bool
Whether the rendered variant includes the optional Rust extension.

Returns
-------
None
The helper returns after the coverage-main workflow contract matches
expectations.

Raises
------
AssertionError
Raised when the triggers, guarded upload, coverage generation, or Rust
manifest wiring are wrong.

Examples
--------
Validate a rendered coverage-main workflow::

assert_coverage_main_workflow_contract(
coverage_main_workflow=coverage_main_workflow,
package_name="example_pkg",
use_rust=True,
)
"""
# ``on:`` is parsed by PyYAML as the boolean key ``True``, so assert the
# triggers against the raw text to stay robust.
assert "branches: [main]" in coverage_main_workflow, (
"expected coverage-main to upload on push to main"
)
assert "workflow_dispatch:" in coverage_main_workflow, (
"expected coverage-main to allow manual dispatch (automerge pushes do "
"not fire push-event workflows)"
)
parsed = parse_yaml_mapping(coverage_main_workflow, "coverage-main workflow")
jobs = require_mapping(parsed, "jobs", "coverage-main workflow")
upload_job = require_mapping(jobs, "coverage-upload", "coverage-main jobs")
permissions = require_mapping(upload_job, "permissions", "coverage-upload job")
assert permissions.get("contents") == "read", (
"expected coverage-main upload job to restrict GITHUB_TOKEN to reads"
)
steps = require_sequence(upload_job, "steps", "coverage-upload job")

generate_steps = [
step
for step in steps
if isinstance(step, dict) and step.get("name") == "Generate coverage"
]
assert len(generate_steps) == 1, "expected one shared coverage generation step"
generate_step = generate_steps[0]
assert (
generate_step.get("uses")
== "leynos/shared-actions/.github/actions/generate-coverage"
"@927edd45ae77be4251a8a18ca9eb5613a2e32cbd"
), "expected coverage-main to use the pinned shared coverage action"
generate_inputs = require_mapping(generate_step, "with", "coverage generation step")
assert generate_inputs.get("output-path") == "coverage.xml", (
"expected coverage-main to write cobertura coverage.xml"
)
assert generate_inputs.get("format") == "cobertura", (
"expected coverage-main to generate cobertura coverage for CodeScene"
)
assert generate_inputs.get("with-ratchet") == "true", (
"expected coverage-main to advance the authoritative ratchet baseline"
)
assert generate_inputs.get("artefact-name-suffix") == package_name.replace(
"_", "-"
), "expected package-specific coverage artefact name suffix"

upload_steps = [
step
for step in steps
if isinstance(step, dict)
and step.get("name") == "Upload coverage data to CodeScene"
]
assert len(upload_steps) == 1, "expected one guarded CodeScene upload step"
upload_step = upload_steps[0]
assert (
upload_step.get("uses")
== "leynos/shared-actions/.github/actions/upload-codescene-coverage"
"@927edd45ae77be4251a8a18ca9eb5613a2e32cbd"
), "expected coverage-main to use the pinned shared upload action"
assert upload_step.get("if") == "env.CS_ACCESS_TOKEN != ''", (
"expected the CodeScene upload to skip when the access token is absent"
)
upload_inputs = require_mapping(upload_step, "with", "CodeScene upload step")
assert upload_inputs.get("format") == "cobertura", (
"expected the CodeScene upload to send cobertura coverage"
)
assert upload_inputs.get("path") == "coverage.xml", (
"expected the CodeScene upload to read coverage.xml"
)

if use_rust:
assert generate_inputs.get("cargo-manifest") == "rust_extension/Cargo.toml", (
"expected Rust variant coverage-main to pass the extension manifest"
)
assert "leynos/shared-actions/.github/actions/setup-rust" in (
coverage_main_workflow
), "expected Rust variant coverage-main to set up Rust"
else:
assert "cargo-manifest" not in coverage_main_workflow, (
"expected pure-Python coverage-main to omit Rust coverage inputs"
)
assert "setup-rust" not in coverage_main_workflow, (
"expected pure-Python coverage-main to omit Rust setup"
)


def _parse_ci_workflow(ci_workflow: str) -> dict[str, Any]:
"""Parse a generated CI workflow as a YAML mapping."""
return parse_yaml_mapping(ci_workflow, "CI workflow")
Expand Down Expand Up @@ -144,6 +272,10 @@ def _assert_ci_workflow_contracts(
assert "leynos/shared-actions/.github/actions/generate-coverage" in ci_workflow, (
"expected generated CI workflow to use the shared coverage action"
)
assert "cs-coverage upload" not in ci_workflow, (
"expected the pull-request CI job to defer the CodeScene upload to "
"coverage-main.yml"
)
if use_rust:
assert "leynos/shared-actions/.github/actions/setup-rust" in ci_workflow, (
"expected Rust variant CI to set up Rust"
Expand Down
11 changes: 11 additions & 0 deletions tests/helpers/tooling_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
_assert_ci_workflow_contracts,
_parse_ci_workflow,
assert_ci_coverage_action_contract,
assert_coverage_main_workflow_contract,
)
from tests.helpers.makefile_contracts import (
_assert_makefile_contracts,
Expand All @@ -33,6 +34,7 @@
__all__ = [
"assert_ci_coverage_action_contract",
"assert_common_make_targets",
"assert_coverage_main_workflow_contract",
"assert_generated_tooling_contracts",
]

Expand All @@ -44,6 +46,7 @@ def assert_generated_tooling_contracts(
pyproject: dict[str, Any],
makefile: str,
ci_workflow: str,
coverage_main_workflow: str,
act_validation_workflow: str,
release_workflow: str,
build_wheels_workflow: str,
Expand All @@ -65,6 +68,8 @@ def assert_generated_tooling_contracts(
UTF-8 text of the generated Makefile.
ci_workflow : str
UTF-8 text of the generated CI workflow.
coverage_main_workflow : str
UTF-8 text of the generated push-to-main coverage upload workflow.
act_validation_workflow : str
UTF-8 text of the generated act-validation workflow.
release_workflow : str
Expand Down Expand Up @@ -99,6 +104,7 @@ def assert_generated_tooling_contracts(
pyproject=pyproject,
makefile=makefile,
ci_workflow=ci_workflow,
coverage_main_workflow=coverage_main_workflow,
act_validation_workflow=act_validation_workflow,
release_workflow=release_workflow,
build_wheels_workflow=build_wheels_workflow,
Expand Down Expand Up @@ -126,6 +132,11 @@ def assert_generated_tooling_contracts(
ci_workflow=ci_workflow,
use_rust=use_rust,
)
assert_coverage_main_workflow_contract(
coverage_main_workflow=coverage_main_workflow,
package_name=package_name,
use_rust=use_rust,
)
_assert_act_validation_workflow_contracts(
act_validation_workflow=act_validation_workflow,
use_rust=use_rust,
Expand Down
Loading
Loading