From 953d4524200cff8d0d642853bc0ce448852cd7dd Mon Sep 17 00:00:00 2001 From: Andreas Richardson Date: Tue, 8 Sep 2026 11:17:44 +0100 Subject: [PATCH 1/3] docs(contrail-gh): document surface-diff tripwire and version tagging Passport's new CLI subcommand slipped past docs/contrail-gh.md's table because nothing mechanically flagged that contrail's public surface had changed. This documents the contrail-gh-side fix: surface-check.yml diffs subcommands and registry ids against a checked-in contrail-surface.json on every pin-bump PR, and tag-pin.yml tags minor/major bumps as vX.Y.Z (never patch-only, since bump-minor-pre-major already makes that split meaningful pre-1.0). Also rewords the "A release is cut" table row, which previously implied merging the Dependabot PR was sufficient on its own. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01X6VSeYrwoBr5FU8ZiwVDTn --- docs/contrail-gh.md | 84 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 14 deletions(-) diff --git a/docs/contrail-gh.md b/docs/contrail-gh.md index dfd85cd..768a2dc 100644 --- a/docs/contrail-gh.md +++ b/docs/contrail-gh.md @@ -19,16 +19,17 @@ whether an instance's TIM key still works. ## Changes here that require a change there -| Change in contrail | What contrail-gh needs | -| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `CSV_FIELDS` gains, loses or reorders a column | Regenerate `flight_emissions.csv` (header row only) | -| `contrail sync` writes a new durable output file | Add it to the `git add` line in `sync.yml` | -| contrail gains an importer that reads a **file** | A directory for it, the env var in _both_ `sync.yml` and `check-instance.yml`, and a guard in `check-template.yml` that the public template never carries one | -| A new column or behaviour users should know about | Update the template's README | -| contrail generates a private derived artifact | Gitignore the default path and document that it must not be committed | -| The config **file** schema changes | Nothing, as long as the environment variable names hold. The template ships no config file and configures contrail entirely by env, so those names are the contract | -| A release is cut | Nothing here — Dependabot opens the version-bump PR in the instance | -| `requires-python` rises above the workflow's version | Raise `python-version` in `sync.yml` | +| Change in contrail | What contrail-gh needs | +| ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `CSV_FIELDS` gains, loses or reorders a column | Regenerate `flight_emissions.csv` (header row only) | +| `contrail sync` writes a new durable output file | Add it to the `git add` line in `sync.yml` | +| contrail gains an importer that reads a **file** | A directory for it, the env var in _both_ `sync.yml` and `check-instance.yml`, and a guard in `check-template.yml` that the public template never carries one | +| A new column or behaviour users should know about | Update the template's README | +| contrail generates a private derived artifact | Gitignore the default path and document that it must not be committed | +| The config **file** schema changes | Nothing, as long as the environment variable names hold. The template ships no config file and configures contrail entirely by env, so those names are the contract | +| contrail gains or loses a CLI subcommand, or an importer/provider/storage/raw-log id | Caught mechanically by the surface-diff check (`surface-check.yml`, called from both `check-template.yml` and `check-instance.yml`) — it fails the pin-bump PR until `contrail-surface.json` is regenerated. Satisfying it only means acknowledging the new id exists; it does not require configuring or testing a new importer, since an importer nothing in config references is never invoked (see `config.py`'s `lookup_type`) | +| A release is cut | Dependabot opens the version-bump PR automatically — that's the trigger, not the whole story. Check whether the release also hit one of the rows above (or below) before merging it | +| `requires-python` rises above the workflow's version | Raise `python-version` in `sync.yml` | The release row used to be the one release-please couldn't take off your hands: it rewrites the version pins in this repo's own README, because @@ -41,6 +42,15 @@ reaches PyPI, which the `publish` job in `release-please.yml` now does on its own. Someone still has to review and merge the Dependabot PR — see contrail-gh's own README for that side of it. +Dependabot closes the mechanical gap — a bump PR always opens, unprompted. It +does not close the content gap: whether that release also needs one of the other +changes in this table is a judgment call, and until the surface-diff check +existed, nothing forced anyone to make it before merging. That check narrows the +content gap without closing it — it can tell that `contrail.cli`'s subcommand set +or a registry's ids changed, which is a mechanical fact, but not that the new +thing is _sensitive_ the way Passport's private output is. It's a gate that +forces a human to look at this table, not a substitute for reading it. + The `flighty_csv` importer is the first of the file-reading kind, and it is the reason that third row exists. An export is a manual file rather than a feed URL, so an instance commits its exports to `flighty/` and points `FLIGHTY_CSV_PATH` at @@ -61,10 +71,31 @@ Regenerating the header, from a checkout of contrail: > ../contrail-gh/flight_emissions.csv ``` -The header must match the **pinned** version's schema, not `main`'s. If a schema -change hasn't been released yet, regenerate the header in the same change that -bumps the pin — not before, or the template ships a header no released contrail -writes. +Regenerating `contrail-surface.json`, from the same checkout: + +```bash +./venv/bin/python -c ' +import json, re, subprocess +from contrail.emissions import PROVIDERS +from contrail.importers import IMPORTERS +from contrail.storage import CSV_FIELDS, RAW_LOGS, STORAGES +help_text = subprocess.run(["./venv/bin/contrail", "--help"], capture_output=True, text=True).stdout +subcommands = sorted(re.search(r"\{([\w,-]+)\}", help_text).group(1).split(",")) +print(json.dumps({ + "csv_fields": list(CSV_FIELDS), + "importers": sorted(IMPORTERS), + "providers": sorted(PROVIDERS), + "storages": sorted(STORAGES), + "raw_logs": sorted(RAW_LOGS), + "subcommands": subcommands, +}, indent=2, sort_keys=True)) +' > ../contrail-gh/contrail-surface.json +``` + +Both must match the **pinned** version's shape, not `main`'s. If a schema or +surface change hasn't been released yet, regenerate both in the same change that +bumps the pin — not before, or the template ships a shape no released contrail +matches. ## Why the pin is never `main` @@ -79,6 +110,31 @@ may sit on an older schema for a while; contrail handles that — a column a row doesn't have is treated as back-fill, not as a changed flight (see [resync.md](resync.md)). +## Tagging contrail-gh by version + +Every push to contrail-gh's `main` that changes `requirements.txt` — i.e. every +merged Dependabot pin-bump PR — is checked by `tag-pin.yml` there, which tags the +merge commit `vX.Y.Z` matching the new pin, but **only for a minor or major bump, +never a patch-only one.** + +That split is reliable because of how contrail is released: +`release-please-config.json` sets `bump-minor-pre-major: true` and +`bump-patch-for-minor-pre-major: false`. While contrail is pre-1.0, `feat:` and +breaking changes both bump the minor version (major stays `0` until a deliberate +1.0 cut), and `fix:`/`perf:` bump patch only. So "minor-or-major" already reliably +means "something feature-level or breaking landed," and "patch" reliably means +"pure fix" — the tag doesn't need to inspect commit messages itself, just compare +version numbers. + +The tag is a convenience pointer for humans — "this template state targets +contrail vX.Y" — not a substitute for the surface-diff check above, which still +runs on every bump regardless of size: a patch-only release cannot add a +subcommand or a registry id under this scheme, but the check doesn't take that on +faith. + +contrail-gh has no semver of its own; the tag borrows contrail's version purely +for traceability across two repos with no other way to correlate history. + ## What must never appear in the template Real flight data. `flight_emissions.csv` is the header row and nothing else, From e73f4330fa34ae73150050a24712c170256f95e9 Mon Sep 17 00:00:00 2001 From: Andreas Richardson Date: Tue, 8 Sep 2026 20:49:14 +0100 Subject: [PATCH 2/3] docs(contrail-gh): fix surface regen recipe against older pins The recipe imported RAW_LOGS and STORAGES unconditionally, which don't exist in contrail 0.4.0, the version contrail-gh's requirements.txt currently pins. Mirror surface-check.yml's optional_import so the recipe produces the same null values the CI check and the committed baseline already expect, instead of an ImportError. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01FiQX66jnRTAVwaYrBF7vQV --- docs/contrail-gh.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/contrail-gh.md b/docs/contrail-gh.md index 768a2dc..0e03ec4 100644 --- a/docs/contrail-gh.md +++ b/docs/contrail-gh.md @@ -78,15 +78,25 @@ Regenerating `contrail-surface.json`, from the same checkout: import json, re, subprocess from contrail.emissions import PROVIDERS from contrail.importers import IMPORTERS -from contrail.storage import CSV_FIELDS, RAW_LOGS, STORAGES +from contrail.storage import CSV_FIELDS + +def optional_import(module, name): + # None means the pin predates this registry, not an error — see + # surface-check.yml, which reads the result the same way. + try: + mod = __import__(module, fromlist=[name]) + return sorted(getattr(mod, name)) + except (ImportError, AttributeError): + return None + help_text = subprocess.run(["./venv/bin/contrail", "--help"], capture_output=True, text=True).stdout subcommands = sorted(re.search(r"\{([\w,-]+)\}", help_text).group(1).split(",")) print(json.dumps({ "csv_fields": list(CSV_FIELDS), "importers": sorted(IMPORTERS), "providers": sorted(PROVIDERS), - "storages": sorted(STORAGES), - "raw_logs": sorted(RAW_LOGS), + "storages": optional_import("contrail.storage", "STORAGES"), + "raw_logs": optional_import("contrail.storage", "RAW_LOGS"), "subcommands": subcommands, }, indent=2, sort_keys=True)) ' > ../contrail-gh/contrail-surface.json From c0f1ee2dd287e68cd6d6cdc2f2146abcb0920cf5 Mon Sep 17 00:00:00 2001 From: Andreas Richardson Date: Tue, 8 Sep 2026 20:57:48 +0100 Subject: [PATCH 3/3] docs(contrail-gh): split the surface-diff table row by default posture One row lumped CLI subcommands together with importer/provider/storage/ raw-log ids under identical guidance, but they don't share a default posture: importers and providers should default to being scaffolded (an env var, a directory, a gitignore/guard if file-based) so a new source is as easy as possible to adopt; storages and raw-logs need the same wiring instinct but no repo-tree scaffolding by default, since they're output choices rather than data-source choices; CLI subcommands get no default at all -- Passport deserved active promotion, a hypothetical diagnostic flag would deserve none, and the table can't tell those apart in one shared line. Matches the per-key guidance just added to surface-check.yml in atdr/contrail-gh#14. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01X6VSeYrwoBr5FU8ZiwVDTn --- docs/contrail-gh.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/contrail-gh.md b/docs/contrail-gh.md index 0e03ec4..24a5a97 100644 --- a/docs/contrail-gh.md +++ b/docs/contrail-gh.md @@ -19,17 +19,19 @@ whether an instance's TIM key still works. ## Changes here that require a change there -| Change in contrail | What contrail-gh needs | -| ------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `CSV_FIELDS` gains, loses or reorders a column | Regenerate `flight_emissions.csv` (header row only) | -| `contrail sync` writes a new durable output file | Add it to the `git add` line in `sync.yml` | -| contrail gains an importer that reads a **file** | A directory for it, the env var in _both_ `sync.yml` and `check-instance.yml`, and a guard in `check-template.yml` that the public template never carries one | -| A new column or behaviour users should know about | Update the template's README | -| contrail generates a private derived artifact | Gitignore the default path and document that it must not be committed | -| The config **file** schema changes | Nothing, as long as the environment variable names hold. The template ships no config file and configures contrail entirely by env, so those names are the contract | -| contrail gains or loses a CLI subcommand, or an importer/provider/storage/raw-log id | Caught mechanically by the surface-diff check (`surface-check.yml`, called from both `check-template.yml` and `check-instance.yml`) — it fails the pin-bump PR until `contrail-surface.json` is regenerated. Satisfying it only means acknowledging the new id exists; it does not require configuring or testing a new importer, since an importer nothing in config references is never invoked (see `config.py`'s `lookup_type`) | -| A release is cut | Dependabot opens the version-bump PR automatically — that's the trigger, not the whole story. Check whether the release also hit one of the rows above (or below) before merging it | -| `requires-python` rises above the workflow's version | Raise `python-version` in `sync.yml` | +| Change in contrail | What contrail-gh needs | +| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `CSV_FIELDS` gains, loses or reorders a column | Regenerate `flight_emissions.csv` (header row only) | +| `contrail sync` writes a new durable output file | Add it to the `git add` line in `sync.yml` | +| contrail gains an importer that reads a **file** | A directory for it, the env var in _both_ `sync.yml` and `check-instance.yml`, and a guard in `check-template.yml` that the public template never carries one | +| A new column or behaviour users should know about | Update the template's README | +| contrail generates a private derived artifact | Gitignore the default path and document that it must not be committed | +| The config **file** schema changes | Nothing, as long as the environment variable names hold. The template ships no config file and configures contrail entirely by env, so those names are the contract | +| contrail gains or loses an importer or emissions-provider id | Caught mechanically by the surface-diff check (`surface-check.yml`, called from both `check-template.yml` and `check-instance.yml`) — it fails the pin-bump PR until `contrail-surface.json` is regenerated. Default posture: scaffold it, don't just acknowledge it — an env var in `sync.yml` and `check-instance.yml` at minimum, plus a directory, a `.gitignore` entry, and a `check-template.yml` guard if it reads a file (`flighty_csv`/`flighty/` is the pattern). Never requires configuring or testing the new source itself, since an importer nothing in config references is never invoked (see `config.py`'s `lookup_type`) | +| contrail gains or loses a storage or raw-log id | Caught the same way. Wire it (an env var) if contrail exposes a way to select it, but unlike importers this doesn't need repo-tree scaffolding by default — a non-local storage or raw log doesn't write into the template. Exception: a raw log that turns out to be file-based, the way `jsonl` is, needs the same treatment as a file-reading importer | +| contrail gains or loses a CLI subcommand | Caught the same way, but with no default action — this is a judgment call every time. Read what the new command actually does (its own `--help`, the CHANGELOG entry for the release) before deciding whether instance owners should be actively shown how to use it (Passport is the worked example: gitignored, documented) or whether it's internal or diagnostic tooling that needs no template reaction. Don't assume either answer just because a subcommand changed | +| A release is cut | Dependabot opens the version-bump PR automatically — that's the trigger, not the whole story. Check whether the release also hit one of the rows above (or below) before merging it | +| `requires-python` rises above the workflow's version | Raise `python-version` in `sync.yml` | The release row used to be the one release-please couldn't take off your hands: it rewrites the version pins in this repo's own README, because