Skip to content

fix: break the calibration chain's dependency on its last step - #41

Merged
koen-vg merged 1 commit into
mainfrom
chore-recalibrate-after-simplifications
Jul 28, 2026
Merged

koen-vg merged 1 commit into
mainfrom
chore-recalibrate-after-simplifications

Conversation

@koen-vg

@koen-vg koen-vg commented Jul 28, 2026

Copy link
Copy Markdown
Member

The question this started from

Are calibration provenance and fingerprints still valid after #36 and #38?

Provenance: yes. #38 updated both stamps itself, and both sets match the
current config.

Artefacts: yes, and verified rather than assumed. #38 is 61 files and
-1430 lines, touching build_model.py, build_model/biomass.py, the GBD prep
scripts and prepare_faostat_animal_production.py. Too large to call inert by
reading, so I regenerated both tracked sets from scratch. Every artefact came
back byte-identical -- the calibrated L1 costs to the last digit:

default:       cropland 1.4406072051746193  grassland 0.23976429357967677  feed 0.06151640376563935
gbd-anchored:  cropland 1.606710895127586   grassland 0.24247737358625074  feed 0.05417951839730158

Fingerprints: stale, and now refreshed by that re-run.

What the re-run exposed

Even immediately after a complete chain, --check reported the first four
steps stale:

[STALE] feed  data/curated/calibration/default/deviation_penalty.yaml (input content changed)

deviation_penalty.yaml is written by stability, the last step, and read
by the first four. That is a lower-triangle violation of the same kind as the
feed/food-waste one fixed earlier: the chain is a strict forward pass, so those
four were fit against whatever vintage happened to be on disk.

None of them actually needs the file:

  • feed, food_waste, food_demand pin production to actuals
    (use_actual_production plus the enforced baselines), so the deviation
    penalty has nothing to act on. They now set deviation_penalty.enabled: false, and the solve gates sentinel resolution on that flag.
  • cost drives production stability through hard bounds. The solve resolves
    the "calibrated" sentinel only when the penalty is enabled and
    penalty_mode == "l1", but calibration_artefact_inputs declared the input
    whenever the sentinel merely appeared. Tightening the declaration to mirror
    the solve's own gate drops the dependency with no config change.

The subtle part: a step does not have to enable a successor's calibration to
depend on it. Inheriting l1_cost: "calibrated" from default.yaml is enough.
Noted in the calibration skill for whoever adds the next calibrated key.

Verification

  • Both sets regenerated again after the config change: zero artefact
    changes, so the fix is inert too.
  • tools/calibrate --check is all-green on both sets with matching
    provenance -- the first time the chain has settled.
  • pixi run -e dev pytest 727 passed; ruff, snakefmt and REUSE hooks clean.

Supersedes #39, which re-stamped against pre-#38 code.

The feed, food_waste, food_demand and cost steps all read
`deviation_penalty.yaml`, which the stability step writes at the end of the
chain. The chain is a strict forward pass, so those four were fit against
whatever vintage happened to be on disk, and `tools/calibrate --check`
reported them stale after every complete run.

None of them needs the file. feed, food_waste and food_demand pin production
to actuals via `use_actual_production` and the enforced baselines, so the
deviation penalty has nothing to act on; they now disable it. The cost step
drives production stability through hard bounds, and the solve resolves the
`"calibrated"` sentinel only for an enabled L1 penalty -- the input
declaration in `calibration_artefact_inputs` now mirrors that gate instead of
keying off the sentinel alone, which drops the dependency there without a
config change.

The dependency was inert: regenerating both tracked sets against the current
model reproduces every artefact byte-identically, and `--check` now settles
all-green after a full chain.

Also re-stamps both sets after #36 and #38. Neither moved the artefacts --
verified by a full re-run, not by inspection.
@koen-vg
koen-vg merged commit 01b878e into main Jul 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant