Skip to content

refactor: simplify workflow and remove obsolete code paths - #38

Merged
koen-vg merged 10 commits into
mainfrom
refactor/codebase-simplification
Jul 28, 2026
Merged

koen-vg merged 10 commits into
mainfrom
refactor/codebase-simplification

Conversation

@koen-vg

@koen-vg koen-vg commented Jul 28, 2026

Copy link
Copy Markdown
Member

Summary

This is a multi-pass codebase cleanup focused on removing dead paths, consolidating duplicated logic, and making the active workflow easier to understand and maintain.

  • remove obsolete export, migration, profiling, synchronization, compatibility, and no-op validation paths
  • remove unused configuration keys and an orphaned sensitivity test config
  • require complete configuration and reject unknown or ignored scenario overrides
  • consolidate shared IHME country mapping and analysis invocation logic
  • track Python package dependencies accurately in Snakemake rules
  • simplify model construction by removing dead water, biomass, grassland, and land-use-change branches
  • tighten calibration provenance tracking for structural inputs
  • align schemas, tests, documentation, and the changelog with the resulting interfaces

Overall, the branch changes 62 files and removes about 1,000 net lines.

Motivation

The repository had accumulated several one-off tools, transitional compatibility branches, duplicated execution adapters, and configuration options that were no longer consumed. Some scenario keys could also be accepted without affecting a solve, and several imported workflow modules were missing from Snakemake's code dependencies.

Keeping these paths made the supported workflow ambiguous and increased the chance of stale results or silently ignored configuration. Since GLADE is under active development and does not require backward compatibility, this PR makes the current interfaces explicit and fail-fast.

User impact

This intentionally removes obsolete command-line tools and configuration keys. Configurations must now be complete after merging with config/default.yaml, and invalid scenario overrides fail instead of being ignored. User-visible removals and migration details are recorded under Unreleased in the changelog.

Validation

  • 716 non-integration/non-plot tests passed
  • targeted provenance, solve namespace, and configuration validation suite: 39 passed
  • complete Snakemake workflow DAG dry-run passed
  • repository-wide Ruff checks passed
  • pre-push hooks passed, including Ruff, snakefmt, REUSE, Furo directive checks, and Sphinx with warnings treated as errors
  • final diff whitespace check passed

The full integration workflow was not executed because it would schedule retrieval rules requiring explicit network authorization. DAG construction and the existing local test coverage passed.

@koen-vg
koen-vg force-pushed the refactor/codebase-simplification branch from 79ec9f2 to 4c9f023 Compare July 28, 2026 21:41
@koen-vg
koen-vg marked this pull request as ready for review July 28, 2026 21:48
@koen-vg
koen-vg merged commit cc412bf into main Jul 28, 2026
4 checks passed
koen-vg added a commit that referenced this pull request Jul 28, 2026
…40)

The deviation-penalty calibration drives its solves in-process, synthesizing
a baseline and a main scenario per Broyden iteration, so the baseline named
in `config/calibration/stability.yaml` is not a scenario the workflow
declares. The stricter check introduced in #38 therefore rejected the config
outright and `tools/calibrate stability` could not build its DAG.
koen-vg added a commit that referenced this pull request Jul 28, 2026
The feed, food_waste, food_demand and cost steps all read
`deviation_penalty.yaml`, which the stability step writes at the end of the
chain. The chain is a strict forward pass, so those four were fit against
whatever vintage happened to be on disk, and `tools/calibrate --check`
reported them stale after every complete run.

None of them needs the file. feed, food_waste and food_demand pin production
to actuals via `use_actual_production` and the enforced baselines, so the
deviation penalty has nothing to act on; they now disable it. The cost step
drives production stability through hard bounds, and the solve resolves the
`"calibrated"` sentinel only for an enabled L1 penalty -- the input
declaration in `calibration_artefact_inputs` now mirrors that gate instead of
keying off the sentinel alone, which drops the dependency there without a
config change.

The dependency was inert: regenerating both tracked sets against the current
model reproduces every artefact byte-identically, and `--check` now settles
all-green after a full chain.

Also re-stamps both sets after #36 and #38. Neither moved the artefacts --
verified by a full re-run, not by inspection.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant