Skip to content

feat(schema): make 0.3.0 the current scaffold.toml schema - #255

Closed
weboko wants to merge 1 commit into
masterfrom
weboko/schema-migration
Closed

feat(schema): make 0.3.0 the current scaffold.toml schema#255
weboko wants to merge 1 commit into
masterfrom
weboko/schema-migration

Conversation

@weboko

@weboko weboko commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

User Story

Every project on the Logos stack carries a scaffold.toml stamped 0.2.0. Moving
the schema to 0.3.0 means init has to take them there, and the naive version of
that change is actively destructive: migrate_to_v0_2_0 short-circuited on
== SCAFFOLD_TOML_SCHEMA_VERSION, so flipping the constant alone would stop a valid
0.2.0 file from short-circuiting and run the pre-0.2.0 rewrites (lssa drop/rename,
url strip, [basecamp] reshape) against it — silently rewriting content the user
is entitled to keep, such as a hand-kept [repos.lssa] for a fork. Detection had the
mirror-image gap: version_stale only matched 0.1.x/0.1/0.0, so a 0.2.0 file
would skip the marker and fall through to the generic "this build expects…" error
instead of the targeted "run init" line.

Surfaced by E3 (init / re-init) and by D1/D7 rerun on a migrated project.

Change Summary

Bumps SCAFFOLD_TOML_SCHEMA_VERSION to 0.3.0 and turns the migrator into a chain:
a migrate dispatcher reads [scaffold].version once, runs the pre-0.2.0 structural
rewrites only for files below the 0.2 series, then applies the 0.3.0 stamp. 0.3.0
changes no sections and no fields — it is a version stamp alone — but it is still a
migration, because the parser exact-matches the current schema and a 0.2.0 file must
be routed through init rather than silently accepted. One init takes a 0.1.x file
all the way to 0.3.0, reporting a single bump ("0.1.0" -> "0.3.0").

Verification

DOGFOODING rerun: E1, E3, D1 (setup/build/deploy), D6, D7 — the last four
run against a project migrated 0.2.0 → 0.3.0, not a fresh one.

  • setup exit 0; doctor 20 PASS / 4 WARN / 0 FAIL (warns: no nix, localnet
    not started)
  • run exit 0 — 5 programs deployed, 0 failed, real sequencer, blocks 5→8
  • D6 runner submitted a real tx (61cf6f66…); wallet account get shows nonce: 1
  • Stale-schema gate: 9 commands + deploy --json / doctor --json all hard-fail at
    init with stdout_bytes=0; scaffold.toml unmutated. After init, none still
    cite the schema.
  • init --dry-run writes nothing; .bak collision refuses, --no-backup proceeds.

Tests: 558 unit + 187 CLI + 3 API + 5 doc, all green; cargo fmt --check clean.
New coverage: 0.3.0 file untouched with a stray [repos.lssa] preserved; 0.2.0 file
stamped with the rewrites proven not to run; the whole 0.2 series gated identically;
pre-0.2.0 migrated in one call; missing [scaffold] stamped; non-init commands
rejecting 0.2.0; fresh init/new emitting 0.3.0.

Checklist

  • CI is green
  • Relevant DOGFOODING scenarios rerun (listed above)
  • Docs updated (README.md, ADR.md) if user-facing behavior changed
  • I have read CONTRIBUTING.md and this PR fits within the weekly contribution cap

Bump SCAFFOLD_TOML_SCHEMA_VERSION to "0.3.0" and turn the migrator into a
chain so one `init` takes any older file all the way to the current schema.

0.3.0 changes no sections and no fields — it is a version stamp alone. It is
still a migration because the parser exact-matches the current schema: a 0.2.0
file must be routed through `init` rather than silently accepted.

Bumping the constant on its own would have been wrong. `migrate_to_v0_2_0`
short-circuited on `== SCAFFOLD_TOML_SCHEMA_VERSION`, so a valid 0.2.0 file
would have stopped short-circuiting and had the pre-0.2.0 rewrites (lssa
drop/rename, url strip, [basecamp] reshape, [basecamp.modules.*] move) run
against it — silently rewriting content it is entitled to keep, such as a
hand-kept [repos.lssa] for a fork.

- migrate.rs: new `migrate` dispatcher reads [scaffold].version once, returns
  an empty report untouched when already current, runs the pre-0.2.0
  structural rewrites only for files below the 0.2 series, then runs the
  0.3.0 stamp. `migrate_to_v0_2_0` keeps its rewrites, loses the stamp and
  the short-circuit, and is private so it cannot be called unguarded. The
  stamp is owned by the last step, so a pre-0.2.0 file reports one bump
  ("0.1.0" -> "0.3.0") instead of one per generation crossed.
- config.rs: `detect_old_schema_markers` flags the whole 0.2 series as stale,
  so a 0.2.0 file — which trips none of the shape markers — gets the targeted
  "run `init`" line instead of the generic version-mismatch bail. The gate is
  a series match in both places, so detection and migration agree on 0.2.1.
- Non-init commands keep hard-failing stale schemas, JSON paths included; the
  parser still accepts exactly one version.
- Fixtures that meant "current schema" move to 0.3.0; legacy 0.1.x inputs stay
  as they are.

Tests: 0.3.0 file untouched with a stray [repos.lssa] preserved; 0.2.0 file
stamped with the rewrites proven not to run (stray section survives, no spel
backfill); whole 0.2 series gated the same; pre-0.2.0 file fully migrated in
one call; missing [scaffold] section stamped; non-init commands rejecting
0.2.0; init dry-run/backup behavior on the 0.2.0 path; fresh init emitting
0.3.0. Full suite green (558 unit + 187 CLI + 3 API + 5 doc).

The crate version is untouched and unrelated to the schema version.
@weboko
weboko requested review from a team and a lite review from Copilot August 20, 2026 22:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the scaffold.toml schema “current version” from 0.2.0 → 0.3.0 and adjusts migration + stale-schema detection so projects with a 0.2.x stamp are routed through init (without accidentally re-running pre-0.2 structural rewrites that could clobber user-kept data such as a stray [repos.lssa]).

Changes:

  • Bump SCAFFOLD_TOML_SCHEMA_VERSION to 0.3.0 and update docs/fixtures accordingly.
  • Convert migration into a chained dispatcher (migrate) that skips pre-0.2 rewrites for 0.2.x files and stamps the file to 0.3.0.
  • Expand stale-schema detection/tests so 0.2.x versions get the targeted “run init” rejection path (including JSON-output commands).

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/common/test_node.rs Updates test scaffold fixture stamp to 0.3.0.
tests/cli.rs Updates minimal fixture stamp, adds coverage for 0.2.0→0.3.0 gating + init behavior.
src/testnode/state.rs Updates test config version to 0.3.0.
src/testnode/pins.rs Updates test config version to 0.3.0.
src/project.rs Updates test config version to 0.3.0.
src/migrate.rs Introduces chained migrator with explicit 0.2.x skip behavior and 0.3.0 stamp step.
src/constants.rs Bumps SCAFFOLD_TOML_SCHEMA_VERSION to 0.3.0 and updates comment.
src/config.rs Expands stale-schema marker detection to include 0.2.x and updates tests/docs.
src/commands/run.rs Updates test config version to 0.3.0.
src/commands/run_state.rs Updates test config version to 0.3.0.
src/commands/init.rs Switches init to call migrate and updates init tests/messages for 0.3.0.
src/commands/idl.rs Updates test config version to 0.3.0.
src/commands/basecamp.rs Updates scaffold fixture stamps in basecamp tests to 0.3.0.
src/api/testnode.rs Updates scaffold fixture stamp to 0.3.0.
src/api/mod.rs Updates fixture writer + comments to schema 0.3.0.
README.md Updates user-facing init semantics/documentation for 0.3.0 and 0.2.x stamping.
ADR.md Documents the 0.3.0 migration model and the role of the 0.2.x gate.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/migrate.rs
Comment on lines +61 to +65
// A file already at 0.2.x carries none of the pre-0.2.0 shapes, so the
// structural rewrites are skipped and only the version stamp runs.
if !is_v0_2_x(&from_version) {
report.merge(migrate_to_v0_2_0(doc)?);
}
@weboko

weboko commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review + targeted dogfooding (E3 / migration paths)

The migrator-as-a-chain refactor is the right shape, and the reasoning in the PR body is correct: flipping the constant alone really would have run the pre-0.2.0 rewrites against valid 0.2.0 files. Splitting the version stamp into migrate_to_v0_3_0 so a pre-0.2.0 file reports one bump ("0.1.0" -> "0.3.0") rather than one per generation is a nice touch, and is_v0_2_x being frozen rather than derived from SCAFFOLD_TOML_SCHEMA_VERSION is exactly right — that constant moves, the waypoint must not.

Test coverage is good: the 0.2-series gate, the stray [repos.lssa] survival case, the one-call pre-0.2.0 path, and the missing-[scaffold] case all pin the behaviour that matters.

I verified the happy paths behave as described — a fresh init stamps 0.3.0, and a 0.2.0 file gets the stamp and nothing else.

One finding: a file newer than the current schema is silently downgraded and structurally rewritten

migrate guards two cases — exact match on the current version, and the 0.2 series. Anything else falls through to the pre-0.2.0 structural rewrites. That includes versions above the current one, because init calls migrate(&mut doc) with no version gate ahead of it.

Reproduced against this branch with a 0.4.0 file carrying a hand-kept [repos.lssa]:

$ lgs init
scaffold.toml in /tmp/futuresch migrated to schema v0.3.0.
  - dropped stale [repos.lssa] (kept [repos.lez])
  - appended [repos.spel] with default pin
  - migrated [basecamp].pin / .source -> [repos.basecamp]
  - bumped [scaffold].version: "0.4.0" -> "0.3.0"

Three things go wrong at once: the hand-kept [repos.lssa] is dropped, the pre-0.2.0 reshapes run against a file that never had those shapes, and the version is walked backwards while being reported as bumped.

This is pre-existing, not a regression — I checked master and it does the same thing ("0.4.0" -> "0.2.0", same [repos.lssa] drop), so it is not something this PR introduces and I would not block on it. But it is the exact mirror-image of the bug this PR exists to fix, and this PR is the one that makes the migrator version-aware, so it is the natural place to close it. Concretely: gate the structural rewrites on "genuinely below the 0.2 series" rather than "not 0.2.x and not current", and treat a version above the current one as an error pointing at a scaffold upgrade rather than something to migrate. The .bak limits the blast radius, but the rewrite is silent and the "bumped … -> 0.3.0" line actively misleads.

Worth noting the detection side already behaves correctly here — version_stale only matches 0.0/0.1.x/0.2.x, so a 0.4.0 file reaches the generic "this build expects…" error on every other command. It is specifically init that has no upper bound.

Minor

is_v0_2_x treats a prerelease like 0.2.0-beta as not-0.2.x (strip_prefix("0.2") leaves -beta, which is neither empty nor .-prefixed), so such a file would take the structural-rewrite path. 0.20.0 is correctly rejected, so the prefix logic is sound for the real cases — just flagging the prerelease edge in case hand-edited stamps are considered in scope.

Nothing else blocking from my side — the core change is well-reasoned and well-tested.

Note on CI: the red validate check is cargo fmt --check failing at src/process.rs:632, which is inherited from master (broken since bd2574d, #34), not caused by this PR. The one-line fix is sitting in #253.


Generated by Claude Code

@weboko weboko left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: make 0.3.0 the current schema — verified ✅

The migration design is clean: migrate() chains one step per generation (pre-0.2.0 structural rewrites → 0.2.0→0.3.0 version stamp), a 0.2.x file skips the structural rewrites and only gets re-stamped, and an already-current file is untouched. The from_version gating (is_v0_2_x) and the single "one bump line, not one per generation" report are the right shape.

Verified on my side:

  • cargo test --lib migrate → 13/13 pass, including the new cases (migrates_v0_2_0_by_stamping_the_version_without_rerunning_v0_2_0_rewrites, migrates_the_whole_0_2_series…, migrates_pre_v0_2_0_to_current_schema_in_one_call). Full lib suite: 574 pass.
  • Real end-to-end with a built binary: a version = "0.2.0" project is rejected by doctor with "Run logos-scaffold init to migrate to v0.3.0; existing settings are preserved"; init migrates it to 0.3.0, writes scaffold.toml.bak, and reports the single bump; doctor then reads it fine. A fresh init stamps 0.3.0.
  • The crate-version-vs-schema-version decoupling is sound and well-documented in the constant's doc comment.

The stale-version detection correctly extends to 0.2/0.2.x while leaving other mismatches to parse_config's more specific error. No blocking issues.


Generated by Claude Code

@weboko

weboko commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Re-review: 0.3.0 as the current schema — verified end-to-end, one documentation gap

Environment: Nix 2.35.2, Rust 1.94.1, r0vm 3.0.5 + risc0 guest toolchains, real LEZ sequencer + wallet from cf3639d8, circuits v0.4.1. cargo test --lib on 28e48cf: 558 passed, 0 failed (migrate: 13/13).

I re-ran the migration against a real 0.2.0 project — one generated by the master binary, not a hand-written fixture — and everything the PR body claims holds up.

Chained migration, one init (0.1.0 file with url, [basecamp].pin/.source, [basecamp.modules.*]):

  - removed [repos.lez].url (use `source` only)
  - appended [repos.spel] with default pin
  - migrated [basecamp].pin / .source -> [repos.basecamp]
  - moved [basecamp.modules.*] -> [modules.*] (1 entry)
  - bumped [scaffold].version: "0.1.0" -> "0.3.0"

One bump line spanning both generations, exactly as designed. The # a user comment that must survive survived, key ordering held, and [localnet] was untouched.

0.2.0 → 0.3.0 stamp-only path: diff scaffold.toml.bak scaffold.toml is a single line (version), confirming the 0.2.x file correctly skips the pre-0.2.0 rewrites — the is_v0_2_x gate is doing its job.

Stale-schema gate: a 0.2.0 file against the new binary gets the targeted line, not the generic mismatch error:

error: scaffold.toml uses an old schema. Run `logos-scaffold init` to migrate to v0.3.0; existing settings are preserved.

Backup safety: --dry-run wrote nothing; a pre-existing scaffold.toml.bak was refused rather than clobbered (I checked the file contents afterward — untouched, and scaffold.toml stayed at 0.2.0); --no-backup proceeded.

Migrated project still works: I migrated the real provisioned project in place and then ran the full path against a live sequencer — localnet start → ready, deploy --json → 5 programs submitted, wallet topup --jsonstatus=success (real tx executed by r0vm). No residue from the migration.

The design reads well too. The chain-with-a-dispatcher shape is right, is_v0_2_x being frozen rather than derived from SCAFFOLD_TOML_SCHEMA_VERSION is a subtle call that the doc comment justifies correctly, and MigrationReport::merge keeping the first hand-edit hint (oldest step) is the right precedence.


The one thing I'd want before this lands: DOGFOODING.md has no migration scenario

The Verification section cites E3 for the init path. E3 is "AI Skills Materialized Into Every Project" — it covers new, init, and init re-run for skill materialization, and asserts shasum stability of the three skill files. It says nothing about schema migration. Grepping the runbook confirms the gap: no scenario mentions scaffold.toml.bak, --no-backup, detect_old_schema, or an older-schema init. The --dry-run hits are all wallet topup --dry-run.

So the behaviours you actually verified for this PR — the stale-schema gate across 9 commands + the two --json surfaces, --dry-run writing nothing, .bak collision refusal, --no-backup, the chained 0.1.x → 0.3.0 report — exist nowhere in the runbook. That matters more here than it usually would, because this PR's user-facing surface is the migration: after it lands, every scaffold.toml on disk hard-fails on every command until its owner runs init. That is the single highest-traffic path in the release, and a dogfooder following DOGFOODING.md end-to-end would never exercise it.

The runbook's own maintenance note is the argument:

update this document whenever first-class commands, templates, supported workflows, or major user-facing behaviors are added, removed, or materially changed. If the product surface changes and this runbook does not, the runbook is wrong.

I'd add an E4 (or extend E3) covering: adopt an existing older-schema project → confirm the gate fires on a representative command → init --dry-runinit → assert the .bak, the one-line report, and that a previously-working project still runs. You have already executed every step of that; it just needs to be written down where the next dogfooder will find it. Also worth adding a line to the "Which scenarios to rerun" table for changes to SCAFFOLD_TOML_SCHEMA_VERSION / migrate.rs / detect_old_schema_markers — that table is otherwise thorough and this is a conspicuous omission.

Minor, non-blocking

detect_old_schema_markers gates version_stale on an explicit prefix list (0.2. / 0.2 / 0.1. / 0.1 / 0.0). Each future bump has to remember to append to it, and forgetting means a stale file silently falls through to the generic error instead of the targeted one — the exact mirror-image bug the PR body describes fixing for 0.2.0. Since is_v0_2_x already establishes "frozen waypoint" as the pattern, a comment on that list noting "append the outgoing series here on every bump" would make the maintenance obligation visible at the point of edit. Not worth changing the logic for.


Verdict: the code is correct and I could not break it. I'd approve once the DOGFOODING scenario is added — for a change whose entire blast radius is a migration every user must run, the runbook not describing that migration is the thing I'd fix before merging, not after.


Generated by Claude Code

@weboko weboko closed this Aug 26, 2026
@weboko
weboko deleted the weboko/schema-migration branch August 26, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants