Skip to content

fix(blockchain): make dev-up actually start the local chain (3 stacked bugs) - #93

Merged
flo2517 merged 1 commit into
mainfrom
fix/local-dev-chain-authority-key-provisioning
Aug 7, 2026
Merged

fix(blockchain): make dev-up actually start the local chain (3 stacked bugs)#93
flo2517 merged 1 commit into
mainfrom
fix/local-dev-chain-authority-key-provisioning

Conversation

@flo2517

@flo2517 flo2517 commented Aug 7, 2026

Copy link
Copy Markdown
Owner

What

`make dev-up` was crash-looping on `openinfra-blockchain-node-1`. Root cause turned out to be three separate, stacked bugs, all pre-existing (none introduced this session) and apparently never exercised together end to end since the project moved from manual sealing to Aura+GRANDPA (ADR-009, "local Aura GRANDPA testnet").

1. Stale CLI flag

`deployments/docker-compose.yml` still passed `--consensus=manual-seal-3000` to the node — a flag the CLI no longer accepts at all now that it runs real Aura block production + GRANDPA finality. This was the literal crash: `error: unexpected argument '--consensus' found`. Replaced with `--alice --force-authoring` (`openinfra-dev`'s chain spec has exactly one genesis authority, Alice).

2. Dead entrypoint script

`blockchain/Dockerfile`'s `ENTRYPOINT` was never wired to `docker/authority-entrypoint.sh`. That script already existed and already correctly inserts an authority's Aura (sr25519) and GRANDPA (ed25519) keys via `key insert`, driven by `OPENINFRA_DEV_AUTHORITY_SEED` — but the Dockerfile's `ENTRYPOINT` was still the plain `openinfra-node` binary, so the script was dead code, copied into every image and never executed. Result: even with (1) fixed, the node started as an authority with a permanently empty keystore and sat at block #0 forever — confirmed directly (`--alice` alone does not populate a usable keystore for this image/polkadot-sdk pinning; only the explicit `key insert` step does).

3. No network-key auto-generation

Unlike a vanilla substrate-node-template, this build does not auto-generate a libp2p identity key on first run with a persistent `--base-path` (confirmed directly: `NetworkKeyNotFound` on an empty volume). Added an idempotent `key generate-node-key` step to `authority-entrypoint.sh`, ahead of the two `key insert` calls — a no-op on a volume that already has one (so an existing dev deployment's peer ID is unaffected), required for `make dev-clean` (or any first boot) to actually work.

`docker-compose.yml` also gains `OPENINFRA_DEV_AUTHORITY_SEED=//Alice`, `OPENINFRA_NODE_BASE_PATH`, and `OPENINFRA_NODE_CHAIN` — the entrypoint script's own required inputs, matched exactly to the `--base-path`/`--chain` CLI args the node itself runs with (a mismatch here would silently insert keys into a keystore path the running node never reads from).

Verified

  • Isolated `docker run` testing at each step (not just the final `docker compose up`): confirmed the exact `--consensus` CLI rejection, confirmed an empty keystore with (1) alone fixed, confirmed real block authoring + GRANDPA finalization (`Imported Complete workload stop and lease completion lifecycle #1..Add CI and protect the main branch #5`, `finalized Complete workload stop and lease completion lifecycle #1`) once (2) was also fixed, confirmed `NetworkKeyNotFound` on a genuinely fresh volume and its fix.
  • Full `make dev-clean && make dev-up`: all six containers (blockchain-node, control-plane, provider-agent, postgres, redis, docker-socket-proxy) reach Healthy. `system_health`/`chain_getHeader` over RPC show a live, advancing, finalizing chain (block `#0x48` and climbing within a minute of startup).
  • This is also the first time this session's local dev chain has actually run the current runtime end to end: `state_getMetadata` confirms `NetworkValidator` (pallet-network-validator, merged across ADR-013/014/015 this session) is present — every prior PR this session could only encoding-verify against this pallet, never dispatch-verify, because the long-running dev chain predated it. That limitation is resolved for local dev going forward (a fresh `make dev-clean && make dev-up` is required to pick this up, since existing `substrate-data` volumes still hold pre-Aura/GRANDPA genesis state).

🤖 Generated with Claude Code

…d bugs)

`make dev-up` was crash-looping on openinfra-blockchain-node-1. Root
cause turned out to be three separate, stacked bugs, all pre-existing
(none introduced this session) and apparently never exercised together
end to end since the project moved from manual sealing to Aura+GRANDPA
(the "local Aura GRANDPA testnet" commit, ADR-009):

1. **deployments/docker-compose.yml still passed `--consensus=manual-
   seal-3000`** to the node, a flag the CLI no longer accepts at all now
   that it runs real Aura block production + GRANDPA finality. This was
   the literal crash: `error: unexpected argument '--consensus' found`.
   Replaced with `--alice --force-authoring` (openinfra-dev's chain spec
   has exactly one genesis authority, Alice).

2. **blockchain/Dockerfile's ENTRYPOINT was never wired to
   docker/authority-entrypoint.sh.** That script already existed and
   already correctly inserts an authority's Aura (sr25519) and GRANDPA
   (ed25519) keys via `key insert`, driven by
   OPENINFRA_DEV_AUTHORITY_SEED -- but the Dockerfile's ENTRYPOINT was
   still the plain `openinfra-node` binary, so the script was dead code,
   copied into every image and never executed. Result: even with (1)
   fixed, the node started as an authority with a permanently empty
   keystore and sat at block #0 forever -- confirmed directly (`--alice`
   alone does not populate a usable keystore for this image/polkadot-sdk
   pinning; only the explicit `key insert` step does).

3. **No network-key auto-generation on a fresh volume.** Unlike a
   vanilla substrate-node-template, this build does not auto-generate a
   libp2p identity key on first run with a persistent --base-path
   (confirmed directly: NetworkKeyNotFound on an empty volume). Added an
   idempotent `key generate-node-key` step to authority-entrypoint.sh,
   ahead of the two `key insert` calls -- a no-op on a volume that
   already has one (so an existing dev deployment's peer ID is
   unaffected), required for `make dev-clean` (or any first boot) to
   actually work.

docker-compose.yml also gains OPENINFRA_DEV_AUTHORITY_SEED=//Alice,
OPENINFRA_NODE_BASE_PATH, and OPENINFRA_NODE_CHAIN -- the entrypoint
script's own required inputs, matched exactly to the --base-path/--chain
CLI args the node itself runs with (a mismatch here would silently
insert keys into a keystore path the running node never reads from).

## Verified

- Isolated `docker run` testing at each step (not just the final
  `docker compose up`): confirmed the exact `--consensus` CLI rejection,
  confirmed an empty keystore with (1) alone fixed, confirmed real block
  authoring + GRANDPA finalization (`Imported #1..#5`, `finalized #1`)
  once (2) was also fixed, confirmed NetworkKeyNotFound on a genuinely
  fresh volume and its fix.
- Full `make dev-clean && make dev-up`: all six containers (blockchain-
  node, control-plane, provider-agent, postgres, redis, docker-socket-
  proxy) reach Healthy. `system_health`/`chain_getHeader` over RPC show
  a live, advancing, finalizing chain (block #0x48 and climbing within
  a minute of startup).
- This is also the first time this session's local dev chain has
  actually run the current runtime end to end: `state_getMetadata`
  confirms `NetworkValidator` (pallet-network-validator, merged across
  ADR-013/014/015 this session) is present -- every prior PR this
  session could only encoding-verify against this pallet, never
  dispatch-verify, because the long-running dev chain predated it. That
  limitation is resolved for local dev going forward (a fresh `make
  dev-clean && make dev-up` is required to pick this up, since existing
  substrate-data volumes still hold pre-Aura/GRANDPA genesis state).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@flo2517
flo2517 merged commit 7111dcf into main Aug 7, 2026
4 checks passed
@flo2517
flo2517 deleted the fix/local-dev-chain-authority-key-provisioning branch August 7, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants