Conversation
An Autobahn bench chain needs four settings nothing defaults: no RPC followers, `evm.http_enabled: true`, `state-store.ss-enable: true`, and `allowEmptyBlocks: true`. Absence of `state-store.ss-enable` killed a benchmark chain: every historical state read panics, and `eth_sendRawTransaction` reaches that path through gas estimation. Also: the measured 3,089 TPS recipe, the field-choice table (configValues vs overrides vs consensus), why a Giga bench runs on Autobahn rather than EVM-only (sei-chain#4168, #4169), the 713714 chain ID, genesis funding, and the `--set` dotted-key limit. Measured 2026-09-15 against eng-brandon/brandon-autobahn-02 and -03. Three claims the run contradicted are corrected in place: Autobahn followers sync, load rescues a height-0 chain, and EVM-only is the only tier whose validators serve RPC. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR SummaryLow Risk Overview
Reviewed by Cursor Bugbot for commit e5957d5. Bugbot is set up for automated code reviews on this repo. Configure here. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| | "Run a load test against chain X" / "bench it" / "stress test chain X" | **PR-based bench** (see Procedure: spin up a load test). Live-fetch chain rpc per-pod URLs, substitute into the profile JSON, render Job + ConfigMap from the templates in `references/sei-load-bench.md`, open a PR against `harbor-engineering-workspace` at `engineers/<alias>/bench-<RUN_ID>/`. Merge → Flux applies → seiload runs → the uploader sidecar pushes the results to S3. When the host lists the `sei-load` MCP tools, call them for the scenario menu and the profile verdict. Fall back to the docker path when it does not, and never block on them (`references/sei-load-bench.md`, *`seiload mcp`*). | | ||
| | "Compare PR 3399 to main on sei-chain" / "bench A against B" / "diff the perf of these two commits" | **PR-based comparative bench** (see Procedure: comparative bench). Renders two ephemeral chains (each running its own seid image) + two sei-load Jobs (identical profile + duration) into a single PR. After merge, watches both chains in parallel and polls both Jobs to terminal. Then fetches both reports from S3 and surfaces a side-by-side metrics table (TPS / latency / success rate / errors). Lives at `engineers/<alias>/compare-<COMPARE_RUN_ID>/`. | | ||
| | "Spin up a giga chain" / "enable the giga executor" / "autobahn network" / "evm-only chain" / "reproduce the autobahn README on harbor" | **Tier check first** (see `references/autobahn-giga.md`). Giga executor/storage = `spec.configValues` on a normal chain. Autobahn consensus and EVM-only = `spec.consensus: {engine: Autobahn, evmOnly}` on the SeiNetwork (`seictl network apply --consensus-engine Autobahn [--evm-only]`) — **gate on the CRD carrying `spec.consensus`** (sei-k8s-controller#553) before rendering; on an older cell offer the Giga subset and ask for a controller pin advance (file it as a PLT-1259 follow-up). Never set `evm-only`, `autobahn-config-file` or the RPC/API/gRPC listener keys through `configValues`: the controller owns them and refuses the entry. | | ||
| | "Spin up a giga chain" / "enable the giga executor" / "autobahn network" / "evm-only chain" / "reproduce the autobahn README on harbor" | **Tier check first** (see `references/autobahn-giga.md`). Giga executor/storage = `spec.configValues` on a normal chain. Autobahn consensus and EVM-only = `spec.consensus: {engine: Autobahn, evmOnly}` on the SeiNetwork (`seictl network apply --consensus-engine Autobahn [--evm-only]`) — **gate on the CRD carrying `spec.consensus`** (sei-k8s-controller#553) before rendering; on an older cell offer the Giga subset and ask for a controller pin advance (file it as a PLT-1259 follow-up). Never set `evm-only`, `autobahn-config-file` or the RPC/API/gRPC listener keys through `configValues`: the controller owns them and refuses the entry. **Bench Giga on Autobahn (Tier 2), not EVM-only** — two open sei-chain defects (#4168, #4169; measured 2026-09-15). An Autobahn bench chain needs four settings: `spec.consensus.autobahn.allowEmptyBlocks: true`, `app.toml` `evm.http_enabled: true`, `app.toml` `state-store.ss-enable: true`, and **no RPC followers**. Nothing defaults them. A follower never syncs under Autobahn, so the bench targets the validators. | |
There was a problem hiding this comment.
📝 [vale] <AgenticWriting.STE-ParagraphLength> reported by reviewdog 🐶
Paragraph has more than 6 sentences. ASD-STE100 allows 6.
| | "Run a load test against chain X" / "bench it" / "stress test chain X" | **PR-based bench** (see Procedure: spin up a load test). Live-fetch chain rpc per-pod URLs, substitute into the profile JSON, render Job + ConfigMap from the templates in `references/sei-load-bench.md`, open a PR against `harbor-engineering-workspace` at `engineers/<alias>/bench-<RUN_ID>/`. Merge → Flux applies → seiload runs → the uploader sidecar pushes the results to S3. When the host lists the `sei-load` MCP tools, call them for the scenario menu and the profile verdict. Fall back to the docker path when it does not, and never block on them (`references/sei-load-bench.md`, *`seiload mcp`*). | | ||
| | "Compare PR 3399 to main on sei-chain" / "bench A against B" / "diff the perf of these two commits" | **PR-based comparative bench** (see Procedure: comparative bench). Renders two ephemeral chains (each running its own seid image) + two sei-load Jobs (identical profile + duration) into a single PR. After merge, watches both chains in parallel and polls both Jobs to terminal. Then fetches both reports from S3 and surfaces a side-by-side metrics table (TPS / latency / success rate / errors). Lives at `engineers/<alias>/compare-<COMPARE_RUN_ID>/`. | | ||
| | "Spin up a giga chain" / "enable the giga executor" / "autobahn network" / "evm-only chain" / "reproduce the autobahn README on harbor" | **Tier check first** (see `references/autobahn-giga.md`). Giga executor/storage = `spec.configValues` on a normal chain. Autobahn consensus and EVM-only = `spec.consensus: {engine: Autobahn, evmOnly}` on the SeiNetwork (`seictl network apply --consensus-engine Autobahn [--evm-only]`) — **gate on the CRD carrying `spec.consensus`** (sei-k8s-controller#553) before rendering; on an older cell offer the Giga subset and ask for a controller pin advance (file it as a PLT-1259 follow-up). Never set `evm-only`, `autobahn-config-file` or the RPC/API/gRPC listener keys through `configValues`: the controller owns them and refuses the entry. | | ||
| | "Spin up a giga chain" / "enable the giga executor" / "autobahn network" / "evm-only chain" / "reproduce the autobahn README on harbor" | **Tier check first** (see `references/autobahn-giga.md`). Giga executor/storage = `spec.configValues` on a normal chain. Autobahn consensus and EVM-only = `spec.consensus: {engine: Autobahn, evmOnly}` on the SeiNetwork (`seictl network apply --consensus-engine Autobahn [--evm-only]`) — **gate on the CRD carrying `spec.consensus`** (sei-k8s-controller#553) before rendering; on an older cell offer the Giga subset and ask for a controller pin advance (file it as a PLT-1259 follow-up). Never set `evm-only`, `autobahn-config-file` or the RPC/API/gRPC listener keys through `configValues`: the controller owns them and refuses the entry. **Bench Giga on Autobahn (Tier 2), not EVM-only** — two open sei-chain defects (#4168, #4169; measured 2026-09-15). An Autobahn bench chain needs four settings: `spec.consensus.autobahn.allowEmptyBlocks: true`, `app.toml` `evm.http_enabled: true`, `app.toml` `state-store.ss-enable: true`, and **no RPC followers**. Nothing defaults them. A follower never syncs under Autobahn, so the bench targets the validators. | |
There was a problem hiding this comment.
Descriptive sentence has 42 words. ASD-STE100 allows 25. Split it.
| - **terminal-check** (`bench:terminal-check`) — the kubectl jsonpath + grep recipe. jsonpath filter expressions do not support `||`, so the recipe iterates on the host side. | ||
| - **teardown** (`bench:teardown`) — `git rm` of `engineers/<alias>/bench-<RUN_ID>/` plus the `kustomization.yaml` entry removal. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, `Idle` on an Autobahn chain is the healthy state, not a failure. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, an Autobahn chain with `allowEmptyBlocks` off reads `Idle` — the healthy state, not a failure. That chain also never takes a first transaction, so a bench chain sets `allowEmptyBlocks: true` at create time (`references/autobahn-giga.md`). |
There was a problem hiding this comment.
Descriptive sentence has 31 words. ASD-STE100 allows 25. Split it.
| - **terminal-check** (`bench:terminal-check`) — the kubectl jsonpath + grep recipe. jsonpath filter expressions do not support `||`, so the recipe iterates on the host side. | ||
| - **teardown** (`bench:teardown`) — `git rm` of `engineers/<alias>/bench-<RUN_ID>/` plus the `kustomization.yaml` entry removal. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, `Idle` on an Autobahn chain is the healthy state, not a failure. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, an Autobahn chain with `allowEmptyBlocks` off reads `Idle` — the healthy state, not a failure. That chain also never takes a first transaction, so a bench chain sets `allowEmptyBlocks: true` at create time (`references/autobahn-giga.md`). |
There was a problem hiding this comment.
Passive voice: 'is *******'. ASD-STE100 asks for the active voice.
| - **terminal-check** (`bench:terminal-check`) — the kubectl jsonpath + grep recipe. jsonpath filter expressions do not support `||`, so the recipe iterates on the host side. | ||
| - **teardown** (`bench:teardown`) — `git rm` of `engineers/<alias>/bench-<RUN_ID>/` plus the `kustomization.yaml` entry removal. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, `Idle` on an Autobahn chain is the healthy state, not a failure. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, an Autobahn chain with `allowEmptyBlocks` off reads `Idle` — the healthy state, not a failure. That chain also never takes a first transaction, so a bench chain sets `allowEmptyBlocks: true` at create time (`references/autobahn-giga.md`). |
There was a problem hiding this comment.
Descriptive sentence has 46 words. ASD-STE100 allows 25. Split it.
| - **A seed cannot be `evmOnly`**, and **a frozen node (`fullNode.freeze`/`archive.freeze`) cannot run Autobahn** — both refused at apply, both because seid refuses the combination at start. | ||
| - **EVM-only Ready means "the 8545 listener answers", not "synced".** The `/lag_status` probe is gone with the CometBFT RPC. Treat `Running` as "process up" and prove progress with the receipt of a transaction you sent (`cast receipt <hash>`; `eth_blockNumber` is method-not-found here, see below). Idle is height 0 (`allow_empty_blocks: false`); that is healthy, not stuck. | ||
| - **Autobahn without `evmOnly` keeps the full Cosmos surface.** Readiness, `restart-seid`, followers, `seiload` receipt tracking and every other `harbor-dev` recipe behave as on CometBFT. Only the `autobahn.json` ceremony differs. Prefer this tier when the engineer says "autobahn" and does not say "evm-only". | ||
| - **EVM-only Ready means "the 8545 listener answers", not "synced".** The `/lag_status` probe is gone with the CometBFT RPC. Treat `Running` as "process up" and prove progress with the receipt of a transaction you sent (`cast receipt <hash>`; `eth_blockNumber` is method-not-found here, see below). Idle at height 0 is `allow_empty_blocks: false`. That is healthy as infrastructure, and a dead end for a bench: the chain takes no first transaction (the four settings above). |
There was a problem hiding this comment.
Passive voice: 'is gone'. ASD-STE100 asks for the active voice.
| - **Profile `chainId` is `713715`.** The EVM-only app runs the compile-time `AutobahnEVMOnlyChainID`, not a genesis value, and `seiload` signs with the envelope `chainId` (`sei-load-bench.md`, *Profile shape* — only the fee cap resolves from the chain). A Tier-1-shaped profile carrying `713714` has every transaction rejected for wrong chain ID, which looks like a dead RPC. The nightly `autobahn_evm_only.json` carries `713715`; copy it. | ||
| - **Height 0 until load.** `allow_empty_blocks: false` (the default) means a healthy idle chain sits at height 0. The controller reports this as `Ready` with `Producing=False/Idle` (`seinetwork-crd.md`): gate the bench start on `Ready`, and expect `Producing` to flip to `HeightAdvancing` only once `seiload` sends. `HeightStalled` is unreachable on this chain: the controller reports every non-advancing height as `Idle` while `allowEmptyBlocks` is off, so a chain that wedges mid-bench also reads `Idle`. Tell the two apart with `.status.observedHeight` (`height`, `time`): a height that advanced and then stopped is the chain. Height 0 under load is ambiguous — misaddressed load generator or a chain wedged before its first block — and seiload's receipt counters cannot settle it here (`trackReceipts` is off, above); read the seiload log for `eth_sendRawTransaction` errors (wrong chain ID rejects every send) and confirm one hash with `cast receipt <hash>` against the validator's `evmJsonRpc` endpoint. `HeightStalled` appears only with `allowEmptyBlocks: true`. Applies to Tier 2 as well as Tier 3. | ||
| - **No funding block.** Absent addresses read as `2^200` wei, so sender accounts need no funding — but only in EVM-only. A Tier-1 Giga chain on a vanilla image still needs funded senders (`seiload` `funding.rootKeyFile`) or a mock-balances image. | ||
| - **Height 0 until load.** `allow_empty_blocks: false` (the default) means an idle chain sits at height 0. The controller reports this as `Ready` with `Producing=False/Idle` (`seinetwork-crd.md`). That chain also never reaches height 1, so load does not rescue it. A bench chain sets `allowEmptyBlocks: true` at create time (the four settings above), reads `HeightAdvancing` before load, and treats `HeightStalled` as its wedge signal. On an `allowEmptyBlocks`-off chain, gate the bench start on `Ready`. `HeightStalled` is unreachable there: the controller reports every non-advancing height as `Idle` while `allowEmptyBlocks` is off, so a chain that wedges mid-bench also reads `Idle`. Tell the two apart with `.status.observedHeight` (`height`, `time`): a height that advanced and then stopped is the chain. Height 0 under load is ambiguous — misaddressed load generator or a chain wedged before its first block — and seiload's receipt counters cannot settle it here (`trackReceipts` is off, above); read the seiload log for `eth_sendRawTransaction` errors (wrong chain ID rejects every send) and confirm one hash with `cast receipt <hash>` against the validator's `evmJsonRpc` endpoint. `HeightStalled` appears only with `allowEmptyBlocks: true`. Applies to Tier 2 as well as Tier 3. |
There was a problem hiding this comment.
Descriptive sentence has 51 words. ASD-STE100 allows 25. Split it.
| - **Profile `chainId` is `713715`.** The EVM-only app runs the compile-time `AutobahnEVMOnlyChainID`, not a genesis value, and `seiload` signs with the envelope `chainId` (`sei-load-bench.md`, *Profile shape* — only the fee cap resolves from the chain). A Tier-1-shaped profile carrying `713714` has every transaction rejected for wrong chain ID, which looks like a dead RPC. The nightly `autobahn_evm_only.json` carries `713715`; copy it. | ||
| - **Height 0 until load.** `allow_empty_blocks: false` (the default) means a healthy idle chain sits at height 0. The controller reports this as `Ready` with `Producing=False/Idle` (`seinetwork-crd.md`): gate the bench start on `Ready`, and expect `Producing` to flip to `HeightAdvancing` only once `seiload` sends. `HeightStalled` is unreachable on this chain: the controller reports every non-advancing height as `Idle` while `allowEmptyBlocks` is off, so a chain that wedges mid-bench also reads `Idle`. Tell the two apart with `.status.observedHeight` (`height`, `time`): a height that advanced and then stopped is the chain. Height 0 under load is ambiguous — misaddressed load generator or a chain wedged before its first block — and seiload's receipt counters cannot settle it here (`trackReceipts` is off, above); read the seiload log for `eth_sendRawTransaction` errors (wrong chain ID rejects every send) and confirm one hash with `cast receipt <hash>` against the validator's `evmJsonRpc` endpoint. `HeightStalled` appears only with `allowEmptyBlocks: true`. Applies to Tier 2 as well as Tier 3. | ||
| - **No funding block.** Absent addresses read as `2^200` wei, so sender accounts need no funding — but only in EVM-only. A Tier-1 Giga chain on a vanilla image still needs funded senders (`seiload` `funding.rootKeyFile`) or a mock-balances image. | ||
| - **Height 0 until load.** `allow_empty_blocks: false` (the default) means an idle chain sits at height 0. The controller reports this as `Ready` with `Producing=False/Idle` (`seinetwork-crd.md`). That chain also never reaches height 1, so load does not rescue it. A bench chain sets `allowEmptyBlocks: true` at create time (the four settings above), reads `HeightAdvancing` before load, and treats `HeightStalled` as its wedge signal. On an `allowEmptyBlocks`-off chain, gate the bench start on `Ready`. `HeightStalled` is unreachable there: the controller reports every non-advancing height as `Idle` while `allowEmptyBlocks` is off, so a chain that wedges mid-bench also reads `Idle`. Tell the two apart with `.status.observedHeight` (`height`, `time`): a height that advanced and then stopped is the chain. Height 0 under load is ambiguous — misaddressed load generator or a chain wedged before its first block — and seiload's receipt counters cannot settle it here (`trackReceipts` is off, above); read the seiload log for `eth_sendRawTransaction` errors (wrong chain ID rejects every send) and confirm one hash with `cast receipt <hash>` against the validator's `evmJsonRpc` endpoint. `HeightStalled` appears only with `allowEmptyBlocks: true`. Applies to Tier 2 as well as Tier 3. |
There was a problem hiding this comment.
📝 [vale] <AgenticWriting.STE-ParagraphLength> reported by reviewdog 🐶
Paragraph has more than 6 sentences. ASD-STE100 allows 6.
| ## Consensus engine (`--consensus-engine`, `--evm-only`) | ||
|
|
||
| Both `network apply` and `node apply` (seictl#255). `--consensus-engine Tendermint|Autobahn` (case-insensitive, rendered canonical) sets `spec.consensus.engine`; `--evm-only` sets `spec.consensus.evmOnly: true` and is refused without `--consensus-engine Autobahn` — locally, after `--set` has run, so `--set spec.consensus.engine=Tendermint` cannot smuggle the pair past it. Both omitted leaves `spec.consensus` absent (controller default Tendermint). **Repeat both on every re-apply**, same force-ownership reason as `--node-isolation`: an apply that omits them drops `spec.consensus`, and the apiserver refuses that as a create-only change on an Autobahn object rather than reverting it. A follower joining an Autobahn chain needs `--consensus-engine Autobahn` too, or `configure-genesis` never fetches `autobahn.json`. Full recipe, controller-owned keys and EVM-only caveats: `autobahn-giga.md`. | ||
| Both `network apply` and `node apply` (seictl#255). `--consensus-engine Tendermint|Autobahn` (case-insensitive, rendered canonical) sets `spec.consensus.engine`; `--evm-only` sets `spec.consensus.evmOnly: true` and is refused without `--consensus-engine Autobahn` — locally, after `--set` has run, so `--set spec.consensus.engine=Tendermint` cannot smuggle the pair past it. Both omitted leaves `spec.consensus` absent (controller default Tendermint). **Repeat both on every re-apply**, same force-ownership reason as `--node-isolation`: an apply that omits them drops `spec.consensus`, and the apiserver refuses that as a create-only change on an Autobahn object rather than reverting it. A follower on an Autobahn chain never syncs, so do not provision one; a follower an engineer adds anyway needs `--consensus-engine Autobahn`, or `configure-genesis` never fetches `autobahn.json`. Full recipe, controller-owned keys and EVM-only caveats: `autobahn-giga.md`. |
There was a problem hiding this comment.
Passive voice: 'is refused'. ASD-STE100 asks for the active voice.
| ## Consensus engine (`--consensus-engine`, `--evm-only`) | ||
|
|
||
| Both `network apply` and `node apply` (seictl#255). `--consensus-engine Tendermint|Autobahn` (case-insensitive, rendered canonical) sets `spec.consensus.engine`; `--evm-only` sets `spec.consensus.evmOnly: true` and is refused without `--consensus-engine Autobahn` — locally, after `--set` has run, so `--set spec.consensus.engine=Tendermint` cannot smuggle the pair past it. Both omitted leaves `spec.consensus` absent (controller default Tendermint). **Repeat both on every re-apply**, same force-ownership reason as `--node-isolation`: an apply that omits them drops `spec.consensus`, and the apiserver refuses that as a create-only change on an Autobahn object rather than reverting it. A follower joining an Autobahn chain needs `--consensus-engine Autobahn` too, or `configure-genesis` never fetches `autobahn.json`. Full recipe, controller-owned keys and EVM-only caveats: `autobahn-giga.md`. | ||
| Both `network apply` and `node apply` (seictl#255). `--consensus-engine Tendermint|Autobahn` (case-insensitive, rendered canonical) sets `spec.consensus.engine`; `--evm-only` sets `spec.consensus.evmOnly: true` and is refused without `--consensus-engine Autobahn` — locally, after `--set` has run, so `--set spec.consensus.engine=Tendermint` cannot smuggle the pair past it. Both omitted leaves `spec.consensus` absent (controller default Tendermint). **Repeat both on every re-apply**, same force-ownership reason as `--node-isolation`: an apply that omits them drops `spec.consensus`, and the apiserver refuses that as a create-only change on an Autobahn object rather than reverting it. A follower on an Autobahn chain never syncs, so do not provision one; a follower an engineer adds anyway needs `--consensus-engine Autobahn`, or `configure-genesis` never fetches `autobahn.json`. Full recipe, controller-owned keys and EVM-only caveats: `autobahn-giga.md`. |
There was a problem hiding this comment.
Descriptive sentence has 35 words. ASD-STE100 allows 25. Split it.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e5957d5. Configure here.
| - **terminal-check** (`bench:terminal-check`) — the kubectl jsonpath + grep recipe. jsonpath filter expressions do not support `||`, so the recipe iterates on the host side. | ||
| - **teardown** (`bench:teardown`) — `git rm` of `engineers/<alias>/bench-<RUN_ID>/` plus the `kustomization.yaml` entry removal. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, `Idle` on an Autobahn chain is the healthy state, not a failure. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, an Autobahn chain with `allowEmptyBlocks` off reads `Idle` — the healthy state, not a failure. That chain also never takes a first transaction, so a bench chain sets `allowEmptyBlocks: true` at create time (`references/autobahn-giga.md`). |
There was a problem hiding this comment.
Follower halt contradicts Autobahn bench rule
High Severity
The Autobahn triage row now forbids RPC followers, but the load-test procedure still requires Running followers before render, and the matching halt still treats a missing follower as a blocker that needs node apply. An Autobahn Giga bench then waits on nodes the skill just refused to create, or adds followers that never sync.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit e5957d5. Configure here.
| - **terminal-check** (`bench:terminal-check`) — the kubectl jsonpath + grep recipe. jsonpath filter expressions do not support `||`, so the recipe iterates on the host side. | ||
| - **teardown** (`bench:teardown`) — `git rm` of `engineers/<alias>/bench-<RUN_ID>/` plus the `kustomization.yaml` entry removal. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, `Idle` on an Autobahn chain is the healthy state, not a failure. | ||
| Also read the chain's `Producing` condition, but only after the seiload log shows sends (funding, prewarm and ramp take minutes on a cold chain, and the Job is `Running` throughout — `references/sei-load-bench.md`): `kubectl get seinetwork <chain-id> -n eng-<alias> -o jsonpath='{.status.conditions[?(@.type=="Producing")].reason}'` must be `HeightAdvancing`. Any `False` reason other than `AwaitingFirstBlock` (the post-genesis grace window, `references/seinetwork-crd.md`) once sends are flowing ends the run as a finding, not a result. The reason string alone does not say which finding: on a Tendermint chain or Autobahn with `allowEmptyBlocks: true` a wedge reads `HeightStalled`, but on the default Autobahn/EVM-only chain (`allowEmptyBlocks` off) the controller reports **every** non-advancing height as `Idle` — a wedged chain and a misaddressed load generator look identical. Discriminate on `.status.observedHeight`, not on the seiload receipt counters (EVM-only kills `trackReceipts`, so "sends, no receipts" is every EVM-only run): `height > 0` with a `time` older than the 2-minute window means the chain produced and then stopped — the chain. `height: 0` has two candidate causes and a logged send does not pick between them (it proves submission, not acceptance): read the seiload log for `eth_sendRawTransaction` errors and confirm one hash out-of-band with `cast receipt <hash>`; errors on the right endpoint and chain ID point at the load generator (`references/autobahn-giga.md`), a clean send that never lands points at a chain wedged before its first block. Before load, an Autobahn chain with `allowEmptyBlocks` off reads `Idle` — the healthy state, not a failure. That chain also never takes a first transaction, so a bench chain sets `allowEmptyBlocks: true` at create time (`references/autobahn-giga.md`). |
There was a problem hiding this comment.
RPC recipe still excludes Autobahn validators
Medium Severity
Recipe #1 and the post-merge report step still tell the agent validators serve no EVM except on an EVM-only chain. That leftover contradicts the new Autobahn exception: evm.http_enabled makes the validator the RPC on either tier. Load and RPC queries keep targeting follower URLs that stay at height 0.
Reviewed by Cursor Bugbot for commit e5957d5. Configure here.
| }, | ||
| "source": "references/autobahn-giga.md — tier table, gate, 'Tier 2/3 — Autobahn and EVM-only via spec.consensus', 'What EVM-only changes for the rest of this skill'; SKILL.md triage row 'Spin up a giga chain'" | ||
| }, | ||
| { |
There was a problem hiding this comment.
Eval still requires EVM-only Giga render
Medium Severity
autobahn-evm-only-spec-consensus still expects a Giga plus sei-load request to render evmOnly: true. The new halt and happy-path eval say offer Tier 2 and name sei-chain#4168 and #4169 instead. The leftover eval grades that intended refusal as non-compliant, and the first Autobahn YAML example still copies the EVM-only plus Giga combo the halt forbids.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit e5957d5. Configure here.
![seidroid[bot]](https://avatars.githubusercontent.com/in/2745135?s=60&v=4){kind=small}
There was a problem hiding this comment.
Adds measured Autobahn/Giga benchmark guidance to harbor-dev — four required create-time settings, a working configValues recipe, a Tier-2-over-EVM-only recommendation — and rewrites follower, height-0 and validator-RPC claims across four references plus the evals. Blocking because the new "allowEmptyBlocks off means the chain never takes a first transaction" claim contradicts guidance left standing in the same paragraph and an eval nobody updated, and because the new ss-enable halt rule fires on any eth_sendRawTransaction failure while prescribing chain recreation; of codex's three findings I kept the halt-rule one and the eth_chainId-on-Tier-3 one, and dropped its Tier-2-preference claim because sei-chain#4169 makes any EVM-only chain fatal on restart, not only a benchmark.
Blocking
2 findings on the changed lines, as inline comments.
- The unchanged eval
bench-producing-idle-after-loadnow contradicts the doctrine this PR introduces: it posits a seiload Job sending against an EVM-only chain with allowEmptyBlocks off at height 0 and asks the agent to discriminate two candidate causes, while the new rule says such a chain can never accept a transaction, which makes recreation the only correct answer. Either update that eval alongsideautobahn-idle-height-zero-not-stuckor reconcile the rule.
Non-blocking
2 findings on the changed lines, as inline comments.
seidroid review · decision request_changes · session 351ce6a043de4cda8c2298d2c2988c97 · turn resp_claude_ec38a235ba6436177bbe3983b4964376 · item 083b725fa0375a3f90e9f417d741b360
Findings: 3 blocking | 2 non-blocking | 4 posted inline
|
|
||
| 1. **No RPC followers.** Under Autobahn, dissemination runs validator-to-validator, so a follower never syncs. The four validators reached height 4062 while both RPC followers sat at height 0 for the whole run. Provision no follower SeiNode on an Autobahn benchmark chain, and point the load profile and every RPC query at the **validators**. This holds on Tier 2 as well as Tier 3, so `sei-load-bench.md`'s "target the followers, never the validators" rule is off here. Its follower pre-flight gate is off too — that gate never passes on an Autobahn chain. | ||
| 2. **`app.toml` `evm.http_enabled: true`.** Autobahn routes through each validator's `evmrpc` listener. Without the key the chain deadlocks at height 0. | ||
| 3. **`spec.consensus.autobahn.allowEmptyBlocks: true`.** A chain that starts with no traffic needs it. Without it the chain never reaches height 1, so nothing can submit the first transaction. The field is create-only, so a chain that missed it is a new chain. |
There was a problem hiding this comment.
blocker — "Without it the chain never reaches height 1, so nothing can submit the first transaction" contradicts text this PR leaves standing: line 230 still tells the agent that height 0 under load is ambiguous between a misaddressed load generator and a wedged chain, and line 229 still says the nightly autobahn_evm_only.json profile benches an EVM-only chain (allowEmptyBlocks defaults off) with inclusion confirmed by cast receipt. It also shares its evidence with setting 4 — both are diagnosed from the same height-0, sends-failing chain, so one may be a misattribution of the other's cause. Either state the mechanism by which a height-0 Autobahn node refuses a transaction and fix the three places that assume otherwise, or narrow the claim.
| - Follower requested on an EVM-only chain → ask what it is for; the validators serve `:8545` and a follower serves nothing the bench reads. | ||
| - Follower requested on an Autobahn chain, either tier → refuse it and say why: under Autobahn a follower never syncs, and the validators are the RPC. | ||
| - Autobahn chain sitting at height 0 with `allowEmptyBlocks` off → the chain cannot take a first transaction. Recreate it with `allowEmptyBlocks: true`; the field is create-only. Do not send load and wait. | ||
| - `eth_sendRawTransaction` fails, or a validator logs `unable to load historical state with SS disabled` → `state-store.ss-enable` is false and the chain accepts no transactions. Recreate with the key set. Do not promise that a day-2 `configValues` edit repairs it. Storage mode is a create-time decision for a SeiNetwork (Tier 1 rules above), and the edit restarts every validator at once. |
There was a problem hiding this comment.
blocker — The antecedent is a bare eth_sendRawTransaction failure, but this same file gives two other causes of exactly that symptom — a copied chain ID (line 242, "invalidates every signature") and unfunded senders on a genesis chain (line 231) — and the prescribed action is to destroy and recreate the chain. Condition the rule on the unable to load historical state with SS disabled log line, or on having ruled the other causes out first. (codex raised this.)
| Before writing any manifest: | ||
|
|
||
| 1. **At least one rpc follower SeiNode is present and `Running`.** `seictl node list -n eng-<alias> -l sei.io/seinetwork=<chain-id>,sei.io/role=node -o json | jq -r '[.items[].status.phase]'` shows at least one `Running` follower. Halt and ask the engineer to wait if not — per-follower URLs are not published otherwise (a SeiNode has no `Ready` phase — terminal is `Running`). | ||
| 1. **At least one rpc follower SeiNode shows phase `Running`.** `seictl node list -n eng-<alias> -l sei.io/seinetwork=<chain-id>,sei.io/role=node -o json | jq -r '[.items[].status.phase]'` shows at least one `Running` follower. Halt and ask the engineer to wait if not — per-follower URLs are not published otherwise (a SeiNode has no `Ready` phase — `Running` is the terminal one). **Skip this gate on an Autobahn chain:** a follower never syncs there, so the validators are the RPC. Skip gate 2 as well, and read the endpoints off the network (`references/autobahn-giga.md` → *Four settings an Autobahn bench chain must carry*). |
There was a problem hiding this comment.
suggestion — Skipping gates 1 and 2 leaves the agent with no documented endpoint source for an Autobahn chain: the section pointed at, Four settings an Autobahn bench chain must carry, contains no endpoint recipe, and the only one that does (status.endpoints.nodes[].evmJsonRpc) sits under What EVM-only changes for the rest of this skill, explicitly scoped to Tier 3. This file's Substrate facts, Inputs table and __RPC_ENDPOINTS__ recipe also still read the follower node list, so the skip is incomplete.
| - Autobahn chain sitting at height 0 with `allowEmptyBlocks` off → the chain cannot take a first transaction. Recreate it with `allowEmptyBlocks: true`; the field is create-only. Do not send load and wait. | ||
| - `eth_sendRawTransaction` fails, or a validator logs `unable to load historical state with SS disabled` → `state-store.ss-enable` is false and the chain accepts no transactions. Recreate with the key set. Do not promise that a day-2 `configValues` edit repairs it. Storage mode is a create-time decision for a SeiNetwork (Tier 1 rules above), and the edit restarts every validator at once. | ||
| - EVM-only requested for a Giga benchmark → offer Tier 2 and name sei-chain#4168 and sei-chain#4169. | ||
| - Load-profile `chainId` copied from another chain or another tier → read `eth_chainId` off a validator first. A wrong chain ID invalidates every signature. |
There was a problem hiding this comment.
suggestion — This halt condition is unscoped, but on Tier 3 eth_chainId returns method-not-found (line 227, the two-method RPC surface) and line 229 tells the agent to copy 713715 rather than probe. Qualify it to Tier 2. (codex raised this.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An Autobahn bench chain needs four settings nothing defaults: no RPC followers (they never sync),
app.tomlevm.http_enabled: true,app.tomlstate-store.ss-enable: true, andallowEmptyBlocks: true. Missingstate-store.ss-enablekilled a benchmark chain — every historical state read panics, andeth_sendRawTransactionreaches that path through gas estimation.Also lands the measured 3,089 TPS recipe, the field-choice table (
configValuesvsoverridesvsconsensus), why a Giga bench runs on Autobahn rather than EVM-only (sei-chain#4168, #4169), the 713714 chain ID, genesis funding, and the--setdotted-key limit.Measured 2026-09-15 against
eng-brandon/brandon-autobahn-02and-03. Three claims the run contradicted are corrected in place acrossSKILL.md,seinetwork-crd.md,sei-load-bench.mdandseictl-cli.md: Autobahn followers sync, load rescues a height-0 chain, and EVM-only is the only tier whose validators serve RPC. One eval criterion changed for the same reason, plus one new happy-path eval.Gates:
vale0 errors on every touched file (+1 suggestion-level finding, -1 warning vs baseline);verify-referencesclean;skill-package-checksno block failure.🤖 Generated with Claude Code