diff --git a/.ci-siblings b/.ci-siblings new file mode 100644 index 00000000..d65fa3ae --- /dev/null +++ b/.ci-siblings @@ -0,0 +1,41 @@ +# Repos the documentation suite needs checked out BESIDE it on the CI venue. +# +# Declaring a sibling here is what makes its cross-repo guards actually run. An +# UNdeclared sibling is not a neutral omission: every guard written against it +# resolves with existsSync, calls this.skip(), and the gate still prints PASS, +# so the run reports green having exercised none of them. Measured 2026-09-13, +# with no roster the venue ran 397 passing / 60 skipped; with every repo below +# beside it the same tree runs 485 tests, and the only skips left are the three +# placeholder vector gaps in test/vectors.test.js and the LikeC4 topology model, +# which lives in the platform workspace rather than in any sibling repo. +# +# ci-dispatch.sh ships each of these at the pushed branch (falling back to its +# origin/master, then local HEAD) and REFUSES the push if one is not a checkout +# beside this repo, so a declared sibling can never be silently absent. + +# The activation registry behind the generated flag-day page, the action +# activation model, XBRIDGE registration, supply/settlement/dividend source +# facts, fee and batch limits, and the schema-table coverage of database.md. +xchain-indexer +# The consensus wall-clock budget the VM pages quote. +xchain-vm +# Endpoint counts, the ExplorerStatus schema, the REST error-code registry, +# contract-state proof availability and the regtest tip-age escape hatch. +xchain-explorer +# The multisign capacity figure the corpus pages publish. +xchain-encoder +# The SDK action surface and the canonical batch limits. +xchain-sdk +# The signer surface and release-key fingerprint channel one (SECURITY.md). +xchain-wallet +# The docs-site slug rule and release-key fingerprint channel two. +xchain-websites +# Schema-table coverage of the decoder and hub database pages. +xchain-decoder +xchain-hub +# The environment-variable coverage fleet floor, which applies only when every +# service component it scans is present (lib/env-var-doc-coverage.js COMPONENTS). +xchain-node +xchain-regtest-miner +xchain-sync +xchain-utxo-tracker diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b48f36cc..458499c2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,9 +16,10 @@ name: CI # A drift in any of those is a wrong answer given to an implementer, and until # now nothing ran on a push to catch it. # -# Hermetic: no sibling checkouts, no services, no secrets. +# No services, no secrets, and exactly one sibling checkout (xchain-indexer, +# read as text by the flag-day literals suite; see that step). # -# The hermeticity has a cost worth naming rather than rediscovering: +# The near-hermeticity has a cost worth naming rather than rediscovering: # a test that needs a SIBLING repo cannot run here, and node:test prints a skip # as a pass. test/schema-table-coverage.test.js is the one that bit: its # cross-repo half resolves xchain-decoder/-indexer/-hub beside this checkout, @@ -62,6 +63,33 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 + with: + path: xchain-documentation + + # test/flag-day-literals.test.js reads xchain-indexer/src/protocol_changes.js + # as a SIBLING checkout (../xchain-indexer relative to this repo's root, + # per bin/generate-flag-days.js) and quietly SKIPS its drift and + # completeness checks when that sibling is absent, which is exactly what + # every push here had been doing: the generated protocol/flag-days.md page + # could rot against the indexer's registry with this workflow green the + # whole way. Checked out beside xchain-documentation, not nested, so the + # generator's '../xchain-indexer' resolution finds it. + # + # Ref: the release branch on a release head (a PR from release/*), the + # pushed/targeted branch otherwise. Matches the ref logic xchain-e2e-test's + # CI already uses for the same sibling. The indexer repo is public, so the + # default token reads it and this repo keeps holding no secrets. + # + # Only ONE sibling is checked out. The suites that read a sibling skip the + # ones that are absent, and the env-var coverage suite applies its fleet + # floor only when every sibling is present, so a partial set is judged on + # what is here rather than accused of a broken scanner. + - name: Check out xchain-indexer (flag-day registry read by the literals suite) + uses: actions/checkout@v4 + with: + repository: XChain-Platform/xchain-indexer + ref: ${{ startsWith(github.head_ref, 'release/') && github.head_ref || (github.ref == 'refs/heads/master' && 'master' || 'develop') }} + path: xchain-indexer # Node 22 exactly: the platform's suites require it, and on some repos an # off-version run degrades into silent skips rather than failing. @@ -73,8 +101,14 @@ jobs: # mathjs is a devDependency here, pinned to the same exact 15.2.0 the # indexer, explorer, sdk and hub carry, because the reference # implementation must compute quorum weights the way the fleet does. + # Only this tree needs installing: the flag-day generator READS the + # indexer's registry as text (see the file-header note in + # bin/generate-flag-days.js) rather than requiring it, so the sibling + # checkout above needs no install of its own. - name: Install dependencies + working-directory: xchain-documentation run: if [ -f package-lock.json ]; then npm ci; else npm install; fi - name: Documentation conformance suite + working-directory: xchain-documentation run: npm test diff --git a/CHANGELOG.md b/CHANGELOG.md index 6399eb43..57b15e1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.19.0] - 2026-09-16 + +### Added +- Published `ADMIT_MARGIN_BLOCKS`, `ADMIT_MIN_FUTURE_BLOCKS` and `ADMIT_MAX_FUTURE_BLOCKS`, the canonical admission-height margins that let a mirror barrier bind rows by block height instead of by the block's timestamp. +- Published `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION`, the paired producer and consumer flag-day maps keyed by coin and network, inert everywhere pending their sizing at the cut. +- Published `ANCHOR_ATTEST_ARRIVAL_MARGIN_S` (64800 s) and `ANCHOR_ATTEST_BARRIER_ACTIVATION`, the maturity-horizon bound that keeps the anchor-attest barrier opening no later than it does today. +- The XBRIDGE action and the cross-chain bridge protocol are documented. +- The bridge settle pass's origin-indexer reads are documented and the computed-read baseline is raised for the coin-keyed lookups. +- The four bridge helper modules are counted in the e2e architecture and README pages. +- Published the `TRAIN_ACTIVATION` 0.19.0 row and the per-chain testnet bridge heights this train arms. + +### Changed +- The bridge activation is documented as keyed per chain. +- Testnet activations are documented as arming at the tip read at the cut. +- The wallet's cross-chain order form is documented. +- Restructured under the platform code-structure standard (feature directories, snake_case files, split test suites, restored comments). + +### Fixed +- CI checks out xchain-indexer beside this repo so the flag-day literals suite runs on GitHub instead of skipping, and the env-var coverage fleet floor applies only when every sibling is present. +- The node and explorer configuration pages document `XCHAIN_NODE_ALLOW_NO_DOGE_READ` and `EXPLORER_FEDERATION_READ_KEY`. +- The undo-window and halted-state pages state the per-network table (every testnet 120 blocks) and the halt's `/status` fields and reset path, replacing the stale LTC value. +- The Pi 5 firmware's `cgroup_disable=memory` clause, the page-cache disk model and the node operations exit-status section are documented. + ## [0.18.0] - 2026-09-11 ### Added @@ -23,6 +46,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The SDK websocket page matches the producer roster, adding the mempool, expiry, bet, attestation and xcall event types. - The explorer API page describes the merged `hub_status` on the validators endpoint. +### Fixed +- Corrected the bare-multisig capacity figure: 60 bytes per output (two 32-byte key slots), not ~61 bytes per key. +- The indexer determinism and replay claims now name the local Hub DB mirror they read during block processing. +- The explorer pages state the hub-mirror schema the explorer creates and writes under `self_sync`, instead of claiming it never writes to any database. +- The hub configuration, architecture and database pages distinguish hub-local suspension from equivocation slashing: missed rounds and price deviation leave on-chain stake untouched. +- Published Token Information Standard v1.1.1, which relaxes the media requirement to `type` plus at least one of `data` or `data_ref` so the fully on-chain form the standard recommends validates; v1.1.0 and v1.0.0 stay frozen as published. +- The TIS media `type` row states the per-array vocabulary the schema enums pin (`images` display role, `audio` and `video` containers, `files` category) instead of calling it a MIME type. +- The gated-pack publishing steps require one `FILE` per transaction: a `BATCH` hands one `rawData` payload to every sub-command, so a second `FILE` is recorded valid carrying the first file's ciphertext. + ## [0.17.0] - 2026-09-10 ### Added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7167914d..6a611333 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,7 +27,7 @@ xchain-documentation/ ├── developer-guide/ tutorials: build tokens, dispensers, query data, integrate, testing ├── ai-agents/ building AI agents with MCP and bounded wallets ├── user-guide/ capabilities, use cases, FAQ (no code required) -├── protocol/ 37 ACTION definitions, Token Information Standard, schemas +├── protocol/ 38 ACTION definitions, Token Information Standard, schemas ├── operations/ deployment, Docker, monitoring, upgrades, troubleshooting ├── legal/ licensing, commercial license, trademark, contributor agreement ├── blockchains.md supported chains, adding new blockchains diff --git a/README.md b/README.md index 60cd1d5c..02bfb11b 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ # XChain Platform -A blockchain-agnostic token protocol currently running on Bitcoin, Litecoin, and Dogecoin. Create, transfer, trade, and manage tokens, run smart contracts that can call out to AI models and the web, and stake for validation using 37 ACTION commands, 36 of them embedded directly in standard blockchain transactions. No sidechains, no bridges, no separate consensus mechanism. The platform includes a built-in DEX, a sandboxed JavaScript virtual machine for on-chain smart contracts, cross-chain swap support, and **cryptographically secure token-gated file publishing**: encrypt single files or multi-file packs on-chain so only holders of a specific token can decrypt them (see [Token-Gated Content](./protocol/token-gated-content.md)). The platform can be extended to run on any Bitcoin-compatible blockchain, including private blockchains for enterprise deployments. +A blockchain-agnostic token protocol currently running on Bitcoin, Litecoin, and Dogecoin. Create, transfer, trade, and manage tokens, run smart contracts that can call out to AI models and the web, and stake for validation using 38 ACTION commands, 37 of them embedded directly in standard blockchain transactions. The `XBRIDGE` action moves supply between XChain's own ledgers under lock-and-mint / burn-and-release custody of the hub federation, with no separate consensus mechanism and no external bridge. The platform includes a built-in DEX, a sandboxed JavaScript virtual machine for on-chain smart contracts, cross-chain swap support, and **cryptographically secure token-gated file publishing**: encrypt single files or multi-file packs on-chain so only holders of a specific token can decrypt them (see [Token-Gated Content](./protocol/token-gated-content.md)). The platform can be extended to run on any Bitcoin-compatible blockchain, including private blockchains for enterprise deployments. **New here?** Start with [What is XChain?](./getting-started/what-is-xchain.md) or jump straight to the [Developer Quickstart](./getting-started/quickstart-developer.md). diff --git a/architecture/component-map.md b/architecture/component-map.md index 21ba53a1..e348de03 100644 --- a/architecture/component-map.md +++ b/architecture/component-map.md @@ -51,7 +51,7 @@ See [`../components/decoder/`](../components/decoder/) for full documentation. | | | |---|---| | **Purpose** | Reads decoded ACTIONs from the Decoder DB, validates them, executes business logic, writes final state | -| **Inputs** | Decoder MariaDB (SQL polling every 5 seconds); local Hub DB (cross-chain price data); inbound JSON-RPC `pushvalidatorrewards` from xchain-hub | +| **Inputs** | Decoder MariaDB (SQL polling every 5 seconds); local Hub DB (cross-chain price data); inbound JSON-RPC `pushvalidatorrewards` from xchain-hub (retired for new anchor rewards, which each indexer now derives from the on-chain ANCHOR bytes) | | **Outputs** | Indexer MariaDB (`XChain_{CHAIN}_{NETWORK}_Indexer`); outbound JSON-RPC pushes to xchain-hub (`pushchaintip`, `pushpriceround`, `pushoracleprice`) | | **Storage** | Three database connections: Decoder DB (read), Indexer DB (read/write, 100+ tables), local Hub DB (read, synced from xchain-hub) | | **Communication** | Outbound SQL reads from Decoder DB and local Hub DB; outbound HTTP/WebSocket to xchain-hub; inbound JSON-RPC API for hub pushes | @@ -78,10 +78,10 @@ See [`../components/indexer/`](../components/indexer/) for full documentation. | | | |---|---| | **Purpose** | Serves REST endpoints, JSON-RPC 2.0, and a web UI over the Indexer DB | -| **Inputs** | Indexer MariaDB (direct SQL reads); xchain-hub (config sync every 60s) | +| **Inputs** | Indexer MariaDB (direct SQL reads); Decoder MariaDB (raw transaction lookups); xchain-hub (config sync every 60s, plus the hub-mirror snapshot and live feed under `self_sync`) | | **Outputs** | HTTP responses (REST, JSON-RPC, HTML) | -| **Storage** | None (stateless read layer) | -| **Communication** | Inbound HTTP from clients; outbound SQL to Indexer DB; outbound JSON-RPC to xchain-hub | +| **Storage** | A hub-mirror schema it owns (created and written under `"self_sync": true`); otherwise a read layer over indexed state | +| **Communication** | Inbound HTTP from clients; outbound SQL to Indexer and Decoder DBs; DDL and row writes to its own hub-mirror DB; outbound JSON-RPC to xchain-hub | Key technical details: @@ -143,7 +143,7 @@ These services support the construction and submission of XChain transactions. Key technical details: -- With `encoding` omitted, selects between `OP_RETURN` (≤80 bytes/output, 76 bytes user data, 1 tx) and `P2SH` (476 bytes/chunk, 2 tx) by payload size. `MULTISIGN` (~61 bytes/key, 1 tx), `P2WSH` (476 bytes/chunk up to the 8,192-byte compiled-payload ceiling, 2 tx) and `TAPROOT` (the envelope, up to 390,000 bytes in one tapscript witness, 2 tx, segwit chains only) are never reached by that size fallback; they are used only when explicitly requested, or when `encoding: AUTO` opts into smallest-footprint selection. +- With `encoding` omitted, selects between `OP_RETURN` (≤80 bytes/output, 76 bytes user data, 1 tx) and `P2SH` (476 bytes/chunk, 2 tx) by payload size. `MULTISIGN` (60 bytes/output, 1 tx), `P2WSH` (476 bytes/chunk up to the 8,192-byte compiled-payload ceiling, 2 tx) and `TAPROOT` (the envelope, up to 390,000 bytes in one tapscript witness, 2 tx, segwit chains only) are never reached by that size fallback; they are used only when explicitly requested, or when `encoding: AUTO` opts into smallest-footprint selection. - P2SH and P2WSH use a two-transaction pattern: fund tx commits funds to a script; reveal tx spends it, embedding the data in the unlocking script. TAPROOT uses a commit/reveal pair returned together from one call. - Obfuscates payloads with AES-128-CTR. Key and IV are derived from the first input's txid, deterministic and reversible by any party with the txid. - Available as a Node.js JSON-RPC service and as a browser bundle via webpack. @@ -167,7 +167,7 @@ Key technical details: - LevelDB key schema uses single-character prefixes: `B`=block, `T`=transaction, `I`=input, `O`=output, `H`/`J`=address hints. - Processes blocks in batches of up to 200 (flush may trigger earlier under heap pressure), writing each batch atomically to LevelDB. -- Maintains a per-chain undo window (BTC: 12 / LTC: 48 / DOGE: 120 blocks, overridable via XCHAIN_UNDO_BLOCKS_) to support chain reorganization rollback. +- Maintains a per-chain, per-network undo window (mainnet and regtest: BTC 12 / LTC 120 / DOGE 120 blocks; testnet: 120 blocks for every coin; overridable via XCHAIN_UNDO_BLOCKS_) to support chain reorganization rollback. - Tracks the mempool for real-time unconfirmed UTXO state. - Supports bootstrap from tar archives to avoid re-indexing from genesis. - Outputs are indexed by scriptPubKey hash, enabling efficient address lookups. @@ -189,7 +189,7 @@ See [`../components/utxo-tracker/`](../components/utxo-tracker/) for full docume Key technical details: -- Exposes 31 action construction methods (one per developer-invocable ACTION type) and 118 explorer query wrappers. +- Exposes 32 action construction methods (one per developer-invocable ACTION type) and 118 explorer query wrappers. - Batch builder allows multiple actions to be combined into a single `BATCH` action string. - Discovers service endpoints via xchain-hub. - Implements retry with exponential backoff and connection pooling for all outbound calls. diff --git a/architecture/data-pipeline.md b/architecture/data-pipeline.md index 9fafedfb..a9b3833f 100644 --- a/architecture/data-pipeline.md +++ b/architecture/data-pipeline.md @@ -35,14 +35,14 @@ The SDK calls the encoder's JSON-RPC API, passing the ACTION string, the sender' 1. **Selects a format**. With `encoding` omitted the choice is by payload length alone (`OP_RETURN`, else `P2SH`); the other lanes are requested explicitly, or via `encoding: AUTO`, which picks the cheapest lane the network and signer support: - `OP_RETURN`: up to 80 bytes per output (76 bytes user data + 4-byte XCHN prefix), single transaction - - `multisig`: up to ~61 bytes per key, single transaction + - `multisig`: 60 bytes per output (two 32-byte key slots carry one chunk), single transaction - `P2SH`: 476 bytes per chunk, split across as many outputs as needed up to the 8,192-byte compiled-payload ceiling, two-transaction pattern - `P2WSH`: the same 476-byte chunking up to the 8,192-byte compiled-payload ceiling, two-transaction pattern - `TAPROOT`: the envelope, up to 390,000 payload bytes in one tapscript witness, commit/reveal pair returned together; segwit chains only, and requested explicitly or via `encoding: AUTO` rather than reached by the size fallback -2. **Obfuscates the payload** using AES-128-CTR. The key is the first 16 hex characters of the first input's txid; the IV is the next 16. This is deterministic (any observer with the txid can reverse it) but it filters casual blockchain scanners. +2. **Prepends the magic prefix** `XCHN` (4 bytes) to the ACTION string, so the decoder can identify an XChain payload once it has deobfuscated the output. The marker sits inside the obfuscated bytes, never ahead of them. -3. **Prepends the magic prefix** `XCHN` (4 bytes) after obfuscation, so the decoder can identify XChain payloads. +3. **Obfuscates the prefixed payload** using AES-128-CTR. The key is the first 16 hex characters of the first input's txid; the IV is the next 16. This is deterministic (any observer with the txid can reverse it) but it filters casual blockchain scanners. 4. **Returns an unsigned PSBT** (Partially Signed Bitcoin Transaction). For two-transaction formats, the encoder returns both PSBTs in sequence: a funding transaction and a reveal transaction. @@ -102,7 +102,7 @@ The indexer polls the Decoder DB every 5 seconds. When it finds a new decoded ac 7. **Detects reorgs** by monitoring the Decoder DB for block hash changes. On reorg, the indexer rolls back across 80+ tables in a single transaction. -The indexer is deterministic: given the same Decoder DB contents, it will always produce the same Indexer DB state. There is no external I/O during block processing. +The indexer is deterministic: given the same Decoder DB contents and the same local Hub DB mirror rows, it will always produce the same Indexer DB state. There is no network I/O during block processing; the hub-mirrored cross-chain tables are read locally over SQL, never fetched from the hub mid-block. --- @@ -198,7 +198,7 @@ Each seam in the pipeline uses polling rather than push notifications or a messa - **Simplicity**: no broker infrastructure (Kafka, RabbitMQ, Redis Pub/Sub) to deploy, monitor, or tune. Each service can be started, stopped, or restarted independently without affecting others. - **Auditability**: the Decoder DB is a complete, queryable record of every raw ACTION the decoder has ever seen. The Indexer DB is a complete record of every validated state transition. Both are inspectable with standard SQL tools. -- **Determinism**: because the indexer only reads from the Decoder DB and applies deterministic logic, running it again from scratch against the same Decoder DB always produces identical output. +- **Determinism**: because the indexer reads only local databases, the Decoder DB and the Hub DB mirror, and applies deterministic logic, running it again from scratch against the same Decoder DB and an equivalent mirror always produces identical output. The cost is latency: a transaction confirmed in a block will not appear in the explorer until the decoder poll finds the block (~seconds), the indexer poll picks up the decoded row (~5 seconds), and the explorer serves the next query. In practice this is 10–30 seconds of additional latency beyond block confirmation, which is acceptable for a protocol where block times are measured in minutes. @@ -218,13 +218,13 @@ In regtest, all services point to `Regtest` network databases (`XChain_BTC_Regte ## Determinism -The indexer's output is fully determined by its input (the Decoder DB) and its code. There is no randomness, no external API calls during block processing, and no dependency on wall-clock time beyond block heights. This means: +The indexer's output is fully determined by its inputs and its code. Those inputs are the Decoder DB and the local Hub DB mirror, the read-only copy of the cross-chain tables described in [Database Design](database-design.md); block processing consults the mirror for fee validation, FIAT dispenser settlement, and VM oracle queries. There is no randomness, no external API calls during block processing (the mirror is a local SQL read, not a hub round-trip), and no dependency on wall-clock time beyond block heights. This means: -- **Replay**: destroy the Indexer DB, run the indexer from block 0 against the existing Decoder DB, and the result is bit-for-bit identical. -- **Verification**: multiple independent indexer instances reading the same Decoder DB will converge to the same state. +- **Replay**: destroy the Indexer DB, run the indexer from block 0 against the existing Decoder DB and an equivalent Hub DB mirror, and the result is bit-for-bit identical. +- **Verification**: multiple independent indexer instances reading the same Decoder DB and the same hub-mirrored rows will converge to the same state. - **Auditability**: a disputed balance or token state can be traced back through ledger entries to the exact action and block that caused it. -The Decoder DB itself is rebuilt from the blockchain: destroy the Decoder DB, run the decoder from block 0, and it re-derives the same rows from on-chain data. Together, the two-stage pipeline means the full indexer state is reproducible from the raw blockchain alone. +Both inputs are themselves chain-derived. The Decoder DB is rebuilt from the blockchain: destroy it, run the decoder from block 0, and it re-derives the same rows from on-chain data. The hub-mirrored rows are chain-derived too: prices arrive as PRICE v0 and v1 actions, and the validator infrastructure tables (`stakes`, `delegations`, `validator_rewards`) are synced from BTC indexer state, both aggregated across chains by the hub, with the mirror rebuilt from the hub's snapshot ([Database Design](database-design.md)). So the full indexer state is reproducible from the chains, though not from one chain's Decoder DB in isolation. --- diff --git a/architecture/database-design.md b/architecture/database-design.md index 2882eb37..df4471a4 100644 --- a/architecture/database-design.md +++ b/architecture/database-design.md @@ -152,7 +152,7 @@ Stores the full UTXO set of the monitored coin node. Key schema uses single-char | `O` | Output records (txid:vout → value, scriptPubKey) | | `H` / `J` | Address hints (scriptPubKey hash → txids) | -Writes are batched in groups of up to 200 blocks (flush may trigger earlier under heap pressure). A per-chain undo window (BTC: 12 / LTC: 48 / DOGE: 120 blocks) is retained to support reorg rollback. +Writes are batched in groups of up to 200 blocks (flush may trigger earlier under heap pressure). A per-chain, per-network undo window (mainnet and regtest: BTC 12 / LTC 120 / DOGE 120 blocks; testnet: 120 blocks for every coin) is retained to support reorg rollback. ### xchain-hub diff --git a/bin/complete_run_reporter.js b/bin/complete_run_reporter.js new file mode 100644 index 00000000..8e25ec93 --- /dev/null +++ b/bin/complete_run_reporter.js @@ -0,0 +1,105 @@ +'use strict'; + +// Refuses a node --test run in which a file's child exited before its event +// stream was whole: the runner trusts exit codes alone, so a lost stream tail +// otherwise grades green with the tests in it never counted. Node 22.10+. + +const fs = require('node:fs'); +const path = require('node:path'); + +const fileKey = (file) => (file ? path.resolve(file) : null); + +// A file the runner loaded that never touched node:test (a helper living +// under test/) emits nothing and is graded by the parent's placeholder pass; +// that is the one shape in which silence from the child is not a lost stream. +const LOADS_NODE_TEST = /['"]node:test['"]|require\(\s*['"]test['"]\s*\)|from\s+['"]test['"]/; +const loadsNodeTest = (file, readFile) => { + try { return LOADS_NODE_TEST.test(readFile(file, 'utf8')); } catch { return true; } +}; + +// Walks a test-event stream and names what is missing: the run summary, a +// file's own summary (the last thing a child writes), or counts that differ. +// Returns { problems: string[], files: number, total: number|null }. +async function audit(events, readFile = fs.readFileSync) { + const files = new Map(); + let total = null; + + for await (const { type, data } of events) { + if (type === 'test:summary') { + if (!data.file) { total = data; continue; } + fileEntry(files, data.file).summary = data; + continue; + } + if (!data || !data.file) continue; + const entry = fileEntry(files, data.file); + // The parent's own bookkeeping for the file (enqueue, complete) is + // named after the file; the last event worth naming is the child's. + if (!(data.name && fileKey(data.name) === fileKey(data.file))) { + entry.lastEvent = `${type} ${data.name || ''}`.trim(); + entry.childEvents += 1; + } + if ((type === 'test:pass' || type === 'test:fail') && data.details?.type !== 'suite') { + entry.seen += 1; + } + } + + const problems = []; + if (total === null) { + problems.push('the runner ended without its cumulative summary (Node 22.10 or later delivers one)'); + } + + let accounted = 0; + for (const [file, entry] of files) { + const rel = path.relative(process.cwd(), file) || file; + if (!entry.summary && entry.childEvents === 0 && !loadsNodeTest(file, readFile)) { + accounted += entry.seen; + continue; + } + if (!entry.summary && entry.childEvents === 0) { + problems.push(`${rel} loads node:test but delivered no events at all: it exited before its ` + + 'first event was written (a helper that registers no tests belongs outside the test glob)'); + continue; + } + if (!entry.summary) { + problems.push(`${rel} ended without reporting its summary after ${entry.seen} of its tests ` + + `were seen (last event: ${entry.lastEvent})`); + continue; + } + const reported = entry.summary.counts.tests; + accounted += reported; + if (reported !== entry.seen) { + problems.push(`${rel} reported ${reported} tests but the runner saw ${entry.seen}`); + } + } + + if (total !== null && problems.length === 0 && accounted !== total.counts.tests) { + problems.push(`the files that reported account for ${accounted} tests but the runner counted ${total.counts.tests}`); + } + + return { problems, files: files.size, total: total ? total.counts.tests : null }; +} + +function fileEntry(files, file) { + const key = fileKey(file); + let entry = files.get(key); + if (!entry) { + entry = { seen: 0, childEvents: 0, summary: null, lastEvent: 'none' }; + files.set(key, entry); + } + return entry; +} + +// The reporter node --test loads: silent on a whole run, otherwise one line +// per problem to its destination and a failing exit code for the run. +async function* completeRunReporter(source) { + const { problems } = await audit(source); + if (problems.length === 0) return; + process.exitCode = 1; + yield 'test run incomplete, refusing to grade it green:\n'; + for (const problem of problems) { + yield ` ${problem}\n`; + } +} + +module.exports = completeRunReporter; +module.exports.audit = audit; diff --git a/bin/generate-flag-days.js b/bin/generate-flag-days.js index c60bc9f1..cc515b2c 100644 --- a/bin/generate-flag-days.js +++ b/bin/generate-flag-days.js @@ -27,17 +27,25 @@ * else in the tree carries a value that can rot. * * WHERE THE VALUES COME FROM. `xchain-indexer/src/protocol_changes.js` is the - * registry: `addChange(name, version, mainnet_time, ...)` plus the handful of - * gates declared as `const NAME_MAINNET_TIME`. Three more time-keyed gates ship - * as standalone sibling modules in the same directory (they are registered next - * to the query they gate rather than in the registry), so those are read too; - * leaving them out would publish an inventory that calls itself complete and is - * not. - * - * THE SOURCE IS READ AS TEXT, NOT REQUIRED. `require('protocol_changes.js')` - * pulls in the indexer's config and database layer, and this must run in a - * documentation checkout with neither. Other tooling in this monorepo reads - * source files as text for the same reason. + * registry's entry, and its rows live in the part files under + * `src/protocol_changes/`: the time table as array rows + * `['NAME', 'X.Y.Z', mainnet_time, ...]` in `changes_*.js` (the older + * `addChange(name, version, mainnet_time, ...)` call shape is still read, for + * a tree that predates the split), plus the handful of gates declared as + * `const NAME_MAINNET_TIME` in `flag_times*.js`. Three more time-keyed gates + * ship as standalone sibling modules in `src/` (they are registered next to + * the query they gate rather than in the registry), so those are read too; + * leaving them out would publish an inventory that calls itself complete and + * is not. + * + * THE SOURCE IS READ AS TEXT, NOT REQUIRED. What this page asserts about the + * registry is asserted about its LITERALS: a retired row parked in a comment + * must not be published, a separator-formatted or arithmetic time slot must be + * refused rather than guessed at, and a declaration in a shape the parse does + * not know must be loud. A required module has already resolved all of that + * away, so the fixtures under test/flag-day-literals.test.js could not drive + * it. The entry plus every part is read as one text (lib/indexer-source.js), + * and every refusal names the part file and line it came from. * * A standalone documentation clone has no sibling indexer. The generated page * is COMMITTED, so such a clone still reads correct values; only regeneration @@ -52,12 +60,31 @@ const fs = require('node:fs'); const path = require('node:path'); +const { locatedModuleSource } = require('../lib/indexer-source.js'); +const { stripComments } = require('../lib/env-var-doc-coverage.js'); const DOC_ROOT = path.resolve(__dirname, '..'); const INDEXER_SRC = path.resolve(DOC_ROOT, '../xchain-indexer/src'); const REGISTRY = path.join(INDEXER_SRC, 'protocol_changes.js'); const OUTPUT = path.join(DOC_ROOT, 'protocol', 'flag-days.md'); +// The registry's own files as ONE text, comments blanked, with the map back to +// the part file and line an offset came from. Every registry pass below reads +// this rather than the entry alone, because the rows moved into the parts and +// a pass that read the entry would find no row and publish an empty page. +function registrySources(indexerSrc) { + const source = locatedModuleSource(path.join(indexerSrc, 'protocol_changes.js')); + const rel = (file) => path.relative(indexerSrc, file); + return { + raw: source.text, + // Blanked FILE BY FILE and re-joined the same way, so an unterminated + // shape in one part cannot blank the next and every offset still maps. + scannable: source.parts.map((p) => withoutComments(p.text)).join('\n'), + fileAt: (index) => rel(source.where(index).file), + where: (index) => { const at = source.where(index); return `${rel(at.file)} line ${at.line}`; }, + }; +} + /** * Lower bound for "this number is a Unix timestamp, not a block height". * Block heights are seven digits today and stay under a billion for centuries; @@ -90,108 +117,15 @@ function lineAt(text, index) { return text.slice(0, index).split('\n').length; } -// The characters after which a `/` cannot be dividing a finished operand, so -// it opens a regex literal. Kept byte-identical to the twin in -// lib/env-var-doc-coverage.js. -const REGEX_POSITION_AFTER = new Set(['(', ',', '=', ':', '[', '!', '&', '|', '?', '{', '}', ';', '+', '-', '*', '%', '<', '>', '~', '^']); -const REGEX_POSITION_KEYWORDS = new Set(['return', 'typeof', 'case', 'in', 'of', 'new', 'delete', 'void', 'do', 'else', 'yield', 'await']); - -/** - * The index just past the regex literal starting at `i`, or -1 when there is - * no literal here to read. - * - * FAIL-SAFE BY CONSTRUCTION. Both ambiguous cases return -1, which leaves the - * caller doing exactly what it did before this existed: a `/` that follows a - * finished operand (so it divides), and a literal with no unescaped closing - * `/` before the newline (so the line is not the shape it looked like). Only - * an unambiguous literal takes the branch, so the walk is a strict superset of - * the previous behaviour rather than a new guess. - * - * RESIDUAL LIMIT, and it is deliberate: `}` and `{` are read as regex position - * even though a division can legally follow a block, and a division after - * `)` or `]` is always read as division even though no regex can follow those. - * Both readings are wrong only for source that does not exist here (measured - * 2026-08-20 across all 696 production files in the 11 gated components: zero - * change to the env-read survey and zero change to the computed-read ratchet). - * - * `[...]` classes are honoured, because a `/` inside one does not close. - */ -function regexLiteralEnd(text, i) { - let k = i - 1; - while (k >= 0 && (text[k] === ' ' || text[k] === '\t')) k--; - if (k >= 0 && text[k] !== '\n' && !REGEX_POSITION_AFTER.has(text[k])) { - if (!/[A-Za-z0-9_$]/.test(text[k])) return -1; - let start = k; - while (start >= 0 && /[A-Za-z0-9_$]/.test(text[start])) start--; - if (!REGEX_POSITION_KEYWORDS.has(text.slice(start + 1, k + 1))) return -1; - } - - let j = i + 1; - let inClass = false; - while (j < text.length) { - const c = text[j]; - if (c === '\n') return -1; - if (c === '\\') { j += 2; continue; } - if (inClass) { if (c === ']') inClass = false; j++; continue; } - if (c === '[') { inClass = true; j++; continue; } - if (c === '/') return j + 1; - j++; - } - return -1; -} - -/** - * Blanks every comment body, keeping length and newlines so offsets and line - * numbers still line up with the raw text. - * - * The completeness scan below reads this rather than the source, because dead - * code inside a block comment is not a declaration: a leading-token test only - * recognises the slash-slash and star shapes, so a commented-out call whose own - * line starts with `this.` read as live and failed the build over nothing. - * - * A REGEX LITERAL IS COPIED WHOLE for the same reason a string is: the `//` - * inside `text.replace(/\/\//, '-')` starts no comment, and blanking from it - * drops every declaration later on that line. See `regexLiteralEnd` for the - * two shapes still read as division rather than as a literal. - */ -function withoutComments(text) { - let out = ''; - let i = 0; - const blank = (s) => s.replace(/[^\n]/g, ' '); - - while (i < text.length) { - const two = text.slice(i, i + 2); - if (two === '//') { - const end = text.indexOf('\n', i); - const stop = end === -1 ? text.length : end; - out += blank(text.slice(i, stop)); i = stop; continue; - } - if (two === '/*') { - const end = text.indexOf('*/', i + 2); - const stop = end === -1 ? text.length : end + 2; - out += blank(text.slice(i, stop)); i = stop; continue; - } - if (text[i] === '/') { - // Copy the regex literal whole: the `//` inside one starts no - // comment. Runs AFTER the two branches above, never before them, - // because `//` is how JavaScript itself spells a comment rather - // than an empty literal: testing this first reads every ordinary - // comment line as a zero-length regex, leaves the body live, and - // any apostrophe in it then opens a string that eats the file. - const end = regexLiteralEnd(text, i); - if (end !== -1) { out += text.slice(i, end); i = end; continue; } - } - const ch = text[i]; - if (ch === "'" || ch === '"' || ch === '`') { - // Copy the string whole: a `//` inside one starts no comment. - let j = i + 1; - while (j < text.length && text[j] !== ch) j += text[j] === '\\' ? 2 : 1; - out += text.slice(i, Math.min(j + 1, text.length)); i = j + 1; continue; - } - out += ch; i++; - } - return out; -} +// The comment stripper is the one lib/env-var-doc-coverage.js owns: it was +// ported from here and the two had to stay byte-identical, so this file now +// reads the one copy instead of carrying a twin. It blanks every comment body, +// keeping length and newlines, so offsets and line numbers still line up with +// the raw text; a regex literal and a string are copied whole, because the +// `//` inside either starts no comment. The completeness scan reads this +// rather than the source, because dead code inside a block comment is not a +// declaration. +const withoutComments = stripComments; /** * The mainnet_time argument of a call, when it is a literal this page covers. @@ -200,10 +134,12 @@ function withoutComments(text) { * identifier, or a call shape with too few arguments. Null means "not a row * this page would have carried", which is what makes the check below quiet * about declarations that were never its business. + * + * `open` is the index of the `(` of a call or the `[` of an array row; the + * arguments are read the same way from either. */ -function mainnetTimeLiteral(text, callIndex) { - const open = text.indexOf('(', callIndex); - if (open === -1) return null; +function mainnetTimeLiteral(text, open) { + if (text[open] !== '(' && text[open] !== '[') return null; const args = []; let depth = 0; @@ -296,15 +232,7 @@ function collectSiblingGates(indexerSrc, add) { const text = withoutComments(fs.readFileSync(path.join(indexerSrc, name), 'utf8')); for (const decl of text.matchAll(SIBLING_MAP)) { const body = objectBody(text, decl.index + decl[0].length - 1); - if (body === null) continue; - for (const slot of body.matchAll(SIBLING_MAINNET_SLOT)) { - const read = readMainnetSlot(slot[2]); - const where = `${name}: ${decl[1]}.${slot[1]} = ${slot[2].trim() || '(nothing this scan can read)'}`; - if (read.kind === 'unreadable') unreadable.push(where); - else if (read.kind !== 'number') continue; - else if (slot[1] === 'mainnet') add(decl[1], read.time, name); - else if (read.time >= TIMESTAMP_FLOOR && read.time < SENTINEL_FLOOR) unreadable.push(`${where} is a block TIME, so this page would carry it`); - } + if (body !== null) collectMapSlots(body, decl[1], name, name, add, unreadable); } } @@ -320,6 +248,59 @@ function collectSiblingGates(indexerSrc, add) { } } +/** + * Every mainnet slot of one activation map body, into `add` as `(gate, time, + * source)`, or into `unreadable` under `label` when the scan refuses it. The + * one reader behind the sibling scan and the gate-row pass: the map is the + * same shape in a `const NAME = {` declaration and in an `addGate` row. + */ +function collectMapSlots(body, gate, source, label, add, unreadable) { + for (const slot of body.matchAll(SIBLING_MAINNET_SLOT)) { + const read = readMainnetSlot(slot[2]); + const where = `${label}: ${gate}.${slot[1]} = ${slot[2].trim() || '(nothing this scan can read)'}`; + if (read.kind === 'unreadable') unreadable.push(where); + else if (read.kind !== 'number') continue; + else if (slot[1] === 'mainnet') add(gate, read.time, source); + else if (read.time >= TIMESTAMP_FLOOR && read.time < SENTINEL_FLOOR) unreadable.push(`${where} is a block TIME, so this page would carry it`); + } +} + +/** An `addGate('.', '', {` row with an object-literal table. */ +const GATE_ROW = /addGate\(\s*'([A-Za-z0-9_/]+)\.([A-Z][A-Z0-9_]*)'\s*,\s*'([a-z]+)'\s*,\s*\{/g; + +/** + * Every time-keyed gate the registry's `addGate(key, unit, table)` rows declare. + * + * These are the maps the sibling scan above found in `*_activation.js` before W3: + * the registry owns every activation table now and each module reads its own + * back from it, so the row is where the threshold and its registration comment + * live. The gate keeps the name the sibling scan published, the key's export + * half (`DISPENSER_CAPS_ACTIVATION`), so a table that moved into a row keeps + * its row on the page. Only `'time'` rows are read: the unit says what the + * value scan had to infer from the number's size, and a height row is not this + * page's subject whatever its mainnet slot holds. The slots go through the + * sibling scan's reader, so the same shapes are quiet and the same are loud. + */ +function collectGateRows(sources, add) { + const unreadable = []; + for (const row of sources.scannable.matchAll(GATE_ROW)) { + if (row[3] !== 'time') continue; + const body = objectBody(sources.scannable, row.index + row[0].length - 1); + if (body === null) continue; + collectMapSlots(body, row[2], sources.fileAt(row.index), sources.where(row.index), add, unreadable); + } + if (unreadable.length > 0) { + throw new Error( + 'a registry addGate row declares a mainnet threshold this generator cannot read, so ' + + 'protocol/flag-days.md would publish an inventory that calls itself complete and is not:\n ' + + unreadable.join('\n ') + + "\n\nThe gate-row pass reads `mainnet: ` inside an addGate('.', 'time', { ... }) " + + 'table and stays quiet for `null` and for an identifier it cannot resolve. Either write the ' + + 'threshold in that shape or widen the parse in bin/generate-flag-days.js deliberately.', + ); + } +} + /** * Refuses a registry that declares a gate in a style the two regexes above * cannot read. @@ -352,53 +333,125 @@ function collectSiblingGates(indexerSrc, add) { * * A call is also fine when its gate was collected some other way, which is how a * constant no call consumes still reaches the page under its own prefix. + * + * The array row `['NAME', 'X.Y.Z', mainnet_time, ...]` is the same declaration + * in the part files' shape, and is checked by the same arm: the row's `[` is + * where a call's `(` is, and the arguments read identically from either. */ -function assertEveryDeclarationParsed(rawRegistry, parsedCalls, parsedConstLines, parsedNames) { +function assertEveryDeclarationParsed(sources, parsedCalls, parsedConstLines, parsedNames) { const unparsed = []; - const registry = withoutComments(rawRegistry); + const registry = sources.scannable; - for (const m of registry.matchAll(/addChange\s*\(\s*(['"])([A-Za-z0-9_]+)\1/g)) { + for (const m of registry.matchAll(/(?:addChange\s*\(|\[)\s*(['"])([A-Za-z0-9_]+)\1\s*,\s*(['"])[0-9.]+\3\s*,/g)) { if (parsedCalls.has(m.index)) continue; if (parsedNames.has(m[2])) continue; - const time = mainnetTimeLiteral(registry, m.index); + const time = mainnetTimeLiteral(registry, m.index + m[0].search(/[([]/)); if (time === null) continue; - unparsed.push(`line ${lineAt(registry, m.index)}: addChange call for ${m[2]} at mainnet_time ${time}`); + unparsed.push(`${sources.where(m.index)}: declaration of ${m[2]} at mainnet_time ${time}`); } for (const m of registry.matchAll(/const\s+[A-Z][A-Z0-9_]*_MAINNET_TIME\s*=/g)) { const line = lineAt(registry, m.index); if (parsedConstLines.has(line)) continue; - unparsed.push(`line ${line}: ${rawRegistry.split('\n')[line - 1].trim()}`); + unparsed.push(`${sources.where(m.index)}: ${sources.raw.split('\n')[line - 1].trim()}`); } if (unparsed.length > 0) { throw new Error( - 'protocol_changes.js declares gates this generator cannot read, so protocol/flag-days.md ' + 'the protocol_changes registry declares gates this generator cannot read, so protocol/flag-days.md ' + 'would publish an inventory that calls itself complete and is not:\n ' + unparsed.join('\n ') - + "\n\ncollectGates reads addChange('NAME', 'version', , ...) with single quotes and a " - + 'literal time, and const NAME_MAINNET_TIME = ;. Either write the declaration in one ' - + 'of those shapes or widen the parse in bin/generate-flag-days.js deliberately.', + + "\n\ncollectGates reads addChange('NAME', 'version', , ...) or the part-file row " + + "['NAME', 'version', , ...] with single quotes and a literal time, and " + + 'const NAME_MAINNET_TIME = ;. A decimal literal may carry `_` separators in either ' + + 'position. Either write the declaration in one of those shapes or widen the parse in ' + + 'bin/generate-flag-days.js deliberately.', ); } } +// A decimal literal, separators and all: the SAME grammar the time-slot parser +// in `registryCalls` uses. The two were written apart, digits-only here and +// separator-aware there, and that asymmetry is what made a separator-formatted +// declaration unreadable to the const pass while the identical value in a call +// argument read fine. +const TIME_LITERAL = /^\d(?:_?\d)*$/; + +// The head of a time-constant declaration. The initializer is read separately, +// up to its `;`, so an unreadable shape is REFUSED rather than left unmatched: +// a regex that demands digits simply does not match a hex or arithmetic +// initializer, and a declaration nothing matched is a declaration nothing can +// report. +const TIME_DECL_HEAD = /const\s+([A-Z][A-Z0-9_]*)_(MAINNET|TESTNET)_TIME\s*=/g; + /** - * The registry's `const NAME_MAINNET_TIME = ;` and - * `const NAME_TESTNET_TIME = ;` declarations as an identifier -> value - * map, read from the comment-stripped text. + * The registry's `const NAME_MAINNET_TIME = ;` and + * `const NAME_TESTNET_TIME = ;` declarations, read from the + * comment-stripped text as `{ name, prefix, network, value, line }`. + * + * ONE SCANNER, and it REFUSES what it cannot read. Both properties are load + * bearing. A bare-digit regex per collector and per constant map means widening + * one leaves four un-widened, and a declaration none of them matches vanishes + * in silence: a `const FOO_TESTNET_TIME = 1_789_257_600;` + * consumed by an addChange call resolved to null, the gate dropped out of the + * testnet-exceptions table, and the page then extended its "genesis-active off + * mainnet" claim over a gate that arms on a date of its own. The completeness + * guard could not catch it either: it scans MAINNET declarations only, and + * `collectTestnetArms`, `collectTestnetUnarmed` and `collectMainnetUnarmed` + * never reach it at all. + * + * Refusal lives here for the reason `registryCalls` gives for its own: this + * runs inside `registryCalls`, which every collector calls, so an unreadable + * declaration is loud on every arm or it is loud on one. The name says it is a + * time, so there is no sentinel-versus-timestamp ambiguity to respect. */ -function registryConstants(scannable) { - const values = new Map(); - for (const m of scannable.matchAll(/const\s+([A-Z][A-Z0-9_]*_(?:MAINNET|TESTNET)_TIME)\s*=\s*(\d+)\s*;/g)) { - values.set(m[1], Number(m[2])); +function declaredTimeConstants(sources) { + const out = []; + const scannable = sources.scannable; + TIME_DECL_HEAD.lastIndex = 0; + for (const m of scannable.matchAll(TIME_DECL_HEAD)) { + const name = `${m[1]}_${m[2]}_TIME`; + const line = lineAt(scannable, m.index); + const rest = scannable.slice(m.index + m[0].length); + const end = rest.indexOf(';'); + const text = (end === -1 ? rest : rest.slice(0, end)).trim(); + + if (end === -1 || !TIME_LITERAL.test(text)) { + throw new Error( + `${sources.where(m.index)}: ${name} is declared with an initializer this ` + + `generator cannot read (\`${text.split('\n')[0].slice(0, 60)}\`), so ` + + 'protocol/flag-days.md would publish an inventory that calls itself complete and is ' + + 'not.\n\nA time constant reads as a decimal literal on one line (`1786060800`, ' + + '`_` separators allowed). Write the declaration in that shape or widen the parse in ' + + 'bin/generate-flag-days.js deliberately.', + ); + } + out.push({ name, prefix: m[1], network: m[2], value: Number(text.replace(/_/g, '')), line, index: m.index }); } + return out; +} + +/** + * The same declarations as an identifier -> value map, for the slot parser. + */ +function registryConstants(sources) { + const values = new Map(); + for (const d of declaredTimeConstants(sources)) values.set(d.name, d.value); return values; } /** * Every single-quoted `addChange('GATE', 'version', mainnet_time, testnet_time, ...)` - * call in the comment-stripped registry, as `{ index, gate, mainnet, testnet }`. + * call and every single-quoted part-file row `['GATE', 'version', mainnet_time, + * testnet_time, ...]` in the comment-stripped registry, as + * `{ index, gate, mainnet, testnet }`. + * + * THE ROW IS THE CALL WITHOUT ITS NAME. The part files hold the time table as + * array literals that core.applyChanges() spreads into addChange(), argument + * for argument, so the two shapes carry the same slots in the same order and + * one pattern reads both. The constants a slot names are declared in + * `flag_times*.js` and consumed in `changes_*.js`; they resolve across the + * parts because the const pass runs over the whole joined text first. * * A time slot holding a digit literal reads as that number. A slot holding an * identifier reads as the value of the registry constant it names, so the GATE @@ -427,8 +480,9 @@ function registryConstants(scannable) { * read the same calls and have no completeness check behind them: a shape this * parse cannot read has to be loud on every arm or it is loud on one. */ -function registryCalls(scannable) { - const constants = registryConstants(scannable); +function registryCalls(sources) { + const scannable = sources.scannable; + const constants = registryConstants(sources); const consumed = new Set(); const slot = (arg, gate, index) => { if (arg === undefined) return null; @@ -449,7 +503,7 @@ function registryCalls(scannable) { } throw new Error( - `protocol_changes.js line ${lineAt(scannable, index)}: the ${gate} gate passes a time slot ` + `${sources.where(index)}: the ${gate} gate passes a time slot ` + `this generator cannot read (\`${text}\`), so protocol/flag-days.md would publish an ` + 'inventory that calls itself complete and is not.\n\n' + 'A time slot reads as a decimal literal (`1786060800`, separators allowed) or as the name ' @@ -457,7 +511,7 @@ function registryCalls(scannable) { + 'those shapes or widen the parse in bin/generate-flag-days.js deliberately.', ); }; - const callRe = /addChange\(\s*'([A-Z0-9_]+)'\s*,\s*'[0-9.]+'\s*,\s*([^,)]+)(?:\s*,\s*([^,)]+))?/g; + const callRe = /(?:addChange\(|\[)\s*'([A-Z0-9_]+)'\s*,\s*'[0-9.]+'\s*,\s*([^,)\]]+)(?:\s*,\s*([^,)\]]+))?/g; const calls = []; for (const m of scannable.matchAll(callRe)) { calls.push({ @@ -492,45 +546,43 @@ function collectGates(indexerSrc = INDEXER_SRC) { const parsedConstLines = new Set(); const parsedNames = new Set(); - const registry = fs.readFileSync(path.join(indexerSrc, 'protocol_changes.js'), 'utf8'); - - // COLLECT FROM THE COMMENT-STRIPPED COPY, which the completeness check below - // already did and the two collectors did not. Dead code inside a comment is - // not a declaration, so a retired gate parked in one used to be collected, - // published as a row, and counted toward the coordinated flag day: the - // generator inventing a gate the indexer does not arm, on the page - // implementers plan fleet upgrades from. `withoutComments` preserves every - // offset and newline, so the bookkeeping below still lines up with the raw - // text the check quotes in its error message. - const scannable = withoutComments(registry); - - // addChange('NAME', 'version', mainnet_time, ...), the time slot a digit - // literal or a registry constant passed by name (see registryCalls). - const { calls, consumed } = registryCalls(scannable); + // COLLECT FROM THE COMMENT-STRIPPED COPY (registrySources blanks it). Dead + // code inside a comment is not a declaration, so a retired gate parked in + // one was collected, published as a row, and counted toward the + // coordinated flag day: the generator inventing a gate the indexer does not + // arm, on the page implementers plan fleet upgrades from. + const sources = registrySources(indexerSrc); + + // addChange('NAME', 'version', mainnet_time, ...) or the part-file row + // ['NAME', 'version', mainnet_time, ...], the time slot a digit literal or + // a registry constant passed by name (see registryCalls). "Declared in" + // names the part the row sits in, where its registration comment is. + const { calls, consumed } = registryCalls(sources); for (const call of calls) { parsedCalls.add(call.index); parsedNames.add(call.gate); - if (call.mainnet !== null) add(call.gate, call.mainnet, 'protocol_changes.js'); + if (call.mainnet !== null) add(call.gate, call.mainnet, sources.fileAt(call.index)); } // const NAME_MAINNET_TIME = 1786060800; (gates the registry declares as a // shared constant because a second repo has to stay byte-identical to it). // A constant some call consumes is published under that call's gate name // above; only a constant no call reads is published under its own prefix. - const constRe = /const\s+([A-Z][A-Z0-9_]*)_MAINNET_TIME\s*=\s*(\d+)\s*;/g; - let match; - while ((match = constRe.exec(scannable)) !== null) { - parsedConstLines.add(lineAt(scannable, match.index)); - if (consumed.has(match[1] + '_MAINNET_TIME')) continue; - parsedNames.add(match[1]); - add(match[1], Number(match[2]), 'protocol_changes.js'); + for (const d of declaredTimeConstants(sources)) { + if (d.network !== 'MAINNET') continue; + parsedConstLines.add(d.line); + if (consumed.has(d.name)) continue; + parsedNames.add(d.prefix); + add(d.prefix, d.value, sources.fileAt(d.index)); } - assertEveryDeclarationParsed(registry, parsedCalls, parsedConstLines, parsedNames); + assertEveryDeclarationParsed(sources, parsedCalls, parsedConstLines, parsedNames); - // Sibling `*_activation.js` modules: a mainnet threshold above the - // timestamp floor is a time-keyed gate; below it, a block height, which - // this page does not cover. + // The registry's addGate('.', 'time', { mainnet: ... }) rows, + // then the sibling `*_activation.js` modules for a tree that still declares + // a map of its own: a mainnet threshold above the timestamp floor is a + // time-keyed gate; below it, a block height, which this page does not cover. + collectGateRows(sources, add); collectSiblingGates(indexerSrc, add); return [...found.values()].sort((a, b) => (a.time - b.time) || a.gate.localeCompare(b.gate)); @@ -546,20 +598,16 @@ function collectGates(indexerSrc = INDEXER_SRC) { * testnet_time slot (the fourth argument) in an addChange call. */ function collectTestnetArms(indexerSrc = INDEXER_SRC) { - const scannable = withoutComments( - fs.readFileSync(path.join(indexerSrc, 'protocol_changes.js'), 'utf8'), - ); + const sources = registrySources(indexerSrc); const found = new Map(); const add = (gate, time) => { if (!Number.isFinite(time) || time < TIMESTAMP_FLOOR || time >= SENTINEL_FLOOR) return; if (!found.has(gate)) found.set(gate, { gate, time }); }; - const { calls, consumed } = registryCalls(scannable); + const { calls, consumed } = registryCalls(sources); for (const call of calls) if (call.testnet !== null) add(call.gate, call.testnet); - let match; - const constRe = /const\s+([A-Z][A-Z0-9_]*)_TESTNET_TIME\s*=\s*(\d+)\s*;/g; - while ((match = constRe.exec(scannable)) !== null) { - if (!consumed.has(match[1] + '_TESTNET_TIME')) add(match[1], Number(match[2])); + for (const d of declaredTimeConstants(sources)) { + if (d.network === 'TESTNET' && !consumed.has(d.name)) add(d.prefix, d.value); } return [...found.values()].sort((a, b) => (a.time - b.time) || a.gate.localeCompare(b.gate)); } @@ -578,20 +626,16 @@ function collectTestnetArms(indexerSrc = INDEXER_SRC) { * armed parse. */ function collectTestnetUnarmed(indexerSrc = INDEXER_SRC) { - const scannable = withoutComments( - fs.readFileSync(path.join(indexerSrc, 'protocol_changes.js'), 'utf8'), - ); + const sources = registrySources(indexerSrc); const found = new Map(); const add = (gate, time) => { if (!Number.isFinite(time) || time < SENTINEL_FLOOR) return; if (!found.has(gate)) found.set(gate, { gate, time }); }; - const { calls, consumed } = registryCalls(scannable); + const { calls, consumed } = registryCalls(sources); for (const call of calls) if (call.testnet !== null) add(call.gate, call.testnet); - let match; - const constRe = /const\s+([A-Z][A-Z0-9_]*)_TESTNET_TIME\s*=\s*(\d+)\s*;/g; - while ((match = constRe.exec(scannable)) !== null) { - if (!consumed.has(match[1] + '_TESTNET_TIME')) add(match[1], Number(match[2])); + for (const d of declaredTimeConstants(sources)) { + if (d.network === 'TESTNET' && !consumed.has(d.name)) add(d.prefix, d.value); } return [...found.values()].sort((a, b) => a.gate.localeCompare(b.gate)); } @@ -607,25 +651,21 @@ function collectTestnetUnarmed(indexerSrc = INDEXER_SRC) { * where the gate stands on each network" and the page would not say. Naming them without * an instant keeps both properties. * - * Scoped to `protocol_changes.js`, exactly like the testnet twin above. A sibling + * Scoped to the registry's own files, exactly like the testnet twin above. A sibling * `*_activation.js` module can also park a mainnet sentinel, and this scan does not reach * it; the note it feeds says so rather than claiming a completeness it does not have. */ function collectMainnetUnarmed(indexerSrc = INDEXER_SRC) { - const scannable = withoutComments( - fs.readFileSync(path.join(indexerSrc, 'protocol_changes.js'), 'utf8'), - ); + const sources = registrySources(indexerSrc); const found = new Map(); const add = (gate, time) => { if (!Number.isFinite(time) || time < SENTINEL_FLOOR) return; if (!found.has(gate)) found.set(gate, { gate, time }); }; - const { calls, consumed } = registryCalls(scannable); + const { calls, consumed } = registryCalls(sources); for (const call of calls) if (call.mainnet !== null) add(call.gate, call.mainnet); - let match; - const constRe = /const\s+([A-Z][A-Z0-9_]*)_MAINNET_TIME\s*=\s*(\d+)\s*;/g; - while ((match = constRe.exec(scannable)) !== null) { - if (!consumed.has(match[1] + '_MAINNET_TIME')) add(match[1], Number(match[2])); + for (const d of declaredTimeConstants(sources)) { + if (d.network === 'MAINNET' && !consumed.has(d.name)) add(d.prefix, d.value); } return [...found.values()].sort((a, b) => a.gate.localeCompare(b.gate)); } @@ -687,7 +727,7 @@ function render(gates, testnetArms = [], testnetUnarmed = [], mainnetUnarmed = [ : `${testnetArms.length === 1 ? 'One gate is the exception' : `${testnetArms.length} gates are the exception`}: ` + testnetArms.map((g) => `\`${g.gate}\` arms testnet at \`${g.time}\` (${utcInstant(g.time)})`).join(', ') + '. The reason it cannot be genesis-active there is written in its registration ' - + 'comment in \`protocol_changes.js\`. The values on this page are otherwise ' + + 'comment under \`xchain-indexer/src/protocol_changes/\`. The values on this page are otherwise ' + 'mainnet values only.'; // The other way a gate can be off the genesis-active invariant: parked on the UNARMED @@ -701,7 +741,7 @@ function render(gates, testnetArms = [], testnetUnarmed = [], mainnetUnarmed = [ + 'post-activation behavior and will not until an operator arms it. A consensus ' + 'change registered after the public testnet launch cannot be genesis-active ' + 'there without re-deciding history that outside nodes have already committed. ' - + 'Each names its reason in its registration comment in `protocol_changes.js`.'; + + 'Each names its reason in its registration comment under `xchain-indexer/src/protocol_changes/`.'; // The symmetric mainnet note. Without it a sentinel-parked gate leaves no trace on the // page at all, so the table below reads as the whole registry and the testnet sentence @@ -714,8 +754,8 @@ function render(gates, testnetArms = [], testnetUnarmed = [], mainnetUnarmed = [ + 'rather than an instant, so mainnet has **never** run the post-activation behavior ' + 'and will not until an operator names a date. They carry no row in the table below, ' + 'because publishing the sentinel as a flag day would put a commitment on this page ' - + 'that nobody made. Each names its reason in its registration comment in ' - + '`protocol_changes.js`. This note covers the registry only; a sibling ' + + 'that nobody made. Each names its reason in its registration comment under ' + + '`xchain-indexer/src/protocol_changes/`. This note covers the registry only; a sibling ' + '`*_activation.js` module can park a mainnet sentinel too, and those are not ' + 'enumerated here.'; @@ -725,9 +765,9 @@ function render(gates, testnetArms = [], testnetUnarmed = [], mainnetUnarmed = [ # Flag-Day Values -**This page is generated** from \`xchain-indexer/src/protocol_changes.js\` and the -time-keyed activation modules beside it. Do not edit it by hand: run -\`node bin/generate-flag-days.js\` from the repository root and commit the result. +**This page is generated** from \`xchain-indexer/src/protocol_changes.js\`, its part files +under \`src/protocol_changes/\`, and the time-keyed activation modules beside them. Do not +edit it by hand: run \`node bin/generate-flag-days.js\` from the repository root and commit the result. Every other page in this documentation set names the **gate** and links here instead of quoting a date, because a flag-day value is not a fact about the diff --git a/components/decoder/architecture.md b/components/decoder/architecture.md index 6a16f16a..c25e1ead 100644 --- a/components/decoder/architecture.md +++ b/components/decoder/architecture.md @@ -53,10 +53,10 @@ flowchart TD |---|---|---| | `src/api.js` | None | Entry point: Express server + JSON-RPC, env var loading, signal handlers (SIGTERM/SIGINT) | | `src/XChainDecoder.js` | `XChainDecoder` | Main orchestrator: block polling loop, transaction parsing, deobfuscation, mempool updates, reorg detection | -| `src/BlockchainConnector.js` | `BlockchainConnector` | JSON-RPC client for coin node: getblock, getrawtransaction, getrawmempool, retry with backoff | +| `src/chain/blockchain_connector.js` | `BlockchainConnector` | JSON-RPC client for coin node: getblock, getrawtransaction, getrawmempool, retry with backoff | | `src/db.js` | `Database` | MariaDB connection pool, table creation, block/tx/dispenser inserts, mempool management, reorg rollback | -| `src/XChainBlockDecoder.js` | `XChainBlockDecoder` | Block and transaction parsing via bitcoinjs-lib with coin-specific fixes (Litecoin MWEB, Dogecoin AuxPoW) | -| `src/CryptoNetworks.js` | `CryptoNetworks` | Network configuration: bitcoinjs-lib network objects and start block indexes for all 9 chain/network combinations | +| `src/chain/XChainBlockDecoder.js` | `XChainBlockDecoder` | Block and transaction parsing via bitcoinjs-lib with coin-specific fixes (Litecoin MWEB, Dogecoin AuxPoW) | +| `src/chain/crypto_networks.js` | `CryptoNetworks` | Network configuration: bitcoinjs-lib network objects and start block indexes for all 9 chain/network combinations | | `src/util.js` | None | Utility functions: sleep, SHA256, hex conversion, timer | | `src/sql/*.sql` | None | Table creation SQL for all 9 database tables | diff --git a/components/decoder/configuration.md b/components/decoder/configuration.md index 7140b0c9..2e9b5f80 100644 --- a/components/decoder/configuration.md +++ b/components/decoder/configuration.md @@ -42,6 +42,7 @@ Configuration is loaded from a `.env` file via `dotenv`. All variables are read | `MIGRATION_STRICT_CHECKSUM` | Set to `1` to make a schema-checksum mismatch fail closed at startup instead of logging and continuing. Off by default so a diverged schema does not cause a surprise fleet-wide boot failure; CI and operators running `node src/migrate.js` get the strict path anyway. | _(unset, non-fatal)_ | | `COIN` | Cosmetic label only, reported in the `/status` response. The decoder takes its chain identity from the node it is pointed at, so it has no coin setting of its own; the label stays empty unless a deploy sets one. | _(unset, empty label)_ | | `DECODER_POLL_SILENT_MS` | How long the block loop may go without completing a single iteration before `/live` reports unhealthy (503) and the container restart policy recycles the process. Measures the loop, not the chain: the stall window `DECODER_STALL_ALERT_MS` (default `900000` ms, documented under Operations) asks whether the chain is advancing, and a caught-up decoder advances nothing for hours while being perfectly healthy, so only an iteration count separates "idle" from "the loop is gone". Defaults to twice the stall window, because every normal path through the loop, the node-outage retry included, returns to the loop top far inside it. | `1800000` (30 minutes) | +| `XCHAIN_INDEXER_DIR` | Path override for the sibling `xchain-indexer` checkout that `bin/sync-batch-limits.js` reads to regenerate the vendored BATCH limit tables (`src/protocol/indexer_batch_limits.js`) from the indexer's own `src/actions/batch.js` and `src/protocol_changes.js`. A maintenance-tool setting only; the decoder service itself never reads it. | `../../xchain-indexer` relative to `bin/` (the sibling checkout layout) | The `AUX_POW` variable should be set to any truthy value when running against Dogecoin nodes. It enables the `getBlockWithoutAuxPow()` code path that strips merge-mining headers before parsing. @@ -138,7 +139,7 @@ The decoder begins parsing from a preconfigured block height per network to skip | `dogecoin-testnet` | 64,800,000 | | `dogecoin-regtest` | 0 | -> **DOGE testnet note:** the DOGE testnet mines min-difficulty blocks roughly every 20 seconds, so the chain runs tens of millions of blocks ahead of the other networks. The start block was re-pinned near the chain tip on 2026-06-19 to avoid indexing millions of pre-launch blocks. See `src/CryptoNetworks.js` for the comment. +> **DOGE testnet note:** the DOGE testnet mines min-difficulty blocks roughly every 20 seconds, so the chain runs tens of millions of blocks ahead of the other networks. The start block was re-pinned near the chain tip on 2026-06-19 to avoid indexing millions of pre-launch blocks. See `src/chain/crypto_networks.js` for the comment. ## Valid ACTION Names diff --git a/components/decoder/operations.md b/components/decoder/operations.md index 4ab3f962..1b013d31 100644 --- a/components/decoder/operations.md +++ b/components/decoder/operations.md @@ -122,12 +122,14 @@ Detailed health status including decoder state. | `blockLag` | `integer\|null` | Alias for `lag` (convenience copy) | | `lag_blocks` | `integer\|null` | Live lag computed from internal decoder state; `null` when either height is still unknown (before the first `getBlockchainInfo`, or nothing processed yet) rather than a misleading `0`. May differ slightly from `lag` during rapid catch-up | | `node_height_stale` | `boolean` | Present and `true` when the last successful node-tip poll is more than two refresh intervals old (node outage): `node_height` is then frozen, so a zero `lag` does not mean caught-up | -| `reorg_halted` | `boolean` | `true` while the database carries a live durable REORG_HALT marker (see [Decoder halted after a deep reorg](#decoder-halted-after-a-deep-reorg-reorg_halt)). The decoder keeps parsing and `status` stays `"healthy"`; the next reorg will stop it | +| `reorg_halted` | `boolean` | `true` while the database carries a live durable REORG_HALT marker (see [Decoder halted after a deep reorg](#decoder-halted-after-a-deep-reorg-reorg_halt)). The decoder keeps parsing and `status` stays `"healthy"`; the next reorg parks it (see `reorg_halt_parked`) | | `reorg_halt_reason` | `string\|null` | Why the halt was written | | `reorg_halted_at` | `string\|null` | When the halt was written (ISO 8601) | | `reorg_halt_cleared_at` | `string\|null` | When an operator cleared the last halt with `clear-reorg-halt`; `null` while a halt is live or none was recorded | | `reorg_halt_cleared_reason` | `string\|null` | The reason the operator recorded with that clear | | `reorg_halt_checked_at` | `integer\|null` | Epoch ms of the last marker probe (cached for one minute) | +| `reorg_halt_parked` | `boolean` | `true` once the parse loop has stopped on the halt and is waiting for the clear. `false` on a decoder that carries the marker but is still parsing forward, which is the only thing that separates the two | +| `reorg_halt_parked_at` | `string\|null` | When the parse loop parked (ISO 8601); `null` when it is not parked | | `rpc_errors` | `integer` | Combined RPC error count from the decoder and its `BlockchainConnector` | | `parse_errors` | `integer` | Number of transactions quarantined due to parse failures | | `error` | `string\|null` | Error message if the decoder crashed, otherwise `null` | @@ -268,7 +270,9 @@ Mempool tracking pauses if the decoder falls more than 3 blocks behind the tip, ### Decoder halted after a deep reorg (REORG_HALT) -Symptoms: the log printed `LATENT REORG_HALT MARKER PRESENT` once at startup, `health` reports `reorg_halted: true`, or `xchain-node ps` shows `running REORG_HALT` on the decoder. The decoder still parses forward; it will stop at the next reorg. +Symptoms: the log printed `LATENT REORG_HALT MARKER PRESENT` once at startup, `health` reports `reorg_halted: true`, or `xchain-node ps` shows `running REORG_HALT` on the decoder. The decoder still parses forward; the next reorg parks it. + +A halted decoder stays up. When a reorg reaches the marker the parse loop parks instead of exiting: the container keeps running at a stable restart count, the API keeps answering, and `health`, `/status` and `/live` report `reorg_halt_parked: true` beside `reorg_halted: true`, which is what tells a parked decoder apart from one still parsing forward on a dormant marker. `xchain-node ps` shows `REORG_HALT` against that steady count rather than a climbing one. The parked loop re-reads the marker every 60 seconds, so once the audited clear below lands, the decoder logs one line and resumes parsing from its stored tip on its own, with no restart. Confirm it (this is the decoder's own marker; the `sync_halt` table in the same database belongs to xchain-sync and is a different signal): @@ -291,12 +295,14 @@ Two recoveries: npm run clear-reorg-halt -- --reason "" ``` - The clear checks that every rolled-back block above the tip has been re-parsed (cannot be forced; wait for the decoder to catch up) and that the database holds no dispenser rows and never decoded a `DISPENSER` action (so the purge could not have lost anything). A database that has held dispensers is refused unless you pass `--force` after comparing its `dispensers` table against a known-good replica; the clear is then recorded as forced. `--dry-run` reports the verdict without writing. + The clear checks that every rolled-back block above the tip has been re-parsed (cannot be forced; wait for the decoder to catch up) and that the database holds no dispenser rows and never decoded a `DISPENSER` action (so the purge could not have lost anything). A database that has held dispensers is refused unless you pass `--force` after comparing its `dispensers` table against a known-good replica; the clear is then recorded as forced. `--dry-run` reports the verdict without writing and needs no `--reason`, so run it first to see what a clear would do. The clear writes a `REORG_HALT_CLEARED` event carrying the reason, the check results and the halt it supersedes. The halt row stays for the audit trail, `health` reports `reorg_halted: false` with `reorg_halt_cleared_at` set on its next probe, and the bootstrap health gate accepts the database again. Never delete the `REORG_HALT` row by hand: that erases the evidence the clear records and leaves nothing for the next operator to read. +One marker is rarely alone: a deep reorg on one chain often coincides with halts on the other decoders of the same box. After finding one, run `xchain-node ps` and check every decoder it lists for `REORG_HALT` before moving on. + ### Database name rejected - Database names must match `/^[A-Za-z0-9_]+$/`. No spaces, backticks, or special characters diff --git a/components/e2e-test/README.md b/components/e2e-test/README.md index 6a34d3dc..415449c8 100644 --- a/components/e2e-test/README.md +++ b/components/e2e-test/README.md @@ -42,9 +42,9 @@ sequenceDiagram ## Features -- **31 ACTION test suites**: ADDRESS, AIRDROP, BATCH, BROADCAST, CALLBACK, COINPAY, COLLECT, DELEGATE, DEPLOY, DEPOSIT, DESTROY, DISPENSER, DIVIDEND, EXECUTE, FILE, ISSUE, LINK, LIST, MESSAGE, MINT, ORDER, PRICE, ROLLCALL, SEND, SLASH, SLEEP, STAKE, SWAP, SWEEP, UNSTAKE, WITHDRAW +- **32 ACTION test suites**: ADDRESS, AIRDROP, BATCH, BROADCAST, CALLBACK, COINPAY, COLLECT, DELEGATE, DEPLOY, DEPOSIT, DESTROY, DISPENSER, DIVIDEND, EXECUTE, FILE, ISSUE, LINK, LIST, MESSAGE, MINT, ORDER, PRICE, ROLLCALL, SEND, SLASH, SLEEP, STAKE, SWAP, SWEEP, UNSTAKE, WITHDRAW, XBRIDGE - **How that number is counted:** one suite per ACTION name, with every version of an action folded into a single entry, so ISSUE V0 through V5 counts once and SEND V0 through V3 counts once. An ACTION name is counted when a suite under `test/actions/` builds a payload for it, whether directly or through a helper it loads, and the name is recognised by the decoder's `VALID_ACTION_NAMES`. The figure is not a file count: 69 files collapse onto these 29 names because reorg, negative, and variant suites re-test actions already listed. Regenerate it with `node scripts/count-action-suites.js` (add `--json` for the per-suite breakdown); `test/unit/scripts/actionSuiteCount.test.js` fails if this list and the tree disagree. Actions exercised only by other tiers, such as BET and VOTE in `test/sdk/` or ATTEST and NODEPROOF in `test/federation/`, are outside this count. + **How that number is counted:** one suite per ACTION name, with every version of an action folded into a single entry, so ISSUE V0 through V5 counts once and SEND V0 through V3 counts once. An ACTION name is counted when a suite under `test/actions/` builds a payload for it, whether directly or through a helper it loads, and the name is recognised by the decoder's `VALID_ACTION_NAMES`. The figure is not a file count: 69 files collapse onto these 29 names because reorg, negative, and variant suites re-test actions already listed. Regenerate it with `node scripts/count-action-suites.js` (add `--json` for the per-suite breakdown); `test/unit/scripts/action_suite_count.test.js` fails if this list and the tree disagree. Actions exercised only by other tiers, such as BET and VOTE in `test/sdk/` or ATTEST and NODEPROOF in `test/federation/`, are outside this count. - **9 service connectors**: BlockchainConnector (axios, Basic Auth), XChainUtxoTrackerConnector, XChainEncoderConnector, XChainDecoderConnector, XChainIndexerConnector, XChainExplorerConnector, XChainHubConnector (multi-endpoint failover), RegtestMinerConnector, and Database (MariaDB connection pool) - **Hub auto-discovery**: falls back to xchain-hub for service endpoint resolution when direct environment variables are not set - **Multi-chain support**: Bitcoin, Litecoin, and Dogecoin today, on regtest (network configs via `CryptoNetworks.js`) @@ -67,7 +67,7 @@ flowchart TD subgraph E2E["xchain-e2e-test"] CH["cryptoHelper
BIP39/BIP32
wallet mgmt"] TH["transactionHelper
PSBT/P2SH"] - AH["action helpers (53 modules)
message construction"] + AH["action helpers (58 modules)
message construction"] SC["Service Connectors (src/)
BlockchainConnector, XChainEncoderConnector
XChainUtxoTrackerConn, XChainDecoderConnector
XChainIndexerConnector, XChainExplorerConnector
XChainHubConnector, RegtestMinerConnector
Database (MariaDB)"] CH --> SC TH --> SC diff --git a/components/e2e-test/architecture.md b/components/e2e-test/architecture.md index 83fd8d6b..f7645eb4 100644 --- a/components/e2e-test/architecture.md +++ b/components/e2e-test/architecture.md @@ -178,8 +178,8 @@ xchain-e2e-test/ │ ├── initialCheck.test.js # Mocha root hooks (beforeAll/afterAll) │ ├── cryptoHelper.js # BIP39/BIP32 wallet management │ ├── transactionHelper.js # PSBT construction, signing, broadcast -│ ├── actions/ # 80 action test files (live, ordered), covering 31 ACTION names -│ ├── helpers/ # 53 modules (action helpers + federation/fee/utility helpers) +│ ├── actions/ # 80 action test files (live, ordered), covering 32 ACTION names +│ ├── helpers/ # 58 modules (action helpers + federation/fee/utility helpers) │ ├── unit/ # 350+ unit tests (stubbed, no services) │ ├── integration/ # 150+ integration tests (stubbed I/O) │ │ ├── fixtures/ # mockMariadb, services, dbRows, hub diff --git a/components/encoder/README.md b/components/encoder/README.md index e1f20d2c..40fae982 100644 --- a/components/encoder/README.md +++ b/components/encoder/README.md @@ -76,9 +76,9 @@ Because SegWit witness data is discounted when calculating transaction weight, P ### Multisig -Payload capacity: **approximately 61 bytes per key** +Payload capacity: **60 bytes per multisig output** (two 32-byte key slots carry the payload) -The payload is split across the public key positions of a bare multisig output (`OP_m ... OP_n OP_CHECKMULTISIG`). This is a single-transaction format. The decoder reads the fake public keys from the output to extract the payload. +The payload is split across the public key positions of a bare multisig output (`OP_m ... OP_n OP_CHECKMULTISIG`). Each output carries one 64-byte chunk, a 4-byte magic prefix plus 60 bytes of data, spread over two fake 32-byte public keys, so capacity does not grow with additional key slots. This is a single-transaction format. The decoder reads the fake public keys from the output to extract the payload. Multisig encoding is an alternative for payloads that exceed OP_RETURN's 76-byte user-data limit but where the caller prefers a single-transaction flow. The encoder handles splitting and padding automatically. @@ -159,6 +159,8 @@ npm run api | `ENCODER_MAX_CONCURRENT_REQUESTS` | No | `50` | Concurrency cap for everything that is not a probe. Same immediate-`429` behaviour, and `0` likewise disables it | | `CORS_ORIGIN` | No | Disabled | CORS origin (`*` to allow all) | | `ENCODER_MAINTENANCE_FILE` | No | `/tmp/xchain-encoder-maintenance.json` | Path, inside the encoder container, to the maintenance-window sentinel that `GET /status` reads before reporting an unreachable UTXO tracker. When xchain-node's bootstrap stops the tracker for a scheduled publish, it drops a small JSON file here declaring the outage planned; `/status` then folds that in as context alongside the unchanged readiness fields, so the public status board can show "Maintenance" instead of "Degraded" without ever making an unready encoder read ready. Must be set to the same path as xchain-node's `XCHAIN_NODE_ENCODER_MAINTENANCE_FILE`, since that variable is what writes and removes the file this one points at | +| `ENCODER_REPLICAS` | No | `1` (unset) | Deploy-manifest declaration of the horizontal replica count, checked at boot. Any value above `1` is refused: the UTXO outpoint-reservation double-spend guard, the recent-build duplicate refusal and the rate limiter are all in-process, so two replicas could build PSBTs spending the same UTXO or journal one byte-identical transaction as two successes. Unset or empty passes as the default single-replica deploy | +| `ENCODER_INSTANCE_LOCK_FILE` | No | `/xchain-encoder-.lock` | Path to the same-host PID lockfile the encoder takes exclusively at boot, so two encoder processes accidentally started on one host fail fast instead of racing UTXO selections. Does not see replicas on other hosts or containers; `ENCODER_REPLICAS` is the cross-host declaration | ## Testing diff --git a/components/explorer/README.md b/components/explorer/README.md index 0ad6a75e..76ac618f 100644 --- a/components/explorer/README.md +++ b/components/explorer/README.md @@ -5,7 +5,7 @@ ## What is xchain-explorer -xchain-explorer is the query and presentation layer of the XChain Platform. It reads from the Indexer database and exposes over 200 REST API endpoints, a JSON-RPC 2.0 interface, and a Bootstrap-based web block explorer, all from a single long-lived Node.js/Express process. The explorer never writes to any database. +xchain-explorer is the query and presentation layer of the XChain Platform. It reads from the Indexer database and exposes over 200 REST API endpoints, a JSON-RPC 2.0 interface, and a Bootstrap-based web block explorer, all from a single long-lived Node.js/Express process. The explorer never writes to the Indexer or Decoder databases during normal serving, but it does own and write one schema of its own, the hub mirror (see [Configuration](configuration.md)). The explorer is the primary integration point for wallets, exchanges, dApps, and any application that needs to query XChain state. Developers interact with the platform through the explorer's REST API (directly or via the xchain-sdk), making this the most externally-facing component of the stack. @@ -14,7 +14,7 @@ The explorer is the primary integration point for wallets, exchanges, dApps, and - **Three interfaces**: REST API, JSON-RPC 2.0, and a web block explorer served from the same process - **200+ REST endpoints**: tokens, balances, transactions, market data, DEX state, addresses, blocks, files, messages, and more - **Multi-chain support**: Bitcoin, Litecoin, and Dogecoin today, on mainnet, testnet, and regtest (9 networks) -- **Read-only**: the explorer never writes to the Indexer database +- **Read-only against indexed state**: the explorer issues no writes to the Indexer or Decoder databases, except the optional icon downloader, which writes the indexer-owned `icons` table and so needs INSERT and UPDATE grants there. It owns and writes its own hub-mirror schema: with `"self_sync": true` it creates that schema and its tables, bootstraps them from a hub snapshot, and follows the hub's live feed, so its mirror database user needs DDL and write privileges - **Config discovery**: fetches configuration from xchain-hub on startup and refreshes every 60 seconds - **SSL/TLS support**: serves both HTTP and HTTPS with configurable certificates - **Rate limiting**: configurable request rate limiting (default 500 requests per minute) diff --git a/components/explorer/api.md b/components/explorer/api.md index 9a9b5a1c..959d0a27 100644 --- a/components/explorer/api.md +++ b/components/explorer/api.md @@ -2055,7 +2055,7 @@ GET /icon/{path} GET /openapi.json ``` -OpenAPI 3.1 specification for all explorer REST endpoints. Regenerated by `docs/openapi.build.js`; kept in sync with the route tables by `test/unit/openapi-coverage.test.js`. +OpenAPI 3.1 specification for all explorer REST endpoints. Regenerated by `docs/openapi.build.js`; kept in sync with the route tables by `test/unit/http/openapi_coverage.test.js`. --- diff --git a/components/explorer/architecture.md b/components/explorer/architecture.md index 4fab33b4..b0f7341d 100644 --- a/components/explorer/architecture.md +++ b/components/explorer/architecture.md @@ -15,6 +15,7 @@ flowchart TD EXPLORER["xchain-explorer"] OUT["REST API / JSON-RPC / Web UI"] HUB["xchain-hub"] + MIRRORDB[("Hub mirror DB (MariaDB, explorer-owned)")] NODE -->|"JSON-RPC polling"| DECODER DECODER --> DECDB @@ -23,9 +24,14 @@ flowchart TD IDXDB -->|"SQL reads (read-only)"| EXPLORER EXPLORER --> OUT HUB -->|"config discovery, 60s refresh"| EXPLORER + HUB -->|"snapshot + live feed (self_sync)"| EXPLORER + EXPLORER -->|"schema DDL + row writes"| MIRRORDB + MIRRORDB -->|"SQL reads"| EXPLORER ``` -The explorer sits at the end of the data pipeline. It reads indexed state from the Indexer database (read-only access) and presents it through three interfaces: a REST API, a JSON-RPC 2.0 endpoint, and a web block explorer. It also connects to the Decoder database for raw transaction data lookups. The explorer never writes to any database. +The explorer sits at the end of the data pipeline. It reads indexed state from the Indexer database (read-only access) and presents it through three interfaces: a REST API, a JSON-RPC 2.0 endpoint, and a web block explorer. It also connects to the Decoder database for raw transaction data lookups. + +The explorer writes nothing to the Indexer or Decoder databases during normal serving, with one optional exception: with the icon downloader enabled it writes the indexer-owned `icons` table, which requires INSERT and UPDATE grants there. It does own and write one schema of its own, the hub mirror. With `"self_sync": true` the explorer creates that schema and its tables, bootstraps them from a hub snapshot, and keeps them current from the hub's live feed, so its mirror database user needs DDL and write privileges. See [Configuration](configuration.md) for how the mirror is provisioned. ## Internal Components @@ -55,20 +61,20 @@ flowchart TD | `src/XChainExplorer.js` | `XChainExplorer` | Main orchestrator: URL routing (130+ routes), request processing, response formatting, icon/relay handlers, SPV proof endpoint dispatch | | `src/db.js` | `Database` | All SQL queries (~9,400 lines), connection pool management, pagination, caching | | `src/config.js` | None | Configuration loading from hub or local config.json, 60-second auto-sync, coin/network discovery | -| `src/utility.js` | `Utility` | BigNumber math, timer functions, sanitization (escapeLike, sanitizeInt), type checking | -| `src/XChainHubConnector.js` | `XChainHubConnector` | JSON-RPC client for xchain-hub (ping, getAllConfig) | -| `src/XChainDecoderConnector.js` | `XChainDecoderConnector` | JSON-RPC client for xchain-decoder's health endpoint; lets `/api/status` expose per-coin chain-tip lag without polling decoder ports separately | -| `src/XChainIndexerConnector.js` | `XChainIndexerConnector` | JSON-RPC client for xchain-indexer; proxies read-only `feequote` and `feeschedule` endpoints so fee logic stays single-sourced in the indexer | -| `src/proofServer.js` | `ProofServer` | SPV light-client proof server (spec §8.1): builds Merkle balance/state proofs from the indexer's `state_tree_nodes` table for client-side verification against quorum-signed checkpoint roots | -| `src/merkle.js` | None | Consensus-critical, DB-free Merkle primitives for the additive state commitment, per-block content root, and top-level state root; shared byte-identically with xchain-indexer and xchain-sdk | +| `src/lib/utility.js` | `Utility` | BigNumber math, timer functions, sanitization (escapeLike, sanitizeInt), type checking | +| `src/connectors/hub.js` | `XChainHubConnector` | JSON-RPC client for xchain-hub (ping, getAllConfig) | +| `src/connectors/decoder.js` | `XChainDecoderConnector` | JSON-RPC client for xchain-decoder's health endpoint; lets `/api/status` expose per-coin chain-tip lag without polling decoder ports separately | +| `src/connectors/indexer.js` | `XChainIndexerConnector` | JSON-RPC client for xchain-indexer; proxies read-only `feequote` and `feeschedule` endpoints so fee logic stays single-sourced in the indexer | +| `src/http/proof_server.js` | `ProofServer` | SPV light-client proof server (spec §8.1): builds Merkle balance/state proofs from the indexer's `state_tree_nodes` table for client-side verification against quorum-signed checkpoint roots | +| `src/consensus/merkle.js` | None | Consensus-critical, DB-free Merkle primitives for the additive state commitment, per-block content root, and top-level state root; shared byte-identically with xchain-indexer and xchain-sdk | | `src/checkpoint_commitment_activation.js` | None | Flag-day gate (SPV Phase 2, spec §6.1/§6.3): determines at which BTC block the signed checkpoint canonical gains `state_root` and `block_merkle_root` fields; consensus-critical, vendored across hub/indexer/explorer | | `src/equivocation_header.js` | None | Consensus-critical equivocation header (`EQUIV|ENGINE|ROUND|VIEW||content`) that prefixes every PBFT canonical at/above its activation height; vendored byte-identically across all consensus-bearing services | | `src/stake_weighted_quorum.js` | None | Consensus-critical source-deduplicated stake predicate (3 x tally > 2 x total stake) used by every settlement gate and the checkpoint verifier; the 2f+1 signer count is the separate pre-activation rule, not this one; vendored byte-identically across all consensus-bearing services | -| `src/IconDownloader.js` | `IconDownloader` | In-process worker that downloads, resizes, and caches token icons from the indexer's `icons` table | -| `src/IconResolver.js` | `IconResolver` | Pure icon URL resolution logic; mirrors the priority chain used in the web UI's `xchain.js` so server and browser select the same source | -| `src/configs/BTC.js` | None | Bitcoin-specific: chain info, network addresses (burn, gas, protocol, community) | -| `src/configs/LTC.js` | None | Litecoin-specific configuration | -| `src/configs/DOGE.js` | None | Dogecoin-specific configuration | +| `src/icons/downloader.js` | `IconDownloader` | In-process worker that downloads, resizes, and caches token icons from the indexer's `icons` table | +| `src/icons/resolver.js` | `IconResolver` | Pure icon URL resolution logic; mirrors the priority chain used in the web UI's `xchain.js` so server and browser select the same source | +| `src/coin-config/BTC.js` | None | Bitcoin-specific: chain info, network addresses (burn, gas, protocol, community) | +| `src/coin-config/LTC.js` | None | Litecoin-specific configuration | +| `src/coin-config/DOGE.js` | None | Dogecoin-specific configuration | | `src/config.json` | None | Local database connection configuration (fallback when hub is unavailable) | ### Static Content (`src/content/`) @@ -181,7 +187,7 @@ Two pagination modes are supported: ## SPV Light-Client Proof Server -The `ProofServer` class (`src/proofServer.js`) serves read-only Merkle proofs for the SPV light-client protocol (Phase 3, spec §8.1). It is instantiated by `XChainExplorer` on startup and handles four proof endpoint families: +The `ProofServer` class (`src/http/proof_server.js`) serves read-only Merkle proofs for the SPV light-client protocol (Phase 3, spec §8.1). It is instantiated by `XChainExplorer` on startup and handles four proof endpoint families: ``` GET /{COIN}/api/proof/balance/:address/:tick - SMT balance inclusion / non-inclusion proof @@ -193,7 +199,7 @@ GET /{COIN}/api/checkpoints/range - Forward-ordered checkpoint sli All proofs are derived from the indexer DB's `state_tree_nodes` and `state_tree_roots` tables, which are NOT replicated by `xchain-sync`. The proof server checks that its local tree assembles to the same root as the signed checkpoint before returning any proof; if they disagree (server bug or divergence), it returns an error rather than a proof the client cannot verify. -The cryptographic primitives used are in `src/merkle.js`, which is vendored byte-identically across `xchain-indexer`, `xchain-explorer`, and `xchain-sdk` so that a proof produced here verifies under `merkle.verifyCompressedSmtProof` (balance/validator) or `merkle.verifyFixedMerkleProof` (action) in the SDK. +The cryptographic primitives used are in `src/consensus/merkle.js`, which is vendored byte-identically across `xchain-indexer`, `xchain-explorer`, and `xchain-sdk` so that a proof produced here verifies under `merkle.verifyCompressedSmtProof` (balance/validator) or `merkle.verifyFixedMerkleProof` (action) in the SDK. See [API.md](api.md) for the full request/response shapes and error codes. @@ -203,10 +209,10 @@ The explorer provides a real-time event streaming API via WebSockets. Four modul ``` src/ws/ -├── WebSocketServer.js # Connection handling, upgrade, WELCOME, message routing -├── ChannelManager.js # Subscription tracking with filters (types, ticks, etc.; statuses accepted, never confirmed active) -├── ChangeDetector.js # Polls DB for new blocks/actions, emits lifecycle events -└── Broadcaster.js # Routes events to subscribed clients through filter pipeline +├── websocket_server.js # Connection handling, upgrade, WELCOME, message routing +├── channel_manager.js # Subscription tracking with filters (types, ticks, etc.; statuses accepted, never confirmed active) +├── change_detector.js # Polls DB for new blocks/actions, emits lifecycle events +└── broadcaster.js # Routes events to subscribed clients through filter pipeline ``` **Data flow:** diff --git a/components/explorer/configuration.md b/components/explorer/configuration.md index 25eec1bd..27de20ff 100644 --- a/components/explorer/configuration.md +++ b/components/explorer/configuration.md @@ -145,6 +145,7 @@ two endpoints return `503` (clients then fall back to paying the protocol fee in | `INDEXER_API_TIMEOUT_MS` | No | `5000` | Per-request timeout for the indexer proxy calls | | `EXPLORER_FEEQUOTE_BUSY_RETRY_MS` | No | `6000` | Wall-clock budget for re-asking `/{COIN}/api/feequote` while the indexer answers `busy: true, retryable: true` (it is processing a block). This hop absorbs the overlap because the wallet reads the endpoint on every fee-bearing compose and has no retry of its own. Only a retryable busy answer is re-asked; a verdict never is. | | `EXPLORER_INDEXER_API_KEY` | No | None | API key presented to the indexer's fail-closed federation-read gate. When the peer indexer sets `INDEXER_API_KEY`, gated methods such as `getstakeweightsbycapability` return `401` without this, which is what a hardened indexer needs in order to still serve the explorer's validator-set proof. | +| `EXPLORER_FEDERATION_READ_KEY` | No | None (federation reads refused) | Key a caller must present as `x-api-key` to use the five federation reads the explorer serves off its replicated indexer databases on `POST /{COIN}/api/`: `getrollcallsigners`, `getanchoraction`, `getanchorconfirmations`, `getarchiveanchor` and `getpricebatches`. A validator without its own Dogecoin indexer sets `DOGE_INDEXER_API_URL` to this explorer and `DOGE_INDEXER_API_KEY` to this key. Unset, those methods answer `-32001 Unauthorized` to everyone; every other route is unaffected. | | `DECODER_API_TIMEOUT_MS` | No | `2500` | Per-request timeout for decoder health calls. Tighter than the indexer timeout on purpose: health aggregation runs on the `/api/status` hot path, so a stalled decoder must not hold the whole status response. | ### Contract simulation (Read Contract card) @@ -167,6 +168,16 @@ The check exists because a deployed explorer's bundled VM can go stale silently: The in-process check is coarser than a byte comparison, because a running process has no canonical copy to compare against. The full comparison is `bin/check-explorer-vm-drift.sh ` in the platform checkout: read-only over SSH, it hashes the deployed VM tree against canonical and reads the flag out of the running process. Run it before enabling the flag on a public explorer, and enable only once it reports `OK`. +### Activation Registry (regtest arming) + +The explorer carries a byte-identical copy of the shared activation rows the indexer, hub, sync and SDK carry (`src/consensus/gate_registry/`), and that copy arms its regtest entries from the same three variables by the same grammar (`src/consensus/gate_registry/regtest_env.js`). The explorer drives no roll call and admits no mirror row itself; honouring the levers keeps its reading of a row identical to the venue's, so a consensus-identity comparison across the venue's processes does not diverge on the explorer. + +| Variable | Required | Default | Description | +|---|---|---|---| +| `XC_ROLLCALL_REGTEST_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_ACTIVATION` in the explorer's copy of the activation registry. `armed` (or `genesis`/`on`/`true`/`yes`) arms at BTC height `0`; a bare non-negative integer arms at that height; `off`/`inert`/`false`/`no`/`none` and unset leave it inert, and anything else is refused with a process warning and stays inert. Applied when a row is read, from the environment as it stands then. mainnet and testnet are fixed in source and cannot be moved from the environment. Never set outside a regtest venue. | +| `XC_ROLLCALL_GATES_REGTEST_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_GATES_ACTIVATION` (ROLLCALL v1, the consensus-gate list roll calls carry) in the explorer's registry copy. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; set identically on every hub, indexer and explorer process in the venue. mainnet and testnet are fixed in source. Never set outside a regtest venue. | +| `XC_MIRROR_ADMISSION_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the per-coin `regtest` entries of `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION` (the mirror-admission heights) and the `regtest` entry of `ANCHOR_ATTEST_BARRIER_ACTIVATION` in the explorer's registry copy, one variable for the whole barrier family. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; the armed form arms at height `0`. mainnet and testnet are fixed in source. Never set outside a regtest venue. | + ### SSL/TLS | Variable | Required | Default | Description | @@ -205,6 +216,31 @@ A value that does not match its expected shape (a non-numeric kit id, a license other than `free`/`pro`) is ignored with a warning rather than passed through to the browser. +### Metrics and Log Shipping + +The shared observability module adds a Prometheus scrape endpoint and a +structured log shim. Both are off unless set here: with no variables the +explorer registers no extra route, starts no timer, and opens no socket. The +log shim is installed before the explorer's first log line, so these variables +also shape startup output. + +| Variable | Required | Default | Description | +|---|---|---|---| +| `METRICS_ENABLED` | No | off | Serve the Prometheus scrape endpoint. | +| `METRICS_PATH` | No | `/metrics` | Scrape path. | +| `METRICS_TOKEN` | No | None | Require `Authorization: Bearer ` on the scrape. Set this (or keep the path behind the fronting proxy) on any internet-reachable box. | +| `METRICS_HTTP` | No | `true` when metrics are on | Per-request counters and a latency histogram. Set `0` for endpoint-only. | +| `LOG_FORMAT` | No | `text` | `json` emits one NDJSON record per log line. | +| `LOG_LEVEL` | No | `info` | `debug`, `info`, `warn`, or `error`. | +| `LOG_SHIP_ENABLED` | No | off | POST batched NDJSON to a collector. Needs `LOG_SHIP_URL` too; either alone stays off. | +| `LOG_SHIP_URL` | No | None | Collector endpoint (http/https). | +| `LOG_SHIP_TOKEN` | No | None | Bearer token for the collector. Never logged or echoed. | +| `LOG_SHIP_BATCH_SIZE` | No | `100` | Lines per POST. | +| `LOG_SHIP_INTERVAL_MS` | No | `5000` | Flush interval. | +| `LOG_SHIP_MAX_BUFFER` | No | `5000` | Bounded buffer; the oldest lines are dropped and counted, never grown without limit. | +| `LOG_SHIP_TIMEOUT_MS` | No | `5000` | Per-batch POST timeout. | +| `XCHAIN_LOG_PATCH` | No | None | Set `0` to leave the global console unpatched, so no structured log shim is installed. The test bootstrap sets it so suites see the stock console. | + ## Local Configuration File The `src/config.json` file provides database connection details when xchain-hub is not available. Structure: @@ -283,13 +319,13 @@ config.onConfigChanged(() => { ## Coin-Specific Configuration -Each supported blockchain has a configuration file in `src/configs/`: +Each supported blockchain has a configuration file in `src/coin-config/`: | File | Chain | |---|---| -| `src/configs/BTC.js` | Bitcoin | -| `src/configs/LTC.js` | Litecoin | -| `src/configs/DOGE.js` | Dogecoin | +| `src/coin-config/BTC.js` | Bitcoin | +| `src/coin-config/LTC.js` | Litecoin | +| `src/coin-config/DOGE.js` | Dogecoin | These files export a `getConfig(network)` function returning: diff --git a/components/hub/README.md b/components/hub/README.md index 443b2437..3619fa79 100644 --- a/components/hub/README.md +++ b/components/hub/README.md @@ -28,7 +28,7 @@ The hub operates in two modes. In **standalone mode** (no `P2P_VALIDATOR_ADDR` s - **Reorg propagation**: cross-chain reorg detection with PBFT consensus, hub state rollback, and downstream notification - **Governance**: off-chain PBFT voting for parameter changes with 7-day voting period, 2/3+ approval, 50% quorum - **Reward tracking**: per-round XCHAIN rewards distributed equally among participating oracle validators -- **Slash detection**: price deviation (>5%), repeated deviation (3+ in 24h), and non-participation (30+ missed rounds) monitoring +- **Offense detection** (`SlashDetector`): price deviation (>5%), repeated deviation (3+ in 24h), and non-participation (30+ missed rounds) monitoring. These are hub-local records; governance can suspend a validator, and on-chain stake is burned only by a SLASH proof of equivocation - **Multi-instance**: multiple hub instances against shared MariaDB with consumer fallback via `HUB_VALIDATORS` - **MariaDB storage**: 20 relational tables with connection pooling, circuit breaker, and exponential backoff - **Single-node fallback**: all consensus-dependent operations fall back to direct execution when no peers are connected diff --git a/components/hub/api.md b/components/hub/api.md index 0111ae00..5a78d2ff 100644 --- a/components/hub/api.md +++ b/components/hub/api.md @@ -380,7 +380,7 @@ Pushes a chain tip update from an indexer. The hub uses this to anchor oracle ro {"status":"success"} ``` -Stored in the `configs` table as `(coin, mainnet, chain_tips, block_height|block_time)`. Read by `OracleRound._executeRound()` at the start of each PBFT round. +Stored in the `configs` table as `(coin, mainnet, chain_tips, block_height|block_time)`. Read by `OracleRound.executeRound()` at the start of each PBFT round. ### `pushpriceround` (write: requires API key) @@ -1164,6 +1164,8 @@ ANCHOR publisher status (read, no auth): cumulative anchor counts plus the last- > **Note:** this method is implemented on `xchain-indexer`, not the hub. The hub's `RewardTracker` calls it to persist anchor-publish reward rows into the indexer's `validator_rewards` table. +> **Retired for new anchor rewards.** At or above `ANCHOR_REWARD_ACTIVATION` (for `anchor_bundle`) and `ARCHIVE_REWARD_ACTIVATION` (for `anchor_archive`) every indexer DERIVES the reward from the on-chain ANCHOR bytes, so this push no longer establishes those rows; it remains documented because it is how pre-flag-day rows arrived, and those rows are not re-derivable from a chain parse. See [ANCHOR](../../protocol/actions/anchor.md). + Accepted `reward_type` values must match `^anchor_[A-Za-z_]+$` (the live types are `anchor_bundle` for a checkpoint bundle and `anchor_archive` for a match-archive batch). The indexer **rejects** `oracle_round` and `attest_fee` because those are derived deterministically during block processing, accepting a push for them would open a replay-divergence window. **Request** (from hub → indexer): @@ -1244,7 +1246,7 @@ The hub serves a machine-readable **OpenRPC 1.3.2** specification at: GET /openrpc.json ``` -No authentication required. The spec is generated by `docs/openrpc.build.js` and kept in lockstep with `jsonRpcController` by `test/unit/openrpc-coverage.test.js`. +No authentication required. The spec is generated by `docs/openrpc.build.js` and kept in lockstep with `jsonRpcController` by `test/unit/api/openrpc_coverage.test.js`. ## Telemetry (REST) diff --git a/components/hub/architecture.md b/components/hub/architecture.md index fe9b2ab6..b4ff216c 100644 --- a/components/hub/architecture.md +++ b/components/hub/architecture.md @@ -39,7 +39,7 @@ flowchart LR direction TB SA_API["api.js
Express + JSON-RPC"] SA_HUB["XChainHub
(orchestrator)"] - SA_DB["db.js
MariaDB pool + circuit breaker"] + SA_DB["db/index.js
MariaDB pool + circuit breaker"] SA_NOTE["Config writes go directly to MariaDB.
No P2P, no consensus."] SA_API --> SA_HUB --> SA_DB SA_DB -.-> SA_NOTE @@ -58,7 +58,7 @@ flowchart LR VA_REWARD["RewardTracker"] VA_SLASH["SlashDetector"] VA_SWAP["SwapTracker"] - VA_DB["db.js
MariaDB pool + circuit breaker"] + VA_DB["db/index.js
MariaDB pool + circuit breaker"] VA_API --> VA_HUB --> VA_PEER VA_PEER --> VA_CONS --> VA_GOV @@ -118,34 +118,34 @@ flowchart TD |---|---|---| | `api.js` | None | Entry point: Express app, JSON-RPC routes, env var validation, starts XChainHub | | `XChainHub.js` | `XChainHub` | Orchestrator: wires all subsystems, exposes JSON-RPC method handlers | -| `db.js` | `Database` | MariaDB connection pool with circuit breaker and exponential backoff | -| `PeerManager.js` | `PeerManager` | WebSocket P2P gossip layer: peer connections, message signing, heartbeats | -| `Consensus.js` | `Consensus` | PBFT consensus for config writes: PRE_PREPARE → PREPARE → COMMIT | -| `ValidatorIdentity.js` | `ValidatorIdentity` | Ed25519 key management: signing, verification, key generation | -| `OracleRound.js` | `OracleRound` | Oracle round lifecycle: timer, price fetching, submission broadcast | -| `OracleConsensus.js` | `OracleConsensus` | PBFT consensus for price finalization: trimmed median, propose/prepare/commit | -| `PriceFetcher.js` | `PriceFetcher` | External price API client: CoinGecko and Kraken (both keyless, always active) plus CoinMarketCap (optional, when `COINMARKETCAP_API_KEY` is set) | -| `CrossChainEngine.js` | `CrossChainEngine` | PBFT attestation for cross-chain actions with per-chain-pair validators | -| `SwapTracker.js` | `SwapTracker` | Cross-chain SWAP lifecycle tracking: initiated → attested → executed → settled | -| `ReorgHandler.js` | `ReorgHandler` | Blockchain reorg detection, PBFT consensus, and hub state rollback | -| `Governance.js` | `Governance` | Off-chain PBFT voting for parameter changes | -| `RewardTracker.js` | `RewardTracker` | Per-round XCHAIN reward distribution to oracle participants; pushes rewards to BTC indexer for `COLLECT` | -| `SlashDetector.js` | `SlashDetector` | Validator misbehavior detection: price deviation, non-participation | -| `PriceAggregator.js` | `PriceAggregator` | Receives validated PRICE v0/v1 actions from indexers, deduplicates by `round_number` (v0) or `(source, action_index)` (v1), writes to `price_snapshots`/`oracle_prices`. EventEmitter: emits `row:inserted` for hub DB sync. | -| `OraclePublisher.js` | `OraclePublisher` | `oracle_publish` capability publisher: deterministic leader rotation, persistent JSONL queue, builds PRICE v0 wire format, broadcasts to DOGE via the encoder pipeline, monitors DOGE balance | -| `EncoderClient.js` | `EncoderClient` | Minimal JSON-RPC client for talking to xchain-encoder (`get_utxos`, `create_tx`, `broadcast_tx`): used by `OraclePublisher` | -| `HubDbBroadcaster.js` | `HubDbBroadcaster` | WebSocket subscriber registry; broadcasts `row:inserted` events from `PriceAggregator`, `StateCheckpointEngine`, `CrossChainDexEngine`, and `CrossChainCallEngine` to all connected indexers' `HubDbSync` clients | -| `StateCheckpointEngine.js` | `StateCheckpointEngine` | Quorum-signed per-chain ledger/actions/contract hash checkpoints: cadence-leader reads each chain's block-hash triple, collects XCHK_SIGN from peers, finalizes at the federation quorum for the checkpoint's snapshot block (stake-weighted and source-deduped at/above `STAKE_WEIGHTED_QUORUM_ACTIVATION`, otherwise the majority-floored count `max(2f+1, ceil((N+1)/2))`; see [Quorum](#quorum)), writes to `state_checkpoints`, streams via `HubDbBroadcaster`, emits `checkpoint:finalized` | -| `StateAnchorPublisher.js` | `StateAnchorPublisher` | Checkpoint-bundle anchor publisher: listens for `checkpoint:finalized`, batches `cross_chain_matches` archive, and commits every checkpointed chain in ONE DOGE [ANCHOR v0](../../protocol/actions/anchor.md) action per network per publishing cycle (one section per chain, one publisher election per bundle), plus the archive, on the `ANCHOR_INTERVAL_MS` cadence | -| `FullNodeChallengeRound.js` | `FullNodeChallengeRound` | Challenge-response rounds that verify `full_node` capability claimants. The elected leader issues a block-hash challenge; each claimant broadcasts its computed answer (`XNODE_ANSWER`); the leader proposes the pass list (`XNODE_SIGN_REQ`); verifiers independently recompute and co-sign (`XNODE_SIGN`); results are finalized on-chain via `XNODE_DONE`. Pass rate feeds into the full-node reward tier. | -| `AttestationPublisher.js` | `AttestationPublisher` | Subscribes to `AttestationConsensus` `request:finalized` events and ships the on-chain ATTEST v1 (response) wire payload via an operator-provided hook. Writes a durable JSONL write-ahead log before any broadcast; the leader broadcasts immediately, followers step in after `failoverWindowBlocks` blocks using a rank-staggered backoff. | -| `AttestationRound.js` | `AttestationRound` | Event-driven per-request lifecycle for the external attestation framework. Polls the indexer for new ATTEST v0 (request) rows, selects the responsible validator set via SHA-256 ordering at `block_index`, fetches the payload via the provider module, and gossips `ATTEST_PROPOSE` for `AttestationConsensus` to drive to quorum. | -| `AttestationSpotChecker.js` | `AttestationSpotChecker` | Spot-checker for synthetic ATTEST v0 requests injected to verify validator honesty. When `AttestationConsensus` finalizes a round, compares the published response against the expected pattern using a provider's `judge_model` comparator. Repeated failures within a 24-hour window trigger a slash proposal via `SlashDetector`. | -| `CapabilityRegistry.js` | `CapabilityRegistry` | Tracks per-validator capability state in the `validator_capabilities` table. A capability is active when all three conditions hold: `qualified` (stake >= configured `MIN_STAKE`), `self_test_ok` (local self-test passed), and `enabled` (operator has not opted out). Hot-reloads the capability config file on change. | -| `CapabilitySnapshot.js` | `CapabilitySnapshot` | Locks the validator set for a capability at a block boundary so every hub in the federation computes the same PBFT quorum for a given round. Queries the BTC indexer at the target `blockIndex`; stake state at a given block is on-chain-deterministic, making the snapshot cross-hub identical. Self-test and enabled flags are excluded (those are local per hub). | -| `CrossChainDexEngine.js` | `CrossChainDexEngine` | Matches cross-chain ORDER/SWAP offers across chain-isolated indexer order books. Polls each chain's `getopencrosschainorders` RPC, pairs compatible offers, drives PBFT finalization via `CrossChainDexConsensus`, writes validator-signed match rows to `cross_chain_matches`, and broadcasts them to indexers via `HubDbBroadcaster`. | -| `CrossChainDexConsensus.js` | `CrossChainDexConsensus` | PBFT consensus engine for cross-chain DEX match finalization. Each peer independently re-derives and validates the canonical match before co-signing. Drives a 3-phase PBFT round (PROPOSE, PREPARE, COMMIT) with VIEW_CHANGE / NEW_VIEW leader failover. Reused as the base engine for `CrossChainCallEngine` with parameterized message types. | -| `ProviderRegistry.js` | `ProviderRegistry` | Hub-authoritative registry of governance-approved attestation providers. Loads provider definitions from the `configs` table under `module='ATTESTATION_PROVIDER'`; falls back to a built-in `http_get` default so a fresh hub works without prior governance configuration. Hot-reloads on `governance proposal:passed` events. | +| `db/index.js` | `Database` | MariaDB connection pool with circuit breaker and exponential backoff | +| `peers/manager.js` | `PeerManager` | WebSocket P2P gossip layer: peer connections, message signing, heartbeats | +| `consensus/pbft.js` | `Consensus` | PBFT consensus for config writes: PRE_PREPARE → PREPARE → COMMIT | +| `validators/identity.js` | `ValidatorIdentity` | Ed25519 key management: signing, verification, key generation | +| `oracle/round.js` | `OracleRound` | Oracle round lifecycle: timer, price fetching, submission broadcast | +| `oracle/consensus.js` | `OracleConsensus` | PBFT consensus for price finalization: trimmed median, propose/prepare/commit | +| `oracle/price_fetcher.js` | `PriceFetcher` | External price API client: CoinGecko and Kraken (both keyless, always active) plus CoinMarketCap (optional, when `COINMARKETCAP_API_KEY` is set) | +| `cross_chain/engine.js` | `CrossChainEngine` | PBFT attestation for cross-chain actions with per-chain-pair validators | +| `cross_chain/swap_tracker.js` | `SwapTracker` | Cross-chain SWAP lifecycle tracking: initiated → attested → executed → settled | +| `anchor/reorg_handler.js` | `ReorgHandler` | Blockchain reorg detection, PBFT consensus, and hub state rollback | +| `validators/governance.js` | `Governance` | Off-chain PBFT voting for parameter changes | +| `anchor/reward_tracker.js` | `RewardTracker` | Per-round XCHAIN reward distribution to oracle participants; pushed anchor rewards to the BTC indexer for `COLLECT` below the anchor-reward flag-days, a rail retired at or above them in favour of indexer-side derivation from the ANCHOR bytes | +| `validators/slash_detector.js` | `SlashDetector` | Validator misbehavior detection: price deviation, non-participation | +| `oracle/price_aggregator.js` | `PriceAggregator` | Receives validated PRICE v0/v1 actions from indexers, deduplicates by `round_number` (v0) or `(source, action_index)` (v1), writes to `price_snapshots`/`oracle_prices`. EventEmitter: emits `row:inserted` for hub DB sync. | +| `oracle/publisher.js` | `OraclePublisher` | `oracle_publish` capability publisher: deterministic leader rotation, persistent JSONL queue, builds PRICE v0 wire format, broadcasts to DOGE via the encoder pipeline, monitors DOGE balance | +| `peers/encoder_client.js` | `EncoderClient` | Minimal JSON-RPC client for talking to xchain-encoder (`get_utxos`, `create_tx`, `broadcast_tx`): used by `OraclePublisher` | +| `peers/hub_db_broadcaster.js` | `HubDbBroadcaster` | WebSocket subscriber registry; broadcasts `row:inserted` events from `PriceAggregator`, `StateCheckpointEngine`, `CrossChainDexEngine`, and `CrossChainCallEngine` to all connected indexers' `HubDbSync` clients | +| `anchor/checkpoint_engine.js` | `StateCheckpointEngine` | Quorum-signed per-chain ledger/actions/contract hash checkpoints: cadence-leader reads each chain's block-hash triple, collects XCHK_SIGN from peers, finalizes at the federation quorum for the checkpoint's snapshot block (stake-weighted and source-deduped at/above `STAKE_WEIGHTED_QUORUM_ACTIVATION`, otherwise the majority-floored count `max(2f+1, ceil((N+1)/2))`; see [Quorum](#quorum)), writes to `state_checkpoints`, streams via `HubDbBroadcaster`, emits `checkpoint:finalized` | +| `anchor/publisher.js` | `StateAnchorPublisher` | Checkpoint-bundle anchor publisher: listens for `checkpoint:finalized`, batches `cross_chain_matches` archive, and commits every checkpointed chain in ONE DOGE [ANCHOR v0](../../protocol/actions/anchor.md) action per network per publishing cycle (one section per chain, one publisher election per bundle), plus the archive, on the `ANCHOR_INTERVAL_MS` cadence | +| `consensus/full_node_challenge_round.js` | `FullNodeChallengeRound` | Challenge-response rounds that verify `full_node` capability claimants. The elected leader issues a block-hash challenge; each claimant broadcasts its computed answer (`XNODE_ANSWER`); the leader proposes the pass list (`XNODE_SIGN_REQ`); verifiers independently recompute and co-sign (`XNODE_SIGN`); results are finalized on-chain via `XNODE_DONE`. Pass rate feeds into the full-node reward tier. | +| `attestation/publisher.js` | `AttestationPublisher` | Subscribes to `AttestationConsensus` `request:finalized` events and ships the on-chain ATTEST v1 (response) wire payload via an operator-provided hook. Writes a durable JSONL write-ahead log before any broadcast; the leader broadcasts immediately, followers step in after `failoverWindowBlocks` blocks using a rank-staggered backoff. | +| `attestation/round.js` | `AttestationRound` | Event-driven per-request lifecycle for the external attestation framework. Polls the indexer for new ATTEST v0 (request) rows, selects the responsible validator set via SHA-256 ordering at `block_index`, fetches the payload via the provider module, and gossips `ATTEST_PROPOSE` for `AttestationConsensus` to drive to quorum. | +| `attestation/spot_checker.js` | `AttestationSpotChecker` | Spot-checker for synthetic ATTEST v0 requests injected to verify validator honesty. When `AttestationConsensus` finalizes a round, compares the published response against the expected pattern using a provider's `judge_model` comparator. Repeated failures within a 24-hour window trigger a slash proposal via `SlashDetector`. | +| `validators/capability_registry.js` | `CapabilityRegistry` | Tracks per-validator capability state in the `validator_capabilities` table. A capability is active when all three conditions hold: `qualified` (stake >= configured `MIN_STAKE`), `self_test_ok` (local self-test passed), and `enabled` (operator has not opted out). Hot-reloads the capability config file on change. | +| `validators/capability_snapshot.js` | `CapabilitySnapshot` | Locks the validator set for a capability at a block boundary so every hub in the federation computes the same PBFT quorum for a given round. Queries the BTC indexer at the target `blockIndex`; stake state at a given block is on-chain-deterministic, making the snapshot cross-hub identical. Self-test and enabled flags are excluded (those are local per hub). | +| `cross_chain/dex_engine.js` | `CrossChainDexEngine` | Matches cross-chain ORDER/SWAP offers across chain-isolated indexer order books. Polls each chain's `getopencrosschainorders` RPC, pairs compatible offers, drives PBFT finalization via `CrossChainDexConsensus`, writes validator-signed match rows to `cross_chain_matches`, and broadcasts them to indexers via `HubDbBroadcaster`. | +| `cross_chain/dex_consensus.js` | `CrossChainDexConsensus` | PBFT consensus engine for cross-chain DEX match finalization. Each peer independently re-derives and validates the canonical match before co-signing. Drives a 3-phase PBFT round (PROPOSE, PREPARE, COMMIT) with VIEW_CHANGE / NEW_VIEW leader failover. Reused as the base engine for `CrossChainCallEngine` with parameterized message types. | +| `validators/provider_registry.js` | `ProviderRegistry` | Hub-authoritative registry of governance-approved attestation providers. Loads provider definitions from the `configs` table under `module='ATTESTATION_PROVIDER'`; falls back to a built-in `http_get` default so a fresh hub works without prior governance configuration. Hot-reloads on `governance proposal:passed` events. | | `constants.js` | None | Shared protocol/consensus constants for the oracle price pipeline. Exports `PRICE_MAX` (the per-pair price ceiling enforced during ingestion and aggregation) and `ORACLE_DEVIATION_THRESHOLD` (used by `OracleConsensus`). | | `bcmath.js` | None | Big-number helpers for cross-chain DEX partial-fill matching. A faithful port of the bignumber utilities in `xchain-indexer` (mathjs bignumber, `bcmul` at precision 18, `bcsub` at precision 64) so hub match quantities stay byte-identical to the indexer's local fill math. | | `stake_weighted_quorum.js` | None | Consensus-critical stake-weighted quorum predicate (WI-1). Vendored byte-identically from `xchain-documentation/protocol/reference-impl/` into the hub and every service that tallies PBFT votes or verifies settlement gates. A CI gate (ConsensusPrimitiveConformance) asserts byte-identity across all repos. | @@ -229,7 +229,7 @@ JSON.stringify({ id, type, sender, timestamp, data }) A verifier must reconstruct this exact string to check the signature. The signing key is the sender's Ed25519 validator key; the verifier looks up `sender` in the validator registry to obtain the 64-hex-char public key. When `REQUIRE_SIGNATURES=true`, unsigned messages and messages from unknown senders are rejected; otherwise they are accepted (bootstrap mode). -**Inbound processing order** (in `_handleInbound`): JSON parse → reject non-object/array values → validate `type`/`id`/`sender`/`timestamp` → self-connection guard (drop messages whose `sender` is this node) → dedup against `seenIds` → per-peer rate limit → signature verification → emit `message` (and type-specific events) → relay to all peers except the source connection and the original `sender`. +**Inbound processing order** (in `PeerManager.handleInbound`): JSON parse → reject non-object/array values → validate `type`/`id`/`sender`/`timestamp` → self-connection guard (drop messages whose `sender` is this node) → dedup against `seenIds` → per-peer rate limit → signature verification → emit `message` (and type-specific events) → relay to all peers except the source connection and the original `sender`. ### Message Types @@ -301,7 +301,7 @@ If the leader fails to drive consensus within `PBFT_TIMEOUT` (default 30s): The quorum rule is activation-gated, keyed on the round's BTC-anchored snapshot block and network. `PREPARE`, `COMMIT` and `PBFT_VIEW_CHANGE` all use the same predicate -(`Consensus._quorumMet`), as do the checkpoint and cross-chain engines. +(`Consensus.quorumMet`), as do the checkpoint and cross-chain engines. **At or above `STAKE_WEIGHTED_QUORUM_ACTIVATION`:** stake-weighted and source-deduplicated. Each voting validator's signing pubkey resolves to its stake source in the federation @@ -499,7 +499,7 @@ Three offense types are monitored: | `repeated_deviation` | 3+ in 24 hours | Three or more deviations within a rolling 24-hour window | | `non_participation` | 30+ missed rounds | `SLASH_MISSED_ROUNDS_THRESHOLD` consecutive rounds without a submission | -Detection is recorded in the `slash_proposals` table. Actual stake slashing is executed by the indexer, not the hub. +Detection is recorded in the `slash_proposals` table for governance review. All three offenses are hub-local: the strongest outcome of a governance vote is `validators.status='suspended'`, which excludes the validator from PBFT rounds and leaves on-chain stake untouched. Stake is burned only when the indexer processes a permissionless SLASH proof of equivocation, which no offense in this table produces. See [Decentralization](decentralization.md) for the three penalty lanes. --- diff --git a/components/hub/configuration.md b/components/hub/configuration.md index b2c9a964..e015c928 100644 --- a/components/hub/configuration.md +++ b/components/hub/configuration.md @@ -224,6 +224,11 @@ Caps on the live-update channel indexers subscribe to. See [API](api.md#get-hub- | `WS_BACKPRESSURE_LIMIT` | No | `50` | Buffered messages a slow subscriber may accumulate before its connection is dropped | | `WS_WATERMARK_INTERVAL_MS` | No | `10000` | Interval between `watermark` heartbeats, which let a subscriber tell "the mirror is behind" apart from "no rows are being produced" | | `WS_WATERMARK_LATE_FACTOR` | No | `2` | Multiple of `WS_WATERMARK_INTERVAL_MS` after which a heartbeat gap counts as late and is logged; `getWatermarkStats()` exposes the tally on `/health`. A value below `1` would mark an exactly-on-time tick late, so anything under `1` falls back to the default rather than raising permanent false alarms | +| `XDEX_ROUND_TIMEOUT_MS` | No | `120000` | The single-round timeout of the cross-chain consensus rail (matches, calls, bridge transfers and policy snapshots), in milliseconds. The admission watermark reads it to size that rail's default terminal bound. A zero, negative or non-numeric value falls back to the default. | +| `XDEX_ROUND_MAX_LIFETIME_MS` | No | `4 × XDEX_ROUND_TIMEOUT_MS` | The terminal bound of a cross-chain round, in milliseconds. A view change re-arms the single-round timeout on a round that is still open, so the watermark trails by this lifetime rather than by one timeout. Widen it together with the rail's rounds. | +| `ADMISSION_ORACLE_INGEST_WINDOW_MS` | No | `600000` | How far the hub's ingest of on-chain `PRICE` v1 rows may trail the chain it reads before the admission watermark stops waiting for it, in milliseconds. The oracle price table has no consensus round, so this is its only bound. | +| `HUB_ADMISSION_RELAY` | No | _(unset)_ | Set to `1` or `true` on a relay hub that serves a mirrored copy of another hub's database. A relay observes no rounds, so it claims no admission height of its own: it republishes its upstream's entry verbatim or none, and its indexers defer fail-closed. | +| `ADMISSION_WATERMARK_SAMPLE_MS` | No | `30000` | How often the hub samples the admission height watermark it publishes on these frames, in milliseconds. Sampling starts only once a hub source is attached. | ### Indexer tip freshness @@ -237,6 +242,7 @@ The hub reads the BTC chain tip to anchor consensus rounds. These gates stop a s | `MAX_INDEXER_LAG_BLOCKS` | No | `200` | Maximum blocks the BTC indexer may lag before its tip is treated as untrustworthy and ignored, degrading gracefully instead of locking in a stale validator set. | | `MAX_TIP_AGE_S` | No | `2 × ORACLE_ROUND_INTERVAL` (seconds) | Maximum age of the indexer-pushed BTC tip before it is considered stale. Rejecting it costs one HTTP call: the hub falls through to a direct `getlatestblock`. | | `MAX_DIRECT_TIP_AGE_S` | No | `7200` (seconds) | Age at which the hub stops trusting a direct `getlatestblock` height that has **not** advanced past the pushed tip just rejected, and reports no BTC tip at all. Separate from `MAX_TIP_AGE_S` on purpose: this gate is terminal, so its bound is sized so an ordinary long block gap on a healthy chain never trips it. A height that beats the pushed tip is always accepted, whatever the tip's age. | +| `ADMISSION_TIP_MAX_AGE_S` | No | `6 × the chain's block interval` (seconds) | How long a chain's decoder tip may stay at one height before the hub treats it as stalled when resolving an admission height. The default is a block count, so it scales with each chain: about an hour on BTC, 15 minutes on LTC and 6 minutes on DOGE. A zero, negative or non-numeric value uses the default. | | `INDEXER_COIN_CHECK` | No | enabled | Set to `0` to disable the per-coin indexer reachability check. | ### Oracle @@ -304,8 +310,8 @@ Controls `OraclePublisher`, which broadcasts finalized price rounds on-chain as | Variable | Required | Default | Description | |---|---|---|---| | `ORACLE_REWARD_PER_ROUND` | No | `"10.00000000"` | XCHAIN distributed per finalized oracle round | -| `SLASH_DEVIATION_THRESHOLD` | No | `"0.05"` | Price deviation threshold (5%) for slash detection | -| `SLASH_MISSED_ROUNDS_THRESHOLD` | No | `"30"` | Consecutive missed rounds before non-participation slash | +| `SLASH_DEVIATION_THRESHOLD` | No | `"0.05"` | Price deviation (5%) at which the hub records a `price_deviation` offense. Hub-local: governance can suspend the validator, on-chain stake is untouched | +| `SLASH_MISSED_ROUNDS_THRESHOLD` | No | `"30"` | Missed rounds at which the hub records a `non_participation` offense. Hub-local: governance can set `validators.status='suspended'`, on-chain stake is untouched. Only a permissionless SLASH proof of equivocation burns stake (see [Decentralization](decentralization.md)) | | `REWARD_PUSH_MAX_ATTEMPTS` | No | `3` | Attempts `RewardTracker` makes when pushing a validator-reward record to the indexer before giving up and recording the failure. The push was previously fire-and-forget, so a dropped push lost the reward record silently. | | `REWARD_PUSH_RETRY_DELAY_MS` | No | `2000` | Delay (ms) between those attempts. | @@ -320,7 +326,7 @@ Controls `StateAnchorPublisher` (commits checkpoints and the cross-chain match a | `ANCHOR_CHUNK_RETRY_MS` | No | `2500` | Delay before retrying a failed archive chunk upload (ms) | | `ANCHOR_ELECTION_TOLERANCE_BLOCKS` | No | `36` | BTC blocks a non-leader hub waits before the next eligible rank may take over | | `ANCHOR_REWARD_PER_PUBLISH` | No | `"10.00000000"` | XCHAIN distributed to the elected ANCHOR publisher per successful publish cycle | -| `ANCHOR_CHECKPOINT_EVERY_N` | No | `1` | Anchor only every Nth `checkpoint_seq` on-chain (per chain). Decouples on-chain ANCHOR spend from checkpoint production cadence: skipped (off-multiple) seqs remain in the off-chain hub-DB mirror and are still verifiable via the explorer. `1` anchors every checkpoint (original behaviour). | +| `ANCHOR_CHECKPOINT_EVERY_N` | No | `1` | Anchor only every Nth checkpoint **ordinal** on-chain, gating the whole round (one bundle covers every chain). Eligibility is `FLOOR(checkpoint_seq / CHECKPOINT_INTERVAL_BLOCKS) % N`, not `checkpoint_seq % N`: the cadence latch advances the seq by exactly one interval per round, so a raw-seq test is a residue class pinned by the seed rather than a 1-in-N sample, and for any N sharing a factor with the interval the federation would anchor every round or never anchor at all. Both this and `CHECKPOINT_INTERVAL_BLOCKS` must be fleet-uniform; `checkpoint_seq` is consensus data, so the predicate is deterministic across every hub. It decouples on-chain ANCHOR spend from checkpoint production cadence, it is not a cadence control (`ANCHOR_INTERVAL_MS` is): skipped (off-multiple) ordinals remain in the off-chain hub-DB mirror and are still verifiable via the explorer. `1` anchors every checkpoint (original behaviour). | | `ANCHOR_ENABLED` | No | `true` | Set to `false` to stop this hub publishing ANCHORs. | | `ANCHOR_MAX_BATCH` | No | `1000` | Maximum `cross_chain_matches` rows drained into one publish cycle. | | `ANCHOR_CHUNK_MAX_BYTES` | No | `6000` | Maximum payload bytes per ANCHOR archive chunk. | @@ -467,6 +473,8 @@ Controls `RollcallRound`, which signs the per-epoch ledger-hash roll call and el | `ROLLCALL_ELECTION_TOLERANCE_BLOCKS` | No | `36` (regtest `3`) | Blocks the elected publisher is given before the next hub in the election ladder may take over. Separate from `ANCHOR_ELECTION_TOLERANCE_BLOCKS` on purpose: the two ladders climb against different anchors. | | `ROLLCALL_SELF_PUBLISH_BLOCKS` | No | `100` (regtest `9`) | Blocks after which any hub still holding an unpublished epoch publishes it itself, whatever the ladder says. | +**Without `DOGE_INDEXER_URL` / `DOGE_INDEXER_API_URL` this hub cannot tell a signature it holds from one already on chain, so it publishes nothing and logs nothing.** That failure is silent: the round leader still sees this hub's own oracle submissions arrive, but no roll call ever lands with this hub's pair in it, and a validator left in that state for two consecutive rolled epochs is evicted. A validator with no Dogecoin indexer of its own points these at the public explorer's replicated read: `DOGE_INDEXER_API_URL=https://explorer.xchain.io/TDOGE/api/` on testnet, `https://explorer.xchain.io/DOGE/api/` on mainnet, with `DOGE_INDEXER_API_KEY` set to the federation read key issued alongside this validator's other per-coin keys. That answer comes from the explorer's own replica, so a replica that has fallen behind makes the round wait longer rather than publish against stale data. + ### Full-Node Challenge Controls `FullNodeChallengeRound`, the periodic possession challenge proving a validator runs a real coin full node rather than mirroring the decoder and indexer databases. Feeds the full-node verified reward tier and the on-chain `NODEPROOF` action. @@ -513,7 +521,8 @@ The XCHAIN/USD price is derived from platform-realized fills rather than an exte The four derivation parameters below are **consensus-uniform**, not per-operator tuning. Every validator has to compute the same window over the same fills, so a hub honoring a local override would produce a different XCHAIN/BTC leg, land -outside the co-sign deviation band, and expose itself to slashing. They are +outside the co-sign deviation band, and expose itself to a recorded +`price_deviation` offense and hub-local suspension. They are therefore **honored on regtest only**: on mainnet and testnet the hub logs a `set but IGNORED` warning and uses the consensus-pinned value regardless of what the environment says, and so does a standalone hub with no `HUB_NETWORK`. @@ -541,13 +550,13 @@ Backs the `ATTEST` path where a contract asks an approved model a question. See | `LLM_SPEND_LOG_PATH` | No | `./data/llm-spend.jsonl` | File the provider appends each spend record to, written before the call so the audit trail cannot be lost to a crash mid-request. | | `LLM_SPEND_LOG_FALLBACK_PATH` | No | `llm-spend.jsonl` inside the OS temp directory | Where a per-dispatch LLM spend audit line is written when the primary sink (`LLM_SPEND_LOG_PATH`) cannot be written. The aggregate spend-state file cannot stand in for it: that file carries a rolling window of costs and no per-dispatch identity, so an operator reconciling a vendor invoice against it cannot tell which call was which. | -> **Cost note.** Each on-chain checkpoint anchor spends real DOGE on three transactions (BTC + LTC + DOGE checkpoints all broadcast on the DOGE chain). State recovery (`recovery.js`) only needs the **latest** anchored checkpoint per chain, so anchoring every intermediate `checkpoint_seq` is optional. With daily checkpoints (`CHECKPOINT_INTERVAL_BLOCKS=144`), `ANCHOR_CHECKPOINT_EVERY_N=2` halves anchor spend (on-chain recovery point then trails the tip by up to ~2 checkpoint intervals). `checkpoint_seq` is consensus data, so the gate is deterministic across every hub. +> **Cost note.** Each on-chain checkpoint anchor spends real DOGE on **one bundle per network**, not one transaction per chain: a single ANCHOR v0 carries BTC, LTC and DOGE as sections of the same payload, broadcast on the DOGE chain over the P2SH lane's funding plus reveal pair. The cost therefore scales with encoded payload bytes at the venue's fee rate rather than with the number of chains checkpointed. State recovery (`recovery.js`) only needs the **latest** anchored checkpoint per chain, so anchoring every intermediate `checkpoint_seq` is optional. With daily checkpoints (`CHECKPOINT_INTERVAL_BLOCKS=144`), `ANCHOR_CHECKPOINT_EVERY_N=2` halves anchor spend (on-chain recovery point then trails the tip by up to ~2 checkpoint intervals). `checkpoint_seq` is consensus data, so the gate is deterministic across every hub. ### Operator Signer | Variable | Required | Default | Description | |---|---|---|---| -| `HUB_SIGNER_MODULE` | No | None | Path to a CommonJS module exporting `walletSign(psbtHex) → Promise`. Used by `OraclePublisher` and `AttestationPublisher` to sign DOGE transactions; `StateAnchorPublisher` borrows the same hooks via `_resolveSigner()`. Optional: without it the publishers stay idle. Set-but-unloadable throws at startup (fail loudly). Falls back to `setWalletSignHook` / `setBroadcastHook` if the module is not provided. | +| `HUB_SIGNER_MODULE` | No | None | Path to a CommonJS module exporting `walletSign(psbtHex) → Promise`. Used by `OraclePublisher` and `AttestationPublisher` to sign DOGE transactions; `StateAnchorPublisher` borrows the same hooks via `resolveSigner()`. Optional: without it the publishers stay idle. Set-but-unloadable throws at startup (fail loudly). Falls back to `setWalletSignHook` / `setBroadcastHook` if the module is not provided. | ### Cross-Chain @@ -564,8 +573,11 @@ Backs the `ATTEST` path where a contract asks an approved model a question. See | `XCHAIN_ATTEST_FINALIZED_MAX` | No | `10000` | Cap on retained finalized cross-chain attestation records held in memory | | `XCHAIN_ATTEST_STORE_RETRIES` | No | `4` | Attempts made when persisting a cross-chain attestation record. The INSERT is idempotent (`ON DUPLICATE KEY UPDATE`), so a retry after a partial failure is safe. | | `XCHAIN_ATTEST_STORE_RETRY_MS` | No | `100` | Base backoff (ms) between those attempts. | +| `XBRIDGE_POLL_MS` | No | `15000` | Poll cadence of the cross-chain bridge engine (`CrossChainBridgeEngine`), which signs `bridge_transfers` and `policy_snapshots` rows. | +| `_INDEXER_URL` | No | _(from config table)_ | Per-coin indexer JSON-RPC URL the cross-chain bridge engine polls for confirmed transfer and policy legs (e.g. `BTC_INDEXER_URL`). Shared knob name with the other per-coin indexer reads on this page; falls back to the config table, then an empty string. | +| `_INDEXER_API_KEY` | No | _(from config table)_ | API key presented to that indexer by the cross-chain bridge engine. Treat as a credential. | -**Regtest-only seams.** Both engines honour these only when the hub's network is `regtest`, and read them as `NaN`/false everywhere else, so a stray environment variable or config row can never reach the signed snapshot anchor or seed a validator on mainnet or testnet. They deliberately share names between the DEX and XCALL engines so a no-BTC regtest stack is configured once. +**Regtest-only seams.** All three engines honour these only when the hub's network is `regtest`, and read them as `NaN`/false everywhere else, so a stray environment variable or config row can never reach the signed snapshot anchor or seed a validator on mainnet or testnet. They deliberately share names between the DEX, XCALL and bridge engines so a no-BTC regtest stack is configured once. | Variable | Required | Default | Description | |---|---|---|---| @@ -584,6 +596,7 @@ Regtest-only genesis overrides, ignored on mainnet and testnet, which always use | `XC_ROLLCALL_REGTEST_ACTIVATION` | Regtest only | unset (inert) | Arms ROLLCALL on a private regtest venue, so this hub signs roll calls and elects publishers there. `armed` (or `genesis`/`on`/`true`/`yes`) activates at BTC height `0`; a bare non-negative integer activates at that height; `off`/`inert`/`false` and anything unrecognised leave it inert. Read once at startup, so a change needs a restart. Ignored on mainnet and testnet, whose heights are fixed in source and unreachable from the environment. | | `XC_ROLLCALL_GATES_REGTEST_ACTIVATION` | Regtest only | unset (inert) | Arms ROLLCALL v1 on a private regtest venue: from the armed epoch height on, this hub publishes roll calls that carry the consensus gates its build knows, and the attestation capability set drops a validator whose recorded list lacks a rule active at the request block. Same grammar as `XC_ROLLCALL_REGTEST_ACTIVATION` (`armed` at height `0`, a bare integer at that height, anything else inert), read once at startup, and separate from the roll-call rail so a rail-armed venue can still drive v0 roll calls as its control. Ignored on mainnet and testnet, whose heights are fixed in source. | +| `XC_MIRROR_ADMISSION_ACTIVATION` | Regtest only | unset (inert) | Arms the per-coin `regtest` entries of `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION` (the mirror-admission heights) and the `regtest` entry of `ANCHOR_ATTEST_BARRIER_ACTIVATION` in the hub's activation registry, one variable for the whole barrier family. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; the armed form arms at height `0`. Applied when a row is read, from the environment as it stands then; set identically on every hub, indexer, sync and explorer process in the venue. Ignored on mainnet and testnet, whose heights are fixed in source. | ROLLCALL arming is a **venue-wide** setting: set `XC_ROLLCALL_REGTEST_ACTIVATION` (and, when the gates rail is wanted, `XC_ROLLCALL_GATES_REGTEST_ACTIVATION`) identically on every hub and every BTC indexer in the venue, and wire the indexers' `DOGE_INDEXER_API_URL`. Regtest ships inert because arming a network commits every BTC indexer on it to a wired DOGE peer, and a single-coin BTC venue would defer forever at its first epoch close. A venue that arms its hubs and forgets an indexer surfaces as a consensus-rules digest mismatch rather than as silent disagreement about which epochs exist. @@ -619,6 +632,9 @@ Read-only operator tools; neither broadcasts nor writes anything and neither is |---|---|---|---| | `HUB_RPC_URL` | No | `http://127.0.0.1:4000` | Hub JSON-RPC base URL `bin/stake-share-drill.js` queries (`getstakeshare`) when no `--hub` flag is given. The drill reports how much more third-party stake the federation can absorb before the stake-weighted quorum commit gate stops being reachable, and what a stake of a given size would do to that margin; used to size a top-up before putting real stake on the network. | | `HUB_RPC_URLS` | No | _(empty; `--hubs` required instead)_ | Comma-separated hub JSON-RPC URLs `bin/oracle-round-presence.js` polls (`getoracleroundpresence`) when no `--hubs` flag is given. Asks every named hub about the same round range and reports whether the federation agrees on which rounds happened, so a round that finalized on some validators and not others shows up as a named divergence instead of looking like ordinary absence. At least two URLs are required; comparing one hub to itself is refused. | +| `XCHAIN_HUB_DIR` | No | `../xchain-hub` | Sibling-checkout override `bin/lib/carrier_logic_pin.js`'s `siblingDir()` resolves for cross-repo carrier-logic comparison; the `repo_guards` twin test points it at a second checkout with `XCHAIN_REQUIRE_SIBLINGS=1`. Never read by the running hub process. | +| `XCHAIN_INDEXER_DIR` | No | `../xchain-indexer` | Sibling-checkout override for the `xchain-indexer` tree the same `siblingDir()` resolves when the pin tool compares against the indexer's canonical copy; read by literal name so the coverage gate can see it. Never read by the running hub process. | +| `XCHAIN_SYNC_DIR` | No | `../xchain-sync` | Sibling-checkout override for the `xchain-sync` tree the same `siblingDir()` resolves when the pin tool compares against sync's copy; read by literal name so the coverage gate can see it. Never read by the running hub process. | ## Database Schema diff --git a/components/hub/database.md b/components/hub/database.md index 1c584c71..200cd5cf 100644 --- a/components/hub/database.md +++ b/components/hub/database.md @@ -3,7 +3,7 @@ # XChain Platform Hub: Database Schema -The hub uses a single MariaDB database (e.g., `XChain_Hub`) for all state. The database and all tables are auto-created on first startup. SQL schema files live in `src/sql/*.sql` and are loaded by `db.js`. +The hub uses a single MariaDB database (e.g., `XChain_Hub`) for all state. The database and all tables are auto-created on first startup. SQL schema files live in `src/sql/*.sql` and are loaded by `src/db/index.js`. ## Config Tables @@ -183,6 +183,8 @@ The durable at-most-once marker for PRICE v0 round broadcasts, written by `Oracl | `reorg_attestations` | Confirmed blockchain reorg events | | `cross_chain_matches` | PBFT-finalized DEX order match records (dispatch to indexers via hub-DB mirror) | | `cross_chain_calls` | PBFT-finalized XCALL dispatch and result records (relay to indexers via hub-DB mirror) | +| `bridge_transfers` | PBFT-finalized bridge transfer records: one signed row per confirmed lock or burn, carrying `tick`, `decimals` and `amount`, from which the destination indexer injects the XBRIDGE settle leg (mirror to indexers) | +| `policy_snapshots` | PBFT-finalized per-token policy snapshots (allow list, block list, tick sleep) a destination chain materializes onto a bridged copy. Append-only, latest-wins by `policy_seq`, on the `state_checkpoints` terms (mirror to indexers) | ### `attestations` @@ -452,7 +454,7 @@ Tracks XCHAIN rewards earned by validators for participating in oracle rounds. R ### `slash_proposals` -Records detected validator misbehavior for governance review. The hub detects violations but does not execute slashing directly, actual slashing occurs via the indexer's staking contract. +Records detected validator misbehavior for governance review. The offenses recorded here (`price_deviation`, `repeated_deviation`, `non_participation`, attestation divergence) are hub-local: a governance vote can suspend the validator, and on-chain stake is untouched. Stake is burned only when the indexer processes a permissionless SLASH proof of equivocation, which is a separate on-chain path this table does not feed. | Column | Type | Description | |---|---|---| diff --git a/components/hub/operations.md b/components/hub/operations.md index bf1be8a1..f4dd075b 100644 --- a/components/hub/operations.md +++ b/components/hub/operations.md @@ -32,7 +32,7 @@ On startup, the hub: - Cross-chain attestation engine - Reorg handler - Governance engine - - Reward tracker (pushes rewards to BTC indexer) and slash detector + - Reward tracker (pushed anchor rewards to the BTC indexer below the anchor-reward flag-days; at or above them each indexer derives them from the ANCHOR bytes) and slash detector - `StateCheckpointEngine` (quorum-signs per-chain ledger/actions/contract hash checkpoints; streams to hub DB subscribers) - `StateAnchorPublisher` (one publisher election per bundle; commits every chain's checkpoint in one DOGE ANCHOR v0 action per network, plus the match archive, on the `ANCHOR_INTERVAL_MS` cadence) @@ -445,7 +445,7 @@ The ANCHOR publisher logs `StateAnchorPublisher: DOGE balance low` and skips pub - Check the DOGE wallet balance at the address configured in `capabilities.json` under `oracle_publish.doge_address`. - Refill the wallet to resume publishing. Once funded, either wait for the next `ANCHOR_INTERVAL_MS` cycle or force an immediate flush with `anchorflush` (see above). -- **Cost / runway.** Each anchor *round* broadcasts one transaction per chain (BTC + LTC + DOGE checkpoints, all on the DOGE chain) at ~0.4 DOGE/tx ≈ ~1.2 DOGE/round, plus the archive transaction(s) when there is cross-chain activity. With daily checkpoints (`CHECKPOINT_INTERVAL_BLOCKS=144`) that is ~1.2 DOGE/day. To cut spend, raise `ANCHOR_CHECKPOINT_EVERY_N` (see CONFIGURATION.md → ANCHOR Publishing): `=2` anchors every other checkpoint → ~0.6 DOGE/day. Size a comfortable refill at roughly `daily_cost × desired_days` (e.g. ~60 DOGE ≈ 100 days at `EVERY_N=2`). +- **Cost / runway.** Each anchor *round* broadcasts **one checkpoint bundle per network**, not one transaction per chain: a single ANCHOR v0 carries BTC, LTC and DOGE as sections of the same payload (see [ANCHOR](../../protocol/actions/anchor.md)), and it rides the P2SH lane, so the bundle is a funding transaction plus a reveal transaction. Pending cross-chain matches add the v1 archive head and its v2 continuation chunks on top, and on a busy cycle the archive leg dominates. Fees are **byte-driven**: they scale with the encoded payload at the venue's current fee rate, not with the number of chains, so bundling the three per-chain anchors that this layout replaced saved per-transaction overhead only, not a multiple. Get your own number rather than trusting a constant: read the fee actually paid by one cycle's anchor transactions on your venue, multiply by cycles per day (`ANCHOR_INTERVAL_MS`), then size a refill at roughly `daily_cost × desired_days` with enough margin to stay clear of the low-balance threshold, since the publisher skips publishing entirely while the wallet sits below it. To cut spend, raise `ANCHOR_CHECKPOINT_EVERY_N` (see CONFIGURATION.md → ANCHOR Publishing): it gates the whole round, so `=2` roughly halves checkpoint-leg spend at the price of an on-chain recovery point that trails the tip by up to two checkpoint intervals. - **Restarts are free** as of the cadence-latch fix; a hub restart restores the checkpoint cadence latch from the last persisted checkpoint and no longer fires an extra (DOGE-spending) off-schedule anchor. Look for `StateCheckpointEngine: cadence latch restored at snapshot block N` in startup logs to confirm. ### Consumers not discovering hub diff --git a/components/indexer/actions.md b/components/indexer/actions.md index 78d096b9..066e5ff1 100644 --- a/components/indexer/actions.md +++ b/components/indexer/actions.md @@ -16,9 +16,9 @@ An action is only processed if: 2. The current block time is >= the action's activation timestamp for the active network 3. The current block height is >= the action's activation block for the active network -21 actions are registered at version `0.1.0` (ADDRESS, AIRDROP, BATCH, BET, BROADCAST, CALLBACK, COINPAY, DESTROY, DISPENSER, DIVIDEND, FILE, ISSUE, LINK, LIST, MESSAGE, MINT, ORDER, SEND, SLEEP, SWAP, SWEEP) and the other 16 at `0.2.0` (the Virtual Machine, Hub Staking, Oracle, Governance, and Validator categories). The two derived rows DISPENSE and COINPAY_EXPIRE are also registered at `0.1.0`. +21 actions are registered at version `0.1.0` (ADDRESS, AIRDROP, BATCH, BET, BROADCAST, CALLBACK, COINPAY, DESTROY, DISPENSER, DIVIDEND, FILE, ISSUE, LINK, LIST, MESSAGE, MINT, ORDER, SEND, SLEEP, SWAP, SWEEP) and the other 17 at `0.2.0` (the Virtual Machine, Hub Staking, Oracle, Governance, Cross-Chain Bridge, and Validator categories). The two derived rows DISPENSE and COINPAY_EXPIRE are also registered at `0.1.0`. -**All 37 actions carry an activation block and timestamp of `0` on every network**, so what gates them is condition 1 above, the indexer's own version, not a height. Non-zero activation values are used by the ~34 *behaviour* changes registered alongside the actions (`ISSUANCE_FEE`, `CONTROLLER_GUARD`, `VM_BANNED_ASYNC`, `CROSS_CHAIN_ROYALTY`, and so on): those are the block-height and timestamp flag-days, and they change how an already-live action behaves rather than introducing a new one. Future protocol upgrades can do either. +**All 38 actions carry an activation block and timestamp of `0` on every network**, so what gates them is condition 1 above, the indexer's own version, not a height. Non-zero activation values are used by the ~34 *behaviour* changes registered alongside the actions (`ISSUANCE_FEE`, `CONTROLLER_GUARD`, `VM_BANNED_ASYNC`, `CROSS_CHAIN_ROYALTY`, and so on): those are the block-height and timestamp flag-days, and they change how an already-live action behaves rather than introducing a new one. Future protocol upgrades can do either. ## Token Lifecycle Actions @@ -135,7 +135,7 @@ Two staking systems share the same four action names. **Capability staking** (ST | **STAKE** | Lock tokens against a signing pubkey. v1 = new capability stake (XCHAIN), v2 = top-up of existing capability stake (XCHAIN), v3 = contract-targeted stake (any token, targets a stakeable contract: see DEPLOY v1) | VERSION valid (1/2/3), AMOUNT positive, SIGNING_PUBKEY is 64-char hex Ed25519. v1/v2: aggregate per-pubkey active stake auto-qualifies the pubkey for each of five capabilities (`price`, `cross_chain`, `oracle_publish`, `attestation`, `full_node`) based on governance `min_stake[capability]`. v3: target contract must be stakeable; row keyed by `(target, pubkey, tick, source)`. | | **UNSTAKE** | Release staked tokens. v0 = full-pubkey capability unstake. v1 = release a single contract-targeted row keyed by `(target, pubkey, tick)`. | Pubkey has active stake of matching type; sets `deactivation_block`. v1 cooldown is per-contract (set at DEPLOY v1 time); v0 uses the global `STAKING.COOLDOWN_BLOCKS`. | | **DELEGATE** | Manage the signing key for a stake. v0 = capability rotate, v1 = contract rotate, v2 = capability revoke, v3 = contract revoke. | Active stake/delegation of matching type exists. For rotates, new pubkey valid and unused. Takes effect after the activation delay: 6 blocks for capability rotate/revoke (v0/v2); the per-chain delay (6 blocks on BTC, 24 on LTC, 60 on DOGE) for contract rotate/revoke (v1/v3). | -| **COLLECT** | Collect accumulated rewards | Address has unclaimed rewards > 0. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rewards are derived by the indexer during block processing; `anchor_bundle` and `anchor_archive` rewards are pushed from `xchain-hub` via `pushvalidatorrewards`. | +| **COLLECT** | Collect accumulated rewards | Address has unclaimed rewards > 0. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rewards are derived by the indexer during block processing; `anchor_bundle` and `anchor_archive` rewards are derived by the indexer from the on-chain ANCHOR bytes at or above their own flag-days (`ANCHOR_REWARD_ACTIVATION` / `ARCHIVE_REWARD_ACTIVATION`); below those heights they were pushed from `xchain-hub` via `pushvalidatorrewards`, which is retired for new anchor rewards. | ### `oracle_publish` capability (formerly "Tier 3") @@ -165,7 +165,7 @@ Virtual Machine actions are available on **all chains** (BTC, LTC, DOGE). DEPLOY | Action | Purpose | Key Validations | |---|---|---| -| [**ANCHOR**](../../protocol/actions/anchor.md) | Commit quorum-signed state checkpoints and a compressed archive of cross-chain match rows on-chain. DOGE-only, validator-broadcast action. | Valid only on the DOGE chain (all networks). No protocol fee. On parse, the indexer writes to `anchor_actions` and records checkpoint hashes. The archived data makes all platform state recoverable from a full chain re-parse. See `src/actions/anchor.js` and `protocol/actions/ANCHOR.md`. | +| [**ANCHOR**](../../protocol/actions/anchor.md) | Commit quorum-signed state checkpoints and a compressed archive of cross-chain match rows on-chain. DOGE-only, validator-broadcast action. | Valid only on the DOGE chain (all networks). No protocol fee. On parse, the indexer writes to `anchor_actions` and records checkpoint hashes. The archived data makes all platform state recoverable from a full chain re-parse. See `src/actions/anchor/index.js` and `protocol/actions/ANCHOR.md`. | ## Attestation & Validator Actions diff --git a/components/indexer/architecture.md b/components/indexer/architecture.md index 9f596192..0c937d0c 100644 --- a/components/indexer/architecture.md +++ b/components/indexer/architecture.md @@ -90,7 +90,7 @@ The indexer's API also exposes a write endpoint that the hub calls: | Method | Sent By | Purpose | |---|---|---| -| `pushvalidatorrewards` | hub `RewardTracker` | Pushes `anchor_bundle` and `anchor_archive` reward rows from the hub to the indexer. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rewards are rejected by this endpoint; they are derived deterministically by the indexer during block processing and do not need to be replicated. | +| `pushvalidatorrewards` | hub `RewardTracker` | Pushes `anchor_bundle` and `anchor_archive` reward rows from the hub to the indexer. **Retired for new anchor rewards:** at or above `ANCHOR_REWARD_ACTIVATION` (for `anchor_bundle`) and `ARCHIVE_REWARD_ACTIVATION` (for `anchor_archive`) the indexer derives the reward from the on-chain ANCHOR bytes instead, so the endpoint carries pre-flag-day history only. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rewards are rejected by this endpoint; they are derived deterministically by the indexer during block processing and do not need to be replicated. | ## VM Runtime Module @@ -109,7 +109,7 @@ flowchart TD end ``` -The indexer's `execute.js` handler bridges the VM and the database: +The indexer's `execute/index.js` handler bridges the VM and the database: 1. Loads contract code and state from the DB 2. Calls `vm.execute()`, receives results 3. Writes state changes via `createContractState()` (append-only) @@ -135,29 +135,29 @@ The VM maintains a per-block cache of V8 compiled script data (`beginBlock()`/`e |---|---|---| | `src/api.js` | None | Entry point: Express server + JSON-RPC, env var validation, indexer startup | | `src/XChainIndexer.js` | `XChainIndexer` | Main orchestrator: block polling loop, reorg detection, block processing pipeline | -| `src/actions.js` | `Actions` | Loads all 48 action handler classes (one per routable ACTION string, including the `UNKNOWN` fallback), routes transactions to the correct handler. The internal `deploy_chunk` sub-handler is loaded by `deploy.js`, not here | -| `src/db.js` | `Database` | MariaDB connection pool management, all SQL queries, table creation, sanity checks | +| `src/actions/index.js` | `Actions` | Loads all 48 action handler classes (one per routable ACTION string, including the `UNKNOWN` fallback), routes transactions to the correct handler. The internal `deploy_chunk` sub-handler is loaded by `deploy/index.js`, not here | +| `src/db/index.js` | `Database` | MariaDB connection pool management, table creation, sanity checks; the SQL queries live in the per-table modules under `src/db/` | | `src/config.js` | None | Merges environment variables with coin-specific config into a single config object | -| `src/configs/BTC.js` | None | Bitcoin-specific: fee schedules, BURN/GAS/DONATE addresses per network | -| `src/configs/LTC.js` | None | Litecoin-specific configuration | -| `src/configs/DOGE.js` | None | Dogecoin-specific configuration | +| `src/coins/BTC.js` | None | Bitcoin-specific: fee schedules, BURN/GAS/DONATE addresses per network | +| `src/coins/LTC.js` | None | Litecoin-specific configuration | +| `src/coins/DOGE.js` | None | Dogecoin-specific configuration | | `src/utility.js` | `Utility` | BigNumber math, timer functions, expiration/cancellation processing, ledger operations, cross-chain settlement injection | -| `src/mapper.js` | `Mapper` | Creates action_index ↔ address/tick cross-reference mappings | -| `src/rollback.js` | `Rollback` | Handles blockchain reorganizations: deletes affected records, recalculates balances | +| `src/chain/mapper.js` | `Mapper` | Creates action_index ↔ address/tick cross-reference mappings | +| `src/rollback/index.js` | `Rollback` | Handles blockchain reorganizations: deletes affected records, recalculates balances | | `src/protocol_changes.js` | `ProtocolChanges` | Defines supported actions and their activation rules (version, block, timestamp) | -| `src/health.js` | None | Assembles the `health` JSON-RPC response payload; separate from `api.js` so it can be unit-tested without a database | -| `src/hub_client.js` | `HubClient` | Lightweight JSON-RPC client for pushing chain tip, PRICE rounds, and price retractions to `xchain-hub`; uses Node built-in `http`/`https` | -| `src/hub_db_sync.js` | `HubDbSync` | Bootstraps and live-syncs the local hub DB mirror (price snapshots, oracle prices, capability snapshots, cross-chain matches) via REST snapshot + WebSocket | -| `src/hub_push_queue.js` | `HubPushQueue` | Durable retry queue for PRICE pushes to the hub; backs the `pending_hub_pushes` table | -| `src/ed25519.js` | None | Ed25519 signature verification using Node built-in crypto; mirrors `xchain-hub/src/ValidatorIdentity.js` format | -| `src/merkle.js` | None | Consensus-critical SPV light-client Merkle primitives: additive state SMT, per-block content root, fixed top-level state root. Vendored byte-identically into `xchain-sync` | +| `src/api/health.js` | None | Assembles the `health` JSON-RPC response payload; separate from `api.js` so it can be unit-tested without a database | +| `src/hub/hub_client.js` | `HubClient` | Lightweight JSON-RPC client for pushing chain tip, PRICE rounds, and price retractions to `xchain-hub`; uses Node built-in `http`/`https` | +| `src/hub/hub_db_sync.js` | `HubDbSync` | Bootstraps and live-syncs the local hub DB mirror (price snapshots, oracle prices, capability snapshots, cross-chain matches) via REST snapshot + WebSocket | +| `src/hub/hub_push_queue.js` | `HubPushQueue` | Durable retry queue for PRICE pushes to the hub; backs the `pending_hub_pushes` table | +| `src/consensus/ed25519.js` | None | Ed25519 signature verification using Node built-in crypto; mirrors `xchain-hub/src/validators/identity.js` format | +| `src/consensus/merkle.js` | None | Consensus-critical SPV light-client Merkle primitives: additive state SMT, per-block content root, fixed top-level state root. Vendored byte-identically into `xchain-sync` | | `src/stateHash.js` | None | Builds the `state_hash` preimage covering in-place mutations (deactivation stamps, slash debits, status flips, cooldown maturities) that the three standard block hashes cannot see | -| `src/stateCommitment.js` | None | Computes per-block `state_tree_roots` (balances SMT + stakes SMT + state root + block Merkle root) and writes them to the DB | +| `src/state_commitment/index.js` | None | Computes per-block `state_tree_roots` (balances SMT + stakes SMT + state root + block Merkle root) and writes them to the DB | | `src/stake_weighted_quorum.js` | None | Consensus-critical stake-weighted quorum predicate (WI-1). Vendored byte-identically across hub, indexer, explorer, sync, and SDK | -| `src/recovery.js` | None | CLI for rebuilding the cross-chain match mirror from on-chain ANCHOR archive data, with no surviving hub database | +| `bin/recovery.js` | None | CLI for rebuilding the cross-chain match mirror from on-chain ANCHOR archive data, with no surviving hub database | | `src/equivocation_header.js` | None | Builds EQUIV-header canonicals for the WI-2 equivocation slashing protocol, one per engine tag | -| `src/migrate.js` | None | Applies incremental SQL migrations from `src/sql/migrations/` at startup | -| `xchain-vm` (external) | `XChainVM` | Standalone module: V8 isolate sandbox, AST-based gas metering, gateway API; loaded by `actions.js`, called by DEPLOY and EXECUTE handlers | +| `src/db/migration/migrate.js` | None | Operator-initiated CLI that applies pending SQL migrations from `src/sql/migrations/`, including the `manual`-tagged ones startup skips (startup auto-applies only `auto`-tagged migrations). A bare run applies every pending migration; `--file ` scopes the run to named files. There is no `--help` and no dry-run flag | +| `xchain-vm` (external) | `XChainVM` | Standalone module: V8 isolate sandbox, AST-based gas metering, gateway API; loaded by `src/actions/index.js`, called by DEPLOY and EXECUTE handlers | ## Action Handlers (`src/actions/*.js`) @@ -178,7 +178,7 @@ Actions with automatic lifecycle events have companion handlers: | `DISPENSER` | `dispenser_close.js`, `dispenser_expire.js`, `dispense.js` | | `ORDER` | `order_expire.js`, `order_match.js` | | `SWAP` | `swap_expire.js`, `swap_match.js` | -| `SWAP` / `ORDER` (cross-chain legs) | `cross_settle.js` (system-injected per hub-mirrored match; no on-chain transaction) | +| `SWAP` / `ORDER` (cross-chain legs) | `cross_settle/index.js` (system-injected per hub-mirrored match; no on-chain transaction) | Action aliases provide backward compatibility and shorthand: diff --git a/components/indexer/configuration.md b/components/indexer/configuration.md index f47f0fc2..72a61d5c 100644 --- a/components/indexer/configuration.md +++ b/components/indexer/configuration.md @@ -21,8 +21,8 @@ Configuration is loaded from a `.env` file and environment variables. Copy the ` | `INDEXER_DB_NAME` | Indexer database name | `XChain_BTC_Mainnet_Indexer` | | `INDEXER_DB_USER` | Indexer database username | `xchain` | | `INDEXER_DB_PASS` | Indexer database password | `secretpassword` | -| `INDEXER_COIN` | Blockchain to index | `BTC`, `LTC`, or `DOGE` | -| `INDEXER_NETWORK` | Network to index | `mainnet`, `testnet`, or `regtest` | +| `INDEXER_COIN` | Blockchain to index. The service requires it; the measurement script `bin/measure-batch-execute-cost.js` defaults to `BTC` when it is unset. | `BTC`, `LTC`, or `DOGE` | +| `INDEXER_NETWORK` | Network to index. The service requires it; the measurement script `bin/measure-batch-execute-cost.js` defaults to `regtest` when it is unset. | `mainnet`, `testnet`, or `regtest` | ### Optional Variables @@ -44,7 +44,8 @@ Configuration is loaded from a `.env` file and environment variables. Copy the ` | `DB_CONNECT_TIMEOUT` | MariaDB connection timeout in milliseconds | `10000` | | `DB_ACQUIRE_TIMEOUT` | Time to wait for a free pooled connection, in milliseconds | `10000` | | `DB_QUERY_TIMEOUT` | MariaDB query execution timeout in milliseconds | `30000` | -| `MIGRATION_STRICT_CHECKSUM` | Set to `1` to make a schema-checksum mismatch fail closed at startup instead of logging and continuing. Off by default so a diverged schema does not cause a surprise fleet-wide boot failure; the operator path (`node src/migrate.js`) fails closed regardless. | _(unset, non-fatal)_ | +| `MIGRATION_STRICT_CHECKSUM` | Set to `1` to make a schema-checksum mismatch fail closed at startup instead of logging and continuing. Off by default so a diverged schema does not cause a surprise fleet-wide boot failure; the operator path (`node src/db/migration/migrate.js`) fails closed regardless. | _(unset, non-fatal)_ | +| `SHUTDOWN_TIMEOUT_MS` | Hard-exit budget for the SIGTERM/SIGINT drain, in milliseconds. On `docker stop` the indexer stops reporting itself running on `/status`, lets the block loop break at its next block boundary (never mid-transaction), drains the API listener, closes its database pools and exits 0; if that has not finished within the budget it logs the overrun and exits 1 instead of lingering until docker's SIGKILL. The default sits under docker's 10 s stop grace because `xchain-node` issues a bare `docker stop`; raise it for a chain whose blocks take longer to apply. A non-numeric or non-positive value keeps the default. | `8000` | ### Migration compatibility harness @@ -75,6 +76,9 @@ Configuration is loaded from a `.env` file and environment variables. Copy the ` | `DOGE_INDEXER_URL` | DOGE indexer JSON-RPC URL the BTC indexer uses to re-prove that a mirrored anchor reward's DOGE anchor was actually mined (`getanchorconfirmations`), before crediting it. `DOGE_INDEXER_API_URL` takes precedence when both are set. Required on a BTC indexer once the anchor-reward derive flag-day is armed: unset, no reward can be proven and the block defers. | _(unset)_ | | `DOGE_INDEXER_API_KEY` | API key sent as `x-api-key` with that read (`getanchorconfirmations` is a federation-read method on the DOGE indexer). | _(unset)_ | | `ANCHOR_PROOF_TIMEOUT_MS` | Per-request timeout for the DOGE anchor proof read, and for the ROLLCALL signer read below. A timeout is treated as "cannot tell", which defers the block; it is never read as "not mined". | `15000` | +| `_INDEXER_URL` | Origin-chain indexer JSON-RPC URL the bridge settle pass uses to fetch the escrow checkpoint proof for an incoming transfer (`getbridgeescrowproof`), one per origin coin: `BTC_INDEXER_URL` on a DOGE or LTC indexer that receives XCHAIN, and so on. `_INDEXER_API_URL` takes precedence when both are set, then the config key of the same name. Unset, no proof can be fetched and the settle leg stalls on the proof barrier; it is never applied unproven. | _(unset)_ | +| `_INDEXER_API_KEY` | API key sent as `x-api-key` with that proof read. Falls back to the config key of the same name. | _(unset)_ | +| `BRIDGE_PROOF_TIMEOUT_MS` | Per-request timeout for the escrow proof read. A timeout is treated as "cannot tell", which stalls the settle leg on the proof barrier; it is never read as "no proof". | `15000` | | `HUB_SYNC_BARRIER_HOLD_CEILING_S` | How long the block loop may sit deferring at a hub-mirror-completeness barrier before the mirror forces itself to resync: tearing down and reconnecting its hub-DB WebSocket (or, in poll mode, re-kicking the bootstrap directly). A mirror's stream watermark only advances while its bootstrap drain is flagged complete, and nothing else re-arms that flag once a drain has stalled, so a socket can sit open and heartbeating while the mirror certifies nothing, indefinitely; this bounds that wait by a re-drive instead of leaving it unbounded. Purely operational: it opens no barrier and commits no block early, a genuinely-behind mirror keeps deferring after the resync, and the forced resync is rate-limited to once per ceiling window. Seconds; `0` disables it (no forced resync). | `900` (15 min) | | `HUB_SYNC_WATERMARK_STALL_S` | How long the hub mirror's stream watermark may stay frozen while the hub's own heartbeat tip runs ahead of it before the mirror forces a fresh subscribe-then-bootstrap. This is the mirror's own bound, distinct from `HUB_SYNC_BARRIER_HOLD_CEILING_S`, which the block loop drives and only while a block is deferring: heartbeats keep arriving during such a stall, so the transport watchdog stays satisfied while the mirror certifies nothing. Suppressed where a frozen watermark is correct: poll mode, an outstanding hub schema-version mismatch, and a mirror that has not yet certified a first watermark. Operational only: it opens no barrier and commits no block early. Seconds; `0` disables the detector. | `180` | | `HUB_SYNC_WATERMARK_STALL_EXIT_S` | How long after that forced resync the watermark still has to stay frozen before the process logs a named fatal and exits non-zero so its supervisor restarts it (the indexer wires the exit; the explorer's vendored copy logs and re-drives). Sized above a full re-bootstrap drain, so an ordinary slow drain finishes and moves the watermark inside the window; any real advance cancels it. Seconds; `0` keeps the forced resync but never exits. | `300` | @@ -84,11 +88,14 @@ Configuration is loaded from a `.env` file and environment variables. Copy the ` | Variable | Description | Default | |---|---|---| | `XC_ROLLCALL_GATES_REGTEST_ACTIVATION` | **Regtest only.** Arms ROLLCALL v1 on this private venue: from the armed epoch height on, roll calls must carry the publisher's consensus-gate list, the epoch close records each verified signer's list in `rollcall_gates`, and the attestation capability set drops a validator whose recorded list lacks a rule active at the request block. Same grammar as `XC_ROLLCALL_REGTEST_ACTIVATION`; read **once at startup**; set identically on every hub and BTC indexer in the venue. mainnet and testnet are fixed in source. | _(unset: inert)_ | +| `XC_MIRROR_ADMISSION_ACTIVATION` | **Regtest only.** Arms the per-coin `regtest` entries of `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION` (the mirror-admission heights) and the `regtest` entry of `ANCHOR_ATTEST_BARRIER_ACTIVATION` in the indexer's activation registry (`src/protocol_changes/shared_rows.js`), one variable for the whole barrier family. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; the armed form arms at height `0`. Applied when a row is read, from the environment as it stands then; set identically on every hub, indexer, sync and explorer process in the venue. mainnet and testnet are fixed in source. | _(unset: inert)_ | | `XC_ROLLCALL_REGTEST_ACTIVATION` | **Regtest only.** Arms ROLLCALL on this private venue. `armed` (or `genesis`/`on`/`true`/`yes`) activates at BTC height `0`; a bare non-negative integer activates at that height, for a venue whose epochs should begin above an already-indexed prefix; `off`/`inert`/`false` and anything unrecognised leave it inert, and an unrecognised value is logged. Read **once at startup**, so a change needs a restart. mainnet and testnet are fixed in source and cannot be moved from the environment. | _(unset: inert)_ | Regtest ships inert on purpose: arming a network commits every BTC indexer on it to a wired DOGE peer, so a hardcoded height wedged every single-coin BTC venue at its first close. Set this on **every** BTC indexer and hub in a two-chain acceptance venue, alongside `DOGE_INDEXER_API_URL`. A venue that arms its hubs and forgets its indexer shows up as a consensus-rules digest mismatch, because `ROLLCALL_ACTIVATION` is one of the shared gates that digest covers. -A BTC indexer with no DOGE wiring **defers every block** from the first epoch close onward, with `stallReason = 'rollcall_proof_unavailable'`, rather than judging absences it cannot prove. The same deferral covers an unreachable or malformed answer, a DOGE tip that has not yet buried the window cut by `ROLLCALL_DOGE_MATURITY`, and a DOGE indexer whose vendored action-manifest hash differs from this indexer's own. That last case is what turns a DOGE indexer running a decoder too old to know `ROLLCALL` from a silent evict-the-federation bug into a loud, safe stall: wire the DOGE indexers and deploy their decoder **before** `ROLLCALL_ACTIVATION` is reached. +`bin/consensus-identity.js`'s `selectedPinBlock()` also reads both names, from the `armed_regtest_venue.env` block of `bin/pins/at1-consensus-identity.json`, to pick between the armed and bare-checkout consensus-identity pin for comparison; that block must be kept in step with whatever the venue actually arms. + +A BTC indexer with no DOGE wiring **defers every block** from the first epoch close onward (epoch height + 144 + 36), with `stallReason = 'rollcall_proof_unavailable'`, rather than judging absences it cannot prove. The same deferral covers an unreachable or malformed answer, a DOGE tip that has not yet buried the window cut by `ROLLCALL_DOGE_MATURITY`, and a DOGE indexer whose vendored action-manifest hash differs from this indexer's own. That last case is what turns a DOGE indexer running a decoder too old to know `ROLLCALL` from a silent evict-the-federation bug into a loud, safe stall: wire the DOGE indexers and deploy their decoder **before** `ROLLCALL_ACTIVATION` is reached. A validator with no Dogecoin indexer of its own points `DOGE_INDEXER_API_URL` at the public explorer instead: `https://explorer.xchain.io/TDOGE/api/` on testnet, `https://explorer.xchain.io/DOGE/api/` on mainnet, with `DOGE_INDEXER_API_KEY` set to the federation read key issued alongside the validator's other per-coin keys. That answer comes from the explorer's own replicated indexer database, so a replica that has fallen behind makes the epoch close wait longer rather than judge the roll call on stale data. ### Hub push queue and mirror @@ -220,6 +227,16 @@ baseline; never by the indexer service itself. The harness also reads the |---|---|---| | `XCHAIN_DECODER_SQL_PATH` | **Harness only.** Path to the decoder's SQL schema directory, used to build the scratch decoder database the measured blocks are read from. The harness refuses to run when it is unset rather than measuring against a schema it guessed at | `/path/to/xchain-decoder/src/sql` | +### Diagnostic Scripts (`bin/`) + +Read-only operator tools; neither broadcasts nor writes anything and neither is read by the running indexer process itself. + +| Variable | Description | Default | +|---|---|---| +| `XCHAIN_INDEXER_DIR` | Sibling-checkout override `bin/lib/carrier_logic_pin.js`'s `siblingDir()` resolves for cross-repo carrier-logic comparison; the `repo_guards` twin test points it at a second checkout with `XCHAIN_REQUIRE_SIBLINGS=1`. Never read by the running indexer process. | `../xchain-indexer` | +| `XCHAIN_SYNC_DIR` | Sibling-checkout override for the `xchain-sync` tree the same `siblingDir()` resolves when the pin tool compares against sync's copy; read by literal name so the coverage gate can see it. Never read by the running indexer process. | `../xchain-sync` | +| `XCHAIN_HUB_DIR` | Sibling-checkout override for the `xchain-hub` tree the same `siblingDir()` resolves when the pin tool compares against the hub's copy; read by literal name so the coverage gate can see it. Never read by the running indexer process. | `../xchain-hub` | + ## Hub DB Price Source Native-coin fee validation and FIAT settlement read the oracle tables (`price_snapshots`, @@ -247,7 +264,7 @@ missing hub DB only logs a warning. ## Coin-Specific Configuration -Each supported blockchain has a configuration file at `src/configs/.js` (BTC.js, LTC.js, DOGE.js) that defines: +Each supported blockchain has a configuration file at `src/coins/.js` (BTC.js, LTC.js, DOGE.js) that defines: | Parameter | Description | Example (BTC) | |---|---|---| @@ -265,7 +282,7 @@ Each supported blockchain has a configuration file at `src/configs/.js` (B ## Unified Gas Fee Schedule -After the activation block, fees for VM and staking actions are calculated using a gas-based schedule rather than the legacy flat fee constants. The following parameters are defined in each coin config file (`src/configs/.js`) and are only applied to blocks at or after the activation height: +After the activation block, fees for VM and staking actions are calculated using a gas-based schedule rather than the legacy flat fee constants. The following parameters are defined in each coin config file (`src/coins/.js`) and are only applied to blocks at or after the activation height: | Parameter | Description | Example (BTC) | |---|---|---| diff --git a/components/indexer/database.md b/components/indexer/database.md index 1425840e..05ee2645 100644 --- a/components/indexer/database.md +++ b/components/indexer/database.md @@ -15,7 +15,7 @@ The indexer reads decoded transaction data and block information from this datab Database name format: `XChain_{CHAIN}_{NETWORK}_Indexer` (e.g., `XChain_BTC_Mainnet_Indexer`) -The indexer creates and manages all tables in this database. SQL schema files live in `src/sql/*.sql` and are loaded by `db.js` to initialize the database on first startup. Tables are organized into several categories: +The indexer creates and manages all tables in this database. SQL schema files live in `src/sql/*.sql` and are loaded by `src/db/index.js` to initialize the database on first startup. Tables are organized into several categories: ### Core Tables @@ -76,6 +76,7 @@ The indexer creates and manages all tables in this database. SQL schema files li | `order_matches` | ORDER match (trade execution) records | | `order_statuses` | ORDER status change history | | `sends` | SEND transfer records | +| `xbridges` | XBRIDGE action records: one row per broadcast lock/burn (v0=lock XCHAIN, v1=burn XCHAIN, v3=lock a token, v4=burn a bridged copy), valid or refused. The system-injected settle legs (v2/v5) write no row here; see `bridge_settlements` | | `sleeps` | SLEEP action records | | `swaps` | SWAP (cross-chain) records | | `swap_cancels` | SWAP cancellation records | @@ -98,6 +99,9 @@ The indexer creates and manages all tables in this database. SQL schema files li | `cross_chain_calls` | Hub-mirrored cross-chain contract call rows (XCALL dispatch + result phases). Populated by `hub_db_sync` | | `cross_chain_call_executions` | Records the system-injected XEXEC action that executed a cross-chain call on this chain. One row per `call_id` (idempotency) | | `cross_chain_call_callbacks` | Records the system-injected callback EXECUTE delivered to the source contract after a cross-chain call result is processed. One row per `call_id` (idempotency) | +| `bridge_transfers` | Hub-mirrored bridge transfer rows. Populated by `hub_db_sync`; carries the source leg (chain, action_index, address), the destination (chain, address), the signed `tick`, `decimals` and `amount`, and the `validator_signatures` the `cross_chain` quorum signed. The XBRIDGE pass injects the settle leg from a finalized row | +| `bridge_settlements` | Settlement records for applied bridge legs on this chain. One row per `(transfer_id, kind)`, where `kind` separates a transfer settle from an applied policy snapshot; used for idempotency so a leg is never applied twice. Rolled back by `action_index` | +| `policy_snapshots` | Hub-mirrored per-token policy snapshots (allow list, block list, tick sleep) for a bridged copy. Populated by `hub_db_sync`; append-only, latest-wins by `policy_seq`, on the same terms as `state_checkpoints`. Membership arrays are transport and are verified against the signed `policy_hash` on apply | | `full_node_verifications` | Validated full-node possession-proof records. One row per (epoch, passing validator) from a NODEPROOF v0 verdict. Presence within `PROOF_WINDOW_BLOCKS` of a block gates the full-node reward tranche | | `rollcalls` | ROLLCALL epoch closes, BTC side. One row per epoch that reached its close block: `epoch_height` (PK), `snapshot_block` (where the responsible set was resolved), `close_block`, and `rolled` (0 = the epoch counted for nobody, recorded so the K-streak knows which epochs to skip) | | `rollcall_signers` | Signatures collected from ROLLCALL actions on the Dogecoin side: `(epoch_height, pubkey)` PK, `sig`, the `ledger_hash` as carried (the BTC close discards a mismatch), `publisher`, `action_index`, and the DOGE `block_index` | @@ -136,8 +140,8 @@ The indexer creates and manages all tables in this database. SQL schema files li | `stakes` | Active and historical capability-staking STAKE records (`version` 1=new / 2=top-up): `signing_pubkey_id`, `amount`, `activation_block` (`block_index + 6`), `deactivation_block` (set on UNSTAKE), `status_id`, `source_id`. Capabilities (`price`, `cross_chain`, `oracle_publish`, `attestation`, `full_node`) are derived from a pubkey's aggregate active `amount` against the governance-configured minimums: there is no `tier` column. | | `unstakes` | Capability UNSTAKE v0 records: `signing_pubkey_id`, `amount`, `cooldown_end_block`, `status_id`; links back to the originating stake by pubkey. The cooldown end is `block_index + STAKING.COOLDOWN_BLOCKS`; the cooldown length is governance-configurable via the `STAKING.COOLDOWN_BLOCKS` parameter (default 1000 blocks), not a hardcoded constant. Contract-targeted UNSTAKE v1 records do **not** appear here; they are written to `contract_unstakes` with a per-contract cooldown (see below). | | `delegations` | Active and historical DELEGATE records: `signing_pubkey_id`, `activation_block`, `deactivation_block` (set on DELEGATE v2 revoke), `status_id` | -| `validator_rewards` | Per-validator accumulated rewards: `source_id`, `signing_pubkey_id`, `reward_type` (`oracle_round`, `oracle_base`, `oracle_full_node`, `attest_fee`, `anchor_bundle`, or `anchor_archive`), `round_reference`, `amount`, `block_index`. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rows are derived by the indexer during block processing (the oracle label depends on whether the full-node reward tier is active: see [COLLECT](../../protocol/actions/collect.md)); `anchor_bundle` and `anchor_archive` rows are pushed from the hub via `pushvalidatorrewards`. One `anchor_bundle` row is written per published bundle, keyed on the bundle's snapshot block, not one per chain. | -| `stake_key_revocations` | Records DELEGATE v2 revocations of the original stake signing key: `source_id`, `signing_pubkey_id`, `deactivation_block`, `action_index`, `block_index`, `status_id`. A later STAKE v2 (higher `action_index`) by the same `(source, pubkey)` clears the revocation. Queried via `createStakeKeyRevocation` / `getStakeKeyRevocation` in `db.js`. | +| `validator_rewards` | Per-validator accumulated rewards: `source_id`, `signing_pubkey_id`, `reward_type` (`oracle_round`, `oracle_base`, `oracle_full_node`, `attest_fee`, `anchor_bundle`, or `anchor_archive`), `round_reference`, `amount`, `block_index`. `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` rows are derived by the indexer during block processing (the oracle label depends on whether the full-node reward tier is active: see [COLLECT](../../protocol/actions/collect.md)); `anchor_bundle` and `anchor_archive` rows are derived by the indexer from the on-chain ANCHOR bytes at or above their own flag-days (`ANCHOR_REWARD_ACTIVATION` / `ARCHIVE_REWARD_ACTIVATION`); rows below those heights were pushed from the hub via `pushvalidatorrewards` and are restored by ANCHOR full-parse recovery rather than re-derived. One `anchor_bundle` row is written per published bundle, keyed on the bundle's snapshot block, not one per chain. | +| `stake_key_revocations` | Records DELEGATE v2 revocations of the original stake signing key: `source_id`, `signing_pubkey_id`, `deactivation_block`, `action_index`, `block_index`, `status_id`. A later STAKE v2 (higher `action_index`) by the same `(source, pubkey)` clears the revocation. Queried via `createStakeKeyRevocation` / `getStakeKeyRevocation` in `src/db/stakes.js`. | | `reward_claims` | COLLECT records: `source_id`, `amount`, `status_id`, `block_index` | Staking tables enforce an activation/deactivation delay via `activation_block` and `deactivation_block` columns. Capability staking (`stakes`) is BTC-only and uses a fixed **6-block** delay. Contract-targeted staking (`contract_stakes`) runs on every chain and uses the per-chain `STAKING.ACTIVATION_DELAY_BLOCKS` default, calibrated for equivalent ~60-min reorg protection (**6 blocks on BTC, 24 on LTC, 60 on DOGE**); see `protocol/Contract_Staking.md`. Active-stake queries filter by `activation_block <= current_block AND (deactivation_block IS NULL OR deactivation_block > current_block)` to prevent short-range reorgs from affecting the active validator set. @@ -240,7 +244,7 @@ After processing, the indexer pushes validated PRICE actions to `xchain-hub` whi **Note:** Contract token balances are tracked via the standard `balances` table using the contract's derived address (`C::` in `index_addresses`). There is no separate `contract_balances` table. DEPOSIT creates credits/debits between the depositor and the derived address; WITHDRAW does the reverse. -**Contract identity manifest.** Gated by the `CONTRACT_META_REQUIRED` flag day (see [Flag Days](../../protocol/flag-days.md)). The four `meta_*` columns hold what the contract declared about itself in its own exports. They are written **only** when the deploy's status is `valid` and the declared `meta` conforms to the consensus grammar, the same gate `contract_permissions` rows use: on every other row all four stay NULL, so a value larger than the byte rule can never reach a column narrower than it. `meta_json` holds the isolate's serialized bytes verbatim, so unknown keys the author declared survive and a later display field needs no reindex. They are explicitly `utf8mb4` because they carry free-form author text, while the table default is `utf8mb3`; they are born that way rather than widened, so they are not in the `utf8mb4Columns.js` widen set. `meta_search` is the schema's only FULLTEXT index. **None of the four is in any hash preimage**: the contracts leg of `contract_hash` is a fixed four-column SELECT (`action_index`, `source_address`, `code_hash`, `status`), so a contract's block hash is identical with and without a recorded identity. The columns are additive and land through `src/sql/migrations/2026-09-08-contract-meta-columns.sql`; there is no backfill, and a reindex from genesis fills them through the ordinary write path. +**Contract identity manifest.** Gated by the `CONTRACT_META_REQUIRED` flag day (see [Flag Days](../../protocol/flag-days.md)). The four `meta_*` columns hold what the contract declared about itself in its own exports. They are written **only** when the deploy's status is `valid` and the declared `meta` conforms to the consensus grammar, the same gate `contract_permissions` rows use: on every other row all four stay NULL, so a value larger than the byte rule can never reach a column narrower than it. `meta_json` holds the isolate's serialized bytes verbatim, so unknown keys the author declared survive and a later display field needs no reindex. They are explicitly `utf8mb4` because they carry free-form author text, while the table default is `utf8mb3`; they are born that way rather than widened, so they are not in the `utf8mb4_columns.js` widen set. `meta_search` is the schema's only FULLTEXT index. **None of the four is in any hash preimage**: the contracts leg of `contract_hash` is a fixed four-column SELECT (`action_index`, `source_address`, `code_hash`, `status`), so a contract's block hash is identical with and without a recorded identity. The columns are additive and land through `src/sql/migrations/2026-09-11-contract-meta-columns.sql`; there is no backfill, and a reindex from genesis fills them through the ordinary write path. **Deferred assembly of chunked DEPLOY groups.** Gated by the `DEPLOY_DEFERRED_ASSEMBLY` flag day (see [Flag Days](../../protocol/flag-days.md)). Below the flag, an assembling DEPLOY (v2/v3) must land after every carrier of its chunk group or it is permanently `invalid: CODE_HASH (no chunks)`. At and above the flag, an assembling DEPLOY that lands early instead goes `pending: CODE_HASH (awaiting chunks)`: its `contracts` and constructor `contract_executions` rows are written at its own action_index exactly as an invalid assembler's are, the base fee is charged, and the status is never mutated afterward. Whichever action then completes the group (the assembler itself, or a later carrier) runs the deployment at that action's own index, and writes `assembler_action_index` on its constructor row pointing back at the pending assembler. A second assembler landing while one is already pending for the same `(source_id, code_hash)` group is `invalid: CODE_HASH (duplicate pending)`. Only the two new `contract_executions` columns record this; no new table, and neither column enters a block-hash preimage. @@ -265,7 +269,7 @@ Several tables require special handling beyond a simple bulk delete: - **`contract_stakes`, `contract_unstakes`, `stakes`, `unstakes`, `delegations`, `contract_delegations`**: In-place `deactivation_block` stamps written by orphaned UNSTAKE/DELEGATE-revoke actions are reset before the bulk delete. Similarly, in-place `amount` reductions from orphaned SLASH executions are restored from the corresponding `*_slash_debits` rows before those rows are deleted, and in-place `signing_pubkey_id` rotations from an orphaned DELEGATE v1 materialization are restored from `contract_delegation_rotations`. - **`attests` (v0 rows), `xcalls` (v0 rows)**: Request-status flips (`fulfilled`/`errored`/`expired` and `completed`/`expired`) written as in-place UPDATEs on surviving rows are reset to `pending` before the bulk delete, keyed on `resolved_block >= reorgBlock`. - **`price_snapshots`, `oracle_prices`**: Not deleted by the generic loops; deleted separately by `reference_block`/`(source_chain, action_index)` respectively. -- **`attest_validator_stats`**: A cross-attestation aggregate with no `action_index` or `block_index` FK; recomputed from surviving response and expired-request rows via `_recomputeAttestationValidatorStats`. +- **`attest_validator_stats`**: A cross-attestation aggregate with no `action_index` or `block_index` FK; recomputed from surviving response and expired-request rows via `recomputeAttestationValidatorStats`. - **`state_checkpoints`, `capability_snapshots`**: Intentionally NOT deleted on reorg. Both use append-only / supersede-by-seq semantics so stale rows are harmless; hub-driven convergence closes any divergence window. --- diff --git a/components/indexer/ledger.md b/components/indexer/ledger.md index bed445e4..9a48bfa4 100644 --- a/components/indexer/ledger.md +++ b/components/indexer/ledger.md @@ -73,7 +73,7 @@ This means contract token movements are covered by the standard sanity check (`t - **VM actions**: DEPLOY and EXECUTE charge gas via the unified gas schedule: `gas_cost × gas_price` XCHAIN per operation (post-activation blocks) - **Staking actions**: STAKE, UNSTAKE, DELEGATE (rotate + revoke), and COLLECT are metered under the same unified gas schedule (post-activation blocks) -The GAS address (defined per-chain, per-network in `src/configs/.js`) is the only address authorized to issue the `XCHAIN` token. It is exempt from the reserved ticker restriction that prevents other addresses from using protocol-reserved names. +The GAS address (defined per-chain, per-network in `src/coins/.js`) is the only address authorized to issue the `XCHAIN` token. It is exempt from the reserved ticker restriction that prevents other addresses from using protocol-reserved names. ## Fee Distribution diff --git a/components/node/architecture.md b/components/node/architecture.md index da559377..e543769c 100644 --- a/components/node/architecture.md +++ b/components/node/architecture.md @@ -37,7 +37,7 @@ Each coin/network combination (e.g., bitcoin/regtest) gets its own Docker networ flowchart TD subgraph NODEBOX["xchain-node"] CLI["cli.js
Commander
21 commands"] - MODOPS["moduleOperations.js
installModules / startModules /
stopModules / restartModules /
uninstallModules / resetModules"] + MODOPS["module_operations.js
installModules / startModules /
stopModules / restartModules /
uninstallModules / resetModules"] PRECHECK["precheck.js
Docker check
Dir creation
MariaDB open
Version fetch"] MODSVC["ModuleService
cloneGit()
buildAndUp()
uninstallModule()"] CONFIGSVC["ConfigService"] @@ -70,28 +70,28 @@ flowchart TD | `src/cli.js` | Commander.js CLI definitions (21 commands, global options, preAction hook) | | `src/precheck.js` | Pre-command validation (Docker, directories, MariaDB connection, versions, networks) | | `src/state.js` | Singleton state (MariaDB pool instance, cached modules, verbose flag) | -| `src/MariaDbStore.js` | MariaDB-backed store for module to container ID persistence; persists mappings in the `xchain_node.modules` table inside the shared `xchain-node-database` container (the same container that managed services use for their decoder/indexer databases) | -| `src/config/constants.js` | Enums (Coin, Network, XChainService), paths, git URLs | -| `src/services/ConfigService.js` | Path/naming helpers, config generation, arg parsing, port validation | -| `src/services/DockerService.js` | Docker CLI wrappers (network, build, run, start, stop, exec, logs, monitor) | -| `src/services/ModuleService.js` | Git clone, Docker build/run, install/uninstall/update flows | -| `src/services/DatabaseService.js` | MariaDB container setup, user/password management, database creation | -| `src/services/StatusService.js` | Container status queries, version display, formatted table output | -| `src/services/VersionService.js` | Local/remote/container version checking via GitHub API | -| `src/services/NodeService.js` | Crypto node download and Docker image building | -| `src/services/HubService.js` | Hub installation, update, and JSON-RPC configuration | -| `src/services/ExplorerService.js` | Explorer installation and configuration | -| `src/services/BootstrapService.js` | Bootstrap snapshot create/restore with SHA-256 verification; creates Ed25519 signatures on `bootstrap create` when `XCHAIN_NODE_BOOTSTRAP_SIGNING_KEY` is set, and enforces signature verification on restore (fail-closed by default) | -| `src/services/TelemetryService.js` | Anonymous usage telemetry: collects install ID, version, running services, and OS info; sends to the hub collector; default-on with opt-out via `--no-telemetry`, `XCHAIN_NODE_NO_TELEMETRY=1`, or a persisted preference | -| `src/services/CredentialsService.js` | Persists per-OS-user MariaDB credentials in `~/.xchain-node/credentials.json`; stores both the bundled-DB password and optional external-DB connection details | -| `src/services/DiscoveryService.js` | Auto-discovers existing xchain-node Docker containers and re-registers them in the MariaDB modules table (`sync` command); classifies containers by naming convention to recover state after a database loss | -| `src/services/ValidatorService.js` | Validator-mode onboarding: generates Ed25519 signing keys and writes validator config files (`validator init`); reads and displays persisted validator settings (`validator status`); injects resulting env vars into the hub container | -| `src/operations/moduleOperations.js` | Bulk operations (install/start/stop/restart/reset/exec/logs/monitor) | -| `src/HubConnector.js` | JSON-RPC 2.0 client for xchain-hub | -| `src/ExplorerConnector.js` | JSON-RPC 2.0 client for xchain-explorer | -| `src/TelemetryConnector.js` | HTTP client that posts telemetry pings to the central hub collector; URL overrideable via `XCHAIN_NODE_TELEMETRY_URL` | - -| `src/GitHubDownloader.js` | GitHub release download with SHA-256 hash verification | +| `src/db/modules.js` | MariaDB-backed store for module to container ID persistence; persists mappings in the `xchain_node.modules` table inside the shared `xchain-node-database` container (the same container that managed services use for their decoder/indexer databases) | +| `src/config/index.js` | Enums (Coin, Network, XChainService), paths, git URLs | +| `src/services/config_service.js` | Path/naming helpers, config generation, arg parsing, port validation | +| `src/services/docker_service.js` | Docker CLI wrappers (network, build, run, start, stop, exec, logs, monitor) | +| `src/services/module_service.js` | Git clone, Docker build/run, install/uninstall/update flows | +| `src/services/database_service.js` | MariaDB container setup, user/password management, database creation | +| `src/services/status_service.js` | Container status queries, version display, formatted table output | +| `src/services/version_service.js` | Local/remote/container version checking via GitHub API | +| `src/services/node_service.js` | Crypto node download and Docker image building | +| `src/services/hub_service.js` | Hub installation, update, and JSON-RPC configuration | +| `src/services/explorer_service.js` | Explorer installation and configuration | +| `src/services/bootstrap_service.js` | Bootstrap snapshot create/restore with SHA-256 verification; creates Ed25519 signatures on `bootstrap create` when `XCHAIN_NODE_BOOTSTRAP_SIGNING_KEY` is set, and enforces signature verification on restore (fail-closed by default) | +| `src/services/telemetry_service.js` | Anonymous usage telemetry: collects install ID, version, running services, and OS info; sends to the hub collector; default-on with opt-out via `--no-telemetry`, `XCHAIN_NODE_NO_TELEMETRY=1`, or a persisted preference | +| `src/services/credentials_service.js` | Persists per-OS-user MariaDB credentials in `~/.xchain-node/credentials.json`; stores both the bundled-DB password and optional external-DB connection details | +| `src/services/discovery_service.js` | Auto-discovers existing xchain-node Docker containers and re-registers them in the MariaDB modules table (`sync` command); classifies containers by naming convention to recover state after a database loss | +| `src/services/validator_service.js` | Validator-mode onboarding: generates Ed25519 signing keys and writes validator config files (`validator init`); reads and displays persisted validator settings (`validator status`); injects resulting env vars into the hub container | +| `src/operations/module_operations.js` | Bulk operations (install/start/stop/restart/reset/exec/logs/monitor) | +| `src/services/hub_connector.js` | JSON-RPC 2.0 client for xchain-hub | +| `src/services/explorer_connector.js` | JSON-RPC 2.0 client for xchain-explorer | +| `src/services/telemetry_connector.js` | HTTP client that posts telemetry pings to the central hub collector; URL overrideable via `XCHAIN_NODE_TELEMETRY_URL` | + +| `src/services/github_downloader.js` | GitHub release download with SHA-256 hash verification | | `src/utils/helpers.js` | Utilities (sleep, stringToCoin, decompressTarGz) | ## Precheck Workflow diff --git a/components/node/configuration.md b/components/node/configuration.md index a20a912e..c4223c87 100644 --- a/components/node/configuration.md +++ b/components/node/configuration.md @@ -7,7 +7,7 @@ xchain-node uses a two-layer configuration system to generate environment variables for each managed service: -1. **Hardcoded defaults**: defined in `ConfigService.js` for each module type (40+ variables per coin-specific service) +1. **Hardcoded defaults**: defined in `config_service.js` for each module type (40+ variables per coin-specific service) 2. **Config file overrides**: read from `config/{coin}-{network}` files in `KEY=VALUE` format Config files are plain text with one variable per line. Values containing `=` (such as base64 tokens or passwords) are handled correctly; only the first `=` on each line is treated as the separator. Blank lines and lines without `=` are skipped. @@ -115,7 +115,7 @@ These variables are read by xchain-node itself at startup. They control runtime | `XCHAIN_NODE_EXTERNAL_DB_PORT` | Port of the external MariaDB when `XCHAIN_NODE_EXTERNAL_DB=1` (default: `3306`). | | `XCHAIN_NODE_EXTERNAL_DB_ROOT_USER` | Root username for the external MariaDB when `XCHAIN_NODE_EXTERNAL_DB=1` (default: `root`). Used during database and user provisioning. | | `XCHAIN_NODE_EXTERNAL_DB_ROOT_PASSWORD` | MariaDB root password for the host-native (non-Docker) database, used alongside `XCHAIN_NODE_EXTERNAL_DB=1`. Avoids an interactive password prompt in headless installs. Supply alongside `XCHAIN_NODE_EXTERNAL_DB_HOST`, `XCHAIN_NODE_EXTERNAL_DB_PORT`, and `XCHAIN_NODE_EXTERNAL_DB_ROOT_USER`. | -| `XCHAIN_NODE_MODULE_MEMORY_MB_` | Explicit container memory limit in MB for one service, e.g. `XCHAIN_NODE_MODULE_MEMORY_MB_XCHAIN_UTXO_TRACKER=4096` or `XCHAIN_NODE_MODULE_MEMORY_MB_XCHAIN_DECODER=1536` (the service name upper-cased with `-` as `_`). Applied as `--memory` and an equal `--memory-swap` at the next `install`, `update` or `recreate`. `0` disables the derived tracker limit. Without it, only the utxo-tracker is limited, to half the host RAM divided by the number of installed trackers (floor 1024 MB, ceiling 16384 MB); other services run unlimited because they do not size themselves to a cgroup limit. See [Memory on a multi-chain host](../../operations/deployment.md#memory-on-a-multi-chain-host). | +| `XCHAIN_NODE_MODULE_MEMORY_MB_` | Explicit container memory limit in MB for one service, e.g. `XCHAIN_NODE_MODULE_MEMORY_MB_XCHAIN_UTXO_TRACKER=4096` or `XCHAIN_NODE_MODULE_MEMORY_MB_XCHAIN_DECODER=1536` (the service name upper-cased with `-` as `_`). Applied as `--memory` and an equal `--memory-swap` at the next `install`, `update` or `recreate`. `0` disables the derived tracker limit. Without it, only the utxo-tracker is limited, to half the host RAM divided by the number of installed trackers (floor 1024 MB, ceiling 16384 MB); other services run unlimited because they do not size themselves to a cgroup limit. A host whose kernel has no memory cgroup controller accepts the flag and discards it, so confirm the limit landed after the create; [Memory on a multi-chain host](../../operations/deployment.md#memory-on-a-multi-chain-host) has the check and the Raspberry Pi OS fix. | | `XCHAIN_NODE_STOP_TIMEOUT_SECONDS` | Seconds a coin node daemon is given to exit cleanly before docker kills it, on `update` and `recreate` and as the container's own `--stop-timeout` (default: `600`). A daemon flushes its chainstate only on a clean exit; a killed one re-validates from its last flushed block when it returns. Raise it on a host where a large `dbcache` flushes slowly (a Pi writing to a USB SSD). The update prints how long the daemon took and warns when the budget ran out. Applied at the next `update` or `recreate`. | | `XCHAIN_NODE_MODULE_STOP_TIMEOUT_SECONDS_` | Seconds one service container is given to exit cleanly before docker kills it, e.g. `XCHAIN_NODE_MODULE_STOP_TIMEOUT_SECONDS_XCHAIN_DECODER=300` (the service name upper-cased with `-` as `_`). Used by `stop`, `update`, `recreate` and `uninstall` and stamped on the container as `--stop-timeout`. Defaults: 120 for `xchain-decoder` and `xchain-utxo-tracker`, which break their loops at a block boundary, 30 for every other service. The coin daemon keeps `XCHAIN_NODE_STOP_TIMEOUT_SECONDS`. Applied at the next `update` or `recreate`. See [Stopping](operations.md#stopping). | | `XCHAIN_NODE_NO_TELEMETRY` | Set to `1` to disable anonymous usage telemetry. Opt-out is also available via the `--no-telemetry` CLI flag or a persisted preference in `~/.xchain-node/telemetry.json`. | @@ -255,6 +255,7 @@ These env vars override where xchain-node stores its filesystem state on the hos | `XCHAIN_NODE_CONFIG_DIR` | `/config` | Generated per-service `.env` files. Small. | | `XCHAIN_NODE_BLOCKS_DIR` | (unset → inside data volume) | Optional host path for the coin node's `blocks/` directory. If set, mounted as `/blocks` into the docker container so chain data can live on a separate disk from the rest of the node state. | | `XCHAIN_NODE_ALLOW_DEGRADED_EXPLORER` | (unset → install fails) | Accepted values `1`, `true`, `yes`, case-insensitive. An `install` that creates a coin stack waits for the explorer to start serving coin data and fails when it never does; an install that adds no coin stack does not wait. Set this to continue anyway, accepting a stack whose explorer answers but serves no coins. Intended for callers that knowingly want the rest of the stack without a converged explorer; leave it unset on any node meant to serve reads. | +| `XCHAIN_NODE_ALLOW_NO_DOGE_READ` | (unset → deploy refused) | Any value other than empty, `0`, `false` or `no` enables it. `install`, `update` and `recreate` refuse a bitcoin `xchain-indexer`, or a validator-mode `xchain-hub`, on a network whose roll call is armed when neither `DOGE_INDEXER_API_URL` nor `DOGE_INDEXER_URL` is set, because that indexer defers every block from the first epoch close and that hub publishes no roll call. Set this to deploy anyway with a warning, for a single-coin regtest venue that runs no roll call; never on a validator. | > **⚠️ Testnet / regtest write to a network-prefixed subdirectory.** Dogecoind and litecoind place block data under a per-network subdirectory of the datadir on every network except mainnet: > @@ -286,10 +287,10 @@ Without these overrides the small `/` partition fills the moment a bootstrap is | Constant | Value | Location | Description | |---|---|---|---| -| `NODE_PREFIX` | `xchain-node` | constants.js | Prefix for all Docker container and network names | -| `SEP` | `-` | constants.js | Separator for Docker naming (`xchain-node-bitcoin-mainnet`) | -| `DB_SEP` | `_` | constants.js | Separator for database naming (`xchain_decoder_bitcoin_mainnet`) | -| `DB_NAME` | `xchain_node` | CredentialsService.js | MariaDB database name used to store module state | +| `NODE_PREFIX` | `xchain-node` | src/config/index.js | Prefix for all Docker container and network names | +| `SEP` | `-` | src/config/index.js | Separator for Docker naming (`xchain-node-bitcoin-mainnet`) | +| `DB_SEP` | `_` | src/config/index.js | Separator for database naming (`xchain_decoder_bitcoin_mainnet`) | +| `DB_NAME` | `xchain_node` | credentials_service.js | MariaDB database name used to store module state | ### NODE_PREFIX Validation diff --git a/components/node/operations.md b/components/node/operations.md index a805485c..f7feb13c 100644 --- a/components/node/operations.md +++ b/components/node/operations.md @@ -68,7 +68,7 @@ The coupling is one-directional, so `reset xchain-indexer` on its own stays avai |---|---|---| | `exec` | `exec ` | Execute a command inside a running container | | `shell` | `shell ` | Open an interactive shell in a container | -| `clear-reorg-halt` | `clear-reorg-halt --reason [--force] [--dry-run]` | Clear the decoder's durable REORG_HALT marker after it verifies its database is intact; the reason is recorded in the decoder's `events` table. See the decoder [Troubleshooting](../decoder/operations.md#decoder-halted-after-a-deep-reorg-reorg_halt) page for the checks | +| `clear-reorg-halt` | `clear-reorg-halt [--reason ] [--force] [--dry-run]` | Clear the decoder's durable REORG_HALT marker after it verifies its database is intact; the reason is recorded in the decoder's `events` table. `--reason` is required unless `--dry-run`, which only reports the verdict. See the decoder [Troubleshooting](../decoder/operations.md#decoder-halted-after-a-deep-reorg-reorg_halt) page for the checks | ### Advanced Operations @@ -103,6 +103,24 @@ The coupling is one-directional, so `reset xchain-indexer` on its own stays avai When `all` is used, the command expands to every valid combination. Regtest-only services (`xchain-regtest-miner`, `xchain-e2e-test`) are automatically excluded from mainnet and testnet expansions. +## Exit status + +`xchain-node` reports the outcome of a command in its exit status, so a wrapper script or a cron entry can act on it without parsing output. + +| Exit status | Meaning | +|---|---| +| `0` | The command completed. For `update`, every requested service is on the requested release. | +| `1` | The command failed. `update` prints `update failed: ` on stderr first, including the case where no requested service matched an installed container. | + +**Read the status without a pipe.** `$?` is the status of the LAST command in a pipeline, so `xchain-node update all | grep -v WARN; echo $?` prints grep's status, not xchain-node's, and a failed update reads as `0`. Two lines show it: + +```bash +( echo out; false ) | grep -v zzz >/dev/null; echo $? # prints 0: grep's status +( echo out; false ); echo $? # prints 1: the command's own +``` + +Either read the status with no pipe in the way, or run `set -o pipefail` first, which makes a pipeline's status the last non-zero status of any command in it. + ## Installation Workflow When `xchain-node install master all bitcoin regtest` is executed: diff --git a/components/regtest-miner/architecture.md b/components/regtest-miner/architecture.md index f0584ba7..54165e27 100644 --- a/components/regtest-miner/architecture.md +++ b/components/regtest-miner/architecture.md @@ -43,8 +43,8 @@ flowchart TD |---|---|---| | `src/api.js` | ~212 | Environment validation, Express server, JSON-RPC routing, miner lifecycle | | `src/XChainRegtestMiner.js` | ~589 | Mining loop, wallet management, fillMempool, timer control | -| `src/BlockchainConnector.js` | ~489 | JSON-RPC 2.0 client wrapping 15 Bitcoin Core methods with retry logic | -| `src/CryptoNetworks.js` | ~132 | Coin-specific bitcoinjs-lib network params (BTC/LTC/DOGE, all networks) | +| `src/rpc/blockchain_connector.js` | ~489 | JSON-RPC 2.0 client wrapping 15 Bitcoin Core methods with retry logic | +| `src/networks/crypto_networks.js` | ~132 | Coin-specific bitcoinjs-lib network params (BTC/LTC/DOGE, all networks) | ## Mining Loop diff --git a/components/sdk/actions.md b/components/sdk/actions.md index e360f764..7bf9e39b 100644 --- a/components/sdk/actions.md +++ b/components/sdk/actions.md @@ -3,7 +3,7 @@ # XChain Platform SDK: ACTION Reference -Complete reference for all 31 ACTION types supported by the XChain Platform SDK. +Complete reference for all 32 ACTION types supported by the XChain Platform SDK. --- @@ -80,14 +80,14 @@ Configure address-level preferences for fee routing and memo requirements. | Param | Type | Required | Description | |---|---|---|---| -| feePreference | integer | No | Fee routing: `1` = destroy, `2` = protocol, `3` = community | +| feePreference | integer | No | Fee routing: `0` = default disposition, `1` = destroy, `2` = protocol | | requireMemo | integer | No | Whether to require a memo on incoming sends (`0` or `1`) | | dispenserPreference | integer | No | Who may open dispensers targeting this address: `1` = owner only (default), `2` = anyone | | memo | string | No | Optional note | **Notes (v0):** - All fields are optional; omitting all fields is valid (no-op update). -- `feePreference` must be `1`, `2`, or `3` if provided. +- `feePreference` must be `0`, `1`, or `2` if provided; consensus rejects any other value. The shipped SDK validator still enforces the retired `{1, 2, 3}` set, so it refuses a valid `0` and admits a `3` the indexer marks invalid: until that is corrected, emit `1` or `2`, which both surfaces accept. **Params (controller bind/unbind (v1):)** @@ -1240,6 +1240,51 @@ See also: [`../actions/BET.md`](../../protocol/actions/bet.md) --- +### XBRIDGE + +Cross-chain lock/burn/settle: moves XCHAIN or a bridged token between chains against a protocol-owned escrow. `sdk.xbridge(params)` is the raw wrapper; the version is taken from `params.version` (0 lock XCHAIN, 1 burn XCHAIN, 3 lock a general token, 4 burn a bridged token). Versions 2 and 5 are the settle legs, system-injected by the indexer from a finalized transfer, and a broadcast one is refused with `invalid: XBRIDGE v2 is system-injected` / `invalid: XBRIDGE v5 is system-injected`. + +**Format Versions:** v0 (lock XCHAIN, BTC only), v1 (burn XCHAIN, never on BTC), v3 (lock a token, its origin chain only), v4 (burn a bridged token) + +**Format v0:** `XBRIDGE|0|DEST_COIN|DEST_ADDRESS|AMOUNT|MEMO` +**Format v1:** `XBRIDGE|1|BTC_ADDRESS|AMOUNT|MEMO` +**Format v3:** `XBRIDGE|3|TICK|DEST_COIN|DEST_ADDRESS|AMOUNT|MEMO` +**Format v4:** `XBRIDGE|4|TICK|ORIGIN_ADDRESS|AMOUNT|MEMO` + +**Params:** + +| Param | Type | Required | Description | +|---|---|---|---| +| version | integer | Yes | `0`, `1`, `3`, or `4` (see Format Versions above) | +| destCoin | string | v0, v3 | Destination coin, a supported coin other than the source chain | +| destAddress | string | v0, v3 | Destination address on `destCoin`, validated coin-and-network aware | +| btcAddress | string | v1 | Destination BTC address, BTC-network validated | +| tick | string | v3, v4 | v3: a native (undotted, non-gas) tick on this chain to lock. v4: a bridged row on this chain (`.`) to burn | +| originAddress | string | v4 | Destination address on the bridged tick's origin chain, validated against that chain | +| amount | string | v0, v1, v3, v4 | Positive decimal, at most the token's decimals, at most the source's balance | +| memo | string | No | Optional note | + +**Notes:** +- This is the raw builder; it does not validate the destination address against the coin and network it lands on before broadcast, because a lock is one-way and unrecoverable. Use the coin-aware recipes below instead of calling `sdk.xbridge()` directly wherever the destination comes from user input. +- A bridged tick is named under its origin chain's root, `.` (`BTC.PEPECASH`, `DOGE.FUFU`); v3's `TICK` is always the native, undotted name on the chain the lock is broadcast on. + +```js +// Lock 500 XCHAIN on BTC into the DOGE escrow (raw wrapper) +await sdk.xbridge({ version: 0, destCoin: 'DOGE', destAddress: 'D8bFJYQ6JZ4tSjzZbXqXYh2vN3xKzQpump', amount: '500' }) + +// The coin-aware recipes validate the destination first and pin the version: +await sdk.workflows.bridgeLock(wif, { destCoin: 'DOGE', destAddress: 'D8bFJYQ6JZ4tSjzZbXqXYh2vN3xKzQpump', amount: '500' }) +await sdk.workflows.bridgeBurn(wif, { btcAddress: '1ExampleAddressXXXXXXXXXXXXXXXXXXX', amount: '200' }) +await sdk.workflows.bridgeTokenLock(wif, { tick: 'PEPECASH', destCoin: 'DOGE', destAddress: 'D8bFJYQ6JZ4tSjzZbXqXYh2vN3xKzQpump', amount: '1000' }) +await sdk.workflows.bridgeTokenBurn(wif, { tick: 'BTC.PEPECASH', originAddress: '1ExampleAddressXXXXXXXXXXXXXXXXXXX', amount: '250' }) +``` + +Read the escrow-versus-supply invariant and in-flight transfers over the indexer's `getpendingbridgetransfers` / `getbridgetransfer` and the hub's `getbridgeinvariant` (see [Cross-Chain Bridge](../../protocol/xchain-bridge.md#reads)). + +See also: [`../actions/XBRIDGE.md`](../../protocol/actions/xbridge.md) + +--- + ## Validation Rules The SDK enforces these rules before serializing any action. Violations throw an `SDKValidationError`. @@ -1314,7 +1359,7 @@ Must be **`1`** (ECIES), **`2`** (ECDH), or **`3`** (AES). ### FEE_PREFERENCE (ADDRESS) -Must be **`1`** (destroy), **`2`** (protocol), or **`3`** (community). +Must be **`0`** (default disposition), **`1`** (destroy), or **`2`** (protocol). There is no community bucket; a `3` indexes invalid. The shipped SDK validator still carries the retired `{1, 2, 3}` set, so `0` is rejected client-side and `3` passes client-side validation only to fail at the indexer. ### LIST TYPE diff --git a/components/sdk/configuration.md b/components/sdk/configuration.md index 7ebe499a..1a9988ad 100644 --- a/components/sdk/configuration.md +++ b/components/sdk/configuration.md @@ -77,7 +77,7 @@ The SDK reads these environment variables at construction time. A `.env` file in | Variable | Description | Used by | |---|---|---| -| `NETWORK` | Network string (e.g. `bitcoin-mainnet`). See [Network Strings](#network-strings). No default in the SDK itself; the interactive REPL (`npm run repl`) falls back to `bitcoin-regtest` when it is unset. | Explorer client, Hub connector, REPL | +| `NETWORK` | Network string (e.g. `bitcoin-mainnet`). See [Network Strings](#network-strings). No default in the SDK itself; the interactive REPL (`npm run repl`) defaults to `bitcoin-regtest` when it is unset. | Explorer client, Hub connector, REPL | | `SDK_API_PORT` | Port for the JSON-RPC microservice API server. Default: `3005`. | API server (`npm run api`) | | `SDK_API_KEY` | Bearer token required for all API server methods except `ping`. No default; the server warns and rejects all non-ping calls when unset. | API server (`npm run api`) | | `SDK_API_MAX_BATCH` | Maximum calls accepted in one batch request. A non-numeric or non-positive value falls back to the default rather than disabling the cap. Default: `20`. | API server (`npm run api`) | @@ -95,9 +95,16 @@ The SDK reads these environment variables at construction time. A `.env` file in | `XCHAIN_MCP_WIF` | Agent signing key for the MCP server's write tools. **Never logged or echoed.** Must be set together with `XCHAIN_MCP_POLICY`; with either missing the server starts read-only. Configured by the operator, never by the conversation. See [MCP Quickstart](../../ai-agents/mcp-quickstart.md). Treat as a credential. | MCP server | | `XCHAIN_MCP_POLICY` | Path to the `AgentSession` policy JSON that bounds what the agent may spend. Required alongside `XCHAIN_MCP_WIF`; a policy with no spend ceiling is refused, because the MCP rail has no human in the loop. | MCP server | | `XCHAIN_INDEXER_PATH` | Path to a sibling `xchain-indexer` checkout, used by `bin/check-preflight-drift.js` to hash the indexer's action handlers. Falls back to `../xchain-indexer`. Development tooling only. | Drift checker | +| `XCHAIN_REQUIRE_SIBLINGS` | Makes `bin/check-preflight-drift.js` treat an unresolved `xchain-indexer` checkout as a finding instead of a skip, so a CI job whose checkout step was dropped fails loudly rather than reporting success having compared nothing. Only the exact value `1` enables it, and it overrides `XCHAIN_ALLOW_NO_INDEXER`. No default. Development tooling only. | Drift checker | +| `XCHAIN_ALLOW_NO_INDEXER` | Declares that this run has no `xchain-indexer` checkout on purpose (a standalone SDK clone), so `bin/check-preflight-drift.js` exits 0 and says it compared nothing instead of failing. Only the exact value `1` enables it, and `XCHAIN_REQUIRE_SIBLINGS=1` overrides it. No default: when unset, an unresolved indexer checkout FAILS the gate, because a missing directory is an accident of layout rather than a decision. Development tooling only. | Drift checker | | `CORS_ORIGIN` | CORS origin for the JSON-RPC microservice API server. Disabled when unset. | API server (`npm run api`) | | `STOP_CHECK_INTERVAL` | Poll interval in milliseconds for `sdk.start()`'s shutdown loop, which sleeps this long between checks of the stop flag set by `sdk.stop()`. Only relevant when running the SDK as a long-lived process. Default: `5000`. | `sdk.start()` | | `COSIGNER_TOKEN` | Shared bearer token for the optional MuSig2 co-signer sidecar (`createCoSignerApp`). The sidecar is a local service: bind it to loopback and gate it with this token. No default; see [MULTISIG](../wallet/multisig.md). | Co-signer sidecar | +| `XC_ROLLCALL_REGTEST_ACTIVATION` | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_ACTIVATION` in the SDK's copy of the activation registry, so the copy reads the row as the venue's armed indexers and hubs do. `armed` (or `genesis`/`on`/`true`/`yes`) arms at BTC height `0`; a bare non-negative integer arms at that height; `off`/`inert`/`false`/`no`/`none` and unset leave it inert, and anything else is refused with a process warning and stays inert. Applied when a row is read, from the environment as it stands then. mainnet and testnet are fixed in source and cannot be moved from the environment. Never set outside a regtest venue. | Activation registry copy (`src/consensus/gate_registry.js`) | +| `XC_ROLLCALL_GATES_REGTEST_ACTIVATION` | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_GATES_ACTIVATION` (ROLLCALL v1, the consensus-gate list roll calls carry) in the SDK's registry copy. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; set identically on every hub, indexer and SDK process in the venue. mainnet and testnet are fixed in source. Never set outside a regtest venue. | Activation registry copy (`src/consensus/gate_registry.js`) | +| `XC_MIRROR_ADMISSION_ACTIVATION` | **Regtest only.** Arms the per-coin `regtest` entries of `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION` (the mirror-admission heights) and the `regtest` entry of `ANCHOR_ATTEST_BARRIER_ACTIVATION` in the SDK's registry copy, one variable for the whole barrier family. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; the armed form arms at height `0`. mainnet and testnet are fixed in source. Never set outside a regtest venue. | Activation registry copy (`src/consensus/gate_registry.js`) | + +The three `XC_*` arming variables exist because the SDK carries a byte-identical copy of the shared activation rows the indexer, hub, sync and explorer carry, and that copy arms its regtest entries from the same variables by the same grammar (`src/consensus/gate_registry/regtest_env.js`). The SDK drives no roll call and admits no mirror row itself; honouring the levers keeps its reading of a row identical to the venue's, so a consensus-identity comparison across the venue's processes does not diverge on the SDK. Regtest ships inert on purpose, and a venue sets each lever **identically** on every process it arms. ## Hub Discovery diff --git a/components/sdk/encoder.md b/components/sdk/encoder.md index ae406b6f..93baa101 100644 --- a/components/sdk/encoder.md +++ b/components/sdk/encoder.md @@ -50,7 +50,7 @@ The encoder supports five encoding strategies: four script-output lanes and the |---|---|---| | `OP_RETURN` | 80 bytes total (76 bytes user data) | Cheapest single-transaction encoding. Each output is 80 bytes: 4-byte `XCHN` magic prefix + 76 bytes for ACTION data. Auto-selected for small payloads. | | `P2SH` | 476 bytes | Two-phase transaction. Data is committed in a P2SH script (520 bytes minus 44 bytes of script overhead). Payloads larger than one chunk are split across multiple P2SH outputs (fund-then-spend pairs), up to the shared 8,192-byte compiled-ACTION ceiling. Auto-selected for anything above the OP_RETURN limit. | -| `P2WSH` | 476 bytes | SegWit two-phase transaction. Same 476-byte chunking as `P2SH`; payloads larger than one chunk are split across multiple P2WSH outputs (fund-then-spend pairs), up to the shared 8,192-byte compiled-ACTION ceiling (`MAX_COMPILED_ACTION_DATA_LENGTH` in `xchain-encoder/src/validator.js`, shared by all four script-output lanes, not P2WSH-specific). Best for FILE actions or large BATCH sequences. | +| `P2WSH` | 476 bytes | SegWit two-phase transaction. Same 476-byte chunking as `P2SH`; payloads larger than one chunk are split across multiple P2WSH outputs (fund-then-spend pairs), up to the shared 8,192-byte compiled-ACTION ceiling (`MAX_COMPILED_ACTION_DATA_LENGTH` in `xchain-encoder/src/common/validator.js`, shared by all four script-output lanes, not P2WSH-specific). Best for FILE actions or large BATCH sequences. | | `MULTISIGN` | 60 bytes per chunk | Data spread across fake public keys in a multisig output. Requires `compressedPubKey`. Rarely used directly. | | `TAPROOT` | 390,000 bytes of payload total, pushed in 520-byte elements | The [Taproot envelope](../../protocol/taproot-envelope.md): one `createTx` call returns the commit and reveal PSBTs together, not the `p2shHash` two-call flow. It replaces the 8,192-byte ceiling with its own `ENVELOPE_MAX_PAYLOAD` of 390,000 bytes. Bitcoin and Litecoin only (Dogecoin has no SegWit), and only at or above that chain's envelope recognition height. Requires `compressedPubKey`, which becomes the envelope's internal key, and a signer that can produce a BIP341 script-path signature. | diff --git a/components/sdk/errors.md b/components/sdk/errors.md index 5e8d4170..7f689c8d 100644 --- a/components/sdk/errors.md +++ b/components/sdk/errors.md @@ -256,7 +256,7 @@ Thrown by `AgentSession.submit()` when a declarative spending policy check fails ### SDKX402Error -Thrown by the `X402Gateway` and `X402Client` during HTTP 402 payment flows. See `src/x402.js`. +Thrown by the `X402Gateway` and `X402Client` during HTTP 402 payment flows. See `src/utils/x402.js`. | Code | Description | |------|-------------| diff --git a/components/sdk/light-client.md b/components/sdk/light-client.md index b1fd3f1a..6e9b784a 100644 --- a/components/sdk/light-client.md +++ b/components/sdk/light-client.md @@ -143,7 +143,7 @@ The signer set used for the quorum check is resolved in this order: never consulted for the set. 2. **Pinned launch set** (spec D4). With no `validators` and no `trustedCheckpoint`, the SDK consults its baked-in per-coin registry - (`src/pinnedCheckpoints.js`). When an entry is pinned, the quorum is checked + (`src/protocol/pinned_checkpoints.js`). When an entry is pinned, the quorum is checked against it and the explorer's `/verify` endpoint is **never** called. See [Pre-launch inertness](#pre-launch-inertness). 3. **Explorer convenience set** (weakest). With nothing pinned, the SDK fetches @@ -278,7 +278,7 @@ block forward-following uses. ## Pre-launch inertness -The pinned registry (`src/pinnedCheckpoints.js`) ships `null` for every real coin +The pinned registry (`src/protocol/pinned_checkpoints.js`) ships `null` for every real coin until launch values are filled in. Until then `sdk.light` behaves exactly as before the registry existed: with no `validators` and no `trustedCheckpoint`, it uses the explorer convenience set. Rotation-following is likewise inert until a diff --git a/components/sdk/nft-and-registry.md b/components/sdk/nft-and-registry.md index 3d3589a4..af81d129 100644 --- a/components/sdk/nft-and-registry.md +++ b/components/sdk/nft-and-registry.md @@ -117,11 +117,18 @@ const { doc, json } = sdk.nft.tisDocument({ imageCoin: null, // optional; base coin ticker when artwork // is on a sibling chain (e.g. 'DOGE') imageUrl: null, // optional; off-chain URL (data_ref preferred) - imageType: 'image/png', // optional; artwork MIME type + imageType: 'image/png', // optional; written verbatim to images[].type imageName: 'artwork.png' // optional; artwork filename }); ``` +`imageType` is written to the entry's `type` unchanged. In the [Token Information +Standard](../../protocol/token-information-standard.md) that field is a display +role, one of `icon`, `standard`, `large` or `hires`, so a MIME value such as +`image/png` is rejected by every published TIS schema and is skipped by the +explorer's icon selection. Pass a role token instead; the artwork's media type +comes from the referenced `FILE` action, not from this field. + **`doc` shape:** ```json diff --git a/components/sdk/sessions.md b/components/sdk/sessions.md index 76f59710..4f2639ca 100644 --- a/components/sdk/sessions.md +++ b/components/sdk/sessions.md @@ -50,7 +50,7 @@ await session.order({ ### Available Action Methods -Thirty of the 31 action types are available as convenience methods. `BATCH` is +Thirty of the 32 action types are available as convenience methods. `BATCH` is the exception: it is composed with the SDK's batch builder (`sdk.batch().send({...}).mint({...}).build()`) rather than by a session method. diff --git a/components/sync/architecture.md b/components/sync/architecture.md index afddb48d..aad4da22 100644 --- a/components/sync/architecture.md +++ b/components/sync/architecture.md @@ -89,39 +89,39 @@ flowchart TD |---|---|---| | `api.js` | None | Entry point: Express app, REST routes, WebSocket upgrade, starts SyncService | | `config.js` | `getConfig()` | Reads environment variables and returns a config object | -| `db.js` | `Database` | MariaDB connection pool with circuit breaker; one instance per chain/network | -| `middleware.js` | `authMiddleware` | API key authentication middleware for REST and WebSocket endpoints | -| `validation.js` | None | Input validation: SQL identifiers, DDL whitelisting, WebSocket event schemas | -| `utility.js` | `Utility` | `sleep()`, `getDataHash()` (SHA256), `isNull()`, timer helpers | -| `sqlUtil.js` | `splitSqlStatements` | Splits `.sql` files on `;`, stripping line comments to avoid false splits | -| `HubClient.js` | `HubClient` | JSON-RPC client for xchain-hub; `getallconfigs()` to discover indexer and decoder DB connections | +| `db/index.js` | `Database` | MariaDB connection pool with circuit breaker; one instance per chain/network | +| `http/middleware.js` | `authMiddleware` | API key authentication middleware for REST and WebSocket endpoints | +| `util/validation.js` | None | Input validation: SQL identifiers, DDL whitelisting, WebSocket event schemas | +| `util/index.js` | `Utility` | `sleep()`, `getDataHash()` (SHA256), `isNull()`, timer helpers | +| `db/sql_util.js` | `splitSqlStatements` | Splits `.sql` files on `;`, stripping line comments to avoid false splits | +| `hub/client.js` | `HubClient` | JSON-RPC client for xchain-hub; `getallconfigs()` to discover indexer and decoder DB connections | | `SyncService.js` | `SyncService` | Orchestrator: hub discovery, DB pool creation, server/client mode branching | -| `ServerPoller.js` | `ServerPoller` | Polls one indexer DB for new blocks; builds block payloads; emits events | -| `BlockBroadcaster.js` | `BlockBroadcaster` | Manages WebSocket subscriptions per chain/network; broadcasts block/reorg events | -| `SnapshotBuilder.js` | `SnapshotBuilder` | Builds full and incremental JSON snapshots with gzip streaming | -| `TransparencyLog.js` | `TransparencyLog` | Writes append-only per-block hash records to `sync_meta` table; every write is a no-op when `REPLICA_DB_READONLY=1` (see Configuration) | -| `replicatedTables.js` | `getTopology()` | Single source of truth for the block/tx/action-scoped table sets that replicate per dbType; shared by ServerPoller and the row-count completeness check | -| `updatedRows.js` | `collectUpdatedRows` | Collects in-place mutations to surviving (earlier-block) rows for a block window; the source side of the "updated rows" replication channel | -| `cooldownCredits.js` | `collectMaturedCooldownCredits` | Collects backdated cooldown-refund credits that the action-scoped join cannot reach; source side | -| `wireCodec.js` | `encodeRow`, `decodeValue` | Binary-safe row serialization: tags BLOB/Buffer column values with a `__xbin__` sentinel so they survive JSON round-trip intact | -| `BlockHasher.js` | `BlockHasher` | Independently recomputes a block's consensus hashes (ledger/actions/contract) from the replicated rows; the source of VERIFY_RECOMPUTE | +| `server/poller.js` | `ServerPoller` | Polls one indexer DB for new blocks; builds block payloads; emits events | +| `server/block_broadcaster.js` | `BlockBroadcaster` | Manages WebSocket subscriptions per chain/network; broadcasts block/reorg events | +| `server/snapshot_builder.js` | `SnapshotBuilder` | Builds full and incremental JSON snapshots with gzip streaming | +| `server/transparency_log.js` | `TransparencyLog` | Writes append-only per-block hash records to `sync_meta` table; every write is a no-op when `REPLICA_DB_READONLY=1` (see Configuration) | +| `schema/replicated_tables.js` | `getTopology()` | Single source of truth for the block/tx/action-scoped table sets that replicate per dbType; shared by ServerPoller and the row-count completeness check | +| `server/updated_rows.js` | `collectUpdatedRows` | Collects in-place mutations to surviving (earlier-block) rows for a block window; the source side of the "updated rows" replication channel | +| `server/cooldown_credits.js` | `collectMaturedCooldownCredits` | Collects backdated cooldown-refund credits that the action-scoped join cannot reach; source side | +| `util/wire_codec.js` | `encodeRow`, `decodeValue` | Binary-safe row serialization: tags BLOB/Buffer column values with a `__xbin__` sentinel so they survive JSON round-trip intact | +| `client/block_hasher.js` | `BlockHasher` | Independently recomputes a block's consensus hashes (ledger/actions/contract) from the replicated rows; the source of VERIFY_RECOMPUTE | | `stateHash.js` | `buildStateHashData` | Builds the canonical preimage for the fourth per-block replication-integrity hash (`state_hash`), covering in-place mutations and backdated credits not captured by the three consensus hashes | | `stateCommitment.js` | `computeFollowerRoots` | Follower twin of the indexer's SPV state-commitment engine; recomputes per-block SMT roots (balances, stakes, state) for VERIFY_STATE_COMMITMENT | | `merkle.js` | None | Consensus-critical SPV Merkle primitives (SHA-256 SMT, block Merkle root, state root); byte-aligned with the indexer twin | -| `MerkleTree.js` | `MerkleTree` | Binary SHA-256 Merkle tree used by TransparencyLog for epoch proof construction | -| `balance-helpers.js` | None | Shared SQL helpers for rebuilding the `balances` aggregate after a block apply or rollback | +| `server/merkle_tree.js` | `MerkleTree` | Binary SHA-256 Merkle tree used by TransparencyLog for epoch proof construction | +| `db/balance_helpers.js` | None | Shared SQL helpers for rebuilding the `balances` aggregate after a block apply or rollback | | `checkpoint.js` | None | Client-side verifier for quorum-signed state checkpoints (SPV spec §6.1/§6.3) | | `stake_weighted_quorum.js` | None | Canonical stake-weighted quorum predicate; vendored byte-identically from xchain-documentation | -| `pinnedValidators.js` | None | Out-of-band pinned validator sets used by VERIFY_CHECKPOINT_QUORUM to anchor checkpoint signatures | +| `client/pinned_validators.js` | None | Out-of-band pinned validator sets used by VERIFY_CHECKPOINT_QUORUM to anchor checkpoint signatures | | `consensus-constants.js` | None | Frozen per-chain consensus constants (e.g. `ACTIVATION_DELAY_BLOCKS`) shared across modules | -| `schema-version.js` | None | Snapshot schema version constant used to detect incompatible snapshot formats | +| `schema/version.js` | None | Snapshot schema version constant used to detect incompatible snapshot formats | | `state_commitment_activation.js` | `isStateCommitmentActive` | Flag-day gate: returns whether the SPV state-commitment feature is active for a given block and network | | `checkpoint_commitment_activation.js` | None | Flag-day gate for quorum-signed checkpoint commitment (SPV spec §6.1/§6.3 Phase 2) | | `equivocation_header.js` | None | Consensus-critical implementation of the uniform signed equivocation header (WI-2 bump 2) | -| `ClientSync.js` | `ClientSync` | Client-mode orchestrator: bootstrap, catch-up, live sync loop per chain/network | -| `ClientApplier.js` | `ClientApplier` | Applies block payloads and snapshots to local replica DB via INSERT IGNORE | -| `ClientRollback.js` | `ClientRollback` | Rollback logic mirroring indexer's Rollback.js table lists | -| `HashVerifier.js` | `HashVerifier` | Cross-source hash comparison and hash chain continuity verification | +| `client/sync.js` | `ClientSync` | Client-mode orchestrator: bootstrap, catch-up, live sync loop per chain/network | +| `client/applier.js` | `ClientApplier` | Applies block payloads and snapshots to local replica DB via INSERT IGNORE | +| `client/rollback.js` | `ClientRollback` | Rollback logic mirroring indexer's Rollback.js table lists | +| `client/hash_verifier.js` | `HashVerifier` | Cross-source hash comparison and hash chain continuity verification | ## Hub Discovery Flow diff --git a/components/sync/configuration.md b/components/sync/configuration.md index 1668c21c..31a6baff 100644 --- a/components/sync/configuration.md +++ b/components/sync/configuration.md @@ -26,6 +26,10 @@ These variables are required regardless of whether the service runs in server or | `MERKLE_EPOCH_SIZE` | No | `100` | Number of blocks per Merkle epoch in the transparency log. Changing this after a log already exists will make existing epoch roots inconsistent; only set at initial deploy. | | `TRANSPARENCY_RATE_LIMIT` | No | `10` | Maximum transparency-proof endpoint requests per minute per IP. | | `SYNC_META_RETENTION_BLOCKS` | No | `0` (off) | Transparency-log retention window in blocks. `0` or unset keeps the full log, so every historical inclusion proof stays serveable. A positive value prunes `sync_meta` rows older than the window at epoch boundaries, which bounds table growth and gives up proofs below the window. Committed Merkle roots (`merkle_epochs`) are kept either way. See [Indexer: Data Retention and Pruning](../indexer/data-retention.md). | +| `XC_ROLLCALL_REGTEST_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_ACTIVATION` in sync's copy of the activation registry (`src/consensus/gate_registry.js`), so the copy reads the row as the venue's armed indexers and hubs do. `armed` (or `genesis`/`on`/`true`/`yes`) arms at BTC height `0`; a bare non-negative integer arms at that height; `off`/`inert`/`false`/`no`/`none` and unset leave it inert, and anything else is refused with a process warning and stays inert. Applied when a row is read, from the environment as it stands then. mainnet and testnet are fixed in source and cannot be moved from the environment. Never set outside a regtest venue. | +| `XC_ROLLCALL_GATES_REGTEST_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the `regtest` entry of `ROLLCALL_GATES_ACTIVATION` (ROLLCALL v1, the consensus-gate list roll calls carry) in sync's registry copy. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; set identically on every hub, indexer and sync process in the venue. mainnet and testnet are fixed in source. Never set outside a regtest venue. | +| `XC_MIRROR_ADMISSION_ACTIVATION` | No | unset (inert) | **Regtest only.** Arms the per-coin `regtest` entries of `MIRROR_ADMISSION_ACTIVATION` and `MIRROR_ADMISSION_CONSUMER_ACTIVATION` (the mirror-admission heights) and the `regtest` entry of `ANCHOR_ATTEST_BARRIER_ACTIVATION` in sync's registry copy, one variable for the whole barrier family. Same grammar and inert default as `XC_ROLLCALL_REGTEST_ACTIVATION`; the armed form arms at height `0`. mainnet and testnet are fixed in source. Never set outside a regtest venue. | +| `SYNC_META_RETENTION_INTERVAL_MS` | No | `3600000` (1 hour) | How often, in client mode, the retention sweep that enforces `SYNC_META_RETENTION_BLOCKS` runs. A clock rather than a per-block hook, because bulk snapshot catch-up applies many blocks at once and would skip epoch-boundary events if the sweep only ran per block. No timer runs at all when the retention window is `0`. Server mode ignores this variable; it prunes at epoch boundaries instead. | ### Server Mode @@ -56,6 +60,7 @@ Setting `REPLICA_DB_HOST` overrides that default: the server instead connects to | `SYNC_REPLICA_MAX_LAG_S` | No | `120` | Replication freshness ceiling, in seconds, when the server's own database is a native SQL replica. Above this many seconds behind its source, `/status` reports `replica_stale: true` instead of certifying the served heights as current. Reading it needs the `SLAVE MONITOR` grant on MariaDB, or `REPLICATION CLIENT` on MySQL, for the server's database user; without that grant, or with the replication threads stopped, the status reads unknown and is reported stale rather than fresh. A database that is not a replica at all is unaffected. | | `REPLICA_DB_READONLY` | No | `false` | Makes the transparency log serve-only, for a server process whose database is itself a replica (for example, kept current by MariaDB binlog replication) that this process must never write to. When `true` (also accepts `1`), the four `TransparencyLog` write entry points (`recordBlock`, `commitEpoch`, `pruneFrom`, `recommitEpoch`) and the startup gap-repair scan (`ServerPoller.backfillGaps`) all become no-ops. Every read path, proofs, epoch roots, the paginated log, and the `/transparency/*` endpoints, is unaffected. See [Operations: read-only-replica deployment](operations.md#read-only-replica-deployment) for the full pattern. | | `SYNC_REPLICA_CONNECTION` | No | _(unset)_ | Name of the replication connection carrying the served schemas, on a multi-source replica. Unset reduces across every connection, worst-case: the server reads as stale if any connection is stopped, and reports the laggiest one. Naming a connection measures that stream alone, so an unrelated lagging connection cannot drag the reading; if the named connection is not present on the server, the status reads unknown and is reported stale rather than fresh. | +| `SYNC_STATUS_MAX_AGE_MS` | No | `180000` (3 min) | How long a cached status measurement stays valid, in milliseconds, before its freshness verdict expires. The server's status cache is overwritten only on a successful poll, so a hung query or a stopped poller leaves the last healthy status in place and every reader (the periodic status broadcast, new-subscriber snapshots, the validator-lag view, REST `/status`) keeps re-serving it even though transport liveness looks fine. Past this age the served heights are kept but no longer certified fresh. Uses the same 3-missed-heartbeat budget as `CLIENT_SOURCE_STALE_MS` on the client side. | ### Client Mode @@ -77,7 +82,7 @@ In client mode, the service connects to remote sync servers and replicates their | `COMPLETENESS_CHECK_INTERVAL` | No | `3600000` | **Advisory only; never halts.** How often, in milliseconds, a live client re-runs the replica-completeness sweep against its primary source: the per-table row counts the source publishes on `/status`, compared against its own, which is the only check that sees a follower missing rows the consensus hashes cannot cover. `0` disables it. Runs only when the replica and the source are at the same height, because a shortfall while behind is ordinary lag. Deliberately slow by default: the sweep makes the source run a `COUNT(*)` per replicated table. | | `REPLICA_GAP_ALERT_SWEEPS` | No | `2` | How many consecutive equal-height completeness sweeps a table must stay short before the client escalates it from the ordinary per-sweep shortfall line to the distinct, rate-limited `REPLICA_GAP_PERSISTENT` alert and records the gap in sync state for monitors. Clamped to at least `1`. Read from the client config first, then the environment. | | `REPLICA_GAP_ALERT_REPEAT_MS` | No | `21600000` (6 h) | Minimum interval, in milliseconds, between repeats of the `REPLICA_GAP_PERSISTENT` alert for a gap that is neither closing nor growing. A gap that grows re-alerts immediately regardless of this window; a gap that closes clears its state. `0` repeats on every sweep. Read from the client config first, then the environment. | -| `VERIFY_CHECKPOINT_QUORUM` | No | `false` | **Default OFF.** When `true`, anchors the replica's independently recomputed `state_root` to the federation quorum: the client fetches the source's signed checkpoint, verifies its Ed25519 signatures against the pinned validator set in `pinnedValidators.js`, and halts if the quorum fails or the checkpoint's `state_root` disagrees with the replica's own computed root. Inert without a pinned set configured for the chain/network. | +| `VERIFY_CHECKPOINT_QUORUM` | No | `false` | **Default OFF.** When `true`, anchors the replica's independently recomputed `state_root` to the federation quorum: the client fetches the source's signed checkpoint, verifies its Ed25519 signatures against the pinned validator set in `client/pinned_validators.js`, and halts if the quorum fails or the checkpoint's `state_root` disagrees with the replica's own computed root. Inert without a pinned set configured for the chain/network. | | `CHECKPOINT_VERIFY_INTERVAL` | No | `50` | How often to probe the `/latest` checkpoint, measured in applied blocks. Only used when `VERIFY_CHECKPOINT_QUORUM=true`. | | `REPLICA_DB_HOST` | Yes | None | MariaDB hostname for local replica databases. This variable, and the three below, are also honored in server mode as an opt-in override; see [Server Mode](#server-mode) above. | | `REPLICA_DB_PORT` | No | `3306` | MariaDB port | @@ -111,6 +116,16 @@ In client mode, the service connects to remote sync servers and replicates their | `STATE_TREE_METRIC_MAX_NODES` | No | `2000000` | Node ceiling for a single state-tree metric pass, bounding the cost of the sweep on a large tree. | | `SYNC_QUERY_METRIC_INTERVAL_MS` | No | `900000` (15 m) | Interval between `[METRIC] sync_action_scoped_queries_per_block` lines, which record how many action-scoped queries a block payload cost and how many of them returned rows. The per-table read loop grows with every replicated table added, so this is how the trend against poll cadence stays observable. Set to `0` to disable. Indexer pollers only (a decoder has no action-scoped tables). | +### Diagnostic Scripts (`bin/`) + +Read-only operator tools; neither broadcasts nor writes anything and neither is read by the running sync process itself. + +| Variable | Required | Default | Description | +|---|---|---|---| +| `XCHAIN_SYNC_DIR` | No | `../xchain-sync` | Sibling-checkout override `bin/lib/carrier_logic_pin.js`'s `siblingDir()` resolves for cross-repo carrier-logic comparison; the `repo_guards` twin test points it at a second checkout with `XCHAIN_REQUIRE_SIBLINGS=1`. Never read by the running sync process. | +| `XCHAIN_INDEXER_DIR` | No | `../xchain-indexer` | Sibling-checkout override for the `xchain-indexer` tree the same `siblingDir()` resolves when the pin tool compares against the indexer's canonical copy; read by literal name so the coverage gate can see it. Never read by the running sync process. | +| `XCHAIN_HUB_DIR` | No | `../xchain-hub` | Sibling-checkout override for the `xchain-hub` tree the same `siblingDir()` resolves when the pin tool compares against the hub's copy; read by literal name so the coverage gate can see it. Never read by the running sync process. | + ## Hub Discovery The service calls the local xchain-hub's `getallconfigs` JSON-RPC method at startup and every 5 minutes thereafter. The hub returns an envelope `{ configs, seq, watermark }`, where `configs` holds the nested configuration tree: @@ -214,7 +229,7 @@ The transparency log table (`sync_meta`) is not created for decoder replicas. ## Connection Pool Configuration -Each chain/network/dbType gets its own MariaDB connection pool (from `db.js`, sized by `poolSizing.js`). Pool sizes are **per dbType**, because the two dbTypes carry very different loads: the indexer pool absorbs the poller's ~113-query-per-block fan-out plus any in-flight snapshot streams, while the decoder pool replicates 8 narrow tables. +Each chain/network/dbType gets its own MariaDB connection pool (from `db/index.js`, sized by `db/pool_sizing.js`). Pool sizes are **per dbType**, because the two dbTypes carry very different loads: the indexer pool absorbs the poller's ~113-query-per-block fan-out plus any in-flight snapshot streams, while the decoder pool replicates 8 narrow tables. | Parameter | indexer | decoder | Description | |---|---|---|---| @@ -232,7 +247,7 @@ Resolution order for every knob above: `_`, then the flat `` Sizing budget: a source serving 3 chains x 2 dbTypes opens 6 pools, so the defaults cost 3 x (12 + 6) = 54 connections. -Measured on a regtest indexer schema with `test/perf/pool-fanout-load.js` (113 queries/block, 5 ms per query, median of 8 blocks): pool 3 = 213 ms/block, pool 5 = 129 ms, pool 12 = 55 ms, pool 20 = 34 ms. Run that script inside a container that already holds the DB credentials to re-measure on your own hardware before raising a pool. +Measured on a regtest indexer schema with `test/perf/helpers/pool_fanout_load.js` (113 queries/block, 5 ms per query, median of 8 blocks): pool 3 = 213 ms/block, pool 5 = 129 ms, pool 12 = 55 ms, pool 20 = 34 ms. Run that script inside a container that already holds the DB credentials to re-measure on your own hardware before raising a pool. ## Circuit Breaker diff --git a/components/sync/operations.md b/components/sync/operations.md index d4ee2c70..71451e05 100644 --- a/components/sync/operations.md +++ b/components/sync/operations.md @@ -781,7 +781,7 @@ Set `SYNC_EXCLUDE=COIN:network:dbType` (comma-separated, e.g. `DOGE:testnet:inde By default, `VERIFY_STATE_COMMITMENT=true` recomputes the per-block SPV state-commitment roots (balances and block Merkle root) from the replica and compares them to the source. If the replica was built from a truncated bootstrap, the root recompute will always diverge; disable it with `VERIFY_STATE_COMMITMENT=false` on those replicas. -`VERIFY_CHECKPOINT_QUORUM=true` anchors the replica's computed `state_root` to a quorum-signed checkpoint instead of trusting the source's claim alone. This is the only mechanism that closes the single-source trust gap that `VERIFY_STATE_COMMITMENT` (which still trusts the same source for the claimed root) cannot close. It requires a pinned validator set in `src/pinnedValidators.js` and a `CHECKPOINT_VERIFY_INTERVAL` probe interval (default: every 50 applied blocks). +`VERIFY_CHECKPOINT_QUORUM=true` anchors the replica's computed `state_root` to a quorum-signed checkpoint instead of trusting the source's claim alone. This is the only mechanism that closes the single-source trust gap that `VERIFY_STATE_COMMITMENT` (which still trusts the same source for the claimed root) cannot close. It requires a pinned validator set in `src/client/pinned_validators.js` and a `CHECKPOINT_VERIFY_INTERVAL` probe interval (default: every 50 applied blocks). `INDEX_MAP_PARITY_CHECK=true` enables an advisory index-address map consistency check. It never halts replication; mismatches are logged and counted only. Requires an index on `index_addresses.block_index` before enabling on a high-volume chain. diff --git a/components/utxo-tracker/README.md b/components/utxo-tracker/README.md index 56cec73c..dc8cfbc5 100644 --- a/components/utxo-tracker/README.md +++ b/components/utxo-tracker/README.md @@ -19,7 +19,7 @@ In addition to confirmed block data, the tracker maintains a separate in-memory - **Active-UTXO-only storage**: only unspent outputs kept in the live index; spent outputs archived temporarily for reorg recovery - **Real-time mempool tracking**: unconfirmed transactions tracked in a separate in-memory LevelDB, updated every 60 seconds - **BigInt precision**: all balance calculations use BigInt arithmetic with `satoshiToDecimalString()` conversion, eliminating floating-point errors -- **Reorg handling**: maintains a per-chain undo history (BTC: 12 blocks, LTC: 48 blocks, DOGE: 120 blocks, overridable via `XCHAIN_UNDO_BLOCKS_`) using K/M archive records and rolls back correctly on chain reorganization +- **Reorg handling**: maintains a per-chain, per-network undo history (mainnet and regtest: BTC 12 blocks, LTC 120 blocks, DOGE 120 blocks; testnet: 120 blocks for every coin; overridable via `XCHAIN_UNDO_BLOCKS_`) using K/M archive records and rolls back correctly on chain reorganization - **Concurrent block prefetch**: pre-fetches up to 10 blocks concurrently via JSON-RPC batch requests with HTTP keep-alive - **Batch writes**: LevelDB writes batched in groups of 200 blocks for throughput efficiency with atomic commit - **Two-pass transaction processing**: outputs inserted before inputs within each block, correctly handling intra-block spends diff --git a/components/utxo-tracker/architecture.md b/components/utxo-tracker/architecture.md index 0be9a180..60bb7c1a 100644 --- a/components/utxo-tracker/architecture.md +++ b/components/utxo-tracker/architecture.md @@ -50,14 +50,13 @@ flowchart TD |---|---|---| | `src/api.js` | None | Entry point: Express server, REST + JSON-RPC endpoints, env var loading, bootstrap/restore tasks | | `src/XChainUtxoTracker.js` | `XChainUtxoTracker` | Main orchestrator: block polling loop, reorg detection, two-pass transaction processing, balance queries, mempool updates | -| `src/LevelUpDb.js` | `LevelUpStore` | LevelDB abstraction: binary key encoding/decoding, batch transactions, range scans, all 12 prefix type operations | -| `src/BlockchainConnector.js` | `BlockchainConnector` | HTTP JSON-RPC client for coin node: block fetching, batch requests, mempool queries, connection pooling (25 sockets) | -| `src/XChainBlockDecoder.js` | `XChainBlockDecoder` | Block and transaction parser: standard Bitcoin blocks, AuxPoW header stripping for Dogecoin/Litecoin HogEx | -| `src/CryptoNetworks.js` | `CryptoNetworks` | Network parameter lookup: maps network names to bitcoinjs-lib network objects for 9 network variants | -| `src/util.js` | None | Utility functions: timing, hex/uint8 conversion, formatting | -| `src/bufferutils.js` | `BufferReader`, `BufferWriter` | Binary buffer reading/writing: UInt8/16/32/64LE, VarInt, slices | -| `src/db.js` | `Database` | Legacy MariaDB abstraction (connection pool, parameterized queries); not used by the main LevelDB pipeline but retained for compatibility | -| `src/fm.js` | `FileManager` | File manager: reads and writes block/transaction/input/output flat-file exports used by offline processing workflows | +| `src/store/level_up_db.js` | `LevelUpStore` | LevelDB abstraction: binary key encoding/decoding, batch transactions, range scans, all 12 prefix type operations | +| `src/chain/blockchain_connector.js` | `BlockchainConnector` | HTTP JSON-RPC facade: wires transport, batch fetching, block queries, mempool tracking, AuxPoW codec, RPC helpers, and connection pooling | +| `src/chain/blockchain_connector/` | (multiple) | Part modules: auxpow_codec (AuxPoW header encoding and decoding), batch_fetch (parallel block and hash fetching), block_queries (blockchain info and block data retrieval), rpc_helpers (RPC error handling and node health tracking), transport_and_mempool (HTTP client and mempool synchronization), constants (logging configuration) | +| `src/chain/XChainBlockDecoder.js` | `XChainBlockDecoder` | Block and transaction parser: standard Bitcoin blocks, AuxPoW header stripping for Dogecoin/Litecoin HogEx | +| `src/chain/crypto_networks.js` | `CryptoNetworks` | Network parameter lookup: maps network names to bitcoinjs-lib network objects for 9 network variants | +| `src/common/util.js` | None | Utility functions: timing, hex/uint8 conversion, formatting | +| `src/chain/bufferutils.js` | `BufferReader`, `BufferWriter` | Binary buffer reading/writing: UInt8/16/32/64LE, VarInt, slices | | `src/bulk-sync/` | (multiple) | Bulk-sync pipeline: offline parallel parse and load for initial database population on an empty DB (orchestrator, parse worker, merger, writers, loader, validator, and supporting utilities) | ## LevelDB Key Schema @@ -91,7 +90,7 @@ Two string keys are also used as checkpoints: **H key (output hint)**: Maps an outpoint (txHash8 + index) back to its scriptHash. When processing an input that spends an output, the tracker reads the H hint to find the scriptHash, then deletes the corresponding O record. Without H, the tracker would need to scan all O records to find the one being spent. -**K/M keys (deleted archives)**: When a UTXO is spent, the O and H records are deleted, but copies are saved as K and M records keyed by blockHash. If a reorg rolls back that block, the K/M records are restored to O/H. After `DEFAULT_UNDO_BLOCKS` (BTC: 12, LTC: 120, DOGE: 120) subsequent blocks, the K/M records are purged. +**K/M keys (deleted archives)**: When a UTXO is spent, the O and H records are deleted, but copies are saved as K and M records keyed by blockHash. If a reorg rolls back that block, the K/M records are restored to O/H. After `DEFAULT_UNDO_BLOCKS` subsequent blocks (mainnet and regtest: BTC 12, LTC 120, DOGE 120; testnet: 120 for every coin), the K/M records are purged. **txHash8 truncation**: Transaction hashes are truncated to 8 bytes in index keys (T, I, O, H, J, K, M, W). The full 32-byte hash is stored in O values for API responses. 8-byte truncation provides sufficient uniqueness for index lookups while halving key sizes. @@ -172,7 +171,7 @@ flowchart TD DETECT --> WALK --> FORK --> ROLLBACK --> RESET --> RESUME ``` -The undo window is determined per chain: BTC 12 blocks, LTC 48 blocks, DOGE 120 blocks (overridable via `XCHAIN_UNDO_BLOCKS_`). Reorgs exceeding the configured window throw an error and require a full re-index. +The undo window is determined per chain and per network (mainnet and regtest: BTC 12 blocks, LTC 120 blocks, DOGE 120 blocks; testnet: 120 blocks for every coin; overridable via `XCHAIN_UNDO_BLOCKS_`). Reorgs exceeding the configured window throw an error and require a full re-index. ## Mempool Tracking diff --git a/components/utxo-tracker/configuration.md b/components/utxo-tracker/configuration.md index 04675c07..9760d66d 100644 --- a/components/utxo-tracker/configuration.md +++ b/components/utxo-tracker/configuration.md @@ -35,9 +35,9 @@ BULK_SYNC_RAM_BUDGET=768 | `UTXO_MAX_PAGE_LIMIT` | Maximum page size a caller may request via `?limit=`. Caps a single request so a caller cannot trigger an OOM by requesting one giant page. Independent of `UTXO_MAX_ADDRESS_OUTPUTS`. | `10000` | | `UTXO_MAX_ADDRESS_OUTPUTS` | Hard ceiling on outputs materialized for a single-address unbounded query; above this limit `/utxos` and `get_balance` return HTTP 413: callers must page via `?limit=&after=` | `500000` | | `CORS_ORIGIN` | Allowed CORS origins: either one origin, or a comma-separated allowlist matched per origin, for example `capacitor://localhost,https://localhost,https://explorer.xchain.io`. Browser shells need the list form because each surface sends a different origin. Entries are trimmed and blank ones dropped, so an empty or all-blank value disables CORS (no CORS header) exactly as leaving it unset does. `*` means "any origin" only when it is the entire value; inside a list it stays a literal entry no browser sends, so `*,https://x` grants `https://x` and nothing more. | `""` (disabled) | -| `XCHAIN_UNDO_BLOCKS_BTC` | Override the BTC reorg recovery window (blocks) | `12` | -| `XCHAIN_UNDO_BLOCKS_LTC` | Override the LTC reorg recovery window (blocks) | `48` | -| `XCHAIN_UNDO_BLOCKS_DOGE` | Override the DOGE reorg recovery window (blocks) | `120` | +| `XCHAIN_UNDO_BLOCKS_BTC` | Override the BTC reorg recovery window (blocks). The default is per network: 12 on mainnet and regtest, 120 on testnet (a testnet's minimum-difficulty rule forks far deeper than block time predicts; bitcoin testnet outran 12 on 2026-09-15). One key per coin is enough because a tracker process serves exactly one network. Values above 126 (the decoder's `DISPENSER_EXPIRE_SAFE_DEPTH`) are honoured but logged as splitting the tracker's and decoder's reorg windows. | `12` mainnet/regtest, `120` testnet | +| `XCHAIN_UNDO_BLOCKS_LTC` | Override the LTC reorg recovery window (blocks). 120 on every network since litecoin testnet outran the previous 48 on 2026-09-01. | `120` | +| `XCHAIN_UNDO_BLOCKS_DOGE` | Override the DOGE reorg recovery window (blocks). 120 on every network. | `120` | | `UTXO_TRACKER_RATE_LIMIT_RPM` | API requests per minute per IP | `500` | | `UTXO_TRACKER_NODE_RPC_STALE_MS` | Staleness window for the tracker`s last usable node-tip read, after which `health` reports the node RPC stale. Five times the loop`s `BLOCKCHAIN_INFO_REFRESH_MS` (30s), so a slow or skipped poll never trips it and only a sustained outage does. | `150000` | | `UTXO_MAX_RPC_BATCH` | Maximum calls accepted in one inbound JSON-RPC batch (array body). The router runs `Promise.all` over every element, so without this cap a single unauthenticated ~100kb POST fans out into thousands of concurrent read scans and node RPCs. Mirrors the decoder and encoder batch guards. | `20` | @@ -126,7 +126,8 @@ These values are defined in `src/XChainUtxoTracker.js` and are not configurable | `DB_TRANSACTION_BLOCKS_QUANTITY` | `200` | Number of blocks per LevelDB batch commit | | `PREFETCH_SIZE` | `10` | Number of blocks pre-fetched concurrently | | `ETA_WINDOW_BLOCKS` | `1000` | Rolling window size for sync ETA calculation | -| `DEFAULT_UNDO_BLOCKS` | BTC: `12` / LTC: `120` / DOGE: `120` | Per-chain K/M archive retention window; override per coin via `XCHAIN_UNDO_BLOCKS_BTC`, `XCHAIN_UNDO_BLOCKS_LTC`, `XCHAIN_UNDO_BLOCKS_DOGE` | +| `DEFAULT_UNDO_BLOCKS` | mainnet BTC: `12` / LTC: `120` / DOGE: `120`; testnet `120` for every coin; regtest same as mainnet | Per-coin, per-network K/M archive retention window (`src/chain/undo_blocks.js`, keyed `_`); override per coin via `XCHAIN_UNDO_BLOCKS_BTC`, `XCHAIN_UNDO_BLOCKS_LTC`, `XCHAIN_UNDO_BLOCKS_DOGE` | +| `MAX_SAFE_UNDO_BLOCKS` | `126` | Ceiling any resolved window is checked against; equals the decoder's `DISPENSER_EXPIRE_SAFE_DEPTH` and the two move in lockstep | ### Storage diff --git a/components/utxo-tracker/operations.md b/components/utxo-tracker/operations.md index c23d3cc5..fb1d17a7 100644 --- a/components/utxo-tracker/operations.md +++ b/components/utxo-tracker/operations.md @@ -69,7 +69,7 @@ The tracker exposes both REST and JSON-RPC interfaces. | `GET` | `/firstseen/:address` | Returns the block height at which the address first appeared (`{"height": N}`) | | `GET` | `/balance/:address` | Returns the confirmed balance as a number (in coin units, not satoshis) | | `GET` | `/info/:address` | Returns comprehensive balance info (confirmed, pending, received, UTXO counts) | -| `GET` | `/status` | Lightweight health probe for Docker HEALTHCHECK and uptime monitors: `{status, db, committed_height}`, HTTP 503 when the LevelDB store is unreachable. Point health checks here; a plain GET against the JSON-RPC root always answers 200 (method-not-found body) even when the DB is down. | +| `GET` | `/status` | Lightweight health probe for Docker HEALTHCHECK and uptime monitors: `{status, db, committed_height}`, HTTP 503 when the LevelDB store is unreachable or the tracker has halted on an unrecoverable reorg (`status: "halted"` plus `halt_reason`; see "Reorg exceeds the undo window" under Troubleshooting). Point health checks here; a plain GET against the JSON-RPC root always answers 200 (method-not-found body) even when the DB is down. | #### GET /utxos/:address @@ -188,7 +188,7 @@ When a blockchain reorganization is detected: 1. The tracker walks back from its tip until it finds a block hash matching the coin node 2. Each rolled-back block's outputs are restored from K/M archive records 3. Normal forward indexing resumes from the fork point -4. Reorgs deeper than the chain's undo window (BTC: 12, LTC: 48, DOGE: 120) require a full re-index +4. Reorgs deeper than the network's undo window (see the table under "Reorg exceeds the undo window" below) halt the tracker; the only exit is a rebuild ### Mempool Error Handling @@ -213,7 +213,27 @@ LevelDB only allows one process to open a database at a time. If the tracker cra Large blocks (e.g., BRC-20 inscription blocks) can contain tens of thousands of transactions. The tracker may appear stalled but is processing normally. Check the console output for progress updates. If the process runs out of memory, increase `--max-old-space-size`. **Reorg exceeds the undo window** -The tracker throws an error and stops if a reorg exceeds the chain's undo window (BTC: 12 blocks, LTC: 48 blocks, DOGE: 120 blocks). This is rare on mainnet but can occur on testnet/regtest. The solution is to delete the LevelDB database and re-index from scratch. +The tracker can only roll back as many blocks as its undo window holds, and the window is sized per coin AND per network (`src/chain/undo_blocks.js`): + +| Network | BTC | LTC | DOGE | +|---|---|---|---| +| mainnet | 12 | 120 | 120 | +| testnet | 120 | 120 | 120 | +| regtest | 12 | 120 | 120 | + +Mainnet windows are block-time-scaled (about two hours of headroom). Every testnet sits at 120 because a testnet's minimum-difficulty rule lets a lone miner extend a private branch regardless of network difficulty, so forks run far deeper than block time predicts: litecoin testnet outran a 48-block window on 2026-09-01 and bitcoin testnet outran mainnet's 12 on 2026-09-15. `XCHAIN_UNDO_BLOCKS_` overrides the resolved value for the process's own network; 126 is the ceiling (the decoder's `DISPENSER_EXPIRE_SAFE_DEPTH`), and a larger override is honoured but logged as splitting the two components' reorg windows. + +When a fork is deeper than the window the tracker does NOT exit. It halts in place: the polling loop stops, the process stays up, `GET /status` answers HTTP 503 with `{"status": "halted", "halt_reason": "...", "halted_at": ..., "halted_height": ...}`, and `get_sync_status` carries `halted: true` and `halt_reason`. The halt is a memory flag, but the state behind it is on disk (the undo window has been walked down, partly or to zero), so a restart or a `recreate` reproduces the same halt within seconds: the log shows `verifyReorg: reorg depth exceeds the recovery window (UNDO_BLOCKS=N)`, or on a window already drained to zero `Can't delete a block from 'last blocks': list is empty`. Both log lines end with the remedy. + +The remedy is a rebuild; the index cannot be walked back onto the node's chain. Under xchain-node: + +```bash +xchain-node reset xchain-utxo-tracker # e.g. bitcoin testnet +``` + +`reset` drops the tracker's data volume and the next start takes the bulk-sync path (`runBulkSyncIfEmpty()`, see "Startup Sequence" above). Standalone: stop the tracker, empty its data directory (`/data/xchain-utxo-tracker` in the container image), and start it again. + +Do not "recover" by restoring the bootstrap you came from: if that bootstrap's tip is the drifted fork, the restore lands on the same block and halts again at the same height. Only restore a bootstrap taken after the fork resolved. ### Data inconsistency diff --git a/components/vm/README.md b/components/vm/README.md index 18ccfe35..ba078339 100644 --- a/components/vm/README.md +++ b/components/vm/README.md @@ -7,7 +7,7 @@ The `xchain-vm` module is a standalone JavaScript library that executes smart co ## What is xchain-vm -A pure function library. Takes contract code + state + inputs + block context. Returns new state + emitted actions + gas used. It has no awareness of the indexer, database, blockchain, or network. The indexer's `execute.js` handler is the bridge between the VM and the platform. +A pure function library. Takes contract code + state + inputs + block context. Returns new state + emitted actions + gas used. It has no awareness of the indexer, database, blockchain, or network. The indexer's `execute/index.js` handler is the bridge between the VM and the platform. ## Features diff --git a/components/vm/architecture.md b/components/vm/architecture.md index 4c5317d4..4f68208d 100644 --- a/components/vm/architecture.md +++ b/components/vm/architecture.md @@ -20,7 +20,7 @@ flowchart TD STATE["state.js -> stateChanges, stateDeletes"] COLLECTOR["collector.js -> emittedActions, logs"] GASRESULT["gas.js -> gasUsed"] - RETURN["Return to indexer (execute.js)"] + RETURN["Return to indexer (execute/index.js)"] SRC --> ACORN --> METER --> ASTR --> ISOLATE ISOLATE --> SANDBOX @@ -49,13 +49,13 @@ flowchart TD | `metering.js` | AST-based gas injection: parses source with acorn, injects `__gas()` at control flow points, regenerates with astring. Also provides `hasGasIdentifier()` for deploy-time validation | | `gas.js` | GasTracker class: validates gas schedule (non-negative integers), accumulates gas charges per operation, enforces ceiling, throws GasExhaustedError on overflow | | `gateway.js` | Builds the `xchain` gateway object: context accessors, state CRUD, ledger queries, oracle, cross-chain, **external attestation (`xchain.attestation.*`)**, **contract-targeted staking (`xchain.contract.*`)**, emit API, math, control flow, logging | -| `gateway-emit.js` | Emit API builder: 19 action types (SEND through MESSAGE and VOTE, plus `execute` for cross-contract calls and `crossExecute` for cross-chain calls), parameter validation, gas charging | +| `gateway_emit.js` | Emit API builder: assembles the 19 action types (SEND through MESSAGE and VOTE, plus `execute` for cross-contract calls and `crossExecute` for cross-chain calls) and charges gas per emit. It keeps `crossExecute`, the id preimage builders and their golden vectors; the same-chain emits (`execute` and SEND through VOTE) live in `gateway_emit/same_chain.js` and the shared parameter checks in `gateway_emit/param_validation.js` | | `math.js` | Deterministic math wrapping mathjs bignumber: all inputs are strings, arithmetic results are strings, `compare` returns a number (-1/0/1) and `gt`/`gte`/`lt`/`lte`/`eq`/`isZero` return booleans; wrapped in `safeMath` for ContractRevertError on failures | | `state.js` | StateManager: reads from initial snapshot, tracks writes/deletes in dirty map, enforces key count, key size, and value size limits, provides `getChanges()` for result collection | | `collector.js` | EmissionCollector: queues emitted actions (with emission cap), collects debug logs (100 entries, 1 KB UTF-8 each, with byte-aware truncation) | | `validator.js` | ActionValidator: pre-validates emitted actions against the 21 allowed action types (`SEND`, `DESTROY`, `ISSUE`, `MINT`, `ORDER`, `DISPENSER`, `DIVIDEND`, `AIRDROP`, `CALLBACK`, `FILE`, `LIST`, `COINPAY`, `SWEEP`, `LINK`, `BROADCAST`, `MESSAGE`, `ATTEST`, `SLASH`, `EXECUTE`, `XCALL`, `VOTE`) and checks params shape | -| `lint-core.js` | Dependency-light (no isolated-vm) acorn-only consensus validation, exporting `lintSource()`, `analyzeContract()`, and per-rule detectors: `findBannedMathCalls`, `findBannedLiterals`, `findBannedAsync`, `findBannedGenerator` (`function*`/`yield`), `findBannedWasm` (global `WebAssembly` reference), `findBannedExponentiation` (`**`/`**=`, hardened-only), `findBannedProtoMethods` (advisory, name-only match, never consensus), `findReservedControlBinding` (`CONTRACT_WRAPPER` bindings, hardened-only), and `codeSizeBytes`/`MAX_CODE_SIZE` for the 64KiB deploy cap. Under `VM_LINT_HARDENING`, the Math ban widens from the five named transcendentals to the complement of the `SAFE_MATH_MEMBERS` whitelist. Holds `CONSENSUS_RULES` (the 8-member set of deploy-blocking rule names: `invalid-type`, `unsupported-syntax`, `reserved-identifier`, `banned-math`, `banned-literal`, `banned-async`, `banned-generator`, `banned-wasm`) and `lintSource()`. Vendored byte-for-byte into `xchain-sdk/src/contract/lint-core.js`; a CI SHA-256 parity guard fails on drift. | -| `syntax.js` | Deploy-time validation: V8 syntax check (throwaway isolate, the one step requiring isolated-vm; blocks via a separate early return and is not itself a `CONSENSUS_RULES` member), then delegates all acorn-coverable consensus rules to `lint-core.lintSource()`. Re-exports `findBannedMathCalls`, `findBannedLiterals`, `findBannedAsync`, `findBannedGenerator`, and `findBannedWasm` for backward compatibility. `validateSyntax(code, opts)` threads four independent consensus gates: `enforceBannedAsync` (the `VM_BANNED_ASYNC` flag-day), `enforceLintHardening` (the `VM_LINT_HARDENING` flag-day, widening the Math ban and adding the exponentiation/reserved-binding/dynamic-import rules), and `enforceBannedGenerator`/`enforceBannedWasm` (the Package 3 per-coin sandbox height gate, active from genesis on testnet/regtest). Each defaults to `true` for author-facing callers so a from-genesis replay reproduces the historical deploy verdict at every activation. | +| `lint_core.js` | Dependency-light (no isolated-vm) acorn-only consensus validation, exporting `lintSource()`, `analyzeContract()`, and per-rule detectors: `findBannedMathCalls`, `findBannedLiterals`, `findBannedAsync`, `findBannedGenerator` (`function*`/`yield`), `findBannedWasm` (global `WebAssembly` reference), `findBannedRest` (unmeterable rest positions), `findBannedExponentiation` (`**`/`**=`, hardened-only), `findBannedProtoMethods` (advisory, name-only match, never consensus), `findReservedControlBinding` (`CONTRACT_WRAPPER` bindings, hardened-only), and `codeSizeBytes`/`MAX_CODE_SIZE` for the 64KiB deploy cap. Under `VM_LINT_HARDENING`, the Math ban widens from the five named transcendentals to the complement of the `SAFE_MATH_MEMBERS` whitelist. Holds `CONSENSUS_RULES` (the 9-member set of deploy-blocking rule names: `invalid-type`, `unsupported-syntax`, `reserved-identifier`, `banned-math`, `banned-literal`, `banned-async`, `banned-generator`, `banned-rest`, `banned-wasm`) and `lintSource()`. Vendored byte-for-byte into `xchain-sdk/src/contract/lint_core.js`; a CI SHA-256 parity guard fails on drift. | +| `syntax.js` | Deploy-time validation: V8 syntax check (throwaway isolate, the one step requiring isolated-vm; blocks via a separate early return and is not itself a `CONSENSUS_RULES` member), then delegates all acorn-coverable consensus rules to `lint-core.lintSource()`. Re-exports `findBannedMathCalls`, `findBannedLiterals`, `findBannedAsync`, `findBannedGenerator`, `findBannedWasm`, and `findBannedRest` for backward compatibility. `validateSyntax(code, opts)` threads five independent consensus gates: `enforceBannedAsync` (the `VM_BANNED_ASYNC` flag-day), `enforceLintHardening` (the `VM_LINT_HARDENING` flag-day, widening the Math ban and adding the exponentiation/reserved-binding/dynamic-import rules), `enforceBannedGenerator`/`enforceBannedWasm` (the Package 3 per-coin sandbox height gate, active from genesis on testnet/regtest), and `enforceBannedRest` (the separate [`REST_PATTERN_METER`](../../protocol/flag-days.md) gate, likewise active from genesis on testnet/regtest). Each defaults to `true` for author-facing callers so a from-genesis replay reproduces the historical deploy verdict at every activation. | | `errors.js` | ContractRevertError (thrown by `revert()`/`require()`) and GasExhaustedError (thrown when gas ceiling exceeded) | ## JSON Bridge Protocol diff --git a/components/vm/configuration.md b/components/vm/configuration.md index ede03b42..b88abadb 100644 --- a/components/vm/configuration.md +++ b/components/vm/configuration.md @@ -100,11 +100,13 @@ These limits are hardcoded in the VM and not configurable: |---|---|---| | Log entries per execution | 100 | `collector.js` | | Log entry size | 1,024 bytes UTF-8 (truncated with `...(truncated)` marker) | `collector.js` | -| Return value size | 65,536 bytes (truncated) | `index.js` | +| Return value size | 65,536 UTF-16 code units (truncated) | `index.js` | | Recursion depth (`__DEPTH_LIMIT`) | 512, or 256 once the Package 3 sandbox gate is active (unconditional on testnet/regtest; per-coin heights on mainnet) | `index.js` | | Throwaway isolate memory | 8 MB | `isolate.js`, `syntax.js` | | Binary expression metering depth | 10 | `metering.js` | +The return-value cap is measured in UTF-16 code units, with `String.length` over the serialized value, not in UTF-8 bytes: a non-ASCII return at the cap can occupy up to roughly 196,608 UTF-8 bytes, so size a return against code units rather than the encoded size. Truncation is a UTF-16 `substring`, so a value cut in the middle of a surrogate pair ends with an unpaired surrogate. The log-entry limit directly above is genuinely byte-measured and is the contrasting case. + ## Bounded Execution Summary | Resource | Limit | Enforcement | diff --git a/components/vm/operations.md b/components/vm/operations.md index aedb7842..b1ec9f23 100644 --- a/components/vm/operations.md +++ b/components/vm/operations.md @@ -34,14 +34,14 @@ The VM maintains **1,250+ total tests** across unit, E2E, security, fuzz, chaos, ## Integration with the Indexer -The VM is instantiated once in the indexer's `actions.js` and shared across all action handlers for the lifetime of the indexer process. +The VM is instantiated once in the indexer's `src/actions/index.js` and shared across all action handlers for the lifetime of the indexer process. ### Lifecycle 1. **Startup:** Indexer creates `new XChainVM({ gasSchedule, gasCeiling, limits })` from its configuration 2. **Per block:** Indexer calls `vm.beginBlock()` before processing transactions, `vm.endBlock()` after -3. **DEPLOY action:** `deploy.js` calls `vm.validateSyntax(code)` to validate contract source, then `vm.execute()` to run the constructor -4. **EXECUTE action:** `execute.js` calls `vm.execute()` with the contract code, current state, method name, parameters, and block context +3. **DEPLOY action:** `deploy/index.js` calls `vm.validateSyntax(code)` to validate contract source, then `vm.execute()` to run the constructor +4. **EXECUTE action:** `execute/index.js` calls `vm.execute()` with the contract code, current state, method name, parameters, and block context 5. **Result processing:** The indexer applies `stateChanges` and `stateDeletes` to the database, processes `emittedActions` through standard action handlers, and records `gasUsed` for fee charging ```mermaid @@ -56,11 +56,11 @@ sequenceDiagram Indexer->>VM: vm.beginBlock() Indexer->>VM: vm.endBlock() - Note over Indexer: DEPLOY action (deploy.js) + Note over Indexer: DEPLOY action (deploy/index.js) Indexer->>VM: vm.validateSyntax(code) Indexer->>VM: vm.execute() (run constructor) - Note over Indexer: EXECUTE action (execute.js) + Note over Indexer: EXECUTE action (execute/index.js) Indexer->>VM: vm.execute(code, state, method, params, block context) Note over Indexer: Result processing @@ -71,7 +71,7 @@ sequenceDiagram ```mermaid flowchart TD - INDEXER["Indexer (execute.js)"] + INDEXER["Indexer (execute/index.js)"] LOAD["Loads contract code + state from DB"] BUILD["Builds balances, tokenInfo,
oracleData, crossChainData"] EXEC["vm.execute({ code, state, method,
params, caller, ... })"] @@ -94,7 +94,7 @@ flowchart TD Beside `validateSyntax` and `execute`, the indexer calls `vm.readManifest(code, opts)` once per DEPLOY. It instantiates the module's top level inside a gas-metered isolate (no state, oracle, or balances) and reports what the exported object declares, **without dispatching any method**, so it works for a contract that exports no constructor. It resolves `{ success, manifest, error }`; on a module-level throw, `success` is `false`. -The VM **reports; it never judges**. Every verdict lives host-side in the indexer's `actions/deploy.js`, which is why the report is deliberately raw and typed: +The VM **reports; it never judges**. Every verdict lives host-side in the indexer's `actions/deploy/index.js`, which is why the report is deliberately raw and typed: | Field | Type | Meaning | |---|---|---| @@ -144,7 +144,7 @@ The indexer uses database savepoints to ensure these guarantees extend to the pe ## Syntax Validation (Deploy-Time) -Before a contract is deployed, `vm.validateSyntax(code)` runs the following checks in order. The V8 syntax check blocks a deploy via a separate early return and is not itself a `lint-core.CONSENSUS_RULES` member; checks 2-8 below are all deploy-blocking consensus rules (`invalid-type`, `unsupported-syntax`, `reserved-identifier`, `banned-math`, `banned-literal`, `banned-async`, `banned-generator`, `banned-wasm`), and all must pass or the DEPLOY action is rejected: +Before a contract is deployed, `vm.validateSyntax(code)` runs the following checks in order. The V8 syntax check blocks a deploy via a separate early return and is not itself a `lint-core.CONSENSUS_RULES` member; checks 2-9 below are all deploy-blocking consensus rules (`invalid-type`, `unsupported-syntax`, `reserved-identifier`, `banned-math`, `banned-literal`, `banned-async`, `banned-generator`, `banned-rest`, `banned-wasm`), and all must pass or the DEPLOY action is rejected: 1. **V8 syntax check**: compiles the code in a throwaway 8 MB isolate to catch syntax errors (the only step requiring `isolated-vm`) 2. **Acorn metering pass**: runs `meterCode()` to ensure acorn can parse the source (effective ES2020 ceiling) @@ -153,7 +153,8 @@ Before a contract is deployed, `vm.validateSyntax(code)` runs the following chec 5. **Banned literal check**: rejects BigInt literals (e.g. `10n`) and RegExp literals (e.g. `/foo/`). BigInt arithmetic is unmetered native computation; catastrophic RegExp backtracking is unmetered and can burn heavy CPU for near-zero gas. 6. **Banned async check** (consensus-gated): rejects `async` functions, `await` expressions, and `Promise` references after the `VM_BANNED_ASYNC` flag-day. The CONTRACT_WRAPPER invokes exports synchronously; an async export returns a pending Promise whose post-`await` effects depend on isolated-vm's version-dependent microtask-drain timing, which is outside the consensus-runtime pin and can diverge across validators. Under `VM_LINT_HARDENING` this also rejects dynamic `import(...)` (it evaluates to a Promise). 7. **Banned generator check** (consensus-gated, Pkg 3 sandbox): rejects `function*`, generator methods, and any `yield`; live from genesis on testnet/regtest. -8. **Banned WebAssembly check** (consensus-gated, Pkg 3 sandbox): rejects any reference to the global `WebAssembly`; live from genesis on testnet/regtest. +8. **Banned rest-pattern check** (consensus-gated, its own [`REST_PATTERN_METER`](../../protocol/flag-days.md) gate, not the Pkg 3 one): rejects a rest pattern in the four positions the metering transform cannot charge, because it charges a rest destructure by wrapping the source expression and these have none: a rest parameter in a function parameter list, a nested rest inside a destructuring pattern, a catch-clause rest, and a rest in a `for-of`/`for-in` loop head. Live from genesis on testnet/regtest, and on mainnet at/after that gate's block time; the `xchain-lint` CLI and the SDK linter enforce it today by default. +9. **Banned WebAssembly check** (consensus-gated, Pkg 3 sandbox): rejects any reference to the global `WebAssembly`; live from genesis on testnet/regtest. ```mermaid flowchart TD @@ -165,7 +166,8 @@ flowchart TD S5{"5. Banned literal check"} S6{"6. Banned async check (VM_BANNED_ASYNC flag-day)"} S7{"7. Banned generator check (live from genesis, testnet/regtest)"} - S8{"8. Banned WebAssembly check (live from genesis, testnet/regtest)"} + S8{"8. Banned rest-pattern check (REST_PATTERN_METER gate)"} + S9{"9. Banned WebAssembly check (live from genesis, testnet/regtest)"} ACCEPT["DEPLOY accepted"] REJECT["DEPLOY rejected"] @@ -184,8 +186,10 @@ flowchart TD S6 -->|"clean, or flag-day not active"| S7 S7 -->|"generator or yield found"| REJECT S7 -->|"clean"| S8 - S8 -->|"WebAssembly reference found"| REJECT - S8 -->|"clean"| ACCEPT + S8 -->|"unmeterable rest pattern found, gate active"| REJECT + S8 -->|"clean, or gate not active"| S9 + S9 -->|"WebAssembly reference found"| REJECT + S9 -->|"clean"| ACCEPT ``` `vm.checkFloatWarnings(code)` additionally scans for non-integer number literals and returns warnings (non-blocking). diff --git a/components/wallet/architecture.md b/components/wallet/architecture.md index cc33a7a3..23d409fb 100644 --- a/components/wallet/architecture.md +++ b/components/wallet/architecture.md @@ -39,9 +39,9 @@ Each shell wraps the core in a small amount of host-specific glue: | `@xchain-wallet/web` | Vite SPA + `hostBridge.js` | IndexedDB | in-memory only | | `@xchain-wallet/extension` | service worker + content script + injected provider + popup + approval window | `chrome.storage.local` | `chrome.storage.session` | | `@xchain-wallet/desktop` | Electron main / preload / renderer | encrypted file via main process | OS keychain (optional) | -| `@xchain-wallet/mobile` | Capacitor WebView wrapping the built `@xchain-wallet/web` SPA verbatim, no UI or glue of its own | IndexedDB (same as web) | in-memory only (same as web) | +| `@xchain-wallet/mobile` | Capacitor WebView over the built `@xchain-wallet/web` SPA, plus native glue of its own: the `XChainVault` plugin and a biometric sidecar on both Android and iOS | app-private file written by the native vault plugin, encrypted under an OS-keystore key (Android Keystore / iOS Keychain, device-bound and excluded from backups) | in-memory only (same as web) | -Every route renders the same React tree across shells; only the host bridge differs. Mobile is a wrapper, not a port: it packages the web shell's own build, so it inherits the web shell's bridge, vault, and session behavior unchanged. +Every route renders the same React tree across shells; only the host bridge differs. Mobile packages the web shell's own build and inherits its bridge and its in-memory session behavior, but it does **not** inherit the web vault. WebView storage is evictable, and in an installed app whose backup posture is deliberately off it would hold the only copy of the vault in existence, so the web build detects the native shell at boot and swaps its IndexedDB and `localStorage` backends for the `XChainVault` plugin that ships in `@xchain-wallet/mobile` (`packages/web/src/storage/backends.js`, `CapacitorStorageBackend.js`). A native shell whose plugin failed to register is a blocking error, never a silent fallback to WebView storage. ## Package boundaries @@ -51,7 +51,7 @@ flowchart TD WEB["@xchain-wallet/web
Vite SPA
hostBridge.js
sdkFactory.js"] EXT["@xchain-wallet/extension
background + content + popup +
approval + inject"] DESKTOP["@xchain-wallet/desktop
Electron main +
preload + renderer"] - MOBILE["@xchain-wallet/mobile
Capacitor (Android/iOS)
wraps the web shell's build"] + MOBILE["@xchain-wallet/mobile
Capacitor (Android/iOS)
hosts the web shell's build +
native XChainVault plugin"] SDK["xchain-sdk (sibling repo)
actions + encoder +
explorer + hub + ws"] CORE --> WEB @@ -78,9 +78,9 @@ Sibling packages alongside the four shells: Each of the three shells that build against core directly (web, extension, desktop) registers *two* host functions with it: 1. **SDK factory**: `core/src/sdk/SDKRegistry` calls a host-supplied factory to mint per-chain SDK instances. Web/desktop instantiate `xchain-sdk` directly; the extension instantiates the SDK in the service worker and routes calls from popup / approval / full-screen via `MessageHost`. -2. **Storage backend**: `core/src/storage/backend.js` selects between IndexedDB (web), `chrome.storage.local` (extension), and a file-backed adapter (desktop main process). Vault encryption / decryption is identical across all three. +2. **Storage backend**: the host selects between IndexedDB (web), `chrome.storage.local` (extension), a file-backed adapter (desktop main process), and the native `XChainVault` plugin (mobile). Vault encryption / decryption is identical across all of them: what changes is where the ciphertext lands. -Mobile registers neither: it has no seam of its own, since it packages the web shell's already-built SDK factory and IndexedDB storage backend verbatim. +Mobile registers one of the two. It reuses the web shell's already-built SDK factory unchanged, so it has no SDK seam of its own; but it does register a storage backend, because the web build resolves that seam at boot (`packages/web/src/storage/backends.js`) and picks the Capacitor backends over IndexedDB whenever the native vault plugin is present. The kdfParams record and the pre-unlock guard record (lockout ladder, duress hash, panic freeze) move into native slots with it, for the same reason the blob does. ## Signal flow @@ -128,7 +128,7 @@ flowchart TD CLICK --> FLOW --> SEND --> CREATE --> SIGN --> BROADCAST --> WAIT ``` -The path is the same in the web shell (signer runs in the page), the extension (signer runs in the service worker), and the desktop app (signer runs in the main process). Mobile follows the web shell's path exactly, since it runs the same build in a Capacitor WebView. Differences live entirely behind the SDK factory + storage backend seams. +The path is the same in the web shell (signer runs in the page), the extension (signer runs in the service worker), and the desktop app (signer runs in the main process). Mobile runs the web shell's build in a Capacitor WebView, so the signing path is the web shell's; its storage reads and writes go out through the native vault plugin instead of IndexedDB. Differences live entirely behind the SDK factory + storage backend seams. ## Vault and state model @@ -153,11 +153,13 @@ Master key derivation: password → Argon2id (calibrated per device, floor 64 Mi Each shell maps this same logical schema onto a different physical store: -| Logical store | Web | Extension | Desktop | -|---|---|---|---| -| Vault (encrypted seed, accounts, addresses, contacts, settings, connected sites) | IndexedDB | `chrome.storage.local` | Electron `userData` (encrypted file) | -| Session (master key after unlock) | in-memory only | `chrome.storage.session` (cleared on browser close) | OS keychain (with consent) or in-memory | -| Ephemeral metadata (toast state, demo flag, last-view) | `localStorage` | `localStorage` | `localStorage` | +| Logical store | Web | Extension | Desktop | Mobile | +|---|---|---|---|---| +| Vault (encrypted seed, accounts, addresses, contacts, settings, connected sites) | IndexedDB | `chrome.storage.local` | Electron `userData` (encrypted file) | app-private file via the native `XChainVault` plugin, under an OS-keystore key (Android Keystore / iOS Keychain) | +| Session (master key after unlock) | in-memory only | `chrome.storage.session` (cleared on browser close) | OS keychain (with consent) or in-memory | in-memory only | +| Ephemeral metadata (toast state, demo flag, last-view) | `localStorage` | `localStorage` | `localStorage` | `localStorage` (WebView) | + +The mobile column is not the web column: the vault blob, the kdfParams record and the pre-unlock guard record all sit in native slots rather than in WebView storage. Only genuinely ephemeral UI state is left in the WebView's own `localStorage`, where losing it costs nothing. ## Schema migrations diff --git a/components/wallet/build-release.md b/components/wallet/build-release.md index eb7c5956..c59f224a 100644 --- a/components/wallet/build-release.md +++ b/components/wallet/build-release.md @@ -121,7 +121,7 @@ The CWS submission packet covers: - Post-approval automation roadmap - Edge / Firefox variants -CWS submission is one of the three remaining user-driven items before v1.0.0 GA. The submitter (Dankest, LLC) hosts the privacy policy at a public URL: `packages/extension/PRIVACY_POLICY.md` is authored to be hosted as-is on either GitHub Pages or `https://dankest.llc/xchain-wallet/privacy`. +CWS submission is one of the three remaining user-driven items before v1.0.0 GA. The submitter (Dankest, LLC) hosts the privacy policy at a public URL: the [privacy policy](privacy/privacy-policy.md) is authored to be hosted as-is on either GitHub Pages or `https://dankest.llc/xchain-wallet/privacy`. ## Release artifact list diff --git a/components/wallet/features.md b/components/wallet/features.md index 43d5a02c..26cfb222 100644 --- a/components/wallet/features.md +++ b/components/wallet/features.md @@ -124,6 +124,7 @@ See [Multisig](multisig.md) for the full state machine. The wallet supports: ## Cross-chain flows - **Cross-chain swap**: `CrossChainSwapForm.jsx`. SWAP action across chains (BTC ↔ LTC ↔ DOGE) coordinated by `xchain-hub`. +- **Cross-chain order**: `CrossChainOrderForm.jsx`. ORDER with `GIVE_COIN` and `GET_COIN` on different chains, escrowed on the give chain, matched by the validator federation with partial fills and settled through CROSS_SETTLE; token-only on both sides, with the receive address resolved on the get chain. - **Cross-chain templates**: `CrossChainTemplates.jsx`. Pre-built parallel-composer presets like "issue token on BTC + seed dispenser on LTC atomically". - **Parallel composer**: `ParallelComposer.jsx`. Custom multi-chain action sequence with per-chain SDK instances and atomic-or-rollback semantics where the protocol allows. - **Per-chain SDK registry**: `core/src/sdk/SDKRegistry.js`. The wallet keeps a registered SDK instance per chain so cross-chain flows can call into multiple chains in one user-confirmed step. diff --git a/components/wallet/keys-signing.md b/components/wallet/keys-signing.md index 25ba251b..783dfc0a 100644 --- a/components/wallet/keys-signing.md +++ b/components/wallet/keys-signing.md @@ -7,7 +7,7 @@ This document covers everything between the user's password and a broadcast tran ## Master key derivation -The user's password is never persisted. On unlock it's stretched through **Argon2id** to a 32-byte master key: +The user's password is not persisted, with one opt-in exception covered under [Biometric unlock](#biometric-unlock-password-wrap) below. On unlock it's stretched through **Argon2id** to a 32-byte master key: | Parameter | Value | |---|---| @@ -23,11 +23,25 @@ Calibration: on first wallet creation the wallet runs Argon2id with the floor pa After derivation the master key: - Decrypts the vault blob via AES-256-GCM -- Is cached for the session (`chrome.storage.session` on the extension, OS keychain on desktop if enabled, in-memory only on web) +- Is cached for the session (`chrome.storage.session` on the extension, OS keychain on desktop if enabled, in-memory only on web and mobile) - Is zeroed when the wallet locks The password itself is zeroed immediately after derivation. +## Biometric unlock (password wrap) + +Biometric unlock is opt-in, off by default, and can only be enabled from inside an already-unlocked wallet. Enabling it persists an encrypted copy of the **password**, and the choice of the password over the master key is deliberate: each wallet record's seed is encrypted under the password (`SignerPool.populate` → `unlockWalletRecord`), so a provider that cached only the master key would open the vault document and show balances, then fail at the first signature. The password stays the KDF root; biometrics only shorten the path back to it. + +Core owns the seam (`core/src/flows/biometricUnlock.js`) and a shell hands its provider in at boot, so the shared unlock UI works unchanged everywhere. Three providers ship today: + +| Shell | Mechanism | Where the wrap lives | Binding | +|---|---|---|---| +| Browser-based (web, extension, desktop) | WebAuthn with the PRF extension, deriving a 32-byte AES-GCM key from the platform authenticator | `localStorage`, beside the credential ID | PRF salt randomized per registration; registration refuses anything without `userVerification: 'required'`; unsupported PRF hides the affordance rather than downgrading it | +| Android | `BiometricPrompt` over an `AndroidKeyStore` AES-256-GCM key | `biometric.wrap`, an app-private sidecar file outside the vault blob | Authentication required per use, Class-3 biometrics only, invalidated by new biometric enrollment | +| iOS | Face ID / Touch ID over a Keychain item | iOS Keychain | Biometry-current-set access control, `kSecAttrAccessibleWhenPasscodeSetThisDeviceOnly`, non-synchronizable, so it cannot reach iCloud Keychain or another device | + +The wrap sits outside the vault on every shell because it has to be readable before the vault opens. Disabling biometric unlock wipes the credential reference and the ciphertext, and losing the wrap (a new fingerprint enrolled, a cleared credential) simply returns the user to typing the password. + ## Vault encryption The vault is a single AES-256-GCM ciphertext. On unlock the wallet: diff --git a/components/wallet/security.md b/components/wallet/security.md index 8c95dd70..4de266a6 100644 --- a/components/wallet/security.md +++ b/components/wallet/security.md @@ -3,7 +3,7 @@ # Security & Threat Model -This document names what the wallet defends against, what it deliberately doesn't, and which mitigations ship today. It tracks the in-repo `docs/Threat_Model.md` and the §12 + §14 sections of the wallet specification. +This document names what the wallet defends against, what it deliberately doesn't, and which mitigations ship today. It tracks the [threat model](threat-model.md) and the §12 + §14 sections of the wallet specification. ## Protected assets @@ -11,9 +11,9 @@ This document names what the wallet defends against, what it deliberately doesn' |---|---|---| | BIP39 seed phrase | Only in the user's possession (paper / hardware). Encrypted at rest in the vault. | Full loss of funds across every chain the wallet derives | | Wallet master key | Derived from password via Argon2id; held in memory while unlocked | Decryption of the persisted vault → access to all private keys | -| Vault blob | `chrome.storage.local` (extension), IndexedDB (web), or encrypted file (desktop). AES-256-GCM with the master key | Offline access to the ciphertext. Still requires the password to decrypt | -| Session master key | `chrome.storage.session` (extension), OS keychain (desktop, optional), or in-memory (web). Cleared on browser close / tab close | Skips the Argon2id cost of re-unlocking. Not the raw password | -| User password | Never persisted. In memory only during unlock / sign, zeroed after use | Full access to every locked vault on the device | +| Vault blob | `chrome.storage.local` (extension), IndexedDB (web), encrypted file (desktop), or an app-private file under an OS-keystore key via the native vault plugin (mobile). AES-256-GCM with the master key | Offline access to the ciphertext. Still requires the password to decrypt | +| Session master key | `chrome.storage.session` (extension), OS keychain (desktop, optional), or in-memory (web and mobile). Cleared on browser close / tab close | Skips the Argon2id cost of re-unlocking. Not the raw password | +| User password | In memory only during unlock / sign, zeroed after use. Persisted in one case and one only: with biometric unlock enabled, an encrypted copy is kept, released only after a successful biometric check (see **Biometric password wrap** below) | Full access to every locked vault on the device | | Connected-site grants | Persisted in the vault's `connectedSites` collection | Silent approval of dApp requests the user previously granted | ## In scope @@ -29,6 +29,7 @@ This document names what the wallet defends against, what it deliberately doesn' - **Offline attacker with the encrypted blob.** Wallet is password-locked with Argon2id (calibrated to ≥ 750 ms per derivation on the device, floor 64 MiB × 3 iterations × 1 parallelism). Without the password, the blob is AES-256-GCM-protected. - **Tampering with the ciphertext.** AES-GCM tag mismatch surfaces as an unlock failure. An attacker who modifies the blob cannot produce a valid plaintext that opens. - **Key recovery from `chrome.storage.session`.** Session key is the derived master key (32 bytes), not the password. On browser close, the session namespace is cleared by Chrome. Attackers with runtime access to the session have already won; the line isn't held there. +- **Biometric password wrap.** Biometric unlock is opt-in and off until the user turns it on after a normal password unlock; with it off, no copy of the password is persisted anywhere. Turning it on persists exactly one encrypted copy of the *password* (never the master key: each wallet record's seed is encrypted under the password, so a master-key-only cache would open the vault and then fail at the first signature). What protects that copy depends on the shell. Browser-based shells use the default WebAuthn provider with the PRF extension: a 32-byte AES-GCM key is derived from the platform authenticator, the ciphertext sits in `localStorage` beside the credential ID, the PRF salt is randomized per registration, and registration refuses any credential without `userVerification: 'required'`; where PRF is unavailable the affordance is hidden rather than downgraded. Android wraps the copy under an `AndroidKeyStore` AES-256-GCM key that requires authentication per use against a Class-3 `BiometricPrompt` and is invalidated by new biometric enrollment, with the ciphertext in a `biometric.wrap` sidecar file rather than inside the vault blob, since it has to be readable before the vault opens. iOS keeps it in the Keychain behind a biometry-current-set access control, `kSecAttrAccessibleWhenPasscodeSetThisDeviceOnly` and non-synchronizable, so it can neither reach iCloud Keychain nor leave the device. Losing the wrap costs the user nothing beyond typing the password again, and disabling biometric unlock wipes both the credential reference and the ciphertext. ### Network threats @@ -61,7 +62,7 @@ The wallet is the first consumer of the platform's light client (`sdk.light`). I - **Zero-day browser sandbox escapes.** If the browser is compromised, so is the wallet. Users with extreme threat models should use air-gapped PSBT-QR flows (see [URI Schemes](uri-schemes.md)) or hardware wallets. - **Vendor firmware bugs.** Hardware-signer firmware (Trezor, Ledger) is out-of-scope for the wallet's audit; the wallet trusts the vendor's signed firmware. -- **Supply-chain attacks on vendored deps.** Mitigated by `pnpm audit --prod --audit-level=high` in CI + the per-dep review in `docs/DEPENDENCIES.md`. Reproducible builds (Level-2, see [Reproducible Builds](reproducible-builds.md)) narrow the blast radius; a verifier can prove the published artifact came from public source. +- **Supply-chain attacks on vendored deps.** Mitigated by `pnpm audit --prod --audit-level=high` in CI + the per-dep review in [Dependencies](dependencies.md). Reproducible builds (Level-2, see [Reproducible Builds](reproducible-builds.md)) narrow the blast radius; a verifier can prove the published artifact came from public source. - **Physical access to an unlocked device.** No wallet can defend against this. Mitigations: foreground auto-lock (configurable in Settings) and manual lock action. - **Raw-password attacks.** Outside the wallet's design; Argon2id raises the cost, the user's password choice does the rest. - **Blockchain consensus.** The wallet trusts the platform's encoding / decoding logic and the underlying coin nodes' chain selection. For displayed balances and actions it no longer trusts a single indexer: those are SPV-verified against checkpoints signed by a stake-weighted quorum of the federation (see [Proof verification](#proof-verification-spv)), so trust there bottoms out at the federation quorum rather than one server. The platform's correctness is audited separately. diff --git a/components/wallet/shell-extension.md b/components/wallet/shell-extension.md index 6bd45c62..97efcf39 100644 --- a/components/wallet/shell-extension.md +++ b/components/wallet/shell-extension.md @@ -159,7 +159,7 @@ Both are subject to Chrome's ~10 MB per-extension quota. The wallet's typical st ## Privacy policy + CWS -`packages/extension/PRIVACY_POLICY.md` is the public-facing policy hosted alongside the CWS listing. It covers: +The [privacy policy](privacy/privacy-policy.md) is the public-facing policy hosted alongside the CWS listing. It covers: - What's stored on-device (encrypted vault, addresses, contacts, dApp grants, queued PSBTs) - What leaves the device (user-configured RPC endpoints; optional vendor hardware-bridge calls) diff --git a/concepts/README.md b/concepts/README.md index 22995611..86f0cf48 100644 --- a/concepts/README.md +++ b/concepts/README.md @@ -14,6 +14,7 @@ This section explains the fundamental ideas behind the XChain Protocol. Each doc | [Encoding](./encoding.md) | How ACTION data is embedded in blockchain transactions via AES-128-CTR obfuscation | | [Cross-Chain](./cross-chain.md) | How XChain coordinates token swaps across the chains it runs on | | [Gas](./gas.md) | The XCHAIN fee token: what it is, how it works, and why it exists | +| [Token Bridge](./token-bridge.md) | Giving any issuer's token, including XCHAIN itself, a provably-backed shadow balance on the chains it wasn't issued on | | [Security Model](./security-model.md) | Threat model, trust assumptions, and protocol-level security guarantees | | [Smart Contracts](./smart-contracts.md) | Programmable contract layer: sandboxed JavaScript VM with gas metering that orchestrates existing ACTIONs | | [Block Hashes](./block-hashes.md) | Per-block cryptographic hashes (ledger, actions, contracts) for state verification and integrity checking | diff --git a/concepts/actions.md b/concepts/actions.md index 5c74be18..67b77ebe 100644 --- a/concepts/actions.md +++ b/concepts/actions.md @@ -55,7 +55,7 @@ Invalid ACTIONs are recorded as failed; they are not silently ignored. This make ## The ACTION Set -The platform defines 37 named ACTIONs; 36 ACTION types are decoded from the wire. Of those 36, 31 are user-submittable (available via the SDK) across ten categories; the remaining 5 (ANCHOR, ATTEST, NODEPROOF, ROLLCALL, SLASH) are validator-broadcast or system-synthesized and are not SDK-invocable. XCALL is a related but separate case: it is mirror-injected into the destination chain's index rather than decoded from a wire transaction, so it is not counted among the 36 wire-decoded ACTION types, though it is documented below alongside the validator/system actions since it is also not user-submittable. Every ACTION is gated by the **indexer's protocol version**, not by a block height: 21 are registered at version `0.1.0` and 16 at `0.2.0`, and all 37 carry an activation block and timestamp of `0` on every network. ROLLCALL is the one action that additionally carries a per-network height gate, and that gate lives in `rollcall_activation.js` rather than in the protocol-version registry. An indexer processes an action once its own version is at least the registered one. Block-height and timestamp flag-days do exist, but they gate *changes in behaviour* to already-live actions (fee rules, validation tightening, hash-preimage ordering), not the arrival of the actions themselves. See [Protocol Activation](../protocol/protocol-activation.md). +The platform defines 38 named ACTIONs; 36 ACTION types are decoded from the wire. Of those 36, 31 are user-submittable (available via the SDK) across ten categories; the remaining 5 (ANCHOR, ATTEST, NODEPROOF, ROLLCALL, SLASH) are validator-broadcast or system-synthesized and are not SDK-invocable. XCALL is a related but separate case: it is mirror-injected into the destination chain's index rather than decoded from a wire transaction, so it is not counted among the 36 wire-decoded ACTION types, though it is documented below alongside the validator/system actions since it is also not user-submittable. XBRIDGE is the newest addition and sits outside that 36/31 split the same way: it decodes from the wire for its user-broadcast versions but its settle versions are system-injected, so it is counted separately (see the Cross-Chain section below). Every ACTION is gated by the **indexer's protocol version**, not by a block height: 21 are registered at version `0.1.0` and 17 at `0.2.0`, and all 38 carry an activation block and timestamp of `0` on every network. ROLLCALL and XBRIDGE additionally carry a per-network height gate on top of the version gate above; those gates live in `rollcall_activation.js`, `xchain_bridge_activation.js` and `token_bridge_activation.js` rather than in the protocol-version registry. An indexer processes an action once its own version is at least the registered one. Block-height and timestamp flag-days do exist, but they gate *changes in behaviour* to already-live actions (fee rules, validation tightening, hash-preimage ordering), not the arrival of the actions themselves. See [Protocol Activation](../protocol/protocol-activation.md). ### Token Lifecycle @@ -85,6 +85,12 @@ The platform defines 37 named ACTIONs; 36 ACTION types are decoded from the wire | `DISPENSER` | Create a vending machine that sells tokens at a fixed rate in exchange for coin payments. Closes when it is depleted, cancelled, or reaches its `EXPIRATION` (90 days by default), returning any tokens still escrowed to the creating address. | | `SWAP` | Initiate or fulfill a cross-chain atomic token swap, coordinated by the hub. | +### Cross-Chain Bridge + +| ACTION | What it does | +|---|---| +| `XBRIDGE` | Move a token between chains by lock-and-mint / burn-and-release against a protocol-owned escrow. v0/v1 lock and burn XCHAIN; v3/v4 generalize the same lifecycle to any bridgeable token; v2/v5 are system-injected settle legs, never user-broadcast. Gated by `XCHAIN_BRIDGE_ACTIVATION` (v0-v2) and `TOKEN_BRIDGE_ACTIVATION` (v3-v5). See [Cross-Chain Bridge](../protocol/xchain-bridge.md) and [Token Bridge](../protocol/token-bridge.md). | + ### Data | ACTION | What it does | diff --git a/concepts/block-hashes.md b/concepts/block-hashes.md index 16bd6474..030b2aa2 100644 --- a/concepts/block-hashes.md +++ b/concepts/block-hashes.md @@ -73,7 +73,7 @@ data = { hash = SHA256(JSON.stringify(data)) ``` -The `hash_version` field is mandatory in every preimage. An implementor that omits it will compute a different SHA-256 digest and will never match a canonical indexer, even if all other fields are identical. The current value of `BLOCK_HASH_VERSION` is `1` (defined in `xchain-indexer/src/db.js` and mirrored in `xchain-sync/src/BlockHasher.js`). +The `hash_version` field is mandatory in every preimage. An implementor that omits it will compute a different SHA-256 digest and will never match a canonical indexer, even if all other fields are identical. The current value of `BLOCK_HASH_VERSION` is `1` (defined in `xchain-indexer/src/db/shared.js` and mirrored in `xchain-sync/src/client/block_hasher.js`). ### Hash Chaining diff --git a/concepts/gas.md b/concepts/gas.md index ecc510e7..cb3ab044 100644 --- a/concepts/gas.md +++ b/concepts/gas.md @@ -3,7 +3,7 @@ # Gas and Fees -XChain uses a unified gas-based fee system. All protocol fees are expressed in **gas units**, converted to XCHAIN via a single GAS_PRICE parameter, and paid in either native coin (BTC/LTC/DOGE) via oracle price conversion or XCHAIN balance deduction (BTC only). +XChain uses a unified gas-based fee system. All protocol fees are expressed in **gas units**, converted to XCHAIN via a single GAS_PRICE parameter, and paid in either native coin (BTC/LTC/DOGE) via oracle price conversion or XCHAIN balance deduction (BTC only for now; the [token bridge](./token-bridge.md) gives XCHAIN a shadow balance on LTC and DOGE, but fee payment by debiting it is a later milestone, not live yet). ## Fee Conversion Paths @@ -13,7 +13,7 @@ flowchart LR GC["gas cost"] --> GP["GAS_PRICE"] --> XA["XCHAIN amount"] --> XU["XCHAIN/USD oracle"] --> USD["USD"] --> UC["USD/coin oracle"] --> NC["native coin"] ``` -**XCHAIN balance payment (BTC only):** +**XCHAIN balance payment (BTC only for now):** ```mermaid flowchart LR GC2["gas cost"] --> GP2["GAS_PRICE"] --> XA2["XCHAIN amount"] --> DB["debit from user's
XCHAIN balance"] @@ -120,13 +120,15 @@ Fees can be paid two ways. **Native-coin fees** (BTC/LTC/DOGE) are collected at | Bucket | `FEE_PREFERENCE` | Purpose | |---|---|---| +| Default disposition | `0` | Same effect as omitting the field: treated as protocol development | | Destroy (burn) | `1` | Permanently removes XCHAIN from supply (deflationary) | | Protocol Development | `2` (default) | Funds ongoing platform development | -| Community Development | `3` | Community grants, ecosystem growth | + +The accepted set is `{0, 1, 2}`, following the indexer's `validValues` for the field. A community-development bucket (`3`) has been discussed but is not accepted by consensus, so an ADDRESS action carrying a `3` indexes invalid; see [ADDRESS](../protocol/actions/address.md). ## The XCHAIN Token -XCHAIN is a standard XChain token issued via ISSUE on the **BTC chain only**. It does not exist natively on LTC or DOGE. The XCHAIN ticker is reserved on all chains to prevent unauthorized issuance. +XCHAIN is a standard XChain token issued via ISSUE on the **BTC chain only**; it is never natively issued on LTC or DOGE, and the ticker is reserved (case-folded) on all chains to prevent unauthorized issuance. The [token bridge](./token-bridge.md) gives XCHAIN a real, provably-backed **shadow balance** on LTC and DOGE (lock on BTC into a protocol-owned escrow, mint a matching amount on the other chain), so XCHAIN can be held and spent there today, but supply, the mint cap and the price reference all still live on BTC alone; a foreign chain's supply is a shadow of the BTC escrow, never a second, independent mint, and XCHAIN-balance fee payment on LTC/DOGE (as opposed to native-coin fee payment, which works everywhere) is a later milestone. See [Cross-Chain Bridge](../protocol/xchain-bridge.md) for the full mechanism. **Fixed supply, zero pre-mint.** XCHAIN has a permanent `MAX_SUPPLY` cap of 100,000,000 (8 decimals) set at genesis, but supply starts at zero: the genesis `ISSUE` carries no `MINT_SUPPLY`. Every unit is minted, either as a pinned genesis distribution credit or by a public mint, up to the cap; once the open mint is exhausted, no further XCHAIN can ever be created, by anyone, including the issuing address. Supply only ever decreases after that, via the burn bucket above. The genesis allocation itself is in the [white paper](../whitepaper.md#133-genesis-and-fair-launch). diff --git a/concepts/metalayer.md b/concepts/metalayer.md index 977800a8..259330e8 100644 --- a/concepts/metalayer.md +++ b/concepts/metalayer.md @@ -23,7 +23,7 @@ The two layers are completely independent. XChain transactions are valid Bitcoin ## Comparison to Other Approaches -**Sidechains** create a separate blockchain with its own consensus, linked to the main chain via a bridge. Funds must be locked on the main chain to mint equivalent assets on the sidechain. Security depends on the bridge; a critical failure point. XChain has no bridge and no separate chain. +**Sidechains** create a separate blockchain with its own consensus, linked to the main chain via a bridge. Funds must be locked on the main chain to mint equivalent assets on the sidechain. Security depends on the bridge; a critical failure point. XChain has no separate chain, so its metalayer needs no bridge to function; the one bridge it does run, XBRIDGE, moves only the platform's own XCHAIN token between chains and never creates a second chain to secure. **Layer 2 rollups** batch off-chain transactions and periodically commit state roots to the main chain. State exists primarily off-chain; the main chain only sees summary commitments. XChain does the opposite; every action is on-chain, and the main chain is the only source of truth. diff --git a/concepts/scope-and-non-goals.md b/concepts/scope-and-non-goals.md index e55eafe7..9dcb2798 100644 --- a/concepts/scope-and-non-goals.md +++ b/concepts/scope-and-non-goals.md @@ -161,7 +161,7 @@ availability depends on where it is hosted; it is not part of the verifiable on- | XChain is a strong fit for… | Look elsewhere if you need… | |---|---| -| Tokens secured directly by Bitcoin-family proof-of-work, no bridge | Confidential balances or private transfers | +| Tokens secured directly by Bitcoin-family proof-of-work, no third-party bridge | Confidential balances or private transfers | | A transparent, auditable, fully-replayable asset ledger | Sub-second / real-time settlement | | A native DEX and trustless cross-chain swaps among BTC/LTC/DOGE | Deep synchronous DeFi composability (EVM-style) | | Contracts that react to real-world data or AI judgments | Interop with Ethereum/Solana assets and liquidity | diff --git a/concepts/security-model.md b/concepts/security-model.md index e7958bd0..516dcba7 100644 --- a/concepts/security-model.md +++ b/concepts/security-model.md @@ -13,7 +13,7 @@ XChain's security properties come from several sources: the underlying blockchai **Sanity checks after every block**: After processing each block, the indexer verifies that total supply equals net ledger credits minus debits for every active token. A mismatch is treated as a fatal invariant violation, processing halts and the block is rolled back. No inconsistent state is ever persisted. -**Reorg handling**: When the decoder detects a chain reorganization (the canonical chain tip has changed), it records the fork point and signals the indexer. The indexer rolls back all affected data across every relevant table atomically, recalculates state from the fork block, and re-indexes forward. The UTXO tracker maintains a per-chain undo window (BTC: 12 / LTC: 48 / DOGE: 120 blocks) for the same purpose. +**Reorg handling**: When the decoder detects a chain reorganization (the canonical chain tip has changed), it records the fork point and signals the indexer. The indexer rolls back all affected data across every relevant table atomically, recalculates state from the fork block, and re-indexes forward. The UTXO tracker maintains a per-chain, per-network undo window (mainnet and regtest: BTC 12 / LTC 120 / DOGE 120 blocks; testnet: 120 blocks for every coin) for the same purpose. ## Protocol Safety diff --git a/concepts/smart-contracts.md b/concepts/smart-contracts.md index f584f4e4..2b1b9a70 100644 --- a/concepts/smart-contracts.md +++ b/concepts/smart-contracts.md @@ -152,7 +152,7 @@ Every figure in a **Gas** column below is in gas units, charged against the call | Method | Returns | |---|---| | `xchain.getBalance(address, tick)` | Balance of address for token, or null | -| `xchain.getTokenInfo(tick)` | Token metadata, or null | +| `xchain.getTokenInfo(tick)` | Token metadata, or null. Keys are UPPERCASE (`TICK`, `TICK_ID`, `DECIMALS`, `SUPPLY`, `OWNER`, `MAX_SUPPLY`, the `LOCK_*` flags), and `DECIMALS` and `TICK_ID` are integers rather than strings. A lowercase read such as `info.decimals` is `undefined`, which quantising helpers answer with the integer part instead of an error, so spell the keys exactly | | `xchain.getPollResult(pollIndex)` | Frozen tally of a resolved [VOTE](../protocol/actions/vote.md) poll (`status` is `finalized` or `failed_quorum`), or null. A poll is readable only from the block after the one it resolved in, so it never reads as decided inside its own finalization block | ### Contract State (metered) diff --git a/concepts/token-bridge.md b/concepts/token-bridge.md new file mode 100644 index 00000000..d8b2661a --- /dev/null +++ b/concepts/token-bridge.md @@ -0,0 +1,93 @@ + + + +# Token Bridge + +XChain runs on three chains, but a token is only ever issued on one of them. The token bridge +lets an issuer give their token a real, provably-backed presence on the others without splitting +its supply into three independent tokens that can drift apart in price. It is the same mechanism +that gives [XCHAIN](./gas.md#the-xchain-token) itself a balance on chains it was never issued +on; see [Token Bridge](../protocol/token-bridge.md) for the protocol-level design and +[`XBRIDGE`](../protocol/actions/xbridge.md) for the wire action. + +## The idea in one paragraph + +Lock a token on the chain it was issued on ("the origin"), into a protocol-owned escrow nobody +holds a key for. A matching copy appears on the destination chain, mintable and spendable there. +Burn the copy, and the same amount unlocks back on the origin. One unit is always either sitting +in the escrow or circulating as exactly one copy somewhere: the supply never doubles, and anyone +can move value back to the origin at any time by burning. + +## What the bridged copy is called + +A token issued as `PEPECASH` on Bitcoin appears on Dogecoin and Litecoin as `BTC.PEPECASH`, +never as a bare `PEPECASH` a squatter could have already registered there. Wallets and explorers +show the bare name with a small origin badge, so a holder sees "PEPECASH, bridged from BTC" and +never has to think about the underlying naming. The prefix is the whole point: it is what lets +anyone verify which chain actually backs a token, with no registry and no possibility of a name +collision blocking an issuer's expansion. + +## Origin chain, one escrow + +Whichever chain a token is issued on is its origin for the token bridge's whole life. The +escrow for that token lives there, in the same keyless role address the platform already uses +for XCHAIN's own bridge. A bridged copy on another chain can never itself be re-bridged onward: +moving value from one non-origin chain to another always passes back through the origin first. +That keeps the accounting simple for every party involved: one escrow, one shadow supply, per +destination. + +## Turning it on: the issuer's opt-in + +Bridging is off by default. An issuer opts a token in by naming which destination chains it may +bridge to, and, optionally, a required confirmation depth beyond the platform's usual default, +a natural knob for an issuer who wants extra assurance against a reorg before a lock is treated +as final. That choice can be locked permanently, which is the issuer's way of promising holders +the settings can never be pulled out from under them, even by a future owner of the token. + +## What holders should know before bridging value + +- **A bridged mint is final.** If something ever goes wrong upstream of the lock (a chain + reorg deep enough to undo it, for instance), the platform has no way to undo a mint that has + already happened on the destination. The confirmation depth an issuer sets is exactly the + price that makes reaching that situation expensive; a higher depth (longer wait, more + security) is always an issuer's option, never a holder's. +- **An issuer's allow list, block list and pause now travel with a bridged token.** Once the + network has turned this on, a token that has ever put itself under an allow list or a block + list can bridge, and a policy the issuer sets on the origin afterward reaches every bridged + copy automatically, enforced identically everywhere, never re-set per chain. The two are still + not instant: an origin-side change reaches a copy after that chain's own confirmation depth + plus a short signing-and-mirror margin (typically a few minutes), so a copy briefly enforces + the *previous* policy right after the issuer changes it, never a *later* one before the issuer + changes it. A list also has a size ceiling (10,000 addresses) an issuer sees at opt-in time, + since every snapshot and every chain's copy carries the full list. A smart-contract-controlled + policy is the one exception that still does not travel: the contract only exists on the chain + it was deployed on, so a controller-bound token still cannot bridge, and a bridged token + still cannot bind one. +- **Sleep is chain-local.** Pausing a token on its origin chain stops new locks from that + chain, but any copies already bridged elsewhere keep trading, and burning them back to the + origin still works. A holder relying on a token being "fully paused" everywhere should know + that, today, it is only paused where it was issued. +- **Subassets aren't bridgeable yet.** A token issued as a child of another token (a name with a + dot in it, like `PARENT.CHILD`) cannot be bridged in this milestone; only plain, undotted + tokens can. + +## Why not just issue the same ticker on every chain? + +Because nothing then proves the three tickers are the same asset, or that whoever issued the +second and third copies is the same person who issued the first. A holder would have to trust +an out-of-band claim instead of reading it off the chain. The bridge instead makes the +relationship provable: every bridged unit traces back, in the protocol's own ledger, to a real +unit locked in a real escrow on the chain it actually came from. + +--- + +**Copyright © 2025–2026 Dankest, LLC** + +**Based on XChain Platform by Dankest, LLC – https://dankest.llc** + +Licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0-or-later) +with a commercial license available for proprietary use. + +You may use, modify, and distribute this material under the terms of the License. +See [LICENSE](../LICENSE.md) and [NOTICE](../NOTICE.md) for full terms. +See the [licensing overview](https://docs.xchain.io/legal/LICENSING.html). diff --git a/developer-guide/adding-a-blockchain.md b/developer-guide/adding-a-blockchain.md index 882fbd58..9c7c80a5 100644 --- a/developer-guide/adding-a-blockchain.md +++ b/developer-guide/adding-a-blockchain.md @@ -65,7 +65,7 @@ xchain-utxo-tracker xchain-sdk xchain-sync xchain-node ``` Each consumer keeps a thin adapter (for example -`xchain-indexer/src/configs/BTC.js`, `xchain-*/src/CryptoNetworks.js`) that reads +`xchain-indexer/src/coins/BTC.js`, `xchain-decoder/src/chain/crypto_networks.js`) that reads the vendored canonical file and returns that service's existing shape, so nothing downstream of the adapter changes when you add a coin. A CI drift guard (`sync-coins.sh --check`, wired into `bin/ci-all.sh`) fails the build if any @@ -201,7 +201,7 @@ chain on regtest first and pin mainnet at its launch. ### 4. Freeze the hash in the unit test -Add `FOO` to `GOLDEN_HASH` in `xchain-hub/test/unit/coins.test.js`. This freeze +Add `FOO` to `GOLDEN_HASH` in `xchain-hub/test/unit/coins/coins.test.js`. This freeze vector means any later accidental change to a consensus value fails the test loudly; updating a value is then a deliberate act (change the value, the golden hash, and the matching pin in one commit). diff --git a/developer-guide/advanced-token-features.md b/developer-guide/advanced-token-features.md index 44ae5c3a..bfe5a2ab 100644 --- a/developer-guide/advanced-token-features.md +++ b/developer-guide/advanced-token-features.md @@ -462,7 +462,8 @@ const selfMessage = sdk.message({ }); // Broadcast each FILE in its own transaction; the self-MESSAGE can ride with the last one. -// (Files are too large to fit many in one BATCH; broadcast one at a time.) +// (A transaction carries one rawData payload, so a BATCH holds at most one FILE +// whatever the file size; batching two would give both the same ciphertext.) ``` ### Transferring a Gated Token diff --git a/developer-guide/smart-contract-development.md b/developer-guide/smart-contract-development.md index a229a183..0d451d86 100644 --- a/developer-guide/smart-contract-development.md +++ b/developer-guide/smart-contract-development.md @@ -268,9 +268,10 @@ The VM performs the following checks before deployment: 5. **Banned DoS literals:** `BigInt` literals (e.g. `10n`) and `RegExp` literals (e.g. `/foo/`) are rejected. Both expose unmetered native computation; a `BigInt` arithmetic loop or a catastrophic regex can exhaust the block watchdog and halt the chain. The `BigInt` global and `RegExp` constructor are also stripped at runtime; use `xchain.math.*` for big-number work. 6. **Banned async check** (consensus-gated): `async` functions, `await` expressions, and `Promise` references are rejected. Under `VM_LINT_HARDENING` this also rejects dynamic `import(...)` (it evaluates to a Promise). Enforced today by the `xchain-lint` CLI, the SDK, and testnet/regtest; on mainnet at/after the [`VM_BANNED_ASYNC` flag day](../protocol/flag-days.md#contract-era-flag-day). 7. **Banned generator check** (consensus-gated, Pkg 3 sandbox): `function*`, generator methods, and any `yield` are rejected. -8. **Banned WebAssembly check** (consensus-gated, Pkg 3 sandbox): any reference to the global `WebAssembly` is rejected. +8. **Banned rest-pattern check** (consensus-gated, its own `REST_PATTERN_METER` gate): a rest pattern is rejected in the four positions the gas-metering transform cannot charge, because it charges a rest destructure by wrapping the source expression and these positions have none. Rewrite each one: a **rest parameter** in a function parameter list (read `arguments.length` or index the parameters instead), a **nested rest** inside a destructuring pattern (destructure in two steps so the rest reads a named binding), a **catch-clause rest** (destructure the caught binding on a following line), and a **rest in a `for-of`/`for-in` loop head** (bind the iteration value and destructure it in the loop body). See the [`REST_PATTERN_METER` flag day](../protocol/flag-days.md). +9. **Banned WebAssembly check** (consensus-gated, Pkg 3 sandbox): any reference to the global `WebAssembly` is rejected. -Checks 7-8 are live today on testnet/regtest (the Pkg 3 sandbox gate is unconditionally active on those networks) even though their mainnet activation is a separate flag-day; a regtest deploy that trips either one is rejected now. The `VM_LINT_HARDENING` widenings under checks 3, 4, and 6 activate at the same instant as `VM_BANNED_ASYNC` and are default-on for the SDK and CLI already. +Checks 7 and 9 are live today on testnet/regtest (the Pkg 3 sandbox gate is unconditionally active on those networks) even though their mainnet activation is a separate flag-day; a regtest deploy that trips either one is rejected now. Check 8 rides its own `REST_PATTERN_METER` gate rather than the Pkg 3 one, and is likewise unconditionally active on testnet/regtest, so the `xchain-lint` CLI, the SDK linter and a regtest deploy all reject an unmeterable rest pattern today. The `VM_LINT_HARDENING` widenings under checks 3, 4, and 6 activate at the same instant as `VM_BANNED_ASYNC` and are default-on for the SDK and CLI already. A non-blocking **float warning** is also generated if decimal number literals are detected in the code. This warning appears in the execution record but does not prevent deployment. diff --git a/getting-started/key-terms.md b/getting-started/key-terms.md index 3e552328..312179d4 100644 --- a/getting-started/key-terms.md +++ b/getting-started/key-terms.md @@ -9,7 +9,7 @@ A reference glossary of XChain terminology, organized by category. ## Protocol -**ACTION**: A command embedded in a blockchain transaction that instructs the XChain indexer to perform an operation, such as issuing a token, sending a balance, or placing an order. All XChain operations are expressed as one of 37 named ACTIONs. +**ACTION**: A command embedded in a blockchain transaction that instructs the XChain indexer to perform an operation, such as issuing a token, sending a balance, or placing an order. All XChain operations are expressed as one of 38 named ACTIONs. **ACTION_INDEX**: A unique sequential integer assigned to every valid XChain ACTION transaction, in the order it was confirmed on-chain. Many actions reference prior actions by their ACTION_INDEX (e.g., an ORDER references the action that created the token being sold). diff --git a/getting-started/quickstart-developer.md b/getting-started/quickstart-developer.md index c8c3f371..463b82eb 100644 --- a/getting-started/quickstart-developer.md +++ b/getting-started/quickstart-developer.md @@ -269,7 +269,7 @@ The SDK covers all 31 user-submittable actions. Thirty of them have convenience - [Full SDK Documentation](../components/sdk/): all methods, configuration options, error types, and examples - [ACTION Concepts](../concepts/actions.md): conceptual overview of the ACTION set and the ACTION format -- [ACTION Protocol Specs](../protocol/actions/): per-action field-level formats and validation rules for all 37 actions +- [ACTION Protocol Specs](../protocol/actions/): per-action field-level formats and validation rules for all 38 actions - [Regtest Development](../developer-guide/regtest-development.md): run a full local stack for free - [Explorer API](../components/explorer/): all 200+ REST and JSON-RPC endpoints diff --git a/getting-started/quickstart-validator.md b/getting-started/quickstart-validator.md index 5779a5b3..8a4dcd14 100644 --- a/getting-started/quickstart-validator.md +++ b/getting-started/quickstart-validator.md @@ -96,6 +96,26 @@ bootstrap restore does the heavy lifting). The second boots the validator with your keys and capabilities wired in. On a fresh machine it asks once for a database root password. +**One more wire, and it matters on testnet today.** ROLLCALL is already +active on testnet: your BTC indexer must prove each epoch's roll-call +signers from a Dogecoin indexer before it can close the epoch, and your hub +separately needs the same read before it publishes. With neither set, your +BTC indexer silently defers every block from the first epoch close onward +and your hub publishes nothing, and a validator stuck like that for two +epochs in a row is evicted. Unless you run your own Dogecoin indexer, add +this to `~/xchain-node/.env`: + +``` +DOGE_INDEXER_API_URL=https://explorer.xchain.io/TDOGE/api/ # mainnet: /DOGE/api/ +DOGE_INDEXER_API_KEY= +``` + +That read is served off the explorer's own replicated indexer database, so a +replica running behind just makes the epoch close wait longer, never judge +the roll call on stale data. See ["Wire the Dogecoin +read"](../operations/run-a-validator.md#step-5-give-the-hub-a-btc-indexer) +for the full detail. + That's it. Peers admit your validator within about 30 seconds of your stake going live. You do not need to tell anyone. diff --git a/getting-started/running-a-validator.md b/getting-started/running-a-validator.md index 59265956..8b623815 100644 --- a/getting-started/running-a-validator.md +++ b/getting-started/running-a-validator.md @@ -31,6 +31,17 @@ Every validator, either tier, needs three things: 2. **A signing identity.** An Ed25519 keypair the hub generates and holds. 3. **A running hub.** The `xchain-hub` service, reachable by its peers. +On a network where ROLLCALL is active (already the case on testnet, and +armed from genesis on mainnet), a fourth thing is required of every +validator's hub, whichever tier it runs: a JSON-RPC read against a +**Dogecoin** indexer, so the hub can tell what roll-call signatures already +landed before it publishes. Without it a hub publishes nothing for its +roll-call rounds and logs nothing about why, and a validator left unable to +sign roll calls for two consecutive epochs is evicted. See ["Wire the +Dogecoin read"](../operations/run-a-validator.md#step-5-give-the-hub-a-btc-indexer) +for how to point at the public explorer's read if you do not run a Dogecoin +indexer of your own. + Note what is *not* on that list: a coin node. The protocol assigns no tiers by decree. Capabilities qualify automatically when your total effective stake clears each capability's floor, and most capabilities never touch a coin node at all. `price` needs a price feed. `attestation` needs a reachable model provider. `cross_chain` verifies source actions against indexer APIs. `oracle_publish` needs a broadcast wallet. Only one capability requires a coin node, and it is the one named after it. diff --git a/getting-started/what-is-xchain.md b/getting-started/what-is-xchain.md index 6cb9ecb2..24b0e138 100644 --- a/getting-started/what-is-xchain.md +++ b/getting-started/what-is-xchain.md @@ -29,7 +29,7 @@ flowchart TD In practice, XChain works by embedding small pieces of data inside ordinary blockchain transactions. Those data packets are invisible to Bitcoin itself, they're just part of a normal transaction. But XChain's software layer reads those packets, interprets them as commands, and maintains its own database of token balances, orders, and state. -**Nothing about Bitcoin, Litecoin, or Dogecoin is changed.** XChain tokens exist on the actual blockchain, secured by the same proof-of-work consensus that secures every other Bitcoin transaction. There are no sidechains and no bridges: your token balances need no separate validators and no new consensus mechanism to trust. (A staked validator federation does provide the optional cross-chain, oracle, and attestation services described later; those services never sit between you and your base-layer token records.) +**Nothing about Bitcoin, Litecoin, or Dogecoin is changed.** XChain tokens exist on the actual blockchain, secured by the same proof-of-work consensus that secures every other Bitcoin transaction. There is no sidechain and no third-party bridge: your token balances need no separate validators and no new consensus mechanism to trust. (A staked validator federation does provide the optional cross-chain, oracle, and attestation services described later, including XBRIDGE, the protocol action that moves the platform's own XCHAIN token between chains; those services never sit between you and your base-layer token records.) --- @@ -121,11 +121,11 @@ XCALL, the platform's cross-chain contract call, works differently: it's emitted There are a lot of blockchain protocols out there. Here's what sets XChain apart. -### No Sidechains, No Bridges +### No Sidechains, No Third-Party Bridge -Many blockchain token systems work by "locking" assets on one chain and "mirroring" them on another; a process that requires bridges. Bridges are one of the most attacked surfaces in all of crypto; billions of dollars have been lost to bridge exploits. +Many blockchain token systems work by "locking" assets on one chain and "mirroring" them on another; a process that requires a bridge. Bridges are one of the most attacked surfaces in all of crypto; billions of dollars have been lost to bridge exploits. -XChain doesn't use bridges. When you hold an XChain token on Bitcoin, that token record literally exists inside a Bitcoin transaction. There's nothing to bridge, nothing to lock and unlock, no separate chain to trust. +XChain has no third-party bridge. Your token record literally exists inside a Bitcoin transaction, with no separate chain to trust. Moving a token between supported chains is a protocol action, XBRIDGE: the balance is locked or burned on its own chain in an ordinary transaction and credited on the destination by the validator federation, with no wrapped asset and no custodian. Everything stays on chains you already trust. ### No New Consensus diff --git a/lib/env-var-doc-coverage.js b/lib/env-var-doc-coverage.js index 15d531a5..e554f27f 100644 --- a/lib/env-var-doc-coverage.js +++ b/lib/env-var-doc-coverage.js @@ -57,7 +57,7 @@ * `COMPUTED_READ_BASELINE` says. That set is RATCHETED by `checkComputedReads`, * so the blind spot can shrink and cannot silently widen. It is not closed: * a key added to an EXISTING map creates no new site (xchain-node/src/services/ - * DatabaseService.js reads three keys through one `process.env[envVar]`), and + * database_service.js reads three keys through one `process.env[envVar]`), and * waiving the sites is barred by the KNOWN_GAPS admission rule below, which * requires the variable's doc row already written. Recovering the names needs * cross-file tracing of the key maps, including prefixes built from constructor @@ -289,6 +289,84 @@ function catFileBatch(repoDir, ref, rels) { const ENV_READ = /process\.env\.([A-Za-z_][A-Za-z0-9_]*)|process\.env\[\s*['"]([A-Za-z_][A-Za-z0-9_]*)['"]\s*\]/g; +/* + * READS THROUGH A CONFIG VIEW. xchain-explorer's src/config.js exports `env`, a + * live read-through Proxy over process.env, and its code reads configuration as + * `env.X`, `this.configInfo.env.X` or `configInfo.env['X']` rather than naming + * process.env. Every such read is an environment read an operator configures, + * but ENV_READ cannot see it, so moving a read behind the view silently dropped + * it from coverage: the variable could lose its doc row and this gate stayed + * green. The view is matched for the components listed here ONLY, because a + * bare `env` elsewhere in the fleet is whatever object that file named `env` + * (a child-process environment, a parsed .env file), not process.env. + * + * The prefix is a bare `env` (not a member of anything else, so `process.env` + * stays ENV_READ's alone and is never counted twice) or `configInfo.env`, with + * or without `this.`. Names are the upper-case environment spelling only, so a + * method on the object (`env.hasOwnProperty`) is not read as a variable, and an + * ASSIGNMENT into an object that happens to be called `env` is not a read. + * + * A module can also reach the view without holding it: straight off the module + * (`require('../config.js').env.X`), or through an accessor the file binds to + * that expression (`const configEnv = () => require('../config.js').env;`, then + * `configEnv().X`). The accessor defers the require to read time, because + * config.js loads modules that load these files back, and it is the form the + * explorer's feature directories use. Its name is whatever the file bound, so + * it is read from the file rather than assumed: `configEnv().X` in a file with + * no such binding is somebody else's function and not counted. + */ +const CONFIG_VIEW_COMPONENTS = new Set(['explorer']); + +const CONFIG_MODULE = String.raw`require\(\s*['"](?:\.{1,2}\/)+config(?:\.js)?['"]\s*\)`; + +const CONFIG_VIEW_ACCESSOR = new RegExp( + String.raw`(?\s*` + + String.raw`${CONFIG_MODULE}\.env(?![A-Za-z0-9_$.\[])`, 'g'); + +/** Names a source binds as zero-argument accessors returning the config view. */ +function configViewAccessors(stripped) { + const names = new Set(); + CONFIG_VIEW_ACCESSOR.lastIndex = 0; + let m; + while ((m = CONFIG_VIEW_ACCESSOR.exec(stripped)) !== null) names.add(m[1]); + return [...names]; +} + +/** + * The named-read and computed-read patterns through the view, for a file that + * binds `accessors`. The `env` alternatives match at `env` itself; an accessor + * call matches from its name, which leaves the read's line and the call-shaped + * default rules unchanged, since both balance the empty `()` they walk over. + */ +function configViewPatterns(accessors = []) { + const alts = [String.raw`(?:(? n.replace(/\$/g, '\\$')).join('|'); + alts.push(String.raw`(?'; +} + // Coercions that take the read as their ONLY argument, so a `|| default` after // them still belongs to the variable. Deliberately numeric-only: broadening this // to any single-identifier call misreads `getConfig(env.X).timeout || 5000`. @@ -536,8 +614,8 @@ function readOperand(text, i) { } // The characters after which a `/` cannot be dividing a finished operand, so -// it opens a regex literal. Kept byte-identical to the twin in -// bin/generate-flag-days.js. +// it opens a regex literal. The one copy: bin/generate-flag-days.js reads +// stripComments from here rather than carrying a twin. const REGEX_POSITION_AFTER = new Set(['(', ',', '=', ':', '[', '!', '&', '|', '?', '{', '}', ';', '+', '-', '*', '%', '<', '>', '~', '^']); const REGEX_POSITION_KEYWORDS = new Set(['return', 'typeof', 'case', 'in', 'of', 'new', 'delete', 'void', 'do', 'else', 'yield', 'await']); @@ -601,8 +679,8 @@ function regexLiteralEnd(source, i) { * * Blanks rather than deletes, so every offset, column and newline survives and * the callers' `idx + 1` line numbers still name the real line. Ported from - * `withoutComments` in xchain-documentation/bin/generate-flag-days.js, which - * solved this for the flag-day generator first. + * the flag-day generator (bin/generate-flag-days.js), which solved this first + * and now imports this copy under its old `withoutComments` name. * * A REGEX LITERAL IS COPIED WHOLE for the same reason a string is: the `//` * inside `str.replace(/\/\//, '-')` starts no comment, and blanking from it @@ -686,21 +764,40 @@ function lineIndex(text) { * name the same place in the original and `lineIndex` still reports the real * line. The line recorded is the line of the READ, not of the default. * + * With `configView`, reads through the config view (see CONFIG_VIEW_READ) are + * collected too, merged in source order so a variable's first site is still + * the first place the file reads it. + * + * @param {string} source + * @param {{configView?: boolean}} [opts] * @returns {Map>} */ -function scanSource(source) { +function scanSource(source, opts = {}) { const found = new Map(); const stripped = stripComments(source); const lineAt = lineIndex(stripped); + const hits = []; ENV_READ.lastIndex = 0; let m; while ((m = ENV_READ.exec(stripped)) !== null) { - const name = m[1] || m[2]; - if (!found.has(name)) found.set(name, []); - found.get(name).push({ - line: lineAt(m.index), - default: extractDefault(stripped, m.index + m[0].length), + hits.push({ name: m[1] || m[2], index: m.index, end: m.index + m[0].length }); + } + if (opts.configView) { + const { read } = configViewPatterns(configViewAccessors(stripped)); + while ((m = read.exec(stripped)) !== null) { + const end = m.index + m[0].length; + if (isAssignmentTarget(stripped, end)) continue; + hits.push({ name: m[1] || m[2], index: m.index, end }); + } + hits.sort((a, b) => a.index - b.index); + } + + for (const h of hits) { + if (!found.has(h.name)) found.set(h.name, []); + found.get(h.name).push({ + line: lineAt(h.index), + default: extractDefault(stripped, h.end), }); } @@ -715,14 +812,33 @@ const COMPUTED_READ = /process\.env\s*\[\s*(?!['"])/g; /** * Every computed env read in a source string, by line. * + * With `configView`, a computed read through the config view counts as well, + * unless it is an assignment target. The view is scanned over the whole text, + * because telling a read from a write needs the closing bracket, which a + * wrapped key puts on a later line. + * + * @param {string} source + * @param {{configView?: boolean}} [opts] * @returns {number[]} */ -function scanComputedReads(source) { - const lines = []; - stripComments(source).split('\n').forEach((code, idx) => { +function scanComputedReads(source, opts = {}) { + const lines = []; + const stripped = stripComments(source); + stripped.split('\n').forEach((code, idx) => { COMPUTED_READ.lastIndex = 0; while (COMPUTED_READ.exec(code) !== null) lines.push(idx + 1); }); + if (opts.configView) { + const lineAt = lineIndex(stripped); + const { computed } = configViewPatterns(configViewAccessors(stripped)); + let m; + while ((m = computed.exec(stripped)) !== null) { + const close = skipToEnclosingClose(stripped, m.index + m[0].length); + if (close !== -1 && isAssignmentTarget(stripped, close)) continue; + lines.push(lineAt(m.index)); + } + lines.sort((a, b) => a - b); + } return lines; } @@ -833,13 +949,8 @@ function assertsDefault(rows, value) { * uses several table shapes, and pinning a column would break on the next one. * Strict about the value, which is the part that goes stale. * - * The boundary guard follows the SHAPE of the value. A number must not be read - * out of a longer number (`3000` inside `30000`), so digits and dots bound it. - * A string default is a word, so it is bounded the way a variable name is in - * `docLinesFor`, which keeps `server` out of `webserver`. - * - * BOOLEANS AND NUMBERS must be ASSERTED, because for them presence alone is - * VACUOUS. A row documenting a switch names both states by + * EVERY value shape must be ASSERTED, because presence alone is VACUOUS. + * A row documenting a switch names both states by * construction: `REQUIRE_SIGNATURES` says "Defaults to `true` ... pass `false` * to bootstrap", so `true` and `false` were equally "documented" and the check * passed whichever one the code held. A number is masked the same way by any @@ -865,16 +976,19 @@ function assertsDefault(rows, value) { * matcher to understand units, which is a different piece of work from telling * an assertion apart from a mention. * - * STRING defaults stay position-free. A word is not masked by a coincidence the - * way a digit is, and the boundary guard below is what keeps `server` out of - * `webserver`. + * STRINGS ARE ASSERTED TOO, under the same one rule. The earlier reading gave + * them a position-free mention, on the ground that a word is not masked by a + * coincidence the way a digit is. The class that actually ships falsifies it: a + * row that ENUMERATES the allowed values names the rival value by construction, + * exactly as a switch row names both states. `SYNC_MODE` is the shipped case, + * `| \`SYNC_MODE\` | Yes | \`server\` | Operating mode: \`server\` or \`client\` |` + * against a code default of `server`, where flipping only the Default cell to + * `client` left the word `server` in the description and the check stayed green. + * One rule for every value shape, so presence alone is vacuous everywhere. */ function defaultDocumented(rows, value) { if (value === '0' && rows.some((r) => MEANS_ZERO.test(r))) return true; - const numeric = /^-?\d+(\.\d+)?$/.test(value); - if (value === 'true' || value === 'false' || numeric) return assertsDefault(rows, value); - const asWritten = new RegExp(`(? asWritten.test(r)); + return assertsDefault(rows, value); } /** @@ -926,13 +1040,14 @@ function buildSurvey({ platformRoot, docRoot, serviceReader, docReader, componen const vars = new Map(); const computed = []; + const scanOpts = { configView: CONFIG_VIEW_COMPONENTS.has(c) }; for (const [rel, source] of files) { - for (const [name, sites] of scanSource(source)) { + for (const [name, sites] of scanSource(source, scanOpts)) { if (NOT_CONFIGURATION(name)) continue; if (!vars.has(name)) vars.set(name, []); for (const s of sites) vars.get(name).push({ ...s, file: rel }); } - for (const line of scanComputedReads(source)) computed.push({ file: rel, line }); + for (const line of scanComputedReads(source, scanOpts)) computed.push({ file: rel, line }); } const mine = docPaths.filter((p) => isDocPath(p, c)); @@ -1106,7 +1221,7 @@ function checkDivergentDefaults(survey) { * ratcheting. * * Honest about what it does not buy: a key added to an EXISTING map creates no - * new site (xchain-node/src/services/DatabaseService.js reads three tuning keys + * new site (xchain-node/src/services/database_service.js reads three tuning keys * through one `process.env[envVar]`), so this holds the line rather than gating * those. Gating them is proposal B and a separate piece of work. * @@ -1117,8 +1232,107 @@ function checkDivergentDefaults(survey) { const COMPUTED_READ_BASELINE = { // Measured 2026-08-11 against the committed trees of all 11 gated // components: 95 sites in 37 files across 10 of them. - decoder: 4, encoder: 4, explorer: 8, hub: 33, indexer: 8, node: 23, - 'regtest-miner': 7, sdk: 4, sync: 10, 'utxo-tracker': 8, vm: 0, + decoder: 4, encoder: 4, explorer: 6, hub: 5, indexer: 7, node: 5, + 'regtest-miner': 7, sdk: 4, sync: 10, 'utxo-tracker': 7, vm: 0, + // hub 6 -> 5, indexer 8 -> 7 and sync 11 -> 10 (activation registry W4, row P1/P3): the + // carrier logic pin's one computed read (bin/lib/carrier_logic_pin.js:119, the + // XCHAIN__DIR key built from the repo name) became three reads by literal name, + // so the pin module scans to zero computed sites in all three repos. Lowered because + // the blind spot shrank: the three variables are now visible to the coverage gate. + // hub 5 -> 6, indexer 6 -> 8 and sync 10 -> 11 on 2026-09-15, re-measured against the + // landed tips (xchain-hub d6a81150, xchain-indexer da12b6e0 and xchain-sync 0264a43): + // the carrier logic pin landed in all three repos (bin/lib/carrier_logic_pin.js), + // one new computed process.env read apiece (line 119 in each). indexer picks up a + // second new site: xchain-indexer f3ef3d16 has bin/consensus-identity.js print and + // pin carrier_logic_digest, reading it from process.env at line 164. Raised because + // the reads are new, not because the blind spot changed shape. + // sync 9 -> 10 on 2026-09-15: the code-structure pass's call-time readEnvNow(key) accessor in src/config.js reads the state-tree metric cap, a new computed site. + // explorer 7 -> 6 and indexer 7 -> 6 at explorer 7b2a28e3 / indexer 29a1b18e: the + // hub-mirror client split (xchain-indexer c3000819, vendored into explorer) + // consolidated 11 process.env reads scattered across hub_db_sync.js's methods into + // one vendored reader, src/hub/hub_db_sync/env.js, whose single readEnvNow(key) + // is the sole process.env[key] site (line 43) every caller now goes through. + // Lowered because the count fell, NOT because the unscannable surface shrank: the + // same keys are read, one hop further away. + // hub 11 -> 5 at hub f3401e34 (52671173 routed the anchor family's six reads through + // config.env(), the same one-hop-further shape as below). What remains: the prune + // tool's one and the four vendored src/coins/index.js sites. + // hub 33 -> 11 on 2026-09-15, re-derived at the hub stack that follows 21cadb00: the + // code-structure pass routed the api boot check, the attestation relay's per-coin + // endpoints, the cross-chain engines' indexer keys, the hub indexer-url and peer + // roster reads, the spend guard, the CLI child env and the roll-call tunables through + // hubConfig.env(), which this scanner does not match. The eleven left are the anchor + // family's six (checkpoint_engine/options.js, publisher/state.js, reorg_handler.js), + // the prune tool's one and the four vendored src/coins/index.js sites. Lowered because + // the count fell, NOT because the unscannable surface shrank: the same computed keys + // are read, one hop further away. + // hub 35 -> 33 on 2026-09-15, re-derived at xchain-hub 21cadb00: the code-structure + // pass routed SpendCeiling's two computed reads through the config home's call-time + // accessor, so src/lib/spend_ceiling.js:60 and :62 now read hubConfig.env()[maxKey] + // and [windowKey] instead of process.env, which this scanner does not match. The + // pass's api.js, cross-chain, anchor, attestation and rollcall splits moved the + // other sites between files without changing the count, which is what counting per + // component rather than per file is for. Lowered because the count fell, NOT because + // the unscannable surface shrank: the same _MAX_PUBLISHES_PER_WINDOW and + // _SPEND_WINDOW_MS keys are read, one hop further away. + // explorer 9 -> 7 on 2026-09-15, re-derived at xchain-explorer 4e73514: the query + // builder split moved the two envPrefix + '_MS' / '_MAX' reads out of + // src/db/query_sql.js into src/db/query_sql/query_limits.js, where they read the + // config view rather than process.env, so the scanner no longer matches them. The + // seven that remain are the four vendored src/coins/index.js sites, the view's own + // passthrough in src/config.js and the two in the vendored src/hub/hub_db_sync.js. + // Lowered because the count fell, NOT because the unscannable surface shrank. + // indexer 10 -> 7 on 2026-09-15, re-derived at xchain-indexer 4759f9eb: the residue + // landing routed the bridge transport's three origin-chain reads through the config + // home (src/consensus/bridge_proof_client/transport.js onto src/config.js). Lowered + // because the count fell, NOT because the surface shrank: the same per-coin keys are + // read, one hop further away. The sdk stays 4: xchain-sdk bin/preflight_indexer_root.js + // reads XCHAIN_ALLOW_NO_INDEXER by its literal name, documented in + // components/sdk/configuration.md. + // node 23 -> 5 on 2026-09-14, re-derived at xchain-node d384ef4: the config-home pass + // (xchain-node 11e4000) declared the computed reads outside the config home as named + // views in src/config/env_views.js, all bound through one process.env[name] at + // src/config/index.js:453. Lowered because the count fell, NOT because the surface + // shrank: the views still build names from a module, coin or network. + // sync 10 -> 9 on 2026-09-14, re-derived at xchain-sync 0e06951: the same pass + // (xchain-sync 0c5f1ab) routed the pinned-validator, pool-sizing and shutdown reads + // through two call-time accessors, src/config.js:155 and :239. Lowered on the same terms. + // indexer 11 -> 10 on 2026-09-14, re-derived at xchain-indexer 427a0b4: the residue fold + // (xchain-indexer 57e49dd0) moved src/api.js's REQUIRED_ENV boot check from + // process.env[key] onto the config home's CONFIG_ENV snapshot, whose every key is a + // named read in src/config.js. The same commit gave SHUTDOWN_TIMEOUT_MS its named + // read there, which is what surfaced its row in components/indexer/configuration.md. + // explorer 7 -> 9 on 2026-09-14, re-derived at xchain-explorer a610975: the scanner + // now matches reads through config.js's `env` view (CONFIG_VIEW_COMPUTED_READ), so + // the two envPrefix + '_MS' / '_MAX' reads in src/db/query_sql.js:90 and :201 are + // counted again. Raised because the blind spot became VISIBLE, not because it grew: + // no new computed read landed. config.js's own process.env[k] (the view's single + // passthrough, src/config.js:138) stays in the count. Still counted nowhere, by + // this or by COMPUTED_READ: a bracket key that OPENS with a quote and then + // concatenates (`env['UTXO_TRACKER_URL_' + code]`, src/db/readers/entities.js:272, + // and the two per-coin tip-age keys in src/db/readers/health.js:112 and :128). + // explorer 8 -> 7 on 2026-09-14: the explorer split db.js into reader modules and + // routed its env reads through config.js's `env` view. The two computed reads of + // envPrefix + '_MS' and envPrefix + '_MAX' left db.js and now read + // this.configInfo.env[...] in src/db/query_sql.js, which this scanner does not + // match, while the view's own process.env[k] (src/config.js) adds one site that + // stands for every read through it. Lowered because the count fell, NOT because + // the unscannable surface shrank: the same keys are read, one hop further away. + // indexer 8 -> 11 on 2026-09-12: the bridge settle pass landed. bridge_proof_client.js + // resolves the origin chain's indexer through process.env[coin + '_INDEXER_API_URL'], + // process.env[coin + '_INDEXER_URL'] and process.env[coin + '_INDEXER_API_KEY'] + // (src/consensus/bridge_proof_client.js:132-134), documented as the _INDEXER_* rows in + // components/indexer/configuration.md. Raised deliberately, for the reason the hub's + // bridge engine was: an escrow proof is only verifiable against the chain the lock + // landed on, so the key name is built from the origin coin and cannot be a literal. + // hub 33 -> 35 on 2026-09-12: the cross-chain bridge engine landed as + // xchain-hub 9d103c8. CrossChainBridgeEngine.js reads process.env[coin + + // '_INDEXER_URL'] and process.env[coin + '_INDEXER_API_KEY'] per origin chain + // (src/CrossChainBridgeEngine.js:158-159), both of which carry rows in + // components/hub/configuration.md. Raised deliberately: the bridge reads one + // indexer per origin chain rather than one global endpoint, because a bridged + // token's source leg is only verifiable against the chain it was locked on, + // so the key name has to be built from the coin and cannot be a literal. // sync 9 -> 10 on 2026-09-11: the persistent replica-gap escalation landed as // xchain-sync a4c23f9. ClientSync._numericSetting() reads process.env[key] // for REPLICA_GAP_ALERT_SWEEPS and REPLICA_GAP_ALERT_REPEAT_MS, both of which @@ -1129,7 +1343,7 @@ const COMPUTED_READ_BASELINE = { // indexer and explorer 7 -> 8 on 2026-09-11: the hub-mirror stream watermark // bound landed as xchain-indexer 74fae9c8 and rode into the explorer with the // twin resync xchain-explorer 6cdcb0f. resolveWatermarkStallMs() in - // src/hub_db_sync.js reads process.env[envKey] for HUB_SYNC_WATERMARK_STALL_S + // src/hub/hub_db_sync.js reads process.env[envKey] for HUB_SYNC_WATERMARK_STALL_S // and HUB_SYNC_WATERMARK_STALL_EXIT_S, both of which carry rows in // components/indexer/configuration.md and components/explorer/configuration.md. // One new site per component, because hub_db_sync.js is a byte twin. Raised @@ -1143,7 +1357,7 @@ const COMPUTED_READ_BASELINE = { // what lets one resolver enforce the regtest-only rule (ignore-with-warning // off regtest, positive-integer check on it) for a consensus input, instead // of repeating that guard at the read site. - // hub 31 -> 33 on 2026-09-03: RollcallRound._resolveTunable() reads + // hub 31 -> 33 on 2026-09-03: RollcallRound.resolveTunable() reads // process.env[name] twice for the three ROLLCALL_*_BLOCKS tunables, and // attest_response_timing.js reads ATTEST_RESPONSE_FORWARD_S_OVERRIDE by its // constant. All four names carry rows in components/hub/configuration.md. @@ -1169,6 +1383,8 @@ const COMPUTED_READ_BASELINE = { // five BULK_SYNC_* knobs, so each is read as process.env[name]. Raised deliberately, // not to clear a gate: all five carry rows in components/utxo-tracker/configuration.md, // and the indirection buys a NaN guard on knobs a bare parseInt degrades silently. + // utxo-tracker 8 -> 7 on 2026-09-12: the numeric operator-override validation + // replaced one computed read with a named one, so the ratchet drops to match. }; function checkComputedReads(survey) { @@ -1202,6 +1418,7 @@ function checkStaleKnownGaps(survey) { module.exports = { COMPONENTS, SOURCE_ROOTS, SKIP_DIRS, KNOWN_GAPS, ENV_READ, COMPUTED_READ, NOT_CONFIGURATION, + CONFIG_VIEW_COMPONENTS, CONFIG_VIEW_READ, CONFIG_VIEW_COMPUTED_READ, configViewAccessors, COMPUTED_READ_BASELINE, isSourcePath, isDocPath, workingTreeReader, committedTreeReader, isGitRepo, diff --git a/lib/indexer-source.js b/lib/indexer-source.js new file mode 100644 index 00000000..0c541956 --- /dev/null +++ b/lib/indexer-source.js @@ -0,0 +1,137 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * Reads one xchain-indexer module's SOURCE TEXT the way the indexer lays it out. + * + * WHY. The indexer splits a long module without moving its require path. The + * entry stays at `.js` and its body moves into parts under a sibling + * directory `/`; a directory module keeps `/index.js` and puts its + * parts beside it. A check that reads the entry alone goes red when the text it + * wants moved into a part, and goes BLIND when it asserts the text is ABSENT, + * because what it must not find now sits in a file it never opens. + * + * WHAT. The entry followed by every `.js` file under its part directory, at any + * depth, in sorted order. Only a directory spelled exactly as the entry's own + * name, in the entry's own directory, counts, so a look-alike or a same-named + * directory elsewhere is never read. On a tree where the module was never split + * that directory does not exist and the read is the entry alone, unchanged. + */ +'use strict'; + +const fs = require('fs'); +const path = require('path'); + +/** Every `.js` file under `dir` at any depth, sorted, without `skip`. */ +function jsFilesUnder(dir, skip) { + const out = []; + for (const e of fs.readdirSync(dir, { withFileTypes: true })) { + const p = path.join(dir, e.name); + if (e.isDirectory()) out.push(...jsFilesUnder(p, skip)); + else if (e.name.endsWith('.js') && p !== skip) out.push(p); + } + return out.sort(); +} + +/** + * The entry as this checkout actually spells it. + * + * A split leaves `.js` in place while the parts move under `/`, but the + * split can go one step further and take the entry into the directory as well, which + * is the shape the sdk pre-flight drift gate requires of a hashed handler (it refuses + * a directory row while a flat `.js` sits beside it). A caller naming the flat + * path then names a file that is not there, so it is followed to `/index.js`. + * The flat path wins whenever it exists, so nothing changes on a tree that still has it. + * + * @param {string} entry absolute path of `.js` or `/index.js` + * @returns {string} the entry that exists, or `entry` unchanged when neither does + */ +function moduleEntry(entry) { + if (fs.existsSync(entry)) return entry; + const inDirectory = path.join(entry.replace(/\.js$/, ''), 'index.js'); + return inDirectory !== entry && fs.existsSync(inDirectory) ? inDirectory : entry; +} + +/** Is the module present at either spelling? The probe a cross-repo check skips on. */ +function moduleExists(entry) { + return fs.existsSync(moduleEntry(entry)); +} + +/** + * The files one module is made of: the entry first, then its parts. + * + * @param {string} entry absolute path of `.js` or `/index.js` + * @returns {string[]} the entry, then every part in sorted order + */ +function modulePaths(entryPath) { + const entry = moduleEntry(entryPath); + const dir = path.basename(entry) === 'index.js' ? path.dirname(entry) : entry.replace(/\.js$/, ''); + if (dir === entry) return [entry]; + // Exact spelling, checked by listing the parent: a case-insensitive + // filesystem would otherwise let `Consensus.js` claim a `consensus/` directory. + const parent = path.dirname(dir); + const named = fs.existsSync(parent) && fs.readdirSync(parent, { withFileTypes: true }) + .some((e) => e.isDirectory() && e.name === path.basename(dir)); + return named ? [entry, ...jsFilesUnder(dir, entry)] : [entry]; +} + +/** + * The module's text, entry then parts, newline-joined. + * + * A module at neither spelling throws naming BOTH, so a claim check that goes red + * when a handler moves says where it looked instead of an ENOENT on the flat path + * alone (which reads as "the file is gone" on a tree where it merely moved). + */ +function readModuleSource(entry) { + if (!moduleExists(entry)) { + throw new Error('indexer module not found at either spelling: ' + + `${entry} or ${path.join(entry.replace(/\.js$/, ''), 'index.js')}`); + } + return modulePaths(entry).map((p) => fs.readFileSync(p, 'utf8')).join('\n'); +} + +/** + * The same text as readModuleSource, plus the map back to the file and line + * an offset in it came from. + * + * A reader that refuses a declaration it cannot parse has to say WHERE, and + * "line 302" of a joined text is no place a person can open. `parts` carries + * each file's own text at its `start` offset in `text` (the join adds one + * newline between files), so a caller that transforms the text file by file + * and re-joins it the same way keeps every offset aligned with `where()`. + * + * @param {string} entry absolute path of `.js` or `/index.js` + * @returns {{text: string, parts: Array<{file: string, start: number, text: string}>, + * where: (index: number) => {file: string, line: number}}} + */ +function locatedModuleSource(entry) { + if (!moduleExists(entry)) { + throw new Error('indexer module not found at either spelling: ' + + `${entry} or ${path.join(entry.replace(/\.js$/, ''), 'index.js')}`); + } + const parts = []; + let start = 0; + for (const file of modulePaths(entry)) { + const text = fs.readFileSync(file, 'utf8'); + parts.push({ file, start, text }); + start += text.length + 1; + } + const where = (index) => { + let part = parts[0]; + for (const p of parts) if (p.start <= index) part = p; + return { file: part.file, line: part.text.slice(0, index - part.start).split('\n').length }; + }; + return { text: parts.map((p) => p.text).join('\n'), parts, where }; +} + +module.exports = { moduleEntry, moduleExists, modulePaths, readModuleSource, locatedModuleSource }; diff --git a/operations/deployment.md b/operations/deployment.md index 50dfdd6a..039f6b4a 100644 --- a/operations/deployment.md +++ b/operations/deployment.md @@ -151,11 +151,23 @@ Three things follow from the formula: - **A host that cannot afford the floor is told.** When half the host divided by the tracker count falls under 1024 MB, each tracker still gets 1024 MB and `install` prints a warning that the host is oversubscribed; run fewer chains there. - **Only the tracker is limited by default.** The decoder, indexer and hub do not size themselves to a cgroup limit, so a limit on them turns a transient spike (a large mempool batch, a deep reorg) into an OOM kill and a restart loop. Cap one explicitly with `XCHAIN_NODE_MODULE_MEMORY_MB_` only after measuring it. +**Confirm the limit landed.** A kernel without the memory cgroup controller does not refuse `--memory`. Docker takes the flag, prints `WARNING: Your kernel does not support memory limit capabilities or the cgroup is not mounted. Limitation discarded.`, exits successfully, and creates the container with no limit at all. Two reads say whether that happened. Ask Docker what the container got: + +```bash +docker inspect -f '{{.HostConfig.Memory}}' xchain-node-dogecoin-mainnet-xchain-utxo-tracker +``` + +A non-zero byte count is the limit in force; `0` means the tracker is running on the whole host. The tracker's own boot line agrees: `docker logs` on it prints `memory budget NNNNMB (cgroup limit)` when a limit binds, and `(host memory)` when none does. `xchain-node` performs this check itself after every create and warns when the limit did not stick, but the two commands above confirm it at any time. + +Raspberry Pi OS ships with the memory cgroup controller off, which is the usual cause. Append `cgroup_enable=memory cgroup_memory=1` to the single line in `/boot/firmware/cmdline.txt`, reboot, then run `xchain-node recreate xchain-utxo-tracker all all` so every tracker is created again with its limit. On a Raspberry Pi 5 the disable comes from the firmware, which injects `cgroup_disable=memory` ahead of whatever `cmdline.txt` holds: the file never carries it, so there is nothing to remove, and after the fix `cat /proc/cmdline` still shows `cgroup_disable=memory` followed by your appended parameters. That is the working state, not a failed one: the kernel takes the last setting on the line, so the appended enable wins. Judge the fix by `docker inspect` returning a non-zero limit and the tracker's boot line reading `(cgroup limit)`, never by grepping the kernel command line for the disable. On any other host, run `docker info` and look for `No memory limit support` among its warnings. + +The limit is what leaves the host its page cache. On a four-chain Pi 5 the same tracker's heap-flush threshold went from 2027 MB, sized against the whole host, to 256 MB under a 2 GB cap, and host memory available went from 3793 MB to 7669 MB with nothing else changed. The section below is why that page cache matters. + To override the derivation for a tracker, either set the container limit (`XCHAIN_NODE_MODULE_MEMORY_MB_XCHAIN_UTXO_TRACKER=4096` before `recreate`; the tracker re-derives its slices from the new limit) or set the slices themselves in the tracker's environment (`LEVELDB_CACHE_BYTES`, `HEAP_FLUSH_THRESHOLD_MB`, `BULK_SYNC_RAM_BUDGET`), documented on the [utxo-tracker configuration](../components/utxo-tracker/configuration.md#memory-budget) page. Setting a slice larger than the container limit allows is the one combination to avoid: the kernel enforces the limit, not the tracker. ### Disk I/O on a multi-chain host -On a host with a single disk, every chain's coin node, decoder, indexer, encoder and UTXO tracker read and write that same disk, and it is often the first resource to saturate, not CPU or RAM. A chain that has already finished its initial block download does not go quiet: its services keep polling the coin node and the database on a fixed cadence, and the UTXO tracker's LevelDB store serves those polls as cold reads, so a synced chain's idle services can still hold the disk at high iowait and starve another chain's initial block download of the throughput it needs. +On a host with a single disk, every chain's coin node, decoder, indexer, encoder and UTXO tracker read and write that same disk, and it is often the first resource to saturate, not CPU or RAM. The mechanism is the page cache, not an idle writer. A synced chain's services are quiet at idle: measured over fourteen minutes on a four-chain host, their block I/O counters did not move and iowait sat at 0.3, with the tracker parsing the mempool every minute. What the tracker does on every mempool pass is read index blocks from its LevelDB store, and on a mainnet Bitcoin tracker that store is around 92,000 table files (176 GiB at the 2 MB default file size) while LevelDB keeps index blocks resident for only its default 1,000 open files. Every other lookup goes to the page cache, and while the page cache holds them the passes cost no disk reads at all. When another process evicts the page cache, which a coin node in initial block download with a large `dbcache` does, every mempool pass re-reads those index blocks off the device at its full bandwidth (41.8 GB in five minutes on one Pi 5), and that is what starves the second chain's initial block download. Measure the effect without stopping anything by sampling each container's block I/O twice, five minutes apart: @@ -163,12 +175,13 @@ Measure the effect without stopping anything by sampling each container's block docker stats --no-stream --format '{{.Name}}\t{{.BlockIO}}' ``` -The container whose BlockIO grew the most between the two samples is the current writer; if a synced chain's services show meaningful growth while idle, they are competing with whatever chain is still in initial block download. +The container whose BlockIO grew the most between the two samples is the current reader or writer. A synced chain's tracker whose BlockIO grows by gigabytes while idle is not busy; it has lost its page cache and is re-reading its index from the disk, which is the signature to look for. -On a single-disk host, prefer one of these in order: +On a single-disk host, prefer these in order: -- **Sync one chain at a time.** Install and let each chain finish its initial block download before installing the next; this avoids the contention entirely. -- **If chains must overlap, stop the synced chain's services for the other chain's initial block download**, then start them again once it catches up: +- **Cap the trackers so the host keeps its page cache.** The memory limit in the section above is what stops a tracker from sizing its own caches against the whole host; with the limit in force the page cache survives and the idle re-reads stop. Confirm the limit landed before going further down this list. +- **Sync one chain at a time.** Install and let each chain finish its initial block download before installing the next, so no initial block download evicts a synced tracker's index while it runs. +- **Only if the cap and the ordering are not enough, stop the synced chain's services for the other chain's initial block download**, then start them again once it catches up: ```bash xchain-node stop all bitcoin mainnet diff --git a/operations/docker.md b/operations/docker.md index 5e71c5a6..67213f63 100644 --- a/operations/docker.md +++ b/operations/docker.md @@ -87,7 +87,7 @@ xchain-node creates networks automatically during install, but this may be neede Configuration is passed to each container as environment variables. These are set by xchain-node during `install` based on: -1. Hardcoded defaults in `xchain-node/src/services/ConfigService.js` +1. Hardcoded defaults in `xchain-node/src/services/config_service.js` 2. Per-chain overrides in `xchain-node/config/-` files Key variables passed to most services: diff --git a/operations/reorg-handling.md b/operations/reorg-handling.md index 87bb70c2..e14cb1d0 100644 --- a/operations/reorg-handling.md +++ b/operations/reorg-handling.md @@ -38,7 +38,7 @@ On each polling cycle (every 5 seconds), the indexer checks the Decoder DB `even ### UTXO Tracker -The UTXO tracker maintains a per-chain undo history (BTC: 12 blocks, LTC: 48 blocks, DOGE: 120 blocks, overridable via XCHAIN_UNDO_BLOCKS_) in its LevelDB store. On detecting a chain tip change from the coin node, it rolls back blocks one at a time until its tip matches the node, then re-indexes forward. +The UTXO tracker maintains a per-chain, per-network undo history in its LevelDB store (mainnet and regtest: BTC 12 blocks, LTC 120 blocks, DOGE 120 blocks; testnet: 120 blocks for every coin; overridable via XCHAIN_UNDO_BLOCKS_). On detecting a chain tip change from the coin node, it rolls back blocks one at a time until its tip matches the node, then re-indexes forward. --- @@ -70,7 +70,7 @@ The `index_addresses` table is intentionally left untouched: it is an append-onl ### UTXO Tracker Rollback -The UTXO tracker keeps a per-chain undo window in its undo log (BTC: 12 blocks, LTC: 48 blocks, DOGE: 120 blocks; overridable via XCHAIN_UNDO_BLOCKS_). Blocks are removed from LevelDB in reverse order until the local tip matches the coin node. A reorg deeper than the configured window would require a full re-sync from scratch; this is extremely rare on any mainnet chain. +The UTXO tracker keeps a per-chain, per-network undo window in its undo log (mainnet and regtest: BTC 12 blocks, LTC 120 blocks, DOGE 120 blocks; testnet: 120 blocks for every coin; overridable via XCHAIN_UNDO_BLOCKS_). Blocks are removed from LevelDB in reverse order until the local tip matches the coin node. A reorg deeper than the configured window would require a full re-sync from scratch; this is extremely rare on any mainnet chain. ```mermaid flowchart TD @@ -130,7 +130,15 @@ Users should not take action based on data from blocks within the last few confi A deep reorg (more than a few blocks) is extremely rare on mainnet but can occur on testnet or regtest. The XChain decoder and indexer handle deep reorgs using the same mechanism as shallow ones, there is no hard limit on rollback depth for the MariaDB-backed services. -The UTXO tracker's per-chain undo window (BTC: 12 / LTC: 48 / DOGE: 120 blocks, overridable via XCHAIN_UNDO_BLOCKS_) is the only component with a depth limit. A reorg deeper than the configured window requires stopping the UTXO tracker, deleting its LevelDB data, and re-syncing from scratch or from a bootstrap archive. +The UTXO tracker's per-chain, per-network undo window is the only component with a depth limit: + +| Network | BTC | LTC | DOGE | +|---|---|---|---| +| mainnet | 12 | 120 | 120 | +| testnet | 120 | 120 | 120 | +| regtest | 12 | 120 | 120 | + +Overridable via `XCHAIN_UNDO_BLOCKS_`. A reorg deeper than the configured window requires stopping the UTXO tracker, deleting its LevelDB data, and re-syncing from scratch or from a bootstrap archive. --- diff --git a/operations/run-a-validator.md b/operations/run-a-validator.md index 974b257a..8b8bd6b4 100644 --- a/operations/run-a-validator.md +++ b/operations/run-a-validator.md @@ -259,6 +259,35 @@ BTC_INDEXER_API_KEY= The validator reads are on the indexer's gated list, so a keyed indexer 401s without the key. The hub names that case in its log. +### Wire the Dogecoin read (required on testnet today) + +ROLLCALL is already active on testnet, and it is armed at genesis on mainnet. +From the epoch it activates, both your BTC indexer and your hub need to read +a **Dogecoin** indexer over JSON-RPC: the BTC indexer proves each epoch's +roll-call signers from Dogecoin before it will close that epoch +(`getrollcallsigners`), and the hub separately asks the same indexer what +already landed before it publishes. Neither failure is loud in the obvious +place. With no DOGE indexer configured, your BTC indexer **defers every +block** from the first epoch close onward (`stallReason: +rollcall_proof_unavailable` on `/status`, and the log line `ROLLCALL PROOF +UNAVAILABLE ... DOGE indexer not configured`), and your hub publishes nothing +for its roll-call rounds while logging nothing about why. A validator stuck +in that state is recorded absent and **evicted after two consecutive +absences**, even while `xchain-node ps` and the hub log otherwise look fine. + +Unless you are running your own Dogecoin testnet or mainnet indexer, point at +the public explorer's replicated read, in `~/xchain-node/.env`: + +``` +DOGE_INDEXER_API_URL=https://explorer.xchain.io/TDOGE/api/ # mainnet: /DOGE/api/ +DOGE_INDEXER_API_KEY= +``` + +That answer comes from the explorer's own replica of the Dogecoin chain, so a +replica that has fallen behind makes your epoch close wait longer rather than +judge the roll call on stale data. If you run a Dogecoin indexer of your own +on the same network, point at it instead. + ## Step 6: decide your capabilities `config/validator/hub-caps/capabilities.json` is ready to go for `price`, @@ -343,6 +372,8 @@ stake activating. You do not need to tell anyone. | Staked and the hub is up, but you never appear in `validator_capabilities` | That table is gossiped from your hub to its peers, not read from the chain | Check the hub log for peer connections and for self-test failures; a capability that fails its self-test is never advertised | | Qualified but never publishing | DOGE wallet empty | Top it up (step 3) | | ROLLCALL signed but never appears on Dogecoin | Hand-built signer module has no `broadcast` export | Add `broadcast(payload)` to the module, or use the CLI-generated signer; `validator status` shows which you have | +| BTC indexer logs `ROLLCALL PROOF UNAVAILABLE ... DOGE indexer not configured`, and it never closes past the first epoch | No `DOGE_INDEXER_API_URL` set | Wire it (see "Wire the Dogecoin read" in step 5) | +| Everything looks fine (`ps`, indexer, decoder all healthy), but your validator is later reported absent or evicted | Hub has no `DOGE_INDEXER_API_URL` either, so it silently publishes nothing for roll-call rounds | Wire it on the hub as well (see "Wire the Dogecoin read" in step 5) | | `bitcoin-cli stop` (or `dogecoin-cli stop`) inside the container, and the daemon is back two seconds later | The container's `unless-stopped` restart policy restarts a daemon that exits, and cannot tell a clean exit from a crash | Stop it from outside: `xchain-node stop node bitcoin mainnet`, which gives the daemon its flush budget. See [Stopping](../components/node/operations.md#stopping) | ## Upgrading diff --git a/operations/xchain-genesis.md b/operations/xchain-genesis.md index a592a079..432c14ac 100644 --- a/operations/xchain-genesis.md +++ b/operations/xchain-genesis.md @@ -33,7 +33,7 @@ validator **reward pool**. XCHAIN exists on the **BTC chain only**. ## How genesis creates the token The token is **not** created by an operator-broadcast transaction. When an indexer parses the -pinned genesis block, `src/genesis.js` injects a synthetic GAS-signed `ISSUE` as the first +pinned genesis block, `src/chain/genesis.js` injects a synthetic GAS-signed `ISSUE` as the first genesis action: | Parameter | Value | Why | diff --git a/overview.md b/overview.md index 25d4c763..1c88960c 100644 --- a/overview.md +++ b/overview.md @@ -11,7 +11,7 @@ A short, plain-language introduction to the platform. For the full protocol spec ## In one paragraph -XChain is a token platform that turns the world's most secure blockchains into programmable, multi-chain token networks: without bridges, sidechains, or a new chain to trust. It embeds a complete protocol (tokens, a built-in exchange, cross-chain swaps, smart contracts, cross-chain contract calls, and on-chain data) directly inside ordinary blockchain transactions, so every token inherits the host chain's proof-of-work security wholesale. It is chain-agnostic by design, live in production today on Bitcoin, Litecoin, and Dogecoin, and built to extend across many blockchains. The product exists, runs, and has settled cross-chain trades end-to-end. +XChain is a token platform that turns the world's most secure blockchains into programmable, multi-chain token networks: no third-party bridge, no sidechain, and no new chain to trust. It embeds a complete protocol (tokens, a built-in exchange, cross-chain swaps, smart contracts, cross-chain contract calls, a native cross-chain token bridge, and on-chain data) directly inside ordinary blockchain transactions, so every token inherits the host chain's proof-of-work security wholesale. It is chain-agnostic by design, live in production today on Bitcoin, Litecoin, and Dogecoin, and built to extend across many blockchains. The product exists, runs, and has settled cross-chain trades end-to-end. ## The problem @@ -23,7 +23,7 @@ The largest, most liquid, most secure chains (Bitcoin and its relatives) were ne XChain is a **metalayer**: a protocol that runs above an unmodified base blockchain, using ordinary transactions to carry its data. Software running the protocol reads those transactions and derives its own state (balances, order books, contract storage) by a fixed, deterministic set of rules anyone can independently replay and verify. -The result is a full digital-token stack expressed as **37 standard ACTIONs**: issue and manage tokens (including non-fungible and limited-edition tokens), transfer and airdrop, trade on a native on-chain exchange, swap tokens trustlessly across chains, deploy smart contracts and call them across chains, publish encrypted token-gated content, run staking, and store data, all secured by the base chain, with no bridge anywhere in the system. +The result is a full digital-token stack expressed as **38 standard ACTIONs**: issue and manage tokens (including non-fungible and limited-edition tokens), transfer and airdrop, trade on a native on-chain exchange, swap tokens trustlessly across chains, deploy smart contracts and call them across chains, publish encrypted token-gated content, run staking, store data, and bridge tokens between chains, all secured by the base chain. Critically, **none of this is Bitcoin-specific.** The metalayer technique works on any suitable chain. Bitcoin, Litecoin, and Dogecoin are the first three; adding another UTXO chain is a configuration change, not a rebuild. XChain is designed to grow into a platform spanning a large number of blockchains over time. @@ -39,7 +39,7 @@ Three things are genuinely hard to replicate: **AI- and web-callable contracts.** XChain contracts can ask the outside world a question (an HTTPS fetch, or a prompt to an approved AI model) and get a *verified* answer back on-chain. A validator network fetches the answer independently, agrees on the result, and writes it so the outcome is reproducible by anyone replaying the chain. This makes a long-promised class of applications finally practical: AI-judged contests and moderation, parametric insurance, prediction markets settled from real sources, data-reactive treasuries. -**Bridgeless multi-chain.** Because XChain never wraps or locks tokens, there's no bridge to attack. Cross-chain swaps are coordinated (never custodied) by a stake-weighted Byzantine-fault-tolerant validator network; tokens stay on their home chains and only ownership changes, and the same rail carries cross-chain contract calls. Cross-chain settlement already works in production. +**No third-party bridge.** XChain never wraps a token or hands it to an outside chain. Cross-chain swaps are coordinated (never custodied) by a stake-weighted Byzantine-fault-tolerant validator network; tokens stay on their home chains and only ownership changes, and the same rail carries cross-chain contract calls. The one exception is XCHAIN, the platform's own fee token: XBRIDGE moves it between supported chains by locking on its own chain and crediting the destination, with no outside contract ever holding the balance. Cross-chain settlement already works in production. Reinforcing these: token-gated encrypted content that unlocks client-side with no key server, a staking primitive that lets any token back any contract on any chain, a fully transparent ledger anyone can replay from genesis, and a light-client path that lets an app verify a balance against quorum-signed checkpoints without trusting any single server. diff --git a/package-lock.json b/package-lock.json index bf0c1cb5..40e3813b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "xchain-documentation", - "version": "0.18.0", + "version": "0.19.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "xchain-documentation", - "version": "0.18.0", + "version": "0.19.0", "license": "AGPL-3.0-or-later", "devDependencies": { "mathjs": "15.2.0" diff --git a/package.json b/package.json index 4ea2226c..4240df99 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "xchain-documentation", "description": "XChain Platform protocol specification, architecture guides, and developer documentation", - "version": "0.18.0", + "version": "0.19.0", "license": "AGPL-3.0-or-later", "repository": { "type": "git", @@ -12,7 +12,7 @@ "node": ">=22.0.0" }, "scripts": { - "test": "node --test --test-force-exit --test-timeout=120000", + "test": "node --test --test-timeout=120000 --test-reporter=tap --test-reporter-destination=stdout --test-reporter=./bin/complete_run_reporter.js --test-reporter-destination=stderr", "ci": "npm test" }, "devDependencies": { diff --git a/protocol/README.md b/protocol/README.md index 02d40e73..e6d0f6d2 100644 --- a/protocol/README.md +++ b/protocol/README.md @@ -19,6 +19,8 @@ This section is the canonical reference for the XChain Protocol; ACTION command | [Contract-Targeted Staking](./contract-staking.md) | Design and VM API for staking any token against a smart contract with contract-decided slashing | | [Cross-Chain Contract Calls](./cross-chain-calls.md) | XCALL: a contract on one chain calls a contract on another, verified by federation capability signatures | | [Cross-Chain DEX](./cross-chain-dex.md) | Network-scoped mirror settlement: trading a token on one chain against a token on another | +| [Cross-Chain Bridge](./xchain-bridge.md) | XBRIDGE: lock-and-mint / burn-and-release for XCHAIN across chains, with a protocol-owned escrow and a signed transfer record | +| [Token Bridge](./token-bridge.md) | The same bridge mechanism opened to any issuer's token, origin-rooted naming, and policy inheritance for allow/block lists and sleep | | [x402 Payments](./x402-payments.md) | HTTP 402 payment interop: machine-payable web resources settled with XChain actions | | [Attestation Providers](./providers/) | Provider specs for the attestation framework (`http_get`, `llm`) | | [Error Codes](./error-codes.md) | Stable machine-readable error-code registry | diff --git a/protocol/action-manifest.json b/protocol/action-manifest.json index ad21bd2e..43a87125 100644 --- a/protocol/action-manifest.json +++ b/protocol/action-manifest.json @@ -20,7 +20,7 @@ "The indexer protocol_changes registry ALSO holds non-action feature-gate flags: any this.addChange(...) entry whose name is NOT a key in this manifest's 'actions' map is a feature gate, not an action, and is excluded here. As of this pass that set is CONTROLLER_GUARD, CROSS_CHAIN_DEX, ISSUANCE_FEE, ISSUANCE_FEE_EMISSION_EXEMPT, LOCK_MAX_SUPPLY_EXACT, UNIFIED_FEES, VM_ACTIONS, VM_BALANCE_TOKENINFO, VM_BANNED_ASYNC, DEPLOY_BASE64_CODE, CROSS_CHAIN_ROYALTY, ISSUE_MINT_SUPPLY_CUMULATIVE_CAP, SLEEP_RESPECTS_LOCK_SLEEP, COINPAY_EXPIRE_TOKEN_AMOUNT, UNSTAKE_COOLDOWN_COMPLETION_ACTION, DELEGATE_REVOKE_NO_REINSERT, CONTRACT_INDEX_CANONICAL, SLASH_BURNS_PENDING_STAKE, NATIVE_FEE_PRICE_TIME_GATE, DEPLOY_INIT_STRICT, but the enumeration is illustrative, not authoritative: trust the exclusion RULE above it, not this list, since new flag-days are added to the registry without a manifest update. The indexer conformance guard compares the dispatch switch, not the registry.", "UNKNOWN is the indexer/explorer catch-all sentinel, not an action; excluded. Each guard drops UNKNOWN before comparing.", "Aliases are expanded to canonical names during canonicalization - before the ACTION-name gate (so no alias appears as its own action entry) but AFTER the compiled-size gate, which therefore measures the alias/wire form, not the expanded canonical record (decoder ACTION_ALIASES / indexer actionAliases).", - "userEncodableVersions is NOT a copy of the SDK Formats keys, it is the audit those keys are checked against. Every entry was read off xchain-indexer/src/actions/.js: the handler's this.formats map is the set of versions the indexer will parse at all, and a version is user-encodable only if nothing in the handler restricts it to indexer-synthesized input. Two versions the indexer parses are therefore absent here: VOTE v2 (finalize) rejects a user broadcast outright via `if(!data['IS_SYNTHETIC'])` in vote.js, and PRICE v0 is the validator COIN/FIAT snapshot, which only validates with a PBFT quorum of Ed25519 signatures from price-capability stakes, so no wallet can author one. Adding a version here that the indexer will not accept from a user is worse than omitting one: the SDK guard would then demand a Format that builds a command guaranteed to be rejected on arrival, so re-run the audit against the handler before editing an array.", + "userEncodableVersions is NOT a copy of the SDK Formats keys, it is the audit those keys are checked against. Every entry was read off xchain-indexer/src/actions/.js: the handler's this.formats map is the set of versions the indexer will parse at all, and a version is user-encodable only if nothing in the handler restricts it to indexer-synthesized input. Two versions the indexer parses are therefore absent here: VOTE v2 (finalize) rejects a user broadcast outright via `if(!data['IS_SYNTHETIC'])` in vote.js, and PRICE v0 is the validator COIN/FIAT snapshot, which only validates with a PBFT quorum of Ed25519 signatures from price-capability stakes, so no wallet can author one. XBRIDGE is the third case and the reason one entry carries a gapped version list: v0/v1 (XCHAIN lock/burn) and v3/v4 (general-token lock/burn) are user-broadcast, while v2 and v5 are the mirror-injected settle legs the indexer synthesizes from a finalized bridge_transfers row and refuses on broadcast ('invalid: XBRIDGE v2 is system-injected'), the ATTEST precedent of one action name carrying mixed formats. Adding a version here that the indexer will not accept from a user is worse than omitting one: the SDK guard would then demand a Format that builds a command guaranteed to be rejected on arrival, so re-run the audit against the handler before editing an array.", "BET is fully rolled out as of the P8 wallet form: wireDecoded + userEncodable (decoder, encoder gate, SDK), indexerHandled + the BET_EXPIRE lifecycle entry (P4), explorerRender (P7), walletForm (P8). It was staged one flag per work package on purpose, because flipping a flag ahead of the code makes that repo's conformance guard red, which is the intended signal rather than an oversight. BET_EXPIRE gained its own explorerRender in a later pass: it owns no table, so its detail reads the bet_feed_statuses row keyed by its action_index and joins through to the feed it expired." ], "aliases": { @@ -230,7 +230,7 @@ "wireDecoded": true, "indexerHandled": true, "userEncodable": true, - "userEncodableVersions": [0, 1, 2, 3, 4, 5, 6], + "userEncodableVersions": [0, 1, 2, 3, 4, 5, 6, 7], "explorerRender": true, "walletForm": true }, @@ -414,6 +414,15 @@ "explorerRender": true, "walletForm": true }, + "XBRIDGE": { + "category": "wire-user", + "wireDecoded": true, + "indexerHandled": true, + "userEncodable": true, + "userEncodableVersions": [0, 1, 3, 4], + "explorerRender": true, + "walletForm": true + }, "XCALL": { "category": "mirror-injected", "indexerHandled": true, diff --git a/protocol/actions/README.md b/protocol/actions/README.md index 0a729cc9..3faff8fe 100644 --- a/protocol/actions/README.md +++ b/protocol/actions/README.md @@ -57,6 +57,12 @@ The same ACTION specifications apply across all chains. Chain-specific behavior | [`DISPENSER`](./dispenser.md) | Creates a vending machine that dispenses tokens when triggered by a send | | [`SWAP`](./swap.md) | Creates a cross-chain token swap offer between supported blockchains | +### Cross-Chain Bridge + +| ACTION | Description | +|---|---| +| [`XBRIDGE`](./xbridge.md) | Lock-and-mint / burn-and-release across chains: v0/v1/v2 move XCHAIN itself, v3/v4/v5 generalize the same lifecycle to any bridgeable token under its origin chain's root | + ### Data and Communication | ACTION | Description | @@ -150,7 +156,7 @@ Every ACTION includes a `VERSION` parameter as its first field. This determines ### TICK -A `TICK` is a token ticker name (1-250 characters). Tickers are case-sensitive and can contain letters, numbers, and symbols. The names `BTC`, `LTC`, `DOGE`, and `XCHAIN` are reserved by the protocol. +A `TICK` is a token ticker name (1-250 characters). Tickers are case-sensitive and can contain letters, numbers, and symbols. The names `BTC`, `LTC`, `DOGE`, and `XCHAIN` are reserved by the protocol, case-folded (`btc`, `Btc`, ... are reserved too) so no case variant can be squatted; see [`ISSUE`](./issue.md) and [Token Bridge](../token-bridge.md) for why: a bridged token is always named under its origin chain's coin root (`BTC.PEPECASH`). Behind `TICK_NAMESPACE_ACTIVATION`, a new top-level tick under four characters is also refused (`invalid: TICK (length)`, creation only, existing tickers unaffected), and a fixed list of chain codes XChain expects to integrate later (`RESERVED_FUTURE_ROOTS`) is reserved the same way, so their roots are free the day each chain actually arrives. ### ACTION_INDEX diff --git a/protocol/actions/anchor.md b/protocol/actions/anchor.md index e7d0476c..42f1ddf9 100644 --- a/protocol/actions/anchor.md +++ b/protocol/actions/anchor.md @@ -267,8 +267,8 @@ and at/above the `EQUIV_HEADER` flag-day: EQUIV|XCHECKPOINT|XANCPUB|archive|NETWORK|MATCH_BATCH_SEQ|SNAPSHOT_BLOCK|0||XANCPUB|anchor_archive|MATCH_BATCH_SEQ|SNAPSHOT_BLOCK|PUBLISHER|ARCHIVE_REWARD_AMOUNT ``` -These bytes are byte-identical across the hub producer (`StateAnchorPublisher._attestationCanonical`), -the indexer verifier (`actions/anchor.js` `_rewardCanonical`) and this spec; a divergence forks the +These bytes are byte-identical across the hub producer (`StateAnchorPublisher.attestationCanonical`), +the indexer verifier (`actions/anchor/index.js` `rewardCanonical`) and this spec; a divergence forks the derived reward row. An `ASIG_n` counts only if its pubkey is in the SAME `oracle_publish` snapshot at `SNAPSHOT_BLOCK` used for the root quorum **and** the Ed25519 signature verifies. @@ -498,7 +498,7 @@ flowchart TD None of these is consensus data: they are per-hub operator knobs, and two hubs running different values still produce mutually verifiable anchors. What follows is the derivation of each magnitude, so a tuner can tell what is load-bearing from what is merely a round number. -The arithmetic is pinned by `xchain-hub/test/unit/StateAnchorPublisher.constant-derivations.test.js`. +The arithmetic is pinned by `xchain-hub/test/unit/anchor/publisher/state_anchor_publisher_constant_derivations.test.js`. **`ANCHOR_CHUNK_MAX_BYTES` (6000).** The hard ceiling is `MAX_ACTION_DATA_LENGTH` = 8192 compiled bytes (`protocol/constants.js`). The decoder is the arbiter and *silently drops* any @@ -559,7 +559,7 @@ which is anti-spam only. ## Recovery procedure (full-parse) 1. Sync DOGE through the decoder/indexer from genesis: `anchor_actions` populates from the chain alone. -2. Run `xchain-indexer/src/recovery.js --skip-stake-verification --i-understand-unverified`: +2. Run `xchain-indexer/bin/recovery.js --skip-stake-verification --i-understand-unverified`: reassembles chunked batches by `MATCH_BATCH_SEQ`, gunzips, verifies `BATCH_CRC32`, verifies each archived match's/call's `validator_signatures` against the archived diff --git a/protocol/actions/attest.md b/protocol/actions/attest.md index 1189193d..c3b12728 100644 --- a/protocol/actions/attest.md +++ b/protocol/actions/attest.md @@ -101,7 +101,7 @@ System-synthesized expiry for request abc...def - `CONTRACT_INDEX` (carried via `EMITTER`) must reference an existing contract. - `REQUEST_ID` is verified by re-deriving from `tx_hash:root_action_index:emitter_path:contract_index:emitter_position` (colon-delimited; defends against compromised VM). - Admission flag-day (`ATTEST_ADMISSION_ACTIVATION` in `protocol/constants.js`; mainnet 961000, testnet/regtest genesis): at/above the height, a request whose responsible set at its own block is smaller than `REDUNDANCY` (e.g. after the stake-weighted-quorum source-dedupe) is rejected at admission, since the v1 path can never collect `REDUNDANCY` signatures from a smaller set. Below the height the request is accepted and expires at `DEADLINE_BLOCK` unchanged (replay bit-identical). -- Per-block admission caps (`ATTEST_REQUEST_CAP_ACTIVATION` and `ATTEST_REQUEST_CAPS` in `protocol/constants.js`; testnet/regtest genesis, mainnet unset): a block admits at most `perContract` (2) requests from any one contract and `perBlock` (10) in total, counted over the admitted v0s earlier in the same block (`action_index` order, which is total, so every node counts the same prefix). An admitted request obliges `REDUNDANCY` validators to make a provider call the requester does not pay for, so the ceiling is what bounds validator spend; the per-contract share stops one contract taking the whole ceiling. Over-cap produces `invalid: ATTEST cap (…)`, which, being an emission validation failure, fails the whole enclosing EXECUTE: the over-cap request, the under-cap requests the same EXECUTE already emitted, and its state writes all roll back together. A contract that needs more than two attestations spaces them across blocks. Where the activation height is unset the rule is inert and admission is uncapped. +- Per-block admission caps (`ATTEST_REQUEST_CAP_ACTIVATION` and `ATTEST_REQUEST_CAPS` in `protocol/constants.js`; genesis-active on every network, mainnet armed at `0` under the 2026-09-09 ruling on a history holding zero attestations): a block admits at most `perContract` (2) requests from any one contract and `perBlock` (10) in total, counted over the admitted v0s earlier in the same block (`action_index` order, which is total, so every node counts the same prefix). An admitted request obliges `REDUNDANCY` validators to make a provider call the requester does not pay for, so the ceiling is what bounds validator spend; the per-contract share stops one contract taking the whole ceiling. Over-cap produces `invalid: ATTEST cap (…)`, which, being an emission validation failure, fails the whole enclosing EXECUTE: the over-cap request, the under-cap requests the same EXECUTE already emitted, and its state writes all roll back together. A contract that needs more than two attestations spaces them across blocks. Where the activation height is unset the rule is inert and admission is uncapped. #### Responsible-set selection @@ -240,7 +240,7 @@ stateDiagram-v2 ### Response delivery: on-chain versus the hub mirror -Each network has its own response-mirror activation height, `ATTEST_RESPONSE_MIRROR_ACTIVATION` in `protocol/constants.js`, checked against the request's own block: mainnet is `null` (inactive; every mainnet request today uses the on-chain path above), testnet is `null` until the operator arms it, and regtest is `0` (active from genesis, so every regtest request already uses the mirror). +Each network has its own response-mirror activation height, `ATTEST_RESPONSE_MIRROR_ACTIVATION` in `protocol/constants.js`, checked against the request's own block: mainnet is `null` (inactive; every mainnet request today uses the on-chain path above), testnet is `151324` (armed on 2026-09-07 at the chain tip in the v0.15.0 train, so a testnet request at or above that block uses the mirror), and regtest is `0` (active from genesis, so every regtest request already uses the mirror). Steps 1 through 3 above are unchanged either way: the request is emitted the same way, and the same top-`REDUNDANCY` validators still agree on the answer together. What changes is what happens with that agreed answer. diff --git a/protocol/actions/batch.md b/protocol/actions/batch.md index 3773c657..3d8502f2 100644 --- a/protocol/actions/batch.md +++ b/protocol/actions/batch.md @@ -53,7 +53,7 @@ This example registers the JDOG parent token and three of its children in one tr - **Batch-level rejections** invalidate the whole `BATCH` as one record, before any command runs: an unknown `VERSION`, a command naming an action that is not enabled (an empty command counts), more than one `MINT` of the SAME token, more than one top-level `ISSUE`, a nested `BATCH`, a sleeping `SOURCE`, going over the command cap, and a source that provably cannot pay for even the cheapest command in the list (`invalid: GAS (insufficient)`). That last check is a lower bound only: gas is billed greedily in list order against one running balance, so a source that can afford some of the commands is let through and lands exactly the ones it can pay for. - **Activation.** The child-issuance exemption, the caret-dot rejection, the per-distinct-token `MINT` rule, the 250-command cap, the cumulative fee and settlement accounting, and the aggregate gas pre-check all activate together at `BATCH_ISSUANCE_LIMITS`. That gate is **active from genesis on testnet and regtest**, and on **mainnet it has been active since `2026-08-16T00:00:00Z`**. Sub-command output capture (`BATCH_SUBCOMMAND_OUTPUT_CAPTURE_ACTIVATION`, which is what lets a batched `COINPAY` or `DISPENSER` be seen at all) armed at the **same instant** on mainnet, so the two halves of batch behavior arrived together rather than leaving a window where one is live and the other is not. Below that instant, mainnet history keeps the behavior it always had: one `ISSUE` per `BATCH` (dotted or not), one `MINT` per `BATCH` whatever token it names, no command cap, and per-command fee checks that each read the transaction's untouched value. Blocks before the instant are unaffected, which is why the instant was set in the future rather than backdated. The non-atomic settlement described above is not gated: it has always been how a `BATCH` behaves. - **Sub-action normalization (flag-day gated):** at and after the `BATCH_SUBACTION_NORMALIZATION` activation, sub-actions inside a `BATCH` get the same alias rewrite and legacy VERSION-0 injection as top-level actions. That gate is **active from genesis on testnet and regtest**, and on **mainnet it has been active since the coordinated [contract-era flag day](../flag-days.md#contract-era-flag-day) at `2026-08-07T00:00:00Z`**, so canonical `ACTION` names and an explicit `VERSION` are no longer required of a `BATCH` command on any live network. Below that instant, mainnet history keeps the unnormalized behavior, and a re-decode of those blocks must reproduce it: `ACTION` aliases (`TRANSFER`, `ADDR`, `DROP`, `CAST`, `MSG`) invalidate the whole `BATCH` (`invalid: ACTION (unknown)`), and legacy `ISSUE`/`MINT`/`SEND` params that omit the `VERSION` field are misparsed (the first param is read as the format version). -- A `BATCH` may contain at most one `FILE` action. The decoder stores one `raw_data` payload per transaction, so a second `FILE` in the same `BATCH` would fail at the `FILE` handler rather than the `BATCH` validator. This is an architectural limit of the wire format, not an explicit `actionLimits` rule in `batch.js`. +- A `BATCH` may contain at most one `FILE` action, and nothing on the indexing path enforces it. The decoder exposes one `raw_data` payload per transaction and the batch dispatcher carries that same payload into every sub-command, so a second `FILE` is not rejected: it is recorded valid and stored with the FIRST file's payload duplicated into its record. Because `FILE` records are immutable once mined, the mistake is silent and permanent. This is an architectural limit of the wire format, not an explicit `actionLimits` rule in `batch.js`; the SDK's batch builder refuses to compose such a `BATCH`, but a caller hand-rolling encoder input must enforce the limit itself. - A `FILE` may be batched with other actions, most commonly a `MESSAGE` v2 (ECIES) carrying the file's symmetric key, so that publishing a [token-gated file](../token-gated-content.md) and committing the key happen in one transaction. The two commands still settle independently, so check that both were recorded valid rather than assuming the pair moved as a unit. - `BATCH(SEND, MESSAGE)` is the canonical composition for transferring a token that has [active gated content](./send.md), the `MESSAGE` is required and re-encrypts the content keys to the recipient. diff --git a/protocol/actions/collect.md b/protocol/actions/collect.md index 5d05b3d2..0652df29 100644 --- a/protocol/actions/collect.md +++ b/protocol/actions/collect.md @@ -61,7 +61,7 @@ Rewards accumulate from multiple validator activities, all stored in the indexer Reward rows reach the indexer's `validator_rewards` table on two rails: - **Derived (replayable):** `oracle_round` / `oracle_base` / `oracle_full_node` and `attest_fee` are computed by the indexer itself during block processing, as deterministic functions of on-chain actions. The oracle reward type used depends on whether the full-node reward tier is active (`FULLNODE.REWARD_SHARE` > 0): when inactive the full per-round budget is credited as `oracle_round`; when active it is split into an `oracle_base` tranche (all qualified signers) and an `oracle_full_node` tranche (verified full-node sources that met the participation threshold). `attest_fee` splits a fulfilled request's fee across its responsible set. A reindex reproduces these rows exactly. -- **Pushed (archived):** `anchor_bundle` / `anchor_archive` are recorded by the hub federation when an anchor publishes and pushed via the `pushvalidatorrewards` JSON-RPC endpoint (which rejects any non-anchor type). Because a chain parse cannot re-derive them, they ride the ANCHOR v1 archive and are restored by full-parse recovery (see [ANCHOR](anchor.md). +- **Derived at or above the reward flag-day, pushed below it:** `anchor_bundle` and `anchor_archive` each have their own boundary, `ANCHOR_REWARD_ACTIVATION` and `ARCHIVE_REWARD_ACTIVATION` in `protocol/constants.js` (mainnet 961000 and 963000; both genesis-active on testnet and regtest). At or above its own flag-day the type's reward is DERIVED by every indexer from the on-chain ANCHOR bytes, the elected `PUBLISHER` plus a quorate `XANCPUB` attestation, crediting the frozen reward amount and never an amount from the wire, so a chain parse reproduces the row exactly. Below it the hub federation recorded the reward when the anchor published and pushed it via the `pushvalidatorrewards` JSON-RPC endpoint (which rejects any non-anchor type); those historical rows a chain parse cannot re-derive, so they ride the ANCHOR v1 archive and are restored by full-parse recovery (see [ANCHOR](anchor.md)). `COLLECT` queries the indexer's `validator_rewards` table directly. No hub round-trip during transaction processing. @@ -80,8 +80,9 @@ If the pool cannot cover the full pending reward, the `COLLECT` is rejected with ```mermaid flowchart TD D1["Indexer computes oracle_round / oracle_base /
oracle_full_node / attest_fee during block processing"] - P1["Hub federation records anchor_bundle / anchor_archive
reward on publish"] - P2["Pushed via pushvalidatorrewards JSON-RPC"] + D2["Indexer derives anchor_bundle / anchor_archive
from the on-chain ANCHOR bytes
(at or above that type's reward flag-day)"] + P1["Hub federation recorded anchor_bundle / anchor_archive
reward on publish (below the flag-day)"] + P2["Pushed via pushvalidatorrewards JSON-RPC
(retired path, pre-flag-day history only)"] VR[("validator_rewards table")] C1["COLLECT sums unclaimed rewards
at or before its own block"] C2{"Reward pool holds
enough XCHAIN?"} @@ -89,6 +90,7 @@ flowchart TD C4["Rejected: insufficient reward pool
(reward stays unclaimed, collectible later)"] D1 -->|"derived, replayable"| VR + D2 -->|"derived, replayable"| VR P1 --> P2 P2 -->|"pushed, archived via ANCHOR v1"| VR VR --> C1 diff --git a/protocol/actions/dividend.md b/protocol/actions/dividend.md index 8bc08171..cfaf9a74 100644 --- a/protocol/actions/dividend.md +++ b/protocol/actions/dividend.md @@ -37,6 +37,7 @@ This example pays a dividend of 0.5 BACON token to every holder of 1 TEST token - To send `TICK` to a large number of users, see the `AIRDROP` or `SEND` commands - If `TICK` is divisible and `DIVIDEND_TICK` is non-divisble, quantities whose calculated share rounds to 0 will receive no `DIVIDEND_TICK`. Such holders are excluded from the recipient list entirely, so they do **not** count toward the per-recipient fee charged to `SOURCE` - `SOURCE` address is excluded from receiving dividends +- If `DIVIDEND_TICK` carries a non-empty `ALLOW_LIST`, only holders on that list receive the dividend, and holders on its non-empty `BLOCK_LIST` are excluded. The lists consulted are those of the payment token (`DIVIDEND_TICK`), not the share token (`TICK`); a configured but empty list admits everyone. Excluded holders do **not** count toward the per-recipient fee charged to `SOURCE` - Any `ADDRESS` may pay out a `DIVIDEND` on any `TICK` - Use `^` (caret) as prefix when passing `TICK_ID` for `TICK` field (^1234 = `TICK_ID` 1234) diff --git a/protocol/actions/issue.md b/protocol/actions/issue.md index 75d3382e..c48b3997 100644 --- a/protocol/actions/issue.md +++ b/protocol/actions/issue.md @@ -37,6 +37,9 @@ This action creates or updates a `TICK`. | `ACTION_CLASS` | String | Which class the binding gates: `transfer`, `trade`, `burn`, `mint`, `stake`, `ownership`, or the catch-all `all` (fallback for any class with no specific binding; most-specific-wins) | | `COOLDOWN_BLOCKS` | String | Drop-cooldown committed at bind time: blocks of friction before a later `UNBIND` takes effect | | `UNBIND` | String | `1` drops the live binding for `ACTION_CLASS`; `0` binds | +| `BRIDGE_CHAINS` | String | Comma list of destination coins this `TICK` may bridge to via [`XBRIDGE`](./xbridge.md), or `-` for none; empty means unchanged | +| `MIN_DEPTH` | String | Confirmation depth the federation must honour for this `TICK`'s bridge locks, raise-only over the platform default; empty means unchanged | +| `LOCK_BRIDGE` | String | `1` permanently locks `BRIDGE_CHAINS` and `MIN_DEPTH` | | `MEMO` | String | An optional memo to include | ## Formats @@ -62,6 +65,9 @@ This action creates or updates a `TICK`. ### Version `6` - Bind/unbind a `CONTROLLER` for one `ACTION_CLASS` - `VERSION|TICK|CONTROLLER|ACTION_CLASS|COOLDOWN_BLOCKS|UNBIND|MEMO` +### Version `7` - Bridge opt-in +- `VERSION|TICK|BRIDGE_CHAINS|MIN_DEPTH|LOCK_BRIDGE|MEMO` + ## Examples ``` ISSUE|0|JDOG @@ -121,6 +127,10 @@ This example issues a TEST token with a max supply of 100, and a maximum mint of - `MAX_SUPPLY` max value is 1,000,000,000,000,000,000,000 (1 Sextillion) - `MAX_SUPPLY` can not be set below existing supply - `LOCK_MAX_SUPPLY` can only be set to `1` when the token's `MAX_SUPPLY` is set (`MIN_TOKEN_SUPPLY` or greater), declared in the same `ISSUE` or already on the token record. Minted supply is NOT required: a fair-mint token may declare its `MAX_SUPPLY` and permanently lock it at issuance, before any supply exists. Setting `LOCK_MAX_SUPPLY` with no `MAX_SUPPLY` declared is invalid. +- On any chain other than BTC, a broadcast `ISSUE` of the `XCHAIN` (GAS) tick is refused with `invalid: TICK (BTC-only)`, from every source including the GAS address, on every network including regtest; the reserved-tick check is case-folded (`btc`, `BTC`, `Btc`, ... are all reserved everywhere), closing a gap where an exact-case check let a mismatched-case ticker through. System-injected creation of the tick (the [`XBRIDGE`](./xbridge.md) v2/v5 settle leg creating a chain's first shadow row) is exempt from this refusal: it is not a broadcast `ISSUE`. See [Cross-Chain Bridge](../xchain-bridge.md) and [Token Bridge](../token-bridge.md). +- **Reserving room for chains XChain integrates later.** Behind `TICK_NAMESPACE_ACTIVATION`, two further rules protect the short and chain-code namespace before a squatter can take it: a top-level `ISSUE` that would CREATE a brand-new tick shorter than four characters is refused with `invalid: TICK (length)` (editing or re-issuing an existing tick, including the `^id` form, is unaffected, so anything issued before the flag keeps its owner and supply); and a fixed list of future chain codes (`RESERVED_FUTURE_ROOTS`, e.g. `ETH`, `SOL`, `AVAX`, ...; see [Flag-Day Values](../flag-days.md) for the exact list) is reserved the same way `BTC`, `LTC` and `DOGE` already are, refused with `invalid: TICK (reserved)`. Both rules apply to top-level creation only; a subasset such as `ABCD.X` is unaffected by the length floor. Below the activation the handler is unchanged. +- **Format `7` refusals**, in addition to the field checks above: a destination not in `COINS` other than this chain is `invalid: BRIDGE_CHAINS`; a format `7` on a row with `LOCK_BRIDGE` already set is `invalid: BRIDGE_CHAINS (locked)`; a non-empty `BRIDGE_CHAINS` naming a native tick that contains a dot is `invalid: TICK (subassets are not bridgeable yet)`; a non-empty `BRIDGE_CHAINS` on a token carrying a live `ALLOW_LIST`, `BLOCK_LIST` or controller binding is `invalid: TICK (policy-bound tokens are not bridgeable yet)`; a non-empty `BRIDGE_CHAINS` whose `ALLOW_LIST` or `BLOCK_LIST` membership exceeds `XPOLICY_MAX_MEMBERS` (10,000) is `invalid: TICK (policy list exceeds XPOLICY_MAX_MEMBERS)`. Behind `TOKEN_POLICY_INHERITANCE_ACTIVATION` the policy-bound refusal lifts (see [Token Bridge](../token-bridge.md#policy-inheritance)); the controller refusal and the membership ceiling never lift on their own. +- **Formats `5` and `6` on a bridged token.** While a row's `BRIDGE_CHAINS` is non-empty or its `bridged` bit is set (the bit is set by the first applied bridge lock and never cleared), a format `5` (list edit) or format `6` (controller bind/unbind), and a format `0` re-issue carrying a non-empty `ALLOW_LIST` or `BLOCK_LIST`, are refused with `invalid: TICK (bridged tokens cannot be policy-bound yet)`. Emptying `BRIDGE_CHAINS` afterward does not reopen the door while bridged copies are outstanding. Behind `TOKEN_POLICY_INHERITANCE_ACTIVATION`, formats `5` and a listed format `0` on a bridged token apply and propagate to every bridged copy (see [Token Bridge](../token-bridge.md#policy-inheritance)); a controller bind (format `6`) on a bridged token stays refused either way. ## Notes - `ISSUE` `TICK` with `MAX_SUPPLY` and `MINT_SUPPLY` set to any non `0` value, to mint supply until `MAX_SUPPLY` is reached (owner can mint beyond `MAX_MINT`) diff --git a/protocol/actions/list.md b/protocol/actions/list.md index 272c187e..2654ee3f 100644 --- a/protocol/actions/list.md +++ b/protocol/actions/list.md @@ -59,6 +59,8 @@ This example creates a new list from an existing list (4321) and removes 2 addre empty list. Read the resulting membership back rather than assuming what you sent - A `TICK` list contains only `TICK` items - A `ADDRESS` list contains only `ADDRESS` items +- An `ADDRESS` item validates against this chain's own coin and network by default. Behind `TOKEN_POLICY_INHERITANCE_ACTIVATION`, an item that is a valid address of ANY coin the platform runs (BTC, LTC, DOGE, at this network) is admitted, not only this chain's own coin; this widens which items an `ADDRESS` list can hold, it never narrows one. See [Token Bridge](../token-bridge.md#policy-inheritance) for why: a bridged token's allow/block list is enforced identically on every chain it has a copy on, so the list has to be able to name a holder on any of them. Below the flag, an item of another chain's address format fails its type check like any other malformed `ADDRESS` and is recorded `invalid`, per the rule above. +- A `LIST` edit (`VERSION 1`) whose `LIST_ACTION_INDEX` names a list created by a chain's own `ADDRESS.BRIDGE_` role address is refused with `invalid: LIST_ACTION_INDEX (bridge-owned)`. Those lists exist only once a bridged token's policy has been carried to this chain (see [Token Bridge](../token-bridge.md#policy-inheritance)); no user key owns them, and only the platform's own injected edits, carrying the finalized policy snapshot's membership, may ever change one. ## Notes - Format version `0` allows for creating a list of `TYPE` diff --git a/protocol/actions/rollcall.md b/protocol/actions/rollcall.md index fdd92d8e..0c8d44ce 100644 --- a/protocol/actions/rollcall.md +++ b/protocol/actions/rollcall.md @@ -77,7 +77,7 @@ EQUIV|XROLLCALL||0||||`: 1. The indexer's own coin must be `DOGE`, else `invalid: ROLLCALL only valid on DOGE`. -2. `ROLLCALL_ACTIVATION[network]` must be finite and `EPOCH_HEIGHT >=` it, else `invalid: VERSION (unknown)`. A non-finite gate (mainnet's `null`) means inert. The value compared is the carried **BTC** `EPOCH_HEIGHT`, the same number the BTC close gates on, so a pre-activation roll call is inert on both chains and no DOGE-height flag day exists. +2. `ROLLCALL_ACTIVATION[network]` must be finite and `EPOCH_HEIGHT >=` it, else `invalid: VERSION (unknown)`. A non-finite gate means inert; mainnet reads `0` (armed at genesis under the 2026-09-09 ruling) and an un-opted-in regtest venue is the `null` case. The value compared is the carried **BTC** `EPOCH_HEIGHT`, the same number the BTC close gates on, so a pre-activation roll call is inert on both chains and no DOGE-height flag day exists. 3. `EPOCH_HEIGHT % ROLLCALL_INTERVAL_BLOCKS[network] == 0`, else `invalid: EPOCH_HEIGHT`. No staleness or accept-window check here: those compare BTC heights and belong to the BTC close. 4. `LEDGER_HASH` and `PUBLISHER` must be 64 hex (lowercased before use), else `invalid: LEDGER_HASH` / `invalid: PUBLISHER`. 5. Every `SIG_i` must verify over the canonical rebuilt from `(network, EPOCH_HEIGHT, LEDGER_HASH)`, with no duplicate pubkey. Dedupe in wire order, and mark a key seen only **after** its signature verifies, so a garbage pair before a valid one cannot suppress the valid one. A roll call with zero valid pairs is `invalid: SIG_COUNT`. @@ -127,7 +127,7 @@ All eight values are **consensus** and frozen in `protocol/constants.js`, with b | Constant | mainnet | testnet | regtest | Unit | |---|---|---|---|---| -| `ROLLCALL_ACTIVATION` | `null` (inert) | 151200 | `null` (inert), arms at `0` on opt-in | BTC height | +| `ROLLCALL_ACTIVATION` | 0 (armed at genesis) | 151200 | `null` (inert), arms at `0` on opt-in | BTC height | | `ROLLCALL_INTERVAL_BLOCKS` | 1008 | 1008 | 30 | BTC blocks | | `ROLLCALL_ACCEPT_WINDOW_BLOCKS` | 144 | 144 | 12 | BTC blocks | | `ROLLCALL_PROOF_DELAY_BLOCKS` | 36 | 36 | 2 | BTC blocks | @@ -136,7 +136,7 @@ All eight values are **consensus** and frozen in `protocol/constants.js`, with b | `ROLLCALL_STREAK_LOOKBACK` | 4 | 4 | 4 | rolled epochs | | `ROLLCALL_REWARD_AMOUNT` | `10.00000000` | `10.00000000` | `10.00000000` | XCHAIN | -Every gate keys on the carried BTC `EPOCH_HEIGHT`, never on either chain's local height. Mainnet ships inert: the operator pins that height with the mainnet federation. +Every gate keys on the carried BTC `EPOCH_HEIGHT`, never on either chain's local height. Mainnet is armed at genesis (`0`) under the 2026-09-09 ruling: the indexed mainnet history holds zero validators and zero roll calls, so arming from block 0 reinterprets nothing. **Regtest is the one network whose height a venue pins for itself.** Every other value here is fixed in source and unreadable from the environment, because on a shared ledger a tunable consensus input is a fork waiting to happen. A regtest chain is private, so no two venues validate the same blocks and nothing a venue pins can fork anybody. It still ships inert, because arming a network commits every BTC indexer on it to a wired DOGE peer, and a single-coin BTC venue would defer forever at its first close. A two-chain venue opts in by setting `XC_ROLLCALL_REGTEST_ACTIVATION=armed` on every BTC indexer and hub it runs, which arms the network at height `0`; the same variable also takes a bare height for a venue whose epochs should begin above an already-indexed prefix. Anything unrecognised leaves the venue inert. Because `ROLLCALL_ACTIVATION` is one of the shared gates in the consensus-rules digest, a venue that arms its hubs and forgets its indexer reports a rules mismatch rather than disagreeing silently about which epochs exist. diff --git a/protocol/actions/xbridge.md b/protocol/actions/xbridge.md new file mode 100644 index 00000000..7b819eb4 --- /dev/null +++ b/protocol/actions/xbridge.md @@ -0,0 +1,99 @@ + + + +# XChain Platform Action - XBRIDGE +This action moves a token between chains by lock-and-mint / burn-and-release against a protocol-owned escrow, never by minting on two ledgers independently. Version `0` locks XCHAIN on BTC for a destination chain; version `1` burns XCHAIN on a destination chain to release it back on BTC; version `2` is the system-injected settle leg that applies a finalized transfer, on either side, and is never user-broadcast. Versions `3`, `4` and `5` generalize the same three-step lifecycle to any bridgeable token, naming the bridged copy under its origin chain's root (`BTC.PEPECASH`, `DOGE.FUFU`). See [Cross-Chain Bridge](../xchain-bridge.md) for XCHAIN's own bridge and [Token Bridge](../token-bridge.md) for the general framework, rooted naming, and the issuer opt-in. + +Every version shares one manifest entry, one decoder name, one doc page (the `ATTEST` precedent: one name, mixed user- and system-formats). A version is refused with `invalid: XBRIDGE before activation` below its activation height (`XCHAIN_BRIDGE_ACTIVATION` for versions 0 to 2, keyed per chain as `':'` because BTC, LTC and DOGE arm at three different heights; `TOKEN_BRIDGE_ACTIVATION` for versions 3 to 5, keyed per network; the latter never lower than the former on any chain). + +## PARAMS +| Name | Type | Description | +| -------------- | ------- | ------------------------------------------------------------------------------------------------- | +| `VERSION` | Integer | Format version (0=lock XCHAIN, 1=burn XCHAIN, 2=settle XCHAIN, 3=lock token, 4=burn bridged token, 5=settle token) | +| `DEST_COIN` | String | Destination coin, a supported coin other than the source chain; v0, v3 | +| `DEST_ADDRESS` | String | Destination address on `DEST_COIN`, validated coin-and-network aware; v0, v3 | +| `BTC_ADDRESS` | String | Destination BTC address, BTC-network validated; v1 | +| `TICK` | String | v3: a native (undotted, non-GAS) tick on this chain to lock. v4: a bridged row on this chain (`.`) to burn | +| `ORIGIN_ADDRESS` | String | Destination address on the bridged tick's origin chain, validated against that chain; v4 | +| `AMOUNT` | String | Positive decimal, at most the token's `DECIMALS` fractional digits, at most the source's balance; v0, v1, v3, v4 | +| `TRANSFER_ID` | String | 64-hex id of the finalized `bridge_transfers` row this settle leg applies; v2, v5 (system-injected only, never user-supplied) | +| `MEMO` | String | An optional memo to include; v0, v1, v3, v4 | + +## Formats + +### Version `0` - Lock XCHAIN (user-broadcast, BTC only) +- `XBRIDGE|0|DEST_COIN|DEST_ADDRESS|AMOUNT|MEMO` + +### Version `1` - Burn XCHAIN (user-broadcast, non-BTC only) +- `XBRIDGE|1|BTC_ADDRESS|AMOUNT|MEMO` + +### Version `2` - Settle XCHAIN (system-injected; never broadcast) +- `XBRIDGE|2|TRANSFER_ID` + +### Version `3` - Lock a token (user-broadcast, origin chain only) +- `XBRIDGE|3|TICK|DEST_COIN|DEST_ADDRESS|AMOUNT|MEMO` + +### Version `4` - Burn a bridged token (user-broadcast, bridged rows only) +- `XBRIDGE|4|TICK|ORIGIN_ADDRESS|AMOUNT|MEMO` + +### Version `5` - Settle a token (system-injected; never broadcast) +- `XBRIDGE|5|TRANSFER_ID` + +## Examples +``` +XBRIDGE|0|DOGE|D8bFJYQ6JZ4tSjzZbXqXYh2vN3xKzQpump|500| +Locks 500 XCHAIN on BTC into the DOGE escrow (ADDRESS.BRIDGE_DOGE); a matching XBRIDGE v2 credits the DOGE address once the federation finalizes the transfer +``` + +``` +XBRIDGE|1|1ExampleAddressXXXXXXXXXXXXXXXXXXX|200| +Burns 200 XCHAIN on DOGE, lowering DOGE's XCHAIN supply by 200; a matching XBRIDGE v2 on BTC releases 200 from ADDRESS.BRIDGE_DOGE to the named BTC address +``` + +``` +XBRIDGE|3|PEPECASH|DOGE|D8bFJYQ6JZ4tSjzZbXqXYh2vN3xKzQpump|1000| +Locks 1000 native PEPECASH on BTC (PEPECASH's origin chain) into the DOGE escrow; the copy arrives on DOGE as BTC.PEPECASH +``` + +``` +XBRIDGE|4|BTC.PEPECASH|1ExampleAddressXXXXXXXXXXXXXXXXXXX|250| +Burns 250 of the bridged BTC.PEPECASH row on DOGE; a matching XBRIDGE v5 releases 250 native PEPECASH on BTC to the named BTC address +``` + +## Rules +- Below `XCHAIN_BRIDGE_ACTIVATION` for this chain (v0 to v2; the map is keyed `':'`, falling back to the bare network key) or `TOKEN_BRIDGE_ACTIVATION` for this network (v3 to v5), every version returns `invalid: XBRIDGE before activation`. The height compared is always the block index of the chain being parsed, never a transfer's snapshot block. +- **Version 0 (lock XCHAIN).** BTC only; on any other chain, `invalid: XBRIDGE (BTC only)`. `DEST_COIN` must be a supported coin other than BTC (`invalid: DEST_COIN`). `DEST_ADDRESS` is validated with the coin-and-network-aware address check for `DEST_COIN` (`invalid: DEST_ADDRESS`). `AMOUNT` must be a positive decimal at up to 8 fractional digits and no more than the source's XCHAIN balance (`invalid: AMOUNT`, `invalid: insufficient funds`). Debits the source and credits `ADDRESS.BRIDGE_`; no supply change on BTC. Fee: `XBRIDGE_BASE` (5,000 gas). +- **Version 1 (burn XCHAIN).** Non-BTC only; on BTC, `invalid: XBRIDGE v1 is not valid on BTC`. `BTC_ADDRESS` is validated as a BTC address (`invalid: BTC_ADDRESS`). Debits the source and lowers this chain's XCHAIN supply by `AMOUNT`. Fee: `XBRIDGE_BASE`, paid in native coin. +- **Version 2 (settle XCHAIN).** System-injected only from a finalized `bridge_transfers` row; a broadcast v2 is refused with `invalid: XBRIDGE v2 is system-injected`. On the destination (from a lock): credits `DEST_ADDRESS` and raises this chain's XCHAIN supply, creating the token row on first use if it does not yet exist. On BTC (from a burn): debits `ADDRESS.BRIDGE_` and credits the named BTC address; an escrow that would go negative is refused outright and logged once, applying nothing. Pays no fee. +- **Version 3 (lock a token).** User-broadcast only on the token's own origin chain. The `TICK` guards run in this order, and the first one that fails is the verdict: an origin-rooted name (`.`) is a bridged copy, not a native row, so it is refused with `invalid: TICK (not native here)` and burned with v4 instead; the GAS tick keeps v0 (`invalid: TICK (use XBRIDGE v0)`, matched case-insensitively because every ticker lookup is); a native name carrying a dot is `invalid: TICK (subassets are not bridgeable yet)`, lifted in a later milestone; a name too long to carry this chain's root and a dot once rooted is `invalid: TICK (too long to bridge)`; and finally the row's [bridge opt-in](./issue.md#version-7---bridge-opt-in) must name `DEST_COIN` (`invalid: TICK (not bridgeable to DEST_COIN)`). `DEST_ADDRESS` and `AMOUNT` validate as v0. A sleeping or list-blocked source cannot lock. Debits the source and credits the destination's escrow; stamps the row's `DECIMALS` and confirmation depth onto the pending transfer. Fee: `XBRIDGE_BASE`. +- **Version 4 (burn a bridged token).** User-broadcast only against a bridged row (`.`) on the chain that received it; a native row, or a rooted name this chain's keyless bridge role address does not own, is refused with `invalid: TICK (not bridged)`. `ORIGIN_ADDRESS` is validated against the tick's origin chain, not against this one, and a failure is `invalid: ORIGIN_ADDRESS`. Debits the source and lowers the bridged row's supply. Fee: `XBRIDGE_BASE`. +- **Version 5 (settle a token).** System-injected only, the v2 lifecycle with a tick; a broadcast v5 is refused with `invalid: XBRIDGE v5 is system-injected`. On the destination (from a lock): creates the origin-rooted root and child rows on first use, credits the destination address, raises the bridged row's supply. On the origin (from a burn): debits the destination coin's escrow, credits the named address on the origin chain; a negative escrow is refused the same way as v2. Injected legs bypass sleep and list checks the way `CROSS_SETTLE` does, so a sleeping origin still releases escrow to a burner. Pays no fee. +- A source `SLEEP`, `ALLOW_LIST` or `BLOCK_LIST` on the token blocks a new v0/v3 lock and v1/v4 burn the same way it blocks a `SEND`; it never blocks the injected v2/v5 settle leg, and it has no effect on a copy already bridged elsewhere. + +## Trust model, stated plainly + +The record a lock or burn produces is signed by the hub's `cross_chain` federation and mirrored to every indexer; the destination applies it once quorum-verified against the mirrored validator set. Milestone 1 is therefore a **hub-trusted mint**: a compromised hub can supply both the transfer record and the roster that verifies it. This is the same trust boundary the cross-chain DEX already runs on, stated here rather than left implicit. A destination-side check against the origin chain's own signed state (a checkpoint cross-check) closes that boundary before this action is armed on mainnet; see [Cross-Chain Bridge](../xchain-bridge.md#trust-model) for the full statement and its status. + +## Reorgs and finality + +If the source lock or burn is reorged out before the federation signs it, no transfer record is ever produced. If a finalized record's source is reorged out before it applies, the federation retracts it and the destination never applies it. **Once a mint or release has applied on the destination, it is final**: the platform has no destination-side unwind, so a later reorg of the *source* leg leaves the applied credit in place and shows up instead as a reported deficit between the escrow and the shadow supply it backs (`getbridgeinvariant`, [Cross-Chain Bridge](../xchain-bridge.md#the-supply-invariant)), which the confirmation depth exists to make expensive to reach. A plain credit landed on an escrow address with no matching transfer record (a stray `SEND`, `AIRDROP`, or DEX fill) is the opposite case, a harmless surplus, never a deficit. + +## Notes +- The escrow addresses (`ADDRESS.BRIDGE_`, one per non-origin coin per network) are ordinary keyless protocol role addresses; nobody holds a key for them, so a lock's credit and a burn's release both settle as plain ledger balance moves with no new escrow table. +- A general token's bridged copy is named under its origin chain's root, `.` (`BTC.PEPECASH`, `DOGE.FUFU`), never a bare name a squatter could pre-register on the destination. See [Token Bridge](../token-bridge.md) for the naming rule and its two limits (dotted origin names, tick length) in milestone 1. +- Bridging a token is opt-in and owner-controlled: see [ISSUE format `7`](./issue.md#version-7---bridge-opt-in) for `BRIDGE_CHAINS`, `MIN_DEPTH`, and `LOCK_BRIDGE`. +- `XBRIDGE` is unrelated to the [Cross-Chain DEX](../cross-chain-dex.md) (`ORDER`/`SWAP` matched across chains through the same federation): the DEX trades a token that already exists on both sides, while `XBRIDGE` is how a token gets a shadow balance on a chain it was never issued on in the first place. +- `getbridgeinvariant` (hub, open read) and `getpendingbridgetransfers` / `getbridgetransfer` (indexer, open read) surface the escrow-versus-supply invariant and in-flight transfers; see [Cross-Chain Bridge](../xchain-bridge.md#reads) for their shape. + +--- + +**Copyright © 2025–2026 Dankest, LLC** + +**Based on XChain Platform by Dankest, LLC – https://dankest.llc** + +Licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0-or-later) +with a commercial license available for proprietary use. + +You may use, modify, and distribute this material under the terms of the License. +See [LICENSE](../../LICENSE.md) and [NOTICE](../../NOTICE.md) for full terms. +See the [licensing overview](https://docs.xchain.io/legal/LICENSING.html). diff --git a/protocol/actions/xcall.md b/protocol/actions/xcall.md index 6c253cfe..fc0e66f8 100644 --- a/protocol/actions/xcall.md +++ b/protocol/actions/xcall.md @@ -73,7 +73,11 @@ Unused target-side gas is not refunded in v1. The callback runs against the fixe Dispatch: XCALL|DISPATCH|call_id|snapshot_block|network|source_chain|source_action_index|source_contract_index|target_chain|target_contract_index|method|sha256(params_json)|gas_limit|cross_hops|effective_time Result: XCALL|RESULT|call_id|snapshot_block|network|target_chain|result_status|sha256(return_payload_b64)|effective_time ``` - Variable-length fields enter as a `sha256` digest so the canonical string stays fixed-arity and `|`-safe + Variable-length fields enter as a `sha256` digest so the canonical string stays fixed-arity and `|`-safe. At/above `EQUIV_HEADER_ACTIVATION` (resolved on the row's `snapshot_block`, so every chain and the hub flip on the same BTC anchor) each string above is wrapped in the uniform equivocation header with `TAG=XCALL`, `VIEW = finalizing_view` (`0` when null) and a phase-specific `ROUND_ID = sha256('XCALLROUND||' + call_id)`, `` being `dispatch` or `result`, so the two legs of one `call_id` never share a round key: + ``` + EQUIV|XCALL|||| + ``` + Below the flag-day the bare bytes are signed. Signing the bare form at/above activation produces signatures every indexer drops during quorum verification (see [Protocol Activation](../protocol-activation.md)) - Target-side execution (`XEXEC`) is an internal action: a depth-0 `EXECUTE` under `gasCeiling = GAS_LIMIT`, with a synthetic chain/network-namespaced `TX_HASH`, the `crossCallable` allowlist enforced, and its own savepoint. A failed run rolls its state back and that failure becomes the relayed result. It is idempotent and reorg-safe via `cross_chain_call_executions` - Lifecycle (source-chain request status): a request starts `pending`. The federation waits for source-chain confirmation depth, then signs the dispatch row; the target chain verifies signatures and injects `XEXEC` at the first block at or after `effective_time`, then the federation waits for target-chain depth and signs the result row. The request becomes `completed` when a verified result arrives, or `expired` once `DEADLINE_BLOCKS` passes with no result @@ -91,7 +95,7 @@ Unused target-side gas is not refunded in v1. The callback runs against the fixe Completed --> [*] Expired --> [*] ``` -- Recoverability: the Version `0` request is reproducible from a pure chain parse, and both relay phases are included in the ANCHOR v1 archive and rebuilt and signature-verified by `xchain-indexer/src/recovery.js`, so a from-genesis reindex re-derives identical injected executions and callbacks +- Recoverability: the Version `0` request is reproducible from a pure chain parse, and both relay phases are included in the ANCHOR v1 archive and rebuilt and signature-verified by `xchain-indexer/bin/recovery.js`, so a from-genesis reindex re-derives identical injected executions and callbacks --- diff --git a/protocol/constants.js b/protocol/constants.js index 3981294e..8778f423 100644 --- a/protocol/constants.js +++ b/protocol/constants.js @@ -197,6 +197,38 @@ const ATTEST_MAX_EXPIRIES_PER_BLOCK = 25; // mainnet block settles the full effective backlog, exactly as it always has. const CROSS_SETTLE_MAX_PER_BLOCK = 25; +// ── Cross-chain bridge (protocol/xchain-bridge.md) ────────────────────────── +// Finalized bridge transfer records the XBRIDGE settle pass applies per DESTINATION +// CHAIN per block. Overflow carries forward in (snapshot_block, transfer_id) order +// and is never dropped, the XCALL and CROSS_SETTLE discipline. +// +// UNGATED, unlike CROSS_SETTLE_MAX_PER_BLOCK above: that cap re-sliced history a +// chain had already indexed and therefore needed its own flag day, while this one +// ships inside XCHAIN_BRIDGE_ACTIVATION, below which no chain has ever applied an +// XBRIDGE settle leg. There is no history for it to reinterpret. +// +// Canonical authority for xchain-indexer/src/protocol/constants.js. +const XBRIDGE_MAX_PER_BLOCK = 25; + +// ── Token-policy inheritance (protocol/token-bridge.md) ───────────────────── +// Finalized policy snapshots applied per block per chain, at the head of the +// XBRIDGE pass and before any in-leg at that block. Lower than the transfer cap +// because one snapshot is up to six injected actions, each rewriting a full list +// membership, where one transfer is a single credit. Overflow carries forward in +// (snapshot_block, snapshot_id) order across ticks and policy_seq within a tick. +const XPOLICY_MAX_PER_BLOCK = 5; + +// Ceiling on the membership of a list a bridged token may carry. No cap existed +// before: LIST items are variadic, the only bound was MAX_ACTION_DATA_LENGTH on one +// action, and every edit persists a complete membership, so a list grew without +// limit across edits. Every snapshot carries the FULL membership and every +// destination rewrites it on apply, so the origin's list length is write +// amplification on every chain holding a copy. +// +// Refused at ISSUE format 7 opt-in and declined at the hub's signing step; never a +// hash input, so a later flag day can raise it. +const XPOLICY_MAX_MEMBERS = 10000; + // ── Token-gated content (PC-29) ───────────────────────────────────────────── // Fixed fractional scale for comparing FILE.GATE_MIN_AMOUNT thresholds against a // holder's balance. The wallet scales both sides to this many fractional digits @@ -1458,6 +1490,18 @@ const TRAIN_ACTIVATION = { // earlier rule set to migrate from: the launch binary IS the first rule set, and // a floor above genesis would leave the pre-floor range resolving to nothing. '1.0.0': { mainnet: 0, testnet: 0, regtest: 0 }, + // The XCHAIN bridge rule set, armed at the v0.19.0 cut. Mainnet holds the house + // sentinel: the mainnet arm is the next milestone and nothing arms there before the + // checkpoint cross-check lands, so no mainnet node ever reaches this boundary. + // Testnet: SIZED 2026-09-16, re-cut 16:33Z, from chain_tip TBTC 152,716 + 71 blocks, + // which is ceil(10 h / 508.8 s per block measured over the preceding 99 blocks), about + // 10.0 h. The first sizing (11:53Z, 152,716 itself) was overrun by the chain while the + // cut waited on the e2e matrix, so the boundary was re-cut from the new tip with a lead + // long enough to cover that wait plus the roll. That is the rolling-upgrade window the + // fleet roll must finish inside (over 6x the 90 minute roll budget), and every testnet + // bridge height below sits above it on the same BTC clock, so a node lacking this rule + // set halts before it can grade a bridge action. + '0.19.0': { mainnet: 9999999999, testnet: 152787, regtest: 0 }, }; // STAKE v1 signing-key REUSE flag day, keyed on the processing chain's OWN @@ -1531,6 +1575,358 @@ const SWEEP_ZERO_LEG_ACTIVATION = { regtest: 0, // genesis-active so the e2e venue exercises the armed rule }; +// XCHAIN bridge flag day, keyed ':' on the block_index of the chain being +// parsed, with the bare network key as the fallback. Canonical authority for the local copy +// in xchain-indexer/src/xchain_bridge_activation.js, which carries the full rationale; the +// indexer's activation-constant parity suite holds the two value-identical, and a one-sided +// edit forks the bridge at the boundary. +// +// At and above a chain's height, XBRIDGE v0 (lock on the origin chain) and v1 (burn on a +// destination chain) are legal and the federation signs transfer records that the +// mirror-injected v2 settle leg applies. Below it a broadcast v0 or v1 is +// 'invalid: XBRIDGE before activation' and no v2 is injected, so pre-activation block +// hashes are unchanged on every chain. +// +// PER COIN, not one height per network: the bridge arms on three chains at once, and their +// tips differ by orders of magnitude (TBTC about 152,110, TLTC about 4,884,193, TDOGE about +// 67,889,993 measured 2026-09-12), so one testnet number is already passed on two of them at +// boot and unreachable on the third. +// +// Mainnet is the house sentinel on every key: bridge milestone 1 is a hub-trusted mint (a +// compromised hub supplies both the transfer record and the roster that verifies it), so +// nothing arms on mainnet before the ANCHOR-checkpoint cross-check of the lock is built. +// Testnet is SIZED AT THE v0.19.0 CUT, one dated instant per chain, from the three chain tips +// and their last-99-block cadences read in one sitting (2026-09-16 16:33Z, the re-cut after +// the 11:53Z sizing was overrun: TBTC 152,716 at 508.8 s per block, TLTC 4,887,644 at 141.8 s, +// TDOGE 67,900,748 at 27.4 s). The two destinations arm ceil(10 h / cadence) blocks above +// their tips and BTC, the origin of the v0 lock, arms ceil(30 h / cadence) above its tip and +// so last in wall clock, because the lock +// handler never checks the destination's own activation; the 3x gap is the band a destination +// cadence can slow by before that ordering breaks. All three sit above the TRAIN_ACTIVATION +// 0.19.0 testnet boundary on the BTC clock. Regtest is genesis-active so the e2e venue +// exercises the armed rule. +const XCHAIN_BRIDGE_ACTIVATION = { + 'BTC:mainnet': 9999999999, + 'LTC:mainnet': 9999999999, + 'DOGE:mainnet': 9999999999, + mainnet: 9999999999, // fallback for a coin with no entry above + 'BTC:testnet': 152929, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 152,716 + 213 (30 h at 508.8 s/blk), about 30.1 h, the origin, last + 'LTC:testnet': 4887898, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 4,887,644 + 254 (10 h at 141.8 s/blk), about 10.0 h + 'DOGE:testnet': 67902062, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 67,900,748 + 1314 (10 h at 27.4 s/blk), about 10.0 h + testnet: 9999999999, // fallback: a testnet coin with no entry above stays dark + regtest: 0, // genesis-active so the e2e rail exercises the armed rule +}; + +// General token-bridge flag day (XBRIDGE v3/v4/v5 and ISSUE format 7), keyed the same way. +// Canonical authority for xchain-indexer/src/token_bridge_activation.js. +// +// ORDERING INVARIANT, asserted by the indexer's parity suite over this map: +// TOKEN_BRIDGE_ACTIVATION >= XCHAIN_BRIDGE_ACTIVATION per network. The general formats ride +// the same hub engine, the same mirrored transfer table and the same settle pass as +// XCHAIN's, so a train that armed v3 without the XCHAIN bridge behind it would admit locks +// that nothing can ever finalize and that no burn can ever return. +// +// Testnet is NOT armed alongside the XCHAIN bridge: no third-party token can be offered on +// a hub-trusted mint, so this gate waits on the checkpoint cross-check landing on that +// network. Regtest is genesis-active. +const TOKEN_BRIDGE_ACTIVATION = { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}; + +// Token-policy inheritance flag day, keyed the same way. Canonical authority for +// xchain-indexer/src/token_policy_activation.js, which carries the full rationale. +// +// At and above a network's height a token's origin-row policy binds every bridged copy: +// the milestone-1 mutual exclusion in ISSUE lifts, LIST address items validate against any +// supported coin at this network (and db.isAddressSleeping judges a foreign-format address +// instead of skipping it), the hub signs XPOLICY snapshots, and the destination applies them +// as the bridged row's own lists and sleep state. Below it every milestone-1 verdict stands +// and no snapshot is signed, so pre-activation block hashes are unchanged on every chain. +// +// TWO ORDERING INVARIANTS, asserted by the indexer's parity suite over this map: +// TOKEN_POLICY_INHERITANCE_ACTIVATION >= TOKEN_BRIDGE_ACTIVATION per network. Inheritance +// has nothing to inherit onto until bridged copies can exist. +// TOKEN_POLICY_INHERITANCE_ACTIVATION >= LIST_EDIT_RESOLUTION_ACTIVATION per chain and +// network (that map is indexer-local: BTC 963000, LTC 3162000, DOGE 6338000 on mainnet, +// 0 on testnet and regtest). The snapshot read resolves a list AS OF origin_block by +// walking the edit chain; below that gate the legacy create-index read runs, and the +// membership the federation signs would not be the membership the chain enforced. +// +// Both public networks hold at the house sentinel until the train that arms them sizes a +// dated instant. Regtest is genesis-active. +const TOKEN_POLICY_INHERITANCE_ACTIVATION = { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}; + +// LIST owner-check flag day, keyed on the block_index of the chain being parsed. +// Canonical authority for xchain-indexer/src/list_owner_activation.js. +// +// At and above a network's height a LIST format 1 whose source is not the address that +// created the list it names is 'invalid: LIST_ACTION_INDEX (not owner)'. Below it the edit +// is judged exactly as it always has been. +// +// It is a flag day rather than an unconditional fix because the check RE-VERDICTS indexed +// history: third-party edits are valid today and list_items is a hashed DERIVED table, so +// refusing them on replay would move block hashes on a live chain. The replay corpus being +// hash-identical below the height is the hard gate on this change. +const LIST_OWNER_ACTIVATION = { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}; + +// Tick-namespace flag day (R8), keyed on the block_index of the chain being +// parsed. Canonical authority for xchain-indexer/src/tick_namespace_activation.js, which +// carries the full rationale. +// +// At and above a network's height two rules bind in the ISSUE handler, beside the reserved +// guard: a top-level ISSUE that would CREATE a tick shorter than four characters is +// 'invalid: TICK (length)' (creation only, so every short token already issued keeps its +// owner and its admin surface), and every ticker in RESERVED_FUTURE_ROOTS below is +// 'invalid: TICK (reserved)'. Together they hold the root of a chain XChain integrates +// later free on every ledger that exists by then. +// +// Its OWN height rather than TOKEN_BRIDGE_ACTIVATION's: the bridge arms only after the base +// spec's D2 checkpoint cross-check, and the namespace has to close before anyone squats. +// Activation-keyed rather than unconditional because the reserved and length checks run +// ahead of the fee and budget checks, so a mined ISSUE of a listed name that is refused +// today on fee would flip its verdict string on replay. +// +// Both public networks hold at the house sentinel until the train that arms them sizes a +// dated instant; mainnet additionally waits on a replica measurement of zero mined ISSUEs +// of a short or listed name, valid or invalid. Regtest is genesis-active. +const TICK_NAMESPACE_ACTIVATION = { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}; + +// Chain tickers held free for chains XChain has not integrated yet, refused as +// 'invalid: TICK (reserved)' at and above TICK_NAMESPACE_ACTIVATION. Canonical authority +// for xchain-indexer/src/reservedRoots.js; the indexer's activation-constant parity suite +// holds the two list-identical, ORDER INCLUDED. +// +// Chains only: no tokens that live on someone else's chain, and no spelled-out names. +// Reserving a root reserves its whole ROOT.* subtree through the subasset parent gate, so +// one entry per chain is the entire cost. When a chain is integrated its ticker moves from +// here to the coin registry; both refuse identically, so nothing re-verdicts on the move. +// +// The first 47 were measured free on every live chain and in both genesis manifests on +// 2026-09-11 (120 names, 720 explorer probes); the last 6 are squatted in the mainnet +// genesis manifests and leave through a separate manifest edit, so for those this guard +// blocks only NEW issuance. +const RESERVED_FUTURE_ROOTS = Object.freeze([ + 'ADA', 'ALGO', 'APT', 'ARB', 'ATOM', 'AVAX', 'BCH', 'BNB', 'BSV', 'BTG', + 'CRO', 'DGB', 'DOT', 'EOS', 'ETC', 'ETH', 'FIL', 'FIRO', 'GRS', 'ICP', + 'INJ', 'KAS', 'MNT', 'NEO', 'NMC', 'OP', 'POL', 'PPC', 'RVN', 'SEI', + 'SOL', 'STX', 'SUI', 'TIA', 'TON', 'TRX', 'VET', 'VTC', 'XCP', 'XDP', + 'XEC', 'XLM', 'XMR', 'XRP', 'XTZ', 'ZEC', 'ZK', + 'BASE', 'DASH', 'HBAR', 'HOOD', 'HYPE', 'NEAR', +]); + +// --------------------------------------------------------------------------- +// The time-keyed mirror barrier family: admission by height +// --------------------------------------------------------------------------- +// +// Eleven barrier hold points in the indexer block loop are keyed on the block's own +// protocol timestamp, and ten of them hold the whole block loop for that stamp's full +// distance plus their grace. Bitcoin consensus accepts a block stamped up to 7200 s +// ahead, so a perfectly VALID block stalls a hub-connected indexer for over two hours +// while /status reports the healthy 'future_block_wait' verdict throughout. +// +// The reason no grace can fix this is a theorem, not a tuning problem. A mirrored row +// binds at block B when its signed effective_time <= t(B), and a producer may mint such +// a row at any wall-clock instant up to t(B) - RELAY_MIN_FUTURE_S. So the SET of rows +// binding at B is not determined until wall clock reaches that instant, and any correct +// barrier under that binding rule must wait for it whatever its grace is. +// +// So the binding rule changes: every mirrored row carries a signed ADMISSION HEIGHT per +// chain that reads it, a row is readable at B on chain C only when admit_blocks[C] <= B, +// and each barrier certifies completeness by comparing a per-table per-chain HEIGHT +// watermark against B rather than a clock against t(B). Heights do not move with stamps, +// so a block stamped 7200 s ahead is height B like any other. +// +// Kept value-identical to the local copies in xchain-{hub,indexer}/src/mirror_admission_activation.js +// by the activation-constants parity suite. + +// ADMIT_MARGIN_BLOCKS: how far ahead of the producer's observed admission tip a row is +// stamped, in BLOCKS of each chain in its map. This is not a new number: producers already +// size their forward margin as DEFAULT_RELAY_MARGIN_BLOCKS (4) blocks of the gating chain +// and then CONVERT it to seconds. On the admission axis the conversion is simply deleted, +// which is why an unknown chain needs no nominal block interval here at all. +// +// A default plus three overrides, never a six-key map: +// attest responses 1, because their 120 s forward margin was chosen to be as SHORT +// as the propagation window allows and converting it back to +// blocks would silently lengthen it. +// oracle prices 1, same reason; effective_at stays the ECONOMIC filter and the +// 24 h lock window, while admission is what the barrier certifies. +// anchor-reward attests 144, the existing ANCHOR_REWARD_MIRROR_MATURITY, already frozen +// fleet-wide. No producer change: this rail's admission height +// already exists. +const ADMIT_MARGIN_BLOCKS = Object.freeze({ + default: 4, + attestation_responses: 1, + oracle_prices: 1, + anchor_reward_attestations: ANCHOR_REWARD_MIRROR_MATURITY, +}); + +// ADMIT_MIN_FUTURE_BLOCKS: a follower refuses any proposal whose admission height for a +// chain is not strictly ahead of that chain's own tip. A row may never be admissible at a +// block that already exists, or a producer could backdate a row into a block its peers +// have already committed. +const ADMIT_MIN_FUTURE_BLOCKS = 1; + +// ADMIT_MAX_FUTURE_BLOCKS: the follower's upper bound, PER CHAIN, sized so each chain's +// height window spans the same 3600 s the existing absolute effective_time ceiling already +// allows: ceil(3600 / nominal block interval), with an unknown chain taking BTC's. +// +// A flat block count here would be a silent tightening. Six blocks is an hour on BTC but +// six minutes on DOGE, so a flat [tip + 1, tip + 6] would collapse the federation's +// clock-skew tolerance from 3600 s to 360 s on DOGE and start refusing honest rows between +// hubs whose tips differ by three blocks. +// +// The absolute TIME bounds on effective_time are NOT retired by this. A follower refuses on +// BOTH axes, so a hub with a broken clock and a hub with a wrong tip are each caught by the +// axis that can actually see them. +const ADMIT_MAX_FUTURE_BLOCKS = Object.freeze({ + BTC: 6, // 600 s nominal + LTC: 24, // 150 s nominal + DOGE: 60, // 60 s nominal + default: 6, // an unrecognised chain takes BTC's interval, as blockIntervalS already does +}); + +const MIRROR_ADMISSION_REGTEST_ENV = 'XC_MIRROR_ADMISSION_ACTIVATION'; +const MIRROR_ADMISSION_REGTEST_ARMED_HEIGHT = 0; + +/** + * Resolve the regtest admission activation from the environment, in the ROLLCALL shape. + * + * The armed form resolves to 0 so a drill block sits ABOVE the armed node's threshold and + * BELOW an inert node's null, which is the only per-process arming seam the codebase has and + * is what lets BF5 put an armed and an inert indexer on ONE venue and show them binding the + * same row at different blocks. Fails closed: anything unrecognised leaves regtest INERT and + * says so, rather than stamping NaN into a height comparison. + * + * @param {object} env the process environment, or a stand-in + * @returns {number|null} + */ +function resolveMirrorAdmissionRegtest(env){ + let raw = (env || {})[MIRROR_ADMISSION_REGTEST_ENV]; + if(raw === undefined || raw === null) return null; + let s = String(raw).trim().toLowerCase(); + if(s === '' || s === 'off' || s === 'inert' || s === 'false' || s === 'no' || s === 'none') return null; + if(s === 'armed' || s === 'genesis' || s === 'on' || s === 'true' || s === 'yes') + return MIRROR_ADMISSION_REGTEST_ARMED_HEIGHT; + if(/^\d+$/.test(s)){ + let h = parseInt(s, 10); + if(Number.isFinite(h) && h >= 0) return h; + } + console.error('MIRROR ADMISSION: ignoring ' + MIRROR_ADMISSION_REGTEST_ENV + '=' + + JSON.stringify(String(raw)) + '; regtest stays INERT. Expected a non-negative ' + + 'height, "armed", or "off".'); + return null; +} + +// MIRROR_ADMISSION_ACTIVATION (the PRODUCER map): the height at/above which a hub stamps an +// admission map into the signed canonical and refuses to finalize a row it cannot stamp. +// MIRROR_ADMISSION_CONSUMER_ACTIVATION (the CONSUMER map): the height at/above which an +// indexer reads that map instead of binding on effective_time <= t(B). +// +// TWO maps, one module, with an ordering rule that is the whole point: every producer height +// is sized strictly BELOW its consumer height, so no row is ever produced legacy and read +// modern. Get that backwards and a consumer above its height reads an admission column that +// the producer below its own height never wrote, and binds nothing. +// +// Keyed by (coin, network), not by network alone. A single per-network height cannot arm a +// family that binds on every chain: one number is an LTC height on an LTC indexer and a BTC +// height on a BTC indexer, so the two legs of one cross-chain match would cross the flag day +// at unrelated instants. The 'COIN:network' key shape is established precedent, not new. +// +// FAIL-CLOSED on every read: a null threshold, a non-finite height or an unknown key reads +// INERT, and INERT is today's behaviour byte for byte. Every read MUST go through a +// Number.isFinite guard; a bare `height >= MAP[key]` arms a null key at height 0, because +// `0 >= null` is true in JavaScript. +// +// Mainnet is null under the 2026-08-29 write hold. Testnet is sized at the release cut from +// the measured tip plus the roll window plus slack, per key, because testnet carries live +// public ledgers and the height is the whole protection. The v7 HUB_SCHEMA_VERSION roll +// completes BEFORE any network's activation height: the heights map rides frames that carry +// no schema_version, so a v7 indexer above the activation against a v6 hub would see no +// heights at all and defer forever under the fail-closed rule. +const MIRROR_ADMISSION_ACTIVATION = Object.freeze({ + 'BTC:mainnet': null, // INERT under the 2026-08-29 mainnet write hold + 'LTC:mainnet': null, + 'DOGE:mainnet': null, + 'BTC:testnet': null, // SIZED AT THE CUT: tip + roll window + slack, strictly below the consumer height + 'LTC:testnet': null, + 'DOGE:testnet': null, + 'BTC:regtest': resolveMirrorAdmissionRegtest(process.env), + 'LTC:regtest': resolveMirrorAdmissionRegtest(process.env), + 'DOGE:regtest': resolveMirrorAdmissionRegtest(process.env), +}); + +const MIRROR_ADMISSION_CONSUMER_ACTIVATION = Object.freeze({ + 'BTC:mainnet': null, + 'LTC:mainnet': null, + 'DOGE:mainnet': null, + 'BTC:testnet': null, // SIZED AT THE CUT, strictly ABOVE the producer height for the same key + 'LTC:testnet': null, + 'DOGE:testnet': null, + 'BTC:regtest': resolveMirrorAdmissionRegtest(process.env), + 'LTC:regtest': resolveMirrorAdmissionRegtest(process.env), + 'DOGE:regtest': resolveMirrorAdmissionRegtest(process.env), +}); + +// --------------------------------------------------------------------------- +// The anchor-attest barrier's maturity horizon (the family's parent item) +// --------------------------------------------------------------------------- +// +// The anchor-attest member is the one place the family's height rule is measurably WORSE +// than the clock it replaces, because the hub cannot advance that rail's height watermark +// past a snapshot whose deferred reward-attest entry is still queued, and that queue's TTL +// is 6 h. So this member keeps its own maturity-horizon bound BESIDE the height rule rather +// than being superseded by it, and the min() below is what guarantees the barrier can only +// ever open EARLIER than it does today, never later. +// +// The derive pass at block B reads exactly the rows with snapshot_block <= B - 144. Every +// such row was written no later than time(snapshot_block) + the hub's arrival lag, so a +// watermark at or past horizonTime + ANCHOR_ATTEST_ARRIVAL_MARGIN_S certifies the node holds +// every row that pass will read, which is the completeness property in full. +// +// Kept value-identical to the local copies in xchain-{hub,indexer}/src/anchor_reward_activation.js +// by the activation-constants parity suite. + +// ANCHOR_ATTEST_ARRIVAL_MARGIN_S: sized against the hub's whole MEASURED write-lag envelope, +// not the DOGE burial alone. The envelope is about 15 h: up to 6 BTC blocks of checkpoint age +// at flush (about 1 h), the publisher's deferred-write queue TTL of 6 h, a receiver hub's +// re-proof through that SAME queue for up to 6 h more, and about 2 h of raw-stamp skew on the +// networks that are off median-time-past. +// +// 64800 s covers that envelope with 3 h of headroom and still opens 6 h before a nominal +// 144-block span, so a +2 h stamp is absorbed entirely. The earlier 21600 s figure was sized +// on the DOGE burial alone and sits BELOW the publisher's own queue TTL, so it was re-put and +// corrected by measurement. A 144-block stretch shorter than 18 h is a three-sigma event and +// falls back to today's wait through the min(), which is the right way for a fail-closed gate +// to fail. +const ANCHOR_ATTEST_ARRIVAL_MARGIN_S = 64800; // 18 h + +// ANCHOR_ATTEST_BARRIER_ACTIVATION: per NETWORK, not per (coin, network), because this member +// is BTC-only by its call-site guard and a second key would be dead weight. Nothing hashed +// moves across this height: two nodes on either side derive the identical set at the identical +// height and differ only in WHEN they get there. The height exists because a rolling deploy +// would otherwise leave the early-opening node alone in carrying a weaker completeness +// guarantee, and one map removes that window. +const ANCHOR_ATTEST_BARRIER_ACTIVATION = Object.freeze({ + mainnet: null, // INERT under the 2026-08-29 mainnet write hold + testnet: null, // SIZED AT THE CUT from the measured tip plus the roll window + regtest: resolveMirrorAdmissionRegtest(process.env), // shares the family's arming seam so one venue lever arms both +}); + module.exports = { MAX_ACTION_DATA_LENGTH, ENVELOPE_MAX_PAYLOAD, @@ -1554,6 +1950,9 @@ module.exports = { XCALL_RESULT_ORPHAN_GRACE_SECONDS, ATTEST_MAX_EXPIRIES_PER_BLOCK, CROSS_SETTLE_MAX_PER_BLOCK, + XBRIDGE_MAX_PER_BLOCK, + XPOLICY_MAX_PER_BLOCK, + XPOLICY_MAX_MEMBERS, THRESHOLD_SCALE, STAKE_WEIGHTED_QUORUM_ACTIVATION, EQUIV_HEADER_ACTIVATION, @@ -1568,6 +1967,16 @@ module.exports = { ANCHOR_ACTIVATION, ANCHOR_REWARD_DERIVE_ACTIVATION, ANCHOR_REWARD_MIRROR_MATURITY, + ADMIT_MARGIN_BLOCKS, + ADMIT_MIN_FUTURE_BLOCKS, + ADMIT_MAX_FUTURE_BLOCKS, + MIRROR_ADMISSION_ACTIVATION, + MIRROR_ADMISSION_CONSUMER_ACTIVATION, + MIRROR_ADMISSION_REGTEST_ENV, + MIRROR_ADMISSION_REGTEST_ARMED_HEIGHT, + resolveMirrorAdmissionRegtest, + ANCHOR_ATTEST_ARRIVAL_MARGIN_S, + ANCHOR_ATTEST_BARRIER_ACTIVATION, ROLLCALL_ACTIVATION, ROLLCALL_REGTEST_ARMED_HEIGHT, ROLLCALL_REGTEST_ENV, @@ -1615,4 +2024,10 @@ module.exports = { TRAIN_ACTIVATION, STAKE_KEY_REUSE_ACTIVATION, SWEEP_ZERO_LEG_ACTIVATION, + XCHAIN_BRIDGE_ACTIVATION, + TOKEN_BRIDGE_ACTIVATION, + TOKEN_POLICY_INHERITANCE_ACTIVATION, + LIST_OWNER_ACTIVATION, + TICK_NAMESPACE_ACTIVATION, + RESERVED_FUTURE_ROOTS, }; diff --git a/protocol/contract-abi.md b/protocol/contract-abi.md index 2c6dfe36..f45b54af 100644 --- a/protocol/contract-abi.md +++ b/protocol/contract-abi.md @@ -93,7 +93,7 @@ ordinary. ## Reader behavior (fail-closed) -Reference readers: `contract-introspect.js` in xchain-explorer (served as the +Reference readers: `contract/introspect.js` in xchain-explorer (served as the `abi` field on `GET /{COIN}/api/contract/{idx}`) and `ContractUtils.parseAbi()` in xchain-sdk. Both apply the same rules: diff --git a/protocol/controller-bound-tokens.md b/protocol/controller-bound-tokens.md index b82ed5c5..ede0021b 100644 --- a/protocol/controller-bound-tokens.md +++ b/protocol/controller-bound-tokens.md @@ -262,6 +262,12 @@ There is no royalty-specific mechanism; "royalty" is simply the most common use tightened by the contract's [permissions manifest](#permissions-manifest)) and stores the legs as JSON on `orders.payout_legs` / `swaps.payout_legs`. A malformed or over-cap set **denies** the listing (fail-closed). No `payoutLegs` ⇒ NULL (an ordinary order). + + Two shapes are read leniently rather than denied, and no activation gate changes that + today (none is registered in [Flag-Day Values](./flag-days.md)): a supplied `payoutLegs` + that is not an ARRAY is read as "no legs" and the listing is created with NULL legs, and + a fractional `bps` is accepted at its truncated integer value, which is also the value + the cap is measured against. An empty array means the same as an absent `payoutLegs`. 2. **At match**: `Utility.applyProceedsSplit(tick, proceeds, seller, legs, decimals, cap)` splits each filled order's proceeds, **seller-remainder first, then each leg**, crediting `floor(proceeds × bps / 10000)` (at token precision) to each `to` and the exact remainder diff --git a/protocol/cross-chain-dex.md b/protocol/cross-chain-dex.md index 6aff8a24..69b2d672 100644 --- a/protocol/cross-chain-dex.md +++ b/protocol/cross-chain-dex.md @@ -177,7 +177,7 @@ closes that gap: the federation periodically publishes, on **DOGE only**: This **does not gate settlement** (settlement remains mirror-driven and verifies the `cross_chain` quorum as above); it guarantees that a full parse of the three chains, with no surviving hub -database, can rebuild the match set via the recovery tool (`xchain-indexer/src/recovery.js`) and +database, can rebuild the match set via the recovery tool (`xchain-indexer/bin/recovery.js`) and re-derive identical state. A match retracted after being archived is re-published in a later batch with `status=retracted`; recovery applies latest-status-wins by `batch_seq`. diff --git a/protocol/error-codes.md b/protocol/error-codes.md index b18dfa91..143638e8 100644 --- a/protocol/error-codes.md +++ b/protocol/error-codes.md @@ -73,6 +73,12 @@ Errors are JSON objects: | `VM_QUERY_DISABLED` | 503 | Contract simulation is disabled on this explorer | No: use another instance | | `VM_QUERY_VM_DRIFT` | 503 | Contract simulation is disabled because the deployed VM is not the canonical one | No: operator action | | `VM_MODULE_UNAVAILABLE` | 503 | Contract simulation: the VM module is not available on this host | No: use another instance | +| `INVALID_ADDRESSES` | 400 | Batch address routes (`POST /{COIN}/api/balances`, `POST /{COIN}/api/coinpay_obligations`): the body's `addresses` field is missing, not an array, empty, or holds a non-string entry, and nothing is read. The SDK's client-side pre-flight throws the same code before a request is sent (see [SDK explorer methods](../components/sdk/explorer.md)) | No: fix the request | +| `TOO_MANY_ADDRESSES` | 400 | Batch address routes: more than 20 addresses in one body, counted before duplicates are collapsed, so a body of repeats is refused rather than quietly served | No: send at most 20 per request | +| `INVALID_ADDRESS` | 400 | Batch address routes: one entry is not a well-formed address; the offending entry is echoed (truncated) in the `error` text. Unrelated to the SDK library's own `INVALID_ADDRESS` typed error, which is a separate surface (see [SDK errors](../components/sdk/errors.md)) | No | +| `READ_FAILED` | 200 (per-address) | Batch address routes only: the fallback code inside a per-address `error` object when the inner read failed without supplying a code of its own. The whole-request status is still 200; branch on the entry's own `status` field | Depends on the entry's `status` | + +The batch address routes fail at two layers, and a client has to read both. A whole-request refusal carries an HTTP 400 with one of the three codes above, or the coin gate's 503 (`COIN_NOT_AVAILABLE` / `COIN_DATA_STALE`), which answers the entire batch rather than being buried per address. Otherwise the response is HTTP 200 and each address may carry an `error` object of `{ code, error, status }`, where `code` is whatever the inner read returned, or `READ_FAILED` when it returned none. A 200 therefore does not mean every address succeeded. Errors on the Explorer WebSocket channel (`INVALID_CHANNEL`, `INVALID_TYPE`, `INVALID_ACTION`, `INVALID_PARAMS`, `SUBSCRIPTION_LIMIT`) are a separate surface with its own message shape; they are documented in [Explorer WebSocket](../components/explorer/websocket.md), not here. diff --git a/protocol/flag-days.md b/protocol/flag-days.md index b69c241d..5e97ced9 100644 --- a/protocol/flag-days.md +++ b/protocol/flag-days.md @@ -4,9 +4,9 @@ # Flag-Day Values -**This page is generated** from `xchain-indexer/src/protocol_changes.js` and the -time-keyed activation modules beside it. Do not edit it by hand: run -`node bin/generate-flag-days.js` from the repository root and commit the result. +**This page is generated** from `xchain-indexer/src/protocol_changes.js`, its part files +under `src/protocol_changes/`, and the time-keyed activation modules beside them. Do not +edit it by hand: run `node bin/generate-flag-days.js` from the repository root and commit the result. Every other page in this documentation set names the **gate** and links here instead of quoting a date, because a flag-day value is not a fact about the @@ -30,58 +30,58 @@ simultaneously on Bitcoin, Litecoin, and Dogecoin. 5 gates do not ride it and carry a date of its own: `BATCH_ISSUANCE_LIMITS` at 2026-08-16 00:00:00 UTC, `CONTRACT_DELEGATION_MATERIALIZE` at 2026-09-15 00:00:00 UTC, `DISPENSER_ORACLE_PER_TOKEN_PRICE` at 2026-09-15 00:00:00 UTC, `CROSS_CHAIN_ROYALTY` at 2027-01-01 00:00:00 UTC, `REST_PATTERN_METER` at 2027-01-01 00:00:00 UTC. Each carries the reason it is armed separately in its registration comment, in the file the **Declared in** column names below. For how a gate is evaluated and what happens to a node that misses one, see [Protocol Activation](./protocol-activation.md). -**One gate is UNARMED on mainnet** (`UNCAPPED_MAX_SUPPLY_ZERO`): each parks the sentinel rather than an instant, so mainnet has **never** run the post-activation behavior and will not until an operator names a date. They carry no row in the table below, because publishing the sentinel as a flag day would put a commitment on this page that nobody made. Each names its reason in its registration comment in `protocol_changes.js`. This note covers the registry only; a sibling `*_activation.js` module can park a mainnet sentinel too, and those are not enumerated here. +**One gate is UNARMED on mainnet** (`UNCAPPED_MAX_SUPPLY_ZERO`): each parks the sentinel rather than an instant, so mainnet has **never** run the post-activation behavior and will not until an operator names a date. They carry no row in the table below, because publishing the sentinel as a flag day would put a commitment on this page that nobody made. Each names its reason in its registration comment under `xchain-indexer/src/protocol_changes/`. This note covers the registry only; a sibling `*_activation.js` module can park a mainnet sentinel too, and those are not enumerated here. **Testnet and regtest are genesis-active** for the time-keyed gates: they carry threshold `0`, so a testnet or regtest stack has always run the -post-activation behavior. 4 gates are the exception: `ISSUE_INHERITED_MINT_WINDOW` arms testnet at `1787961600` (2026-08-29 00:00:00 UTC), `DEPLOY_DEFERRED_ASSEMBLY` arms testnet at `1788868800` (2026-09-08 12:00:00 UTC), `CONTRACT_META_REQUIRED` arms testnet at `1789257600` (2026-09-13 00:00:00 UTC), `UNIFIED_FEES_SWEEP_CALLBACK` arms testnet at `1790812800` (2026-10-01 00:00:00 UTC). The reason it cannot be genesis-active there is written in its registration comment in `protocol_changes.js`. The values on this page are otherwise mainnet values only. +post-activation behavior. 4 gates are the exception: `ISSUE_INHERITED_MINT_WINDOW` arms testnet at `1787961600` (2026-08-29 00:00:00 UTC), `DEPLOY_DEFERRED_ASSEMBLY` arms testnet at `1788868800` (2026-09-08 12:00:00 UTC), `CONTRACT_META_REQUIRED` arms testnet at `1789257600` (2026-09-13 00:00:00 UTC), `UNIFIED_FEES_SWEEP_CALLBACK` arms testnet at `1790812800` (2026-10-01 00:00:00 UTC). The reason it cannot be genesis-active there is written in its registration comment under `xchain-indexer/src/protocol_changes/`. The values on this page are otherwise mainnet values only. ## Mainnet time-keyed gates | Gate | Block time | UTC instant | Rides | Declared in | |---|---|---|---|---| -| `ATTEST_CANONICAL_LOWERCASE_ID` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `ATTEST_RELAY_ORIGIN` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `BATCH_SUBACTION_NORMALIZATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `COINPAY_EXPIRE_TOKEN_AMOUNT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `COINPAY_NATIVE_RECIPROCITY` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `CONTRACT_INDEX_CANONICAL` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `CONTROLLER_GUARD` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `COOLDOWN_BLOCKS_INTEGER` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DELEGATE_REVOKE_NO_REINSERT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DEPLOY_BASE64_CODE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DEPLOY_INIT_STRICT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DEPLOY_SLASH_DEST_ADDRESS_VALID` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DISPENSE_CANCELLING_MATCH_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `dispense_cancelling_match_activation.js` | -| `DISPENSER_CAPS_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `dispenser_caps_activation.js` | -| `DISPENSER_CLOSE_PER_UNIT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `DISPENSER_OWNERSHIP_CANCEL_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `dispenser_ownership_cancel_activation.js` | -| `FIX_OUTPUT_FANOUT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `ISSUANCE_FEE_EMISSION_EXEMPT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `ISSUE_MINT_SUPPLY_CUMULATIVE_CAP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `LEGACY_FEE_NUMERIC_DBHITS` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `LOCK_MAX_SUPPLY_EXACT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `MINT_SELF_MINTED_ONLY` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `NATIVE_FEE_PRICE_TIME_GATE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `PARTIAL_UNSTAKE_COLLECT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `SLEEP_RESPECTS_LOCK_SLEEP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `SYNTH_EXEC_TX_HASH` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `UNSTAKE_CONTRACT_COOLDOWN_STRICT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `UNSTAKE_COOLDOWN_COMPLETION_ACTION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VM_ATTESTATION_GETRESPONSE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VM_BALANCE_TOKENINFO` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VM_BANNED_ASYNC` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VM_LINT_HARDENING` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VOTE_BINDING_MINIMUMS` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VOTE_CALLBACK_TIMELOCK` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VOTE_POLL_TICK_VISIBLE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `VOTE_RESPECTS_SLEEP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `XCALL_RESULT_ORPHAN_RETIREMENT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes.js` | -| `BATCH_ISSUANCE_LIMITS` | `1786838400` | 2026-08-16 00:00:00 UTC | own date | `protocol_changes.js` | -| `CONTRACT_DELEGATION_MATERIALIZE` | `1789430400` | 2026-09-15 00:00:00 UTC | own date | `protocol_changes.js` | -| `DISPENSER_ORACLE_PER_TOKEN_PRICE` | `1789430400` | 2026-09-15 00:00:00 UTC | own date | `protocol_changes.js` | -| `CROSS_CHAIN_ROYALTY` | `1798761600` | 2027-01-01 00:00:00 UTC | own date | `protocol_changes.js` | -| `REST_PATTERN_METER` | `1798761600` | 2027-01-01 00:00:00 UTC | own date | `protocol_changes.js` | +| `ATTEST_CANONICAL_LOWERCASE_ID` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `ATTEST_RELAY_ORIGIN` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `BATCH_SUBACTION_NORMALIZATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `COINPAY_EXPIRE_TOKEN_AMOUNT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `COINPAY_NATIVE_RECIPROCITY` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `CONTRACT_INDEX_CANONICAL` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `CONTROLLER_GUARD` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `COOLDOWN_BLOCKS_INTEGER` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `DELEGATE_REVOKE_NO_REINSERT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `DEPLOY_BASE64_CODE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `DEPLOY_INIT_STRICT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `DEPLOY_SLASH_DEST_ADDRESS_VALID` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `DISPENSE_CANCELLING_MATCH_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/gates_1.js` | +| `DISPENSER_CAPS_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/gates_1.js` | +| `DISPENSER_CLOSE_PER_UNIT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `DISPENSER_OWNERSHIP_CANCEL_ACTIVATION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/gates_1.js` | +| `FIX_OUTPUT_FANOUT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `ISSUANCE_FEE_EMISSION_EXEMPT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `ISSUE_MINT_SUPPLY_CUMULATIVE_CAP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `LEGACY_FEE_NUMERIC_DBHITS` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_4.js` | +| `LOCK_MAX_SUPPLY_EXACT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `MINT_SELF_MINTED_ONLY` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `NATIVE_FEE_PRICE_TIME_GATE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `PARTIAL_UNSTAKE_COLLECT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_4.js` | +| `SLEEP_RESPECTS_LOCK_SLEEP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `SYNTH_EXEC_TX_HASH` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `UNSTAKE_CONTRACT_COOLDOWN_STRICT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `UNSTAKE_COOLDOWN_COMPLETION_ACTION` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_3.js` | +| `VM_ATTESTATION_GETRESPONSE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `VM_BALANCE_TOKENINFO` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `VM_BANNED_ASYNC` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `VM_LINT_HARDENING` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `VOTE_BINDING_MINIMUMS` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `VOTE_CALLBACK_TIMELOCK` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `VOTE_POLL_TICK_VISIBLE` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_2.js` | +| `VOTE_RESPECTS_SLEEP` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_1.js` | +| `XCALL_RESULT_ORPHAN_RETIREMENT` | `1786060800` | 2026-08-07 00:00:00 UTC | contract-era flag day | `protocol_changes/changes_4.js` | +| `BATCH_ISSUANCE_LIMITS` | `1786838400` | 2026-08-16 00:00:00 UTC | own date | `protocol_changes/changes_4.js` | +| `CONTRACT_DELEGATION_MATERIALIZE` | `1789430400` | 2026-09-15 00:00:00 UTC | own date | `protocol_changes/changes_3.js` | +| `DISPENSER_ORACLE_PER_TOKEN_PRICE` | `1789430400` | 2026-09-15 00:00:00 UTC | own date | `protocol_changes/changes_2.js` | +| `CROSS_CHAIN_ROYALTY` | `1798761600` | 2027-01-01 00:00:00 UTC | own date | `protocol_changes/changes_2.js` | +| `REST_PATTERN_METER` | `1798761600` | 2027-01-01 00:00:00 UTC | own date | `protocol_changes/changes_2.js` | Thresholds keyed on a **block height** rather than a block time (the validator-era Cohort B rules and the per-chain Cohort C rules) are not listed diff --git a/protocol/index-id-references.md b/protocol/index-id-references.md index c5399753..d9d02a34 100644 --- a/protocol/index-id-references.md +++ b/protocol/index-id-references.md @@ -42,7 +42,15 @@ Two different questions are answered here: which fields RECEIVE an index id when introduces a new value, and in which fields a `^` written on the wire is RESOLVED on input. The first set is the consensus surface; the second is what a client may send. -**Ticker fields:** `TICK`, `GIVE_TICK`, `GET_TICK`, `DIVIDEND_TICK`, `CALLBACK_TICK`. +**Ticker fields:** `TICK`, `GIVE_TICK`, `GET_TICK`, `DIVIDEND_TICK`, `CALLBACK_TICK`, and +`LIST.ITEM` when the list `TYPE` is ticker. The ticker-typed list item carries two +qualifications the five single-value fields do not. A `^` written there is only ever +RESOLVED, never minted: the reference has to match a block-stamped ticker row of an +existing token, and an item that matches none is recorded `invalid: TICK (unknown)` and +left out of the materialized item set while the `LIST` action itself stays `valid`. And +the reference SDK does not compact it, so a client that wants the shorter form writes +`^` itself. +See [LIST](./actions/list.md). **Address fields that receive an index id:** the destination/transfer/get-address style fields of an action: @@ -122,8 +130,12 @@ automatically (opt out with `{ compactTickers: false }` / `{ compactAddresses: f It only ever emits a `^` for a value it has already resolved to an existing id via the explorer, and it falls back to the full value whenever an id cannot be resolved, so a client never emits an id the indexer would not recognize. Multi-recipient (array) and -type-gated list fields are left in full form by the SDK, which the rules above require: -the indexer resolves no `^` in `SEND.DESTINATION` or `LIST.ITEM`. The SDK also leaves +type-gated list fields are left in full form by the SDK. For `SEND.DESTINATION`, and for +`LIST.ITEM` when the list `TYPE` is address, the rules above require it: the indexer +resolves no `^` there. For `LIST.ITEM` when the list `TYPE` is ticker the SDK is being +conservative rather than obeying a protocol limit, because the indexer does resolve a +`^` item and stores it under the resolved ticker id; that compaction is left to +the client. The SDK also leaves `DISPENSER.GET_ADDRESS` and `DISPENSER.ORACLE_ADDRESS` in full form, for the decoder reason above, even though the indexer would resolve a reference there. diff --git a/protocol/json/README.md b/protocol/json/README.md index dd7a5fd2..163a9422 100644 --- a/protocol/json/README.md +++ b/protocol/json/README.md @@ -5,15 +5,18 @@ Machine-readable JSON artifacts for the XChain protocol. -## Token Information Standard (v1.1.0, current) +## Token Information Standard (v1.1.1, current) The off-chain token metadata document referenced by a token's on-chain `DESCRIPTION` URI. -- [Schema](./token-information-standard-v1.1.0-schema.json): JSON Schema for the metadata document. -- [Example](./token-information-standard-v1.1.0-example.json): a worked example that conforms to the schema. +- [Schema](./token-information-standard-v1.1.1-schema.json): JSON Schema for the metadata document. +- [Example](./token-information-standard-v1.1.1-example.json): a worked example that conforms to the schema. + +v1.1.1 relaxes one constraint and adds nothing: an `images`, `audio`, `video` or `files` entry now requires `type` plus at least one of `data` (off-chain URL) or `data_ref` (on-chain `FILE` reference), where v1.1.0 required `data` outright and so rejected the fully on-chain form the standard itself recommends. It is a pure relaxation, so every v1.1.0 and v1.0.0 document remains valid. ### Previous versions +- v1.1.0: [schema](./token-information-standard-v1.1.0-schema.json), [example](./token-information-standard-v1.1.0-example.json). Frozen as published; its four media definitions require `["type", "data"]`, so a `data_ref`-only entry fails validation against it. - v1.0.0: [schema](./token-information-standard-v1.0.0-schema.json), [example](./token-information-standard-v1.0.0-example.json). Frozen as published; it predates the token-gating fields (`packs`, `title`, `data_ref`, `locked`, `pack_id`). v1.1.0 is additive over it, so every v1.0.0 document is a valid v1.1.0 document. See the [Token Information Standard](../token-information-standard.md) for the field-by-field reference. diff --git a/protocol/json/token-information-standard-v1.1.1-example.json b/protocol/json/token-information-standard-v1.1.1-example.json new file mode 100644 index 00000000..4357d079 --- /dev/null +++ b/protocol/json/token-information-standard-v1.1.1-example.json @@ -0,0 +1,174 @@ +{ + "tick": "MYTOKEN", + "description": "This is a text description of MYTOKEN", + "website": "http://www.mysite.com", + "name": "Token Name", + "html": "", + + "owner": { + "name": "John Smith", + "title": "Chief Technology Officer (CFO)", + "organization": "ABC Technologies, Inc." + }, + + "contacts": [{ + "type": "address", + "data": "1234 Main Street, Seattle, WA 98104" + },{ + "type": "email", + "data": "info@domain.com" + },{ + "type": "phone", + "data": "1-949-555-1234" + },{ + "type": "fax", + "data": "1-949-555-1234" + },{ + "type": "url", + "data": "https://domain.com" + }], + + "categories":[{ + "type": "main", + "data": "Art" + },{ + "type": "sub", + "data": "Photographer" + },{ + "type": "other", + "data": "Skyline/Sunset Photographs" + }], + + "social": [{ + "type": "github", + "data": "https://github.com/XChain-Platform" + },{ + "type": "facebook", + "data": "https://facebook.com/XChain-Platform" + },{ + "type": "twitter", + "data": "https://twitter.com/xchain_io" + },{ + "type": "telegram", + "data": "https://t.me/xchain_io" + }], + + + "images": [{ + "type": "icon", + "size": "48x48", + "data": "https://domain.com/icon.png", + "hash": "8031025a667824a188dd5479ca5cb20c5306be06ed01875f7bcc11ecb48251be" + },{ + "type": "icon", + "size": "128x128", + "data": "https://domain.com/icon128.png" + },{ + "type": "standard", + "data": "https://domain.com/image.png" + },{ + "type": "large", + "name": "Image Name / Title", + "data": "https://domain.com/image_large.png" + },{ + "type": "hires", + "name": "Image Name / Title", + "data": "https://domain.com/image_hires.png" + },{ + "type": "standard", + "name": "On-chain artwork", + "data_ref": "action:12345" + }], + + "audio": [{ + "type": "m4a", + "data": "https://domain.com/audio.m4a", + "name": "Audio Name / Title", + "hash": "8031025a667824a188dd5479ca5cb20c5306be06ed01875f7bcc11ecb48251be" + },{ + "type": "mp3", + "name": "Audio Name / Title", + "data": "https://domain.com/audio.mp3" + },{ + "type": "wav", + "name": "Audio Name / Title", + "data": "https://domain.com/audio.wav" + }], + + "video": [{ + "type": "mp4", + "data": "https://domain.com/video.mp4", + "name": "Video Name / Title", + "hash": "8031025a667824a188dd5479ca5cb20c5306be06ed01875f7bcc11ecb48251be" + },{ + "type": "mov", + "name": "Video Name / Title", + "data": "https://domain.com/video.mov" + },{ + "type": "wmv", + "name": "Video Name / Title", + "data": "https://domain.com/video.wmv" + }], + + "files": [{ + "type": "doc", + "data": "https://domain.com/word.doc", + "name": "File Name / Title", + "hash": "8031025a667824a188dd5479ca5cb20c5306be06ed01875f7bcc11ecb48251be" + },{ + "type": "pdf", + "name": "File Name / Title", + "data": "https://domain.com/document.pdf" + },{ + "type": "xls", + "name": "File Name / Title", + "data": "https://domain.com/excel.xls" + },{ + "type": "other", + "name": "File Name / Title", + "data": "https://domain.com/filename.ext" + },{ + "type": "pdf", + "name": "liner-notes.pdf", + "title": "Liner Notes", + "data": "https://domain.com/liner-notes-preview.pdf", + "data_ref": "action:12345", + "locked": true, + "pack_id": "deluxe" + },{ + "type": "other", + "name": "stems.zip", + "title": "Stem Pack", + "data": "https://domain.com/stems-preview.zip", + "data_ref": "action:DOGE:67890", + "locked": true, + "pack_id": "deluxe" + }], + + "packs": { + "deluxe": { + "name": "Deluxe Edition", + "description": "Liner notes and stems, unlocked by holding the gate token" + } + }, + + "dns": [{ + "type": "A", + "host": "@", + "value": "123.123.123.123" + },{ + "type": "CNAME", + "host": "www", + "value": "domain.com" + },{ + "type": "TXT", + "host": "@", + "value": "google-site-verification=ihWf7hO1uxOcEyEW5KWRI1NtscPyJtQ6ko4BYQuC1Q8" + },{ + "type": "MX", + "host": "@", + "priority": 10, + "value": "aspmx.l.google.com" + }] + +} \ No newline at end of file diff --git a/protocol/json/token-information-standard-v1.1.1-schema.json b/protocol/json/token-information-standard-v1.1.1-schema.json new file mode 100644 index 00000000..65805220 --- /dev/null +++ b/protocol/json/token-information-standard-v1.1.1-schema.json @@ -0,0 +1,437 @@ +{ + "title": "Token Information Standard Schema", + "type": "object", + "$schema": "http://json-schema.org/draft-04/schema", + "version": "1.1.1", + + "properties": { + "tick": { + "type": "string", + "description": "The TICK name that represents the token" + }, + "description": { + "type": "string", + "maxLength": 2048, + "description": "A full text description of the token" + }, + "html": { + "type": "string", + "maxLength": 10000, + "description": "A snippet of HTML which can be displayed to provide additional token information and functionality" + }, + "website": { + "type": "string", + "format": "uri", + "maxLength": 255, + "description": "A URI with more information the token" + }, + "name": { + "type": "string", + "maxLength": 127, + "description": "The full name of the token" + }, + "owner": { + "$ref": "#/definitions/owner", + "description": "Information about the owner of this token" + }, + "contacts": { + "type": "array", + "items": { "$ref": "#/definitions/contacts" }, + "uniqueItems": true, + "description": "Information about how to contact the owner of this token" + }, + "categories": { + "type": "array", + "items": { "$ref": "#/definitions/categories" }, + "uniqueItems": true, + "description": "Information on what type of categories this token falls into" + }, + "social": { + "type": "array", + "items": { "$ref": "#/definitions/social" }, + "description": "Social media accounts related to this token" + }, + "images": { + "type": "array", + "items": { "$ref": "#/definitions/images" }, + "uniqueItems": true, + "description": "One or more images used to represent the token." + }, + "audio": { + "type": "array", + "items": { "$ref": "#/definitions/audio" }, + "uniqueItems": true, + "description": "One or more audio files related to the token." + }, + "video": { + "type": "array", + "items": { "$ref": "#/definitions/video" }, + "uniqueItems": true, + "description": "One or more video files related to the token." + }, + "files": { + "type": "array", + "items": { "$ref": "#/definitions/files" }, + "uniqueItems": true, + "description": "One or more files related to the token." + }, + "dns": { + "type": "array", + "items": { "$ref": "#/definitions/dns" }, + "uniqueItems": true, + "description": "One or more DNS records related to the token." + }, + "packs": { + "type": "object", + "additionalProperties": { "$ref": "#/definitions/pack" }, + "description": "Display metadata for token-gated content packs, keyed by pack id. A file entry joins a pack through its pack_id." + } + }, + + "required": ["tick", "name"], + + "definitions": { + + "owner": { + "type": "object", + "properties": { + "name": { + "type": "string", + "maxLength": 128, + "description": "The full name of the contact for the owner of this token" + }, + "title": { + "type": "string", + "maxLength": 128, + "description": "The organization title for the owner of this token" + }, + "organization": { + "type": "string", + "maxLength": 128, + "description": "The organization name that owns this token" + } + } + }, + + "categories": { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["main", "sub", "other"], + "description": "Type of category being given (main, subcategory, other)" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "Description of the category" + } + }, + "required": ["type", "data"] + }, + + "contacts": { + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["address", "email", "phone", "fax", "url"], + "description": "Type of contact information that is being given" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "The contact information (address, email, phone, etc)" + } + }, + "required": ["type", "data"], + "additionalProperties": false + }, + + "social": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Type of social media account that is being given (github, facebook, twitter, telegram, discord, etc)" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "A URI for the social media account" + } + }, + "required": ["type", "data"] + }, + + "pack": { + "type": "object", + "properties": { + "name": { + "type": "string", + "maxLength": 255, + "description": "Display name of the content pack" + }, + "description": { + "type": "string", + "maxLength": 2048, + "description": "A description of what the content pack contains" + } + } + }, + + "images": { + "type": "object", + "properties": { + "type": { + "enum": ["icon", "standard", "large", "hires"], + "description": "The type of image being given" + }, + "size": { + "description": "The size of the image for pixel-based images or svg for SVG images" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "A URI to the image file" + }, + "name": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "Name / Title of the image or artwork" + }, + "hash": { + "type": "string", + "format": "string", + "maxLength": 64, + "description": "A sha256 hash of the image file" + }, + "title": { + "type": "string", + "maxLength": 255, + "description": "Display title for this entry" + }, + "data_ref": { + "type": "string", + "maxLength": 255, + "description": "Reference to an on-chain FILE action carrying the image bytes, by ACTION_INDEX: action: on the token's own chain, or action:: on a sibling chain. Clients prefer data_ref over data when both are present." + }, + "locked": { + "type": "boolean", + "description": "true when the referenced FILE is encrypted and token-gated, so a client can render a locked state without fetching it" + }, + "pack_id": { + "type": "string", + "maxLength": 255, + "description": "Identifier of the content pack this entry belongs to; keys into the top-level packs map for display metadata" + } + }, + "required": ["type"], + "anyOf": [ + { "required": ["data"] }, + { "required": ["data_ref"] } + ] + }, + + "audio": { + "type": "object", + "properties": { + "type": { + "enum": ["m4a", "mp3", "wav"], + "description": "The type of audio file being given" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "A URI to the audio file" + }, + "name": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "Name / Title of the audio or artwork" + }, + "hash": { + "type": "string", + "format": "string", + "maxLength": 64, + "description": "A sha256 hash of the audio file" + }, + "title": { + "type": "string", + "maxLength": 255, + "description": "Display title for this entry" + }, + "data_ref": { + "type": "string", + "maxLength": 255, + "description": "Reference to an on-chain FILE action carrying the audio bytes, by ACTION_INDEX: action: on the token's own chain, or action:: on a sibling chain. Clients prefer data_ref over data when both are present." + }, + "locked": { + "type": "boolean", + "description": "true when the referenced FILE is encrypted and token-gated, so a client can render a locked state without fetching it" + }, + "pack_id": { + "type": "string", + "maxLength": 255, + "description": "Identifier of the content pack this entry belongs to; keys into the top-level packs map for display metadata" + } + }, + "required": ["type"], + "anyOf": [ + { "required": ["data"] }, + { "required": ["data_ref"] } + ] + }, + + "video": { + "type": "object", + "properties": { + "type": { + "enum": ["mp4", "mov", "wmv"], + "description": "The type of video file being given" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "A URI to the video file" + }, + "name": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "Name / Title of the video or artwork" + }, + "hash": { + "type": "string", + "format": "string", + "maxLength": 64, + "description": "A sha256 hash of the video file" + }, + "title": { + "type": "string", + "maxLength": 255, + "description": "Display title for this entry" + }, + "data_ref": { + "type": "string", + "maxLength": 255, + "description": "Reference to an on-chain FILE action carrying the video bytes, by ACTION_INDEX: action: on the token's own chain, or action:: on a sibling chain. Clients prefer data_ref over data when both are present." + }, + "locked": { + "type": "boolean", + "description": "true when the referenced FILE is encrypted and token-gated, so a client can render a locked state without fetching it" + }, + "pack_id": { + "type": "string", + "maxLength": 255, + "description": "Identifier of the content pack this entry belongs to; keys into the top-level packs map for display metadata" + } + }, + "required": ["type"], + "anyOf": [ + { "required": ["data"] }, + { "required": ["data_ref"] } + ] + }, + + "files": { + "type": "object", + "properties": { + "type": { + "description": "The type of file being given (doc, xls, pdf, other)" + }, + "data": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "A URI to the file" + }, + "name": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "Name / Title of the file" + }, + "hash": { + "type": "string", + "format": "string", + "maxLength": 64, + "description": "A sha256 hash of the file" + }, + "title": { + "type": "string", + "maxLength": 255, + "description": "Display title for this entry" + }, + "data_ref": { + "type": "string", + "maxLength": 255, + "description": "Reference to an on-chain FILE action carrying the file bytes, by ACTION_INDEX: action: on the token's own chain, or action:: on a sibling chain. Clients prefer data_ref over data when both are present." + }, + "locked": { + "type": "boolean", + "description": "true when the referenced FILE is encrypted and token-gated, so a client can render a locked state without fetching it" + }, + "pack_id": { + "type": "string", + "maxLength": 255, + "description": "Identifier of the content pack this entry belongs to; keys into the top-level packs map for display metadata" + } + }, + "required": ["type"], + "anyOf": [ + { "required": ["data"] }, + { "required": ["data_ref"] } + ] + }, + + "dns": { + "type": "object", + "properties": { + "type": { + "enum": [ "A", "AAAA", "ALIAS", "CAA", "CNAME", "NS", "SRV", "TXT", "URL", "MX", ""], + "description": "The type of DNS record" + }, + "host": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "The hostname for this DNS record" + }, + "value": { + "type": "string", + "format": "string", + "maxLength": 255, + "description": "The DNS value you wish to use for this record" + }, + "priority": { + "type": "integer", + "format": "integer", + "maxLength": 3, + "description": "The record priority level" + } + }, + "if": { + "properties": { + "type": { "const": "MX" } + }, + "required": ["type"] + }, + "then": { + "required": ["type", "host", "value", "priority"] + }, + "else": { + "required": ["type", "host", "value"] + }, + "additionalProperties": false + } + } +} \ No newline at end of file diff --git a/protocol/protocol-activation.md b/protocol/protocol-activation.md index 921db348..bea9b86a 100644 --- a/protocol/protocol-activation.md +++ b/protocol/protocol-activation.md @@ -78,7 +78,7 @@ gates ([below](#decoder-carried-gates)), which sit outside all three cohorts bec evaluated in the decoder rather than the indexer. Each consuming service carries a **byte-identical twin** of the maps it needs, and a cross-repo conformance gate fails CI if a twin drifts. Cohort A (contract-era) values are not carried in `constants.js` at all: they are -service-carried in `xchain-indexer/protocol_changes.js` and the `xchain-vm` gate constants (see the +service-carried in `xchain-indexer/src/protocol_changes.js` and the `xchain-vm` gate constants (see the table below), byte-guarded against each other rather than against this file, pending a future consolidation. @@ -93,7 +93,7 @@ re-runs an action handler, a deploy validator, or the VM. | Service | Carries | |---|---| | `xchain-indexer` | `protocol_changes.js` (contract-era gates) + the state-commitment and validator-era activation modules | -| `xchain-vm` | the seven contract-era VM gate constants (async ban, binary-alloc metering, deploy-linter hardening, state-key NUL-reject, state-key type normalization, metering eval-order fix, call-spread metering) plus three per-coin height-keyed maps: `PKG3_SANDBOX_ACTIVATION` (the armed runtime half of VM deploy-lint Pkg 3, [below](#additional-armed-gates-service-carried)), and the genesis-armed `EXEC_LINT_ACTIVATION` and `LINT_GLOBAL_ALIAS_ACTIVATION` ([VM gates](#vm-gates-service-carried)) | +| `xchain-vm` | the seven contract-era VM gate constants (async ban, binary-alloc metering, deploy-linter hardening, state-key NUL-reject, state-key type normalization, metering eval-order fix, call-spread metering) plus five constant-less contract-era riders that key on the binary-alloc instant instead of minting a constant ([Cohort A riders that mint no constant](#cohort-a-riders-that-mint-no-constant)) plus three per-coin height-keyed maps: `PKG3_SANDBOX_ACTIVATION` (the armed runtime half of VM deploy-lint Pkg 3, [below](#additional-armed-gates-service-carried)), and the genesis-armed `EXEC_LINT_ACTIVATION` and `LINT_GLOBAL_ALIAS_ACTIVATION` ([VM gates](#vm-gates-service-carried)) | | `xchain-hub` | the nine validator-era gate modules it consumes (checkpoint, equivocation header, stake-weighted quorum, anchor reward, archive reward, cross-chain royalty canonical, retraction signing, attestation relay, price signature tally). The tenth Cohort B gate, attestation admission, is indexer-only | | `xchain-decoder` | the five activation maps consumed in the decoder's own parse path: `ORACLE_FEE_OUTPUT_ACTIVATION`, `ORACLE_FEE_SET_CAPTURE_ACTIVATION`, `DISPENSER_EXPIRY_REALIGN_ACTIVATION` and `BATCH_SUBCOMMAND_OUTPUT_CAPTURE_ACTIVATION` (block-time-keyed) plus `ENVELOPE_RECOGNITION_ACTIVATION` (per-chain local height) | | `xchain-sync`, `xchain-explorer`, `xchain-sdk` | the subset each needs to verify or display | @@ -108,7 +108,7 @@ coordinated fleet rollout retire a whole batch at once. | Cohort | Keyed on | Rules | Straggler behavior | |---|---|---|---| -| **A (contract era)** | one shared **time** (all three chains) | base64 DEPLOY encoding, VM async ban, VM binary-alloc metering, VM deploy-linter hardening, VM state-key NUL-reject, VM state-key type normalization, VM metering eval-order fix, VM call-spread metering, VM consensus wall-clock budget, controller guards, VM balance/token-info surface, issuance-fee exemption, unstake-cooldown completion, cross-chain royalty create-side, XCALL undeliverable-result retirement | **forks** | +| **A (contract era)** | one shared **time** (all three chains) | base64 DEPLOY encoding, VM async ban, VM binary-alloc metering, VM deploy-linter hardening, VM state-key NUL-reject, VM state-key type normalization, VM metering eval-order fix, VM call-spread metering, VM consensus wall-clock budget, controller guards, VM balance/token-info surface, issuance-fee exemption, unstake-cooldown completion, cross-chain royalty create-side, XCALL undeliverable-result retirement, VM slash token-delimiter guard, VM slash amount-precision widening, VM math-output metering, VM emission prototype-key strip, VM non-finite gas clamp | **forks** | | **B (validator era)** | a **BTC height** (not always the same height across every Cohort B rule; see below) | checkpoint commitment, equivocation header, stake-weighted quorum, anchor reward, cross-chain royalty canonical, attestation admission, archive reward, retraction signing, attestation relay, price signature tally | **forks** | | **C (state commitment)** | per-chain **local height** | light-client state commitment (state root + block-merkle root) and its state-hash classes (e.g. token-supply, poll-finalize) | **halts, recoverable** | @@ -123,7 +123,39 @@ documented default so nothing a default-configured node ever executed changes ou the boundary. Tightening the value later is a different change and would need a flag day of its own. Enforcement detail is on [VM Configuration](../components/vm/configuration.md#resource-limits); the constant lives in -`xchain-vm/src/consensus-wall-clock.js` and the activation beside it in `xchain-vm/src/index.js`. +`xchain-vm/src/consensus_wall_clock.js` and the activation beside it in `xchain-vm/src/index.js`. + +### Cohort A riders that mint no constant + +The wall-clock bound is not the only rule of that shape. Five further VM rules key on the same +shared contract-era instant and mint no activation constant of their own, so anything that +inventories gates by constant name cannot see them. They ride +`BINARY_ALLOC_GATE_BLOCK_TIME` (`xchain-vm/src/index.js`, the binary-alloc metering leg already in +this cohort; the value is on [Flag-Day Values](./flag-days.md#contract-era-flag-day)) rather than +minting one, which is deliberate: a new constant would move the frozen six-gate consensus pin and +the cross-repo `CONTROLLER_GUARD` equality check that goes with it. They fork like the rest of +Cohort A, so a straggler node that has not upgraded reaches a different verdict. + +| Rider | Resolved in | Enforced in | What changes at the instant | +|---|---|---|---| +| **contract.slash token delimiter guard** (`isSlashTokenDelimGuardActive`) | `xchain-vm/src/index.js` | `xchain-vm/src/gateway.js`, on `readOnlyData.slashTokenDelimGuardOn` | A `contract.slash` whose `token` carries a `\|` stops emitting and throws, closing the one emit path that never rejected a character the indexer may pipe-join | +| **contract.slash amount-precision widening** (`isSlashAmountPrecisionActive`) | `xchain-vm/src/index.js` | `xchain-vm/src/gateway.js`, which swaps an 8-fractional-digit amount pattern for an 18-digit one | A slash amount carrying 9 to 18 fractional digits stops throwing and emits. The ceiling is `MAX_SLASH_AMOUNT_DECIMALS` 18, which must equal the indexer's `MAX_TOKEN_DECIMALS`: STAKE v3 admits a stake at the token's own decimals and the slash arithmetic computes the deduction at that precision, so the narrower pattern made an exact partial slash of a 9-to-18-decimal token impossible | +| **Math-output metering** (F-MO, the `mathOutputMeterOn` predicate) | `xchain-vm/src/index.js` | the gateway's math hook | An oversized `pow()` or format result is charged gas, which moves `gasUsed` | +| **Emission prototype-key strip** (F-PS, the `emissionDeepStrip` predicate) | `xchain-vm/src/index.js` | `EmissionCollector` | Prototype-shaped own keys are stripped recursively rather than only at the top level, which can drop a key from a pathological emitted param and so moves that emission's hash | +| **Non-finite gas clamp** (F-NR, the `nonFiniteFailClosed` predicate) | `xchain-vm/src/index.js` | the sandbox gas reference | A non-finite metering size resolves to `Number.MAX_SAFE_INTEGER` and yields a ceiling-clamped `out_of_gas` instead of collapsing to 1 gas, which moves the hashed status and `gasUsed` | + +The two `contract.slash` riders are **network-aware** and resolve exactly like the state-key gates: +`testnet` and `regtest` are active from genesis, and any other network (including an unrecognized +one, read conservatively as mainnet) activates at the shared instant, with a non-finite block time +reading as pre-activation. The three inline riders are **not** network-aware: they compare block +time alone, so they activate at the shared instant on every network, and a non-finite block time +likewise reads as pre-activation. + +No mainnet verdict is reinterpreted by any of the five. The shared instant has already passed, and +mainnet has carried no DEPLOY, EXECUTE or SLASH action and no contract at all (see +[The mainnet genesis arm](#the-mainnet-genesis-arm)), so the riders are live on arrival rather than +live and diverging. Changing any of these predicates later is a different change and would need a +flag day of its own under the [notice policy](./upgrade-notice-policy.md). The ten Cohort B rules arm in two batches. Six share mainnet BTC height 961000: checkpoint commitment, equivocation header, stake-weighted quorum, anchor reward, cross-chain royalty canonical, @@ -132,20 +164,24 @@ reward, retraction signing, attestation relay, and price signature tally. So "on shorthand for "a BTC height per rule, in two batches" rather than a single shared value across the whole cohort. -The cohort is its **armed** rules. `constants.js` also carries validator-era maps that are inert: -`SNAPSHOT_BURIAL_ACTIVATION`, `ANCHOR_REWARD_DERIVE_ACTIVATION`, `ATTEST_BROADCAST_FEE_ACTIVATION`, +The cohort is its **armed** rules. `constants.js` also carries validator-era maps that sit outside +it. Six of them are **armed at genesis** (`0`) on mainnet under the +[mainnet genesis arm](#the-mainnet-genesis-arm): `SNAPSHOT_BURIAL_ACTIVATION`, +`ANCHOR_REWARD_DERIVE_ACTIVATION`, `ATTEST_BROADCAST_FEE_ACTIVATION`, `ATTEST_REQUEST_CAP_ACTIVATION` (the per-block attestation admission ceiling, a sibling of the attestation-admission gate in the cohort table above), `ROLLCALL_ACTIVATION` (keyed on the BTC -`EPOCH_HEIGHT` a ROLLCALL carries), `ATTEST_RESPONSIBLE_WIDENING_ACTIVATION` and -`ATTEST_RESPONSE_MIRROR_ACTIVATION` each hold `null` on mainnet, which is the -encoding of "never" and the fail-closed default until an operator ratifies a height. They are not -counted above and carry no mainnet flag day yet. `ATTEST_RESPONSE_MIRROR_ACTIVATION` was the one of -them unarmed on **testnet** as well; it was ratified there at block 151324 in the v0.15.0 train, so +`EPOCH_HEIGHT` a ROLLCALL carries) and `ATTEST_RESPONSIBLE_WIDENING_ACTIVATION`. Arming them at `0` +was state-neutral on the indexed mainnet history, which holds no validator, roll call or +attestation, so none of them needed a mainnet flag day; they are still not counted in the cohort +table above. `ATTEST_RESPONSE_MIRROR_ACTIVATION` is the one of the seven that still holds `null` on +mainnet, which is the encoding of "never" and the fail-closed default until an operator ratifies a +height; the 2026-09-09 ruling holds it back until its quorum defect is closed. It was also the one +of them unarmed on **testnet**; it was ratified there at block 151324 in the v0.15.0 train, so testnet exercises the hub response-mirror path alongside regtest from that height on. The enumeration is the **height-keyed validator-era** maps specifically: the block-time [decoder-carried gates](#decoder-carried-gates) also read `null` as -disarmed, and `PRICE_PAIR_WIDEN_ACTIVATION` encodes the same "not yet" as a far-future sentinel -instant rather than as `null`. +disarmed, and a block-time map can encode the same "not yet" as a far-future sentinel instant +rather than as `null`. `ANCHOR_ACTIVATION` is height-keyed and **armed on both live networks**, but sits outside the three cohorts: it is keyed on the anchor's own DOGE mined height (`DOGE:mainnet` 6360000, `DOGE:testnet` @@ -156,13 +192,21 @@ so the restarted wire set has not activated there yet. Stragglers **fork**. Regtest runs every cohort **genesis-active** (threshold 0), so a fresh regtest stack exercises the post-activation behavior end to end. Testnet runs the time-keyed (Cohort A) and BTC-height-keyed -(Cohort B) gates genesis-active as well, with **three** exceptions: - -- **`ISSUE_INHERITED_MINT_WINDOW` (Cohort A) arms testnet at its own future instant, not from - genesis** (value on [Flag-Day Values](./flag-days.md)): the ISSUE mint-window - re-parameterization fix is a validity loosening, and testnet already held a recorded rejection - under the pre-fix rule, so a genesis-active arm would fork an already-synced testnet node - against a fresh reindex. It is the only Cohort A rule not genesis-active on testnet. +(Cohort B) gates genesis-active as well, with exceptions in every cohort: + +- **Four Cohort A rules arm testnet at their own future instants, not from genesis** (values on + [Flag-Day Values](./flag-days.md), which derives them from the registry and is the one place they + are written down). In each case testnet already carries history the rule would reinterpret, so a + genesis-active arm would fork an already-synced testnet node against a fresh reindex: + - `ISSUE_INHERITED_MINT_WINDOW`, because the ISSUE mint-window re-parameterization fix is a + validity loosening and testnet already held a recorded rejection under the pre-fix rule. + - `DEPLOY_DEFERRED_ASSEMBLY`, because testnet holds a recorded out-of-order assembler group that + a genesis-active arm would re-decide. + - `CONTRACT_META_REQUIRED`, because testnet already holds deployed contracts that export no + meta-shaped object, so a genesis-active arm would flip every one of them from its recorded + verdict. + - `UNIFIED_FEES_SWEEP_CALLBACK`, because the public testnet has carried real SWEEP and CALLBACK + traffic since launch, so a genesis-active arm would re-price fees already committed there. - **Cohort C (state commitment) is armed at future _per-chain_ heights on testnet, not from genesis** (`STATE_COMMITMENT_ACTIVATION`: `BTC:testnet 145000`, `LTC:testnet 4805000`, `DOGE:testnet 67000000`), because it gates on each chain's own local block height rather than a @@ -173,8 +217,11 @@ post-activation behavior end to end. Testnet runs the time-keyed (Cohort A) and checkpoint. It is the only Cohort B rule not genesis-active on testnet; the other nine carry `testnet: 0`. -Mainnet is genesis-active for nothing: every cohort is armed at a real, non-zero threshold. Several -of those have since been crossed. Cohort C's mainnet heights are past (`BTC:mainnet` 958500 and +No **cohort** rule is genesis-active on mainnet: every cohort gate is armed at a real, non-zero +threshold. Gates outside the three cohorts are a separate question, and several of them do read `0` +there under [the mainnet genesis arm](#the-mainnet-genesis-arm): the validator-era maps above and +`DISPENSER_EXPIRY_REALIGN_ACTIVATION` in the decoder-carried table. Several of the cohort thresholds +have since been crossed. Cohort C's mainnet heights are past (`BTC:mainnet` 958500 and `LTC:mainnet` 3143000 both sit below the envelope heights that crossed 2026-08-03), the main Cohort B anchor 961000 went by with them, and the shared Cohort A instant has passed as well. A crossed value stays in the tree for the same reason the envelope map does: below its threshold each gate still runs @@ -202,15 +249,15 @@ inventoried on this page, the armed ones in this table and the mainnet-unarmed V | **SWQ source cap** (`SWQ_SOURCE_CAP_ACTIVATION`, caps `STAKE_WEIGHT_MAX_SOURCES=1000`, `STAKE_WEIGHT_MAX_KEYS_PER_SOURCE=64`) | BTC height | `BTC:mainnet` 960000 (after state commitment 958500, before stake-weighted quorum 961000; LTC/DOGE inert) | forks | `xchain-indexer` / `xchain-sync` `src/swq_source_cap_activation.js` | | **Slash burns pending stake** (`SLASH_BURNS_PENDING_STAKE`) | BTC height | 961000 (Cohort-B anchor) | forks | `xchain-indexer/src/protocol_changes.js` | | **Slash oracle-round discriminated** (`SLASH_ORACLE_ROUND_DISCRIMINATED`, the sibling registry entry one row from slash-burns) | BTC height | 961000 (Cohort-B anchor) | forks | `xchain-indexer/src/protocol_changes.js` | -| **VM deploy-lint Pkg 3** (`VM_DEPLOY_LINT_PKG3_ACTIVATION`, adds the two Package-3 deploy-blocking `CONSENSUS_RULES`, so it changes which contracts the chain accepts) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | `xchain-indexer/src/vm_deploy_lint_pkg3_activation.js`; the runtime half is `PKG3_SANDBOX_ACTIVATION` in `xchain-vm/src/index.js`, pinned to the same three heights so the deploy-time and execution-time halves stay coherent | -| **Oracle snapshot-age causality** (`ORACLE_SNAPSHOT_AGE_CAUSALITY_ACTIVATION`, caps the snapshot-age query at the processing block; the uncapped value is VM-visible and forks `contract_hash` between a synced and a catching-up node) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | `xchain-indexer/src/oracle_snapshot_age_causality_activation.js` | -| **Dispenser freshness** (`DISPENSER_FRESHNESS_ACTIVATION`, redefines freshness against indexer-local chain state instead of the external utxo tracker, changing which historical DISPENSER creates were valid) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | `xchain-indexer/src/dispenser_freshness_activation.js` | +| **VM deploy-lint Pkg 3** (`VM_DEPLOY_LINT_PKG3_ACTIVATION`, adds the two Package-3 deploy-blocking `CONSENSUS_RULES`, so it changes which contracts the chain accepts) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | registry row `vm_deploy_lint_pkg3_activation.VM_DEPLOY_LINT_PKG3_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_3.js`; the runtime half is `PKG3_SANDBOX_ACTIVATION` in `xchain-vm/src/index.js`, pinned to the same three heights so the deploy-time and execution-time halves stay coherent | +| **Oracle snapshot-age causality** (`ORACLE_SNAPSHOT_AGE_CAUSALITY_ACTIVATION`, caps the snapshot-age query at the processing block; the uncapped value is VM-visible and forks `contract_hash` between a synced and a catching-up node) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | registry row `oracle_snapshot_age_causality_activation.ORACLE_SNAPSHOT_AGE_CAUSALITY_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_2.js` | +| **Dispenser freshness** (`DISPENSER_FRESHNESS_ACTIVATION`, redefines freshness against indexer-local chain state instead of the external utxo tracker, changing which historical DISPENSER creates were valid) | per-chain local height | `BTC:mainnet` 961000, `LTC:mainnet` 3154250, `DOGE:mainnet` 6319000 (armed 2026-07-22) | forks | registry row `dispenser_freshness_activation.DISPENSER_FRESHNESS_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_1.js` | | **List-edit resolution** (`LIST_EDIT_RESOLUTION_ACTIVATION`, resolves a list to its newest valid edit; `getList` gates BET place, ORDER/SWAP match, DISPENSE, DIVIDEND, CALLBACK and AIRDROP, so action acceptance changes) | per-chain local height | `BTC:mainnet` 963000, `LTC:mainnet` 3162000, `DOGE:mainnet` 6338000 | forks | `xchain-indexer` / `xchain-explorer` `src/list_edit_resolution_activation.js` | -| **Caret-ref strict** (`CARET_REF_STRICT_ACTIVATION`, makes an unresolvable address reference a hard reject at three sites that previously failed open, which moves the block's credits and debits) | per-chain local height | `BTC:mainnet` 963000, `LTC:mainnet` 3162000, `DOGE:mainnet` 6338000 (kept value-equal to list-edit resolution) | forks | `xchain-indexer/src/caret_ref_strict_activation.js` | -| **Oracle stale-round visibility** (`ORACLE_STALE_ROUND_VISIBILITY_ACTIVATION`, keeps a stale tip round in the `getPrice()` view with its price withheld instead of dropping the round outright, so a contract can tell an oracle stall apart from an oracle that never ran; VM-visible, so it changes `contract_hash`) | per-chain local height | `BTC:mainnet` 966500, `LTC:mainnet` 3175500, `DOGE:mainnet` 6370000 (pinned ahead of the tip the first release carrying the gate deploys at, so the flag day has no retroactive window; it does NOT share the list-edit resolution boundary, which rides an earlier release) | forks | `xchain-indexer/src/oracle_stale_round_visibility_activation.js` | -| **Ledger amount precision** (`LEDGER_AMOUNT_PRECISION_ACTIVATION`, quantizes every ledger write at 18 dp, the finest precision a tick can be issued with, instead of the written tick's own `decimals`; `db.createLedgerChangeRecord` takes the scale from `ledgerWriteScale` and applies it as `bcadd(amount, 0, decimals)`, so the persisted amounts and the balances projected from them both move) | per-chain local height | `BTC:mainnet` 966500, `LTC:mainnet` 3175500, `DOGE:mainnet` 6370000 (pinned to the same boundary as oracle stale-round visibility so both arm in one fleet deploy, and above each chain's tip at pinning so the flag day has no retroactive window) | forks | `xchain-indexer/src/ledger_amount_precision_activation.js` (indexer-only: the rule sits on the ledger write path, which `xchain-sync` never re-runs, so it has no twin and no twin drift guard) | +| **Caret-ref strict** (`CARET_REF_STRICT_ACTIVATION`, makes an unresolvable address reference a hard reject at three sites that previously failed open, which moves the block's credits and debits) | per-chain local height | `BTC:mainnet` 963000, `LTC:mainnet` 3162000, `DOGE:mainnet` 6338000 (kept value-equal to list-edit resolution) | forks | `xchain-indexer/src/db/database/caret_ref_strict_gate.js` | +| **Oracle stale-round visibility** (`ORACLE_STALE_ROUND_VISIBILITY_ACTIVATION`, keeps a stale tip round in the `getPrice()` view with its price withheld instead of dropping the round outright, so a contract can tell an oracle stall apart from an oracle that never ran; VM-visible, so it changes `contract_hash`) | per-chain local height | `BTC:mainnet` 966500, `LTC:mainnet` 3175500, `DOGE:mainnet` 6370000 (pinned ahead of the tip the first release carrying the gate deploys at, so the flag day has no retroactive window; it does NOT share the list-edit resolution boundary, which rides an earlier release) | forks | registry row `oracle_stale_round_visibility_activation.ORACLE_STALE_ROUND_VISIBILITY_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_2.js` | +| **Ledger amount precision** (`LEDGER_AMOUNT_PRECISION_ACTIVATION`, quantizes every ledger write at 18 dp, the finest precision a tick can be issued with, instead of the written tick's own `decimals`; `db.createLedgerChangeRecord` takes the scale from `ledgerWriteScale` and applies it as `bcadd(amount, 0, decimals)`, so the persisted amounts and the balances projected from them both move) | per-chain local height | `BTC:mainnet` 966500, `LTC:mainnet` 3175500, `DOGE:mainnet` 6370000 (pinned to the same boundary as oracle stale-round visibility so both arm in one fleet deploy, and above each chain's tip at pinning so the flag day has no retroactive window) | forks | `xchain-indexer/src/consensus/ledger_amount_precision_gate.js` (indexer-only: the rule sits on the ledger write path, which `xchain-sync` never re-runs, so it has no twin and no twin drift guard) | | **State-key collation** (`STATE_KEY_COLLATION_ACTIVATION`) | per-chain local height | `BTC:mainnet` 962500, `LTC:mainnet` 3160000, `DOGE:mainnet` 6335000 (armed 2026-07-10, ~10 days past Cohort-B) | halts, recoverable | `xchain-indexer` / `xchain-sync` `src/state_key_collation_activation.js` | -| **DISPENSE cancelling-dispenser match** (`DISPENSE_CANCELLING_MATCH_ACTIVATION`, corrects the `db.findMatchingDispensers` latest-status correlation on the native-coin DISPENSE trigger path) | block time | the coordinated 2.0.0 [contract-era flag day](./flag-days.md#contract-era-flag-day); deploy all indexers before it | forks | `xchain-indexer/src/dispense_cancelling_match_activation.js` | +| **DISPENSE cancelling-dispenser match** (`DISPENSE_CANCELLING_MATCH_ACTIVATION`, corrects the `db.findMatchingDispensers` latest-status correlation on the native-coin DISPENSE trigger path) | block time | the coordinated 2.0.0 [contract-era flag day](./flag-days.md#contract-era-flag-day); deploy all indexers before it | forks | registry row `dispense_cancelling_match_activation.DISPENSE_CANCELLING_MATCH_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_1.js` | The SWQ source cap, slash-burns and slash-oracle-round gates are BTC-height forking rules that belong with **Cohort B**; state-key collation is a per-chain additive gate that behaves like **Cohort C** @@ -239,8 +286,8 @@ BOTH copies is a flag day under the [notice policy](./upgrade-notice-policy.md). | Gate | Keyed on | Mainnet | Straggler | Lives in | |---|---|---|---|---| -| **Execute-time source lint** (`EXEC_LINT_ACTIVATION`, re-runs the deploy syntax validation against a contract's stored source at execute time and fails the execution deterministically when that source no longer passes the bans active for the block; the check is metered as gas, so it moves `gasUsed`) | per-chain local height | **armed at genesis** (0 for BTC, LTC and DOGE, ruled 2026-09-09); testnet and regtest genesis-active | forks | `xchain-vm/src/index.js` (`EXEC_LINT_ACTIVATION`, resolver `isExecLintActive`); twin `xchain-indexer/src/vm_exec_lint_activation.js`, pinned to byte equality by the consensus-params suites in both repos. A height armed on one side only forks the fleet | -| **Deploy-lint global-alias refinement** (`LINT_GLOBAL_ALIAS_ACTIVATION`, makes the banned-global deploy rules resolve sloppy-mode `this` and the `globalThis` self-reference chain as reads of the same global object, which moves DEPLOY verdicts on error-severity `CONSENSUS_RULES`) | per-chain local height | **armed at genesis** (0 for BTC, LTC and DOGE, ruled 2026-09-09); testnet and regtest genesis-active | forks | `xchain-vm/src/index.js` (`LINT_GLOBAL_ALIAS_ACTIVATION`, resolver `isLintGlobalAliasActive`); twin `xchain-indexer/src/vm_lint_global_alias_activation.js`, pinned the same way | +| **Execute-time source lint** (`EXEC_LINT_ACTIVATION`, re-runs the deploy syntax validation against a contract's stored source at execute time and fails the execution deterministically when that source no longer passes the bans active for the block; the check is metered as gas, so it moves `gasUsed`) | per-chain local height | **armed at genesis** (0 for BTC, LTC and DOGE, ruled 2026-09-09); testnet and regtest genesis-active | forks | `xchain-vm/src/index.js` (`EXEC_LINT_ACTIVATION`, resolver `isExecLintActive`); twin registry row `vm_exec_lint_activation.VM_EXEC_LINT_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_3.js`, pinned to byte equality by the consensus-params suites in both repos. A height armed on one side only forks the fleet | +| **Deploy-lint global-alias refinement** (`LINT_GLOBAL_ALIAS_ACTIVATION`, makes the banned-global deploy rules resolve sloppy-mode `this` and the `globalThis` self-reference chain as reads of the same global object, which moves DEPLOY verdicts on error-severity `CONSENSUS_RULES`) | per-chain local height | **armed at genesis** (0 for BTC, LTC and DOGE, ruled 2026-09-09); testnet and regtest genesis-active | forks | `xchain-vm/src/index.js` (`LINT_GLOBAL_ALIAS_ACTIVATION`, resolver `isLintGlobalAliasActive`); twin registry row `vm_lint_global_alias_activation.VM_LINT_GLOBAL_ALIAS_ACTIVATION` in `xchain-indexer/src/protocol_changes/gates_3.js`, pinned the same way | ## Decoder-carried gates @@ -258,7 +305,7 @@ height. | **Oracle-fee set-membership capture** (`ORACLE_FEE_SET_CAPTURE_ACTIVATION`, resolves a DISPENSER v2 edit/refill oracle-fee output by set membership over every open Mode B dispenser of the paying source) | block time | mainnet **armed** at the same contract-era instant as the oracle-fee output capture above, which it may never precede (ruled 2026-09-09 under the [mainnet genesis arm](#the-mainnet-genesis-arm); identity on mainnet history, which holds no dispenser); testnet and regtest genesis-active (0) | forks | `protocol/constants.js`, vendored into `xchain-decoder/src/protocol/constants.js` | | **Dispenser expiry realignment** (`DISPENSER_EXPIRY_REALIGN_ACTIVATION`, soft-expires open dispensers *after* the block's transaction loop, where the indexer's measurement point already is) | block time | mainnet **armed at genesis** (0, ruled 2026-09-09 under the [mainnet genesis arm](#the-mainnet-genesis-arm)); testnet and regtest genesis-active (0) | forks | `protocol/constants.js`, vendored into `xchain-decoder/src/protocol/constants.js` | | **BATCH sub-command output capture** (`BATCH_SUBCOMMAND_OUTPUT_CAPTURE_ACTIVATION`, decides which native-coin outputs to persist from a BATCH's sub-commands instead of only its top-level ACTION name, so a batched COINPAY or Mode B DISPENSER stops spending a coin and settling nothing) | block time | mainnet **armed**, at the same instant the indexer's `BATCH_ISSUANCE_LIMITS` carries; testnet and regtest genesis-active (0) | forks | `protocol/constants.js`, vendored into `xchain-decoder/src/protocol/constants.js` | -| **Taproot envelope recognition** (`ENVELOPE_RECOGNITION_ACTIVATION`, and with it every [envelope consensus rule](./taproot-envelope.md): end-indexed witness parsing, annex refusal, input-0 binding, mixed-carrier and multi-envelope rejection) | per-chain **local height** | `BTC:mainnet` 960850, `LTC:mainnet` 3153500 (both crossed 2026-08-03), `DOGE` **null on every network**; testnet and regtest genesis-active (0) | forks | `xchain-decoder/src/XChainDecoder.js`, mirrored in `xchain-encoder/src/CryptoNetworks.js` | +| **Taproot envelope recognition** (`ENVELOPE_RECOGNITION_ACTIVATION`, and with it every [envelope consensus rule](./taproot-envelope.md): end-indexed witness parsing, annex refusal, input-0 binding, mixed-carrier and multi-envelope rejection) | per-chain **local height** | `BTC:mainnet` 960850, `LTC:mainnet` 3153500 (both crossed 2026-08-03), `DOGE` **null on every network**; testnet and regtest genesis-active (0) | forks | `xchain-decoder/src/XChainDecoder.js`, mirrored in `xchain-encoder/src/build/crypto_networks.js` | **Neither armed decoder gate appears by name on [Flag-Day Values](./flag-days.md).** That page is generated from the indexer's registry and the activation modules beside it, so a gate declared only diff --git a/protocol/providers/llm.md b/protocol/providers/llm.md index f8b169b3..2b91dde8 100644 --- a/protocol/providers/llm.md +++ b/protocol/providers/llm.md @@ -83,7 +83,7 @@ flowchart TD ## Auth & Transport (operator-managed) -Each validator's hub picks a transport per `xchain-hub/src/lib/hub-credentials.js`. Both transports return `{ body, meta: }`; the choice is invisible to contracts and to other validators (only response bytes feed PBFT). +Each validator's hub picks a transport per `xchain-hub/src/lib/hub_credentials.js`. Both transports return `{ body, meta: }`; the choice is invisible to contracts and to other validators (only response bytes feed PBFT). | Transport | Auth source | Cost model | Determinism | | --------------- | --------------------------------------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------ | @@ -137,7 +137,7 @@ Validators on `claude_spawn` amortize their subscription across requests they se ## References -- Provider def: `xchain-hub/src/ProviderRegistry.js` (DEFAULTS.llm) +- Provider def: `xchain-hub/src/validators/provider_registry.js` (DEFAULTS.llm) - Provider module: `xchain-hub/src/providers/llm.js` -- Auth resolver: `xchain-hub/src/lib/hub-credentials.js` -- CLI wrapper: `xchain-hub/src/lib/claude-spawn.js` +- Auth resolver: `xchain-hub/src/lib/hub_credentials.js` +- CLI wrapper: `xchain-hub/src/providers/llm/claude_spawn.js` diff --git a/protocol/reference-impl/consensus/gate_registry.js b/protocol/reference-impl/consensus/gate_registry.js new file mode 100644 index 00000000..fb9e9cf9 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry.js @@ -0,0 +1,77 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The documentation repo's activation registry: every flag-day table the + * twinned reference-impl carriers beside this directory read, as (key, + * value) rows keyed '.', the spelling the fleet's rules digest + * and signed GATES field already use. The carriers keep their predicates and + * read their tables from here, so a table lives in ONE place per repo and a + * moved carrier can no longer read as "not yet active". + * + * This file is the ENTRY. The rows live in the part files under + * gate_registry/: shared_rows_1.js to shared_rows_5.js are BYTE TWINS of + * xchain-indexer/src/protocol_changes/shared_rows_1.js to _5.js (the SHARED + * block every consumer of this platform judges), shared_rows.js is the twin + * of the queue they write into and regtest_env.js the twin of the arming + * grammar it reads, and core.js is the consumer core the other consumers copy + * byte for byte. Copy with cp, prove with cmp: nothing in a twin is edited + * here, and no key, spelling or order in the block changes inside a window. + * + * REGTEST ARMING is applied WHEN A ROW IS READ (shared_rows.js registerRows + * installs it as the core's read overlay): the block writes the five + * venue-armed regtest entries UNPINNED, the registry stores that committed + * table, and every get(), copy(), rows() and activeAt() arms the entry from + * this process's environment as it stands at that moment. This repo carries + * no runtime service and no config layer (protocol/reference-impl is a + * conformance source of record, not a deployed carrier), so the overlay + * reads process.env directly. + * + * Readers get(), copy(), has(), keys(), rows() and activeAt(). A miss THROWS a + * RegistryMissError naming the key: a row a build lacks is a build defect and + * never a network state. Nothing may add a row after this module loads. + * + * DOCUMENTATION-ONLY GATES: none. Every twinned carrier's table is in the + * SHARED block, so this repo registers no rows of its own. + * + ********************************************************************/ + +'use strict'; + +const core = require('./gate_registry/core.js'); +const { registerRows } = require('./gate_registry/shared_rows.js'); + +// The SHARED block, loaded for effect: each part queues its rows into +// shared_rows.js as it loads; registerRows() below replays them, in part +// order, into the one registry and installs the venue's regtest arming as +// its read overlay. +require('./gate_registry/shared_rows_1.js'); +require('./gate_registry/shared_rows_2.js'); +require('./gate_registry/shared_rows_3.js'); +require('./gate_registry/shared_rows_4.js'); +require('./gate_registry/shared_rows_5.js'); + +const { registry } = core; +registerRows(registry, process.env); + +module.exports = { + get: (key) => registry.get(key), + copy: (key) => registry.copy(key), + has: (key) => registry.has(key), + keys: () => registry.keys(), + rows: () => registry.rows(), + activeAt: (key, network, coin, height, time) => registry.activeAt(key, network, coin, height, time), + UNARMED: core.UNARMED, + UNPINNED: core.UNPINNED, + RegistryMissError: core.RegistryMissError, +}; diff --git a/protocol/reference-impl/consensus/gate_registry/core.js b/protocol/reference-impl/consensus/gate_registry/core.js new file mode 100644 index 00000000..e362dfd9 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/core.js @@ -0,0 +1,270 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The consumer's activation registry core: rows, sentinels and the one + * generic predicate. + * + * A row is (key, value). The key is literal data in the `.` + * spelling the rules digest and the signed GATES field already use, never + * derived from where a file lives, so a module can move without a row moving. + * The value is a gate's table of thresholds or a constant. + * + * This is the CONSUMER core: the reader half of xchain-indexer's + * src/protocol_changes/core.js with the same semantics for get(), copy(), + * has(), keys(), rows() and activeAt(), and the same addGate() validation, + * minus the ProtocolChanges time table the indexer alone keeps. The same + * bytes sit in xchain-sync, xchain-explorer and xchain-sdk, so a shim vendored + * byte-identical across the repos behaves the same whichever registry backs it. + * + * Only the part files beside this one call addGate(); every other module + * reads. A miss THROWS, because a row a build lacks is a build defect and not + * a network state, and a null answer would let a moved carrier read as "not + * yet active". Every read passes through one overlay hook, which is where + * shared_rows.js applies a regtest venue's arming at the moment of the read + * (see setReadOverlay). + * + * The core requires nothing, so no feature module can form a cycle with it. + * + ********************************************************************/ + +'use strict'; + +// Key grammar, the one the indexer's canonicaliser spells. The first segment +// admits upper case because today's module stems do, and the key must be +// today's spelling, the same one knownGateKeys() uses in the rules digest. +const KEY_RE = /^[A-Za-z0-9_/-]+(\.[A-Za-z0-9_]+)+$/; + +// The house sentinel for a gate whose instant or height the operator has not +// named yet: a real number (year 2286), so it never fires before then and so +// the fingerprint tells it apart from UNPINNED. Never write the bare literal. +const UNARMED = 9999999999; + +// A network the gate has not been ratified for at all. Never active: the +// predicate below refuses a null threshold explicitly, because `0 >= null` is +// true in JavaScript and would arm the gate at genesis. +const UNPINNED = null; + +// The five row kinds addGate() accepts. `height` and `time` compare a block +// height or block time against the table; `epoch` is a BTC epoch height +// (the roll-call shape); `ruleset` is a version-keyed table of heights (the +// train shape); `constant` is any canonicalisable value with no predicate. +const UNITS = Object.freeze(['height', 'time', 'epoch', 'ruleset', 'constant']); + +class RegistryMissError extends Error { + constructor(key) { + super('activation registry has no row ' + JSON.stringify(key)); + this.name = 'RegistryMissError'; + this.key = key; + } +} + +function isPlainObject(value) { + if (value === null || typeof value !== 'object') return false; + const proto = Object.getPrototypeOf(value); + return proto === Object.prototype || proto === null; +} + +// A frozen deep copy of plain data, so a part file's literal cannot be edited +// through the registry and the registry cannot be edited through the literal. +function frozenCopy(value) { + if (Array.isArray(value)) return Object.freeze(value.map(frozenCopy)); + if (!isPlainObject(value)) return value; + const out = {}; + for (const k of Object.keys(value)) out[k] = frozenCopy(value[k]); + return Object.freeze(out); +} + +// The mutable mirror of frozenCopy(): the same walk, nothing frozen. +function mutableCopy(value) { + if (Array.isArray(value)) return value.map(mutableCopy); + if (!isPlainObject(value)) return value; + const out = {}; + for (const k of Object.keys(value)) out[k] = mutableCopy(value[k]); + return out; +} + +function hasOwn(obj, key) { + return Object.prototype.hasOwnProperty.call(obj, key); +} + +// ':' first, then the bare network key, the resolution order +// every coin-keyed activation module uses today. Own properties only: the +// network name comes off configuration, and an inherited member such as +// `constructor` must read as absent, not as a threshold. +function resolveThreshold(table, network, coin) { + if (coin !== null && coin !== undefined && hasOwn(table, coin + ':' + network)) return table[coin + ':' + network]; + return hasOwn(table, network) ? table[network] : undefined; +} + +// The parse-and-fail-closed body of the predicates, written once. An +// unparseable clock, an absent network, UNPINNED and an unknown network all +// read as inactive; UNARMED is a number and reads as inactive until 2286. +function reached(threshold, clock) { + const c = parseInt(clock); + if (!Number.isFinite(c)) return false; + if (typeof threshold !== 'number' || !Number.isFinite(threshold)) return false; + return c >= threshold; +} + +function checkKey(key, where) { + if (typeof key !== 'string' || !KEY_RE.test(key)) { + throw new Error(where + ': key ' + JSON.stringify(key) + ' does not match the key grammar ' + String(KEY_RE)); + } +} + +// A threshold table is a plain object whose entries are heights or instants +// (a finite number, UNARMED included) or UNPINNED; a ruleset table nests one +// such object per version. Anything else is refused at registration. +function checkTable(key, unit, table) { + if (!isPlainObject(table)) throw new Error('addGate: ' + key + ' table must be a plain object of network keys'); + const leaves = unit === 'ruleset' ? Object.values(table) : [table]; + for (const leaf of leaves) { + if (!isPlainObject(leaf)) throw new Error('addGate: ' + key + ' ruleset table must map versions to network tables'); + for (const [network, v] of Object.entries(leaf)) { + if (v === UNPINNED || (typeof v === 'number' && Number.isFinite(v))) continue; + throw new Error('addGate: ' + key + ' entry ' + network + ' must be a finite number or UNPINNED, got ' + JSON.stringify(v)); + } + } +} + +// What the indexer's canonicaliser refuses, refused here at registration for +// the same reason: a value the fingerprint cannot serialise by value (a +// function, a class instance, a Map, a Set, a Date, a BigInt, a non-finite +// number, an undefined) has no meaning as a row. The hub carries no copy of +// that canonicaliser, so the acceptance set is restated as a walk. +function checkValue(key, value, where) { + if (value === null || typeof value === 'boolean' || typeof value === 'string') return; + if (typeof value === 'number') { + if (!Number.isFinite(value)) throw new Error('addGate: ' + key + ' refused non-finite number at ' + where); + return; + } + if (value instanceof RegExp) return; + if (Array.isArray(value)) { + value.forEach((v, i) => checkValue(key, v, where + '[' + i + ']')); + return; + } + if (!isPlainObject(value)) throw new Error('addGate: ' + key + ' refused ' + (typeof value === 'object' ? 'class instance' : typeof value) + ' at ' + where); + for (const k of Object.keys(value)) { + if (value[k] === undefined) continue; + checkValue(key, value[k], where + '.' + k); + } +} + +class GateRegistry { + constructor() { + // key -> { unit, value }, in insertion order; a Map so no action name + // off the wire can ever resolve to an inherited member. + this.entries = new Map(); + // (key, committed value) -> the value a reader sees. Null until + // registerRows() installs the regtest arming, which is applied here at + // READ time so a module that reads a row when it loads sees the venue's + // environment as it stands at that moment, exactly as its own literal + // did, and the fingerprint sees it as it stands when it runs. + this.overlay = null; + } + + /** + * Installs the one read overlay. Registration stays the committed table; + * every read (get, copy, rows, activeAt) passes through `fn`. + * @param {(key: string, value: *) => *} fn + */ + setReadOverlay(fn) { + if (typeof fn !== 'function') throw new Error('setReadOverlay: expected a function'); + this.overlay = fn; + } + + // The one read path: the committed value through the overlay, or as is. + read(key) { + const row = this.entries.get(key); + if (!row) throw new RegistryMissError(key); + return this.overlay ? this.overlay(key, row.value) : row.value; + } + + /** + * Registers one gate or constant row. + * @param {string} key `.` per the fingerprint key grammar + * @param {string} unit one of UNITS + * @param {*} table the threshold table (frozen copy stored), or for + * `constant` any value the canonicaliser accepts + */ + addGate(key, unit, table) { + checkKey(key, 'addGate'); + if (this.entries.has(key)) throw new Error('addGate: duplicate key ' + key); + if (!UNITS.includes(unit)) throw new Error('addGate: ' + key + ' unit must be one of ' + UNITS.join('|') + ', got ' + JSON.stringify(unit)); + if (unit !== 'constant') checkTable(key, unit, table); + checkValue(key, table, '$'); + this.entries.set(key, { unit, value: frozenCopy(table) }); + } + + has(key) { return this.entries.has(key); } + + /** @returns {*} the row's value; throws RegistryMissError on a miss (never null). */ + get(key) { + return this.read(key); + } + + /** + * The row's value as a fresh MUTABLE deep copy (primitives and RegExps as + * they are); throws RegistryMissError on a miss. For a module that owned a + * plain table before the registry and whose tests patch it: the copy is the + * module's to mutate, the stored row never moves. + * @returns {*} + */ + copy(key) { + return mutableCopy(this.get(key)); + } + + /** @returns {string} the row's unit; throws RegistryMissError on a miss. */ + unitOf(key) { + const row = this.entries.get(key); + if (!row) throw new RegistryMissError(key); + return row.unit; + } + + keys() { return [...this.entries.keys()]; } + + /** @returns {Array<[string, *]>} every row in insertion order, the fingerprint's input. */ + rows() { return [...this.entries.keys()].map((key) => [key, this.read(key)]); } + + /** + * The one generic predicate. Resolves the coin key before the network key + * and applies the unit: `height` and `epoch` compare `height`, `time` + * compares `time`. `ruleset` needs the version this signature lacks and + * `constant` rows have no predicate, so those throw. + * @param {string} key + * @param {string} network mainnet|testnet|regtest + * @param {string|null} coin BTC|LTC|DOGE, or null for a network-only lookup + * @param {number|string} height block height (or the epoch height for `epoch`) + * @param {number|string} time block time + * @returns {boolean} + */ + activeAt(key, network, coin, height, time) { + const row = this.entries.get(key); + if (!row) throw new RegistryMissError(key); + if (row.unit !== 'height' && row.unit !== 'epoch' && row.unit !== 'time') { + throw new Error('activeAt: unsupported unit ' + row.unit + ' for ' + key); + } + if (typeof network !== 'string') return false; + const threshold = resolveThreshold(this.read(key), network, coin); + return reached(threshold, row.unit === 'time' ? time : height); + } +} + +function createRegistry() { return new GateRegistry(); } + +// The one registry this process reads. The entry (../gate_registry.js) fills +// it from the part files once, at its own load, and exports the readers. +const registry = createRegistry(); + +module.exports = { UNARMED, UNPINNED, UNITS, KEY_RE, GateRegistry, RegistryMissError, createRegistry, registry }; diff --git a/protocol/reference-impl/consensus/gate_registry/regtest_env.js b/protocol/reference-impl/consensus/gate_registry/regtest_env.js new file mode 100644 index 00000000..883312a9 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/regtest_env.js @@ -0,0 +1,68 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The one regtest arming grammar, applied by the registry when a row is read. + * + * Five gate rows (ROLLCALL, ROLLCALL gates, the two mirror-admission heights + * and the anchor-attest barrier) let a regtest venue arm their regtest entry + * from an environment variable instead of a committed height: that is how one + * venue carries an armed and an inert indexer and shows them binding the same + * row at different blocks. Their modules keep their own resolver functions + * (the exported ones tests drive), but the VALUE a running process applies is + * the registry row, so the environment is parsed here, by the same grammar + * every one of those resolvers uses, each time shared_rows.js arms a row for + * a reader (cached per value, so a refused value is reported once). + * + * UNSET SHIPS INERT. Arming a network commits every BTC indexer on it to a + * wired DOGE peer, so a venue opts in; a BTC-only venue that cannot answer a + * roll-call close is left alone. + * + ********************************************************************/ + +'use strict'; + +/** + * The regtest arming height named by `raw`, the value of one env variable. + * + * Accepted forms, case-insensitive and trimmed: + * armed | genesis | on | true | yes -> armedHeight + * a non-negative integer -> that height, for a venue whose epochs + * should begin above an indexed prefix + * unset | '' | off | inert | false | no | none -> null (INERT) + * Anything else fails CLOSED to null and says so on stderr as a process + * warning (the registry depends on no logger), because a typo that silently + * armed a venue would produce closes nobody meant to drive. + * + * @param {string|undefined} raw the env variable's value + * @param {number} armedHeight the height the armed form resolves to + * @param {string} label the family named on stderr for a refused value + * @param {string} envName the env variable named on stderr + * @returns {number|null} + */ +function regtestHeight(raw, armedHeight, label, envName) { + if (raw === undefined || raw === null) return null; + const s = String(raw).trim().toLowerCase(); + if (s === '' || s === 'off' || s === 'inert' || s === 'false' || s === 'no' || s === 'none') return null; + if (s === 'armed' || s === 'genesis' || s === 'on' || s === 'true' || s === 'yes') return armedHeight; + if (/^\d+$/.test(s)) { + const h = parseInt(s, 10); + if (Number.isFinite(h) && h >= 0) return h; + } + process.emitWarning(label + ': ignoring ' + envName + '=' + JSON.stringify(String(raw)) + + '; regtest stays INERT. Expected a non-negative height, "armed", or "off".', + 'RegtestArmingWarning'); + return null; +} + +module.exports = { regtestHeight }; diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows.js b/protocol/reference-impl/consensus/gate_registry/shared_rows.js new file mode 100644 index 00000000..10ffca25 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows.js @@ -0,0 +1,130 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The gate-row queue: the wrapper every row part file writes into. + * + * The rows themselves live in the part files beside this one. shared_rows_N.js + * hold the SHARED block, the gate rows every consumer of this platform judges: + * the region between their `// SHARED-GATES BEGIN` and `// SHARED-GATES END` + * markers is BYTE-TWINNED into the registry of xchain-sync, xchain-hub, + * xchain-explorer and xchain-sdk, each of which wraps the same bytes in its own + * copy of this queue and replaces only the require line above the markers. + * gates_N.js hold the rows no other repo twins. Every part file calls + * `addGate(key, unit, table)` at column zero, with literal values only, so the + * calls are queued here as the parts load and replayed into the registry the + * assembler hands registerRows(); a function body around 300 rows would grow + * past the readability limit, and column-zero bytes are what the consumers + * can twin without a shared receiver name. + * + * REGTEST ARMING. Five rows let a regtest venue arm their regtest entry from + * an environment variable (the modules' own resolvers document the grammar; + * regtest_env.js carries it for the registry). The block writes those entries + * UNPINNED, the inert default, so it stays data that every consumer can copy; + * the registry stores that committed table and this wrapper arms the entry + * WHEN THE ROW IS READ, from the environment as it stands at that moment. A + * module reading its table at require time therefore sees what its own + * literal saw (a test that sets the variable and re-requires the module sees + * the new value, with no registry purge), and the fingerprint sees the + * environment as it stands when it runs. The bare reading is the block + * literal and the armed reading is the venue's, exactly what the fingerprint + * pinned bare and armed before the rows moved here. + * + ********************************************************************/ + +'use strict'; + +const { UNARMED, UNPINNED } = require('./core.js'); +const { regtestHeight } = require('./regtest_env.js'); + +const queued = []; +function addGate(key, unit, table) { queued.push([key, unit, table]); } + +// key -> { env, label, armedHeight, keys }: the regtest entries a venue arms. +const REGTEST_ARMING = { + 'rollcall_activation.ROLLCALL_ACTIVATION': + { env: 'XC_ROLLCALL_REGTEST_ACTIVATION', label: 'ROLLCALL', armedHeight: 0, keys: ['regtest'] }, + 'rollcall_gates_activation.ROLLCALL_GATES_ACTIVATION': + { env: 'XC_ROLLCALL_GATES_REGTEST_ACTIVATION', label: 'ROLLCALL gates', armedHeight: 0, keys: ['regtest'] }, + 'mirror_admission_activation.MIRROR_ADMISSION_ACTIVATION': + { env: 'XC_MIRROR_ADMISSION_ACTIVATION', label: 'MIRROR ADMISSION', armedHeight: 0, + keys: ['BTC:regtest', 'LTC:regtest', 'DOGE:regtest'] }, + 'mirror_admission_activation.MIRROR_ADMISSION_CONSUMER_ACTIVATION': + { env: 'XC_MIRROR_ADMISSION_ACTIVATION', label: 'MIRROR ADMISSION', armedHeight: 0, + keys: ['BTC:regtest', 'LTC:regtest', 'DOGE:regtest'] }, + 'anchor_reward_activation.ANCHOR_ATTEST_BARRIER_ACTIVATION': + { env: 'XC_MIRROR_ADMISSION_ACTIVATION', label: 'MIRROR ADMISSION', armedHeight: 0, keys: ['regtest'] }, +}; + +// env name -> its reader. Each variable is read BY NAME, once, here: the +// documentation coverage gate resolves `env.NAME` to a doc row and counts a +// computed `env[name]` as a blind spot it ratchets, so the rule's `env` string +// (kept for the warning text) selects a reader instead of indexing the object. +// A rule naming a variable with no reader here is a defect, not an inert row. +const ENV_READERS = { + XC_ROLLCALL_REGTEST_ACTIVATION: (env) => env.XC_ROLLCALL_REGTEST_ACTIVATION, + XC_ROLLCALL_GATES_REGTEST_ACTIVATION: (env) => env.XC_ROLLCALL_GATES_REGTEST_ACTIVATION, + XC_MIRROR_ADMISSION_ACTIVATION: (env) => env.XC_MIRROR_ADMISSION_ACTIVATION, +}; + +// The raw value of the rule's env variable, through its named reader. +function readRuleEnv(rule, env) { + const read = ENV_READERS[rule.env]; + if (!read) throw new Error('REGTEST_ARMING names ' + rule.env + ' but ENV_READERS has no reader for it'); + return read(env); +} + +// The table with its regtest entries armed from `raw`, one env variable's +// value, or the table itself when the venue named nothing (an unset or refused +// value leaves UNPINNED in place). Frozen like the committed row it stands for. +function armed(rule, table, raw) { + const height = regtestHeight(raw, rule.armedHeight, rule.label, rule.env); + if (height === null) return table; + const out = Object.assign({}, table); + for (const k of rule.keys) out[k] = height; + return Object.freeze(out); +} + +// The read overlay: `env` is read at every call, so it follows the process +// environment as it changes, and the result is cached per key against the raw +// string it was armed from, so a refused value warns once per value and a +// steady environment costs one property read per get(). +function regtestArming(env) { + const cache = new Map(); + return function armAtRead(key, table) { + const rule = REGTEST_ARMING[key]; + if (!rule) return table; + const raw = readRuleEnv(rule, env); + const hit = cache.get(key); + if (hit && hit.raw === raw) return hit.value; + const value = armed(rule, table, raw); + cache.set(key, { raw, value }); + return value; + }; +} + +/** + * Registers every queued row into `registry`, in part-file order, as the block + * commits it, and installs the read overlay that arms the regtest entries from + * `env` at the time of each read. + * @param {{addGate: Function, setReadOverlay: Function}} registry + * @param {object} env the process environment (the assembler passes it), or + * a stand-in; read by reference, never copied + */ +function registerRows(registry, env) { + if (env === null || typeof env !== 'object') throw new Error('registerRows: env must be the environment object to arm from'); + for (const [key, unit, table] of queued) registry.addGate(key, unit, table); + registry.setReadOverlay(regtestArming(env)); +} + +module.exports = { addGate, UNARMED, UNPINNED, registerRows, REGTEST_ARMING }; diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows_1.js b/protocol/reference-impl/consensus/gate_registry/shared_rows_1.js new file mode 100644 index 00000000..59c78a94 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows_1.js @@ -0,0 +1,385 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The SHARED block, part 1 of 5: anchor_reward_activation to attest_responsible_widening_activation + * + * One SHARED block part. The region between the two marker lines is + * BYTE-TWINNED into the registry of xchain-sync, xchain-hub, xchain-explorer + * and xchain-sdk: each consumer keeps the same bytes and replaces only the + * require line below with its own queue module. What may live between the + * markers: `addGate(key, unit, table)` calls with LITERAL values (a table, a + * number, a string or literals joined by +, a RegExp, an array), one call per + * row, at column zero, and comments. No require, no computed value, nothing + * from outside the block but addGate, UNARMED and UNPINNED. A regtest entry a + * venue arms from its environment is written UNPINNED here and armed by the + * wrapper at registration (shared_rows.js), so the block stays data. + * + * Rows are grouped by module stem in alphabetical order; a stem's rows keep + * the order the module declared them. Keys never change (I4). + * + ********************************************************************/ + +'use strict'; + +const { addGate, UNARMED, UNPINNED } = require('./shared_rows.js'); + +// SHARED-GATES BEGIN +// anchor_reward_activation +// Per-network activation height, interpreted as the BTC-anchored snapshot_block +// carried by the ANCHOR canonical (NOT the local processing height), so every chain +// + the hub flip the reward-derivation path on the same anchor. +addGate('anchor_reward_activation.ANCHOR_REWARD_ACTIVATION', 'height', { + mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers before this height + testnet: 0, + regtest: 0, +}); + +// The frozen validator anchor-publish reward. This is a CONSENSUS CONSTANT: the hub +// signs it into the XANCPUB attestation and the indexer re-derives it, never from +// the wire. Changing it is itself a flag-day. Kept equal to the hub's historical +// default (ANCHOR_REWARD_PER_PUBLISH = '10.00000000'). +addGate('anchor_reward_activation.ANCHOR_REWARD_AMOUNT', 'constant', '10.00000000'); + +// Archive-reward re-derivation flag-day. Same shape as ANCHOR_REWARD_ACTIVATION, +// gating the ARCHIVE leg: at/above this BTC-anchored snapshot_block the elected +// archive leader emits a publisher-bearing archive head (v1 since the version +// restart, which always carries the PUBLISHER|ATTEST_SIG_COUNT|... tail; the +// retired v6 was that tail bolted onto a tail-less v1), attested +// over an 'anchor_archive' XANCPUB canonical) and the indexer derives the +// anchor_archive reward from those bytes; the key-authenticated +// pushvalidatorrewards rail is rejected for anchor_archive, closing the +// insider-with-key forge surface the per-chain flag-day left open. Below the +// threshold the legacy tail-less archive wire and the push path stand, and a +// publisher-bearing archive head is rejected. +addGate('anchor_reward_activation.ARCHIVE_REWARD_ACTIVATION', 'height', { + mainnet: 963000, // ARMED 2026-07-16, RE-PINNED 2026-08-12 off block 969500 onto the mainnet pre-freeze deploy-train boundary (tip 959,853 on 07-27 at ~144 blocks/day + 21d); deploy every consumer before this height + testnet: 0, + regtest: 0, +}); + +// The frozen archive-publish reward, signed into the archive XANCPUB attestation and +// re-derived by the indexer, never from the wire. Kept equal to the hub's historical +// default (ANCHOR_REWARD_PER_PUBLISH = '10.00000000'). Changing it is itself a flag-day. +addGate('anchor_reward_activation.ARCHIVE_REWARD_AMOUNT', 'constant', '10.00000000'); + +// ANCHOR_REWARD_DERIVE_ACTIVATION (Option C: derive on the BTC side). +// Relocates anchor-reward derivation from the DOGE indexer to the BTC indexer. +// ANCHOR is DOGE-only, but capability staking (the stake source +// createValidatorReward needs) is BTC-only, so DOGE-side derivation silently +// drops every publisher reward (no local stake -> _resolveActiveStakeSourceId +// returns null). At/above this gate: the DOGE indexer stops attempting the +// reward write; the hub inserts one append-only `anchor_reward_attestations` +// row per attested reward tuple after the XANCPUB quorum resolves, mirrored to +// every indexer; and the BTC indexer derives the reward from the mirrored row, +// re-verifying the XANCPUB signatures against its own locally-computed +// oracle_publish/stake set at snapshot_block (the mirror is transport, not +// trust) before materializing validator_rewards there. Below the gate, +// behavior is byte-identical to legacy (DOGE-side write still attempted and +// silently dropped, no BTC-side derivation). +// +// Consensus-relevant (validator_rewards is COLLECT-spendable): must deploy hub +// + all indexers atomically, on the same BTC-anchored snapshot_block space as +// ANCHOR_REWARD_ACTIVATION. It cannot ride the existing 961000/963000 +// boundaries because those are already live (0) on testnet/regtest, which +// would flip this relocation the instant code deploys with no coordinated +// deploy-first-then-flip window, risking a COLLECT-mediated fork mid-upgrade. +// That window is what the table was held inert for; every network is now armed +// at genesis, mainnet by the 2026-09-09 ruling, because there is no pre-flag +// reward history on any of them to flip under a partially-upgraded fleet. +// +// PRE-ARMING BLOCKERS (ALL THREE LANDED 2026-08-13, well before any network +// armed). Three consensus defects sat on the derive path +// and were harmless only while this table was inert; arming mainnet or testnet +// before they landed would have forked the COLLECT rail. Their remedies are now +// in code and are described here because the remedies, not the defects, are what +// a ratifier has to verify deployed fleet-wide before picking a height. +// (1) No mined-anchor proof: the hub wrote the attestation row on mempool +// acceptance, not confirmation, and the mirrored schema carried no DOGE +// txid, so a dropped or reorged anchor left its COLLECT-spendable reward +// intact. CLOSED: the hub holds the row in _deferredRewardAttest until +// _verifyAnchorOnChain binds that exact txid at that exact ANCHOR version +// buried dogeConfirmations deep, the mirrored schema gained +// doge_anchor_txid (idempotent forward migration in xchain-hub AND +// xchain-indexer), and the BTC indexer re-proves mined depth itself +// against the DOGE indexer's getanchorconfirmations federation read +// before createValidatorReward. +// (2) Non-deterministic materialization: attestations mirrored in with no +// block-loop barrier, and derivation keyed on snapshot_block <= +// blockIndex, unrelated to the mirror's arrival time. Two nodes whose +// copies differed derived the same reward at different heights, forking +// the ledger hash for identical BTC blocks. CLOSED by the operator's +// 2026-08-11 ruling (a): derivation is re-keyed onto the fleet-agreed +// mirror-completeness watermark below (ANCHOR_REWARD_MIRROR_MATURITY), +// and a node whose mirror is not provably caught up DEFERS the block +// rather than deriving a partial set. +// (3) The attestation row was never federated: it fanned out only to that +// hub's own indexer subscribers, so each hub held a disjoint subset of +// rows and an indexer derived only what its own hub happened to publish. +// CLOSED: the confirmed write broadcasts the authenticated XANCREWARD +// peer message, and every receiver re-verifies the XANCPUB quorum against +// its OWN oracle_publish set at snapshot_block (and re-proves the anchor +// mined) before writing its copy. The wire is transport, never trust. +// +// PRE-ARMING BLOCKER (4), LANDED 2026-08-24, same class as the three above and +// held to the same rule: remedy in code while this table is inert, operator +// ratifies a height later. The reward LEDGER key could not tell two genuinely +// distinct archive anchors apart. validator_rewards keys on (source_id, +// signing_pubkey_id, reward_type, round_reference), and for 'anchor_archive' +// round_reference is MATCH_BATCH_SEQ - a DENSE counter the hub allocates from +// its own tables, which a wipe-and-replay rebase resets, so the hub reissues +// seq values earlier archive batches already used. The per-chain legs are safe +// by construction (CHECKPOINT_SEQ == snapshot_block, a height that only +// advances). The SIGNED side always distinguished them - the XANCPUB reward +// canonical carries SNAPSHOT_BLOCK and anchor_reward_attestations' +// uq_reward_tuple includes snapshot_block - so the attestation layer knew there +// were two rewards while the ledger conserved one: the pending-attestation NOT +// EXISTS matched round-only and suppressed the second derive outright, and where +// both rows did land the MIN(pubkey) reconcile deleted one real, quorum-attested +// publisher's pay. CLOSED: validator_rewards and anchor_reward_reconcile_log +// carry round_qualifier (snapshot_block for the archive leg, 0 for every other +// reward type, so non-archive rows keep exactly the key they had), +// reward_unique includes it, and the pending join, the reconcile predicate, the +// derive grouping, the reorg restore and xchain-sync's replica-side mirror all +// key on it. +// +// This one has a LEDGER half the other three do not, and it must be verified +// separately before ratification: the columns converge on their own (declared +// with DEFAULTs, so the startup drift reconciler ADDs them), but the UNIQUE KEY +// does NOT - reconcileTableIndexes will not DROP an index it did not create, so +// an AGED database keeps the old four-column reward_unique and merely logs a +// drift warning each boot. So a node can run the new binary and still +// re-collapse two distinct archive rewards inside its own index. Every node +// must therefore carry BOTH the build AND the applied migration +// (xchain-indexer/src/sql/migrations/ +// 2026-08-24-validator-rewards-round-qualifier.sql) before any mainnet/testnet +// height is ratified. xchain-sync reads and writes the same replicated table, +// so its build has to be qualifier-aware in the same deploy. +// +// THE HUB'S OWN LEDGER now carries the same qualifier, which it did not when the +// paragraph above was written. src/anchor_reward_key.js is the vendored twin of the +// indexer rule (byte-checked by test/unit/anchorRewardKeyTwinParity.test.js), the +// hub-local validator_rewards.uq_reward includes round_qualifier, and the three sites +// that key on the reduced identity read it: the cross-pubkey dedup guard, the follower +// co-sign cross-check and the archive batch_seq stamp. The hub table is hub-local +// (xchain-sync's replicated validator_rewards is indexer-owned), so this half needs no +// fleet coordination, but an AGED hub still needs runMigrations to widen the key and +// backfill the pre-column archive rows before its verdicts can be trusted. +// +// PRE-ARMING DEPLOY STEP (already fixed in code): a derived reward earns at +// the checkpoint's snapshot_block but materializes at a later BTC block, and +// a reorg delete scoped only on the earn-block leaves a COLLECT-spendable +// reward a from-genesis replay has not derived yet. +// validator_rewards now also carries derive_block_index, and rollback deletes +// on both keys. On the INDEXER side the schema half needs no fleet coordination: +// the columns and the index are declared in xchain-indexer/src/sql/ +// validator_rewards.sql and .../anchor_reward_reconcile_log.sql, and the startup +// drift reconciler converges them before runMigrations runs, so any node that +// boots this build has them (the dated migration +// 2026-08-12-validator-rewards-derive-block-index.sql remains the explicit apply +// path, and the runner baselines it once that shape is present). What must +// actually be true fleet-wide before ratifying a mainnet/testnet height is the +// BINARY half: every node running a build whose rollback scopes the delete on +// both keys. A node on an older build has the columns and still scopes on the +// earn-block alone, and forks the COLLECT rail after a reorg. The migration +// ledger never enforced that; the deploy does. +// +// TESTNET IS ARMED AT 0 (operator ruling 2026-08-11, applied 2026-08-14). The +// deploy-first-then-flip window this table was held null for is a MAINNET +// concern: mainnet carries live COLLECT-spendable history, so flipping under a +// partially-upgraded fleet could fork the rail. Testnet was re-genesised with no +// pre-flag history, so there is no legacy set to diverge from and no mid-upgrade +// window to protect; what testnet has instead is the only chance to run the +// relocated derive path (hub attestation write, XANCREWARD federation, BTC-side +// re-verification, mirror-maturity deferral) on a real multi-host network before +// mainnet ratifies a height. +// +// MAINNET IS ARMED AT 0 (operator ruling 2026-09-09). The deploy-first-then-flip +// window above assumed live COLLECT-spendable mainnet history to fork; there is +// none. Mainnet carries 0 anchor reward attestations and 0 validator_rewards rows +// on any chain (measured 2026-09-09), so relocating the derive reinterprets no +// existing reward, and the from-genesis OLD-vs-ON replay per chain is the witness. +// +// TESTNET DEPLOY ORDER, unchanged by the arming: every testnet hub and indexer +// must carry the schema of BOTH +// (2026-08-12-validator-rewards-derive-block-index.sql and +// 2026-08-13-anchor-reward-attestations-doge-anchor-txid.sql) before it processes +// an anchor, since a node on the old schema cannot record the materialization +// block or bind the mined-anchor txid. Read that as a DEPLOY requirement, not a +// ledger one: on a BTC indexer the derive columns arrive with the build (see the +// pre-arming note above), so what to check before an anchor is which build each +// host runs, not which rows its schema_migrations happens to hold. +addGate('anchor_reward_activation.ANCHOR_REWARD_DERIVE_ACTIVATION', 'height', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 anchor reward attestations, 0 validator_rewards rows, measured 2026-09-09) + testnet: 0, // ARMED at genesis 2026-08-14 per the 2026-08-11 operator ruling; see the testnet note above + regtest: 0, +}); + +// The fleet-agreed mirror-completeness watermark, in BTC blocks (operator ruling (a), +// 2026-08-11, settling AML #4172). +// +// Keying on snapshot_block alone matures a mirrored attestation the instant snapshot_block <= the BTC +// block being processed. snapshot_block is the height the XANCPUB signing set was resolved +// at, and it is ALREADY IN THE PAST when the row is written: the hub writes only after the +// DOGE anchor is buried dogeConfirmations deep, after a failover ladder that can hand the +// publish to a later hub, and after the XANCREWARD federation hop. The maturity key was +// therefore unrelated to the row's arrival, so two nodes whose mirrors differed by one row +// derived the same reward at different BTC heights and forked the ledger hash for identical +// blocks. A hub_db_sync barrier alone cannot fix that: snapshot_block is not a maturity key +// for a mirror whose arrival is governed by DOGE confirmation and hub failover. +// +// Re-keyed: a row matures at snapshot_block + ANCHOR_REWARD_MIRROR_MATURITY, a frozen +// constant every node applies identically, sized to exceed the worst-case arrival lag (60 +// DOGE confirmations is ~1h, plus the anchor failover ladder, plus federation). The height +// is only half the barrier. The other half is fail-closed: a node whose attestation mirror +// is not provably caught up DEFERS the block (wait-then-retry, never a partial-set commit; +// see HubDbSync.waitForAnchorAttestationSync), so every node either derives the +// identical set at the identical height or does not advance at all. Changing this value +// moves the block a reward materializes at, so it is a hashed value: it is frozen with the +// activation map above and a change needs its own flag-day. +addGate('anchor_reward_activation.ANCHOR_REWARD_MIRROR_MATURITY', 'constant', 144); // ~24h of BTC blocks + +// The DOGE burial depth deriveAnchorRewards() requires before it will mint a mirrored +// attestation's reward. Frozen HERE, beside the maturity watermark and the activation map, +// because it is a LEDGER input: it decides the BTC height at which a reward materializes, +// so two nodes applying different depths derive the same reward at different heights and +// fork the ledger hash for identical blocks (the same failure ANCHOR_REWARD_MIRROR_MATURITY +// above was re-keyed to close). +// +// It must NOT be read from the coin registry as coins.DEFAULT_CONFIRMATIONS.DOGE, which is +// the wrong authority twice over: the registry classifies `confirmations` as display / +// operator-tunable depth and deliberately leaves it OUT of the pinned consensus subset +// (coins/index.js consensusSubset, coins.test.js NON_CONSENSUS_TOP_LEVEL_KEYS), so a node +// bundling a divergent value forks the derive height while verifyConsensusPin() passes +// clean; and coins.resolveConfirmations() lets an operator move the same number per node +// via XCHAIN_CONFIRMATIONS_DOGE. A ledger input cannot be sourced from a field nothing pins +// and anyone may tune, so the block-transaction path takes it from here and the registry +// field means what it is classified as: local, hub-side trust policy. +// +// The value equals the registry default at rest and a drift alarm in +// test/unit/anchorRewardDerive.test.js fails if the two ever part. Changing it moves the +// block a reward materializes at, so it is frozen with the activation map above and a +// change needs its own flag-day. +addGate('anchor_reward_activation.ANCHOR_REWARD_DOGE_MIN_CONFIRMATIONS', 'constant', 60); // DOGE confirmations, ~1h + +// Sized against the hub's whole MEASURED write-lag envelope, not the DOGE burial alone. The +// envelope is about 15 h: up to 6 BTC blocks of checkpoint age at flush (~1 h), the +// publisher's deferred-write queue TTL of 6 h, a receiver hub's re-proof through that SAME +// queue for up to 6 h more, and ~2 h of raw-stamp skew on the networks that are off +// median-time-past. +// +// 64800 s covers that envelope with 3 h of headroom and still opens 6 h before a nominal +// 144-block span, so a +2 h stamp is absorbed entirely. The earlier 21600 s figure was sized +// on the DOGE burial alone and sat BELOW the publisher's own 6 h queue TTL, so it was +// corrected by measurement. A 144-block stretch shorter than 18 h is a three-sigma event and +// falls back to today's wait through the min(), which is the right way for a fail-closed +// gate to fail. Changing this value moves the block a barrier opens at, so it is frozen with +// the activation map below and a change needs its own flag-day. +addGate('anchor_reward_activation.ANCHOR_ATTEST_ARRIVAL_MARGIN_S', 'constant', 64800); // 18 h + +// Per NETWORK, not per (coin, network), because this member is BTC-only by its call-site +// guard and a second key would be dead weight. Nothing hashed moves across this height: two +// nodes on either side derive the identical set at the identical height and differ only in +// WHEN they get there. The height exists because a rolling deploy would otherwise leave the +// early-opening node alone in carrying a weaker completeness guarantee, and one map removes +// that window. +addGate('anchor_reward_activation.ANCHOR_ATTEST_BARRIER_ACTIVATION', 'height', { + mainnet: null, // INERT under the 2026-08-29 mainnet write hold + testnet: null, // SIZED AT THE CUT from the measured tip plus the roll window + regtest: UNPINNED, // shares the family's arming seam so one venue lever arms both +}); + +// archive_rollback_author_scope_activation +// Per-network activation, interpreted against the block index a rollback targets, +// on the DOGE scale (see the KEYED ON note above). +addGate('archive_rollback_author_scope_activation.ARCHIVE_ROLLBACK_AUTHOR_SCOPE_ACTIVATION', 'height', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 archive chunks, measured 2026-09-09), and ARCHIVE_BATCH_AUTHOR is 0 there too, so the precondition holds + testnet: 67915000, // testnet runs a public chain with live history, so 0 would be retroactive rather than a flag day; TDOGE tip 67881714 on 2026-09-09 + 33286 blocks @1440/day = ~23 days, to ride the v0.17.0 train + regtest: 9999999999, // INERT sentinel: keeps the flag-day-off control path drivable on a throwaway stack +}); + +// The joins that bind an orphaned chunk to its own head's author. Spliced into +// the reset UPDATE by both the source indexer and the replica so the two cannot +// drift; `c` is the orphaned chunk and `p` the surviving head, as named there. +addGate('archive_rollback_author_scope_activation.ARCHIVE_AUTHOR_SCOPE_JOIN_SQL', 'constant', 'JOIN actions pact ON pact.action_index = p.action_index ' + + 'JOIN index_addresses padr ON padr.id = pact.source_id ' + + 'JOIN actions cact ON cact.action_index = c.action_index ' + + 'JOIN index_addresses cadr ON cadr.id = cact.source_id AND cadr.address = padr.address '); + +// attest_relay_activation +// Per-network activation height, interpreted as the BTC-anchored SNAPSHOT_BLOCK +// carried by the relay canonical (NOT the local processing height), so BTC, LTC, +// DOGE and the hub all flip the relay legs on one anchor. +addGate('attest_relay_activation.ATTEST_RELAY_ACTIVATION', 'height', { + mainnet: 963000, // ARMED 2026-07-30, RE-PINNED 2026-08-12 off block 969500 with the rest of the coordinated mainnet activation cohort; deploy every indexer + hub before this height + testnet: 0, + regtest: 0, +}); + +// attest_relay_reject_slot_activation +// Per-network activation, interpreted against the LANDING block's consensus +// timestamp (data['BLOCK_TIME']) on the home chain. +addGate('attest_relay_reject_slot_activation.ATTEST_RELAY_REJECT_SLOT_ACTIVATION', 'time', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 attestations, measured 2026-09-09) + testnet: 0, + regtest: 0, +}); + +// attest_response_mirror_activation +// Per-network activation height (LOCAL COPY, parity-tested). Compared against +// the ATTEST v0 request's own BTC block_index (the v3's, for a relayed request). +addGate('attest_response_mirror_activation.ATTEST_RESPONSE_MIRROR_ACTIVATION', 'height', { + mainnet: null, // INERT: operator-owned height, unratified. The legacy on-chain response path runs byte for byte. + testnet: 151324, // ARMED 2026-09-07 at the chain tip on the operator ruling: exercising the mirror on testnet is the point of this train, so it activates on deploy rather than waiting on a future height. + regtest: 0, // ARMED at genesis so the e2e mirror venue exercises the mirror path +}); + +// attest_responsible_widening_activation +// Per-network activation height (LOCAL COPY, parity-tested). Compared against +// the ATTEST v0 request's own BTC block_index. +// MAINNET IS ARMED AT 0 by the 2026-09-09 ruling. Widening only changes who may sign a +// round that has already failed to finalize, and 0 attestations have ever been recorded on +// any mainnet chain (measured 2026-09-09), so no admitted request is reinterpreted; the +// from-genesis OLD-vs-ON replay per chain is the witness. +addGate('attest_responsible_widening_activation.ATTEST_RESPONSIBLE_WIDENING_ACTIVATION', 'height', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 attestations, measured 2026-09-09) + testnet: 150780, // ARMED 2026-09-02. Tip was 150760 at 17:08Z running 20 min/block, so ~20 blocks (~6.5h). Sized to OUR fleet's deploy wave, not to the community's, and the SAFETY comes from deploy ORDER rather than from this margin: only an upgraded hub can PRODUCE a widened ATTEST v1, so indexers upgraded before hubs leaves no divergence window even if the height arrives mid-deploy. + regtest: 0, // ARMED at genesis so the e2e venue exercises the ladder +}); + +// The ladder's own constants (LOCAL COPY, parity-tested). +// +// FROZEN, and deliberately NOT the hub's operator-tunable ATTESTATION_CONFIRMATIONS / +// ATTESTATION_LEADER_ROTATION_BLOCKS. Those two shape only which hub goes first, which no +// validator checks; these shape WHO MAY SIGN, which every indexer checks. Sourcing them from +// per-hub config would let one operator's tuning fork the set. (Above +// ATTEST_ZERO_CONF_ACTIVATION the hub's confirmations knob is inert anyway: the hub serves +// at the tip, AttestationRound.confirmationsFor, and its ladders start where this one does.) +// +// PROPORTIONAL TO THE REQUEST'S OWN WINDOW, not a fixed block count, and that choice is the +// whole reason this ladder is usable. A fixed window sized to sit after leader rotation's cap of +// 3 never fires at all inside a short deadline: the case this exists for (deadlineBlocks 10, +// confirmations 3) leaves 7 serviceable blocks, and rotation alone consumes every one of them. +// So the serviceable span is divided into `maxSlots + 1` equal segments, one per widening level, +// exactly as attestation_escalation.modelIndex divides the same span across approved models. A +// contract that asks for a long window gets a long grace period before its set widens; one that +// asks for a short window gets a proportionally short one, and both still widen. +// +// maxSlots 2 bounds how far the pool can grow: enough to absorb two dead members of a set, small +// enough that the deterministic assignment stays the dominant property. The first segment is +// always the unwidened set, so a healthy round never sees a widened set at all. +addGate('attest_responsible_widening_activation.ATTEST_RESPONSIBLE_WIDENING', 'constant', { + confirmations: 3, + maxSlots: 2, +}); +// SHARED-GATES END diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows_2.js b/protocol/reference-impl/consensus/gate_registry/shared_rows_2.js new file mode 100644 index 00000000..3b03a0b4 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows_2.js @@ -0,0 +1,398 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The SHARED block, part 2 of 5: attest_responsible_widening_activation to rollcall_activation + * + * One SHARED block part. The region between the two marker lines is + * BYTE-TWINNED into the registry of xchain-sync, xchain-hub, xchain-explorer + * and xchain-sdk: each consumer keeps the same bytes and replaces only the + * require line below with its own queue module. What may live between the + * markers: `addGate(key, unit, table)` calls with LITERAL values (a table, a + * number, a string or literals joined by +, a RegExp, an array), one call per + * row, at column zero, and comments. No require, no computed value, nothing + * from outside the block but addGate, UNARMED and UNPINNED. A regtest entry a + * venue arms from its environment is written UNPINNED here and armed by the + * wrapper at registration (shared_rows.js), so the block stays data. + * + * Rows are grouped by module stem in alphabetical order; a stem's rows keep + * the order the module declared them. Keys never change (I4). + * + ********************************************************************/ + +'use strict'; + +const { addGate, UNARMED, UNPINNED } = require('./shared_rows.js'); + +// SHARED-GATES BEGIN +// attest_responsible_widening_activation (continued) +// STAGE 2, selected by ATTEST_ZERO_CONF_ACTIVATION on the request block (LOCAL COPY, +// parity-tested). Two things change and one does not: +// +// startOffset 0: the ladder starts AT the request block, where the hub now starts its +// own leader and model ladders. It is named startOffset rather than confirmations because +// it is a ladder-start offset and the hub's ATTESTATION_CONFIRMATIONS is a different knob. +// +// headroom 1: one extra slot from step 0, before any segment has elapsed. The failure +// this exists for, measured on testnet, is one member that can never sign: an old key seated in 1 +// of 7 slots stalls every 3-of-7 draw that includes it at 2 of 3 until the ladder opens, +// a third of the deadline window later. Headroom makes such a round finalize inside the +// first segment with no clock at all, and keeps the assignment deterministic. The +// assigned set (the persisted RESPONSIBLE_SET_JSON, the missed_count charge) is still +// the unwidened slice: headroom widens who may EARN, never who is CHARGED. +// +// maxSlots 2 is kept, so the set can reach redundancy + 3: headroom plus two ladder +// steps for two dead members. +addGate('attest_responsible_widening_activation.ATTEST_RESPONSIBLE_WIDENING_V2', 'constant', { + startOffset: 0, + headroom: 1, + maxSlots: 2, +}); + +// attest_zero_conf_activation +// Per-network activation height (LOCAL COPY, parity-tested). Compared against +// the ATTEST v0 request's own BTC block_index. +addGate('attest_zero_conf_activation.ATTEST_ZERO_CONF_ACTIVATION', 'height', { + mainnet: null, // INERT: operator-owned height, unratified. Ratified only after the mirror arms there. + testnet: 151800, // SIZED 2026-09-08 (tip 151483 at 07:32Z, about 5 blocks/h): above the 151324 mirror floor and past the v0.16.0 indexer-then-hub roll; keyed on the request block. + regtest: 0, // ARMED at genesis so the e2e mirror venue exercises the flip +}); + +// checkpoint_commitment_activation +// Per-network activation height, interpreted as the BTC-anchored snapshot_block +// carried by the checkpoint/ANCHOR canonical (NOT the local processing height), so +// every chain + the hub flip the signed shape on the same anchor. +addGate('checkpoint_commitment_activation.CHECKPOINT_COMMITMENT_ACTIVATION', 'height', { + mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers (+ sdk/explorer/sync copies) before this height + testnet: 146000, // ARMED 2026-07-22: first BTC-testnet anchor past all three STATE_COMMITMENT testnet thresholds; was 0, which forced the SPV root suffix from testnet genesis before the indexer computes roots, so the hub refused to sign every testnet checkpoint + regtest: 0, +}); + +// cross_chain_royalty_activation +// Per-network activation height, interpreted as the BTC-anchored snapshot_block +// carried by the XMATCH canonical (NOT the local processing height), so every +// chain + the hub flip the match format on the same anchor. +addGate('cross_chain_royalty_activation.CROSS_CHAIN_ROYALTY_ACTIVATION', 'height', { + mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers before this height + testnet: 0, + regtest: 0, +}); + +// equivocation_header +// Per-network activation height (LOCAL COPY of the canonical map in +// xchain-documentation/protocol/constants.js, kept equal by the cross-service +// regression suite). Keyed on the BTC-anchored snapshot_block, NOT the local +// processing height, so every chain + the hub flip on the same anchor. +addGate('equivocation_header.EQUIV_HEADER_ACTIVATION', 'height', { + mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers (+ sdk/explorer/sync copies) before this height + testnet: 0, + regtest: 0, +}); + +// Fixed per-engine tag (spec §4.1.1). One slashable canonical family per engine. +addGate('equivocation_header.ENGINE_TAGS', 'constant', { + DEX: 'XDEX', + XCALL: 'XCALL', + ATTEST: 'XATTEST', + ORACLE: 'XORACLE', + // PRICE batches. A DISTINCT tag from ORACLE, not a reuse: a batch canonical + // carries first_round/last_round and no scalar `round`, and SLASH v0 reads + // `round` out of an ORACLE-tagged content to judge equivocation, skipping its + // distinct-rounds guard when either side lacks one. Under a shared tag an + // honest validator that signed one per-round consensus canonical and one batch + // at the same BTC anchor would be provably equivocating, for a full bond burn plus permanent + // capability disqualification. The batch ROUND_ID is + // `||` (pipes are safe here; equivKey treats + // the round id as opaque), so two honest batches that split one window + // differently do not collide on one key either. + ORACLE_BATCH: 'XORACLEB', + CHECKPOINT: 'XCHECKPOINT', + CONFIG: 'XCONFIG', + NODEPROOF: 'XNODEPROOF', + // ROLLCALL presence proofs. Namespacing ONLY, exactly like XNODEPROOF: the + // tag is deliberately absent from SLASH's ENGINE_CAPABILITY map, so no + // ROLLCALL canonical is a slashable family. Several valid ROLLCALLs per + // epoch are expected (a leader's, sweepers', self-publishes), every one + // carrying signatures over the SAME canonical for that epoch, so two of + // them are never conflicting content for one key. ROUND_ID is the BTC + // EPOCH_HEIGHT in decimal, VIEW is 0. + ROLLCALL: 'XROLLCALL', + // Cross-chain bridge transfer records. ROUND_ID is the transfer_id, VIEW is the live + // PBFT view on the hub and the row's finalizing_view on an indexer. Mapped to the + // cross_chain capability in SLASH's ENGINE_CAPABILITY: a forged transfer record directs + // value, so two conflicting canonicals for one transfer_id must be slashable. + BRIDGE: 'XBRIDGE', + // Per-token policy snapshots (allow list, block list, sleep) carried from an origin row + // to every bridged copy. A DISTINCT tag from BRIDGE, not a reuse: the two canonicals + // share no field layout, and SLASH judges equivocation within one tag family, so one tag + // over both would make a validator that signed one transfer and one snapshot at the same + // round id provably equivocating. ROUND_ID is the snapshot_id. + POLICY: 'XPOLICY', +}); + +// list_edit_resolution_activation +// Per-chain activation heights, interpreted against the chain's own +// block_index. Pinned to the same pre-freeze activation train as +// BET_STATUS_STATE_HASH_ACTIVATION in stateHash.js: BET members-only markets +// are the loudest consumer of a mutable list, so the two flip together and +// operators reason about one boundary. RE-PINNED 2026-07-27 against live tips +// in lockstep with that map. These two maps must stay equal value for value; +// the explorer vendors this one byte-identically behind a guard test. CONFIRM +// again at train assembly: they are only as good as the day measured. +addGate('list_edit_resolution_activation.LIST_EDIT_RESOLUTION_ACTIVATION', 'height', { + 'BTC:mainnet': 963000, // tip 959,853 (2026-07-27) + 21d @144/day = 962,877 + 'LTC:mainnet': 3162000, // tip 3,149,481 + 21d @576/day = 3,161,577 + 'DOGE:mainnet': 6338000, // tip 6,307,307 + 21d @1440/day = 6,337,547 + // Testnet is genesis-active as of the 2026-08-10 fresh testnet genesis: the + // chain restarts at firstBlock (BTC 147500 / LTC 4855000 / DOGE 67815000) with + // no pre-rule history to preserve. Kept value-equal to + // CARET_REF_STRICT_ACTIVATION and BET_STATUS_STATE_HASH_ACTIVATION, which CI + // asserts, so all three moved in this one change. + 'BTC:testnet': 0, + 'LTC:testnet': 0, + 'DOGE:testnet': 0, + regtest: 0, // armed from genesis: fresh regtest stacks exercise list edits end to end +}); + +// mirror_admission_activation +// How far ahead of the producer's observed admission tip a row is stamped, in BLOCKS of each +// chain in its map. Not a new number: producers already size their forward margin as 4 blocks +// of the gating chain and then CONVERT it to seconds. On the admission axis the conversion is +// deleted, which is why an unknown chain needs no nominal block interval here at all. +addGate('mirror_admission_activation.ADMIT_MARGIN_BLOCKS', 'constant', { + default: 4, + attestation_responses: 1, // their 120 s forward margin was chosen to be as SHORT as propagation allows + oracle_prices: 1, // effective_at stays the economic filter; admission is what the barrier certifies + anchor_reward_attestations: 144, // the existing ANCHOR_REWARD_MIRROR_MATURITY, already frozen fleet-wide +}); + +// A row may never be admissible at a block that already exists, or a producer could backdate +// a row into a block its peers have already committed. +addGate('mirror_admission_activation.ADMIT_MIN_FUTURE_BLOCKS', 'constant', 1); + +// The follower's upper bound, PER CHAIN, sized so each chain's height window spans the same +// 3600 s the existing absolute effective_time ceiling already allows: ceil(3600 / interval). +// +// A flat block count here would be a silent tightening. Six blocks is an hour on BTC but six +// minutes on DOGE, so a flat [tip + 1, tip + 6] would collapse clock-skew tolerance from +// 3600 s to 360 s on DOGE and refuse honest rows between hubs whose tips differ by three blocks. +addGate('mirror_admission_activation.ADMIT_MAX_FUTURE_BLOCKS', 'constant', { + BTC: 6, + LTC: 24, + DOGE: 60, + default: 6, +}); + +/* + * TWO maps, one module, with an ordering rule that is the whole point: every PRODUCER height is + * sized strictly BELOW its CONSUMER height for the same key, so no row is ever produced legacy + * and read modern. Get that backwards and a consumer above its height reads an admission column + * the producer below its own height never wrote, and binds nothing. + * + * Keyed by (coin, network), not by network alone. A single per-network height cannot arm a + * family that binds on every chain: one number is an LTC height on an LTC indexer and a BTC + * height on a BTC indexer, so the two legs of one cross-chain match would cross the flag day at + * unrelated instants. The 'COIN:network' key shape is established precedent. + * + * Mainnet is null under the 2026-08-29 write hold. Testnet is sized at the release cut from the + * measured tip plus the roll window plus slack, per key. The v7 HUB_SCHEMA_VERSION roll + * completes BEFORE any network's activation height: the heights map rides frames carrying no + * schema_version, so a v7 indexer above the activation against a v6 hub would see no heights at + * all and defer forever under the fail-closed rule. + */ +addGate('mirror_admission_activation.MIRROR_ADMISSION_ACTIVATION', 'height', { + 'BTC:mainnet': null, + 'LTC:mainnet': null, + 'DOGE:mainnet': null, + 'BTC:testnet': null, // SIZED AT THE CUT, strictly below the consumer height for this key + 'LTC:testnet': null, + 'DOGE:testnet': null, + 'BTC:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration + 'LTC:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration + 'DOGE:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration +}); + +addGate('mirror_admission_activation.MIRROR_ADMISSION_CONSUMER_ACTIVATION', 'height', { + 'BTC:mainnet': null, + 'LTC:mainnet': null, + 'DOGE:mainnet': null, + 'BTC:testnet': null, // SIZED AT THE CUT, strictly above the producer height for this key + 'LTC:testnet': null, + 'DOGE:testnet': null, + 'BTC:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration + 'LTC:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration + 'DOGE:regtest': UNPINNED, // ARMS by XC_MIRROR_ADMISSION_ACTIVATION at registration +}); + +addGate('mirror_admission_activation.MIRROR_ADMISSION_REGTEST_ENV', 'constant', 'XC_MIRROR_ADMISSION_ACTIVATION'); + +addGate('mirror_admission_activation.MIRROR_ADMISSION_REGTEST_ARMED_HEIGHT', 'constant', 0); + +// A chain code is a closed vocabulary: upper-case letters and digits, nothing else. The +// injectivity argument rests on that, so the check lives here and not only in a test. +addGate('mirror_admission_activation.CHAIN_CODE_RE', 'constant', /^[A-Z0-9]{1,10}$/); + +// Canonical base-10 spelling of a non-negative integer: digits only, no sign, no leading +// zeros. The rule the hub's lib/canonical_int.js applies to its other signed integers, +// restricted to non-negative because a height never is. +addGate('mirror_admission_activation.CANONICAL_HEIGHT_RE', 'constant', /^(?:0|[1-9][0-9]*)$/); + +// One nullable BIGINT UNSIGNED column per chain the federation serves (C28), spelled +// `admit_block_` in every mirror table's DDL. The hub writes them at +// finalization and every mirror client reads them back to rebuild the signed field, so the +// list lives in the twin rather than on one side: a chain the hub writes and an indexer does +// not read back is a row every indexer refuses (the rebuilt field misses a chain and no +// signature verifies), fail-closed but still an outage. Adding a chain adds it here and in +// the mirror .sql twins; it does NOT make rows signed before that chain existed admissible +// on it (C38), which is why the map is read from the columns actually set and never from +// this list. +addGate('mirror_admission_activation.ADMIT_COLUMN_CHAINS', 'constant', ['BTC', 'LTC', 'DOGE']); + +// price_batching_floor_activation +// Per-network pre-batch era floor, as a unix-second block time. 0 (or an +// absent/unknown network) means the barrier applies at every block. A +// ':' key wins over the bare network key, so one chain's rail +// start can differ from its siblings' without splitting the map. +addGate('price_batching_floor_activation.PRICE_BATCHING_FLOOR_ACTIVATION', 'time', { + mainnet: 0, + testnet: 0, + regtest: 0, +}); + +// price_pair_activation +// Ticker-side bounds either side of the gate (LOCAL COPY, see header). +addGate('price_pair_activation.PRICE_PAIR_TICKER_MAX_LEGACY', 'constant', 5); + +addGate('price_pair_activation.PRICE_PAIR_TICKER_MAX_WIDE', 'constant', 6); + +// Per-network activation TIME (LOCAL COPY of the canonical map in +// xchain-documentation/protocol/constants.js). Keyed on the action's own block time. +// +// ARMED at genesis on every network. Mainnet was ruled on 2026-09-09: no PRICE action has +// ever been indexed on any mainnet chain (measured 2026-09-09), so the widened ticker bound +// reinterprets nothing and the from-genesis OLD-vs-ON replay is the witness. Arming at 0 +// rather than at a launch instant is what keeps LTC/DOGE native-coin fees payable from the +// first mainnet block that carries one; the contract-era stamp 1786060800 (2026-08-07) +// would have left them unpayable up to that instant. +addGate('price_pair_activation.PRICE_PAIR_WIDEN_ACTIVATION', 'time', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 PRICE actions, measured 2026-09-09) + testnet: 0, + regtest: 0, +}); + +// Pre-built per-bound matchers. Anchored, uppercase-only, and without the /g flag +// so .test() carries no lastIndex state between calls. +// {3,5}: PRICE_PAIR_TICKER_MAX_LEGACY wide; the two move together. +addGate('price_pair_activation.PRICE_PAIR_RE_LEGACY', 'constant', /^[A-Z]{3,5}\/[A-Z]{3,5}$/); + +// {3,6}: PRICE_PAIR_TICKER_MAX_WIDE wide; the two move together. +addGate('price_pair_activation.PRICE_PAIR_RE_WIDE', 'constant', /^[A-Z]{3,6}\/[A-Z]{3,5}$/); + +// price_scale_activation +// Decimal-side bound in force at/above the gate. The producers' bcformat width. +addGate('price_scale_activation.PRICE_SCALE_MAX_DECIMALS', 'constant', 8); + +// Per-network activation TIME, keyed on the action's own block time. +// +// ARMED at genesis on every network, mainnet by the 2026-09-09 ruling on the measurement +// the header records (0 PRICE actions ever indexed on any mainnet chain). +addGate('price_scale_activation.PRICE_SCALE_ACTIVATION', 'time', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 PRICE actions, measured 2026-09-09) + testnet: 0, + regtest: 0, +}); + +// The two price-value matchers. Anchored and without the /g flag so .test() +// carries no lastIndex state between calls. +// +// LEGACY is byte-for-byte the pattern both v0 ingest sites carry today; it is +// what keeps a below-gate replay identical, so it is never "tidied". +addGate('price_scale_activation.PRICE_VALUE_RE_LEGACY', 'constant', /^[0-9]+(\.[0-9]+)?$/); + +// {1,8}: PRICE_SCALE_MAX_DECIMALS wide; the two move together. +addGate('price_scale_activation.PRICE_VALUE_RE_CANONICAL', 'constant', /^(0|[1-9][0-9]*)(\.[0-9]{1,8})?$/); + +// price_sig_tally_activation +// Per-network activation height (LOCAL COPY of the canonical map in +// xchain-documentation/protocol/constants.js). Keyed on the round's BTC-anchored +// BTC_BLOCK_HEIGHT, NOT the landing chain's local height, so the hub and the BTC, +// LTC and DOGE indexers all flip on the same anchor. +// +// mainnet is ARMED to 963000, the one BTC-height boundary a whole family of +// cross-chain-verdict gates now shares (retraction signing, archive reward, +// attest relay and the hub's governance snapshot), so operators reason about +// one boundary rather than five. That cohort was RE-PINNED 2026-08-12 off an +// earlier 969500 pin: 969500 was derived alongside a TIME anchor that has since +// been repinned twice and now sits at 1786060800 (2026-08-07), which left the +// height half ~8 weeks behind the time half of the same ratified flag-day set. +// 963000 is the pre-freeze train boundary already armed for BTC:mainnet in stateHash.js, +// caret_ref_strict_activation.js and list_edit_resolution_activation.js (tip +// 959,853 on 2026-07-27 at ~144 blocks/day + 21 days), so this reuses a ratified +// boundary rather than minting a new one. Deliberately NOT the nearer 961000 +// anchor, whose train shipped 2026-07-23 and whose BTC anchor (~2026-08-04) has +// already passed: a height in the past is not a flag day at all. 963000 leaves +// the usual "deploy every consumer before this era" runway. +// +// testnet/regtest activate at genesis (same convention as +// STAKE_WEIGHTED_QUORUM_ACTIVATION and ATTEST_ADMISSION_ACTIVATION, which are +// also verdict-changing): the test venues run the corrected tally from block 0. +addGate('price_sig_tally_activation.PRICE_SIG_TALLY_ACTIVATION', 'height', { + mainnet: 963000, // ARMED, RE-PINNED 2026-08-12 off 969500 onto the shared pre-freeze train boundary; deploy ALL indexers + hubs before this height + testnet: 0, + regtest: 0, +}); + +// retraction_signing_activation +// Per-network activation, interpreted as a BTC-anchored snapshot_block era. +addGate('retraction_signing_activation.RETRACTION_SIGNING_ACTIVATION', 'height', { + mainnet: 963000, // ARMED 2026-07-16, RE-PINNED 2026-08-12 off 969500 onto the shared pre-freeze train boundary (tip 959,853 on 07-27 at ~144 blocks/day + 21d); deploy every consumer before this era + testnet: 0, + regtest: 0, +}); + +// rollcall_activation +// Per-network BTC height at/above which ROLLCALL epochs exist at all. +// MAINNET ARMS AT 0 by the 2026-09-09 ruling: eviction can only reinterpret a chain that +// has validators to evict, and mainnet carries 0 validators, 0 stakes and 0 roll-calls +// (measured 2026-09-09), so every epoch below the tip closes empty and the from-genesis +// OLD-vs-ON replay per chain is the witness. null is still a legitimate value here (regtest +// holds it until the venue opts in), so every read MUST go through the Number.isFinite +// guard below: a bare `height >= ROLLCALL_ACTIVATION[network]` would arm a null network at +// height 0, since `0 >= null` is true in JS. +addGate('rollcall_activation.ROLLCALL_ACTIVATION', 'epoch', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 validators, 0 stakes, 0 roll-calls, measured 2026-09-09) + testnet: 151200, // 1008 x 150 = 144 x 1050; tip was 150400 on 2026-08-30, ~5.5 days out + regtest: UNPINNED, // ARMS AT 0 when the venue sets XC_ROLLCALL_REGTEST_ACTIVATION +}); + +// The documented regtest arming height: genesis. It is a multiple of the +// 30-block regtest interval, so epoch 0 is a real epoch and the first close is +// not skipped. This is the height a regtest venue arms AT, not a height it is +// armed at by default -- see resolveRegtestActivation for why the default is +// inert and how a venue opts in. +addGate('rollcall_activation.ROLLCALL_REGTEST_ARMED_HEIGHT', 'constant', 0); + +// The one environment variable this module reads, and only ever for regtest. +addGate('rollcall_activation.ROLLCALL_REGTEST_ENV', 'constant', 'XC_ROLLCALL_REGTEST_ACTIVATION'); + +// Epoch cadence in BTC blocks. Weekly on the live networks per the 2026-08-30 +// ruling: with K=2 an outage shorter than one epoch minus the accept window +// (~6 days) can never evict, and 2-3 weeks idle always does. Regtest uses 30 so +// an acceptance run does not have to mine 2 x 1008 blocks. +addGate('rollcall_activation.ROLLCALL_INTERVAL_BLOCKS', 'constant', { mainnet: 1008, testnet: 1008, regtest: 30 }); + +// How long after the epoch block a signature may still land, in BTC blocks. The +// BTC header stamp at E + this value is what cuts the DOGE chain (see +// rollcallWindowEndHeight / the epoch close). +addGate('rollcall_activation.ROLLCALL_ACCEPT_WINDOW_BLOCKS', 'constant', { mainnet: 144, testnet: 144, regtest: 12 }); +// SHARED-GATES END diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows_3.js b/protocol/reference-impl/consensus/gate_registry/shared_rows_3.js new file mode 100644 index 00000000..3855da51 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows_3.js @@ -0,0 +1,358 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The SHARED block, part 3 of 5: rollcall_activation to stateHash + * + * One SHARED block part. The region between the two marker lines is + * BYTE-TWINNED into the registry of xchain-sync, xchain-hub, xchain-explorer + * and xchain-sdk: each consumer keeps the same bytes and replaces only the + * require line below with its own queue module. What may live between the + * markers: `addGate(key, unit, table)` calls with LITERAL values (a table, a + * number, a string or literals joined by +, a RegExp, an array), one call per + * row, at column zero, and comments. No require, no computed value, nothing + * from outside the block but addGate, UNARMED and UNPINNED. A regtest entry a + * venue arms from its environment is written UNPINNED here and armed by the + * wrapper at registration (shared_rows.js), so the block stays data. + * + * Rows are grouped by module stem in alphabetical order; a stem's rows keep + * the order the module declared them. Keys never change (I4). + * + ********************************************************************/ + +'use strict'; + +const { addGate, UNARMED, UNPINNED } = require('./shared_rows.js'); + +// SHARED-GATES BEGIN +// rollcall_activation (continued) +// BTC blocks after the window closes before the epoch closes, giving the DOGE +// side time to bury. MUST be >= 1 on every network: a block's `block_time` is +// written by createBlock AFTER that block's own processing, so the window +// endpoint has to be a strictly earlier block than the close, or the close +// reads a timestamp that does not exist yet. +addGate('rollcall_activation.ROLLCALL_PROOF_DELAY_BLOCKS', 'constant', { mainnet: 36, testnet: 36, regtest: 2 }); + +// DOGE blocks past the window cut before a DOGE indexer's answer is admissible; +// the anchor rail's own burial depth. This is what bounds the one residual the +// design accepts: a DOGE reorg deeper than this, removing a counted signature +// after the BTC close has recorded its epoch, cannot be undone from BTC, +// because nothing there observes it and no un-evict rail exists. +addGate('rollcall_activation.ROLLCALL_DOGE_MATURITY', 'constant', { mainnet: 60, testnet: 60, regtest: 2 }); + +// K: consecutive ROLLED epochs a source must be absent for before eviction. +addGate('rollcall_activation.ROLLCALL_EVICT_MISSES', 'constant', 2); + +// 2K: how many rolled epochs back the K-streak may reach. Bounds how far an old +// absence can travel, so a source that leaves for months and returns starts +// clean rather than resuming a stale streak. +addGate('rollcall_activation.ROLLCALL_STREAK_LOOKBACK', 'constant', 4); + +// The frozen rollcall-publish reward, minted BTC-side to the ELECTED LEADER +// only -- never to whoever published first, which would be a fee-bidding race +// no hub can bump, since there is no fee-bump or RBF path anywhere in the hub. +// Parity with ANCHOR_REWARD_AMOUNT per the 2026-08-30 ruling. Never from the wire. +addGate('rollcall_activation.ROLLCALL_REWARD_AMOUNT', 'constant', '10.00000000'); + +// rollcall_gates_activation +// Per-network EPOCH height at/above which ROLLCALL is published as v1 with the +// GATES field and the epoch close records each signer's list. +addGate('rollcall_gates_activation.ROLLCALL_GATES_ACTIVATION', 'epoch', { + mainnet: null, // INERT placeholder: the operator owns this height + testnet: 152208, // SIZED 2026-09-08: the first epoch boundary (151200 + 1008) after the v0.16.0 roll, which lands between the 151200 and 152208 closes + regtest: UNPINNED, // ARMS AT 0 when the venue sets XC_ROLLCALL_GATES_REGTEST_ACTIVATION +}); + +// The documented regtest arming height: genesis, a multiple of the 30-block +// regtest interval, so epoch 0 is a real v1 epoch. +addGate('rollcall_gates_activation.ROLLCALL_GATES_REGTEST_ARMED_HEIGHT', 'constant', 0); + +// The one environment variable this module reads, and only ever for regtest. +addGate('rollcall_gates_activation.ROLLCALL_GATES_REGTEST_ENV', 'constant', 'XC_ROLLCALL_GATES_REGTEST_ACTIVATION'); + +// snapshot_reorg_buffer +// The reorg-depth buffer every party in a federation must resolve capability +// snapshots at. 6 = the BTC confirmation depth the platform already treats as +// buried (XCHAIN_CONFIRMATIONS_BTC). CONSENSUS-CRITICAL: the hub subtracts this +// before every snapshot lookup and refuses to boot on mainnet/testnet when a local +// override diverges (CapabilitySnapshot._resolveReorgBuffer), so a verifier that +// buries by a different depth resolves a different set than the signer. +addGate('snapshot_reorg_buffer.CANONICAL_REORG_BUFFER', 'constant', 6); + +// Per-network activation height (LOCAL COPY of the canonical map in +// xchain-documentation/protocol/constants.js, kept equal by the cross-service +// regression suite). Keyed on the BTC-anchored declared snapshot_block. +// +// Arming this changes acceptance itself, so a one-sided or partially-rolled-out arm would +// fork the fleet rather than fix it, and it re-reads every checkpoint already signed and +// anchored under the current reading. There is no such checkpoint on any network: mainnet +// was ruled at genesis on 2026-09-09 after measuring 0 validators, 0 stakes and 0 +// quorum-signed artifacts on every mainnet chain, so burying reinterprets nothing there and +// the from-genesis OLD-vs-ON replay is the witness. Regtest is active from genesis (no +// history to preserve; the regtest suites exercise the buried resolution from block 0). +addGate('snapshot_reorg_buffer.SNAPSHOT_BURIAL_ACTIVATION', 'height', { + mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 validators, 0 stakes, measured 2026-09-09) + // ARMED AT GENESIS, operator-ratified 2026-08-18 as part of the pre-launch "every + // feature active on testnet" ruling. Safe because testnet's indexer state is being + // REBUILT from the chain before launch, and because testnet carries no artifacts + // signed under the current reading for this to reinterpret: the live explorer reports + // 0 validators, 0 capability stakes and 0 checkpoints on BTC testnet, so nothing has + // ever been quorum-signed there. Mainnet was measured the same way on 2026-09-09. + testnet: 0, + regtest: 0, +}); + +// stake_weight_collation_activation +// The collation the consensus ordering is pinned to once the rule is live. +// FROZEN: this string is the emitted SQL of a consensus query, so changing it +// after any chain arms re-orders the cap survivors on a replay, which is a +// fork. utf8_bin is the collation the sibling consensus reads already pin, and +// it is charset-compatible with the utf8/utf8mb3 columns involved. +addGate('stake_weight_collation_activation.STAKE_WEIGHT_COLLATION', 'constant', 'utf8_bin'); + +// Per-chain activation heights, interpreted against the chain's own block_index. +// `null` = NOT YET PINNED = inert (legacy unpinned ordering, byte-identical +// replay). Mainnet is armed at genesis by the 2026-09-09 ruling: the ordering +// this gate pins only decides which stake sources and keys survive the snapshot +// cap, and mainnet holds 0 stakes (measured 2026-09-09), so the binary order and +// the folding order select the same empty set and a height of 0 reinterprets +// nothing. The usual retroactivity hazard (a height a carrying fleet has already +// passed) is what a from-genesis OLD-vs-ON replay witness per chain proves away +// here. Testnet stays unpinned: it carries live stakes, so its height is pinned +// at flag-day assembly above the tip recorded then, in ONE coordinated deploy of +// BOTH fleets (a height armed while one fleet is behind halts the follower). +addGate('stake_weight_collation_activation.STAKE_WEIGHT_COLLATION_ACTIVATION', 'height', { + 'BTC:mainnet': 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 stakes, measured 2026-09-09) + 'LTC:mainnet': 0, + 'DOGE:mainnet': 0, + 'BTC:testnet': null, + 'LTC:testnet': null, + 'DOGE:testnet': null, + regtest: 0, +}); + +// Columns whose charset/collation the consensus ordering depends on, and the +// charset/collation src/sql declares for each. +addGate('stake_weight_collation_activation.STAKE_WEIGHT_ORDERING_COLUMNS', 'constant', [ + { table: 'index_addresses', column: 'address', charset: 'utf8', collation: 'utf8_general_ci' }, + { table: 'index_pubkeys', column: 'pubkey', charset: 'utf8', collation: 'utf8_general_ci' }, +]); + +// stake_weighted_quorum +// Per-network activation height (LOCAL COPY of the canonical map in +// xchain-documentation/protocol/constants.js, kept equal by the cross-service +// regression suite). Keyed on the BTC-anchored snapshot_block, NOT each chain's +// local height, so every chain + the hub flip on the same anchor. +addGate('stake_weighted_quorum.STAKE_WEIGHTED_QUORUM_ACTIVATION', 'height', { + mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers (+ sdk/explorer/sync copies) before this height + testnet: 0, + regtest: 0, +}); + +// stateHash +// Launch genesis version = 1. Folded into the hash so two preimage schemes can +// never compare equal. A dev iteration briefly numbered a changed preimage 2; +// pre-launch that was collapsed back to 1 (mirroring the BLOCK_HASH_VERSION 2->1 +// collapse) because there is no launch-committed state to migrate, only a clean +// fleet-wide reindex. Bump ONLY on a deliberate preimage change AFTER launch. +// Independent of BLOCK_HASH_VERSION (the three-hash baseline is untouched by this +// additive, non-consensus integrity hash). +addGate('stateHash.STATE_HASH_VERSION', 'constant', 1); + +addGate('stateHash.DEACTIVATION_TABLES', 'constant', ['stakes', 'delegations', 'contract_stakes', 'contract_delegations']); + +addGate('stateHash.SLASH_SPECS', 'constant', [ + { table: 'stakes', debits: 'capability_slash_debits', target: 'stakes' }, + { table: 'unstakes', debits: 'capability_slash_debits', target: 'unstakes' }, + { table: 'contract_stakes', debits: 'contract_slash_debits', target: 'contract_stakes' }, + { table: 'contract_unstakes', debits: 'contract_slash_debits', target: 'contract_unstakes' } +]); + +addGate('stateHash.REQUEST_STATUS_TABLES', 'constant', ['attests', 'xcalls']); + +addGate('stateHash.COOLDOWN_TABLES', 'constant', ['unstakes', 'contract_unstakes']); + +// ── Index-map state-hash flag-day (id-determinism P4) ──────────────────────── +// Promotes the index_addresses / index_tickers id->string MAP from the advisory +// /status checksum (BlockHasher.computeIndexMapChecksum) to an ENFORCED per-block +// class of this replication-integrity hash: the follower recomputes state_hash and +// HALTS on mismatch, so an id-map divergence (a wire ^id resolving to a different +// entity, or a recovered node that built a different map) is caught at the block +// that introduced it instead of silently forking. +// +// This is the ONE place state_hash deliberately hashes the surrogate id (not just +// the resolved string): the id IS the value under protection. It is sound only +// because the compaction + F1a work made every id-assignment path deterministic +// (in-block dense counter + rollback; recovery stages by string and the apply hook +// assigns the deterministic id) - so a from-genesis node and a recovered node now +// produce byte-identical (id, address) pairs over the same chain. +// +// Per-block DELTA (block_index = B), mirroring the rest of this preimage's per-block +// shape: the rows whose deterministic id was first assigned at B. A cumulative +// checksum would chain every block on all history (the credits-chaining trap the +// header warns about); the delta catches a divergence at its origin block. +// +// Gated on the chain's OWN local block_index (like state_commitment_activation, not +// the BTC-anchored snapshot_block): each chain arms at its own flag-day height. +// Landed DEFAULT INERT (placeholder 999999999) so shipping the class was a strict +// no-op on the live fleet - the class is omitted from the preimage below the +// threshold, leaving state_hash byte-identical to the pre-feature shape. Armed +// fleet-atomically (real per-chain heights) exactly as WI-2 / state_commitment +// did; regtest was the last inert key (armed 2026-07-16). No +// STATE_HASH_VERSION bump: a block is unambiguously pre- or post-activation on a +// given network, so the two preimage shapes cannot collide. +addGate('stateHash.INDEX_MAP_STATE_HASH_ACTIVATION', 'height', { + // Heights are the chain's OWN local block_index at/after which the index-map + // folds into state_hash. One-way door: once a chain crosses its height, any + // process still on a different height computes a divergent state_hash and a + // follower HALTS, so every indexer + sync process must run this exact map + // BEFORE the chain reaches the height. Keep this map byte-identical to the + // xchain-{indexer,sync}/src/stateHash.js twin. + mainnet: 0, // ARMED at the genesis launch reindex (folds from genesis, no mid-chain flag-day) + testnet: 0, // ARMED at the genesis launch reindex (clean reseed accompanies it) + regtest: 0, // ARMED from genesis 2026-07-16: fresh stacks exercise the class end to end; pre-existing regtest venues need a clean reseed +}); + +// ── VOTE poll-finalization state-hash flag-day ──────────────────────────────── +// Promotes the polls finalization flip (VOTE v2 mutates a SURVIVING polls row +// terminal IN PLACE) from an unhashed updated_rows mutation to an ENFORCED +// per-block class of this replication-integrity hash, closing the one mutation +// class where a follower silently dropping the flip upsert diverged with no halt +// (the attests/xcalls v0 flips have had this coverage from the start). +// +// Same gating model as INDEX_MAP_STATE_HASH_ACTIVATION above (keyed on the +// chain's OWN local block_index), but ARMED MID-CHAIN, which forces per-chain +// keys: unlike the genesis-armed index-map class, one shared 'mainnet' height +// cannot fit BTC (~957k) and DOGE (~6.28M) simultaneously. Lookup is +// ':' first, then the bare network key (regtest keeps one key; +// unknown -> inert). ARMED 2026-07-07 at tip + margin per chain: the whole +// fleet (every indexer + sync process) MUST run this map before the EARLIEST +// chain crosses its height (see the deploy-by note per line), or followers +// still on the old map false-halt at the boundary. A missed deadline is +// recoverable by bumping the not-yet-crossed heights before deploy. No +// STATE_HASH_VERSION bump: a block is unambiguously pre- or post-activation. +// Keep byte-identical to the xchain-sync twin. +addGate('stateHash.POLL_FINALIZE_STATE_HASH_ACTIVATION', 'height', { + 'BTC:mainnet': 958500, // armed 2026-07-07 at tip 957062; ~10 days of margin + 'LTC:mainnet': 3143000, // armed 2026-07-07 at tip 3138154; ~8 days + 'DOGE:mainnet': 6291000, // armed 2026-07-07 at tip 6280094; ~7.5 days + 'BTC:testnet': 145000, // armed 2026-07-07 at tip 143299 + 'LTC:testnet': 4805000, // armed 2026-07-07 at tip 4797675 + 'DOGE:testnet': 67000000, // armed 2026-07-07 at tip 66498605 (fast chain, wide margin) + regtest: 0, // armed from genesis: fresh regtest stacks exercise the class end to end +}); + +// ── tokens.supply state-hash flag-day (F-1 closure) ────────────────────────── +// The hash twin of the updated_rows tokens-supply replication class: supply is +// mutated IN PLACE on a surviving token row (its action_index stays at the +// DEPLOY action), so no consensus block hash and no other state_hash class +// covers it; a follower silently dropping the supply upsert served a stale +// supply with no halt. Supply changes exactly when a credit/debit/escrow row is +// written for the tick, so the per-block class hashes (tick, supply) for every +// tick touched by a ledger row at block B. Same per-chain arming map and +// deploy-by constraint as POLL_FINALIZE above; the two classes flip together. +addGate('stateHash.TOKEN_SUPPLY_STATE_HASH_ACTIVATION', 'height', { + 'BTC:mainnet': 958500, // armed 2026-07-07, same heights as POLL_FINALIZE + 'LTC:mainnet': 3143000, + 'DOGE:mainnet': 6291000, + 'BTC:testnet': 145000, + 'LTC:testnet': 4805000, + 'DOGE:testnet': 67000000, + regtest: 0, +}); + +// ── BET status-flip state-hash flag-day ─────────────────────────────────────── +// The hash twin of the updated_rows BET replication classes. BET carries THREE +// in-place mutations on surviving rows, all stamped with the block that made +// them: the closed latch (bet_feeds.closed_block), the feed terminal flip +// (bet_feeds.terminal_block: resolved/resolved_void/cancelled/expired) and the +// per-bet settlement flip (bets.settled_block: won/lost/refunded). None of them +// is visible to the action-scoped ledger hashes once the row's creating action +// is below the block, so a follower silently dropping one diverged with no halt +// (the exact class this file's header documents). Although the BET action is +// genesis-active, the hash class CANNOT be: mainnet/testnet fleets already +// compare state hashes every block, so an ungated preimage-shape change would +// halt a mixed-version fleet instantly. Same per-chain arming model as +// POLL_FINALIZE/TOKEN_SUPPLY above. +addGate('stateHash.BET_STATUS_STATE_HASH_ACTIVATION', 'height', { + // Heights pinned via roundUp1000(tip + 21 days x nominal blocks/day), never + // lowered once set: a height that falls in the past is not a flag day at all, + // since a node replaying from genesis applies the rule from it while a + // long-running node never did, and the two diverge at the first hash + // comparison (caught once on LTC:testnet, whose first pinned height the chain + // had already passed). Re-verify these against live tips before each deploy. + 'BTC:mainnet': 963000, // tip 959,853 (2026-07-27) + 21d @144/day = 962,877 + 'LTC:mainnet': 3162000, // tip 3,149,481 + 21d @576/day = 3,161,577 + 'DOGE:mainnet': 6338000, // tip 6,307,307 + 21d @1440/day = 6,337,547 + // Testnet is genesis-active as of the 2026-08-10 fresh testnet genesis: the + // chain restarts at firstBlock (BTC 147500 / LTC 4855000 / DOGE 67815000) with + // no pre-rule history, so there is nothing for a mid-chain boundary to protect. + // Kept value-equal to CARET_REF_STRICT_ACTIVATION and + // LIST_EDIT_RESOLUTION_ACTIVATION, which CI asserts. + 'BTC:testnet': 0, + 'LTC:testnet': 0, + 'DOGE:testnet': 0, + regtest: 0, // armed from genesis: fresh regtest stacks exercise the class end to end +}); + +// ── Archive-head anchor versions ────────────────────────────────────────────── +// The anchor_actions versions that carry an archive HEAD (a signed batch header +// whose v2 continuation chunks reassemble against it): v1, the publisher-bearing +// archive anchor, and nothing else. It is a one-member set because the wire +// families are version-disjoint by construction (bundle {0}, archive head {1}, +// chunk {2}); it stays a SET rather than a scalar so a later archive version joins +// it without touching the ten splice sites that read the SQL fragment. +// Every predicate that selects "the archive parent of a v2 chunk" MUST +// use this set: the invalid_archive stamp, its reorg reset, the forward +// updated_rows class and the state-hash class below all target the same rows. +// SINGLE SOURCE OF TRUTH for xchain-indexer rollback.js + this file's class 6, +// and (via the byte-identical xchain-sync twin) ClientRollback.js + +// updatedRows.js. db.js/recovery.js carry matching predicates. +addGate('stateHash.ARCHIVE_HEAD_VERSIONS', 'constant', [1]); + +// SQL fragment form, spliced as `p.version ` + ARCHIVE_HEAD_VERSIONS_SQL. +// The IN list of ARCHIVE_HEAD_VERSIONS above; the two move together. +addGate('stateHash.ARCHIVE_HEAD_VERSIONS_SQL', 'constant', 'IN (1)'); + +// ── invalid_archive archive-head-coverage state-hash flag-day ───────────────── +// Widens the anchor_invalid state-hash class (class 6 below) from a hard-coded +// v1 parent to whatever ARCHIVE_HEAD_VERSIONS holds, closing the integrity-hash +// blind spot where an invalid_archive stamp on a non-v1 archive head was invisible +// to the follower's recompute (a follower silently dropping that upsert diverged +// with no halt). With the version set restarted the two predicates coincide (the +// archive head IS v1 again), so the gate is inert in effect and kept only so the +// widening stays landed for the next archive version. Changes the class's row +// selection, hence the hashed preimage, so it +// is gated exactly like POLL_FINALIZE above (per-chain keys on the chain's OWN +// local block_index). No STATE_HASH_VERSION bump: a block is unambiguously pre- +// or post-activation on a given network. Keep byte-identical to the xchain-sync +// twin; every indexer AND sync process on a network must run this code, since a +// straggler on the v1-only predicate recomputes a different preimage and halts. +// TESTNET IS ARMED AT 0 (operator ruling 2026-08-11, applied 2026-08-14): the +// re-genesised testnet carries no pre-flag blocks, so there is no legacy preimage +// to stay byte-identical with, and arming at genesis made testnet the network that +// exercises the widened class first. MAINNET IS ARMED AT 0 by the 2026-09-09 +// ruling: mainnet holds 0 archive chunks (measured 2026-09-09), so the widened +// predicate and the legacy v1-only one select the same empty class and the +// genesis-armed preimage is identical to the deployed one; the 56 DOGE ANCHOR +// actions on record are checked by the from-genesis replay witness. +addGate('stateHash.ARCHIVE_INVALID_STATE_HASH_ACTIVATION', 'height', { + 'BTC:mainnet': 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 archive chunks, measured 2026-09-09) + 'LTC:mainnet': 0, + 'DOGE:mainnet': 0, + 'BTC:testnet': 0, // armed from genesis 2026-08-11 ruling + 'LTC:testnet': 0, // armed from genesis 2026-08-11 ruling + 'DOGE:testnet': 0, // armed from genesis 2026-08-11 ruling + regtest: 0, // armed from genesis: fresh regtest stacks exercise the widened class end to end +}); +// SHARED-GATES END diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows_4.js b/protocol/reference-impl/consensus/gate_registry/shared_rows_4.js new file mode 100644 index 00000000..4a5063c8 --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows_4.js @@ -0,0 +1,393 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The SHARED block, part 4 of 5: stateHash to token_bridge_activation + * + * One SHARED block part. The region between the two marker lines is + * BYTE-TWINNED into the registry of xchain-sync, xchain-hub, xchain-explorer + * and xchain-sdk: each consumer keeps the same bytes and replaces only the + * require line below with its own queue module. What may live between the + * markers: `addGate(key, unit, table)` calls with LITERAL values (a table, a + * number, a string or literals joined by +, a RegExp, an array), one call per + * row, at column zero, and comments. No require, no computed value, nothing + * from outside the block but addGate, UNARMED and UNPINNED. A regtest entry a + * venue arms from its environment is written UNPINNED here and armed by the + * wrapper at registration (shared_rows.js), so the block stays data. + * + * Rows are grouped by module stem in alphabetical order; a stem's rows keep + * the order the module declared them. Keys never change (I4). + * + ********************************************************************/ + +'use strict'; + +const { addGate, UNARMED, UNPINNED } = require('./shared_rows.js'); + +// SHARED-GATES BEGIN +// stateHash (continued) +// ── invalid_archive chunk-height key repair state-hash flag-day ─────────────── +// Class 6 scopes the invalid_archive stamp to the block the COMPLETING v2 chunk +// landed in. It has always keyed that scope on `c.block_index`, and that column +// is NEVER populated on a v2 row: `block_index` carries BLOCK_INDEX_CHECKPOINTED +// (the checkpointed height on the OTHER chain, see anchor_actions.sql), which is +// assigned only in anchor.js `_parseCheckpoint`; `_parseContinuation` never sets +// it, and db.js binds the column NULL when the key is absent. `NULL BETWEEN x AND +// y` is never true, so the class has selected ZERO rows on every node since it +// landed, on every network. The completing chunk's real height is +// `block_index_doge` (the DOGE block the ANCHOR action landed in, NOT NULL by +// schema), the same height the class is being scoped to, and the same distinction +// anchor.js `_archiveAuthorScope` already draws. +// +// Repairing the key CHANGES THE PREIMAGE the moment a stamped batch exists: a +// node on the repaired predicate hashes the parent row, a node on the broken one +// hashes nothing, and the fleet halts at that block. So the repair is a flag day +// like every other class-shape change here, gated per chain on the chain's OWN +// local block_index, DEFAULT INERT: below the threshold the query keeps the +// broken `c.block_index` key and the preimage stays byte-identical to what every +// deployed node computes today. +// +// MAINNET IS ARMED AT 0 by the 2026-09-09 ruling. The repair only moves the +// preimage where a stamped archive batch exists, and mainnet holds 0 archive +// chunks (measured 2026-09-09), so the repaired key and the broken one select +// the same empty class: arming at genesis leaves the preimage every deployed +// mainnet node computes unchanged, and the from-genesis replay witness is the +// proof of that identity (the 56 DOGE ANCHOR actions on record are covered by it). +// TESTNET TAKES REAL FUTURE HEIGHTS, not 0: testnet is a public chain that has +// run since the 2026-08-10 re-genesis, so a height of 0 here would be retroactive +// rather than a flag day. They are sized from the 2026-09-09 tips (TBTC 151701, +// TLTC 4883295, TDOGE 67881714) plus about 22 days, so the v0.17.0 train is live +// on every testnet process before the earliest chain crosses. +// Deploy order is not free: xchain-sync updatedRows.js carries the SAME broken key +// on the replication side (fixed there un-gated, since shipping a row is not a +// preimage) and must be live FIRST, or a follower is asked to hash a stamped +// parent row it was never sent. regtest is armed at 0 so fresh regtest stacks +// exercise the repaired class end to end. No STATE_HASH_VERSION bump: a block is +// unambiguously pre- or post-activation. Keep byte-identical to the +// xchain-sync twin. +addGate('stateHash.ARCHIVE_INVALID_HEIGHT_KEY_ACTIVATION', 'height', { + 'BTC:mainnet': 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 archive chunks, measured 2026-09-09) + 'LTC:mainnet': 0, + 'DOGE:mainnet': 0, // DOGE is the anchor chain, and the 56 mainnet ANCHORs there carry no archive chunk + 'BTC:testnet': 155000, // tip 151701 (2026-09-09) + 3299 blocks @144/day = ~23 days + 'LTC:testnet': 4896000, // tip 4883295 + 12705 blocks @576/day = ~22 days + 'DOGE:testnet': 67915000, // tip 67881714 + 33286 blocks @1440/day = ~23 days + regtest: 0, // armed from genesis: fresh regtest stacks exercise the repaired class end to end +}); + +// The column class 6 scopes the completing v2 chunk by, as a SQL fragment. Broken +// legacy key below the flag day, repaired key at/after it. Exported so the twin +// repos and the drift guards can assert on ONE definition rather than a literal. +addGate('stateHash.ARCHIVE_CHUNK_HEIGHT_COL', 'constant', 'c.block_index_doge'); + +addGate('stateHash.ARCHIVE_CHUNK_HEIGHT_COL_LEGACY', 'constant', 'c.block_index'); + +// state_commitment_activation +// Per-chain activation height, interpreted as the processing chain's OWN +// block_index. At/after this height the new roots are committed; below it the +// state_tree_roots row is absent and the getblockhashes RPC returns null roots. +// ARMED MID-CHAIN like the stateHash.js class maps, which forces per-chain keys +// (one shared 'mainnet' height cannot fit BTC ~957k and DOGE ~6.28M at once). +// Lookup is ':' first, then the bare network key (regtest keeps +// one key; unknown -> inert/off, which is safe: roots simply stay absent). +// Same heights as the two state-hash gates armed 2026-07-07, so ONE deploy-by +// date governs all Cohort-C flips; each height precedes the Cohort-B BTC +// anchor (961000) as the checkpoint-commitment ordering requires. +addGate('state_commitment_activation.STATE_COMMITMENT_ACTIVATION', 'height', { + 'BTC:mainnet': 958500, // ARMED 2026-07-07 at tip 957062; ~10 days of margin + 'LTC:mainnet': 3143000, // ARMED 2026-07-07 at tip 3138154; ~8 days + 'DOGE:mainnet': 6291000, // ARMED 2026-07-07 at tip 6280094; ~7.5 days + 'BTC:testnet': 145000, // ARMED 2026-07-07 at tip 143299 + 'LTC:testnet': 4805000, // ARMED 2026-07-07 at tip 4797675 + 'DOGE:testnet': 67000000, // ARMED 2026-07-07 at tip 66498605 (fast chain, wide margin) + regtest: 0, // armed from genesis: fresh regtest stacks exercise the roots end to end +}); + +// state_key_collation_activation +// Per-chain activation height, interpreted as the processing chain's OWN +// block_index. At/after the height the binary-collation (utf8_bin) queries +// run; below it the legacy folding (utf8_general_ci) queries run. +// +// *** ARMED 2026-07-10 *** Heights are each processing chain's OWN block_index, +// sequenced after the other armed cohorts (state_commitment 958500/3143000/ +// 6291000, swq_source_cap 960000, Cohort-B anchor 961000 on BTC) with fleet +// deploy-by margin; testnets flip late July so they prove the binary path +// before mainnet. The xchain-sync twin mirrors this file byte-for-byte in the +// same change. Regtest is armed from genesis so fresh regtest stacks and the +// e2e conformance scenario exercise the binary path end to end. +addGate('state_key_collation_activation.STATE_KEY_COLLATION_ACTIVATION', 'height', { + 'BTC:mainnet': 962500, // ARMED 2026-07-10 at tip 957491 (~Aug 13; ~10 days past Cohort-B 961000) + 'LTC:mainnet': 3160000, // ARMED 2026-07-10 at tip 3140024 (~Aug 14) + 'DOGE:mainnet': 6335000, // ARMED 2026-07-10 at tip 6284614 (~Aug 15) + // Testnet is genesis-active as of the 2026-08-10 fresh testnet genesis. LTC was + // the one entry still ahead of the new firstBlock (4890000 > 4855000); BTC and + // DOGE were already behind it and are zeroed with it so the map reads as one + // rule rather than three coincidences. The documented ordering still holds: + // state_commitment is genesis-active too, so this never precedes it. + 'BTC:testnet': 0, + 'LTC:testnet': 0, + 'DOGE:testnet': 0, + regtest: 0, // armed from genesis: fresh regtest stacks exercise the binary path end to end +}); + +// state_subtree_activation +// The reserved slots, in merkle.STATE_SUBTREES order. Order matters: it IS the +// leaf order of the top-level fixed Merkle tree. The conformance test asserts +// this equals STATE_SUBTREES minus the two v1 slots, so the two lists cannot drift. +addGate('state_subtree_activation.RESERVED_SUBTREES', 'constant', ['ownership_root', 'tokens_root', 'contract_state_root']); + +// Per-slot, per-chain activation height, interpreted as the processing chain's +// OWN block_index. At/after the height the slot MAY carry a real sub-root; below +// it (and for any chain absent from the map) the slot commits EMPTY_SMT_ROOT. +// +// *** ARMED: contract_state_root on BTC:regtest at 10000 (2026-07-28) and from +// *** GENESIS on BTC, LTC and DOGE testnet. +// The testnet entries read 0 rather than a measured height because those chains +// are rebuilt from chain before launch, so every block is derived under this rule +// and no row written without it survives to disagree. BTC:testnet held 146500 +// until the 2026-08-10 re-genesis moved firstBlock above it, which left the +// height inert but readable as a boundary that no longer exists; LTC and DOGE had +// no entry at all. Genesis on all three states the intent directly and matches +// the escrow leaf below, so one reindex covers both stages. +// +// The ordering rule is satisfied at 0: a slot must never arm below its chain's +// state_key_collation_activation height, or the SMT is built over a +// collation-FOLDED key set and forks. All three testnets are genesis-active +// there too (see state_key_collation_activation.js), so nothing precedes it. +// +// Everything else is still inert, on every chain and network. MAINNET IS UNARMED +// for every slot. +// +// Regtest armed FIRST and alone, and only once the derivation existed. The +// earlier rule here ("a slot stays off even on regtest, because an armed slot +// with no derivation commits a WRONG root rather than a missing one") was about +// the carrier era; Stage A's derivation, its golden vectors, its serving surface +// and its real-venue conformance all landed before that height was set. Regtest +// is where being wrong costs a chain reset and nothing more, so it is the venue +// that earns the right to arm testnet, and testnet earns mainnet. +// +// All three testnet chains join it from genesis. Both hard preconditions hold, +// and they hold for DIFFERENT reasons than on regtest (spec §3 Stage A): +// 1. collation: state_key_collation_activation is genesis-active on BTC, LTC +// and DOGE testnet, so at height 0 nothing precedes it and the SMT is built +// over the binary-collation key set rather than a collation-FOLDED one. +// Regtest satisfies the same rule trivially, being armed from genesis too. +// 2. NUL keys: isStateKeyNulRejectActive returns true unconditionally for +// testnet AND regtest (xchain-vm), so on neither network can a contract +// plant a key that throws in joinFields and halts the arming block's +// buildFull. Mainnet is a DATE and is why no mainnet height may be set +// here yet. +// +// No per-chain argument separates the three: since the fresh testnet genesis +// their collation heights are all 0, so precondition 1 is satisfied identically +// and none of them is closer to or further from eligibility than another. +// +// DOGE:testnet is SYNC_EXCLUDE'd on the sync client, so its arming exercises the +// source alone. That bounds the EVIDENCE, not the eligibility: BTC:testnet is +// the only testnet chain with a live source-plus-follower pair, so it is the one +// whose arming gets a cross-twin check, and the chain to read when asking +// whether the twins agree. +// +// Honest limit on what genesis arming proves: a testnet chain with no +// contract_state rows commits EMPTY_SMT_ROOT for the slot, so state_root does +// not move at all (a named-but-empty slot is byte-identical to a padded one, see +// the header). What arming really exercises there is the version path: +// state_root_version reads 2 and flows through getblockhashes into the signed +// checkpoint canonical. Exercising the DERIVATION needs contract state on the +// chain first, which BTC:testnet now has and the other two do not. +// +// Arming order is fixed by the design doc: contract_state_root first (Stage A, +// and never below that chain's state_key_collation_activation height, or the SMT +// is built over a collation-FOLDED key set and forks), ownership/tokens later. +addGate('state_subtree_activation.STATE_SUBTREE_ACTIVATION', 'constant', { + ownership_root: {}, + tokens_root: {}, + contract_state_root: { + 'BTC:regtest': 10000, + 'BTC:testnet': 0, + 'LTC:testnet': 0, + 'DOGE:testnet': 0, + }, +}); + +// SHADOW-COMPUTE WINDOW (spec §7 step 1). INERT: every map empty. +// +// Where STATE_SUBTREE_ACTIVATION decides what a chain COMMITS, this decides what +// it merely COMPUTES AND RECORDS. Arming a chain here makes both twins derive the +// slot's would-be sub-root for every block and persist it in that slot's SHADOW +// column, while state_root stays byte-identical to the v1 assembly. Zero +// cross-twin divergence over the window is an arming PRECONDITION, not a +// nice-to-have: it is how a derivation bug is found before a flag day rather than +// as a fleet halt after one. +// +// Two properties make this safe to leave on: +// - it never reaches assembleStateRoot (gateSubRoots is driven by the +// ACTIVATION map alone), so no committed root can move; and +// - it writes a column nothing else reads. The explorer reassembles proofs +// from the COMMITTED column only, which is precisely why the shadow may not +// share it: a below-arming value there would reassemble to a state_root +// nobody signed and take every proof at that height down (spec §7, amended +// 2026-07-28 when Stage A work item 4 dropped the explorer's read gate). +// +// A chain may be shadowing and armed at once; ARMED WINS, so the boundary is +// clean: at and above the armed height the value is committed and written to the +// real column, below it the value is shadow-only. Nothing computes twice. +addGate('state_subtree_activation.STATE_SUBTREE_SHADOW', 'constant', { + ownership_root: {}, + tokens_root: {}, + contract_state_root: {}, +}); + +// Locked-balance leaf inside balances_root (SPV sub-tree spec §3 Stage B, +// ). ARMED ON BTC:regtest AT BLOCK 11200 (2026-07-30), and nowhere else. +// The derivation exists: an append-only, source-authored and +// replicated escrow_leaf_journal whose totals are the escrows LEDGER rows +// re-keyed to their locker (xchain-indexer/src/escrowJournalWriter.js), read +// by the byte-identical escrowLeafSubtree.js twin. Arming this moves +// balances_root, the one sub-root every deployed light client already depends +// on, which is why Stage B arms after Stage A and on its own flag day. +// +// LIVENESS RUNS THROUGH THIS MAP AT BOTH ENDS, unlike a reserved slot. A +// slot's stored column carries the armed decision for its own height, but no +// stored signal distinguishes a balances_root that covers the XCHAIN_ESC +// domain from one that does not (an armed-but-idle domain and an inert one +// commit byte-identical roots). So the explorer refuses locked-balance proofs +// below the armed height using ITS carrier of this file, and the SDK verifier +// independently refuses using its own, so neither a lagging nor a hostile +// server can turn "not committed" into a verified absence (spec §4). +// Armed from genesis on every testnet chain, and on regtest at 11200. +// +// Two conditions make a genesis height correct here rather than merely convenient. Stage A +// (contract_state_root) must already be live on the chain, since this leaf moves +// balances_root and Stage B is defined to follow Stage A; on the testnet chains it is. +// And the chain's key collation must be genesis-active, which it is on all three. +// +// A genesis height also removes the arming BOUNDARY entirely: there is no below-arming +// region for a locked-balance proof to fall into, so the seeding order that matters when +// arming mid-chain does not apply. That holds only where indexer state is rebuilt from the +// chain, which is a precondition of this height. +// +// The derivation carries no coin gate (see escrowLeafSubtree.js and escrowJournalWriter.js), +// so the three chains arm together. Mainnet stays unarmed because live light clients depend +// on balances_root there, which is the whole reason this is staged at all. +addGate('state_subtree_activation.ESCROW_LOCKED_LEAF_ACTIVATION', 'height', { + 'BTC:regtest': 11200, + 'BTC:testnet': 0, + 'LTC:testnet': 0, + 'DOGE:testnet': 0, +}); + +// SHADOW-COMPUTE WINDOW for the escrow leaf (spec §7 step 1, Stage B). INERT. +// +// Same contract as STATE_SUBTREE_SHADOW: arming a chain here makes BOTH twins +// derive the WOULD-BE balances_root (the spendable leaves threaded exactly as +// committed, plus the journal's locked leaves) and persist it in +// state_tree_roots.balances_root_escrow_shadow, while the committed +// balances_root stays byte-identical to v1. It also starts the SOURCE's +// journal writer below the armed height, which is consensus-free (the journal +// is not a commitment; its rows replicate to the follower exactly as when +// armed), so the window exercises writer, replication and leaf application +// end to end, and zero cross-twin divergence over it is the §7 arming +// precondition. +// +// ARMED WINS: the predicate below answers false once the leaf is really live, +// and the arming block still runs its own full ledger replay, so a drifted +// shadow journal is CORRECTED rather than inherited (the replay is +// change-logged; a wrong shadow value gets a correction row, vectored). +// +// *** OPEN ON BTC:testnet FROM BLOCK 148000 (2026-08-11, operator-approved), +// *** and nowhere else. Chosen at tip 147969, so the window starts ~31 blocks +// *** of lead ahead of the deploy, per §4's deploy-before-the-height rule. +// +// WHY THIS CHAIN AND WHY NOW. BTC:testnet is being re-seeded after the +// 2026-08-10 re-genesis, and the escrow seed (xchain-e2e-test +// bin/seed-escrow-state.js) posts its locking ORDERs after this height. That +// ordering is the whole point: with the window already open, those locks are +// journaled by the ORDINARY PER-BLOCK INCREMENTAL PATH - the one that runs on +// every block forever - rather than by the window-start replay. Both produce +// the same rows, and the harness already proves they agree on BTC:regtest +// (97 live keys, incremental against arming replay, measured 2026-08-11), but +// only the incremental path is the one no public chain has ever exercised. +// +// Nothing here is committed. balances_root stays byte-identical to v1 while a +// chain is only shadowing, and its locked-balance proofs stay refused, because +// ESCROW_LOCKED_LEAF_ACTIVATION above is what the explorer and the SDK verifier +// gate on. +// +// EMPTY, and deliberately so. The BTC:testnet entry that sat here at 148000 was +// dead the moment the leaf armed at genesis on all three testnets: ARMED WINS +// over a shadow, so the window could never open, while the surrounding prose +// still read as though the leaf were unarmed there. A shadow height below its +// own chain's arming height is unreachable by construction, so leaving it in +// place taught the next reader something false about what testnet commits. +addGate('state_subtree_activation.ESCROW_LOCKED_LEAF_SHADOW', 'height', {}); + +// swq_source_cap_activation +// CONSENSUS-CRITICAL caps on the source-keyed stake-weight snapshot. MUST be equal +// in xchain-indexer + xchain-sync (a drift forks the stakes_root at/after the +// activation height). +// STAKE_WEIGHT_MAX_SOURCES - cap on DISTINCT staking SOURCES in a weighted +// snapshot (the consensus unit; Σ weight over +// distinct sources = S). Over-fetched by one so a +// genuinely larger federation is flagged truncated +// and the primitive fails closed (a coordinated +// cap raise then re-opens liveness). +// STAKE_WEIGHT_MAX_KEYS_PER_SOURCE - cap on effective keys returned per source. Bounds +// only the row/leaf count for a key-spamming source; +// dropping a source's excess keys does NOT change its +// weight (weight is per source, counted once) and does +// NOT set truncated. Generous: no legit source +// delegates near this many keys. +addGate('swq_source_cap_activation.STAKE_WEIGHT_MAX_SOURCES', 'constant', 1000); + +addGate('swq_source_cap_activation.STAKE_WEIGHT_MAX_KEYS_PER_SOURCE', 'constant', 64); + +// Per-chain activation height, interpreted as the processing chain's OWN block_index +// (same semantics as STATE_COMMITMENT_ACTIVATION). At/after the height the windowed +// source-cap is applied; below it the legacy uncapped key-LIMIT path runs. +// +// Option B (separate later height): the BTC:mainnet cap arms AFTER STATE_COMMITMENT +// (958500, ~2026-07-17) and AT/BEFORE STAKE_WEIGHTED_QUORUM arms (961000, ~2026-08-04), +// so the eviction fix is live when weighted quorum goes live without an 8-day +// hashed-root fleet-deploy race. For sub-cap honest federations the capped and +// uncapped stakes_root are byte-identical, so this mid-stream height introduces no +// real root discontinuity - only the >cap case (the attack) diverges, deterministically. +addGate('swq_source_cap_activation.SWQ_SOURCE_CAP_ACTIVATION', 'height', { + 'BTC:mainnet': 960000, // Option B: after STATE_COMMITMENT (958500), before STAKE_WEIGHTED_QUORUM (961000) + 'LTC:mainnet': 3143000, // inert (LTC commits the EMPTY stakes_root; capability staking is BTC-only) - pinned == STATE_COMMITMENT for parity + 'DOGE:mainnet': 6291000, // inert (DOGE stakes_root empty) - pinned == STATE_COMMITMENT for parity + 'BTC:testnet': 0, // capped from genesis; STATE_COMMITMENT testnet (145000) > 0, so testnet only ever commits capped roots (no discontinuity) + 'LTC:testnet': 0, + 'DOGE:testnet': 0, + regtest: 0, // armed from genesis: fresh regtest stacks exercise the capped path end to end +}); + +// token_bridge_activation +// TOKEN_BRIDGE_ACTIVATION: the height (per network) on the chain being parsed at/above +// which XBRIDGE v3/v4 and ISSUE format 7 are legal. Below it v3 and v4 return the base +// spec's own string 'invalid: XBRIDGE before activation', v5 is never injected, and an +// ISSUE|7 keeps the parse verdict 'invalid: VERSION (unknown)' so no historical ISSUE on +// any chain changes status on replay. +// +// Keyed on the chain's OWN block_index, as XCHAIN_BRIDGE_ACTIVATION. +// +// Mainnet and testnet sit at the house sentinel 9999999999. Testnet is NOT armed with the +// XCHAIN bridge: no third-party token can be offered on a hub-trusted mint, so this gate +// waits on the base spec's D2 checkpoint cross-check being built and armed on that +// network. Regtest is 0 so the e2e rail exercises the armed rule from genesis. +addGate('token_bridge_activation.TOKEN_BRIDGE_ACTIVATION', 'height', { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}); +// SHARED-GATES END diff --git a/protocol/reference-impl/consensus/gate_registry/shared_rows_5.js b/protocol/reference-impl/consensus/gate_registry/shared_rows_5.js new file mode 100644 index 00000000..92f3979b --- /dev/null +++ b/protocol/reference-impl/consensus/gate_registry/shared_rows_5.js @@ -0,0 +1,146 @@ +/********************************************************************* + * + * Copyright © 2025-2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. A commercial + * license (without AGPL source-disclosure terms) is available - + * contact legal@dankest.llc. + * + ********************************************************************** + * + * The SHARED block, part 5 of 5: token_policy_activation to xchain_bridge_activation + * + * One SHARED block part. The region between the two marker lines is + * BYTE-TWINNED into the registry of xchain-sync, xchain-hub, xchain-explorer + * and xchain-sdk: each consumer keeps the same bytes and replaces only the + * require line below with its own queue module. What may live between the + * markers: `addGate(key, unit, table)` calls with LITERAL values (a table, a + * number, a string or literals joined by +, a RegExp, an array), one call per + * row, at column zero, and comments. No require, no computed value, nothing + * from outside the block but addGate, UNARMED and UNPINNED. A regtest entry a + * venue arms from its environment is written UNPINNED here and armed by the + * wrapper at registration (shared_rows.js), so the block stays data. + * + * Rows are grouped by module stem in alphabetical order; a stem's rows keep + * the order the module declared them. Keys never change (I4). + * + ********************************************************************/ + +'use strict'; + +const { addGate, UNARMED, UNPINNED } = require('./shared_rows.js'); + +// SHARED-GATES BEGIN +// token_policy_activation +// TOKEN_POLICY_INHERITANCE_ACTIVATION: the height (per network) on the chain being +// parsed at/above which policy inheritance is in effect. Keyed on the chain's OWN +// block_index, never on a snapshot's snapshot_block or origin_block, because what it +// gates is the verdict of an action mined here. +// +// What it gates, all of it consensus-visible: +// - the milestone-1 refusals lifted in issue.js (format 7 on a listed token; format +// 5, and a format 0 carrying lists, on a bridged token); +// - any-coin address items in list.js and the same widening in db.isAddressSleeping; +// - application of a mirrored policy_snapshots row on the destination, and with it +// the in-leg barrier that holds a v5 credit until the tick has a policy; +// - the hub engine's policy poll, so no XPOLICY row is signed below it. +// +// Below it every milestone-1 verdict stands unchanged, so the replay corpus is +// hash-identical on every chain with this code present. +// +// Mainnet and testnet are the house sentinel 9999999999: this rides the same MAJOR +// train as the two bridges and the operator sizes the dated instant at the cut. A +// height in the map ahead of the fleet's deploy tip is the operator's act, not a +// build's. Regtest is 0 so the e2e rail exercises the armed rule from genesis. +// +// TWO ORDERING INVARIANTS, asserted by test/unit/activationConstantsParity.test.js +// over the canonical constants.js rather than over this copy: +// - >= TOKEN_BRIDGE_ACTIVATION per network. Inheritance has nothing to inherit onto +// before bridged copies can exist. +// - >= LIST_EDIT_RESOLUTION_ACTIVATION per chain and network. The snapshot read +// resolves a list AS OF origin_block through getListAtBlock, which walks the edit +// chain; below that gate the legacy create-index read runs and the membership the +// federation signs would not be the membership the chain actually held. +addGate('token_policy_activation.TOKEN_POLICY_INHERITANCE_ACTIVATION', 'height', { + mainnet: 9999999999, + testnet: 9999999999, + regtest: 0, +}); + +// train_activation +// Keyed by platform version, then network, to a BTC block height. Only MAJOR +// trains and consensus-classified hotfixes get a row; a MINOR or PATCH train adds +// none, and resolveRuleSet then keeps the fleet on the previous entry with no +// ceremony change. Regtest is 0 on every row because regtest stacks are rebuilt +// from genesis and so exercise the new rule set end to end rather than the +// migration. Mainnet is armed ABOVE the tip at cut time on purpose: the fleet runs +// the new binary under the OLD rules until that height, which is the rolling-upgrade +// window. Nothing here is edited outside a train cut. +addGate('train_activation.TRAIN_ACTIVATION', 'ruleset', { + // The launch rule set and the floor. Zero on every network because there is no + // earlier rule set to migrate from: the launch binary IS the first rule set, and + // a floor above genesis would leave the pre-floor range resolving to nothing. + '1.0.0': { mainnet: 0, testnet: 0, regtest: 0 }, + // The XCHAIN bridge rule set, armed at the v0.19.0 cut. Mainnet holds the house + // sentinel: the mainnet arm is the next milestone and nothing arms there before the + // checkpoint cross-check lands, so no mainnet node ever reaches this boundary. + // Testnet: SIZED 2026-09-16, re-cut 16:33Z, from chain_tip TBTC 152,716 + 71 blocks, + // which is ceil(10 h / 508.8 s per block measured over the preceding 99 blocks), about + // 10.0 h. The first sizing (11:53Z, 152,716 itself) was overrun by the chain while the + // cut waited on the e2e matrix, so the boundary was re-cut from the new tip with a lead + // long enough to cover that wait plus the roll. That is the rolling-upgrade window the + // fleet roll must finish inside (over 6x the 90 minute roll budget), and every testnet + // bridge height below sits above it on the same BTC clock, so a node lacking this rule + // set halts before it can grade a bridge action. + '0.19.0': { mainnet: 9999999999, testnet: 152787, regtest: 0 }, +}); + +// xchain_bridge_activation +// XCHAIN_BRIDGE_ACTIVATION: the height on the chain being parsed at/above which XBRIDGE is +// legal. Below it a broadcast v0 or v1 is 'invalid: XBRIDGE before activation' (the +// per-feature shape anchor.js uses; the central 'invalid: ACTION is not yet activated' only +// fires on software that predates the action) and no v2 is ever injected, so pre-activation +// block hashes are unchanged on every chain. +// +// Keyed on the chain's OWN block_index, never on a transfer's snapshot_block: the row being +// judged is the action mined here. The hub reads the same map for the chain a leg was mined +// on, and signs nothing for a chain that has not reached its own height. +// +// KEYED ':', with the bare network key as the fallback (the shape +// stake_key_reuse_activation.js already uses one map over). One testnet number cannot serve +// three chains: the bridge arms on TBTC, TLTC and TDOGE, whose tips differ by orders of +// magnitude (about 152,110 / 4,884,193 / 67,889,993 measured 2026-09-12), so a single height +// is either already passed on two of them at boot or unreachable on the third. A coin with +// no entry of its own inherits the bare network key, which leaves an unlisted chain inert +// rather than undecided. +// +// Mainnet is the house sentinel 9999999999 on every key: milestone 1 is a hub-trusted mint +// (spec section 12), and nothing arms on mainnet before the D2 checkpoint cross-check lands. +// Testnet is SIZED AT THE v0.19.0 CUT, one dated instant PER CHAIN, from the three chain +// tips and their last-99-block cadences read in one sitting (2026-09-16 16:33Z, the re-cut +// after the 11:53Z sizing was overrun: TBTC 152,716 at 508.8 s per block, TLTC 4,887,644 at +// 141.8 s, TDOGE 67,900,748 at 27.4 s). The two destinations arm ceil(10 h / cadence) blocks +// above their tips and BTC, the ORIGIN of the v0 lock, arms ceil(30 h / cadence) above its +// tip and so LAST in wall clock, because the lock +// handler never checks the destination's own activation: a destination arming later would +// admit a lock nothing can mint, and the 3x gap is the band a destination cadence can slow by +// before that ordering breaks. All three sit above the TRAIN_ACTIVATION 0.19.0 testnet +// boundary on the BTC clock, so a node lacking the rule set halts before it grades a bridge +// action. Regtest is 0 and stays bare, because one regtest number fits every chain and the +// e2e rail exercises the armed rule from genesis. +addGate('xchain_bridge_activation.XCHAIN_BRIDGE_ACTIVATION', 'height', { + 'BTC:mainnet': 9999999999, + 'LTC:mainnet': 9999999999, + 'DOGE:mainnet': 9999999999, + mainnet: 9999999999, // fallback for a coin with no entry above + 'BTC:testnet': 152929, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 152,716 + 213 (30 h at 508.8 s/blk), about 30.1 h, the origin, last + 'LTC:testnet': 4887898, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 4,887,644 + 254 (10 h at 141.8 s/blk), about 10.0 h + 'DOGE:testnet': 67902062, // SIZED 2026-09-16, re-cut 16:33Z: chain_tip 67,900,748 + 1314 (10 h at 27.4 s/blk), about 10.0 h + testnet: 9999999999, // fallback: a testnet coin with no entry above stays dark + regtest: 0, // genesis-active so the e2e rail exercises the armed rule +}); +// SHARED-GATES END diff --git a/protocol/reference-impl/equivocation_header.js b/protocol/reference-impl/equivocation_header.js index 588f1489..6d262cda 100644 --- a/protocol/reference-impl/equivocation_header.js +++ b/protocol/reference-impl/equivocation_header.js @@ -43,45 +43,11 @@ * ********************************************************************/ -// Per-network activation height (LOCAL COPY of the canonical map in -// xchain-documentation/protocol/constants.js, kept equal by the cross-service -// regression suite). Keyed on the BTC-anchored snapshot_block, NOT the local -// processing height, so every chain + the hub flip on the same anchor. -const EQUIV_HEADER_ACTIVATION = { - mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers (+ sdk/explorer/sync copies) before this height - testnet: 0, - regtest: 0, -}; +const { get, copy, activeAt } = require('./consensus/gate_registry'); -// Fixed per-engine tag (spec §4.1.1). One slashable canonical family per engine. -const ENGINE_TAGS = { - DEX: 'XDEX', - XCALL: 'XCALL', - ATTEST: 'XATTEST', - ORACLE: 'XORACLE', - // PRICE batches. A DISTINCT tag from ORACLE, not a reuse: a batch canonical - // carries first_round/last_round and no scalar `round`, and SLASH v0 reads - // `round` out of an ORACLE-tagged content to judge equivocation, skipping its - // distinct-rounds guard when either side lacks one. Under a shared tag an - // honest validator that signed one per-round consensus canonical and one batch - // at the same BTC anchor would be provably equivocating, for a full bond burn plus permanent - // capability disqualification. The batch ROUND_ID is - // `||` (pipes are safe here; equivKey treats - // the round id as opaque), so two honest batches that split one window - // differently do not collide on one key either. - ORACLE_BATCH: 'XORACLEB', - CHECKPOINT: 'XCHECKPOINT', - CONFIG: 'XCONFIG', - NODEPROOF: 'XNODEPROOF', - // ROLLCALL presence proofs. Namespacing ONLY, exactly like XNODEPROOF: the - // tag is deliberately absent from SLASH's ENGINE_CAPABILITY map, so no - // ROLLCALL canonical is a slashable family. Several valid ROLLCALLs per - // epoch are expected (a leader's, sweepers', self-publishes), every one - // carrying signatures over the SAME canonical for that epoch, so two of - // them are never conflicting content for one key. ROUND_ID is the BTC - // EPOCH_HEIGHT in decimal, VIEW is 0. - ROLLCALL: 'XROLLCALL', -}; +const EQUIV_HEADER_ACTIVATION = copy('equivocation_header.EQUIV_HEADER_ACTIVATION'); + +const ENGINE_TAGS = copy('equivocation_header.ENGINE_TAGS'); // Whether the EQUIV header is in effect for a settlement whose BTC-anchored snapshot // is at `snapshotBlock` on `network`. Below this -> legacy headerless bytes. diff --git a/protocol/reference-impl/snapshot_reorg_buffer.js b/protocol/reference-impl/snapshot_reorg_buffer.js index 5f298789..7e2bb9e3 100644 --- a/protocol/reference-impl/snapshot_reorg_buffer.js +++ b/protocol/reference-impl/snapshot_reorg_buffer.js @@ -72,36 +72,11 @@ 'use strict'; -// The reorg-depth buffer every party in a federation must resolve capability -// snapshots at. 6 = the BTC confirmation depth the platform already treats as -// buried (XCHAIN_CONFIRMATIONS_BTC). CONSENSUS-CRITICAL: the hub subtracts this -// before every snapshot lookup and refuses to boot on mainnet/testnet when a local -// override diverges (CapabilitySnapshot._resolveReorgBuffer), so a verifier that -// buries by a different depth resolves a different set than the signer. -const CANONICAL_REORG_BUFFER = 6; +const { get, copy, activeAt } = require('./consensus/gate_registry'); -// Per-network activation height (LOCAL COPY of the canonical map in -// xchain-documentation/protocol/constants.js, kept equal by the cross-service -// regression suite). Keyed on the BTC-anchored declared snapshot_block. -// -// Arming this changes acceptance itself, so a one-sided or partially-rolled-out arm would -// fork the fleet rather than fix it, and it re-reads every checkpoint already signed and -// anchored under the current reading. There is no such checkpoint on any network: mainnet -// was ruled at genesis on 2026-09-09 after measuring 0 validators, 0 stakes and 0 -// quorum-signed artifacts on every mainnet chain, so burying reinterprets nothing there and -// the from-genesis OLD-vs-ON replay is the witness. Regtest is active from genesis (no -// history to preserve; the regtest suites exercise the buried resolution from block 0). -const SNAPSHOT_BURIAL_ACTIVATION = { - mainnet: 0, // ARMED at genesis by the 2026-09-09 ruling: identity on the indexed mainnet history (0 validators, 0 stakes, measured 2026-09-09) - // ARMED AT GENESIS, operator-ratified 2026-08-18 as part of the pre-launch "every - // feature active on testnet" ruling. Safe because testnet's indexer state is being - // REBUILT from the chain before launch, and because testnet carries no artifacts - // signed under the current reading for this to reinterpret: the live explorer reports - // 0 validators, 0 capability stakes and 0 checkpoints on BTC testnet, so nothing has - // ever been quorum-signed there. Mainnet was measured the same way on 2026-09-09. - testnet: 0, - regtest: 0, -}; +const CANONICAL_REORG_BUFFER = copy('snapshot_reorg_buffer.CANONICAL_REORG_BUFFER'); + +const SNAPSHOT_BURIAL_ACTIVATION = copy('snapshot_reorg_buffer.SNAPSHOT_BURIAL_ACTIVATION'); // Whether a verifier must bury the declared snapshot_block before re-deriving the // validator set for it, on `network`. diff --git a/protocol/reference-impl/stake_weighted_quorum.js b/protocol/reference-impl/stake_weighted_quorum.js index 9774ae55..493d7f11 100644 --- a/protocol/reference-impl/stake_weighted_quorum.js +++ b/protocol/reference-impl/stake_weighted_quorum.js @@ -32,17 +32,11 @@ * ********************************************************************/ +const { get, copy, activeAt } = require('./consensus/gate_registry'); + const mathjs = require('mathjs'); -// Per-network activation height (LOCAL COPY of the canonical map in -// xchain-documentation/protocol/constants.js, kept equal by the cross-service -// regression suite). Keyed on the BTC-anchored snapshot_block, NOT each chain's -// local height, so every chain + the hub flip on the same anchor. -const STAKE_WEIGHTED_QUORUM_ACTIVATION = { - mainnet: 961000, // ARMED 2026-07-07: BTC anchor ~2026-08-04; deploy hub + ALL indexers (+ sdk/explorer/sync copies) before this height - testnet: 0, - regtest: 0, -}; +const STAKE_WEIGHTED_QUORUM_ACTIVATION = copy('stake_weighted_quorum.STAKE_WEIGHTED_QUORUM_ACTIVATION'); // Whether stake-weighted quorum is in effect for a round whose BTC-anchored // snapshot is at `snapshotBlock` on `network`. Below this -> legacy count quorum. diff --git a/protocol/test-vectors/equivocation_header.json b/protocol/test-vectors/equivocation_header.json index cf8d2430..0f590f11 100644 --- a/protocol/test-vectors/equivocation_header.json +++ b/protocol/test-vectors/equivocation_header.json @@ -9,7 +9,9 @@ "CHECKPOINT": "XCHECKPOINT", "CONFIG": "XCONFIG", "NODEPROOF": "XNODEPROOF", - "ROLLCALL": "XROLLCALL" + "ROLLCALL": "XROLLCALL", + "BRIDGE": "XBRIDGE", + "POLICY": "XPOLICY" }, "equivKey": [ { "name": "basic", "engineTag": "XDEX", "roundId": "r1", "view": "0", "expected": "XDEX|r1|0" }, @@ -80,6 +82,22 @@ "view": "0", "content": "regtest|0|0000000000000000000000000000000000000000000000000000000000000000", "expected": "EQUIV|XROLLCALL|0|0||regtest|0|0000000000000000000000000000000000000000000000000000000000000000" + }, + { + "name": "BRIDGE: transfer_id is the round id, the finalizing view is the view, and the pipe-separated transfer canonical survives the || boundary", + "engineTag": "XBRIDGE", + "roundId": "3b1f8c0e5a47d29e6b04f7a1c8d35e92074b6af18c2d9e35017a4bc6d8e2f309", + "view": "3", + "content": "XBRIDGE|3b1f8c0e5a47d29e6b04f7a1c8d35e92074b6af18c2d9e35017a4bc6d8e2f309|151200|XCHAIN|8|BTC|4211|bcrt1qsourceaddress|DOGE|ndestinationaddress|12.50000000|1757620800|regtest", + "expected": "EQUIV|XBRIDGE|3b1f8c0e5a47d29e6b04f7a1c8d35e92074b6af18c2d9e35017a4bc6d8e2f309|3||XBRIDGE|3b1f8c0e5a47d29e6b04f7a1c8d35e92074b6af18c2d9e35017a4bc6d8e2f309|151200|XCHAIN|8|BTC|4211|bcrt1qsourceaddress|DOGE|ndestinationaddress|12.50000000|1757620800|regtest" + }, + { + "name": "POLICY: snapshot_id is the round id, and a policy snapshot under its own tag never collides with a transfer at the same view", + "engineTag": "XPOLICY", + "roundId": "5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d", + "view": "3", + "content": "XPOLICY|5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d|151200|BTC|FUFU|1|151140|0a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8f9|1757620800|regtest", + "expected": "EQUIV|XPOLICY|5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d|3||XPOLICY|5e6f708192a3b4c5d6e7f8091a2b3c4d5e6f708192a3b4c5d6e7f8091a2b3c4d|151200|BTC|FUFU|1|151140|0a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8f9|1757620800|regtest" } ] } diff --git a/protocol/test-vectors/responsible_set.json b/protocol/test-vectors/responsible_set.json index abc50a83..fef43578 100644 --- a/protocol/test-vectors/responsible_set.json +++ b/protocol/test-vectors/responsible_set.json @@ -1,5 +1,5 @@ { - "_description": "Canonical conformance vectors for the attestation responsible-set selection rule (_computeResponsibleSet). This rule is implemented independently in xchain-hub (AttestationRound._computeResponsibleSet, AttestationPublisher._computeResponsible) and xchain-indexer (actions/attest._computeResponsibleSet, plus rollback._responsibleSet for the reorg recompute); they MUST produce identical ordered output for the same inputs or attestation quorum evaluation forks (the hub signs with set S_hub, the indexer filters verified signatures against S_idx, and any divergence silently expires every affected request). Unlike the stake_weighted_quorum primitive this rule is not a single vendored file, so there is no byte-identity guard; each repo's conformance test instead runs these behaviour vectors against its own copy. Rule: (1) when weighted, drop every validator whose staking source's aggregate `weight` is below `minStake`, the request provider's block-anchored min_stake_xchain floor, treating an unusable weight or an unusable floor as NOT clearing it; (2) for each surviving validator compute SHA256(request_id || lowercased-pubkey) and sort ascending by the hex hash string; (3) when weighted, dedupe by staking source keeping the lowest-hash key per source and always keeping null-source keys; (4) take the first max(1, redundancy) pubkeys. Steps 1 and 3 apply ONLY at/above the STAKE_WEIGHTED_QUORUM activation, which is the only path carrying per-source weight; below it `minStake` is ignored entirely and the legacy per-key selection runs. This file is the single source of truth; do not fork it. Any change to either implementation must update these vectors and stay green in both repos. HEADROOM (ATTEST_RESPONSIBLE_WIDENING_V2, spec attest-zero-confirmation-flip.md §4.1): _computeResponsibleSet also takes an optional `widen` argument that adds directly to the take-count in step 4 (`slice(0, max(1, redundancy) + widen)`), so a call with `redundancy: N, widen: W` is byte-identical to one with `redundancy: N + W, widen: 0`. Neither conformance harness threads a `widen` argument through today, so the headroom vectors below encode the widened take-count directly in `redundancy` and carry a `widen` field purely as documentation of the base/headroom split; a `_note` on each spells out the equivalence.", + "_description": "Canonical conformance vectors for the attestation responsible-set selection rule (computeResponsibleSet). This rule is implemented independently in xchain-hub (AttestationRound.computeResponsibleSet, AttestationPublisher.computeResponsible) and xchain-indexer (actions/attest.computeResponsibleSet, plus rollback.responsibleSet for the reorg recompute); they MUST produce identical ordered output for the same inputs or attestation quorum evaluation forks (the hub signs with set S_hub, the indexer filters verified signatures against S_idx, and any divergence silently expires every affected request). Unlike the stake_weighted_quorum primitive this rule is not a single vendored file, so there is no byte-identity guard; each repo's conformance test instead runs these behaviour vectors against its own copy. Rule: (1) when weighted, drop every validator whose staking source's aggregate `weight` is below `minStake`, the request provider's block-anchored min_stake_xchain floor, treating an unusable weight or an unusable floor as NOT clearing it; (2) for each surviving validator compute SHA256(request_id || lowercased-pubkey) and sort ascending by the hex hash string; (3) when weighted, dedupe by staking source keeping the lowest-hash key per source and always keeping null-source keys; (4) take the first max(1, redundancy) pubkeys. Steps 1 and 3 apply ONLY at/above the STAKE_WEIGHTED_QUORUM activation, which is the only path carrying per-source weight; below it `minStake` is ignored entirely and the legacy per-key selection runs. This file is the single source of truth; do not fork it. Any change to either implementation must update these vectors and stay green in both repos. HEADROOM (ATTEST_RESPONSIBLE_WIDENING_V2, spec attest-zero-confirmation-flip.md §4.1): computeResponsibleSet also takes an optional `widen` argument that adds directly to the take-count in step 4 (`slice(0, max(1, redundancy) + widen)`), so a call with `redundancy: N, widen: W` is byte-identical to one with `redundancy: N + W, widen: 0`. Neither conformance harness threads a `widen` argument through today, so the headroom vectors below encode the widened take-count directly in `redundancy` and carry a `widen` field purely as documentation of the base/headroom split; a `_note` on each spells out the equivalence.", "computeResponsibleSet": [ { "name": "unweighted hash-order, redundancy 3", diff --git a/protocol/test-vectors/rollcall_canonical.json b/protocol/test-vectors/rollcall_canonical.json index f6b662c1..ad7c98ac 100644 --- a/protocol/test-vectors/rollcall_canonical.json +++ b/protocol/test-vectors/rollcall_canonical.json @@ -1,5 +1,5 @@ { - "$schema_note": "FROZEN ROLLCALL canonical + WIRE vectors. The signed preimage is the EQUIV-wrapped canonical; the wire payload is the byte-for-byte broadcast form of a ROLLCALL action on DOGE. Three independent implementations must agree on these bytes: the xchain-hub PRODUCER (RollcallRound signs the canonical), the xchain-indexer DOGE PARSER (src/actions/rollcall.js rebuilds the canonical from the carried fields and verifies each signature), and the xchain-indexer BTC CLOSE (rebuilds the same canonical from its OWN ledger_hash and re-verifies). A drift between any two silently drops real presence proofs and evicts live validators, with no build or test failure unless these vectors are asserted on every side. The signatures below are REAL Ed25519 signatures over the canonical, from the fixed seeds named in signers[].seed, so a verifier vector fails on a wrong canonical rather than merely on a wrong shape.", + "$schema_note": "FROZEN ROLLCALL canonical + WIRE vectors. The signed preimage is the EQUIV-wrapped canonical; the wire payload is the byte-for-byte broadcast form of a ROLLCALL action on DOGE. Three independent implementations must agree on these bytes: the xchain-hub PRODUCER (RollcallRound signs the canonical), the xchain-indexer DOGE PARSER (src/actions/rollcall/index.js rebuilds the canonical from the carried fields and verifies each signature), and the xchain-indexer BTC CLOSE (rebuilds the same canonical from its OWN ledger_hash and re-verifies). A drift between any two silently drops real presence proofs and evicts live validators, with no build or test failure unless these vectors are asserted on every side. The signatures below are REAL Ed25519 signatures over the canonical, from the fixed seeds named in signers[].seed, so a verifier vector fails on a wrong canonical rather than merely on a wrong shape.", "authority": "xchain-documentation/protocol/test-vectors/rollcall_canonical.json is authoritative. Do not fork.", "canonical": { "form": "EQUIV|XROLLCALL||||||", diff --git a/protocol/token-bridge.md b/protocol/token-bridge.md new file mode 100644 index 00000000..54c376f9 --- /dev/null +++ b/protocol/token-bridge.md @@ -0,0 +1,194 @@ + + + +# XChain Platform: Token Bridge + +The [Cross-Chain Bridge](./xchain-bridge.md) gives XCHAIN, the platform's own fee token, a +shadow balance on every chain it runs on. This document describes the same mechanism opened up +to any issuer's token: one supply on its origin chain, provably backed one for one in a +protocol-owned escrow, with shadow balances anyone can mint and redeem elsewhere. See +[`XBRIDGE`](./actions/xbridge.md) for the wire action (versions `3`, `4` and `5`) and +[Token Bridge](../concepts/token-bridge.md) for the issuer-facing explanation. + +## Who this is for + +- **Quote assets.** A stablecoin issuer who issues on one chain and wants the same supply + spendable on the others, so every token on those chains can be listed against it locally with + no `COINPAY` leg. +- **Project tokens.** An issuer with a community on more than one chain who wants one supply and + one price rather than three independent issuances that drift apart. + +## Principle + +Nothing here is specific to XCHAIN except which chain is the origin. The origin chain is +wherever the token was issued (`ISSUE`, not necessarily BTC): its escrow lives there, lock +happens there, burn happens on a bridged copy, and a copy never bridges onward. Spoke-to-spoke +travel is two hops through the origin, so every token has exactly one escrow and the invariant +stays per token per destination chain, exactly as it does for XCHAIN. + +No key can touch a bridged row. The escrow is the origin chain's keyless role address +(`ADDRESS.BRIDGE_`); the bridged copy on the destination is owned by that chain's own +keyless bridge role address, not by any issuer key. An owner key could set lists, bind a +controller, or transfer the row, all of which milestone 1 keeps off the bridged copy entirely +(see [What does not generalize](#what-does-not-generalize)). With a keyless owner, nobody can, +and the row's own locks are belt and braces. + +## Naming: the bridged row lives under its origin chain's root + +The bare-name question decides whether this is usable at all: three options were weighed, and +only one survives a squatter. + +- **Same bare name, required free on every destination.** Fails the day one squatter registers + the name on one chain for one issuance fee; a stablecoin's expansion would be blockable by + anyone. +- **A mapping registry.** Nothing on the wire says two tokens on two chains are one asset, the + map is state somebody has to administer, and it confuses users. +- **Origin-rooted namespace (adopted).** A token native to BTC lands on DOGE and LTC as + `BTC.`; a token native to DOGE lands on BTC as `DOGE.`. + +`BTC`, `LTC` and `DOGE` are reserved ticks on every chain, and the platform's subasset parent +gate refuses any child whose parent does not exist or is owned by another address. The bridge +creates the root row itself the first time a token from that origin arrives: protocol-owned, +supply zero, every lock set. From that block a user `ISSUE` of `.anything` on that chain +fails with the ordinary parent-owner refusal, and no new guard is needed for children. This is +the same origin-badge convention wrapped tokens use elsewhere: the name on the wire says where +the asset is native, there is no registry, and squatting cannot block anyone. Wallets and +explorers render the bare name with an origin badge (`BTC.PEPECASH` shows as "PEPECASH, +bridged from BTC"). The one bare-name exception is XCHAIN itself, reserved on every chain and +needing no root. + +### Two limits in milestone 1 + +Both are refused at lock time, so no transfer can strand: + +- **Dotted origin names.** A subasset (`BTC.PEPE.CASH`) would need an intermediate row the + bridge does not yet create; a lock or opt-in of a dotted native tick is refused with + `invalid: TICK (subassets are not bridgeable yet)`. A later milestone walks the prefix and + creates the missing intermediate rows. +- **Length.** A native tick that would not fit once rooted (origin prefix plus a dot) is refused + with `invalid: TICK (too long to bridge)`. + +## The bridged row on the destination + +Created by the first settle leg for that tick on that chain, the way the XCHAIN row is created +on first use: + +1. **The root row** (``), if absent: owned by this chain's own bridge role address for + that origin, uncapped supply, zero decimals, every lock set, mint permanently disabled. One + per origin chain per destination chain, ever. +2. **The child row** (`.`): the same owner and locks, decimals taken from the + signed transfer record, uncapped supply (nothing off the origin can ever mint it, and a + copied cap would go stale the moment the origin's own `MAX_SUPPLY` changes). Nothing else is + copied from the origin: the bridged row starts with no description, no lists, no + controller. + +If the child row already exists, a later settle leg compares the signed decimals against it: +equal applies normally; unequal with zero supply re-parameterizes the row (the same rule every +token gets: decimals may move until supply exists); unequal with supply refuses the leg outright +and logs it, rather than silently drifting two chains' views of the same token's precision. + +Supply on a bridged row moves only by a settle-in (up) and a burn (down): a broadcast `ISSUE` of +a coin-rooted tick is refused by the ordinary parent-ownership gate, and a `DESTROY` of a bridged +row is refused in favor of the burn leg of `XBRIDGE`. + +## Issuer opt-in + +Bridgeability is a property the token owner sets on the origin row, **default off**, through +[`ISSUE` format `7`](./actions/issue.md#version-7---bridge-opt-in): + +- `BRIDGE_CHAINS`: which destination chains a lock may target. Empty means unchanged, so a + later format `7` can extend the list without restating it. +- `MIN_DEPTH`: a confirmation depth the federation must honour for this token's locks, on top of + the platform default; raise-only, never lower. +- `LOCK_BRIDGE`: freezes both fields forever, the holder's assurance against the owner (and + against a new owner after an ownership transfer, which inherits both fields). + +A lock checks the origin row's `BRIDGE_CHAINS` at its own block; a destination not on the list +is refused with `invalid: TICK (not bridgeable to DEST_COIN)`. `MIN_DEPTH` is stamped onto the +pending transfer at lock time and never re-read later, so an issuer raising it afterward can +never make an already-accepted lock un-signable. Removing a chain from `BRIDGE_CHAINS` only +stops new locks; it never touches balances already bridged, and burns are always allowed, so an +issuer can close a door but never strand anyone on the other side of it. + +## What does not generalize + +- **Trust.** [Cross-Chain Bridge: Trust model](./xchain-bridge.md#trust-model) applies + unchanged, and matters more here: XCHAIN's milestone-1 hub-trusted mint is a statement about + the platform's own token. A third-party issuer bridging real value should read that section + before opting in. This spec arms nowhere before the checkpoint cross-check described there is + built and armed on that network. +- **Reorg finality.** An applied mint is final and the origin escrow can drop on a reorg (see + [Reorgs and finality](./xchain-bridge.md#reorgs-and-finality)). For a third-party token that is + the issuer's own unbacked liability, not the platform's; `MIN_DEPTH` is the issuer's own price + for that risk, and the token's page shows it. +- **Issuer policy across chains.** Allow lists, block lists, controller bindings and sleep live + on the origin row and, in milestone 1, do not carry to a bridged copy at all. A regulated + issuer who needs a block on one chain to also hold on another cannot rely on this milestone. + Milestone 1 keeps the two states mutually exclusive rather than silently under-enforcing: a + token with any live list or controller binding cannot opt into bridging, and a bridged token + cannot set one. Sleep is chain-local by the same reasoning: sleeping the origin stops new + locks but never touches outstanding copies, which keep trading, and burns still release. + Behind `TOKEN_POLICY_INHERITANCE_ACTIVATION` the list and sleep half of this exclusion lifts; + see [Policy inheritance](#policy-inheritance) below. Controller bindings never lift on their + own (a controller names a local contract; nothing about it is portable to another chain). + +**Milestone ladder:** plain, undotted tokens with no policy first; then activation on a proven +network; then policy inheritance and generic (any-coin) lists; mainnet arming last, gated the +same way XCHAIN's own bridge is. + +## Policy inheritance + +Once `TOKEN_POLICY_INHERITANCE_ACTIVATION` is active on a network, a token's allow list, block +list and sleep state are no longer mutually exclusive with bridging: the issuer's policy is +signed once on the origin row and **inherited** on every bridged copy, never re-issued per +chain. See the platform's `policy-propagation` specification for the full mechanism; the +shape that matters to a reader of this page: + +- **One list, everywhere.** The origin's `ALLOW_LIST` and `BLOCK_LIST` membership is signed by + the `cross_chain` quorum into a `policy_snapshots` row (the same mirror channel + `bridge_transfers` rides) and materialized on each destination as its own local lists, owned + by that chain's bridge role address so no user key can edit them (`invalid: LIST_ACTION_INDEX + (bridge-owned)`, [`LIST`](./actions/list.md)). Sleep carries the same way, as an injected + `SLEEP`. +- **A confirmed lag, not a live mirror.** Between an origin-side edit and its effect on a copy: + the origin's own confirmation depth, a signing round, the mirror, and an `effective_time` + margin, the same discipline a bridge lock's own finality already accepts. The copy enforces + the *previous* policy for that window; nothing is retroactive. + `getappliedpolicy(tick)` (indexer, open read) shows exactly what a given chain has applied and + as of which origin block. +- **A membership ceiling.** A token whose `ALLOW_LIST` or `BLOCK_LIST` exceeds + `XPOLICY_MAX_MEMBERS` (10,000 addresses) cannot opt into bridging in the first place + (`invalid: TICK (policy list exceeds XPOLICY_MAX_MEMBERS)`, [`ISSUE` format + `7`](./actions/issue.md)); every snapshot and every destination materialization carries the + full membership, so this bounds both the mirror's transport size and the write amplification + on every chain holding a copy. +- **Any-coin list items.** Because one list now has to be enforced identically on every chain a + token has a copy on, a `LIST` of type `ADDRESS` accepts an address of *any* coin the platform + runs, not only the chain it was broadcast on; see [`LIST`](./actions/list.md). +- **Controller bindings are the one thing that still does not travel.** A controller names a + local contract deployed on one chain (`utility.js` runs its `guard` in that chain's own VM); + nothing about a snapshot can make that contract exist elsewhere. A controller-bound token + cannot opt into bridging and a bridged token cannot bind one, under this activation or any + later one, without a separate controller-portability build the platform does not carry today. + +## Reads and surfaces + +`getbridgeinvariant` (hub) takes an optional tick and, without one, returns the map keyed by +tick with XCHAIN always present. `getpendingbridgetransfers` and `getbridgetransfer` (indexer) +gain the tick, decimals, and effective confirmation depth fields the general case needs. The +explorer, SDK and wallet surfaces are the same ones XCHAIN's own bridge uses, extended with a +tick dimension; see [Token Bridge](../concepts/token-bridge.md) for the issuer- and +holder-facing view. + +--- + +**Copyright © 2025–2026 Dankest, LLC** + +**Based on XChain Platform by Dankest, LLC – https://dankest.llc** + +Licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0-or-later) +with a commercial license available for proprietary use. + +You may use, modify, and distribute this material under the terms of the License. +See [LICENSE](../LICENSE.md) and [NOTICE](../NOTICE.md) for full terms. +See the [licensing overview](https://docs.xchain.io/legal/LICENSING.html). diff --git a/protocol/token-gated-content.md b/protocol/token-gated-content.md index f1eb2985..e58eb009 100644 --- a/protocol/token-gated-content.md +++ b/protocol/token-gated-content.md @@ -47,7 +47,7 @@ The publisher is part of the key because token ownership transfers. A former iss 1. Issuer generates one `K` and `KEY_HASH`. 2. Issuer encrypts each file plaintext under `K`. Each file gets a fresh 12-byte nonce, but they all share `K`. -3. Issuer publishes a `FILE|0|...` action per file (one `rawData` per transaction; small files can be combined in a single `BATCH`). +3. Issuer publishes one `FILE|0|...` action per file, each in its own transaction. A transaction carries exactly one `rawData` payload and a `BATCH` hands that same payload to every sub-command, so a `BATCH` holds at most one `FILE` whatever the file sizes are; a second `FILE` batched beside the first is recorded valid carrying the FIRST file's ciphertext, permanently. To publish a pack in one transaction, combine the files into a single archive and publish that archive as one `FILE`. 4. After (or alongside) the last file, issuer publishes a self-`MESSAGE` whose binary payload contains the single shared `K`. Because every file in the pack shares the same `K`, one 32-byte entry in the handoff unlocks every file in the pack regardless of how many there are. diff --git a/protocol/token-information-standard.md b/protocol/token-information-standard.md index a0269bd9..2a329a88 100644 --- a/protocol/token-information-standard.md +++ b/protocol/token-information-standard.md @@ -7,14 +7,24 @@ The Token Information Standard (TIS) defines standardized formats to associate i ## JSON Specifications -### v1.1.0 (current) +### v1.1.1 (current) +- [Token Information Standard JSON Schema](./json/token-information-standard-v1.1.1-schema.json) +- [Token Information Standard JSON Example](./json/token-information-standard-v1.1.1-example.json) + +v1.1.1 relaxes a single constraint over v1.1.0 and adds no field. An entry in `images`, +`audio`, `video` or `files` requires `type` plus at least one of `data` or `data_ref`, +where v1.1.0 required `data` outright and so rejected the fully on-chain form this +standard recommends below. Relaxing a constraint cannot invalidate a document, so every +v1.1.0 and v1.0.0 document is also valid under v1.1.1. + +### v1.1.0 - [Token Information Standard JSON Schema](./json/token-information-standard-v1.1.0-schema.json) - [Token Information Standard JSON Example](./json/token-information-standard-v1.1.0-example.json) -v1.1.0 is additive over v1.0.0. It declares the token-gating fields (`packs`, `title`, -`data_ref`, `locked`, `pack_id`) that clients already emit and read, adds no required -field, and forbids nothing v1.0.0 allowed, so every document valid under v1.0.0 is also -valid under v1.1.0. +Frozen as published. It is additive over v1.0.0: it declares the token-gating fields +(`packs`, `title`, `data_ref`, `locked`, `pack_id`) that clients already emit and read, +adds no required field, and forbids nothing v1.0.0 allowed. Its four media definitions +require `["type", "data"]`, which is the constraint v1.1.1 relaxes. ### v1.0.0 - [Token Information Standard JSON Schema](./json/token-information-standard-v1.0.0-schema.json) @@ -26,8 +36,8 @@ generator run against it drops them. #### JSON Field Definitions -The tables below describe **v1.1.0**. Rows marked *(since v1.1.0)* are absent from the -v1.0.0 schema. +The tables below describe **v1.1.1**, which declares the same fields as v1.1.0. Rows +marked *(since v1.1.0)* are absent from the v1.0.0 schema. | Field | Type | Description | :--- | :--- | :--- @@ -56,7 +66,7 @@ Entries inside the `files`, `audio`, `video`, and `images` arrays can carry the | data | String | URL to the file (off-chain). Used for non-gated content. | data_ref | String | *(since v1.1.0)* Reference to an on-chain [`FILE`](./actions/file.md) action by `ACTION_INDEX`: `action:` (same chain as the token) or `action::` (sibling chain: base coin ticker `BTC`/`LTC`/`DOGE`, network tier implied by the token's network, same convention as [`LINK`](./actions/link.md)'s `COIN1`/`COIN2`). Lets cheap chains carry the bytes for tokens on expensive ones: e.g. a BTC token whose artwork FILE lives on DOGE. When both `data` and `data_ref` are present, clients prefer `data_ref`. | name | String | Filename -| type | String | MIME type +| type | String | Entry classification, drawn from the vocabulary of the array the entry sits in, and NOT a MIME type. `images`: display role, one of `icon`, `standard`, `large`, `hires`, paired with `size` so clients can pick a token icon. `audio`: container, one of `m4a`, `mp3`, `wav`. `video`: container, one of `mp4`, `mov`, `wmv`. `files`: free-form category such as `doc`, `pdf`, `xls`, `other`. The schema pins the first three lists as enums and leaves the `files` vocabulary open. The media type of the bytes comes from elsewhere: a `data_ref` entry inherits it from the referenced [`FILE`](./actions/file.md) action's `TYPE`, and a `data` URL from the server's `Content-Type`. | title | String | *(since v1.1.0)* Display title | locked | Boolean | *(since v1.1.0)* `true` if the file is encrypted and gated. Clients use this to render locked/unlocked states without first fetching the FILE action. | pack_id | String | *(since v1.1.0)* (Optional) Pack identifier grouping files that share an unlock key. References the top-level `packs` map for display name and description. Does not need to be present for unlocking to work; the protocol groups by `KEY_HASH` directly. diff --git a/protocol/upgrade-notice-policy.md b/protocol/upgrade-notice-policy.md index a474f75a..e32372b1 100644 --- a/protocol/upgrade-notice-policy.md +++ b/protocol/upgrade-notice-policy.md @@ -18,6 +18,15 @@ The clock starts when the release carrying the new activation values is tagged a announcement is published; it ends at the earliest activation moment in the release (first chain to cross an armed height, or the armed timestamp). +## Testnet + +The floors above govern mainnet only. On testnet, a consensus activation value is armed at +each chain's current tip, read at the moment the release is cut, with no forward margin; the +value goes live the instant the fleet is running the release. Testnet exists to build and test +ahead of mainnet, so there is no notice window to wait out there. Accepted caveat: an action +mined between the cut and the roll grades differently under a fresh replay than under the +still-running fleet, since nothing on testnet can encode the new value before the roll. + ## Rules 1. An activation value may be **deferred** (moved later) at any time before it is crossed by diff --git a/protocol/xchain-bridge.md b/protocol/xchain-bridge.md new file mode 100644 index 00000000..93429ad2 --- /dev/null +++ b/protocol/xchain-bridge.md @@ -0,0 +1,179 @@ + + + +# XChain Platform: Cross-Chain Bridge + +This document describes how XCHAIN, the platform's own fee token, moves between the chains +XChain runs on. XCHAIN is issued, minted, capped and priced on **BTC only**: one supply means +one price, and it keeps a Dogecoin XCHAIN, a Litecoin XCHAIN and a Bitcoin XCHAIN from drifting +apart. The bridge gives every other chain a **shadow balance** of that one supply, provably +backed one for one in a protocol-owned escrow on BTC, redeemable by anyone. See +[`XBRIDGE`](./actions/xbridge.md) for the wire action and [Token Bridge](./token-bridge.md) for +the general framework this same mechanism extends to any issuer's token. + +## The problem + +The fee role already works everywhere: every chain can pay gas in native coin at the oracle +rate. Two roles do not: + +- **Base pair.** On BTC a new token lists against XCHAIN and the DEX fills it, fire and forget. + On DOGE and LTC there is no XCHAIN balance to list against, so a seller's only same-chain + option is native coin, which cannot be escrowed and needs a `COINPAY` leg. +- **Distribution.** A genesis-style allocation to BTC addresses is easy. Holders on another + chain have nowhere to land it: there is no XCHAIN on that chain's ledger. + +## The model: lock and mint, burn and release + +A unit on DOGE that is provably one unit held in a BTC escrow address, and that anyone can burn +to get the BTC unit back, is the same asset in a second place. Arbitrage through the bridge +holds the two venues within the round-trip cost (two fees plus the confirmation delay each way), +exactly as the same coin on two exchanges is held together; a persistent price gap is free money +and closes on its own. + +1. **Lock (BTC).** A holder broadcasts [`XBRIDGE`](./actions/xbridge.md) version `0` on BTC + naming a destination chain and address. The indexer moves the amount from the holder's + balance to that chain's escrow address on BTC, an ordinary protocol address nobody holds a + key for. +2. **Attest.** After the source chain's confirmation depth (BTC 6 / LTC 12 / DOGE 60 by + default), the hub federation's `cross_chain` quorum signs a transfer record and writes it to + `bridge_transfers`, streamed to every indexer over the existing hub-DB mirror, the same + channel `cross_chain_matches` and `capability_snapshots` already ride. No per-transfer + on-chain transaction. +3. **Mint (destination).** The destination indexer applies the record at the first block whose + protocol time is at or past the record's `effective_time`, verifies the quorum against the + mirrored capability snapshot at the record's `snapshot_block`, and injects `XBRIDGE` version + `2`: credit the destination address, raise that chain's XCHAIN supply by the amount. +4. **Burn (destination) and release (BTC).** The reverse: `XBRIDGE` version `1` on DOGE or LTC + debits the holder and lowers that chain's supply; the federation attests after that chain's + own depth; the BTC indexer injects `XBRIDGE` version `2` that moves the amount from the + escrow address to the named BTC address. +5. **Retract.** If the source action is rolled back by a reorg before the transfer is applied, + the federation retracts the record (mirror deletion) and it is never applied. A leg that was + already applied on the other chain is **not** unwound; the platform has no such path and the + cross-chain DEX ships with the same residual (see [Reorgs and finality](#reorgs-and-finality)). + +```mermaid +sequenceDiagram + participant BTC as BTC indexer + participant Fed as cross_chain quorum + participant Mirror as bridge_transfers (hub mirror) + participant DOGE as DOGE indexer + BTC->>BTC: XBRIDGE v0 (lock): holder -> ADDRESS.BRIDGE_DOGE + Fed->>BTC: poll getpendingbridgetransfers (depth 6) + Fed->>Fed: PBFT, sign EQUIV-wrapped XBRIDGE canonical + Fed->>Mirror: finalized transfer + Mirror->>DOGE: mirrored row + DOGE->>DOGE: verify quorum at snapshot_block, inject XBRIDGE v2: credit, supply += amount + DOGE->>DOGE: XBRIDGE v1 (burn): holder debited, supply -= amount + Fed->>DOGE: poll (depth 60) + Fed->>Mirror: finalized transfer (return leg) + Mirror->>BTC: mirrored row + BTC->>BTC: inject XBRIDGE v2: ADDRESS.BRIDGE_DOGE -> BTC address +``` + +## The supply invariant + +For every foreign chain C: `balance_BTC(ADDRESS.BRIDGE_C, XCHAIN) >= supply_C(XCHAIN)`, modulo +in-flight transfers, and equality is the expected state. The inequality is deliberate: nothing +on the platform refuses a plain credit to a protocol role address, so an ordinary `SEND`, an +`ORDER` fill, a `DISPENSER` or an `AIRDROP` on BTC can land XCHAIN on the escrow with no transfer +record behind it. That is a **surplus**: the sender's own loss, the same as a send to a burn +address, and no other holder is unbacked by it. A **deficit** (`supply_C` exceeds the escrow) is +the only direction where someone else's units have nothing behind them, and is either a forgery +or a reorg (see [Reorgs and finality](#reorgs-and-finality)). + +`getbridgeinvariant` (below) reports the signed delta per chain: positive is a surplus (WARN), +negative a deficit (CRIT). A transfer counts as **in flight** from the block its source leg +(lock or burn) applies until the block its destination leg (mint or release) applies, which +includes the confirmation wait and the attestation round. + +The `MAX_SUPPLY` cap binds on BTC only: the BTC token row's supply is every unit ever minted, +circulating plus escrowed, and a foreign chain's supply is a shadow of its escrow, never counted +against the cap. Nothing off BTC can raise supply except an `XBRIDGE` v2 credit, and nothing off +BTC can lower it except `XBRIDGE` v1: a broadcast `ISSUE` of XCHAIN off BTC is refused +unconditionally, and so is a `DESTROY` of it, both closures explained under +[`XBRIDGE`](./actions/xbridge.md#rules). + +## Trust model + +Today the cross-chain DEX's mirror can delay a settlement but cannot forge one, because both +legs of a match are pre-escrowed on chain before the mirror ever gets involved. A bridge is +different: a forged transfer record mints on the destination with nothing held on the source. +Off BTC, the validator set itself is pulled from the hub's own mirror +(`capability_snapshots`, applied with `INSERT IGNORE`), which is the same authority that verifies +the transfer record. So in **milestone 1** a compromised hub can supply both the record and the +roster that verifies it, and every destination indexer mints: milestone 1 is a **hub-trusted +mint**. That is defensible on testnet if stated, and it is stated here and in the testnet +announcement. + +Closing that trust boundary is a stated follow-on, gating mainnet: the destination indexer checks +the lock against the quorum-signed BTC state checkpoint that [`ANCHOR`](./actions/anchor.md) +version `0` carries, so a mint needs a signed transfer record **and** a BTC ledger that agrees. +That reduces the assumption to "the `cross_chain` quorum and the checkpoint quorum both lied," +the same assumption every validator action on the platform already rests on. Nothing arms on +mainnet before that check lands and is proven. + +## Reorgs and finality + +A lock or burn reorged out of the source chain before the federation signs it never produces a +transfer record. A finalized record whose source is reorged out before it applies is retracted +(fenced, co-signed) and the destination never applies it. **Once a mint or release has applied, +it stays applied**: the destination chain did not itself reorg, so there is nothing on it to roll +back, and there is no forward "un-mint" path on the platform. Milestone 1 ships no destination- +side unwind: an applied mint is final, the confirmation depth (BTC 6 / LTC 12 / DOGE 60 by +default, raise-only per token) is the attacker's price for forcing that outcome, and the checkpoint +cross-check above (before mainnet) is the defence beyond it. + +## The XCHAIN token off BTC + +The token row on a foreign chain is created lazily, by the first `XBRIDGE` v2 credit on that +chain, with parameters byte-identical to the BTC genesis row (`MAX_SUPPLY` 100,000,000, +`DECIMALS` 8, mint disabled). No chain but BTC ever gains a genesis pass or a genesis credit; +distributing XCHAIN to another chain is an ordinary treasury operation on the bridge's own rails +(mint on BTC, lock to an operator-held address on the destination, `AIRDROP` there), not a +protocol-level concern. + +Once the row exists, handlers that resolve XCHAIN unconditionally (guard-gas reservations, fee +mode detection) start seeing it where they previously saw nothing; see +[Gas and Fees](../concepts/gas.md) for the fee side of that boundary. + +## Reads + +- **`getbridgeinvariant`** (hub, open read): escrow balance, shadow supply, in-flight amount and + the signed delta, per destination chain. +- **`getpendingbridgetransfers`** and **`getbridgetransfer(transfer_id)`** (indexer, open read): + confirmed locks and burns awaiting attestation, and any transfer by id. Escrow balances need + no new read; they are ordinary balances on a role address. + +## Watch + +The platform's operator watch item carries one entry over the signed delta from +`getbridgeinvariant`: a deficit beyond the in-flight set on any network is CRIT, a surplus is +WARN. + +## Activation + +`XCHAIN_BRIDGE_ACTIVATION` is a standalone height-keyed module, the same shape as every other +flag day: regtest active from genesis, testnet and mainnet held at the platform's sentinel until +armed. It is keyed `':'`, with the bare network key as the fallback for a chain +that has no slot of its own. The bridge arms on three chains at once and their heights are not +comparable (a BTC testnet tip is around 152,000 while a DOGE testnet tip is around 67,900,000), +so one number per network would be already passed on two chains and out of reach on the third; +each chain therefore gets its own instant, sized at the train that arms it. Roll order is the +**reverse** of the cross-chain DEX precedent: indexers and readers +first, the hub last, because a hub rolled ahead of the fleet stamps a schema version every +mirror closed against an un-upgraded indexer would fail. See +[Flag-Day Values](./flag-days.md) for where the height stands on each network. + +--- + +**Copyright © 2025–2026 Dankest, LLC** + +**Based on XChain Platform by Dankest, LLC – https://dankest.llc** + +Licensed under the **GNU Affero General Public License v3.0** (AGPL-3.0-or-later) +with a commercial license available for proprietary use. + +You may use, modify, and distribute this material under the terms of the License. +See [LICENSE](../LICENSE.md) and [NOTICE](../NOTICE.md) for full terms. +See the [licensing overview](https://docs.xchain.io/legal/LICENSING.html). diff --git a/protocol/xchain-uri-scheme.md b/protocol/xchain-uri-scheme.md index ae0a2a1c..7c85d476 100644 --- a/protocol/xchain-uri-scheme.md +++ b/protocol/xchain-uri-scheme.md @@ -181,7 +181,7 @@ xchain:///?amount=&to=&memo=&kind=receive Where `` is the descriptor id (e.g. `bitcoin-mainnet`). Wallets MUST accept this form for backwards compatibility but SHOULD NOT generate new QRs in this format. The coin-code opaque form is preferred for new QRs because it's shorter, uses the platform-wide short identifier, and surfaces the action explicitly. -The full descriptor-to-coin-code mapping (source: `xchain-sdk/src/networks.js`): +The full descriptor-to-coin-code mapping (source: `xchain-sdk/src/protocol/networks.js`): | `` descriptor | Coin code | | :--- | :--- | diff --git a/test/action-activation-model.test.js b/test/action-activation-model.test.js index 7d2b85b0..b02a0438 100644 --- a/test/action-activation-model.test.js +++ b/test/action-activation-model.test.js @@ -33,9 +33,13 @@ * 4. The docs do not reintroduce the "actions activate at block heights" * phrasing. * - * The registry is parsed rather than required: protocol_changes.js is a class - * that wants a live indexer to construct, and the addChange(...) calls are - * literal enough to read directly. + * The registry is REQUIRED, not parsed. Its rows live in the part files under + * src/protocol_changes/ as array literals the entry assembles, and the entry + * pulls in nothing past the canonicaliser (crypto), so the class builds its + * table here from the same rows the indexer runs on. It wants an indexer for + * its config and database, and every row is parsed before either is touched, + * so a bare stub is enough to read the table. A text scan would have to know + * the row shape, and this file asserts values, not literals. * * xchain-indexer is a sibling repo, not a dependency. Source-derived * assertions skip when it is absent; the prose check always runs. @@ -46,10 +50,12 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const REGISTRY = path.resolve(DOC_ROOT, '../xchain-indexer/src/protocol_changes.js'); -const haveRegistry = fs.existsSync(REGISTRY); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the registry is unreadable. +const indexer = sibling('xchain-indexer', [REGISTRY]); // The 36 documented ACTIONs: one page per action under protocol/actions/. const ACTIONS = fs.readdirSync(path.join(DOC_ROOT, 'protocol/actions')) @@ -58,31 +64,29 @@ const ACTIONS = fs.readdirSync(path.join(DOC_ROOT, 'protocol/actions')) // uppercase protocol identifier, so derive the identifier from the file. .map((f) => f.replace(/\.md$/, '').toUpperCase()); +/** Every registered change as `{ name, version, thresholds }`, thresholds in addChange order as strings. */ function readRegistry() { - const src = fs.readFileSync(REGISTRY, 'utf8'); - const re = /this\.addChange\(\s*'([A-Z_]+)'\s*,\s*'([\d.]+)'\s*,([^)]*)\)/g; - const out = []; - let m; - while ((m = re.exec(src)) !== null) { - out.push({ - name: m[1], - version: m[2], - thresholds: m[3].split(',').map((s) => s.trim()).filter((s) => s !== ''), - }); - } - assert.ok(out.length > 50, 'parsed only ' + out.length + ' addChange calls; the registry format changed'); + const ProtocolChanges = require(REGISTRY); + const stub = { config: {}, util: { throwError(message) { throw new Error(message); } } }; + const out = Object.entries(new ProtocolChanges(stub).changes).map(([name, c]) => ({ + name, + version: `${c.version_major}.${c.version_minor}.${c.version_revision}`, + thresholds: [c.mainnet_time, c.testnet_time, c.regtest_time, c.mainnet_block, c.testnet_block, c.regtest_block] + .map(String), + })); + assert.ok(out.length > 50, 'the registry built only ' + out.length + ' changes; the part files under src/protocol_changes/ changed shape'); return out; } describe('ACTION activation model', () => { - test('every ACTION is registered', { skip: !haveRegistry && 'xchain-indexer not present in this checkout' }, () => { + test('every ACTION is registered', { skip: indexer.skip }, () => { const names = new Set(readRegistry().map((r) => r.name)); const missing = ACTIONS.filter((a) => !names.has(a)); assert.deepEqual(missing, [], 'documented actions absent from protocol_changes.js: ' + missing.join(', ')); }); - test('no ACTION carries a non-zero activation time or height', { skip: !haveRegistry && 'xchain-indexer not present in this checkout' }, () => { + test('no ACTION carries a non-zero activation time or height', { skip: indexer.skip }, () => { const gated = readRegistry() .filter((r) => ACTIONS.includes(r.name)) .filter((r) => r.thresholds.some((t) => t !== '0')) @@ -92,7 +96,7 @@ describe('ACTION activation model', () => { 'version alone gates them:\n ' + gated.join('\n ')); }); - test('the documented 21/15 version split matches the registry', { skip: !haveRegistry && 'xchain-indexer not present in this checkout' }, () => { + test('the documented 21/17 version split matches the registry', { skip: indexer.skip }, () => { const acts = readRegistry().filter((r) => ACTIONS.includes(r.name)); const byVersion = {}; for (const a of acts) (byVersion[a.version] = byVersion[a.version] || []).push(a.name); @@ -100,7 +104,7 @@ describe('ACTION activation model', () => { assert.deepEqual(Object.keys(byVersion).sort(), ['0.1.0', '0.2.0'], 'actions are now registered at versions beyond 0.1.0/0.2.0: ' + Object.keys(byVersion).join(', ')); assert.equal(byVersion['0.1.0'].length, 21, 'v0.1.0 action count changed; update the docs'); - assert.equal(byVersion['0.2.0'].length, 16, 'v0.2.0 action count changed; update the docs'); + assert.equal(byVersion['0.2.0'].length, 17, 'v0.2.0 action count changed; update the docs'); assert.ok(byVersion['0.1.0'].includes('BET'), 'BET moved off 0.1.0; concepts/actions.md and components/indexer/actions.md name it as a 0.1.0 action'); }); diff --git a/test/action-count-claims.test.js b/test/action-count-claims.test.js index 4aa0bccc..124402fa 100644 --- a/test/action-count-claims.test.js +++ b/test/action-count-claims.test.js @@ -118,7 +118,7 @@ const SCOPED = [ + 'scan, and the gate gates the commit. count-action-suites.js scans the working tree, so ' + 'its file tally runs ahead of this number whenever a lane holds unlanded suites' }, { file: 'components/indexer/architecture.md', claim: '48 action', count: 1, - why: 'handler classes in xchain-indexer src/actions.js, not the ACTION set; ' + why: 'handler classes in xchain-indexer src/actions/index.js, not the ACTION set; ' + 'requires, instantiations and dispatch cases all counted 48 on 2026-08-06' }, { file: 'components/indexer/actions.md', claim: '21 actions', count: 1, why: 'the subset registered at protocol version 1.0.0; 21 + 16 = 37 on the same page' }, diff --git a/test/action-example-fields.test.js b/test/action-example-fields.test.js index b2dd44cf..8421f30a 100644 --- a/test/action-example-fields.test.js +++ b/test/action-example-fields.test.js @@ -34,7 +34,7 @@ * declared count. DEPLOY is precise about it: v0/v2 take CONSTRUCTOR_PARAMS * as a rest field while v1/v3 do not, because COOLDOWN_BLOCKS + * SLASH_DESTINATION trail the constructor args, and the doc marks exactly - * v0/v2 (matching deploy.js). + * v0/v2 (matching deploy/index.js). * - Some formats end in a bare `...` meaning "the preceding group repeats" * (ANCHOR/ATTEST signature pairs, PRICE pair lists). Also unbounded. * - BATCH is not pipe-counted at all: `VERSION|COMMAND;COMMAND` embeds whole @@ -341,7 +341,7 @@ describe('DISPENSER v0 examples match the declared format', () => { test('does not claim a first oracle price is effective immediately', () => { // dispenser.md and price.md contradicted each other and the code: // dispenser.md said the first price for a feed took effect immediately - // and only updates were delayed, while PriceAggregator.js applies a flat + // and only updates were delayed, while oracle/price_aggregator.js applies a flat // +86400 to EVERY publish (verified live: three rows, first publishes // included, all delay_seconds = 86400). price.md already documented the // uniform rule and the consensus reason for it. Someone following the old diff --git a/test/activation-narrative-claims.test.js b/test/activation-narrative-claims.test.js new file mode 100644 index 00000000..94fd4cb9 --- /dev/null +++ b/test/activation-narrative-claims.test.js @@ -0,0 +1,197 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * Activation-narrative claims in protocol/protocol-activation.md. + * + * WHY. That page is the only hand-written inventory of which consensus gates + * are live on which network, and it has now drifted from BOTH of its machine + * sources at once. The 2026-09-09 mainnet genesis arm set six validator-era + * maps to `mainnet: 0` in protocol/constants.js and the page went on calling + * them inert, and three further testnet time arms were pinned in the indexer + * registry while the page went on saying one gate was the only Cohort A rule + * not genesis-active on testnet. Neither drift touched a test: the only suite + * that read the page at all checked an unrelated wall-clock bound, so the page + * contradicted itself, contradicted constants.js, and contradicted the + * generated protocol/flag-days.md with everything green. + * + * WHAT IT CHECKS. + * + * 1. Mainnet claims. Every sentence on the page that says a named + * `*_ACTIVATION` map is null/inert/unset on mainnet must name a map that + * really holds `mainnet: null` in protocol/constants.js, and every + * sentence that says one is armed at genesis on mainnet must name a map + * that really holds `mainnet: 0`. + * 2. Testnet arms. Every gate with a nonzero testnet arm in the indexer + * registry is named in the page's testnet-exceptions list, and the page + * carries no "the only Cohort A rule not genesis-active on testnet" + * claim while more than one such arm exists. + * + * WHY THE COUNT ASSERTIONS ARE HERE. A prose parse that stops matching returns + * an empty set, and an empty set satisfies every "each of these must" loop + * ever written: the guard goes inert and stays green forever. So each half + * asserts a floor on what it actually matched, and the floor's failure message + * says the parse broke rather than that the page is wrong. + * + * Run: node --test test/activation-narrative-claims.test.js (Node 22) + * + ********************************************************************/ + +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); + +const gen = require('../bin/generate-flag-days.js'); +const { sibling } = require('./helpers/sibling_checkout.js'); + +const DOC_ROOT = path.join(__dirname, '..'); +const PAGE_REL = 'protocol/protocol-activation.md'; +const PAGE = fs.readFileSync(path.join(DOC_ROOT, PAGE_REL), 'utf8'); +const CONSTANTS = require(path.join(DOC_ROOT, 'protocol', 'constants.js')); +/* A bare clone skips the registry half below by name. A run that declared the sibling + * supplied (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set, with + * xchain-indexer in .ci-siblings) throws in the helper instead: there an absent registry + * means a dropped checkout, and the page's testnet claims would go unchecked while the + * file still reported green. */ +const noIndexer = sibling('xchain-indexer', [gen.REGISTRY]).skip; + +/** Every exported map with a `mainnet` slot, by name. */ +const MAPS = new Map( + Object.entries(CONSTANTS).filter( + ([k, v]) => /_ACTIVATION$/.test(k) && v && typeof v === 'object' && 'mainnet' in v, + ), +); + +/** + * Prose sentences, with the page's hard wraps flattened away. + * + * Table rows are dropped first. A row states its thresholds in a COLUMN, not + * in a sentence, and rows carry so few full stops that several of them + * flatten into one pseudo-sentence holding both an inert and an armed claim + * about different gates. Parsing those is how a prose guard starts inventing + * failures; the cohort and decoder tables are covered by the columns the + * generated flag-day page checks. + */ +function sentences(markdown) { + return markdown + .split('\n') + .filter((line) => !/^\s*[|#]/.test(line)) + .join(' ') + .split(/(?<=\.)\s+/) + .map((s) => s.trim()) + .filter(Boolean); +} + +/** Backticked map names in one sentence that constants.js actually declares. */ +function namedMaps(sentence) { + const out = []; + for (const m of sentence.matchAll(/`([A-Z][A-Z0-9_]*_ACTIVATION)`/g)) { + if (MAPS.has(m[1])) out.push(m[1]); + } + return out; +} + +// A claim is about mainnet only when the sentence says so. "unarmed on +// **testnet**" sits two sentences from the mainnet roster and is not this +// guard's business. +const MAINNET = /mainnet/i; +const INERT = /\bnull\b|inert|unset|unarmed|disarmed|not armed/i; +const ARMED = /armed at genesis|genesis-active|genesis arm|arms at|reads `0`/i; + +test('mainnet inert/armed claims on the activation page match constants.js', () => { + let checked = 0; + for (const sentence of sentences(PAGE)) { + if (!MAINNET.test(sentence)) continue; + const inert = INERT.test(sentence); + const armed = ARMED.test(sentence); + // A sentence carrying both cues is describing the SPLIT, naming armed + // and unarmed maps side by side; the genesis-arm section's exception + // roster is one long sentence of exactly that shape. Guessing which + // name goes with which cue is how a prose parse starts inventing + // failures, so those are left to the per-claim sentences around them. + if (inert === armed) continue; + for (const name of namedMaps(sentence)) { + checked += 1; + if (inert) { + assert.strictEqual( + MAPS.get(name).mainnet, null, + `${PAGE_REL} calls ${name} inert/unset on mainnet, but protocol/constants.js ` + + `ships mainnet: ${JSON.stringify(MAPS.get(name).mainnet)}. The page is stale ` + + 'against the constant; correct the prose, never the constant.', + ); + } else { + assert.strictEqual( + MAPS.get(name).mainnet, 0, + `${PAGE_REL} calls ${name} armed at genesis on mainnet, but ` + + `protocol/constants.js ships mainnet: ${JSON.stringify(MAPS.get(name).mainnet)}.`, + ); + } + } + } + assert.ok( + checked >= 7, + `only ${checked} mainnet activation claims matched on ${PAGE_REL}, which is fewer than the ` + + 'seven this page has carried since the 2026-09-09 genesis arm. Either the roster was ' + + 'deleted or the sentence parse in this guard stopped matching it; an unmatched roster ' + + 'passes every assertion above, so this floor is the only thing that reports it.', + ); +}); + +const LEADIN = 'genesis-active as well, with'; + +/** The testnet-exceptions bullet list, from its lead-in to the blank line that ends it. */ +function exceptionsSection() { + const start = PAGE.indexOf(LEADIN); + assert.notStrictEqual( + start, -1, + `the testnet-exceptions lead-in ("${LEADIN}") is gone from ${PAGE_REL}. If the section was ` + + 'renamed, retarget this guard; it cannot check a list it cannot find.', + ); + const rest = PAGE.slice(start); + // The bullets and their continuation lines start with "-" or with spaces, + // so the first blank line followed by anything else ends the list. + const end = rest.match(/\n\n(?![-\s])/); + return end ? rest.slice(0, end.index) : rest; +} + +test('every nonzero testnet arm is named in the page\'s exception list', { + skip: noIndexer, +}, () => { + const arms = gen.collectTestnetArms(); + assert.ok( + arms.length >= 1, + 'collectTestnetArms() returned nothing. Either every testnet arm was un-pinned, or the ' + + 'registry parse in bin/generate-flag-days.js stopped matching; an empty set would ' + + 'satisfy the loop below without reading the page at all.', + ); + const section = exceptionsSection(); + for (const { gate } of arms) { + assert.ok( + section.includes(`\`${gate}\``), + `${gate} arms testnet at an instant of its own in the xchain-indexer protocol_changes registry, ` + + `but ${PAGE_REL} does not name it in the testnet-exceptions list. A new testnet arm ` + + 'was pinned and the activation narrative needs rewording: name the gate and link to ' + + 'Flag-Day Values, and do not write the instant into the prose ' + + '(test/flag-day-literals.test.js enforces that).', + ); + } + if (arms.length > 1) { + assert.doesNotMatch( + PAGE, + /only Cohort A rule not genesis-active on testnet/, + `${PAGE_REL} still claims one gate is the only Cohort A rule not genesis-active on ` + + `testnet, but the registry declares ${arms.length} nonzero testnet arms.`, + ); + } +}); diff --git a/test/bridge-docs-verdict-strings.test.js b/test/bridge-docs-verdict-strings.test.js new file mode 100644 index 00000000..908fcb19 --- /dev/null +++ b/test/bridge-docs-verdict-strings.test.js @@ -0,0 +1,152 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * Bridge and token-bridge doc verdict-string conformance (lane L9, + * build-through run: base spec row B12, token spec row T9, policy spec + * row 7's docs share). + * + * WHY. These docs are read by wallet, SDK and dApp authors who copy the + * verdict strings verbatim to detect a specific refusal. A doc that + * paraphrases or typos a verdict is worse than one that omits it: a client + * built against the paraphrase never matches the real `STATUS` string the + * indexer emits. This test pins the verdict strings this lane's docs state + * against the wire-exact strings the three specs rule (byte for byte, + * quoted from `xchain-bridge.md`, `xchain-token-bridge.md` + * and `xchain-token-bridge-policy.md`), so a doc edit that drifts from the + * consensus string fails loudly here instead of shipping silently wrong. + */ + +'use strict'; + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); + +const ROOT = path.join(__dirname, '..'); +const read = (p) => fs.readFileSync(path.join(ROOT, p), 'utf8'); + +// Verdict strings this lane's docs must state, quoted byte for byte from +// the three ruled specs (never retyped or paraphrased at the call site). +const VERDICTS = { + 'protocol/actions/xbridge.md': [ + 'invalid: XBRIDGE before activation', + 'invalid: XBRIDGE (BTC only)', + 'invalid: XBRIDGE v1 is not valid on BTC', + 'invalid: XBRIDGE v2 is system-injected', + 'invalid: ORIGIN_ADDRESS', + 'invalid: TICK (not native here)', + 'invalid: XBRIDGE v5 is system-injected', + ], + 'protocol/actions/issue.md': [ + 'invalid: TICK (BTC-only)', + 'invalid: BRIDGE_CHAINS', + 'invalid: BRIDGE_CHAINS (locked)', + 'invalid: TICK (subassets are not bridgeable yet)', + 'invalid: TICK (policy-bound tokens are not bridgeable yet)', + 'invalid: TICK (bridged tokens cannot be policy-bound yet)', + 'invalid: TICK (policy list exceeds XPOLICY_MAX_MEMBERS)', + // R8, amended into the token spec at 04:20Z: reserved future + // chain roots and the four-character floor on new top-level ticks. + 'invalid: TICK (length)', + 'invalid: TICK (reserved)', + ], + 'protocol/actions/list.md': [ + 'invalid: LIST_ACTION_INDEX (bridge-owned)', + ], +}; + +test('bridge action docs state the exact, unparaphrased verdict strings the specs rule', () => { + for (const [file, verdicts] of Object.entries(VERDICTS)) { + const text = read(file); + for (const verdict of verdicts) { + assert.ok( + text.includes(verdict), + `${file} is missing the exact verdict string "${verdict}"` + ); + } + } +}); + +test('the XBRIDGE format table on the action page names all six versions', () => { + const text = read('protocol/actions/xbridge.md'); + for (let v = 0; v <= 5; v += 1) { + assert.ok( + text.includes(`Version \`${v}\``), + `xbridge.md is missing a "Version \`${v}\`" heading` + ); + } +}); + +test('ISSUE format 7 (bridge opt-in) params are documented', () => { + const text = read('protocol/actions/issue.md'); + for (const field of ['BRIDGE_CHAINS', 'MIN_DEPTH', 'LOCK_BRIDGE']) { + assert.ok(text.includes(`\`${field}\``), `issue.md is missing the \`${field}\` param`); + } + assert.ok( + text.includes('Bridge opt-in'), + 'issue.md format 7 must be labelled "Bridge opt-in"' + ); +}); + +test('R8 tick-namespace reservation is documented in issue.md and the action index', () => { + for (const file of ['protocol/actions/issue.md', 'protocol/actions/README.md']) { + const text = read(file); + assert.ok( + text.includes('TICK_NAMESPACE_ACTIVATION'), + `${file} must name the TICK_NAMESPACE_ACTIVATION flag` + ); + assert.ok( + /four.character/.test(text), + `${file} must state the four-character floor on new top-level ticks` + ); + assert.ok( + /RESERVED_FUTURE_ROOTS/.test(text), + `${file} must name the RESERVED_FUTURE_ROOTS list` + ); + } +}); + +test('the policy-inheritance section exists exactly where the doc pages link it', () => { + const text = read('protocol/token-bridge.md'); + assert.match(text, /^## Policy inheritance$/m); + // every cross-reference this lane added to "#policy-inheritance" must + // resolve to that exact heading slug (mirrors what + // internal-link-integrity.test.js checks repo-wide, scoped here to this + // lane's own edits so a slug rename is caught even before that suite runs). + const referrers = ['protocol/actions/issue.md', 'protocol/actions/list.md', 'concepts/token-bridge.md']; + for (const referrer of referrers) { + const referrerText = read(referrer); + if (referrerText.includes('#policy-inheritance')) { + assert.match( + referrerText, + /token-bridge\.md#policy-inheritance/, + `${referrer} links #policy-inheritance but not via token-bridge.md` + ); + } + } +}); + +test('gas.md no longer states XCHAIN is BTC-only without qualification', () => { + const text = read('concepts/gas.md'); + // The three lines D50 named must each acknowledge the bridge's shadow + // balance now that XBRIDGE exists, not read as an absolute platform-wide + // BTC-only claim for XCHAIN's existence. + assert.ok( + /BTC only for now/.test(text), + 'gas.md fee-payment lines must qualify "BTC only" now that XCHAIN-balance fees off BTC are a stated future milestone, not a permanent limit' + ); + assert.ok( + /shadow balance/.test(text), + 'gas.md must mention the bridge shadow balance next to the BTC-only issuance statement' + ); +}); diff --git a/test/complete_run_reporter.test.js b/test/complete_run_reporter.test.js new file mode 100644 index 00000000..bd2ec48c --- /dev/null +++ b/test/complete_run_reporter.test.js @@ -0,0 +1,208 @@ +'use strict'; + +// A run in which a file's child exited 0 before its event stream was whole +// must not grade green: proved over synthetic streams and by driving a real +// runner over a throwaway file that exits mid-suite. + +const assert = require('node:assert/strict'); +const { test, describe } = require('node:test'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const { spawnSync } = require('node:child_process'); + +const reporter = require('../bin/complete_run_reporter.js'); +const { audit } = reporter; + +const REPORTER = path.join(__dirname, '..', 'bin', 'complete_run_reporter.js'); + +const A = '/tree/test/a.test.js'; +const B = '/tree/test/b.test.js'; + +const counts = (tests) => ({ tests, failed: 0, passed: tests, cancelled: 0, skipped: 0, todo: 0, topLevel: tests, suites: 0 }); +const pass = (file, name) => ({ type: 'test:pass', data: { file, name, nesting: 0, details: { type: 'test' } } }); +const suitePass = (file, name) => ({ type: 'test:pass', data: { file, name, nesting: 0, details: { type: 'suite' } } }); +const start = (file, name) => ({ type: 'test:start', data: { file, name, nesting: 0 } }); +const fileSummary = (file, tests) => ({ type: 'test:summary', data: { file, success: true, counts: counts(tests) } }); +const runSummary = (tests) => ({ type: 'test:summary', data: { file: undefined, success: true, counts: counts(tests) } }); +// The parent's own placeholder pass for a file whose child reported nothing. +const placeholder = (file) => ({ type: 'test:pass', data: { file, name: file, nesting: 0, details: { type: 'test' } } }); +const source = (text) => () => text; + +describe('audit over an event stream', () => { + test('a whole run, every file summarised and the counts adding up, has no problems', async () => { + const verdict = await audit([ + pass(A, 'one'), pass(A, 'two'), suitePass(A, 'group'), fileSummary(A, 2), + pass(B, 'three'), fileSummary(B, 1), + runSummary(3), + ]); + assert.deepEqual(verdict, { problems: [], files: 2, total: 3 }); + }); + + // The captured shape: the file's stream stops on a test:start, the parent + // still prints a summary of what it saw, and the exit code was 0. + test('a file whose stream ended before its summary is named, with what was seen last', async () => { + const { problems } = await audit([ + pass(A, 'one'), pass(A, 'two'), start(A, 'three'), + pass(B, 'four'), fileSummary(B, 1), + runSummary(3), + ]); + assert.equal(problems.length, 1); + assert.match(problems[0], /a\.test\.js ended without reporting its summary after 2 of its tests were seen/); + assert.match(problems[0], /last event: test:start three/); + }); + + test('a file that loads node:test but delivered nothing at all is refused', async () => { + const { problems } = await audit([ + { type: 'test:enqueue', data: { file: A, name: A, nesting: 0 } }, + placeholder(A), + pass(B, 'four'), fileSummary(B, 1), + runSummary(2), + ], source("const { test } = require('node:test');")); + assert.equal(problems.length, 1); + assert.match(problems[0], /a\.test\.js loads node:test but delivered no events at all/); + }); + + test('a helper under test/ that never touches node:test is graded by its placeholder pass alone', async () => { + const verdict = await audit([ + { type: 'test:enqueue', data: { file: A, name: A, nesting: 0 } }, + placeholder(A), + pass(B, 'four'), fileSummary(B, 1), + runSummary(2), + ], source("module.exports = { resolve: () => 1 };")); + assert.deepEqual(verdict, { problems: [], files: 2, total: 2 }); + }); + + test('a silent file that cannot be read back is refused rather than assumed to be a helper', async () => { + const { problems } = await audit([placeholder(A), runSummary(1)], () => { throw new Error('ENOENT'); }); + assert.equal(problems.length, 1); + assert.match(problems[0], /a\.test\.js loads node:test but delivered no events at all/); + }); + + test('a file summary that disagrees with the events the parent saw is a problem', async () => { + const { problems } = await audit([pass(A, 'one'), fileSummary(A, 2), runSummary(1)]); + assert.deepEqual(problems, ['test/a.test.js reported 2 tests but the runner saw 1'.replace('test/a.test.js', path.relative(process.cwd(), A))]); + }); + + test('per-file totals that do not add up to the run total are a problem', async () => { + const { problems } = await audit([ + pass(A, 'one'), fileSummary(A, 1), + pass(B, 'two'), fileSummary(B, 1), + runSummary(3), + ]); + assert.deepEqual(problems, ['the files that reported account for 2 tests but the runner counted 3']); + }); + + test('a run with no cumulative summary at all is refused rather than trusted', async () => { + const { problems } = await audit([pass(A, 'one'), fileSummary(A, 1)]); + assert.equal(problems.length, 1); + assert.match(problems[0], /ended without its cumulative summary/); + }); + + test('the reporter prints nothing for a whole run and one line per problem otherwise', async () => { + const collect = async (events) => { + const out = []; + for await (const chunk of reporter(events)) out.push(chunk); + return out.join(''); + }; + const before = process.exitCode; + try { + assert.equal(await collect([pass(A, 'one'), fileSummary(A, 1), runSummary(1)]), ''); + assert.equal(process.exitCode, before); + + const text = await collect([pass(A, 'one'), start(A, 'two'), runSummary(1)]); + assert.match(text, /^test run incomplete, refusing to grade it green:\n {2}.*a\.test\.js ended without reporting its summary/); + assert.equal(process.exitCode, 1); + } finally { + process.exitCode = before; + } + }); +}); + +// Driving the real runner: one healthy file and one whose second test calls +// process.exit(0), so that child exits clean before its third test or its +// summary can be written, the same cut the force-exit truncation made. + +function runner(cwd, files) { + // This test itself runs inside a runner child; the nested runner must not + // inherit that marker or it declines to run files at all. + const env = { ...process.env }; + delete env.NODE_TEST_CONTEXT; + return spawnSync(process.execPath, [ + '--test', '--test-timeout=20000', + '--test-reporter=tap', '--test-reporter-destination=stdout', + `--test-reporter=${REPORTER}`, '--test-reporter-destination=stderr', + ...files, + ], { cwd, env, encoding: 'utf8' }); +} + +const HEALTHY = ` +'use strict'; +const { test } = require('node:test'); +test('one', () => {}); +test('two', () => {}); +`; + +const EARLY_EXIT = ` +'use strict'; +const { test } = require('node:test'); +test('runs', () => {}); +test('exits the child clean before the rest can report', () => { process.exit(0); }); +test('never runs', () => {}); +`; + +// The same exit after a turn of the event loop, so the events before it +// have reached the parent: the stream is cut instead of empty. +const LATE_EXIT = ` +'use strict'; +const { test } = require('node:test'); +const { setTimeout: sleep } = require('node:timers/promises'); +test('runs', () => {}); +test('exits the child clean after its first events were written', async () => { await sleep(200); process.exit(0); }); +test('never runs', () => {}); +`; + +const HELPER = ` +'use strict'; +module.exports = { answer: () => 42 }; +`; + +describe('driving node --test with the reporter attached', () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'complete-run-reporter-')); + fs.writeFileSync(path.join(dir, 'healthy.test.js'), HEALTHY); + fs.writeFileSync(path.join(dir, 'early_exit.test.js'), EARLY_EXIT); + fs.writeFileSync(path.join(dir, 'late_exit.test.js'), LATE_EXIT); + fs.writeFileSync(path.join(dir, 'helper.js'), HELPER); + + test('a run whose files all finish exits 0 and the reporter stays silent', () => { + const r = runner(dir, ['healthy.test.js']); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /^# tests 2$/m); + assert.equal(r.stderr, ''); + }); + + test('a file that exits 0 before its stream is whole fails the run and is named', () => { + const r = runner(dir, ['healthy.test.js', 'late_exit.test.js']); + // Without the reporter this run is green: the child's exit code was 0. + assert.match(r.stdout, /^# fail 0$/m, r.stdout); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match(r.stderr, /test run incomplete, refusing to grade it green:/); + assert.match(r.stderr, /late_exit\.test\.js ended without reporting its summary after 1 of its tests/); + assert.doesNotMatch(r.stderr, /healthy\.test\.js/); + }); + + test('a file that exits 0 before writing a single event fails the run too', () => { + const r = runner(dir, ['healthy.test.js', 'early_exit.test.js']); + assert.match(r.stdout, /^ok \d+ - early_exit\.test\.js$/m, r.stdout); + assert.equal(r.status, 1, r.stdout + r.stderr); + assert.match(r.stderr, /early_exit\.test\.js loads node:test but delivered no events at all/); + }); + + test('a helper the glob swept in, with no tests and no node:test, passes as the runner grades it', () => { + const r = runner(dir, ['healthy.test.js', 'helper.js']); + assert.equal(r.status, 0, r.stdout + r.stderr); + assert.match(r.stdout, /^ok \d+ - helper\.js$/m); + assert.match(r.stdout, /^# tests 3$/m); + assert.equal(r.stderr, ''); + }); +}); diff --git a/test/consensus-wall-clock-claims.test.js b/test/consensus-wall-clock-claims.test.js index 75370448..7d0d51a4 100644 --- a/test/consensus-wall-clock-claims.test.js +++ b/test/consensus-wall-clock-claims.test.js @@ -44,15 +44,30 @@ const assert = require('node:assert/strict'); const test = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const ROOT = path.resolve(__dirname, '..'); const VM_SRC = path.resolve(ROOT, '../xchain-vm/src'); -const WALL_CLOCK_JS = path.join(VM_SRC, 'consensus-wall-clock.js'); +// The VM's layout pass renamed src/consensus-wall-clock.js to +// src/consensus_wall_clock.js and left nothing at the old path, so a sibling +// checkout sits on one side of that move or the other. Pinning one spelling +// skips every assertion in this file against the other side while the run still +// reports green, so try the post-move spelling and fall back to the pre-move one. +const WALL_CLOCK_JS = [path.join(VM_SRC, 'consensus_wall_clock.js'), + path.join(VM_SRC, 'consensus-wall-clock.js')] + .find((p) => fs.existsSync(p)) || path.join(VM_SRC, 'consensus_wall_clock.js'); const VM_INDEX_JS = path.join(VM_SRC, 'index.js'); -const haveVm = fs.existsSync(WALL_CLOCK_JS) && fs.existsSync(VM_INDEX_JS); -const noVm = 'sibling xchain-vm not present in this checkout'; +/* The skips below are for a bare clone, by name. A run that declared the sibling + * supplied (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set) throws in + * the helper instead, naming both spellings tried: the rule these pages describe lives + * in that checkout, and skipping leaves the drift this file exists to catch unchecked + * but green. */ +const noVm = sibling('xchain-vm', [ + [path.join(VM_SRC, 'consensus_wall_clock.js'), path.join(VM_SRC, 'consensus-wall-clock.js')], + VM_INDEX_JS, +]).skip; const readDoc = (rel) => fs.readFileSync(path.join(ROOT, rel), 'utf8'); const readVm = (file) => fs.readFileSync(file, 'utf8'); @@ -85,9 +100,9 @@ function activationBody(src) { } test('the wall-clock budget the VM pages quote is the constant xchain-vm declares', - { skip: !haveVm && noVm }, () => { + { skip: noVm }, () => { const declared = sourceConstant(readVm(WALL_CLOCK_JS), 'CONSENSUS_MAX_WALL_MS', - 'xchain-vm/src/consensus-wall-clock.js'); + WALL_CLOCK_JS); const printed = `${declared.toLocaleString('en-US')} ms`; for (const page of [CONFIG_PAGE, OPERATIONS_PAGE, ACTIVATION_PAGE]) { @@ -102,19 +117,19 @@ test('the wall-clock budget the VM pages quote is the constant xchain-vm declare }); test('the enforcing VM re-exports the same constant the budget module declares', - { skip: !haveVm && noVm }, () => { + { skip: noVm }, () => { const wallClock = readVm(WALL_CLOCK_JS); const index = readVm(VM_INDEX_JS); assert.match(wallClock, /module\.exports\s*=\s*\{[\s\S]*CONSENSUS_MAX_WALL_MS/, - 'consensus-wall-clock.js no longer exports CONSENSUS_MAX_WALL_MS; the docs ' + `${WALL_CLOCK_JS} no longer exports CONSENSUS_MAX_WALL_MS; the docs ` + 'present it as a readable protocol constant'); assert.match(index, /module\.exports\.CONSENSUS_MAX_WALL_MS\s*=\s*CONSENSUS_MAX_WALL_MS;/, 'xchain-vm/src/index.js no longer re-exports CONSENSUS_MAX_WALL_MS'); }); test('the activation the VM configuration page describes is the one the VM resolves', - { skip: !haveVm && noVm }, () => { + { skip: noVm }, () => { const body = activationBody(readVm(VM_INDEX_JS)); // Pre-launch networks: unconditional, no flag-day comparison in that arm. @@ -146,7 +161,7 @@ test('the activation the VM configuration page describes is the one the VM resol }); test('the flag day the VM rides is the contract-era instant the generated page publishes', - { skip: !haveVm && noVm }, () => { + { skip: noVm }, () => { const gate = sourceConstant(readVm(VM_INDEX_JS), 'BINARY_ALLOC_GATE_BLOCK_TIME', 'xchain-vm/src/index.js'); const flagDays = readDoc(FLAG_DAYS_PAGE); @@ -165,9 +180,9 @@ test('the flag day the VM rides is the contract-era instant the generated page p }); test('the CPU-time knob is documented as non-binding for consensus executions', - { skip: !haveVm && noVm }, () => { + { skip: noVm }, () => { const index = readVm(VM_INDEX_JS); - assert.match(index, /_wallClockBudgetMs/, + assert.match(index, /wallClockBudgetMs/, 'the per-execution budget resolver is gone from xchain-vm/src/index.js; the ' + 'docs describe a resolved budget rather than the raw knob'); diff --git a/test/contract-state-proof-availability.test.js b/test/contract-state-proof-availability.test.js index 2e62e9c0..1b0a7a14 100644 --- a/test/contract-state-proof-availability.test.js +++ b/test/contract-state-proof-availability.test.js @@ -49,11 +49,13 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.resolve(__dirname, '..'); const ACTIVATION = path.resolve(__dirname, '../../xchain-explorer/src/state_subtree_activation.js'); -const haveExplorer = fs.existsSync(ACTIVATION); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the armed map is unreadable. +const explorer = sibling('xchain-explorer', [ACTIVATION]); // Every tracked .md page except history and vendored trees. function docPages(dir, out) { @@ -86,7 +88,7 @@ function staleLines(file) { } describe('contract-state proof availability in documentation', () => { - test('no page calls the endpoint unimplemented while the slot is armed somewhere', { skip: !haveExplorer && 'xchain-explorer not present in this checkout' }, () => { + test('no page calls the endpoint unimplemented while the slot is armed somewhere', { skip: explorer.skip }, () => { const { STATE_SUBTREE_ACTIVATION } = require(ACTIVATION); const armed = Object.keys(STATE_SUBTREE_ACTIVATION.contract_state_root || {}); if (armed.length === 0) return; // pre-arming: the stale sentences are true diff --git a/test/corpus-consistency-claims.test.js b/test/corpus-consistency-claims.test.js new file mode 100644 index 00000000..5c406ee6 --- /dev/null +++ b/test/corpus-consistency-claims.test.js @@ -0,0 +1,244 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * Drift lint for four cross-page claims this corpus has already contradicted + * itself about. Each block below guards one of them. + * + * WHY, per claim: + * + * 1. Multisig payload capacity. The encoder's MULTISIGN_SIZE is 69: a chunk + * is 4 magic bytes plus 60 data bytes, and one chunk fills BOTH 32-byte + * fake-pubkey halves of a SINGLE output. So capacity is 60 bytes per + * output. Four pages said "~61 bytes per key", a figure traceable to a + * superseded MULTISIGN_SIZE = 71, and the whitepaper table said "60 data + * bytes per key slot", right number and wrong unit, contradicting its own + * "Per-output data capacity" column header. Someone sizing a transaction + * off the per-key reading budgets twice the real capacity. + * + * 2. Indexer inputs and replay. The indexer reads a local Hub DB mirror + * during block processing (architecture/database-design.md documents it, + * and data-pipeline.md's own PRICE oracle flow draws it), yet the + * Determinism sections promised bit-for-bit replay against the Decoder DB + * ALONE. A third-party implementer reproducing state from one chain's + * Decoder DB would diverge on every fee validation and oracle read. + * + * 3. Explorer database writes. The explorer owns and writes a hub-mirror + * schema under `"self_sync": true`, the RECOMMENDED provisioning mode, and + * optionally writes the indexer-owned `icons` table. Two pages still said + * it "never writes to any database", so an operator provisioning grants + * from them hands the explorer a read-only user and the recommended mode + * fails at startup. + * + * 4. The three penalty lanes. Stake is burned only on a permissionless SLASH + * proof of equivocation. Price deviation, repeated deviation and missed + * rounds are hub-local offenses whose strongest outcome is + * `validators.status='suspended'`, with on-chain stake untouched, and + * ROLLCALL eviction burns nothing. Calling the missed-rounds knob a + * "non-participation slash" tells a validator operator their stake is at + * risk when it is not. This claim has drifted back once already after a + * partial correction, which is why it is pinned here. + * + * The capacity figure is read out of the sibling xchain-encoder checkout + * rather than typed here, and SKIPS when that sibling is absent: the + * convention fee-and-limit-claims.test.js and consensus-wall-clock-claims.js + * both use. The prose assertions are doc-internal and always run. + * + ********************************************************************/ + +const assert = require('node:assert/strict'); +const test = require('node:test'); +const fs = require('node:fs'); +const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); + +const ROOT = path.resolve(__dirname, '..'); +const ENCODER_SRC = path.resolve(ROOT, '../xchain-encoder/src/XChainEncoder.js'); +const ENCODER_CONSTANTS = path.resolve(ROOT, '../xchain-encoder/src/XChainEncoder/constants.js'); + +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the +// sibling checkout itself is absent or hollow. Which file inside it declares +// MULTISIGN_SIZE is not a skip condition (see multisignDataBytes): a repo that +// is checked out but declares the constant nowhere is a hard failure, not a skip. +const noEncoder = sibling('xchain-encoder', []).skip; + +const readDoc = (rel) => fs.readFileSync(path.join(ROOT, rel), 'utf8'); + +// MULTISIGN_SIZE minus the magic word and the five single-byte script fields +// the encoder subtracts at prepareData time. Kept as arithmetic over the +// declared constant so a change to the constant moves this guard by itself. +// +// Reads src/XChainEncoder/constants.js first (the per-feature split) and +// falls back to the monolithic src/XChainEncoder.js (the pre-split layout), +// so this guard survives either shape of the encoder checkout. Fails loudly +// naming both paths when neither declares the constant, rather than skipping: +// an unreadable sibling is a skip, but a readable one with the declaration +// gone from both known homes is a corpus-guard defect that must not go quiet. +function multisignDataBytes() { + for (const src of [ENCODER_CONSTANTS, ENCODER_SRC]) { + if (!fs.existsSync(src)) continue; + const m = /^const MULTISIGN_SIZE\s*=\s*(\d+)/m.exec(fs.readFileSync(src, 'utf8')); + if (m) { + const MAGIC_LEN = 4; + return Number(m[1]) - MAGIC_LEN - 5; + } + } + assert.fail('MULTISIGN_SIZE declaration not found in xchain-encoder/src/XChainEncoder/' + + 'constants.js or xchain-encoder/src/XChainEncoder.js; the declaration shape ' + + 'changed, re-point this regex'); +} + +/* ---------------------------------------------------------------- claim 1 */ + +const CAPACITY_PAGES = [ + 'components/encoder/README.md', + 'architecture/data-pipeline.md', + 'architecture/component-map.md', + 'whitepaper.md', +]; + +// Multisig capacity stated on a PER-KEY basis, in any of the spellings the +// corpus used. Deliberately anchored on the word "key", so it cannot match +// protocol/token-gated-content.md's unrelated "+~61 bytes envelope" ECIES +// overhead, which is a correct sentence about a different subject. +const PER_KEY_CAPACITY = /\d{2}\s*(?:data\s+)?bytes\s*(?:per|\/)\s*key/i; + +test('no page states multisig payload capacity on a per-key basis', () => { + const offenders = []; + for (const rel of CAPACITY_PAGES) { + readDoc(rel).split('\n').forEach((line, i) => { + if (PER_KEY_CAPACITY.test(line)) offenders.push(`${rel}:${i + 1} ${line.trim()}`); + }); + } + assert.deepEqual(offenders, [], + 'multisig capacity is 60 bytes per OUTPUT, spread over two 32-byte fake ' + + 'pubkey halves of one output, not a per-key quantity. Offending lines:\n' + + offenders.join('\n')); +}); + +test('the capacity the pages publish equals what xchain-encoder computes', + { skip: noEncoder }, () => { + const bytes = multisignDataBytes(); + assert.equal(bytes, 60, + 'the encoder no longer yields 60 data bytes per MULTISIGN chunk; ' + + 'sweep the capacity figure through ' + CAPACITY_PAGES.join(', ')); + const perOutput = new RegExp(`${bytes}\\s*(?:data\\s+)?bytes\\s*(?:per|\\/)\\s*(?:multisig\\s+)?output`, 'i'); + for (const rel of CAPACITY_PAGES) { + assert.ok(perOutput.test(readDoc(rel)), + `${rel} no longer states the multisig capacity as ${bytes} bytes per output`); + } + }); + +/* ---------------------------------------------------------------- claim 2 */ + +const REPLAY_PAGES = ['architecture/data-pipeline.md', 'whitepaper.md']; + +// Any sentence promising replay or convergence against the decoder DB must +// name the hub mirror in the same sentence. Matching per sentence rather than +// per file is the point: a qualifier three paragraphs away does not reach the +// reader of the bullet. +const REPLAY_CLAIM = /(bit-for-bit|converge to|pure function of the decoder db|only reads from the decoder db)/i; +const MIRROR_MENTION = /hub[\s-]?(db|mirror|mirrored)/i; + +test('every replay or convergence claim names the hub mirror in the same sentence', () => { + const offenders = []; + for (const rel of REPLAY_PAGES) { + for (const sentence of readDoc(rel).split(/(?<=[.!?])\s+|\n/)) { + if (REPLAY_CLAIM.test(sentence) && !MIRROR_MENTION.test(sentence)) { + offenders.push(`${rel} ${sentence.trim()}`); + } + } + } + assert.deepEqual(offenders, [], + 'the indexer reads the local Hub DB mirror during block processing ' + + '(architecture/database-design.md), so replay holds given the Decoder DB ' + + 'AND an equivalent mirror. Unqualified claims:\n' + offenders.join('\n')); +}); + +/* ---------------------------------------------------------------- claim 3 */ + +const EXPLORER_PAGES = [ + 'components/explorer/README.md', + 'components/explorer/architecture.md', + 'architecture/component-map.md', +]; + +test('no page claims the explorer never writes to any database', () => { + const offenders = []; + for (const rel of EXPLORER_PAGES) { + readDoc(rel).split('\n').forEach((line, i) => { + if (/never writes to (any|the Indexer) database/i.test(line)) { + offenders.push(`${rel}:${i + 1} ${line.trim()}`); + } + }); + } + assert.deepEqual(offenders, [], + 'the explorer creates and writes its own hub-mirror schema under ' + + '"self_sync": true, and optionally writes the indexer-owned icons table. ' + + 'Offending lines:\n' + offenders.join('\n')); +}); + +test('the explorer pages an operator reads name the hub-mirror writes', () => { + for (const rel of EXPLORER_PAGES) { + assert.match(readDoc(rel), /self_sync|hub[\s-]?mirror/i, + `${rel} does not mention the hub mirror, so an operator reading it alone ` + + 'provisions the wrong database grants'); + } +}); + +/* ---------------------------------------------------------------- claim 4 */ + +const PENALTY_PAGES = [ + 'components/hub/configuration.md', + 'components/hub/architecture.md', + 'components/hub/database.md', + 'components/hub/README.md', +]; + +// A non-equivocation offense tied to a burn. The identifiers SLASH_*, +// SlashDetector and slash_proposals are legitimate names and must survive, so +// the penalty alternation ends at a word boundary that a following `_` or +// letter defeats: SLASH_MISSED_ROUNDS_THRESHOLD, SlashDetector and +// slash_proposals cannot match it, while the bare noun in "non-participation +// slash" can. That bare form is the one the corpus actually shipped, so +// leaving it out made this guard green against the very drift it names. +// Neither pattern crosses a sentence end or a table-cell boundary, which is +// what keeps the corrected rows (offense recorded in one sentence, equivocation +// SLASH named in the next) from tripping it. +const PENALTY = String.raw`(?:\bslash(?:es|ed|ing)?\b|\bburn(?:s|ed|ing)?\b)`; +const OFFENSE = String.raw`(?:missed[\s-]?round|non[\s-]?participation|price deviation)`; +const FALSE_BURN = new RegExp(`${OFFENSE}[^.\\n|]{0,90}${PENALTY}`, 'i'); +const FALSE_BURN_REVERSED = new RegExp(`${PENALTY}[^.\\n|]{0,90}${OFFENSE}`, 'i'); + +test('no hub page ties a non-equivocation offense to a stake burn', () => { + const offenders = []; + for (const rel of PENALTY_PAGES) { + readDoc(rel).split('\n').forEach((line, i) => { + if (FALSE_BURN.test(line) || FALSE_BURN_REVERSED.test(line)) { + offenders.push(`${rel}:${i + 1} ${line.trim()}`); + } + }); + } + assert.deepEqual(offenders, [], + 'stake burns only on a permissionless SLASH proof of equivocation; deviation ' + + 'and missed rounds are hub-local offenses that leave on-chain stake untouched ' + + '(components/hub/decentralization.md). Offending lines:\n' + offenders.join('\n')); +}); + +test('decentralization.md still carries all three penalty lanes', () => { + const page = readDoc('components/hub/decentralization.md'); + assert.match(page, /burned only on a permissionless SLASH proof of \*\*equivocation\*\*/, + 'the equivocation-only burning lane is no longer stated'); + assert.match(page, /\*\*On-chain stake is untouched\*\*/, + 'the hub-local suspension lane no longer states that stake is untouched'); + assert.match(page, /nothing is burned/, + 'the ROLLCALL eviction lane no longer states that nothing is burned'); +}); diff --git a/test/deploy-code-encoding.test.js b/test/deploy-code-encoding.test.js index c949c01a..69e7debc 100644 --- a/test/deploy-code-encoding.test.js +++ b/test/deploy-code-encoding.test.js @@ -14,7 +14,7 @@ * * WHY. Inline DEPLOY v0/v1 contract code is base64 at and after the * DEPLOY_BASE64_CODE activation and hex before it. The indexer round-trips the - * field and rejects non-canonical base64 (xchain-indexer/src/actions/deploy.js), + * field and rejects non-canonical base64 (xchain-indexer/src/actions/deploy/index.js), * and protocol_changes enables the branch from genesis on testnet and regtest, * so a hex worked example has never run on the networks a developer tries * first. Mainnet passed the flag day on 2026-08-07. diff --git a/test/dividend-eligibility-claims.test.js b/test/dividend-eligibility-claims.test.js new file mode 100644 index 00000000..1eec7e74 --- /dev/null +++ b/test/dividend-eligibility-claims.test.js @@ -0,0 +1,122 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * DIVIDEND recipient-eligibility claims. + * + * WHY. The user guide promised "Every holder receives their proportional cut", + * which is a promise a distributor acts on with their own money. dividend.js + * pays a strict subset of holders and drops three groups: + * + * 1. Holders who fail the PAYMENT token's allow or block list. The lists read + * are those of DIVIDEND_TICK, not of the share token TICK, and a + * configured but empty list still admits everyone. + * 2. The paying address itself (address == SOURCE). + * 3. Holders whose share floors to zero. The share is computed with + * bcmulfloor at the PAYMENT token's precision, and a recipient is added + * only when share != 0. Dropped holders also never reach the per-recipient + * fee, so the payer is not charged for them. + * + * The allow/block-list exclusion was documented nowhere in this repo until the + * Notes bullet this guard now pins, which is why the guide could not have been + * corrected from the spec page alone. + * + * WHAT IT CHECKS. Both halves. The SOURCE half reads the sibling indexer, so + * the guard goes red if the handler changes and the prose becomes stale in the + * other direction. The PROSE half runs unconditionally, so the guard can never + * come back all-skip. + * + * XCHAIN_DOCS_ROOT overrides the docs root, matching + * settlement-and-delivery-claims.test.js. It exists so the negative control is + * runnable: point it at a checkout of an older commit and the prose assertions + * below go red, which is how they were verified to be capable of failing. + */ +const test = require('node:test'); +const assert = require('node:assert'); +const fs = require('node:fs'); +const path = require('node:path'); +// an entry plus every part it was split into, at whichever spelling the sibling checkout uses +const { readModuleSource } = require('../lib/indexer-source.js'); // an entry plus every part it was split into +const { sibling } = require('./helpers/sibling_checkout.js'); + +const DOC_ROOT = process.env.XCHAIN_DOCS_ROOT || path.join(__dirname, '..'); +const INDEXER = path.resolve(path.join(__dirname, '..'), '../xchain-indexer/src'); + +/* Skips by name on a bare clone. A run that declared the sibling supplied + * (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set, with xchain-indexer + * in .ci-siblings) throws in the helper instead, so a dropped checkout cannot leave the + * source half uncompared while the file still reports green. Both spellings of the entry + * are tried, since the indexer's split convention moves `dividend.js` to `dividend/index.js`. */ +const skipNoIndexer = sibling('xchain-indexer', + [[path.join(INDEXER, 'actions', 'dividend.js'), path.join(INDEXER, 'actions', 'dividend', 'index.js')]]).skip; +const readSrc = (rel) => readModuleSource(path.join(INDEXER, rel)); +const readDoc = (rel) => fs.readFileSync(path.join(DOC_ROOT, rel), 'utf8'); + +const useCases = readDoc('user-guide/use-cases.md'); +const creating = readDoc('user-guide/creating-tokens.md'); +const spec = readDoc('protocol/actions/dividend.md'); + +test('the source facts the DIVIDEND eligibility wording rests on still hold', { skip: skipNoIndexer }, () => { + const dividend = readSrc('actions/dividend.js'); + + assert.match(dividend, /if\(address==data\['SOURCE'\]\)/, + 'dividend.js no longer excludes the paying address from the recipient list, so the ' + + 'guide\'s "the paying address does not pay a dividend to itself" is now wrong'); + assert.match(dividend, /bcmulfloor\(/, + 'dividend.js no longer floors each share with bcmulfloor, so the guide\'s "rounded ' + + 'down to the payment token\'s smallest unit" may no longer describe the handler'); + assert.match(dividend, /if\(share\s*!=\s*0\)/, + 'dividend.js no longer drops zero-share holders, so the guide\'s zero-share exclusion ' + + 'and the spec page\'s per-recipient-fee note are both stale'); + assert.match(dividend, /allowList\.size\s*&&\s*!allowList\.has\(address\)/, + 'dividend.js no longer filters recipients through the payment token\'s allow list'); + assert.match(dividend, /blockList\.size\s*&&\s*blockList\.has\(address\)/, + 'dividend.js no longer filters recipients through the payment token\'s block list'); + assert.match(dividend, /dividendTokenInfo\['ALLOW_LIST'\]/, + 'dividend.js no longer reads ALLOW_LIST off dividendTokenInfo. If the lists now come ' + + 'from the share token instead, the spec Notes bullet naming DIVIDEND_TICK is wrong.'); +}); + +test('the use-cases dividend passage names all three exclusions', () => { + assert.ok(!useCases.includes('Every holder receives their proportional cut'), + 'user-guide/use-cases.md still promises every holder a proportional cut. dividend.js ' + + 'excludes the payer, list-filtered holders, and holders whose share floors to zero.'); + assert.match(useCases, /eligible holder/i, + 'user-guide/use-cases.md no longer scopes the dividend payout to eligible holders'); + assert.match(useCases, /rounded \*\*down\*\*|rounds to zero/i, + 'user-guide/use-cases.md no longer states the round-down / zero-share exclusion'); + assert.match(useCases, /allow list or a block list|allow or block list/i, + 'user-guide/use-cases.md no longer states the payment token\'s allow/block-list ' + + 'exclusion, which is the one a reader cannot find anywhere else in the guide'); + assert.match(useCases, /paying address does not pay a dividend to itself/i, + 'user-guide/use-cases.md no longer states that the payer is excluded'); +}); + +test('the token-owner capability list does not promise dividends to all holders', () => { + const bullet = creating.split('\n').find((l) => l.startsWith('- **Pay dividends**')); + assert.ok(bullet, 'creating-tokens.md no longer carries a "- **Pay dividends**" bullet'); + assert.ok(!/all holders/i.test(bullet), + 'the creating-tokens.md dividend bullet still says dividends reach all holders'); + assert.match(bullet, /eligible holders/i, + 'the creating-tokens.md dividend bullet no longer says eligible holders'); +}); + +test('the DIVIDEND spec page documents the allow/block-list exclusion', () => { + assert.match(spec, /ALLOW_LIST/, + 'protocol/actions/dividend.md never mentions ALLOW_LIST. The payment token\'s lists ' + + 'filter the recipient set in dividend.js and this page is the only normative place ' + + 'a reader can learn that.'); + assert.match(spec, /BLOCK_LIST/, 'protocol/actions/dividend.md never mentions BLOCK_LIST'); + assert.match(spec, /payment token \(`DIVIDEND_TICK`\), not the share token/, + 'protocol/actions/dividend.md no longer says which token\'s lists apply. dividend.js ' + + 'reads them off dividendTokenInfo, the DIVIDEND_TICK record, and a reader who assumes ' + + 'the share token\'s lists will predict the wrong recipient set.'); +}); diff --git a/test/env-var-doc-coverage.test.js b/test/env-var-doc-coverage.test.js index 600c21ef..20a4f64d 100644 --- a/test/env-var-doc-coverage.test.js +++ b/test/env-var-doc-coverage.test.js @@ -55,6 +55,7 @@ const path = require('node:path'); const { execFileSync } = require('node:child_process'); const cov = require('../lib/env-var-doc-coverage.js'); +const { sibling } = require('./helpers/sibling_checkout.js'); const { ENV_READ, extractDefault, scanSource, docLinesFor, defaultDocumented, isSourcePath, @@ -285,7 +286,7 @@ describe('scanSource across line boundaries', () => { assert.equal(found.get('SENTINEL')[0].line, 1); }); - // xchain-node/src/services/EncoderMaintenanceWindow.js:46-47, verbatim. + // xchain-node/src/services/encoder_maintenance_window.js:46-47, verbatim. test('the semicolon-less wrapped read in xchain-node is no longer exempt', () => { const found = scanSource([ 'const SENTINEL_PATH = process.env.XCHAIN_NODE_ENCODER_MAINTENANCE_FILE', @@ -297,7 +298,7 @@ describe('scanSource across line boundaries', () => { ); }); - // xchain-hub/src/StateCheckpointEngine.js:190-191, verbatim: the chain + // xchain-hub/src/anchor/checkpoint_engine.js:190-191, verbatim: the chain // continues through a non-literal `cfg.X` on the wrapped line. test('a wrapped chain inside parseInt reaches the literal past a cfg lookup', () => { const found = scanSource([ @@ -310,7 +311,7 @@ describe('scanSource across line boundaries', () => { assert.equal(site.line, 1); }); - // xchain-hub/src/AttestationBatchPublisher.js:175-176: the `||` itself ends + // xchain-hub/src/attestation/batch_publisher.js:175-176: the `||` itself ends // the line, so the operand is read off the NEXT one. test('a `||` at end of line reaches its operand on the next line', () => { const found = scanSource([ @@ -399,6 +400,145 @@ describe('scanComputedReads (the blind spot the gate cannot see into)', () => { }); }); +// The explorer reads configuration through config.js's `env` view, a live +// Proxy over process.env. Before the scanner knew the view, every read moved +// behind it left the survey, so its doc row could vanish with the gate green. +describe('reads through the explorer config view', () => { + const VIEW_SOURCE = [ + "const a = env.ALPHA || 1;", + "const b = env['BETA'] || 'x';", + "const c = parseInt(this.configInfo.env.GAMMA, 10) || 15000;", + "const d = this.configInfo.env['DELTA'];", + "const e = configInfo.env.EPSILON === '1';", + ].join('\n'); + + test('every view shape is a named read, with its line and default', () => { + const found = scanSource(VIEW_SOURCE, { configView: true }); + assert.deepEqual([...found.keys()], ['ALPHA', 'BETA', 'GAMMA', 'DELTA', 'EPSILON']); + assert.deepEqual([...found.values()].map((s) => s[0].line), [1, 2, 3, 4, 5]); + assert.equal(found.get('ALPHA')[0].default.value, '1'); + assert.equal(found.get('BETA')[0].default.value, 'x'); + assert.equal(found.get('GAMMA')[0].default.value, '15000'); + }); + + test('without the option the view is invisible, which is every component but the explorer', () => { + assert.equal(scanSource(VIEW_SOURCE).size, 0); + assert.deepEqual(cov.scanComputedReads('const t = this.configInfo.env[prefix + "_MS"];'), []); + assert.deepEqual([...cov.CONFIG_VIEW_COMPONENTS], ['explorer']); + }); + + test('process.env is counted once, and other objects named env are not the view', () => { + const found = scanSource([ + "const a = process.env.ALPHA;", + "const b = options.env.BETA;", + "const c = cfg.env['GAMMA'];", + "const d = env.hasOwnProperty('X');", + "env.DELTA = 'set, not read';", + "if (env.EPSILON == null) {}", + ].join('\n'), { configView: true }); + assert.deepEqual([...found.keys()], ['ALPHA', 'EPSILON']); + assert.equal(found.get('ALPHA').length, 1); + }); + + test('a computed read through the view counts; a literal key or a write into an env object does not', () => { + const lines = cov.scanComputedReads([ + "const ttl = parseInt(this.configInfo.env[envPrefix + '_MS'], 10) || 15000;", + "const k = env[key];", + "const lit = env['LITERAL'];", + "env[tokens[i].value.slice(0, eq)] = tokens[i].value.slice(eq + 1);", + "const p = process.env[k];", + "const wrapped = configInfo.env[", + " prefix + '_MAX'", + "];", + ].join('\n'), { configView: true }); + assert.deepEqual(lines, [1, 2, 5, 6]); + }); + + // The feature directories hold the view through a read-time accessor, since + // config.js loads modules that load them back. + const ACCESSOR_SOURCE = [ + "const configEnv = () => require('../config.js').env;", + "this.maxAttempts = Number(configEnv().RETRY_ATTEMPTS) || 4;", + "const url = configEnv()['UPSTREAM_URL'] || '';", + "const direct = require('./config.js').env.DIRECT_KEY || 'on';", + "const bracket = require('../../config.js').env['BRACKET_KEY'];", + "if (configEnv().FLAG == null) {}", + ].join('\n'); + + test('reads through a config accessor or straight off the module are named reads, with defaults', () => { + const found = scanSource(ACCESSOR_SOURCE, { configView: true }); + assert.deepEqual([...found.keys()], ['RETRY_ATTEMPTS', 'UPSTREAM_URL', 'DIRECT_KEY', 'BRACKET_KEY', 'FLAG']); + assert.deepEqual([...found.values()].map((s) => s[0].line), [2, 3, 4, 5, 6]); + assert.equal(found.get('RETRY_ATTEMPTS')[0].default.value, '4'); + assert.equal(found.get('UPSTREAM_URL')[0].default.value, ''); + assert.equal(found.get('DIRECT_KEY')[0].default.value, 'on'); + assert.equal(scanSource(ACCESSOR_SOURCE).size, 0); + }); + + test('an accessor counts only when this file binds it to the config view', () => { + assert.deepEqual(cov.configViewAccessors("const configEnv = () => require('../config.js').env;"), ['configEnv']); + const found = scanSource([ + "const hubEnv = () => require('../hub.js').env;", + "const cfg = () => require('../config.js').env.NOT_AN_ACCESSOR;", + "const a = configEnv().UNBOUND;", + "const b = hubEnv().OTHER_MODULE;", + "const c = cfg().VIA_NON_ACCESSOR;", + "const d = process.env.ONCE || require('../config.js').env.ONCE;", + ].join('\n'), { configView: true }); + assert.deepEqual([...found.keys()], ['NOT_AN_ACCESSOR', 'ONCE']); + assert.equal(found.get('ONCE').length, 2, 'process.env and the view are two reads, each counted once'); + }); + + test('through an accessor, a write or a lower-case member is not a read, and a computed key counts', () => { + const source = [ + "const configEnv = () => require('../config.js').env;", + "configEnv().WRITTEN = 'set';", + "const own = configEnv().hasOwnProperty('X');", + "const k = configEnv()[prefix + '_MS'];", + "const lit = configEnv()['LITERAL'];", + "const m = require('../config.js').env[key];", + ].join('\n'); + assert.deepEqual([...scanSource(source, { configView: true }).keys()], ['LITERAL']); + assert.deepEqual(cov.scanComputedReads(source, { configView: true }), [4, 6]); + assert.deepEqual(cov.scanComputedReads(source), []); + }); + + test('the survey applies the view to the explorer and to no other component', () => { + const GIT_ID = [ + '-c', 'user.name=fixture', '-c', 'user.email=fixture@example.invalid', + '-c', 'commit.gpgsign=false', '-c', 'core.hooksPath=/dev/null', + ]; + const git = (dir, ...args) => + execFileSync('git', ['-C', dir, ...GIT_ID, ...args], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }); + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'envvar-view-')); + const docRoot = path.join(root, 'xchain-documentation'); + for (const component of ['explorer', 'hub']) { + const repo = path.join(root, `xchain-${component}`); + fs.mkdirSync(path.join(repo, 'src'), { recursive: true }); + fs.writeFileSync(path.join(repo, 'package.json'), JSON.stringify({ name: `xchain-${component}` })); + fs.writeFileSync(path.join(repo, 'src', 'reader.js'), + "const a = this.configInfo.env.VIEW_KEY;\nconst b = this.configInfo.env[prefix + '_MS'];\n"); + git(repo, 'init', '-q', '-b', 'main'); + git(repo, 'add', '-A'); + git(repo, 'commit', '-q', '-m', component); + fs.mkdirSync(path.join(docRoot, 'components', component), { recursive: true }); + fs.writeFileSync(path.join(docRoot, 'components', component, 'configuration.md'), 'nothing documented\n'); + } + const survey = cov.buildSurvey({ + platformRoot: root, + docRoot, + serviceReader: cov.committedTreeReader('HEAD'), + docReader: cov.workingTreeReader(), + components: ['explorer', 'hub'], + }); + assert.deepEqual([...survey.get('explorer').vars.keys()], ['VIEW_KEY']); + assert.equal(survey.get('explorer').computed.length, 1); + assert.match(cov.checkUndocumented('explorer', survey.get('explorer'))[0], /^VIEW_KEY \(read at src\/reader\.js:1\)/); + assert.equal(survey.get('hub').vars.size, 0); + assert.equal(survey.get('hub').computed.length, 0); + }); +}); + describe('checkComputedReads (the blind-spot ratchet)', () => { const entryWith = (n) => ({ vars: new Map(), docLines: [], docProse: [], sourceFiles: 1, @@ -774,6 +914,22 @@ describe('doc matching', () => { assert.equal(defaultDocumented(['| `SYNC_MODE` | proxied by a webserver |'], 'server'), false); }); + // The shipped row, not a synthetic one. An allowed-values description names + // the rival value by construction, so a mention of `server` anywhere on the + // row must not credit a Default cell that has drifted to `client`. + test('an allowed-values description does not credit a drifted string default', () => { + const correct = '| `SYNC_MODE` | Yes | `server` | Operating mode: `server` or `client` |'; + const drifted = '| `SYNC_MODE` | Yes | `client` | Operating mode: `server` or `client` |'; + assert.equal(defaultDocumented([correct], 'server'), true); + assert.equal(defaultDocumented([drifted], 'server'), false); + }); + + test('a string default mentioned only mid-sentence is not an assertion', () => { + const row = '| `NETWORK` | the REPL falls back to `bitcoin-regtest` when it is unset | REPL |'; + assert.equal(defaultDocumented([row], 'bitcoin-regtest'), false); + assert.equal(defaultDocumented(['`NETWORK` defaults to `bitcoin-regtest` when unset.'], 'bitcoin-regtest'), true); + }); + test('a dotted string default matches literally, not as a wildcard', () => { assert.equal(defaultDocumented(['| `DB_HOST` | host | `127.0.0.1` |'], '127.0.0.1'), true); assert.equal(defaultDocumented(['| `DB_HOST` | host | `127a0b0c1` |'], '127.0.0.1'), false); @@ -1014,6 +1170,10 @@ describe('checkStaleKnownGaps (the waiver ratchet)', () => { * The gate itself * ------------------------------------------------------------------ */ +// Every gated component is a declared sibling (.ci-siblings), so under +// XCHAIN_REQUIRE_SIBLINGS=1 an absent or hollow one throws here by name instead +// of dropping out of the survey and shrinking the floor while the gate reads green. +for (const c of cov.COMPONENTS) sibling(`xchain-${c}`); const present = cov.presentComponents(PLATFORM_ROOT); // Reading a sibling at HEAD needs its object database. A service checked out @@ -1039,12 +1199,22 @@ describe('environment-variable documentation coverage', { skip: siblingsMissing assert.deepEqual(unreadable, [], `checked out but not a git repo, so not gated: ${unreadable.join(', ')}`); }); - test('the survey actually found something to check', () => { - // A refactor that breaks the scanner must not read as a clean bill of - // health. The services read hundreds of variables between them. - const total = cov.totalReads(survey); - assert.ok(total > 300, `only ${total} env reads found across ${readable.length} components; the scanner is probably broken`); - }); + // The floor is a FLEET figure: the services read hundreds of variables + // between them, so a full platform checkout that surveys under 300 has a + // broken scanner, not a quiet fleet. A partial checkout (GitHub CI checks + // out only the siblings a suite names, one today) cannot be held to it: + // the indexer alone reads about a hundred, and the per-component checks + // below plus the empty-scan check still judge every sibling present. The + // floor itself is enforced where the whole fleet is, in the platform + // checkout, the same way the schema-table coverage invariant is. + const fleetMissing = cov.COMPONENTS.filter((c) => !readable.includes(c)); + test('the survey actually found something to check', + { skip: fleetMissing.length ? `fleet floor needs every sibling; absent: ${fleetMissing.join(', ')}` : false }, () => { + // A refactor that breaks the scanner must not read as a clean bill of + // health. + const total = cov.totalReads(survey); + assert.ok(total > 300, `only ${total} env reads found across ${readable.length} components; the scanner is probably broken`); + }); test('every surveyed component contributed source files', () => { // The fleet floor above is not enough on its own: the hub is about a diff --git a/test/error-code-registry-coverage.test.js b/test/error-code-registry-coverage.test.js new file mode 100644 index 00000000..8f6e5d4e --- /dev/null +++ b/test/error-code-registry-coverage.test.js @@ -0,0 +1,142 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * Explorer error-code registry coverage gate. + * + * WHY. protocol/error-codes.md opens by calling itself the machine-readable + * registry for the platform's HTTP APIs and promises the codes are append-only, + * so a client may branch on them. Nothing enforced that promise from the emit + * side: the explorer's batch address routes shipped INVALID_ADDRESSES, + * TOO_MANY_ADDRESSES, INVALID_ADDRESS and the per-address READ_FAILED fallback + * with no registry row, and the explorer's published OpenAPI enumerates no codes + * at all, so the registry was the only contract surface and it was incomplete. + * + * This test re-derives the emitted set from the explorer source on every run and + * fails naming any code that has no registry row. + * + * Collection is by `code:` line rather than by a bare literal scan, so the + * ternary fallback form (`code: cond ? x : 'READ_FAILED'`) is caught alongside + * the plain `code: 'X'` form. Only the three REST-side sources are read: + * src/ws/ carries the WebSocket channel codes, which are a separate surface + * documented in components/explorer/websocket.md and explicitly excluded by the + * registry page's own closing note. + * + * xchain-explorer is a sibling repo in the monorepo checkout, not a dependency + * of xchain-documentation. When the REPO is absent (docs repo cloned on its + * own) the source-derived assertion skips. When the repo is present but one of + * the emit sites below has moved, the gate FAILS naming the missing path: + * keying the skip on the files rather than the repo is how a move would + * silently unpin the whole registry check. + * + ********************************************************************/ + +const assert = require('node:assert/strict'); +const { test } = require('node:test'); +const fs = require('node:fs'); +const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); + +const ROOT = path.resolve(__dirname, '..'); +const EXPLORER = path.resolve(ROOT, '../xchain-explorer'); +const EXPLORER_SRC = path.join(EXPLORER, 'src'); +const REGISTRY_PAGE = path.join(ROOT, 'protocol/error-codes.md'); + +// REST-side emit sites. src/ws/ is deliberately absent (separate surface). +const SOURCES = ['XChainExplorer.js', 'api.js', 'http/concurrency_gate.js'] + .map((file) => path.join(EXPLORER_SRC, file)); + +// An entry file whose long methods were split keeps its path and gains a +// sibling directory of parts, so an emit site can sit in either. The explorer +// names those directories after the entry, except for its two top-level entries +// (XChainExplorer.js to src/explorer/, api.js to src/http/api_boot/), which is +// the same alias map the explorer's own source-text helper carries. +const PART_DIRS = { + 'XChainExplorer.js': path.join(EXPLORER_SRC, 'explorer'), + 'api.js': path.join(EXPLORER_SRC, 'http/api_boot'), + 'http/concurrency_gate.js': path.join(EXPLORER_SRC, 'http/concurrency_gate'), +}; + +// Every .js file under a directory, sorted, so the emit sites are read in a +// stable order however deep the split goes. +function jsFilesUnder(dir) { + if (!fs.existsSync(dir) || !fs.statSync(dir).isDirectory()) return []; + const out = []; + for (const name of fs.readdirSync(dir).sort()) { + const abs = path.join(dir, name); + if (fs.statSync(abs).isDirectory()) out.push(...jsFilesUnder(abs)); + else if (name.endsWith('.js')) out.push(abs); + } + return out; +} + +// Codes documented elsewhere on purpose. Each entry names where it lives, so a +// future addition is a documentation decision rather than a way to quiet the +// test. Loosening the collection regex is not the way to make this pass. +const DOCUMENTED_ELSEWHERE = Object.create(null); + +// Repo presence only: a pinned file gone from a PRESENT explorer fails inside the test by +// name (readSource below). Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1. +const noExplorer = sibling('xchain-explorer').skip; + +// The observability logger's first argument is an EVENT name, not a response +// code; a log line also carrying the Node errno as `code: err.code` would +// otherwise read that event name as a REST code the registry owes a row. +const LOGGER_EVENT = /\blog\.(?:trace|debug|info|warn|error|fatal)\(\s*'[A-Z][A-Z0-9_]*'/g; + +// Every SCREAMING_SNAKE string literal on a line that also carries `code:`, +// minus a logger event name leading that line's log call. +function emittedCodes() { + const found = new Map(); + const files = SOURCES.slice(); + for (const [entry, dir] of Object.entries(PART_DIRS)) { + if (!SOURCES.includes(path.join(EXPLORER_SRC, entry))) continue; + files.push(...jsFilesUnder(dir)); + } + for (const file of files) { + // With the repo present, a missing emit site means the code moved and + // this list has to follow it, never that the check may skip. + assert.ok(fs.existsSync(file), + path.relative(EXPLORER, file) + ' is gone from xchain-explorer; repoint SOURCES at the file the emit site moved to'); + const lines = fs.readFileSync(file, 'utf8').split('\n'); + lines.forEach((line, i) => { + if (!line.includes('code:')) return; + const emitted = line.replace(LOGGER_EVENT, ''); + for (const quoted of emitted.match(/'([A-Z][A-Z0-9_]{2,})'/g) || []) { + const code = quoted.slice(1, -1); + if (!found.has(code)) + found.set(code, `${path.basename(file)}:${i + 1}`); + } + }); + } + return found; +} + +test('every explorer REST error code has a registry row', { skip: noExplorer }, () => { + const registry = fs.readFileSync(REGISTRY_PAGE, 'utf8'); + const emitted = emittedCodes(); + + assert.ok(emitted.size > 20, + `collected only ${emitted.size} codes from the explorer source; the emit ` + + 'shape changed, re-point the collection in this test'); + + const missing = []; + for (const [code, where] of emitted) { + if (code in DOCUMENTED_ELSEWHERE) continue; + if (!registry.includes('| `' + code + '` |')) + missing.push(`${code} (emitted at ${where})`); + } + + assert.deepEqual(missing, [], + 'protocol/error-codes.md calls itself the complete append-only registry, ' + + 'so every code the explorer emits needs a row there. Missing:\n ' + + missing.join('\n ')); +}); diff --git a/test/explorer-endpoint-counts.test.js b/test/explorer-endpoint-counts.test.js index 17b64ef8..4767bf2c 100644 --- a/test/explorer-endpoint-counts.test.js +++ b/test/explorer-endpoint-counts.test.js @@ -40,8 +40,11 @@ * the enclosing method also touches `this.app` and cannot run standalone. * * xchain-explorer is a sibling repo in the monorepo checkout, not a dependency - * of xchain-documentation. When it is absent (docs repo cloned on its own) the - * source-derived assertions skip and only the doc's internal arithmetic runs. + * of xchain-documentation. When the REPO is absent (docs repo cloned on its + * own) the source-derived assertions skip and only the doc's internal arithmetic + * runs. When the repo is present but a file this gate reads has moved, the gate + * FAILS naming the missing path, because a skip or an empty stand-in keyed on + * the file is how a move would silently unpin it. * ********************************************************************/ @@ -49,19 +52,83 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const COMPONENT_MAP = path.resolve(__dirname, '../architecture/component-map.md'); -const EXPLORER_SOURCE = path.resolve(__dirname, '../../xchain-explorer/src/XChainExplorer.js'); +const EXPLORER = path.resolve(__dirname, '../../xchain-explorer'); +const EXPLORER_SOURCE = path.join(EXPLORER, 'src/XChainExplorer.js'); +// The three route-table modules setupUrls() now builds its object from, in the +// order it declares them. The explorer's own identity tool joins the same three +// in the same order (bin/explorer-identity.js routeTableSource), which is what +// keeps this gate and the explorer's route digest reading one surface. +const ROUTE_TABLES = ['static_and_html', 'api_methods', 'explorer_feeds'] + .map((name) => path.join(EXPLORER, 'src/explorer/routes', name + '.js')); + +// The dispatch table's declaration text, wherever it lives: the entry file while +// setupUrls() held the literal, the joined route modules once they carry it. +const ROUTES_INDEX = path.join(EXPLORER, 'src/explorer/routes/index.js'); + +// The dispatch table itself. The explorer builds it in src/explorer/routes/, +// which exports routeTables() returning a fresh object with the same keys in +// the same order setupUrls() declared, so this gate loads it rather than +// re-parsing a literal out of source text. The text path stays for a checkout +// from before that move. +function dispatchTable() { + if (fs.existsSync(ROUTES_INDEX)) { + const { routeTables } = require(requireExplorerFile(ROUTES_INDEX)); + assert.equal(typeof routeTables, 'function', + 'src/explorer/routes/index.js no longer exports routeTables(); this gate needs updating'); + return routeTables(); + } + return readDispatchTable(fs.readFileSync(requireExplorerFile(EXPLORER_SOURCE), 'utf8')); +} + +// The class entry's text followed by every .js module under src/explorer/, in +// path order, as one string for the route-registration scans. +function explorerClassSource() { + const files = [requireExplorerFile(EXPLORER_SOURCE)]; + if (fs.existsSync(EXPLORER_STAGES)) { + const walk = (dir) => { + for (const name of fs.readdirSync(dir).sort()) { + const full = path.join(dir, name); + if (fs.statSync(full).isDirectory()) walk(full); + else if (name.endsWith('.js')) files.push(full); + } + }; + walk(EXPLORER_STAGES); + } + return files.map((f) => fs.readFileSync(f, 'utf8')).join('\n'); +} +const STATIC_MOUNTS = path.join(EXPLORER, 'src/http/static_mounts.js'); +// The hand-registered routes live in the class entry or in the stage modules +// it delegates to under src/explorer/ (mount.js today), so this gate reads the +// entry plus every module in that tree; a text read pinned to one file would +// count zero the moment the registrations moved and never fail. +const EXPLORER_STAGES = path.join(EXPLORER, 'src/explorer'); const doc = fs.readFileSync(COMPONENT_MAP, 'utf8'); -const haveExplorer = fs.existsSync(EXPLORER_SOURCE); +// Repo presence only: a pinned file gone from a PRESENT explorer fails inside the test by +// name (readSource below). Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1. +const noExplorer = sibling('xchain-explorer').skip; + +// Reads an explorer source file this gate is pinned to, failing with the path +// named when it is gone: with the repo present, a missing file means the code +// moved and this pin has to follow it, never that the assertion may skip. +function requireExplorerFile(file) { + assert.ok(fs.existsSync(file), + path.relative(EXPLORER, file) + ' is gone from xchain-explorer; repoint this gate at the file the behaviour moved to'); + return file; +} // Pull the `let urls = { ... }` literal out of setupUrls() by brace matching and // evaluate it. Returns { html, api, explorer, static } exactly as the running // explorer sees it, duplicate keys already collapsed. function readDispatchTable(source) { - const anchor = source.indexOf('let urls = {'); - assert.notEqual(anchor, -1, 'setupUrls() no longer declares `let urls = {`; this gate needs updating'); + // `let urls = {` while the literal sat in setupUrls(), `urls : {` once the + // tables became modules that declare the same object under the same name. + let anchor = source.indexOf('urls : {'); + if (anchor === -1) anchor = source.indexOf('let urls = {'); + assert.notEqual(anchor, -1, 'neither setupUrls() nor the route-table modules declare the urls object; this gate needs updating'); const open = source.indexOf('{', anchor); let depth = 0; let close = -1; @@ -74,11 +141,10 @@ function readDispatchTable(source) { } assert.notEqual(close, -1, 'the urls literal in setupUrls() is unbalanced'); // The literal's `static` bucket reads the explorer's mount list module by - // name, so the evaluation is given that one binding (the real module when - // the sibling checkout carries it, an empty list otherwise); the counts this - // gate checks live in the `api` and `explorer` buckets, which are inline. - const mountsPath = path.join(path.dirname(EXPLORER_SOURCE), 'staticMounts.js'); - const staticMounts = fs.existsSync(mountsPath) ? require(mountsPath) : { STATIC_DIRECTORIES: [] }; + // its local binding name `staticMounts`, so the evaluation is given that one + // binding, loaded from the real module; the counts this gate checks live in + // the `api` and `explorer` buckets, which are inline. + const staticMounts = require(requireExplorerFile(STATIC_MOUNTS)); return new Function('staticMounts', 'return (' + source.slice(open, close + 1) + ')')(staticMounts); } @@ -126,8 +192,8 @@ describe('explorer REST endpoint counts in component-map.md', () => { 'the breakdown ' + api + ' + ' + expl + ' + ' + hand + ' does not sum to the stated total ' + total); }); - test('the dispatch-table counts match xchain-explorer source', { skip: !haveExplorer && 'xchain-explorer not present in this checkout' }, () => { - const urls = readDispatchTable(fs.readFileSync(EXPLORER_SOURCE, 'utf8')); + test('the dispatch-table counts match xchain-explorer source', { skip: noExplorer }, () => { + const urls = dispatchTable(); const api = Object.keys(urls.api); const expl = Object.keys(urls.explorer); @@ -151,17 +217,17 @@ describe('explorer REST endpoint counts in component-map.md', () => { ' HTML page routes, not the documented ' + docHtml); }); - test('the hand-registered /api route count matches xchain-explorer source', { skip: !haveExplorer && 'xchain-explorer not present in this checkout' }, () => { - const routes = readHandRegisteredApiRoutes(fs.readFileSync(EXPLORER_SOURCE, 'utf8')); + test('the hand-registered /api route count matches xchain-explorer source', { skip: noExplorer }, () => { + const routes = readHandRegisteredApiRoutes(explorerClassSource()); const docHand = documentedFigure('hand-registered'); assert.equal(routes.length, docHand, 'the explorer hand-registers ' + routes.length + ' /api routes, not the documented ' + docHand + ':\n ' + routes.join('\n ')); }); - test('the surfaces the doc calls out by name are really registered', { skip: !haveExplorer && 'xchain-explorer not present in this checkout' }, () => { - const source = fs.readFileSync(EXPLORER_SOURCE, 'utf8'); - const urls = readDispatchTable(source); + test('the surfaces the doc calls out by name are really registered', { skip: noExplorer }, () => { + const source = explorerClassSource(); + const urls = dispatchTable(); const hand = readHandRegisteredApiRoutes(source).join('\n'); // The three surfaces added after the stale 2026-06-20 count, named in the diff --git a/test/explorer-status-contract.test.js b/test/explorer-status-contract.test.js index 91c30e22..b5b6a828 100644 --- a/test/explorer-status-contract.test.js +++ b/test/explorer-status-contract.test.js @@ -42,12 +42,14 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const API_DOC = path.resolve(__dirname, '../components/explorer/api.md'); const SCHEMA = path.resolve(__dirname, '../../xchain-explorer/src/content/json/xchain-platform-api.json'); const doc = fs.readFileSync(API_DOC, 'utf8'); -const haveSchema = fs.existsSync(SCHEMA); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the schema is unreadable. +const noSchema = sibling('xchain-explorer', [SCHEMA]).skip; // The "Get Status" section only: a field name mentioned under some other // endpoint must not count as documented here. @@ -71,7 +73,7 @@ function documentedFields(section) { describe('explorer /status contract in components/explorer/api.md', () => { const section = statusSection(doc); - test('documents every field the published ExplorerStatus schema declares', { skip: !haveSchema && 'xchain-explorer not present in this checkout' }, () => { + test('documents every field the published ExplorerStatus schema declares', { skip: noSchema }, () => { const spec = JSON.parse(fs.readFileSync(SCHEMA, 'utf8')); const declared = Object.keys(spec.components.schemas.ExplorerStatus.properties); const rows = documentedFields(section); diff --git a/test/fee-and-limit-claims.test.js b/test/fee-and-limit-claims.test.js index c4f1b4be..2a56c04a 100644 --- a/test/fee-and-limit-claims.test.js +++ b/test/fee-and-limit-claims.test.js @@ -42,6 +42,7 @@ const assert = require('node:assert/strict'); const test = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const ROOT = path.resolve(__dirname, '..'); const INDEXER = path.resolve(ROOT, '../xchain-indexer/src'); @@ -49,8 +50,13 @@ const INDEXER = path.resolve(ROOT, '../xchain-indexer/src'); const CONFIG_JS = path.join(INDEXER, 'config.js'); const COIN_JS = ['BTC', 'LTC', 'DOGE'].map((c) => [c, path.join(INDEXER, 'coins', `${c}.js`)]); -const haveConfig = fs.existsSync(CONFIG_JS); -const haveCoins = COIN_JS.every(([, p]) => fs.existsSync(p)); +/* Two skips cover the source half for a bare clone, one per source the claims read. A run + * that declared the sibling supplied (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the + * venue set, with xchain-indexer in .ci-siblings) throws in the helper instead, naming every + * path not readable, so a dropped checkout cannot leave a fee or limit claim uncompared while + * the file still reports green. */ +const noConfig = sibling('xchain-indexer', [CONFIG_JS]).skip; +const noCoins = sibling('xchain-indexer', COIN_JS.map(([, p]) => p)).skip; const readDoc = (rel) => fs.readFileSync(path.join(ROOT, rel), 'utf8'); const normalize = (s) => s.replace(/[\s`]/g, ''); @@ -71,7 +77,7 @@ function scheduleValue(src, name, where) { } test('the ownership-escrow premium the fee docs quote matches the gas schedule', - { skip: !haveCoins && 'sibling xchain-indexer not present in this checkout' }, () => { + { skip: noCoins }, () => { const escrow = new Map(); const price = new Map(); for (const [coin, file] of COIN_JS) { @@ -118,7 +124,7 @@ const TICK_SET_PAGES = [ ]; test('every page restating the ticker character set matches TICK_CHARACTERS', - { skip: !haveConfig && 'sibling xchain-indexer not present in this checkout' }, () => { + { skip: noConfig }, () => { const chars = configValue(fs.readFileSync(CONFIG_JS, 'utf8'), 'TICK_CHARACTERS'); const tail = chars.slice(chars.indexOf('0123456789') + '0123456789'.length); assert.ok(tail.length > 4, @@ -134,7 +140,7 @@ test('every page restating the ticker character set matches TICK_CHARACTERS', }); test('the betting guide states the enforced refund-window bounds and per-market bet cap', - { skip: !haveConfig && 'sibling xchain-indexer not present in this checkout' }, () => { + { skip: noConfig }, () => { const src = fs.readFileSync(CONFIG_JS, 'utf8'); const min = Number(configValue(src, 'MIN_BET_REFUND_WINDOW')); const max = Number(configValue(src, 'MAX_BET_REFUND_WINDOW')); @@ -161,7 +167,7 @@ test('the betting guide states the enforced refund-window bounds and per-market }); test('the unified free-listing window every fee page quotes matches the coin configs', - { skip: !haveCoins && 'sibling xchain-indexer not present in this checkout' }, () => { + { skip: noCoins }, () => { const free = new Map(); for (const [coin, file] of COIN_JS) { const src = fs.readFileSync(file, 'utf8'); diff --git a/test/flag-day-literals.test.js b/test/flag-day-literals.test.js index 31a77f58..5ee6b671 100644 --- a/test/flag-day-literals.test.js +++ b/test/flag-day-literals.test.js @@ -56,10 +56,12 @@ const os = require('node:os'); const path = require('node:path'); const gen = require('../bin/generate-flag-days.js'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const GENERATED = path.join(DOC_ROOT, 'protocol', 'flag-days.md'); -const HAS_INDEXER = fs.existsSync(gen.REGISTRY); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the registry is unreadable. +const noIndexer = sibling('xchain-indexer', [gen.REGISTRY]).skip; /** * Flag-day dates the platform has retired. Pinned literally, which is the @@ -104,7 +106,7 @@ function proseLines() { return out; } -test('the generated flag-day page matches the indexer registry', { skip: HAS_INDEXER ? false : 'no sibling xchain-indexer checkout' }, () => { +test('the generated flag-day page matches the indexer registry', { skip: noIndexer }, () => { assert.ok(fs.existsSync(GENERATED), 'protocol/flag-days.md is missing. Run node bin/generate-flag-days.js'); assert.strictEqual( fs.readFileSync(GENERATED, 'utf8'), @@ -115,7 +117,7 @@ test('the generated flag-day page matches the indexer registry', { skip: HAS_IND ); }); -test('all three gate-collection paths still find their gates', { skip: HAS_INDEXER ? false : 'no sibling xchain-indexer checkout' }, () => { +test('all three gate-collection paths still find their gates', { skip: noIndexer }, () => { // collectGates reads the registry with two independent regexes and then // scans the sibling `*_activation.js` modules, and the check above cannot // tell you when one of the three stops matching: it compares the COMMITTED @@ -193,11 +195,63 @@ test('an addChange call the parse cannot read is refused, not skipped', () => { ); }); +// The exemplar moved when the const grammar was widened to the slot parser's. +// `1_786_060_800` is a readable decimal literal now, so the unreadable case has +// to be a shape the literal grammar genuinely refuses. The intent this test +// pins, that a declaration the generator cannot read is REFUSED rather than +// dropped, is unchanged. test('a drifted _MAINNET_TIME constant is refused, not skipped', () => { - const dir = fixtureRegistry('const FOO_MAINNET_TIME = 1_786_060_800;\n'); + const dir = fixtureRegistry('const FOO_MAINNET_TIME = 0x6A7B8C9D;\n'); assert.throws(() => gen.collectGates(dir), /FOO_MAINNET_TIME/); }); +/* ------------------------------------------------------------------ + * The CONSTANT DECLARATION, read by the same literal grammar as a slot + * ------------------------------------------------------------------ + * + * The const scanners demanded bare digits while the slot parser accepted + * separators, so `const FOO_TESTNET_TIME = 1_789_257_600;` consumed by an + * addChange call resolved to null and its gate dropped out of the testnet + * table in silence, extending the page's "genesis-active off mainnet" claim + * over a gate that arms on a date of its own. Nothing threw: the completeness + * guard scans MAINNET declarations only, and three of the four collectors + * never reach it. + */ + +test('a separator-bearing testnet constant consumed by a call still publishes its gate', () => { + const dir = fixtureRegistry( + 'const FOO_TESTNET_TIME = 1_787_961_600;\n' + + "this.addChange('FOO', '1.0.0', 9999999999, FOO_TESTNET_TIME, 0, 0, 0, 0);\n", + ); + assert.deepStrictEqual( + gen.collectTestnetArms(dir).map((g) => [g.gate, g.time]), + [['FOO', 1787961600]], + ); +}); + +test('a separator-bearing testnet constant no call consumes publishes under its prefix', () => { + const dir = fixtureRegistry('const FOO_TESTNET_TIME = 1_787_961_600;\n'); + assert.deepStrictEqual( + gen.collectTestnetArms(dir).map((g) => [g.gate, g.time]), + [['FOO', 1787961600]], + ); +}); + +test('a separator-bearing mainnet constant reaches collectGates', () => { + const dir = fixtureRegistry('const FOO_MAINNET_TIME = 1_786_060_800;\n'); + assert.deepStrictEqual( + gen.collectGates(dir).map((g) => [g.gate, g.time]), + [['FOO', 1786060800]], + ); +}); + +test('an unreadable _TESTNET_TIME declaration is loud on every arm', () => { + const dir = fixtureRegistry('const FOO_TESTNET_TIME = 1787961600 + 86400;\n'); + for (const arm of ['collectGates', 'collectTestnetArms', 'collectTestnetUnarmed', 'collectMainnetUnarmed']) { + assert.throws(() => gen[arm](dir), /FOO_TESTNET_TIME/, `${arm} dropped the declaration in silence`); + } +}); + /* ------------------------------------------------------------------ * The call's TIME ARGUMENT, read whole * ------------------------------------------------------------------ @@ -510,11 +564,11 @@ test('the legitimately quiet sibling shapes do not throw', () => { assert.deepStrictEqual(gen.collectGates(dir).map((g) => g.gate), ['REAL']); }); -test('the real registry parses clean, so the check is not merely strict', { skip: HAS_INDEXER ? false : 'no sibling xchain-indexer checkout' }, () => { +test('the real registry parses clean, so the check is not merely strict', { skip: noIndexer }, () => { assert.ok(gen.collectGates().length > 0); }); -test('the generated page is the only place a live flag-day value appears', { skip: HAS_INDEXER ? false : 'no sibling xchain-indexer checkout' }, () => { +test('the generated page is the only place a live flag-day value appears', { skip: noIndexer }, () => { const gates = gen.collectGates(); const liveDates = new Set(gates.map((g) => gen.utcDate(g.time))); const liveStamps = new Set(gates.map((g) => String(g.time))); @@ -554,7 +608,7 @@ test('no retired flag-day value survives anywhere in the prose', () => { + bad.join('\n')); }); -test('every retired value is genuinely retired in the current registry', { skip: HAS_INDEXER ? false : 'no sibling xchain-indexer checkout' }, () => { +test('every retired value is genuinely retired in the current registry', { skip: noIndexer }, () => { const live = new Set(gen.collectGates().map((g) => String(g.time))); for (const stamp of RETIRED_TIMESTAMPS) { assert.ok(!live.has(stamp), diff --git a/test/helpers/sibling_checkout.js b/test/helpers/sibling_checkout.js new file mode 100644 index 00000000..cd661865 --- /dev/null +++ b/test/helpers/sibling_checkout.js @@ -0,0 +1,110 @@ +/********************************************************************* + * + * Copyright © 2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * The one place a documentation suite resolves a sibling xchain-* checkout. + * + * WHY. Every cross-repo guard in this suite reads a sibling repo beside this + * one and skips when it is not there, which is right for a bare clone and + * wrong for a venue that declared the sibling supplied. Before this helper + * each suite carried its own existsSync and its own skip string, and only the + * six claims suites honoured XCHAIN_REQUIRE_SIBLINGS: measured 2026-09-14, + * with the switch set and xchain-vm removed from the checkout, npm test still + * exited 0 with nine fresh skips nobody saw. A gate that reads green having + * exercised less than yesterday is the failure this file exists to remove. + * + * WHAT. sibling(repo, wants) answers "is this sibling here, with the files I + * read?" once, the same way for every suite. With XCHAIN_REQUIRE_SIBLINGS=1 + * (bin/ci-all.sh and the venue set it, with the roster in .ci-siblings) an + * absent, hollow or incomplete sibling THROWS, naming the repo and every path + * tried, so the file fails instead of skipping. Without the switch it returns + * a skip reason the suite passes straight to node:test, so a bare clone still + * skips by name and never silently. + * + * Hollow: a directory with no package.json is not a checkout (an empty mount + * point, a half-finished clone, a stale symlink target). The decoder's gate + * learned this the hard way (a bare `[ -d ]` test let every guard behind it + * skip), so hollow reads as absent here. + */ +'use strict'; + +const fs = require('node:fs'); +const path = require('node:path'); + +const DOC_ROOT = path.resolve(__dirname, '..', '..'); +const PLATFORM_ROOT = path.resolve(DOC_ROOT, '..'); +const REQUIRED = process.env.XCHAIN_REQUIRE_SIBLINGS === '1'; + +/** Absolute path of a sibling repo beside this checkout. */ +function siblingRoot(repo, platformRoot = PLATFORM_ROOT) { + return path.join(platformRoot, repo); +} + +/** + * What is on disk at the sibling's expected location. + * + * @param {string} repo e.g. 'xchain-indexer' + * @param {string} [platformRoot] directory holding the xchain-* repos + * @returns {{root: string, exists: boolean, hollow: boolean, real: boolean}} + */ +function checkoutState(repo, platformRoot = PLATFORM_ROOT) { + const root = siblingRoot(repo, platformRoot); + let exists = false; + try { exists = fs.statSync(root).isDirectory(); } catch { exists = false; } + const hollow = exists && !fs.existsSync(path.join(root, 'package.json')); + return { root, exists, hollow, real: exists && !hollow }; +} + +/** + * Resolve a sibling and the paths a suite reads from it. + * + * `wants` lists what the caller is about to read: each entry is a path + * (absolute, or relative to the sibling root) or an array of alternative + * spellings of which one must exist (a module that moved and left a re-export, + * a file the layout pass renamed). Every alternative tried is named when none + * is found, so a failure says exactly where the suite looked. + * + * @param {string} repo + * @param {Array} [wants] + * @param {{required?: boolean, platformRoot?: string}} [opts] test seams; + * production callers leave both to the environment + * @returns {{root: string, have: boolean, skip: false|string, missing: string[]}} + * @throws when the switch is set and the sibling is absent, hollow or incomplete + */ +function sibling(repo, wants = [], opts = {}) { + const required = opts.required === undefined ? REQUIRED : opts.required; + const state = checkoutState(repo, opts.platformRoot); + const missing = []; + let reason = null; + if (!state.exists) { + reason = `sibling ${repo} is not checked out at ${state.root}`; + missing.push(state.root); + } else if (state.hollow) { + reason = `sibling ${repo} at ${state.root} is a hollow directory (no package.json)`; + missing.push(path.join(state.root, 'package.json')); + } + for (const want of wants) { + const alternatives = (Array.isArray(want) ? want : [want]) + .map((p) => (path.isAbsolute(p) ? p : path.join(state.root, p))); + if (!alternatives.some((p) => fs.existsSync(p))) missing.push(alternatives.join(' or ')); + } + if (reason === null && missing.length) { + reason = `sibling ${repo} is missing ${missing.join(', ')}`; + } + const have = reason === null; + if (!have && required) { + throw new Error(`XCHAIN_REQUIRE_SIBLINGS=1 but ${reason}. Check ${repo} out beside this ` + + 'repo rather than letting this suite skip.'); + } + return { root: state.root, have, skip: have ? false : reason, missing }; +} + +module.exports = { DOC_ROOT, PLATFORM_ROOT, REQUIRED, siblingRoot, checkoutState, sibling }; diff --git a/test/internal-link-integrity.test.js b/test/internal-link-integrity.test.js index bb59ca59..aa1482ea 100644 --- a/test/internal-link-integrity.test.js +++ b/test/internal-link-integrity.test.js @@ -44,9 +44,12 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const SITE_BUILD = path.resolve(DOC_ROOT, '../xchain-websites/docs.xchain.io/build/docs.build.js'); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the site build is unreadable. +const noSite = sibling('xchain-websites', [SITE_BUILD]).skip; // Must match the site's markdown-it-anchor slugify. Punctuation is removed, // not replaced; runs of whitespace collapse to one hyphen. @@ -275,7 +278,7 @@ describe('internal link integrity', () => { // Without this, the copy above could drift from the renderer and quietly // start validating against a rule the site does not use. - test('the slug rule still matches the docs site', { skip: !fs.existsSync(SITE_BUILD) && 'xchain-websites not present in this checkout' }, () => { + test('the slug rule still matches the docs site', { skip: noSite }, () => { const source = fs.readFileSync(SITE_BUILD, 'utf8'); const m = /slugify:\s*\(s\)\s*=>\s*([^\n]+?)\s*\}\)/.exec(source); assert.ok(m, 'could not find the slugify option in docs.build.js; this gate needs updating'); diff --git a/test/protocol-constant-claims.test.js b/test/protocol-constant-claims.test.js index efa8cbe5..8df07a07 100644 --- a/test/protocol-constant-claims.test.js +++ b/test/protocol-constant-claims.test.js @@ -53,6 +53,9 @@ const test = require('node:test'); const assert = require('node:assert'); const fs = require('node:fs'); const path = require('node:path'); +// an entry plus every part it was split into, at whichever spelling the sibling checkout uses +const { moduleEntry, readModuleSource } = require('../lib/indexer-source.js'); +const { sibling } = require('./helpers/sibling_checkout.js'); const ROOT = path.join(__dirname, '..'); const CONSTANTS = require(path.join(ROOT, 'protocol', 'constants.js')); @@ -246,7 +249,7 @@ test('prose command counts for the BATCH cap match the canonical value', () => { * protocol/constants.js is this repo's canonical copy, but the number that * actually runs lives in two sibling repos, each with its own literal: * xchain-indexer/src/actions/batch.js (`this.commandLimit`) and - * xchain-sdk/src/batchLimits.js (`BATCH_COMMAND_LIMIT`). Prose drifting from + * xchain-sdk/src/protocol/batch_limits.js (`BATCH_COMMAND_LIMIT`). Prose drifting from * this file is the smaller failure mode; code drifting from this file, or * the two services drifting from each other, is the one that actually * breaks something on chain. @@ -258,14 +261,48 @@ test('prose command counts for the BATCH cap match the canonical value', () => { * sibling checkout is absent, the same convention every other cross-repo test * in this directory uses. */ -const INDEXER_BATCH = path.resolve(ROOT, '../xchain-indexer/src/actions/batch.js'); -const SDK_BATCH_LIMITS = path.resolve(ROOT, '../xchain-sdk/src/batchLimits.js'); -const haveIndexerBatch = fs.existsSync(INDEXER_BATCH); -const haveSdkBatchLimits = fs.existsSync(SDK_BATCH_LIMITS); +const INDEXER_BATCH = moduleEntry(path.resolve(ROOT, '../xchain-indexer/src/actions/batch.js')); + +/* The limits are read as SOURCE TEXT and matched with a regex, so the file this + * resolves to has to be the one that declares them. The SDK's layout pass moved + * batchLimits.js to src/protocol/batch_limits.js and left a one-line re-export + * behind, which keeps require() working but carries none of the declarations, and a + * sibling checkout can sit on either side of that move. Follow a bare re-export to + * its target and fall back to the pre-move spelling, then report the path that was + * READ: a failure naming the stub would send the next reader to a file with none of + * the numbers in it. */ +function resolveSdkSource(pinned, premove) { + const read = (rel) => { + const abs = path.resolve(ROOT, rel); + if (!fs.existsSync(abs)) return null; + const body = fs.readFileSync(abs, 'utf8'); + const reexport = body.match(/^module\.exports\s*=\s*require\('([^']+)'\);\s*$/m); + const code = body.replace(/\/\*[\s\S]*?\*\/|\/\/[^\n]*/g, '').trim(); + if (!reexport || code !== reexport[0].trim()) return { rel, body }; + const target = path.join(path.dirname(rel), reexport[1]); + const abs2 = path.resolve(ROOT, target); + return fs.existsSync(abs2) ? { rel: target, body: fs.readFileSync(abs2, 'utf8') } : null; + }; + return read(pinned) || read(premove); +} +const SDK_BATCH = resolveSdkSource('../xchain-sdk/src/protocol/batch_limits.js', + '../xchain-sdk/src/batchLimits.js'); +const SDK_BATCH_LIMITS = SDK_BATCH ? SDK_BATCH.rel : '../xchain-sdk/src/protocol/batch_limits.js'; +/* The skips below are for a bare clone, by name. A run that declared the siblings + * supplied (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set, with both + * repos in .ci-siblings) throws in the helper instead, naming every spelling tried: the + * number that actually runs on chain lives in those two files, and skipping leaves the + * drift they exist to catch unchecked but green. */ +const noIndexerBatch = sibling('xchain-indexer', + [[INDEXER_BATCH, path.join(INDEXER_BATCH.replace(/\.js$/, ''), 'index.js')]]).skip; +const noSdkBatchLimits = sibling('xchain-sdk', SDK_BATCH ? [] : [[ + path.resolve(ROOT, '../xchain-sdk/src/protocol/batch_limits.js'), + path.resolve(ROOT, '../xchain-sdk/src/batchLimits.js'), +]]).skip; test('xchain-indexer commandLimit matches the canonical BATCH_COMMAND_LIMIT', - { skip: !haveIndexerBatch && 'sibling xchain-indexer not present in this checkout' }, () => { - const src = fs.readFileSync(INDEXER_BATCH, 'utf8'); + { skip: noIndexerBatch }, () => { + const src = readModuleSource(INDEXER_BATCH); const m = /this\.commandLimit\s*=\s*(\d+)\s*;/.exec(src); assert.ok(m, 'this.commandLimit assignment not found in xchain-indexer/src/actions/batch.js; ' + 'the declaration shape changed, re-point this regex'); @@ -275,10 +312,10 @@ test('xchain-indexer commandLimit matches the canonical BATCH_COMMAND_LIMIT', }); test('xchain-sdk BATCH_COMMAND_LIMIT matches the canonical value', - { skip: !haveSdkBatchLimits && 'sibling xchain-sdk not present in this checkout' }, () => { - const src = fs.readFileSync(SDK_BATCH_LIMITS, 'utf8'); + { skip: noSdkBatchLimits }, () => { + const src = SDK_BATCH.body; const m = /const\s+BATCH_COMMAND_LIMIT\s*=\s*(\d+)\s*;/.exec(src); - assert.ok(m, 'BATCH_COMMAND_LIMIT declaration not found in xchain-sdk/src/batchLimits.js; ' + assert.ok(m, `BATCH_COMMAND_LIMIT declaration not found in ${SDK_BATCH_LIMITS}; ` + 'the declaration shape changed, re-point this regex'); assert.strictEqual(Number(m[1]), CONSTANTS.BATCH_COMMAND_LIMIT, `xchain-sdk's BATCH_COMMAND_LIMIT is ${m[1]}, but protocol/constants.js BATCH_COMMAND_LIMIT ` @@ -292,7 +329,7 @@ test('xchain-sdk BATCH_COMMAND_LIMIT matches the canonical value', * and the numbers again live in three places: protocol/constants.js * (BATCH_WEIGHT_BUDGET and BATCH_COMMAND_WEIGHTS, canonical), * xchain-indexer/src/actions/batch.js (`this.weightBudget` and the - * `this.commandWeights[...]` assignments) and xchain-sdk/src/batchLimits.js + * `this.commandWeights[...]` assignments) and xchain-sdk/src/protocol/batch_limits.js * (`BATCH_WEIGHT_BUDGET` and the `BATCH_COMMAND_WEIGHTS` literal). These are * consensus values: a budget that drifts moves batch verdicts, and a weight * table that drifts moves them per action, either of which forks the SDK's @@ -300,8 +337,8 @@ test('xchain-sdk BATCH_COMMAND_LIMIT matches the canonical value', * command-cap tests above. */ test('xchain-indexer weightBudget matches the canonical BATCH_WEIGHT_BUDGET', - { skip: !haveIndexerBatch && 'sibling xchain-indexer not present in this checkout' }, () => { - const src = fs.readFileSync(INDEXER_BATCH, 'utf8'); + { skip: noIndexerBatch }, () => { + const src = readModuleSource(INDEXER_BATCH); const m = /this\.weightBudget\s*=\s*(\d+)\s*;/.exec(src); assert.ok(m, 'this.weightBudget assignment not found in xchain-indexer/src/actions/batch.js; ' + 'the declaration shape changed, re-point this regex'); @@ -311,10 +348,10 @@ test('xchain-indexer weightBudget matches the canonical BATCH_WEIGHT_BUDGET', }); test('xchain-sdk BATCH_WEIGHT_BUDGET matches the canonical value', - { skip: !haveSdkBatchLimits && 'sibling xchain-sdk not present in this checkout' }, () => { - const src = fs.readFileSync(SDK_BATCH_LIMITS, 'utf8'); + { skip: noSdkBatchLimits }, () => { + const src = SDK_BATCH.body; const m = /const\s+BATCH_WEIGHT_BUDGET\s*=\s*(\d+)\s*;/.exec(src); - assert.ok(m, 'BATCH_WEIGHT_BUDGET declaration not found in xchain-sdk/src/batchLimits.js; ' + assert.ok(m, `BATCH_WEIGHT_BUDGET declaration not found in ${SDK_BATCH_LIMITS}; ` + 'the declaration shape changed, re-point this regex'); assert.strictEqual(Number(m[1]), CONSTANTS.BATCH_WEIGHT_BUDGET, `xchain-sdk's BATCH_WEIGHT_BUDGET is ${m[1]}, but protocol/constants.js BATCH_WEIGHT_BUDGET ` @@ -322,8 +359,8 @@ test('xchain-sdk BATCH_WEIGHT_BUDGET matches the canonical value', }); test('xchain-indexer commandWeights matches the canonical BATCH_COMMAND_WEIGHTS', - { skip: !haveIndexerBatch && 'sibling xchain-indexer not present in this checkout' }, () => { - const src = fs.readFileSync(INDEXER_BATCH, 'utf8'); + { skip: noIndexerBatch }, () => { + const src = readModuleSource(INDEXER_BATCH); const table = {}; const entry = /this\.commandWeights\[['"]([A-Z]+)['"]\]\s*=\s*(\d+)\s*;/g; let m; @@ -338,10 +375,10 @@ test('xchain-indexer commandWeights matches the canonical BATCH_COMMAND_WEIGHTS' }); test('xchain-sdk BATCH_COMMAND_WEIGHTS matches the canonical table', - { skip: !haveSdkBatchLimits && 'sibling xchain-sdk not present in this checkout' }, () => { - const src = fs.readFileSync(SDK_BATCH_LIMITS, 'utf8'); + { skip: noSdkBatchLimits }, () => { + const src = SDK_BATCH.body; const block = /const\s+BATCH_COMMAND_WEIGHTS\s*=\s*Object\.freeze\(\{([^}]*)\}\)/.exec(src); - assert.ok(block, 'BATCH_COMMAND_WEIGHTS literal not found in xchain-sdk/src/batchLimits.js; ' + assert.ok(block, `BATCH_COMMAND_WEIGHTS literal not found in ${SDK_BATCH_LIMITS}; ` + 'the declaration shape changed, re-point this regex'); const table = {}; const entry = /([A-Z]+)\s*:\s*(\d+)/g; diff --git a/test/regtest-tip-age-escape-hatch.test.js b/test/regtest-tip-age-escape-hatch.test.js index 7088cbff..4f914114 100644 --- a/test/regtest-tip-age-escape-hatch.test.js +++ b/test/regtest-tip-age-escape-hatch.test.js @@ -37,9 +37,22 @@ * the built-in regtest default the operator turned down, this goes red * and the decision gets re-made deliberately instead of by patch. * + * WHERE THE EXPLORER HALF LIVES. The explorer's db.js was split into a + * composition root, now `src/db/index.js`, plus per-family reader modules. The + * tip-age constant and `tipMaxAgeSeconds` both live in `src/db/readers/health.js`, + * and src/db/index.js mixes that module into Database.prototype, so each source + * assertion reads the one file its behaviour lives in, and src/db/index.js is + * pinned only for the fact that it still composes health.js (otherwise the + * regex would guard an orphan). Never widen these to a glob over `src/db/`: a + * glob matches wherever the text happens to appear, not where the method + * Database actually runs is defined. + * * xchain-explorer is a sibling repo in the monorepo checkout, not a dependency - * of xchain-documentation. When it is absent (docs repo cloned on its own) the - * source-derived assertions skip and the prose assertions still run. + * of xchain-documentation. When the REPO is absent (docs repo cloned on its + * own) the source-derived assertions skip and the prose assertions still run. + * When the repo is present but a file this gate reads has moved, the gate + * FAILS: keying the skip on the file rather than the repo is how a later move + * would silently unpin it. * ********************************************************************/ @@ -47,14 +60,36 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DEV_DOC = path.resolve(__dirname, '../developer-guide/regtest-development.md'); const CFG_DOC = path.resolve(__dirname, '../components/explorer/configuration.md'); -const DB_SRC = path.resolve(__dirname, '../../xchain-explorer/src/db.js'); +const EXPLORER = path.resolve(__dirname, '../../xchain-explorer'); +const DB_SRC = path.join(EXPLORER, 'src/db/index.js'); +const HEALTH_SRC = path.join(EXPLORER, 'src/db/readers/health.js'); const devDoc = fs.readFileSync(DEV_DOC, 'utf8'); const cfgDoc = fs.readFileSync(CFG_DOC, 'utf8'); -const haveDb = fs.existsSync(DB_SRC); +// Repo presence only: a pinned file gone from a PRESENT explorer fails inside the test by +// name (readSource below). Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1. +const noExplorer = sibling('xchain-explorer').skip; + +// Reads an explorer source file this gate is pinned to, failing with the path +// named when it is gone: with the repo present, a missing file means the code +// moved and this pin has to follow it, never that the assertion may skip. +function readExplorerSource(file) { + assert.ok(fs.existsSync(file), + path.relative(EXPLORER, file) + ' is gone from xchain-explorer; repoint this gate at the file the behaviour moved to'); + // A reader family is an entry file plus a sibling directory of parts named + // after it, so the behaviour this gate pins can sit in either. Read both, + // in sorted path order, the way the explorer's own source-text helper does. + const parts = file.replace(/\.js$/, ''); + let text = fs.readFileSync(file, 'utf8'); + if (fs.existsSync(parts) && fs.statSync(parts).isDirectory()) + for (const name of fs.readdirSync(parts).sort()) + if (name.endsWith('.js')) text += '\n' + fs.readFileSync(path.join(parts, name), 'utf8'); + return text; +} // The body of `tipMaxAgeSeconds`, from its signature to the closing brace of // the method, by brace depth. Read as source text rather than by calling it: @@ -94,15 +129,36 @@ describe('regtest tip-age escape hatch is documented for dev setups', () => { 'the configuration page no longer shows the per-coin gate disabled for dev/regtest'); }); - test('the documented default matches the explorer default', { skip: !haveDb && 'xchain-explorer not present in this checkout' }, () => { - const m = /TIP_MAX_AGE_DEFAULT_S\s*=\s*(\d+)/.exec(fs.readFileSync(DB_SRC, 'utf8')); + test('the explorer composition root still composes the health readers this gate reads', { skip: noExplorer }, () => { + assert.match(readExplorerSource(DB_SRC), /require\(\s*['"]\.\/readers\/health(\.js)?['"]\s*\)/, + 'xchain-explorer src/db/index.js no longer requires ./readers/health.js, so the tip-age assertions below read a file Database does not use'); + }); + + test('the documented default matches the explorer default', { skip: noExplorer }, () => { + const m = /TIP_MAX_AGE_DEFAULT_S\s*=\s*(\d+)/.exec(readExplorerSource(HEALTH_SRC)); assert.ok(m, 'xchain-explorer no longer defines TIP_MAX_AGE_DEFAULT_S'); assert.match(devDoc, new RegExp('\\b' + m[1] + '\\b'), 'the regtest guide states a tip-age default other than the explorer\'s ' + m[1] + 's'); }); - test('no built-in regtest exemption was added to the explorer', { skip: !haveDb && 'xchain-explorer not present in this checkout' }, () => { - const body = tipMaxAgeSource(fs.readFileSync(DB_SRC, 'utf8')); + // The guide documents a per-coin knob and a global one. If the method stops + // reading either, the page describes an escape hatch that no longer opens, + // and a developer on an idle regtest stack is back to 503s with no recourse. + test('the explorer still reads both documented knobs, per-coin before global', { skip: noExplorer }, () => { + const code = tipMaxAgeSource(readExplorerSource(HEALTH_SRC)) + .replace(/\/\*[\s\S]*?\*\//g, '').replace(/\/\/[^\n]*/g, ''); + const perCoin = code.search(/['"]EXPLORER_TIP_MAX_AGE_S_['"]\s*\+/); + const global = code.search(/\bEXPLORER_TIP_MAX_AGE_S\b(?!_)/); + assert.notEqual(perCoin, -1, + 'tipMaxAgeSeconds no longer reads the per-coin EXPLORER_TIP_MAX_AGE_S_ knob the regtest guide documents'); + assert.notEqual(global, -1, + 'tipMaxAgeSeconds no longer reads the global EXPLORER_TIP_MAX_AGE_S knob the regtest guide documents'); + assert.ok(perCoin < global, + 'tipMaxAgeSeconds reads the global knob before the per-coin one, so a per-coin 0 no longer overrides it'); + }); + + test('no built-in regtest exemption was added to the explorer', { skip: noExplorer }, () => { + const body = tipMaxAgeSource(readExplorerSource(HEALTH_SRC)); // Comments explain the regtest case, and should: it is the reason the // hatch exists. Only executable text is searched for a network name. const code = body.replace(/\/\*[\s\S]*?\*\//g, '').replace(/\/\/[^\n]*/g, ''); diff --git a/test/release-key-fingerprint-channels.test.js b/test/release-key-fingerprint-channels.test.js index 266e5021..a487e72a 100644 --- a/test/release-key-fingerprint-channels.test.js +++ b/test/release-key-fingerprint-channels.test.js @@ -45,6 +45,7 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const RECIPE = path.join(DOC_ROOT, 'components/wallet/release/verify-release.md'); @@ -230,12 +231,14 @@ describe('release key fingerprint cross-references', () => { describe('the channels are documents that publish a fingerprint', () => { const wallet = path.resolve(DOC_ROOT, '../xchain-wallet/SECURITY.md'); const website = path.resolve(DOC_ROOT, '../xchain-websites/xchain.io/security/index.html'); + // Each channel skips by name on a bare clone and throws under XCHAIN_REQUIRE_SIBLINGS=1. + const noWallet = sibling('xchain-wallet', [wallet]).skip; + const noWebsite = sibling('xchain-websites', [website]).skip; // Pending the key ceremony neither channel carries a VALUE yet, so // what is checked is the named slot plus its stated empty state. A // channel with no slot at all is the defect this guard exists for. - test('channel one: SECURITY.md in xchain-wallet', (t) => { - if (!fs.existsSync(wallet)) return t.skip('sibling xchain-wallet not checked out'); + test('channel one: SECURITY.md in xchain-wallet', { skip: noWallet }, () => { const src = read(wallet); assert.ok( FINGERPRINT_VALUE.test(src) || /PGP fingerprint:/i.test(src), @@ -248,8 +251,7 @@ describe('release key fingerprint cross-references', () => { ); }); - test('channel two: https://xchain.io/security', (t) => { - if (!fs.existsSync(website)) return t.skip('sibling xchain-websites not checked out'); + test('channel two: https://xchain.io/security', { skip: noWebsite }, () => { const src = read(website); assert.ok( /id="release-key-fingerprint"/.test(src), diff --git a/test/schema-table-coverage.test.js b/test/schema-table-coverage.test.js index 478572a7..b7253238 100644 --- a/test/schema-table-coverage.test.js +++ b/test/schema-table-coverage.test.js @@ -51,6 +51,7 @@ const fs = require('node:fs'); const path = require('node:path'); const cov = require('../lib/schema-table-coverage.js'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const PLATFORM_ROOT = path.resolve(DOC_ROOT, '..'); @@ -136,10 +137,13 @@ describe('schema table coverage', () => { const { doc, repo } = component; const sqlDir = path.resolve(PLATFORM_ROOT, repo, cov.SQL_ROOT); const dbDoc = path.join(DOC_ROOT, cov.docPathFor(doc)); - const havePair = fs.existsSync(sqlDir) && fs.existsSync(dbDoc); + // The sibling half skips by name on a bare clone and throws under + // XCHAIN_REQUIRE_SIBLINGS=1; the doc half is this repo's own page. + const noSibling = sibling(repo, [sqlDir]).skip; + const noDoc = !fs.existsSync(dbDoc) && `${path.relative(DOC_ROOT, dbDoc)} is missing from this repo`; test(`${doc}: every table in src/sql is named in database.md`, - { skip: !havePair && `${repo} not present in this checkout` }, () => { + { skip: noSibling || noDoc }, () => { const reader = cov.workingTreeReader(); const survey = cov.buildSurvey({ diff --git a/test/sdk-action-surface.test.js b/test/sdk-action-surface.test.js index b295fbde..c1ae944a 100644 --- a/test/sdk-action-surface.test.js +++ b/test/sdk-action-surface.test.js @@ -31,20 +31,53 @@ * 2. components/sdk/actions.md names exactly the SDK-invocable set, and * never the five that are not invocable, so nobody goes looking for a * builder that does not exist. - * 3. The sessions.md convenience table lists exactly the action types - * walletSession.js actually exposes. + * 3. The sessions.md convenience table lists exactly the action types the + * SDK's wallet-session module actually exposes, whichever side of the + * SDK's layout move the sibling checkout sits on. */ const test = require('node:test'); const assert = require('node:assert'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); +// an entry plus every part it was split into, the platform's split convention +const { readModuleSource } = require('../lib/indexer-source.js'); const ROOT = path.join(__dirname, '..'); const SPECS = path.join(ROOT, 'protocol', 'actions'); const SDK = path.join(ROOT, '..', 'xchain-sdk', 'src'); + +/* The session surface is read as SOURCE TEXT and matched with a regex, so the file + * this resolves to has to be the one that declares the methods. The SDK's layout pass + * moved walletSession.js to utils/wallet_session.js and left a one-line re-export at + * the old path, which keeps require() working but carries none of the declarations, + * and a sibling checkout can sit on either side of that move. Follow a bare re-export + * to its target and fall back to the pre-move spelling, then report the path that was + * READ: pinning one spelling makes this guard fail against a stub on one side of the + * move and skip silently on the other, and neither reading checks the docs. + * + * The declarations are then read as the ENTRY PLUS ITS PARTS: the SDK's structure pass + * moved the per-action shortcuts out of utils/wallet_session.js into + * utils/wallet_session/action_shortcuts.js, and reading the entry alone reported every + * session method as missing while the docs and the code still agreed. */ +function resolveSdkSource(pinned, premove) { + const read = (rel) => { + const abs = path.join(SDK, rel); + if (!fs.existsSync(abs)) return null; + const body = fs.readFileSync(abs, 'utf8'); + const reexport = body.match(/^module\.exports\s*=\s*require\('([^']+)'\);\s*$/m); + const code = body.replace(/\/\*[\s\S]*?\*\/|\/\/[^\n]*/g, '').trim(); + if (!reexport || code !== reexport[0].trim()) return { rel, body: readModuleSource(abs) }; + const target = path.join(path.dirname(rel), reexport[1]); + const abs2 = path.join(SDK, target); + return fs.existsSync(abs2) ? { rel: target, body: readModuleSource(abs2) } : null; + }; + return read(pinned) || read(premove); +} + const SDK_MAIN = path.join(SDK, 'XChainSDK.js'); -const SDK_SESSION = path.join(SDK, 'walletSession.js'); -const haveSdk = fs.existsSync(SDK_MAIN) && fs.existsSync(SDK_SESSION); +const SESSION = resolveSdkSource('utils/wallet_session.js', 'walletSession.js'); +const SESSION_PATH = SESSION ? `xchain-sdk/src/${SESSION.rel}` : 'xchain-sdk/src/utils/wallet_session.js'; /** Actions that exist but are never user-submittable, so never in the SDK. */ const NOT_INVOCABLE = ['ANCHOR', 'ATTEST', 'NODEPROOF', 'ROLLCALL', 'SLASH', 'XCALL']; @@ -68,6 +101,15 @@ function named(text) { return NAMED.filter((n) => new RegExp(`\\b${n}\\b`).test(text)).sort(); } +/* The skips below are for a bare clone, by name. A run that declared the sibling + * supplied (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set) throws in + * the helper instead, naming the entry and both session spellings tried: the surface + * these pages describe lives in that checkout, and skipping leaves the drift this file + * exists to catch unchecked but green. */ +const noSdk = sibling('xchain-sdk', [SDK_MAIN].concat(SESSION ? [] : [[ + path.join(SDK, 'utils/wallet_session.js'), path.join(SDK, 'walletSession.js'), +]])).skip; + test('concepts/actions.md names every action that has a spec', () => { const missing = NAMED.filter((n) => !named(doc('concepts/actions.md')).includes(n)); assert.deepStrictEqual(missing, [], @@ -75,8 +117,14 @@ test('concepts/actions.md names every action that has a spec', () => { }); test('the SDK reference covers exactly the invocable set', - { skip: !haveSdk && 'xchain-sdk not present in this checkout' }, () => { - const invocable = methodsFor(fs.readFileSync(SDK_MAIN, 'utf8')); + { skip: noSdk }, () => { + // Same entry-plus-parts reading as SESSION above: the SDK's structure pass moved + // the action shorthands (send, deploy, batch, ...) out of XChainSDK.js into + // src/XChainSDK/*.js parts that installMethods() attaches to the prototype, so + // the entry alone no longer declares them. readModuleSource follows the platform's + // split convention (entry first, then every part under a same-named directory) and + // reads the entry alone, unchanged, on a tree that was never split. + const invocable = methodsFor(readModuleSource(SDK_MAIN)); assert.deepStrictEqual(invocable, NAMED.filter((n) => !NOT_INVOCABLE.includes(n)), 'the SDK builder methods no longer match "every action except ' + NOT_INVOCABLE.join(', ') + '". Re-derive the split before touching the docs.'); @@ -93,8 +141,8 @@ test('the SDK reference covers exactly the invocable set', }); test('the session convenience table lists exactly the session methods', - { skip: !haveSdk && 'xchain-sdk not present in this checkout' }, () => { - const sessionActions = methodsFor(fs.readFileSync(SDK_SESSION, 'utf8')); + { skip: noSdk }, () => { + const sessionActions = methodsFor(SESSION.body); // Only the convenience-method table, not the whole page: the prose below it // discusses BATCH and the version-pinned variants by name on purpose. @@ -108,7 +156,7 @@ test('the session convenience table lists exactly the session methods', const absent = listed.filter((n) => NAMED.includes(n) && !sessionActions.includes(n)); assert.deepStrictEqual(missing, [], - 'walletSession.js exposes these action types and the table omits them: ' + missing.join(', ')); + `${SESSION_PATH} exposes these action types and the table omits them: ` + missing.join(', ')); assert.deepStrictEqual(absent, [], - 'the table offers action methods the session does not have: ' + absent.join(', ')); + `the table offers action methods ${SESSION_PATH} does not have: ` + absent.join(', ')); }); diff --git a/test/settlement-and-delivery-claims.test.js b/test/settlement-and-delivery-claims.test.js index 4881566d..4944da12 100644 --- a/test/settlement-and-delivery-claims.test.js +++ b/test/settlement-and-delivery-claims.test.js @@ -50,14 +50,21 @@ const test = require('node:test'); const assert = require('node:assert'); const fs = require('node:fs'); const path = require('node:path'); +const { moduleExists, readModuleSource } = require('../lib/indexer-source.js'); // an entry plus every part it was split into +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = process.env.XCHAIN_DOCS_ROOT || path.join(__dirname, '..'); const INDEXER = path.resolve(path.join(__dirname, '..'), '../xchain-indexer/src'); -const haveIndexer = fs.existsSync(path.join(INDEXER, 'actions', 'order.js')); -const skipNoIndexer = !haveIndexer && 'sibling xchain-indexer not present in this checkout'; +/* Skips by name on a bare clone. A run that declared the sibling supplied + * (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set, with xchain-indexer + * in .ci-siblings) throws in the helper instead, so a dropped checkout cannot leave the + * source half uncompared while the file still reports green. Both spellings of the entry + * are tried, since the indexer's split convention moves `order.js` to `order/index.js`. */ +const skipNoIndexer = sibling('xchain-indexer', + [[path.join(INDEXER, 'actions', 'order.js'), path.join(INDEXER, 'actions', 'order', 'index.js')]]).skip; -const readSrc = (rel) => fs.readFileSync(path.join(INDEXER, rel), 'utf8'); +const readSrc = (rel) => readModuleSource(path.join(INDEXER, rel)); const readDoc = (rel) => fs.readFileSync(path.join(DOC_ROOT, rel), 'utf8'); const creating = readDoc('user-guide/creating-tokens.md'); @@ -181,10 +188,11 @@ test('the ownership-sale and key-handoff source facts still hold', { skip: skipN 'send.js no longer enforces the key-handoff MESSAGE, so the guide\'s ' + '"only a direct send carries the key" wording is no longer accurate'); - const others = ['actions/order_match.js', 'actions/dispense.js', 'actions/cross_settle.js'] - .filter((rel) => fs.existsSync(path.join(INDEXER, rel))); - assert.ok(others.length > 0, 'none of the DEX settlement handlers were found to check'); + const others = ['actions/order_match.js', 'actions/dispense.js', 'actions/cross_settle/index.js']; for(const rel of others){ + assert.ok(moduleExists(path.join(INDEXER, rel)), + `${rel} is missing from the sibling indexer checkout. A moved or renamed settlement ` + + 'handler must be repointed here, not silently dropped from the set this test checks.'); assert.ok(!/requires key handoff message/.test(readSrc(rel)), `${rel} now enforces a key handoff. If a settlement path delivers the key, the ` + 'guide\'s "a buyer on the DEX gets no key" wording must change.'); @@ -196,9 +204,9 @@ test('the ownership-sale and key-handoff source facts still hold', { skip: skipN 'transferTokenOwnership now emits a MESSAGE. If an ownership sale delivers key ' + 'material, use-cases.md\'s archive bullet must change back.'); - const crossSettle = readSrc('actions/cross_settle.js'); + const crossSettle = readSrc('actions/cross_settle/index.js'); assert.match(crossSettle, /transferTokenOwnership\(/, - 'cross_settle.js no longer settles an ownership leg locally, which is the fact behind ' + 'cross_settle/index.js no longer settles an ownership leg locally, which is the fact behind ' + 'the "each chain hands over its own side" wording'); }); @@ -208,7 +216,7 @@ test('the guide scopes ownership-sale atomicity to a single chain', () => { assert.match(answer, /settles on one chain|single-chain/, 'faq.md again claims an issuer-rights sale settles in a single blockchain transaction ' + 'without scoping it to one chain. A cross-chain swap settles each leg separately ' - + '(cross_settle.js).'); + + '(cross_settle/index.js).'); assert.match(answer, /cross-chain\.md#residual-risk/, 'faq.md no longer points at the cross-chain residual-risk section'); diff --git a/test/sibling-checkout-helper.test.js b/test/sibling-checkout-helper.test.js new file mode 100644 index 00000000..7c669b34 --- /dev/null +++ b/test/sibling-checkout-helper.test.js @@ -0,0 +1,111 @@ +/********************************************************************* + * + * Copyright © 2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * The sibling-resolution helper every cross-repo suite skips through, driven + * against a throwaway platform root so each state it must tell apart (real, + * hollow, absent, present but missing a file) is built on disk rather than + * assumed from this checkout. The switch is passed explicitly so the throw + * path runs in every environment, not only where XCHAIN_REQUIRE_SIBLINGS is + * set. + */ +'use strict'; + +const { test, describe, before, after } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); + +const helper = require('./helpers/sibling_checkout.js'); + +let platformRoot; +before(() => { + platformRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'xchain-docs-sibling-')); + fs.mkdirSync(path.join(platformRoot, 'xchain-real', 'src'), { recursive: true }); + fs.writeFileSync(path.join(platformRoot, 'xchain-real', 'package.json'), '{}\n'); + fs.writeFileSync(path.join(platformRoot, 'xchain-real', 'src', 'present.js'), ''); + fs.mkdirSync(path.join(platformRoot, 'xchain-hollow')); +}); +after(() => { + fs.rmSync(platformRoot, { recursive: true, force: true }); +}); + +describe('sibling checkout resolution', () => { + test('the roots are this checkout and its parent', () => { + assert.equal(helper.DOC_ROOT, path.resolve(__dirname, '..')); + assert.equal(helper.PLATFORM_ROOT, path.resolve(__dirname, '..', '..')); + assert.equal(helper.siblingRoot('xchain-indexer'), path.join(helper.PLATFORM_ROOT, 'xchain-indexer')); + }); + + test('a checkout with a package.json is real', () => { + const state = helper.checkoutState('xchain-real', platformRoot); + assert.deepEqual(state, { root: path.join(platformRoot, 'xchain-real'), exists: true, hollow: false, real: true }); + const r = helper.sibling('xchain-real', ['src/present.js'], { required: true, platformRoot }); + assert.equal(r.have, true); + assert.equal(r.skip, false); + assert.deepEqual(r.missing, []); + }); + + test('a directory without package.json is hollow, and hollow reads as absent', () => { + const state = helper.checkoutState('xchain-hollow', platformRoot); + assert.equal(state.exists, true); + assert.equal(state.hollow, true); + assert.equal(state.real, false); + const r = helper.sibling('xchain-hollow', [], { required: false, platformRoot }); + assert.equal(r.have, false); + assert.match(r.skip, /xchain-hollow/); + assert.match(r.skip, /hollow/); + assert.match(r.skip, /package\.json/); + }); + + test('an absent sibling skips by name without the switch', () => { + const r = helper.sibling('xchain-absent', ['src/anything.js'], { required: false, platformRoot }); + assert.equal(r.have, false); + assert.match(r.skip, /xchain-absent is not checked out at /); + // The root first, then every path tried under it, so the message says where it looked. + assert.deepEqual(r.missing, [path.join(platformRoot, 'xchain-absent'), + path.join(platformRoot, 'xchain-absent', 'src', 'anything.js')]); + }); + + test('a real sibling missing a wanted file skips naming that file', () => { + const r = helper.sibling('xchain-real', ['src/present.js', 'src/gone.js'], { required: false, platformRoot }); + assert.equal(r.have, false); + assert.deepEqual(r.missing, [path.join(platformRoot, 'xchain-real', 'src', 'gone.js')]); + assert.match(r.skip, /is missing .*src\/gone\.js/); + }); + + test('alternative spellings pass when any one exists and name every one tried when none does', () => { + const ok = helper.sibling('xchain-real', [['src/moved.js', 'src/present.js']], { required: false, platformRoot }); + assert.equal(ok.have, true); + const bad = helper.sibling('xchain-real', [['src/a.js', 'src/b.js']], { required: false, platformRoot }); + assert.equal(bad.have, false); + assert.match(bad.missing[0], /src\/a\.js or .*src\/b\.js$/); + }); + + test('absolute wants are taken as given', () => { + const abs = path.join(platformRoot, 'xchain-real', 'src', 'present.js'); + assert.equal(helper.sibling('xchain-real', [abs], { required: false, platformRoot }).have, true); + }); + + test('with the switch set, absent, hollow and incomplete siblings throw naming the repo and the path', () => { + assert.throws(() => helper.sibling('xchain-absent', [], { required: true, platformRoot }), + /XCHAIN_REQUIRE_SIBLINGS=1 but sibling xchain-absent is not checked out/); + assert.throws(() => helper.sibling('xchain-hollow', [], { required: true, platformRoot }), + /XCHAIN_REQUIRE_SIBLINGS=1 but sibling xchain-hollow .* hollow/); + assert.throws(() => helper.sibling('xchain-real', ['src/gone.js'], { required: true, platformRoot }), + /XCHAIN_REQUIRE_SIBLINGS=1 but sibling xchain-real is missing .*src\/gone\.js/); + }); + + test('the switch defaults to the environment', () => { + assert.equal(helper.REQUIRED, process.env.XCHAIN_REQUIRE_SIBLINGS === '1'); + }); +}); diff --git a/test/sibling-source-path-existence.test.js b/test/sibling-source-path-existence.test.js new file mode 100644 index 00000000..8998371e --- /dev/null +++ b/test/sibling-source-path-existence.test.js @@ -0,0 +1,431 @@ +/********************************************************************* + * + * Copyright © 2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC - https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * Every sibling source path the documentation names must exist in that sibling. + * + * WHY. Nothing here caught a stale path into a sibling repo: proven 2026-09-14, + * with `node src/migrate.js` restored into both indexer pages after the rename + * had moved it to `src/migration/migrate.js`, this suite still read 493 pass and + * exit 0. A layout pass renames dozens of files per repo, and each rename left + * the pages that cite the old path reading green. This file resolves every + * cited path against the checkout beside this repo and fails naming each one + * that is gone, so the next rename fails here instead of on a reader. + * + * WHAT counts as a reference, and against what it resolves: + * - `xchain-/` anywhere in the corpus, against that repo; + * - a bare `/` on a page under components//, against + * xchain-, when is a top-level entry of that repo (so `api/v1` + * and other non-path slashes never count). A line that names another + * component ("the indexer's src/actions/index.js" on a VM page) may resolve + * in that repo too, and "platform" adds the platform checkout itself. + * Either form may carry `:`, and the line must exist in the file. + * Resolution follows what a reader would do: the path as written, then a + * Node specifier (`+.js`, `/index.js`), then the split convention this + * platform uses (`name.js` moved to `name/index.js`). + * + * WHAT the venue decides. The bare form is classified against the top-level + * entries of the component's checkout, so a component whose sibling is absent + * yields no bare references at all, not a skip per reference. GitHub CI checks + * out ONE sibling (xchain-indexer, see .github/workflows/ci.yml) and so finds + * the repo-qualified corpus plus the indexer pages' bare paths, about a quarter + * of what the platform checkout finds: measured 2026-09-16, 133 against 466. + * The corpus floor therefore has two parts: the repo-qualified half, which + * needs no sibling and is judged everywhere, and the whole-corpus floor, which + * is judged where every declared component sibling is present and otherwise + * skips NAMING the absent trees (the env-var coverage suite gates its fleet + * floor the same way). Under XCHAIN_REQUIRE_SIBLINGS=1 an absent declared + * sibling throws before either floor is read. + * + * WHAT is left alone, each for a reason a reader can check: + * - HTML comments: the `` stamps record where a page + * came from, and the source file was deleted by that port; + * - CHANGELOG.md: history, true when written; + * - placeholders (``, `vX.Y.Z`, `FOO`) and build outputs (dist/, + * release-artifacts/), which no checkout carries; + * - a path the sibling's own .gitignore ignores (a generated config), since + * the page describes a runtime file, not source; + * - a path the line places "in the platform checkout" that no sibling has: + * the platform root is not a declared sibling, so the venue cannot grade it; + * - URLs, where the path after the host is someone else's namespace. + * + * Sibling absence goes through test/helpers/sibling_checkout.js like every + * other cross-repo suite: a named skip on a bare clone, a throw under + * XCHAIN_REQUIRE_SIBLINGS=1. A repo the docs cite that .ci-siblings does not + * declare skips by name even under the switch, since the venue never has it. + */ +'use strict'; + +const assert = require('node:assert/strict'); +const { test, describe } = require('node:test'); +const fs = require('node:fs'); +const os = require('node:os'); +const path = require('node:path'); +const { execFileSync } = require('node:child_process'); + +const { DOC_ROOT, PLATFORM_ROOT, sibling } = require('./helpers/sibling_checkout.js'); + +/* ------------------------------------------------------------------ * + * Corpus and repo roster + * ------------------------------------------------------------------ */ + +// Every tracked page except history and vendored trees, the convention the +// claim suites share. +function markdownFiles(dir, out = []) { + for (const e of fs.readdirSync(dir, { withFileTypes: true })) { + if (e.name === 'node_modules' || e.name === '.git' || e.name === 'dist') continue; + const p = path.join(dir, e.name); + if (e.isDirectory()) markdownFiles(p, out); + else if (e.name.endsWith('.md') && e.name !== 'CHANGELOG.md') out.push(p); + } + return out; +} + +/** Repos the venue checks out beside this one, from .ci-siblings. */ +function declaredSiblings() { + return fs.readFileSync(path.join(DOC_ROOT, '.ci-siblings'), 'utf8').split('\n') + .map((l) => l.trim()).filter((l) => l && !l.startsWith('#')); +} + +const DECLARED = new Set(declaredSiblings()); +// A component page names its repo by directory: components// is about xchain-. +const COMPONENT_REPOS = fs.readdirSync(path.join(DOC_ROOT, 'components'), { withFileTypes: true }) + .filter((e) => e.isDirectory()).map((e) => `xchain-${e.name}`); +const KNOWN_REPOS = new Set([...DECLARED, ...COMPONENT_REPOS, 'xchain-documentation']); + +/** Where a repo's tree is read from: this checkout for itself, the sibling slot otherwise. */ +function repoRoot(repo) { + return repo === 'xchain-documentation' ? DOC_ROOT : path.join(PLATFORM_ROOT, repo); +} + +/* ------------------------------------------------------------------ * + * Classifying a line + * ------------------------------------------------------------------ */ + +// A repo-qualified reference. The character before it may not be part of a +// longer path or a URL (`/`, `.`), an npm scope (`@`), a shell variable or +// home (`$`, `~`), or a word. +const EXPLICIT_RE = /(^|[^\w/.@$~-])(xchain-[a-z0-9-]+)\/((?:[\w.-]+\/)*[\w.-]*)(?::(\d+))?/g; +// A bare path with at least one slash, same left context. +const BARE_RE = /(^|[^\w/.@$~-])((?:[\w.-]+\/)+[\w.-]*)(?::(\d+))?/g; + +const OUTPUT_DIRS = new Set(['dist', 'build', 'release-artifacts', 'coverage', 'node_modules']); + +/** A token no checkout could carry: a template slot or a build product. */ +function isPlaceholder(rel) { + if (/X\.Y\.Z|[<>*{}$]|\b(FOO|BAR|BAZ)\b/.test(rel)) return true; + return rel.split('/').some((seg) => OUTPUT_DIRS.has(seg)); +} + +/** The path as a reader would take it: no trailing slash or sentence dot. */ +function cleanRel(rel) { + return rel.replace(/[/.]+$/, ''); +} + +/** The match runs into a template slot (`src/configs/.js`), so it is a prefix, not a path. */ +function runsIntoSlot(line, m) { + return /[<{$*]/.test(line.charAt(m.index + m[0].length)); +} + +/** `true` when the match at `index` sits inside a URL on this line. */ +function insideUrl(line, index) { + return /https?:\/\/[^\s)>\]]*$/.test(line.slice(0, index + 1)); +} + +/** + * The sibling-path references one line makes. + * + * @param {string} rawLine + * @param {object} ctx + * @param {string|null} ctx.sectionRepo xchain- for a components// page + * @param {(repo: string) => Set} ctx.topsOf top-level entries of a repo's tree + * @param {Set} ctx.known repo names a reference may name + * @returns {Array<{repo: string, rel: string, line: number|null, candidates: string[], form: 'explicit'|'bare'}>} + * `form` says which grammar matched: 'explicit' needs no sibling tree to be + * classified, 'bare' needs the section repo's top-level entries, so the two + * are floored apart (see the corpus test). + */ +function referencesIn(rawLine, ctx) { + const line = rawLine.replace(//g, (m) => ' '.repeat(m.length)); + const out = []; + let m; + EXPLICIT_RE.lastIndex = 0; + while ((m = EXPLICIT_RE.exec(line))) { + const [, lead, repo, rawRel, ln] = m; + const rel = cleanRel(rawRel); + if (!ctx.known.has(repo) || !rel || isPlaceholder(rel) || runsIntoSlot(line, m)) continue; + if (insideUrl(line, m.index + lead.length)) continue; + out.push({ repo, rel, line: ln ? Number(ln) : null, candidates: [repo], form: 'explicit' }); + } + if (!ctx.sectionRepo) return out; + const sectionTops = ctx.topsOf(ctx.sectionRepo); + BARE_RE.lastIndex = 0; + while ((m = BARE_RE.exec(line))) { + const [, lead, rawRel, ln] = m; + const rel = cleanRel(rawRel); + const top = rel.split('/')[0]; + if (!rel || !sectionTops.has(top) || isPlaceholder(rel) || runsIntoSlot(line, m)) continue; + if (insideUrl(line, m.index + lead.length)) continue; + // The section repo first; then any repo the line names by its short + // name, and the platform checkout when the line says "platform". + const candidates = [ctx.sectionRepo]; + for (const repo of ctx.known) { + if (repo === ctx.sectionRepo) continue; + const word = repo.slice('xchain-'.length).replace(/-/g, '[- ]'); + if (new RegExp(`\\b${word}\\b`, 'i').test(line) && ctx.topsOf(repo).has(top)) candidates.push(repo); + } + if (/\bplatform\b/i.test(line)) candidates.push('.'); + out.push({ repo: ctx.sectionRepo, rel, line: ln ? Number(ln) : null, candidates, form: 'bare' }); + } + return out; +} + +/* ------------------------------------------------------------------ * + * Resolving a reference against a tree + * ------------------------------------------------------------------ */ + +/** + * Where `rel` lands under `root`, the way a reader or require() would look. + * + * @returns {{found: string|null, tried: string[]}} + */ +function resolveUnder(root, rel) { + const abs = path.join(root, rel); + const tried = [abs, `${abs}.js`, path.join(abs, 'index.js')]; + if (abs.endsWith('.js')) tried.push(path.join(abs.slice(0, -3), 'index.js')); + return { found: tried.find((p) => fs.existsSync(p)) || null, tried }; +} + +/** Line `n` exists in the file (a directory has no lines). */ +function hasLine(file, n) { + let stat; + try { stat = fs.statSync(file); } catch { return false; } + if (!stat.isFile()) return false; + return fs.readFileSync(file, 'utf8').split('\n').length >= n; +} + +/** + * Whether the sibling's own .gitignore ignores `rel`: a generated file the page + * describes at runtime, not a source path. Read from the checkout's real + * location, since git refuses a pathspec that crosses a symlink. + */ +function ignoredBy(root, rel) { + let real; + try { real = path.dirname(fs.realpathSync(path.join(root, 'package.json'))); } catch { return false; } + try { + execFileSync('git', ['-C', real, 'check-ignore', '-q', '--', rel], { stdio: 'ignore' }); + return true; + } catch (err) { + return false; // status 1: not ignored; anything else: not a repo, so not ignored either + } +} + +/** + * @returns {{ok: boolean, why: string}} where `why` names what was tried + */ +function checkReference(ref, roots) { + const tried = []; + for (const repo of ref.candidates) { + const root = roots(repo); + if (!root) continue; + const r = resolveUnder(root, ref.rel); + tried.push(...r.tried); + if (!r.found) continue; + if (ref.line === null) return { ok: true, why: r.found }; + if (hasLine(r.found, ref.line)) return { ok: true, why: `${r.found}:${ref.line}` }; + tried.push(`${r.found} has fewer than ${ref.line} lines`); + } + if (ref.line === null && ref.candidates.some((repo) => roots(repo) && ignoredBy(roots(repo), ref.rel))) { + return { ok: true, why: 'gitignored in the sibling' }; + } + if (ref.candidates.includes('.')) return { ok: true, why: 'placed in the platform checkout, which is not a sibling' }; + return { ok: false, why: tried.map((p) => path.relative(PLATFORM_ROOT, p) || p).join(', ') }; +} + +/* ------------------------------------------------------------------ * + * The scan + * ------------------------------------------------------------------ */ + +const topsCache = new Map(); +function topsOf(repo) { + if (!topsCache.has(repo)) { + const root = repo === '.' ? PLATFORM_ROOT : repoRoot(repo); + let entries = []; + try { entries = fs.readdirSync(root); } catch { entries = []; } + topsCache.set(repo, new Set(entries)); + } + return topsCache.get(repo); +} + +function scanCorpus() { + const refs = []; + for (const file of markdownFiles(DOC_ROOT)) { + const rel = path.relative(DOC_ROOT, file); + const section = rel.match(/^components\/([a-z0-9-]+)\//); + const sectionRepo = section && KNOWN_REPOS.has(`xchain-${section[1]}`) ? `xchain-${section[1]}` : null; + const lines = fs.readFileSync(file, 'utf8').split('\n'); + lines.forEach((text, i) => { + for (const ref of referencesIn(text, { sectionRepo, topsOf, known: KNOWN_REPOS })) { + refs.push({ ...ref, at: `${rel}:${i + 1}` }); + } + }); + } + return refs; +} + +const REFS = scanCorpus(); +const BY_REPO = new Map(); +for (const ref of REFS) { + if (!BY_REPO.has(ref.repo)) BY_REPO.set(ref.repo, []); + BY_REPO.get(ref.repo).push(ref); +} + +/* ------------------------------------------------------------------ * + * The classifier and resolver on synthetic input + * ------------------------------------------------------------------ */ + +describe('sibling path reference classification', () => { + const known = new Set(['xchain-indexer', 'xchain-vm', 'xchain-hub']); + const tops = { 'xchain-indexer': new Set(['src', 'bin', 'package.json']), 'xchain-vm': new Set(['src', 'toolkit.js']), 'xchain-hub': new Set(['src']), '.': new Set(['bin']) }; + const ctx = { sectionRepo: null, topsOf: (r) => tops[r] || new Set(), known }; + const onIndexerPage = { ...ctx, sectionRepo: 'xchain-indexer' }; + + test('a repo-qualified path is a reference to that repo, with its line number', () => { + assert.deepEqual(referencesIn('see `xchain-indexer/src/migration/migrate.js:12` for the loop', ctx), + [{ repo: 'xchain-indexer', rel: 'src/migration/migrate.js', line: 12, candidates: ['xchain-indexer'], form: 'explicit' }]); + }); + + test('a repo the roster and the components tree do not know is not a reference', () => { + assert.deepEqual(referencesIn('xchain-nothing/src/a.js', ctx), []); + }); + + test('a provenance comment, a URL and a placeholder are not references', () => { + assert.deepEqual(referencesIn('', ctx), []); + assert.deepEqual(referencesIn('https://github.com/XChain-Platform/xchain-indexer/blob/master/src/a.js', ctx), []); + assert.deepEqual(referencesIn('cp xchain-hub/src/coins/FOO.js', ctx), []); + assert.deepEqual(referencesIn('`xchain-indexer/src/configs/.js`', ctx), []); + assert.deepEqual(referencesIn('unzip release-artifacts/vX.Y.Z/x.zip', onIndexerPage), []); + }); + + test('a bare path counts only on a component page and only under a top-level entry of that repo', () => { + assert.deepEqual(referencesIn('run `node src/migrate.js` then', ctx), []); + assert.deepEqual(referencesIn('run `node src/migrate.js` then', onIndexerPage), + [{ repo: 'xchain-indexer', rel: 'src/migrate.js', line: null, candidates: ['xchain-indexer'], form: 'bare' }]); + assert.deepEqual(referencesIn('GET api/v1/blocks', onIndexerPage), []); + assert.deepEqual(referencesIn('a/b in prose', onIndexerPage), []); + }); + + test('a bare path on a page whose sibling tree is absent cannot be classified, an explicit one still is', () => { + // The venue effect the corpus floor is built around: no tree, no + // top-level entries, so the bare grammar has nothing to match against. + const onAbsentSiblingPage = { ...ctx, sectionRepo: 'xchain-sync' }; + assert.deepEqual(referencesIn('run `node src/migrate.js` then', onAbsentSiblingPage), []); + assert.equal(referencesIn('see xchain-indexer/src/a.js', onAbsentSiblingPage).length, 1); + }); + + test('a line that names another component may resolve there, and "platform" adds the platform root', () => { + assert.deepEqual(referencesIn("the VM's src/index.js", onIndexerPage)[0].candidates, ['xchain-indexer', 'xchain-vm']); + assert.deepEqual(referencesIn('bin/check.sh in the platform checkout', onIndexerPage)[0].candidates, ['xchain-indexer', '.']); + }); + + test('a trailing slash or sentence dot is not part of the path', () => { + assert.equal(referencesIn('under xchain-indexer/src/actions/.', ctx)[0].rel, 'src/actions'); + }); +}); + +describe('sibling path resolution', () => { + let root; + test.before(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'xchain-docs-paths-')); + fs.mkdirSync(path.join(root, 'src', 'batch'), { recursive: true }); + fs.writeFileSync(path.join(root, 'src', 'batch', 'index.js'), 'a\nb\nc\n'); + fs.writeFileSync(path.join(root, 'toolkit.js'), ''); + }); + test.after(() => fs.rmSync(root, { recursive: true, force: true })); + + test('the path as written, a Node specifier, and the split convention all resolve', () => { + assert.ok(resolveUnder(root, 'src/batch/index.js').found); + assert.ok(resolveUnder(root, 'toolkit').found, 'require specifier without .js'); + assert.ok(resolveUnder(root, 'src/batch').found, 'directory module'); + assert.ok(resolveUnder(root, 'src/batch.js').found, 'name.js moved to name/index.js'); + assert.equal(resolveUnder(root, 'src/gone.js').found, null); + }); + + test('a cited line must exist in the file', () => { + const roots = () => root; + assert.equal(checkReference({ rel: 'src/batch/index.js', line: 4, candidates: ['x'] }, roots).ok, true); + assert.equal(checkReference({ rel: 'src/batch/index.js', line: 5, candidates: ['x'] }, roots).ok, false); + assert.equal(checkReference({ rel: 'src/batch', line: 1, candidates: ['x'] }, roots).ok, false, 'a directory has no lines'); + }); + + test('a dead reference names every path tried', () => { + const r = checkReference({ rel: 'src/gone.js', line: null, candidates: ['x'] }, () => root); + assert.equal(r.ok, false); + assert.match(r.why, /src\/gone\.js/); + assert.match(r.why, /src\/gone\/index\.js/); + }); +}); + +/* ------------------------------------------------------------------ * + * The corpus against the checkouts + * ------------------------------------------------------------------ */ + +describe('every sibling source path the documentation cites exists', () => { + test('the scan found the repo-qualified corpus', () => { + // The explicit grammar needs only the roster and the components tree, + // both in this repo, so it is judged on every venue. Measured + // 2026-09-16: 133 references. A scan reading far under that has lost + // the corpus or the classifier and must not pass as "nothing to check". + const explicit = REFS.filter((r) => r.form === 'explicit').length; + assert.ok(explicit >= 80, `only ${explicit} repo-qualified references found; the scanner is probably broken`); + }); + + // The bare grammar reads each component's checkout, so the whole-corpus + // floor is a full-checkout figure. A venue missing a declared component + // sibling (GitHub CI checks out xchain-indexer alone) cannot be held to it + // and is told which trees it lacks rather than accused of a broken + // scanner. Undeclared component repos (none of the ci venues ship them) + // do not gate the floor; their pages' references are a bonus where present. + const absentTrees = COMPONENT_REPOS.filter((repo) => DECLARED.has(repo) && !sibling(repo, [], { required: false }).have); + test('the scan found the whole corpus', + { skip: absentTrees.length ? `whole-corpus floor needs every declared component sibling; absent: ${absentTrees.join(', ')}` : false }, () => { + // Measured 2026-09-16 in the platform checkout: 466 references + // (133 repo-qualified and 333 bare) into 15 repos. + assert.ok(REFS.length >= 300, `only ${REFS.length} references found; the scanner is probably broken`); + assert.ok(BY_REPO.size >= 8, `references to only ${BY_REPO.size} repos found`); + }); + + for (const repo of [...BY_REPO.keys()].sort()) { + const refs = BY_REPO.get(repo); + // Declared siblings go through the switch; a cited repo the venue never + // checks out skips by name so the switch cannot demand it. + const declared = DECLARED.has(repo) || repo === 'xchain-documentation'; + const state = repo === 'xchain-documentation' + ? { skip: false } + : sibling(repo, [], declared ? {} : { required: false }); + const skip = state.skip ? `${state.skip}${declared ? '' : ' (not in .ci-siblings)'}` : false; + + test(`${repo}: ${refs.length} cited path(s) resolve in the checkout`, { skip }, () => { + const roots = (r) => { + if (r === '.') return PLATFORM_ROOT; + const st = r === repo ? { have: true } : sibling(r, [], { required: false }); + return st.have ? repoRoot(r) : null; + }; + const dead = []; + for (const ref of refs) { + const r = checkReference(ref, roots); + if (!r.ok) dead.push(`${ref.at} ${repo}/${ref.rel}${ref.line ? `:${ref.line}` : ''} (tried ${r.why})`); + } + assert.deepEqual(dead, [], `${dead.length} documentation reference(s) into ${repo} point at nothing; ` + + `repoint each at the file the code moved to:\n ${dead.join('\n ')}`); + }); + } +}); diff --git a/test/supply-lock-claims.test.js b/test/supply-lock-claims.test.js index 39cc1c72..f81abe81 100644 --- a/test/supply-lock-claims.test.js +++ b/test/supply-lock-claims.test.js @@ -26,6 +26,17 @@ * the separate LOCK_MINT_SUPPLY flag, and LOCK_MINT is consulted nowhere * in the ISSUE handler, so an owner can still create supply with * LOCK_MINT set and LOCK_MINT_SUPPLY unset. + * 3. MAX_SUPPLY = 0 was described as plain "unlimited supply". That reading + * is the UNCAPPED_MAX_SUPPLY_ZERO protocol change, whose MAINNET arm sits + * on the unarmed sentinel 9999999999 while testnet and regtest arm at + * genesis. Below the gate mint.js compares SUPPLY + AMOUNT against a zero + * ceiling, so every positive mint on a zero-cap mainnet token is refused + * and such a token can never issue anything. + * 4. LOCK_CALLBACK was described as proving the recall terms cannot be + * altered. callback.js refuses the action outright when the flag is set, + * so the flag does not preserve a usable recall, it removes recall + * entirely. It was the only lock in the list not phrased as a + * prohibition, which is how the consequence went unstated. * * WHAT IT CHECKS. Both sides, because either one alone is a half guard: * @@ -38,19 +49,27 @@ * * The prose side runs unconditionally; only the source side skips when the * sibling checkout is absent, so the guard can never come back all-skip. + * + * XCHAIN_DOCS_ROOT overrides the docs root, matching + * settlement-and-delivery-claims.test.js. It exists so the negative control is + * runnable: point it at a checkout of an older commit and the prose assertions + * below go red, which is how they were verified to be capable of failing. */ const test = require('node:test'); const assert = require('node:assert'); const fs = require('node:fs'); const path = require('node:path'); +const { readModuleSource } = require('../lib/indexer-source.js'); // an entry plus every part it was split into +const { sibling } = require('./helpers/sibling_checkout.js'); -const DOC_ROOT = path.join(__dirname, '..'); -const INDEXER = path.resolve(DOC_ROOT, '../xchain-indexer/src'); +const DOC_ROOT = process.env.XCHAIN_DOCS_ROOT || path.join(__dirname, '..'); +const INDEXER = path.resolve(path.join(__dirname, '..'), '../xchain-indexer/src'); const GUIDE = path.join(DOC_ROOT, 'user-guide', 'creating-tokens.md'); +const USECASES = path.join(DOC_ROOT, 'user-guide', 'use-cases.md'); -const haveIndexer = fs.existsSync(path.join(INDEXER, 'actions', 'mint.js')); -const readSrc = (rel) => fs.readFileSync(path.join(INDEXER, rel), 'utf8'); +const readSrc = (rel) => readModuleSource(path.join(INDEXER, rel)); const guide = fs.readFileSync(GUIDE, 'utf8'); +const useCases = fs.readFileSync(USECASES, 'utf8'); // Slice a markdown section by its heading, up to the next heading of any depth. function section(md, heading){ @@ -67,20 +86,31 @@ function section(md, heading){ const supplySection = section(guide, '### Supply'); const lockSection = section(guide, '## Building Trust: Locking Parameters'); const lockMintBullet = lockSection.split('\n').find((l) => l.startsWith('- **LOCK_MINT**:')); +const lockCallbackBullet = lockSection.split('\n').find((l) => l.includes('LOCK_CALLBACK')); -const skipNoIndexer = !haveIndexer && 'sibling xchain-indexer not present in this checkout'; +/* Skips by name on a bare clone. A run that declared the sibling supplied + * (XCHAIN_REQUIRE_SIBLINGS=1, which bin/ci-all.sh and the venue set, with xchain-indexer + * in .ci-siblings) throws in the helper instead, so a dropped checkout cannot leave the + * source half uncompared while the file still reports green. Both spellings of the entry + * are tried, since the indexer's split convention moves `mint.js` to `mint/index.js`. */ +const skipNoIndexer = sibling('xchain-indexer', + [[path.join(INDEXER, 'actions', 'mint.js'), path.join(INDEXER, 'actions', 'mint', 'index.js')]]).skip; test('the source facts the supply wording rests on still hold', { skip: skipNoIndexer }, () => { const mint = readSrc('actions/mint.js'); - const db = readSrc('db.js'); + // getTokenSupply moved out of the monolithic src/db.js into the credits mixin when + // the database layer was split. Read that one mixin rather than the whole src/db/ + // tree: a match anywhere in 59 files would not prove the ledger formula still lives + // in the function the guide's "outstanding at one time" wording rests on. + const credits = readSrc('db/credits.js'); const destroy = readSrc('actions/destroy.js'); assert.match(mint, /bcadd\(data\['SUPPLY'\],data\['AMOUNT'\]/, 'mint.js no longer compares SUPPLY + AMOUNT against the ceiling; the guide\'s ' + '"outstanding at one time" wording may need to change back'); assert.match(mint, /MAX_SUPPLY/, 'mint.js no longer names MAX_SUPPLY'); - assert.match(db, /bcadd\(this\.util\.bcsub\(credits, debits, exact\), escrows, decimals\)/, - 'db.js getTokenSupply no longer computes supply as credits - debits + escrows, ' + assert.match(credits, /bcadd\(this\.util\.bcsub\(credits, debits, exact\), escrows, decimals\)/, + 'db/credits.js getTokenSupply no longer computes supply as credits - debits + escrows, ' + 'so burning may no longer return mint headroom'); assert.match(destroy, /debits\.push\(\[destroy\['TICK'\], destroy\['AMOUNT'\], destroy\['SOURCE'\]\]\)/, 'destroy.js no longer debits the burned amount, so DESTROY may no longer lower supply'); @@ -124,3 +154,68 @@ test('the LOCK_MINT bullet scopes itself to the MINT command and names the compa 'the LOCK_MINT bullet does not name LOCK_MINT_SUPPLY, so a reader is not told that ' + 'the issuer path stays open'); }); + +test('use-cases.md does not promise permanently closed issuance', () => { + for(const phrase of ['no one (including you) can ever create more', + 'no more tokens can be created']){ + assert.ok(!useCases.includes(phrase), + `user-guide/use-cases.md states "${phrase}". Locking the ceiling or closing the ` + + 'public mint window closes neither issuance path for good: DESTROY frees headroom ' + + 'under the cap and issue.js still credits MINT_SUPPLY unless LOCK_MINT_SUPPLY is ' + + 'set. Permanent closure needs LOCK_MINT and LOCK_MINT_SUPPLY together.'); + } + assert.match(useCases, /LOCK_MINT_SUPPLY/, + 'user-guide/use-cases.md never names LOCK_MINT_SUPPLY, so its supply-permanence ' + + 'passages do not tell the reader what actually closes the issuer path'); +}); + +test('the source facts the zero-max-supply wording rests on still hold', { skip: skipNoIndexer }, () => { + const changes = readSrc('protocol_changes.js'); + const mint = readSrc('actions/mint.js'); + + assert.match(changes, /UNCAPPED_MAX_SUPPLY_ZERO_MAINNET_TIME\s*=\s*9999999999/, + 'protocol_changes.js no longer parks UNCAPPED_MAX_SUPPLY_ZERO_MAINNET_TIME on the ' + + 'unarmed 9999999999 sentinel. If the operator has named the mainnet launch instant, ' + + 'the "### Supply" wording in user-guide/creating-tokens.md must be updated in the ' + + 'same commit and this assertion retired: zero max supply then means uncapped on ' + + 'mainnet too, and the guide must stop telling mainnet issuers to avoid it.'); + assert.match(mint, /isEnabled\('UNCAPPED_MAX_SUPPLY_ZERO'/, + 'mint.js no longer resolves the uncapped exemption through ' + + 'isEnabled(\'UNCAPPED_MAX_SUPPLY_ZERO\'), so the guide\'s gate-conditional wording ' + + 'no longer describes how a zero ceiling is treated'); +}); + +test('the Supply section qualifies the zero-max-supply sentinel by network', () => { + assert.match(supplySection, /mainnet/i, + 'creating-tokens.md "### Supply" describes a zero max supply without naming mainnet. ' + + 'The uncapped reading is live on testnet/regtest only; on mainnet the gate is ' + + 'unarmed and mint.js refuses every positive mint against a zero ceiling.'); + assert.match(supplySection, /protocol-activation\.md/, + 'creating-tokens.md "### Supply" no longer links protocol/protocol-activation.md, so a ' + + 'reader cannot look up when the uncapped reading turns on for mainnet'); + assert.ok(!/zero means the supply is unlimited/.test(supplySection), + 'creating-tokens.md "### Supply" states unconditionally that a max supply of zero means ' + + 'unlimited supply. That is true on testnet and regtest only while the mainnet arm of ' + + 'UNCAPPED_MAX_SUPPLY_ZERO sits on its sentinel.'); +}); + +test('the source fact the LOCK_CALLBACK wording rests on still holds', { skip: skipNoIndexer }, () => { + const callback = readSrc('actions/callback.js'); + + assert.match(callback, /tokenInfo\['LOCK_CALLBACK'\]==1/, + 'callback.js no longer refuses the CALLBACK action when LOCK_CALLBACK is set, so the ' + + 'guide\'s "recall becomes impossible" wording may no longer be accurate'); + assert.match(callback, /invalid: LOCK_CALLBACK/, + 'callback.js no longer emits the invalid: LOCK_CALLBACK refusal'); +}); + +test('the LOCK_CALLBACK bullet is phrased as a prohibition, not an assurance', () => { + assert.ok(lockCallbackBullet, 'creating-tokens.md no longer carries a LOCK_CALLBACK bullet'); + assert.ok(!lockCallbackBullet.includes('proves the recall terms cannot be altered'), + 'the LOCK_CALLBACK bullet still frames the flag as proving the recall terms are fixed. ' + + 'callback.js refuses CALLBACK outright when the flag is set, so recall becomes ' + + 'impossible rather than fixed.'); + assert.match(lockCallbackBullet, /never|impossible/i, + 'the LOCK_CALLBACK bullet does not tell the issuer that recall becomes impossible, ' + + 'which is the irreversible consequence of setting it'); +}); diff --git a/test/tis-schema-field-coverage.test.js b/test/tis-schema-field-coverage.test.js index 67eee371..91e4531f 100644 --- a/test/tis-schema-field-coverage.test.js +++ b/test/tis-schema-field-coverage.test.js @@ -27,8 +27,16 @@ * the images/audio/video/files definitions). * 2. Every character bound the prose states equals the schema's maxLength. * 3. The worked example parses and uses no key the schema does not declare. - * 4. v1.0.0 stays frozen: still stamped 1.0.0 and still without the gating - * fields, so drift is never "fixed" by rewriting a published version. + * 4. v1.0.0 and v1.1.0 stay frozen: each still stamped with its own version, + * v1.0.0 still without the gating fields and v1.1.0 still carrying the + * `["type", "data"]` media requirement v1.1.1 relaxed, so drift is never + * "fixed" by rewriting a published version. + * 5. A media entry carrying only `data_ref` satisfies the CURRENT schema's + * media requirement, and an entry carrying neither `data` nor `data_ref` + * still fails it. The pair runs the requirement rather than asserting it: + * v1.1.0 required `data` outright and so rejected the fully on-chain form + * the prose recommends, and a one-sided check would have passed on the + * relaxed schema and on a schema that required nothing at all. * * FLOORS, BECAUSE A PARSER THAT MATCHES NOTHING READS AS GREEN. A markdown * table lint that silently stops matching passes forever. The row floors below @@ -47,7 +55,7 @@ const DOC_ROOT = path.join(__dirname, '..'); const SPEC = path.join(DOC_ROOT, 'protocol/token-information-standard.md'); const JSON_DIR = path.join(DOC_ROOT, 'protocol/json'); -const CURRENT = '1.1.0'; +const CURRENT = '1.1.1'; const MEDIA = ['images', 'audio', 'video', 'files']; // A row is `| field | Type | Description`. The header and the `| :--- |` @@ -205,6 +213,52 @@ describe('TIS field table / schema coverage', () => { `v1.0.0 ${def}.${field} appeared; publish v${CURRENT} instead`); }); + test('v1.1.0 stays frozen at what it published', () => { + const published = readJson('token-information-standard-v1.1.0-schema.json'); + assert.equal(published.version, '1.1.0'); + for (const def of MEDIA) + assert.deepEqual(published.definitions[def].required, ['type', 'data'], + `v1.1.0 ${def}.required moved; a published version is superseded, never edited`); + }); + + test('a data_ref-only media entry satisfies the current schema, and an entry with neither does not', () => { + // Runs the requirement the way a validator does: every name in `required` + // must be present, and when the definition carries an `anyOf` of required + // clauses at least one branch must also be satisfied. + const satisfies = (def, entry) => { + for (const name of def.required || []) + if (!(name in entry)) return false; + if (!Array.isArray(def.anyOf)) return true; + return def.anyOf.some((branch) => + (branch.required || []).every((name) => name in entry)); + }; + + const frozen = readJson('token-information-standard-v1.1.0-schema.json'); + for (const name of MEDIA) { + const def = schema.definitions[name]; + const type = (def.properties.type.enum || ['other'])[0]; + + assert.equal(satisfies(def, { type, data_ref: 'action:12345' }), true, + `v${CURRENT} ${name} rejects a data_ref-only entry, which is the fully ` + + 'on-chain form this standard recommends'); + assert.equal(satisfies(def, { type, data: 'https://domain.com/f' }), true, + `v${CURRENT} ${name} rejects a data-only entry, which every published ` + + 'document uses'); + assert.equal(satisfies(def, { type }), false, + `v${CURRENT} ${name} accepts an entry carrying neither data nor data_ref, ` + + 'so the requirement is gone rather than relaxed'); + assert.equal(satisfies(def, { data_ref: 'action:12345' }), false, + `v${CURRENT} ${name} accepts an entry with no type`); + + // The negative control for the checker itself: the same data_ref-only + // entry must still fail against the frozen v1.1.0 definition, which is + // the defect v1.1.1 exists to fix. + assert.equal(satisfies(frozen.definitions[name], { type, data_ref: 'action:12345' }), false, + `v1.1.0 ${name} accepts a data_ref-only entry, so this checker cannot ` + + 'tell the relaxed schema from the frozen one'); + } + }); + test(`the current schema and example are stamped v${CURRENT}`, () => { assert.equal(schema.version, CURRENT); assert.ok(SPEC_TEXT.includes(`token-information-standard-v${CURRENT}-schema.json`), diff --git a/test/wallet-signer-surface.test.js b/test/wallet-signer-surface.test.js index 4f026b1c..f619c536 100644 --- a/test/wallet-signer-surface.test.js +++ b/test/wallet-signer-surface.test.js @@ -46,12 +46,14 @@ const assert = require('node:assert/strict'); const { test, describe } = require('node:test'); const fs = require('node:fs'); const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); const DOC_ROOT = path.join(__dirname, '..'); const WALLET = path.resolve(DOC_ROOT, '../xchain-wallet'); const SIGNER_SRC = path.join(WALLET, 'packages/core/src/signers/Signer.js'); const ARCH_DOC = path.join(DOC_ROOT, 'components/wallet/architecture.md'); -const haveWallet = fs.existsSync(SIGNER_SRC); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the signer is unreadable. +const noWallet = sibling('xchain-wallet', [SIGNER_SRC]).skip; // A member-position `name(` and a member-position `if (` are the same shape, so // shape alone cannot tell them apart. Refuse the keywords that can legally sit @@ -106,7 +108,7 @@ function concreteSignerClasses() { describe('wallet signer surface', () => { test('every Signer base-class method is listed in architecture.md', - { skip: !haveWallet && 'xchain-wallet not present in this checkout' }, () => { + { skip: noWallet }, () => { const doc = fs.readFileSync(ARCH_DOC, 'utf8'); const methods = signerMethods(); @@ -121,7 +123,7 @@ describe('wallet signer surface', () => { }); test('no doc presents a signer class that does not exist', - { skip: !haveWallet && 'xchain-wallet not present in this checkout' }, () => { + { skip: noWallet }, () => { const real = concreteSignerClasses(); assert.ok(real.size >= 4, 'found only ' + real.size + ' signer classes; the scan is probably broken'); @@ -169,7 +171,7 @@ describe('wallet signer surface', () => { }); test('every spelled-out concrete-signer count matches the code', - { skip: !haveWallet && 'xchain-wallet not present in this checkout' }, () => { + { skip: noWallet }, () => { // Count from source only: the extension's built bundles re-declare the // same classes, and a dist copy is not a fifth signer. diff --git a/test/xbridge-action-registration.test.js b/test/xbridge-action-registration.test.js new file mode 100644 index 00000000..c44ad203 --- /dev/null +++ b/test/xbridge-action-registration.test.js @@ -0,0 +1,62 @@ +/********************************************************************* + * + * Copyright © 2025–2026 Dankest, LLC + * Based on XChain Platform by Dankest, LLC – https://dankest.llc + * + * SPDX-License-Identifier: AGPL-3.0-or-later + * + * This file is part of XChain Platform. Licensed under the GNU Affero + * General Public License v3.0 or later; see LICENSE.md. + * + ********************************************************************** + * + * L9b follow-on guard: XBRIDGE landed in protocol/action-manifest.json this + * wave but was missing from three places the wider suite reads. This file + * pins the fixes narrowly, so a future edit that removes one of them fails + * loudly here instead of only in the broader (and much slower) test files + * this guard is a companion to. + */ +const assert = require('node:assert/strict'); +const { test, describe } = require('node:test'); +const fs = require('node:fs'); +const path = require('node:path'); +const { sibling } = require('./helpers/sibling_checkout.js'); + +const DOC_ROOT = path.join(__dirname, '..'); +const REGISTRY = path.resolve(DOC_ROOT, '../xchain-indexer/src/protocol_changes.js'); +// Skips by name on a bare clone; throws under XCHAIN_REQUIRE_SIBLINGS=1 when the registry is unreadable. +const indexer = sibling('xchain-indexer', [REGISTRY]); + +describe('XBRIDGE action registration (L9b follow-on)', () => { + + test('protocol_changes.js registers XBRIDGE at all-zero columns', + { skip: indexer.skip }, () => { + // Read through the class, not a text scan: the row is an array + // literal in a part file under src/protocol_changes/ now, and this + // guard asserts the columns' VALUES, which the class already parsed. + const ProtocolChanges = require(REGISTRY); + const stub = { config: {}, util: { throwError(message) { throw new Error(message); } } }; + const row = new ProtocolChanges(stub).changes.XBRIDGE; + assert.ok(row, 'XBRIDGE is not registered in the protocol_changes registry'); + const thresholds = [row.mainnet_time, row.testnet_time, row.regtest_time, + row.mainnet_block, row.testnet_block, row.regtest_block].map(String); + assert.deepEqual(thresholds, ['0', '0', '0', '0', '0', '0'], + 'XBRIDGE must be registered at all-zero columns like every other ACTION; ' + + 'its real height gates are XCHAIN_BRIDGE_ACTIVATION / TOKEN_BRIDGE_ACTIVATION, ' + + 'not this registry (the ROLLCALL precedent)'); + }); + + test('concepts/actions.md documents an XBRIDGE row', () => { + const text = fs.readFileSync(path.join(DOC_ROOT, 'concepts/actions.md'), 'utf8'); + assert.match(text, /`XBRIDGE`/, 'concepts/actions.md must name XBRIDGE'); + assert.match(text, /XCHAIN_BRIDGE_ACTIVATION/, + 'concepts/actions.md must name the XBRIDGE activation gate'); + }); + + test('the ACTION count claims were bumped from 37/31 to 38/32', () => { + const overview = fs.readFileSync(path.join(DOC_ROOT, 'overview.md'), 'utf8'); + assert.match(overview, /38 standard ACTIONs/, 'overview.md count claim is stale'); + const sdkActions = fs.readFileSync(path.join(DOC_ROOT, 'components/sdk/actions.md'), 'utf8'); + assert.match(sdkActions, /32 ACTION types/, 'components/sdk/actions.md count claim is stale'); + }); +}); diff --git a/user-guide/creating-tokens.md b/user-guide/creating-tokens.md index 8ded7f72..ca4f847c 100644 --- a/user-guide/creating-tokens.md +++ b/user-guide/creating-tokens.md @@ -47,7 +47,7 @@ When you create a token, you configure a set of properties that define how it be **Max Supply** is the ceiling on how many tokens can be outstanding at one time. Every mint is checked against the current supply plus the amount being minted, so while supply sits at the ceiling, further minting is refused. Destroying tokens lowers the current supply and frees that much headroom again, so a max supply is not a limit on how much can be issued over a token's lifetime. Think of it as a tank with a fixed capacity rather than a mine with a finite amount of ore: draining it makes room to refill. To close issuance for good, lock the minting paths (see Locking below) rather than relying on the ceiling alone. -Setting a max supply of zero means the supply is unlimited, which is appropriate for some use cases (like reward points that grow over time) but not others (like collectibles where scarcity matters). +Setting a max supply of zero is the protocol's "uncapped" sentinel: no ceiling at all, which is appropriate for some use cases (like reward points that grow over time) but not others (like collectibles where scarcity matters). That reading is in force on testnet and regtest today. On mainnet it switches on at the network's launch instant and not before, so a mainnet token created now with a max supply of zero refuses every mint and every positive mint supply, and can never issue anything. Until then, give a mainnet token a positive ceiling; you can raise it later unless you set `LOCK_MAX_SUPPLY`. See [Protocol Activation](../protocol/protocol-activation.md) for how the switch-on works. ### Decimals @@ -134,7 +134,7 @@ Parameters you can lock include: - **LOCK_MAX_MINT**: the `MAX_MINT` per-transaction amount cap is frozen permanently and can never be edited again - **LOCK_DESCRIPTION**: proves the token's description cannot be swapped out - **LOCK_SLEEP**: the token can never be paused by the SLEEP command; useful for tokens that must always be tradeable -- **Callback settings** (`LOCK_CALLBACK`): proves the recall terms cannot be altered after the fact +- **LOCK_CALLBACK**: the `CALLBACK` command can never be run against this token again. It does not preserve the recall terms for later use; it makes recall impossible. Do not set it if you may ever need to recall, revoke or settle the token No single flag forecloses all supply creation. Set **LOCK_MINT** and **LOCK_MINT_SUPPLY** together to close both issuance paths, and add **LOCK_MAX_SUPPLY** if you also want the ceiling itself frozen. @@ -175,7 +175,7 @@ Once your token exists, you can: - **Update** any parameters you did not lock - **List** it on the built-in exchange (see the Trading guide) - **Airdrop** it to a list of addresses at once -- **Pay dividends** to all holders proportionally +- **Pay dividends** to eligible holders proportionally (see [DIVIDEND](../protocol/actions/dividend.md) for who is left out) - **Sleep** it temporarily to pause all trading - **Callback** (recall) tokens from all holders if you configured a callback at creation - **Bind a controller** to hand enforcement of transfers, trades, mints, burns, staking, or ownership changes to a contract you deploy, and drop it later subject to the cooldown you set diff --git a/user-guide/use-cases.md b/user-guide/use-cases.md index 9d519dab..0f244de6 100644 --- a/user-guide/use-cases.md +++ b/user-guide/use-cases.md @@ -11,7 +11,7 @@ XChain is a general-purpose token protocol. It does not prescribe what tokens ar ### Limited-Edition Collectible Tokens -Create a token with a fixed, locked supply (say, exactly 100 units) and distribute them to collectors. Because the max supply is locked on-chain, no one (including you) can ever create more. Buyers can verify the scarcity themselves without trusting your promises. +Create a token with a fixed, locked supply (say, exactly 100 units) and distribute them to collectors. Locking the max supply on-chain (`LOCK_MAX_SUPPLY`) caps how many can be held at once and freezes that ceiling, but it does not on its own close issuance: burning tokens frees headroom under the cap, and both minting paths stay open. To make "no one, including you, can ever create more" literally true, also set `LOCK_MINT` (which closes the public `MINT` command) and `LOCK_MINT_SUPPLY` (which closes the issuer's own `MINT_SUPPLY` on a re-issue). With all three set, buyers can verify the scarcity themselves without trusting your promises. XChain actions involved: ISSUE (to create and lock the supply), SEND (to distribute to collectors). @@ -61,13 +61,13 @@ XChain actions involved: ISSUE, LIST, SEND. ### Revenue Sharing and Dividends -If your token has multiple holders and you want to pay them proportionally (like distributing profits to shareholders) the DIVIDEND action does this automatically. You specify the token representing shares, the payment token (which could be XCHAIN or any other token), and the amount per unit. Every holder receives their proportional cut in a single transaction. +If your token has multiple holders and you want to pay them proportionally (like distributing profits to shareholders) the DIVIDEND action does this automatically. You specify the token representing shares, the payment token (which could be XCHAIN or any other token), and the amount per unit. Every eligible holder receives their proportional cut in a single transaction. Three groups are left out: the paying address does not pay a dividend to itself; each share is rounded **down** to the payment token's smallest unit, so a holder whose share rounds to zero receives nothing; and if the payment token carries an allow list or a block list, only addresses that pass it are paid. Holders who are left out do not count toward the per-recipient fee. See [DIVIDEND](../protocol/actions/dividend.md) for the exact rules. XChain actions involved: ISSUE (to create the share token), DIVIDEND (to make distributions), SEND (for ongoing transfers). ### Public Distribution Tokens with Minting Windows -Set up a token with a defined minting window (a start block and a stop block) during which the public can mint the token themselves. Each minter chooses how much to mint in a given transaction, bounded by the caps you set on the token: an optional per-transaction cap (`MAX_MINT`), an optional per-address cap (`MINT_ADDRESS_MAX`), and the token's overall `MAX_SUPPLY`. This is a permissionless, on-chain distribution mechanism: anyone can mint directly from the blockchain during the window, with no intermediary and no allocation list. MINT carries no payment field and moves no funds to the issuer; it only creates the new supply and credits it to the minter. After the window closes or `MAX_SUPPLY` is reached, no more tokens can be created. +Set up a token with a defined minting window (a start block and a stop block) during which the public can mint the token themselves. Each minter chooses how much to mint in a given transaction, bounded by the caps you set on the token: an optional per-transaction cap (`MAX_MINT`), an optional per-address cap (`MINT_ADDRESS_MAX`), and the token's overall `MAX_SUPPLY`. This is a permissionless, on-chain distribution mechanism: anyone can mint directly from the blockchain during the window, with no intermediary and no allocation list. MINT carries no payment field and moves no funds to the issuer; it only creates the new supply and credits it to the minter. After the window closes or `MAX_SUPPLY` is reached, the public can mint no more for now. That is not a permanent close: burning tokens frees headroom under the cap, and as the issuer you can still add supply with `MINT_SUPPLY` on a re-issue unless `LOCK_MINT_SUPPLY` is set. Setting `LOCK_MINT` and `LOCK_MINT_SUPPLY` together is what closes both issuance paths for good. XChain actions involved: ISSUE (with mint window configuration), MINT (by the public during the window). diff --git a/whitepaper.md b/whitepaper.md index f6b26ccc..63e262e7 100644 --- a/whitepaper.md +++ b/whitepaper.md @@ -15,7 +15,7 @@ ## Abstract -XChain is a token-and-settlement **metalayer** for UTXO blockchains. It embeds a complete digital-asset protocol (tokens, a native decentralized exchange, trustless cross-chain swaps, on-chain data and messaging, and a deterministic smart-contract virtual machine) inside ordinary transactions on an unmodified base chain, so that every asset and every state transition is secured directly by the host chain's existing proof-of-work consensus. There are no sidechains, no bridges, and no new consensus layer to trust. +XChain is a token-and-settlement **metalayer** for UTXO blockchains. It embeds a complete digital-asset protocol (tokens, a native decentralized exchange, trustless cross-chain swaps, a cross-chain token bridge, on-chain data and messaging, and a deterministic smart-contract virtual machine) inside ordinary transactions on an unmodified base chain, so that every asset and every state transition is secured directly by the host chain's existing proof-of-work consensus. There is no sidechain and no new consensus layer to trust; cross-chain trading settles without a bridge, and the one bridge the platform runs moves only its own fee token between chains. The protocol is chain-agnostic by construction. It is deployed and running on **Bitcoin, Litecoin, and Dogecoin**, on mainnet as well as testnet, with the XCHAIN distribution and the protocol freeze still ahead of it (§13.3, §16); adding any further UTXO chain is a configuration change rather than a protocol change, and the platform is designed to extend toward a broad set of blockchains over time. The same protocol, the same ACTION set, and the same tooling operate identically across every supported chain. @@ -37,7 +37,7 @@ XChain takes the opposite path. It is a **metalayer**: a protocol layered *above Three commitments run through every layer of the system. -**Inherited security.** XChain introduces no new chain, no new consensus for transaction ordering, and no bridge. Finality, ordering, and double-spend resistance come entirely from the host chain. The validator network described in §10 exists only for configuration, price data, cross-chain coordination, and attestation, never for ordering or settling token state. +**Inherited security.** XChain introduces no new chain and no new consensus for transaction ordering; finality, ordering, and double-spend resistance come entirely from the host chain, and no bridge sits in that path. The validator network described in §10 exists for configuration, price data, cross-chain coordination, attestation, and the lock-and-mint bridge that moves the platform's own fee token between chains; none of that authority extends to ordering or settling base-layer token state on any host chain. **Determinism.** Every node that processes the same base-chain data computes byte-identical state. There is no randomness, no wall-clock-dependent branching, and no un-replayable external input anywhere in state processing. This is the property that makes the system independently verifiable: anyone can run the software, replay the chain from genesis, and confirm every balance for themselves. @@ -98,7 +98,7 @@ XChain is a pipeline of independent services, each runnable separately and most | **utxo-tracker** | Indexes every transaction output from a coin node; serves address balances and spendable UTXOs | LevelDB | | **encoder** | Stateless; turns an ACTION string plus UTXOs plus pubkey into an unsigned PSBT | none | | **decoder** | Polls a coin node, extracts and de-obfuscates XChain transactions from blocks, writes raw decoded data | MariaDB (decoder DB) | -| **indexer** | Reads the decoder DB, validates and applies ACTION logic, runs the VM, maintains the ledger | MariaDB (indexer DB) | +| **indexer** | Reads the decoder DB and a local read-only hub mirror, validates and applies ACTION logic, runs the VM, maintains the ledger | MariaDB (indexer DB, plus the local hub mirror) | | **explorer** | Stateless REST plus JSON-RPC plus WebSocket plus web UI over the indexer DB | none | | **hub** | Decentralized config oracle, price oracle, cross-chain coordinator, attestation engine, governance | MariaDB (hub DB) | | **sync** | Replicates the decoder and indexer DBs to validators via REST snapshots and WebSocket streaming | none | @@ -139,9 +139,9 @@ The pipeline is strictly unidirectional: raw data enters at the decoder, is prom The separation of *extraction* (decoder) from *interpretation* (indexer) is deliberate and yields three properties the protocol depends on: -- **Replay.** The indexer DB is a pure function of the decoder DB. Destroy it and re-run the indexer against the same decoder DB and you obtain bit-for-bit identical state. -- **Independent verification.** Multiple indexers reading the same decoder DB converge to identical state, including identical per-block integrity hashes (§5.5). -- **Auditability.** Any balance traces through ledger entries to an exact block and action; the decoder DB itself is reproducible from the raw chain, so the entire indexer state is ultimately derivable from the blockchain alone. +- **Replay.** The indexer DB is a pure function of the decoder DB and the local hub-mirror tables the indexer reads during block processing. Destroy it and re-run the indexer against the same decoder DB and an equivalent hub mirror and you obtain bit-for-bit identical state. +- **Independent verification.** Multiple indexers reading the same decoder DB and the same hub-mirrored rows converge to identical state, including identical per-block integrity hashes (§5.5). +- **Auditability.** Any balance traces through ledger entries to an exact block and action; the decoder DB itself is reproducible from the raw chain and the hub-mirrored rows are chain-derived as well (PRICE actions, plus validator stake and reward tables synced from BTC indexer state, aggregated across chains by the hub), so the entire indexer state is ultimately derivable from the chains. --- @@ -172,7 +172,7 @@ The encoder measures the obfuscated payload and selects a format that fits. The | Format | Per-output data capacity | Txs | Mechanism | Notes | |---|---|---|---|---| | **OP_RETURN** | 80 bytes total (incl. 4-byte prefix) | 1 | Data in a provably-unspendable output | No UTXO bloat; the common case | -| **Bare multisig** | 60 data bytes per key slot | 1 | Payload packed into fake pubkey slots of an `m-of-n` multisig | Single-tx flow for medium payloads; leaves a spendable (dust) output | +| **Bare multisig** | 60 data bytes per output (two 32-byte key slots) | 1 | Payload packed into fake pubkey slots of an `m-of-n` multisig | Single-tx flow for medium payloads; leaves a spendable (dust) output | | **P2SH** | 476 bytes per chunk, many outputs | 2 | Fund a script hash, then reveal the redeem script in the spend's scriptSig | Fund must reach mempool before the spend is valid | | **P2WSH** | 476 bytes per chunk, many outputs | 2 | Fund a witness script hash, then reveal the witness script | SegWit witness discount makes this the most fee-efficient chunked format | | **Taproot envelope** | up to 390,000 bytes in one witness | 2 | Commit to a P2TR output whose script tree holds one data leaf; reveal it through the script path | BTC and LTC only (DOGE has no SegWit); the large-payload carrier | @@ -279,13 +279,13 @@ All database writes for a block commit inside one MariaDB transaction: the whole token_supply == SUM(credits) - SUM(debits) ``` -A mismatch is a fatal violation: the transaction rolls back and the indexer halts rather than persist inconsistent state. On a host-chain reorganization, the decoder detects the divergent block hash, records the fork point, and the indexer rolls back all affected tables atomically (deleting rows at or above the fork's first action index), recomputes balances from the remaining ledger, and re-indexes the canonical fork. The utxo-tracker keeps a per-chain reorg undo window (default BTC 12, LTC 48, DOGE 120 blocks, env-overridable) for the same purpose. +A mismatch is a fatal violation: the transaction rolls back and the indexer halts rather than persist inconsistent state. On a host-chain reorganization, the decoder detects the divergent block hash, records the fork point, and the indexer rolls back all affected tables atomically (deleting rows at or above the fork's first action index), recomputes balances from the remaining ledger, and re-indexes the canonical fork. The utxo-tracker keeps a per-chain, per-network reorg undo window (mainnet/regtest default BTC 12, LTC 120, DOGE 120 blocks; testnet 120 for every coin; env-overridable) for the same purpose. --- ## 6. The ACTION Set -The protocol defines 37 named ACTIONs across ten categories. Of these, 31 are user-submittable (all except ANCHOR, ATTEST, NODEPROOF, ROLLCALL, SLASH, and XCALL); the remaining six are validator-broadcast, VM-emitted, or permissionless-proof actions described where relevant, along with system-synthesized actions (order/swap matching and expiry, betting market close and expiry, cross-chain settlement, XCALL relay, and so on). Thirty-six of the 37 are decoded from a wire transaction; XCALL alone is mirror-injected into the destination chain's index instead. All user ACTIONs are available on every supported chain unless noted. A `^`-prefixed ticker field passes a numeric token id instead of a name. +The protocol defines 38 named ACTIONs across ten categories. Of these, 32 are user-submittable (all except ANCHOR, ATTEST, NODEPROOF, ROLLCALL, SLASH, and XCALL); the remaining six are validator-broadcast, VM-emitted, or permissionless-proof actions described where relevant, along with system-synthesized actions (order/swap matching and expiry, betting market close and expiry, cross-chain settlement, XCALL relay, and so on). Thirty-seven of the 38 are decoded from a wire transaction; XCALL alone is mirror-injected into the destination chain's index instead. All user ACTIONs are available on every supported chain unless noted. A `^`-prefixed ticker field passes a numeric token id instead of a name. ### 6.1 Token lifecycle @@ -747,7 +747,7 @@ XChain demonstrates that a complete digital-asset platform, including tokens, an | Stake-weighted quorum (at/above `STAKE_WEIGHTED_QUORUM_ACTIVATION`; gated on the validator-era batch, §10.2) | combined signer stake, deduplicated by stake SOURCE, > 2/3 of total active stake | | Trimmed-median trim | top/bottom 15% | | Governance | 7-day vote, 50% quorum, two-thirds approval, 14-day re-proposal cooldown | -| utxo-tracker reorg undo window | BTC 12 / LTC 48 / DOGE 120 blocks (default, env-overridable) | +| utxo-tracker reorg undo window | mainnet/regtest BTC 12 / LTC 120 / DOGE 120; testnet 120 for every coin (default, env-overridable) | | Capability stake activation / cooldown | ~6 BTC blocks / 1,000 blocks (governance-set) | | XCHAIN supply | 100,000,000 (8 decimals), capped at genesis, zero pre-mint, BTC-chain only | | XCHAIN genesis distribution (§13.3; **pre-launch, not final**) | 30% holder airdrop / 25% open mint / 20% treasury / 10% liquidity / 9.7% validators / 5.3% reward pool / 0% team |