|
| 1 | +# Plan: Port all portable example programs to local light-sdk + light-client |
| 2 | + |
| 3 | +**Date:** 2026-06-23 |
| 4 | + |
| 5 | +## IMPORTANT - Autonomous Execution Mode |
| 6 | + |
| 7 | +- This plan executes without user intervention (mode-auto active). |
| 8 | +- All questions resolved: proceed even though the test's start-time `light test-validator --stop` will kill any user-run prover on :3001 (the test starts its own stack). |
| 9 | +- Use subagents for parallel porting (max 5 concurrent), spawned so they never prompt. |
| 10 | +- If blocked, find an alternative approach; do not stop. Skip-and-note only as last resort. |
| 11 | +- Keep working until ALL examples are ported, compiled, and (Phase 2) tested green. |
| 12 | + |
| 13 | +## IMPORTANT (user instructions & constraints) |
| 14 | + |
| 15 | +- User instruction: "Plan to port all portable ones, use subagents to do it in parallel up to 5 at a time." |
| 16 | +- **Batching is explicitly ALLOWED for this plan** (user requested parallel subagents). This overrides the default no-batching rule. |
| 17 | +- Split the work into todos; work through them one wave at a time; do not collapse waves. |
| 18 | +- Use subagents to do the per-example porting in parallel, max 5 concurrent. |
| 19 | +- If a subagent gets stuck or starts doing random things, it must stop and report rather than improvise; escalate research to another subagent. |
| 20 | +- Test EARLY: subagents must get each example to compile before reporting done. |
| 21 | +- **Hard constraint — validator is a singleton:** the rewritten tests spawn `light test-validator` (ports 8899/8784/3001) and each run calls `--stop` first. Therefore **only one example's test may run at a time.** Subagents do PORT + COMPILE only (parallel-safe). Test EXECUTION is serialized in a separate phase. |
| 22 | +- Do not add Claude as a commit co-author. No emojis. Never index slices with `[i]` in any Rust we write. |
| 23 | + |
| 24 | +## Goal |
| 25 | + |
| 26 | +Switch every portable example from released `light-*` crates + `light-program-test` to: |
| 27 | +- local `~/dev/light/light-sdks` path deps via `[patch.crates-io]`, |
| 28 | +- `light-client` + a real `light test-validator` for integration tests, |
| 29 | +- the anchor-1.0.2 / solana-4.0 dependency stack. |
| 30 | + |
| 31 | +Pinocchio examples (`counter/pinocchio`) are OUT until `light-sdk-pinocchio` is added to the local workspace. |
| 32 | + |
| 33 | +## The canonical recipe (proven on counter/anchor) |
| 34 | + |
| 35 | +### A. Workspace-root `Cargo.toml` (the one with `[workspace]`) |
| 36 | +Add: |
| 37 | +```toml |
| 38 | +[patch.crates-io] |
| 39 | +light-sdk = { path = "/Users/jorrit/dev/light/light-sdks/sdk" } |
| 40 | +light-hasher = { path = "/Users/jorrit/dev/light/light-sdks/hasher" } |
| 41 | +light-client = { path = "/Users/jorrit/dev/light/light-sdks/client" } |
| 42 | +``` |
| 43 | +(Patching these three pulls the whole transitive `light-*` graph from the local workspace.) |
| 44 | + |
| 45 | +### B. Program `Cargo.toml` |
| 46 | +- `anchor-lang = "1.0.2"` (anchor examples only; was 0.31.1). |
| 47 | +- Enable `keccak` on the SDK so address derivation works at runtime: |
| 48 | + `light-sdk = { version = "0.23.0", features = ["anchor", "cpi-context", "keccak"] }` |
| 49 | + (native examples: same but without `"anchor"`). |
| 50 | +- `[dev-dependencies]`: remove `light-program-test`; remove the `solana-sdk` umbrella; use split crates matching the local stack: |
| 51 | + ```toml |
| 52 | + light-client = "0.23.0" |
| 53 | + solana-instruction = "3.4" |
| 54 | + solana-keypair = "3.1.2" |
| 55 | + solana-pubkey = { version = "4.2", features = ["curve25519", "sha2"] } |
| 56 | + solana-signature = "3.4" |
| 57 | + solana-signer = "3.0" |
| 58 | + # solana-keypair -> five8 1.0 -> five8_core 0.1.2 gates `impl Error for DecodeError` behind `std` |
| 59 | + five8_core = { version = "0.1.2", features = ["std"] } |
| 60 | + tokio = "1.49.0" |
| 61 | + ``` |
| 62 | + Keep any other existing dev-deps the test genuinely uses. |
| 63 | + |
| 64 | +### C. Program `src/lib.rs` (anchor only) |
| 65 | +- anchor 1.0 collapsed `Context` to one lifetime: |
| 66 | + `Context<'_, '_, '_, 'info, T>` -> `Context<'info, T>` (replace all). |
| 67 | +- Fix any other anchor-1.0 / borsh-1.x breaks the compiler surfaces (iterate `cargo check`). |
| 68 | + |
| 69 | +### D. Test files (`tests/*.rs`, and native `src/test_helpers.rs`) |
| 70 | +- Swap imports: drop `light_program_test::*` and the `solana_sdk` umbrella. |
| 71 | + Use `light_client::{rpc::{LightClient, LightClientConfig, Rpc, RpcError}, indexer::{Indexer, AddressWithTree, CompressedAccount, TreeInfo, ...}}` |
| 72 | + and the split solana crates (`solana_instruction::Instruction`, `solana_keypair::Keypair`, `solana_signature::Signature`, `solana_signer::Signer`). |
| 73 | +- Replace `LightProgramTest::new(...)` setup with a `start_validator_and_connect()` helper that: |
| 74 | + - builds the `.so` path: `format!("{}/../../target/deploy/<prog>.so", env!("CARGO_MANIFEST_DIR"))` (adjust `../..` to the workspace target dir for that example), |
| 75 | + - runs `light test-validator --stop` (cleanup), then spawns `light test-validator --sbf-program <ID> <so_path>`, |
| 76 | + - connects `LightClient::new(LightClientConfig::local())`, polls `get_slot()` until ready, then sleeps ~10s. |
| 77 | +- Fund a fresh payer: `rpc.airdrop_lamports(&payer.pubkey(), 10_000_000_000).await`. |
| 78 | +- Mark every test `#[tokio::test(flavor = "multi_thread", worker_threads = 4)]` (LightClient wraps the blocking RPC client). |
| 79 | +- Do NOT add a `--stop` at the END of the test (it kills the test's own process group -> SIGKILL). Rely on the next run's start-time `--stop`. |
| 80 | +- Helper fns generic over `R: Rpc + Indexer` need no body changes. |
| 81 | + |
| 82 | +### Reference diff |
| 83 | +`counter/anchor` is the worked example. Subagents should read these files as the template: |
| 84 | +- `counter/anchor/Cargo.toml` (patch block) |
| 85 | +- `counter/anchor/programs/counter/Cargo.toml` (deps) |
| 86 | +- `counter/anchor/programs/counter/src/lib.rs` (Context fix) |
| 87 | +- `counter/anchor/programs/counter/tests/test.rs` (validator+client test) |
| 88 | + |
| 89 | +## Examples to port |
| 90 | + |
| 91 | +### Anchor (10) |
| 92 | +1. `basic-operations/anchor/create` |
| 93 | +2. `basic-operations/anchor/update` |
| 94 | +3. `basic-operations/anchor/burn` |
| 95 | +4. `basic-operations/anchor/close` |
| 96 | +5. `basic-operations/anchor/reinit` |
| 97 | +6. `read-only` |
| 98 | +7. `account-comparison` |
| 99 | +8. `create-and-update` |
| 100 | +9. `zk/nullifier` |
| 101 | +10. `zk/zk-id` (also has a Noir circuit test `tests/circuit.rs` — leave circuit alone, only port the light test) |
| 102 | + |
| 103 | +### Native (7) |
| 104 | +11. `counter/native` |
| 105 | +12. `airdrop-implementations/simple-claim` |
| 106 | +13. `basic-operations/native/create` |
| 107 | +14. `basic-operations/native/update` |
| 108 | +15. `basic-operations/native/burn` |
| 109 | +16. `basic-operations/native/close` |
| 110 | +17. `basic-operations/native/reinit` |
| 111 | + |
| 112 | +Native specifics: no anchor migration; bump `solana-program` to the local 4.0-compatible set; tests build instructions manually (no anchor `InstructionData`/`ToAccountMetas`); a `src/test_helpers.rs` also imports `light_program_test` and must be ported too. |
| 113 | + |
| 114 | +## Execution phases |
| 115 | + |
| 116 | +### Phase 0 — Finalize & prove the recipe on counter (me, sequential) [PREREQUISITE] |
| 117 | +- Apply the `keccak` feature fix to `counter/anchor/programs/counter/Cargo.toml`. |
| 118 | +- Run `cargo test-sbf` for counter against the validator; confirm `test_counter ... ok`. |
| 119 | +- Lock the recipe text above against whatever the run reveals. Only after green do we fan out. |
| 120 | + |
| 121 | +### Phase 1 — Port + compile in parallel (subagents, max 5 concurrent) |
| 122 | +Each subagent ports ONE example per the recipe and must achieve BOTH: |
| 123 | +- `cargo check --manifest-path <program Cargo.toml> --tests --features test-sbf` green (anchor) / appropriate features (native), |
| 124 | +- `cargo build-sbf --manifest-path <program Cargo.toml>` green. |
| 125 | +Subagents MUST NOT run `cargo test-sbf` / spawn a validator (port singleton). Report deviations from the recipe. |
| 126 | + |
| 127 | +- Wave A (5): basic-operations/anchor/{create, update, burn, close, reinit} |
| 128 | +- Wave B (5): read-only, account-comparison, create-and-update, zk/nullifier, counter/native |
| 129 | +- Wave C (5): airdrop-implementations/simple-claim, basic-operations/native/{create, update, burn, close} |
| 130 | +- Wave D (2): basic-operations/native/reinit, zk/zk-id |
| 131 | + |
| 132 | +### Phase 2 — Run tests serially (me, one at a time) |
| 133 | +For each ported example, run `cargo test-sbf ... -- --nocapture`, confirm the test passes, fix runtime issues (most likely the same keccak/feature class). Never two validators at once. |
| 134 | + |
| 135 | +## Acceptance criteria |
| 136 | +- Every listed example: program lib + tests compile against local path deps; `cargo build-sbf` produces a `.so`. |
| 137 | +- Each example's integration test passes against `light test-validator` (Phase 2). |
| 138 | +- No example still references `light-program-test` or the `solana-sdk` umbrella. |
| 139 | +- `[patch.crates-io]` points only at local paths; no released `light-*` crates cross an API boundary. |
| 140 | +- Pinocchio examples untouched (documented as blocked). |
| 141 | + |
| 142 | +## STATUS (2026-06-23) |
| 143 | + |
| 144 | +Phase 0: DONE — counter/anchor ported + `test_counter` PASSED with keccak fix. Recipe locked. |
| 145 | +Phase 1 (port + compile): DONE for all portable examples. `cargo check --tests` + `cargo build-sbf` green for: |
| 146 | +- counter/anchor (also test-passed) |
| 147 | +- basic-operations/anchor/{create,update,burn,close,reinit} |
| 148 | +- account-comparison (test_solana_account.rs kept on LiteSVM, bumped to 0.12), create-and-update |
| 149 | +- read-only, zk/nullifier, zk/zk-id (vendored `read_state_merkle_tree_root`; circuit.rs imports redirected) |
| 150 | +- counter/native, basic-operations/native/{create,update,close,reinit,burn} |
| 151 | + |
| 152 | +BLOCKED / not ported: |
| 153 | +- airdrop-implementations/simple-claim — depends on the compressed-TOKEN stack (light-token, light-compressed-token-sdk, light-token-types, light-token-interface) which is NOT in the local workspace, and local light-client dropped `get_compressed_token_accounts_by_owner`. REVERTED to released deps (left working). |
| 154 | +- counter/pinocchio — needs light-sdk-pinocchio (not local). Untouched. |
| 155 | + |
| 156 | +Phase 2 (run tests): DONE — user authorized; prover killed first. 16 integration-test crates PASS against `light test-validator`: |
| 157 | +- counter/anchor (test_counter) |
| 158 | +- basic-operations/anchor/{create,update,burn,close,reinit} |
| 159 | +- read-only, account-comparison (LightClient + LiteSVM), create-and-update (both files) |
| 160 | +- zk/nullifier (2 tests — after adapting the duplicate-rejection check: the real indexer rejects a non-inclusion proof for an existing address at proof-build time) |
| 161 | +- counter/native, basic-operations/native/{create,update,close,reinit,burn} |
| 162 | + |
| 163 | +Phase 2 fixes applied: |
| 164 | +- native update/close/reinit/burn: `test-sbf -> test-helpers` dragged host-only `light-client`/`solana-keypair` (getrandom 0.2, no solana target) into the SBF build. Moved the 4 test-helpers optional deps under `[target.'cfg(not(target_os = "solana"))'.dependencies]` and gated `pub mod test_helpers` with `not(target_os = "solana")`. |
| 165 | +- zk/nullifier: adapted duplicate-rejection assertion (see above). |
| 166 | + |
| 167 | +zk/zk-id: Rust port migrated and `cargo check --tests` GREEN. Its integration test cannot LINK without the native `libcircuit` produced by the example's own `scripts/setup.sh` (node/npm/circom ZK toolchain) — a pre-existing requirement independent of the SDK migration (the released-deps version needs it too). Not run. |
| 168 | + |
| 169 | +## Dependencies / risks |
| 170 | +- Requires the local `~/dev/light/light-sdks` workspace to stay buildable (it is). |
| 171 | +- Requires `light` CLI + the locally-built `photon` (0.51.2 w/ `--prover-url`) + a prover. User is managing a prover server; Phase 2 must coordinate with whatever prover/validator is running (the test's `--stop` will kill a user-run prover on :3001). |
| 172 | +- Each example is its own cargo workspace (separate Cargo.lock/target) — parallel edits touch disjoint paths, so no git/worktree isolation needed. |
| 173 | +- `zk/zk-id` is highest-risk (circuit deps); scheduled last/alone. |
0 commit comments