Skip to content

feat: upgrade to codama-nodes 0.13, drop the convert.cjs backfill - #322

Merged
senzenn merged 1 commit into
mainfrom
feat/codama-0.13
Sep 16, 2026
Merged

senzenn merged 1 commit into
mainfrom
feat/codama-0.13

Conversation

@senzenn

@senzenn senzenn commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

Codama's JavaScript omits empty collections when it serializes, and codama-nodes 0.9.1
declares three of them without #[serde(default)], so the output of rootNodeFromAnchor
could not be fed to the macro at all:

missing field `additionalPrograms` at line 66 column 1

Which field surfaces depends on the program. That is why docs/codama-parser-generation.md
shipped a 40-line convert.cjs backfill just to load a standard IDL.

0.10.0 added the defaults, and 0.11 through 0.13 keep them:

0.9.1   missing field 'additionalPrograms'
0.9.2   missing field 'additionalPrograms'
0.9.3   missing field 'accounts'
0.10.0  loads

This takes 0.13.2, the current release, so include_shipstern_parser! reads standard Codama
output with no backfill.

Stopping at 0.10.0 would not be cheaper. cargo check -p shipstern-proc-macro against the
old code reports the same 52 errors at both versions, same distribution (8 E0277, 39 E0308,
1 E0432, 3 E0599). 0.9.3 is where the shapes changed and every version above carries it.

Most of the diff is rote. 0.9.3 boxed children, moved counts/sizes/offsets from usize to
u64, moved fixed/isOptional from bool to Option, made origin a closed anchor|shank
enum, and renamed two enum variants. build_*.rs, helpers.rs, schema_ir.rs and the renderers
carry only that. The 39 E0308 group as:

11  TypeNode -> Box<TypeNode>     box it
11  u64 -> usize                  as_index()
 6  usize -> u64                  widening cast
 4  Box<TypeNode> -> TypeNode     deref
 3  bool -> Option<bool>          unwrap_or(false)
 4  misc: CountNode boxing, Option map, one borrow

The one change to behaviour is 16 lines in utils.rs. as_index narrows a codama u64 to the
usize every use here indexes a slice with, and panics rather than truncating. It runs at
macro expansion, so a panic surfaces as a compile error naming the value, which is what you
want for an IDL declaring an offset past addressable memory.

fixed and isOptional are the part worth reading closely. Absent changed from false to None
and is read here as .unwrap_or(false). codama's own constructors say that is right:
option_type_node.rs:9 sets fixed: None for the plain constructor and :20 sets Some(true) for
the fixed one, and instruction_account_node.rs:13 and :44 do the same for is_optional. No
fixture in this repo omits either flag (213 optionTypeNode and 3067 instructionAccountNode,
all explicit), so the corpus does not cover that path and field_presence_tests builds the
absent case directly.

The two fixtures that declared origin: "custom" drop the field instead of claiming a
toolchain. Codama accepts an absent origin, and both are synthetic fixtures for inline-struct
and name-collision handling, so no generator produced them.

docs/codama-parser-generation.md is updated in the same commit, since it documented the
workaround this removes. The replacement snippet was run against @codama/nodes-from-anchor
1.5.5 before committing: the .mjs form works under both "type": "module" and
"type": "commonjs", while a .js file using require fails with require is not defined under
the former.

Generated output does not move. All 32 IDLs in the repo (31 in tests/idls plus
examples/idl-parser/pump_fun.json) were expanded on main and on this branch and the emitted
token streams compared: identical apart from the literal-suffix change from #321. The dump
embeds PROTOBUF_SCHEMA, so the proto schemas are covered by the same comparison. Running the
dump twice on one side gives 32/32 identical, so it is not picking up ordering noise.

Also checked against live traffic: a 45 second Richat window, 2449 transactions and 1895
accounts across pump_fun, raydium, order_engine and dynamic_bonding_curve, parsed with both
the 0.9.1 and the 0.13.2 parser. 126212 comparisons, 0 divergence, 0 panics, 3551 successful
decodes. Live traffic cannot exercise the absent fixed/isOptional path, since that is a
compile-time property of the IDL rather than of the bytes.

Tested: 370 passed, 0 failed. Events workspace 45 passed. clippy -Dwarnings and fmt --check
clean.

Based on #321.

@senzenn
senzenn marked this pull request as ready for review September 13, 2026 12:08
@senzenn
senzenn force-pushed the fix/unsuffixed-literals branch from 5dfb2cd to 6a8270b Compare September 14, 2026 16:49
@senzenn
senzenn force-pushed the fix/unsuffixed-literals branch from 6a8270b to 3f9e100 Compare September 14, 2026 18:04
@senzenn
senzenn force-pushed the fix/unsuffixed-literals branch from 3f9e100 to c4ab88f Compare September 16, 2026 15:41
@senzenn
senzenn deleted the branch main September 16, 2026 16:01
@senzenn senzenn closed this Sep 16, 2026
Codama's JavaScript omits empty collections when it serializes, and codama-nodes 0.9.1
declares three of them without #[serde(default)], so the output of rootNodeFromAnchor
could not be fed to the macro at all:

    missing field `additionalPrograms` at line 66 column 1

Which field surfaces depends on the program. That is why docs/codama-parser-generation.md
shipped a 40-line convert.cjs backfill just to load a standard IDL.

0.10.0 added the defaults and 0.11 through 0.13 keep them:

    0.9.1   missing field 'additionalPrograms'
    0.9.2   missing field 'additionalPrograms'
    0.9.3   missing field 'accounts'
    0.10.0  loads

This takes 0.13.2, the current release, so include_shipstern_parser! reads standard Codama
output with no backfill.

Stopping at 0.10.0 would not be cheaper. cargo check -p shipstern-proc-macro against the
old code reports the same 52 errors at both versions, same distribution (8 E0277, 39 E0308,
1 E0432, 3 E0599). 0.9.3 is where the shapes changed and every version above carries it.

Most of the diff is rote. 0.9.3 boxed children, moved counts/sizes/offsets from usize to
u64, moved fixed/isOptional from bool to Option<bool>, made origin a closed anchor|shank
enum, and renamed two enum variants. build_*.rs, helpers.rs, schema_ir.rs and the renderers
carry only that. The 39 E0308 group as 11 boxings, 11 u64 to usize, 6 usize to u64, 4
derefs, 3 bool to Option<bool>, and 4 miscellaneous.

The one change to behaviour is 16 lines in utils.rs. as_index narrows a codama u64 to the
usize every use here indexes a slice with, and panics rather than truncating. It runs at
macro expansion, so a panic surfaces as a compile error naming the value.

fixed and isOptional are the part worth reading closely. Absent changed from false to None
and is read here as .unwrap_or(false). codama's own constructors say that is right:
option_type_node.rs:9 sets fixed: None for the plain constructor and :20 sets Some(true) for
the fixed one, and instruction_account_node.rs:13 and :44 do the same for is_optional. Every other fixture
in the corpus states both flags explicitly, so tests/idls/omitted_presence_flags.json
adds the absent case and field_presence_tests pins it at the node level as well.

The two fixtures that declared origin: "custom" drop the field instead of claiming a
toolchain. Codama accepts an absent origin, and both are synthetic fixtures for
inline-struct and name-collision handling, so no generator produced them.

docs/codama-parser-generation.md is updated in the same commit, since it documented the
workaround this removes. The replacement snippet was run against @codama/nodes-from-anchor
1.5.5 before committing. The .mjs extension is what keeps it portable: node resolves .mjs
as ESM whatever the nearest package.json says, and while node also detects import syntax
in a plain .js file, a project that sets "type": "commonjs" overrides that detection
and the same code fails to parse.

Generated output does not move. All 32 IDLs in the repo were expanded on main and on this
branch and the emitted token streams compared: identical apart from the literal-suffix
change that lands separately. A 45 second live check against a Richat feed, 2449
transactions and 1895 accounts parsed with both the 0.9.1 and 0.13.2 parsers, gives 126212
comparisons with 0 divergence, 0 panics and 3551 successful decodes.
@senzenn senzenn reopened this Sep 16, 2026
@senzenn
senzenn changed the base branch from fix/unsuffixed-literals to main September 16, 2026 16:02
///
pub(crate) fn as_index(value: u64) -> usize {
usize::try_from(value)
.unwrap_or_else(|_| panic!("IDL declares offset/size {value}, too large for this target"))
@senzenn
senzenn merged commit f187ca1 into main Sep 16, 2026
6 of 7 checks passed
@senzenn
senzenn deleted the feat/codama-0.13 branch September 16, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants