feat: misc compiler, VM, and IDL improvements

[adiman9]

Summary

This PR contains various improvements to the compiler, VM, macros, and IDL systems:

Compiler & VM (interpreter)

• Various compiler and VM improvements

Macros

• Vixen runtime: Enhanced tracing and queue handling
• Data structures: Added support for packed structs and large arrays
• CPI events: Added support with camelCase field handling
• Bug fix: Prevented cross-entity instruction hook contamination

IDL

• Added packed representation support to IdlRepr

Chore

• Updated dependencies in ore stack and examples

Changes

• 9 commits ahead of main

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperstack-docs	Ready	Preview, Comment	Mar 14, 2026 4:34pm

[greptile-apps]

Greptile Summary

This PR adds CPI event support end-to-end (IDL codegen → instruction parser → VM routing → TypeScript output), fixes cross-entity instruction hook contamination, and improves observability throughout the Vixen runtime flush path. It also adds packed-struct support and large-array serde helpers to the IDL code generator.

Key changes:

• CPI event pipeline: idl_codegen.rs emits an events submodule; idl_parser_gen.rs extends the instruction enum with Event_<Name> variants matched via a 16-byte prefix (anchor tag + event discriminator); vm.rs and ast/writer.rs treat CpiEvent-suffixed types identically to IxState for queuing and flush logic
• Cross-entity hook contamination fix: collect_pda_registrations_per_entity replaces the global pda_registrations list so each entity only receives its own register_from hooks — however the new function only walks one level of section struct nesting, which may silently drop registrations in deeper-nested sections
• TypeScript dedup: Per-entity already_emitted_types tracking prevents re-emitting builtin resolver schemas and IDL enum schemas across entities; a PascalCase/raw-name inconsistency in extract_emitted_enum_type_names may miss tracking of snake_case IDL enum types
• IdlRepr::packed: New field lacks #[serde(skip_serializing_if = "Option::is_none")], so serialized IDL JSON will contain "packed": null for all non-packed types
• Observability: Silent Err(_) => {} discards in the flush path are replaced with tracing::warn!, and QueueUntil / flush-loop success paths gain tracing::info!
• camelCase account lookup: snake_to_lower_camel helper and dual lookup (ctx.account(camel).or_else(|| ctx.account(raw))) accommodate both Pumpfun-style camelCase and Raydium-style snake_case IDL account names

Confidence Score: 3/5

• Functional correctness risk from the per-entity PDA registration scope limitation and enum dedup name mismatch warrants review before merging to production.
• The core CPI event pipeline is well-structured and internally consistent. However, two logic issues reduce confidence: (1) collect_pda_registrations_per_entity only scans one level of section struct nesting — a silent regression relative to the old global scan for nested section layouts; (2) extract_emitted_enum_type_names may fail to track snake_case IDL enum names due to a PascalCase mismatch, causing duplicate TypeScript schema emission in multi-entity stacks. Both issues would be silent (no panic, no compile error) and may only manifest with specific user configurations.
• hyperstack-macros/src/stream_spec/idl_spec.rs (per-entity PDA scan scope) and interpreter/src/typescript.rs (enum dedup name mismatch)

Important Files Changed

Filename	Overview
interpreter/src/vm.rs	Extends CPI event handling to mirror IxState behaviour: PDA lookup miss queuing, pending-update flushing with write_version, and null-key warning suppression. Also adds tracing::info/warn throughout the flush path and converts silent Err(_) => {} ignores to logged warnings. load_field and set_field_sum gain trace-level logging.
interpreter/src/typescript.rs	Adds cross-entity deduplication of emitted TypeScript types via already_emitted_types. Introduces extract_builtin_resolver_type_names and extract_emitted_enum_type_names to track what was actually emitted per entity. A PascalCase/raw-name mismatch in extract_emitted_enum_type_names may fail to track snake_case IDL enum types, causing duplicate emissions in multi-entity stacks.
interpreter/src/compiler.rs	Improves the LookupIndex resolver path: a CopyRegister from resolved_key_reg now takes priority, with CopyRegisterIfNull falling back to the LookupIndex result. This ensures flush-provided resolved keys supersede first-arrival lookups, matching the Embedded/Computed key strategies.
hyperstack-macros/src/idl_codegen.rs	Adds generate_event_types / generate_event_type for a new events submodule with try_from_bytes (including discriminator validation), packed-struct to_json support, and #[serde(with = "big_array")] for arrays exceeding 32 elements. Missing type definition now panics at macro-expansion time.
hyperstack-macros/src/idl_parser_gen.rs	Extends the generated instruction enum with Event_<Name> variants for CPI events. Adds 16-byte prefix matching (anchor event tag + event discriminator) in unpack, and wires all enum methods (to_json, event_type, to_value, to_value_with_accounts) to handle event variants. The comment on the anchor event tag byte order is a known issue flagged in a prior review thread.
hyperstack-macros/src/stream_spec/idl_spec.rs	Introduces collect_pda_registrations_per_entity to fix cross-entity instruction hook contamination. The new function only scans section structs that are direct field types of each entity, which may silently drop registrations in section structs nested more than one level deep.
hyperstack-macros/src/codegen/vixen_runtime.rs	Adds tracing instrumentation throughout the pending-update flush path (QueueUntil, flushing loop, reprocess success/failure). Converts silent Err(_) => {} ignores to tracing::warn!. Previously noted unused hooks_count variable and missing warning messages addressed in prior review threads.
hyperstack-macros/src/codegen/core.rs	Extends RegisterPdaMapping hook action to try both camelCase (via snake_to_lower_camel) and raw snake_case when looking up accounts in InstructionContext, accommodating both Pumpfun-style camelCase and Raydium-style snake_case IDL account names.
hyperstack-macros/src/event_type_helpers.rs	Adds snake_to_lower_camel helper function. Single-word and multi-word snake_case identifiers are handled correctly; leading underscores are silently dropped (edge case) but the call sites always fall back to the raw snake_case name, so this is low-risk in practice.
hyperstack-idl/src/types.rs	Adds packed: Option<bool> to IdlRepr with #[serde(default)] for deserialization. Missing #[serde(skip_serializing_if = "Option::is_none")] means round-tripped IDL JSON will contain "packed": null for all non-packed types.
hyperstack-macros/src/ast/writer.rs	Adds is_cpi_event detection (via ::events:: path containment) to route CPI event fields under data.* and assign the CpiEvent type suffix, mirroring the instruction-event (IxState) path.
hyperstack-macros/src/stream_spec/mod.rs	Extracts resolve_snapshot_source as a shared helper replacing duplicated inline logic in both entity.rs and sections.rs. Also handles a new field attribute (single-field extraction) alongside the existing transform-based whole-source capture.

Sequence Diagram

sequenceDiagram
    participant RPC as Yellowstone gRPC
    participant VH as VmHandler (vixen_runtime)
    participant IP as InstructionParser (idl_parser_gen)
    participant VM as VmContext (vm.rs)
    participant TS as TypeScriptCompiler

    RPC->>VH: raw transaction instruction bytes
    VH->>IP: parser.unpack(data)
    note over IP: Match 16-byte prefix<br/>[anchor_tag(8) + event_disc(8)]
    IP-->>VH: Ok(Enum::Event_Swap(SwapEvent))
    VH->>VH: event_type = "program::SwapCpiEvent"
    VH->>VM: process_event(bytecode, event_value, "SwapCpiEvent", context)
    alt PDA lookup miss
        VM->>VM: queue_account_update(state_id, QueuedAccountUpdate{write_version})
        note over VM: CpiEvent treated same as IxState<br/>for pending-update flush trigger
    else PDA found
        VM->>VM: execute_handler → mutations
        VM->>VM: flush_pending_updates(lookup_keys)
        loop pending updates
            VM->>VM: execute_handler(pending) → more mutations
        end
    end
    VM-->>VH: all_mutations

    note over TS: compile_stack_spec (multi-entity)
    loop each entity
        TS->>TS: extract_builtin_resolver_type_names(spec)
        TS->>TS: compile with already_emitted_types
        TS->>TS: extract_emitted_enum_type_names(output)
        TS->>TS: emitted_types.extend(enum_names + builtin_names)
    end

Loading

Comments Outside Diff (3)

1. hyperstack-macros/src/stream_spec/idl_spec.rs, line 600-637 (link)

   Per-entity scan misses non-directly-referenced section structs

   collect_register_from_specs (which this function replaces for entity processing) scans all section structs in the module flat, without regard to which entity they belong to. This caused cross-entity contamination, which this PR correctly fixes.

   However, collect_pda_registrations_per_entity only adds a section struct to structs_to_scan when it appears as a direct field type of the entity struct. Any section struct whose register_from attribute is relevant to an entity, but which is not a direct field type (e.g., a section struct used as a field type within another section struct one level deeper), will not be scanned — and its PDA registrations will be silently dropped.

   Consider:

   struct MyEntity {
       pool_id: PoolId,           // ← scanned
   }
   struct PoolId {
       position_id: PositionId,   // ← NOT scanned — only one level deep is walked
   }
   struct PositionId {
       #[map(source = ..., register_from(...))]
       address: String,           // ← silently missed
   }
   

   collect_register_from_specs would find this registration (it scans every struct in section_structs); collect_pda_registrations_per_entity would not.

   Since pda_registrations (the globally-collected list) is no longer passed to process_entity_struct_with_idl, any registrations in deeper-nested sections are now lost rather than contaminated. A simple fix is to recurse into each discovered section struct:

   let mut queue: VecDeque<&syn::ItemStruct> = VecDeque::from([entity_struct]);
   while let Some(scan_struct) = queue.pop_front() {
       structs_to_scan.push(scan_struct);
       if let syn::Fields::Named(fields) = &scan_struct.fields {
           for field in &fields.named {
               if let syn::Type::Path(type_path) = &field.ty {
                   if let Some(seg) = type_path.path.segments.last() {
                       if let Some(s) = section_structs.get(&seg.ident.to_string()) {
                           if !structs_to_scan.contains(&s) { queue.push_back(s); }
                       }
                   }
               }
           }
       }
   }

2. hyperstack-idl/src/types.rs, line 233-240 (link)

   packed: None serializes as "packed": null

   #[serde(default)] only affects deserialization (missing field → None). When IdlRepr is serialized (e.g., when writing back a processed stack JSON or running round-trip tests), every struct that does not explicitly set packed will emit "packed": null in the output, which may break external Anchor IDL consumers that don't expect this key.

   Add skip_serializing_if to suppress the field when absent:

3. interpreter/src/typescript.rs, line 1688-1720 (link)

   PascalCase/raw-name mismatch may cause duplicate enum emission

   extract_idl_enum_type_names collects the raw enum type names from the IDL JSON (e.g., "direction_kind"). However, generate_idl_enum_schemas converts each type name to PascalCase before emitting the schema:

   let interface_name = to_pascal_case(type_name);   // "direction_kind" → "DirectionKind"
   // emits: export const DirectionKindSchema = z.enum([...])

   extract_emitted_enum_type_names extracts "DirectionKind" from the generated output and then checks:

   if idl_enum_names.contains(schema_name)  // contains("DirectionKind") → false

   …because idl_enum_names holds the raw name "direction_kind", not "DirectionKind". The type is therefore not recorded as emitted, and the next entity in the stack will re-emit it, producing a duplicate DirectionKindSchema declaration in the TypeScript output.

   Typical Anchor IDL type names are already PascalCase so this only triggers for snake_case enum names, but the function should be resilient regardless. One fix is to normalise when inserting into idl_enum_names:

   names.insert(to_pascal_case(type_name));

   Or alternatively, normalise schema_name when comparing:

   if idl_enum_names.contains(schema_name)
       || idl_enum_names.contains(&to_pascal_case(schema_name))

Prompt To Fix All With AI

This is a comment left during a code review.
Path: hyperstack-macros/src/stream_spec/idl_spec.rs
Line: 600-637

Comment:
**Per-entity scan misses non-directly-referenced section structs**

`collect_register_from_specs` (which this function replaces for entity processing) scans **all** section structs in the module flat, without regard to which entity they belong to. This caused cross-entity contamination, which this PR correctly fixes.

However, `collect_pda_registrations_per_entity` only adds a section struct to `structs_to_scan` when it appears as a **direct field type** of the entity struct. Any section struct whose `register_from` attribute is relevant to an entity, but which is not a direct field type (e.g., a section struct used as a field type within another section struct one level deeper), will not be scanned — and its PDA registrations will be **silently dropped**.

Consider:
```
struct MyEntity {
    pool_id: PoolId,           // ← scanned
}
struct PoolId {
    position_id: PositionId,   // ← NOT scanned — only one level deep is walked
}
struct PositionId {
    #[map(source = ..., register_from(...))]
    address: String,           // ← silently missed
}
```

`collect_register_from_specs` would find this registration (it scans every struct in `section_structs`); `collect_pda_registrations_per_entity` would not.

Since `pda_registrations` (the globally-collected list) is no longer passed to `process_entity_struct_with_idl`, any registrations in deeper-nested sections are now lost rather than contaminated. A simple fix is to recurse into each discovered section struct:

```rust
let mut queue: VecDeque<&syn::ItemStruct> = VecDeque::from([entity_struct]);
while let Some(scan_struct) = queue.pop_front() {
    structs_to_scan.push(scan_struct);
    if let syn::Fields::Named(fields) = &scan_struct.fields {
        for field in &fields.named {
            if let syn::Type::Path(type_path) = &field.ty {
                if let Some(seg) = type_path.path.segments.last() {
                    if let Some(s) = section_structs.get(&seg.ident.to_string()) {
                        if !structs_to_scan.contains(&s) { queue.push_back(s); }
                    }
                }
            }
        }
    }
}
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: hyperstack-idl/src/types.rs
Line: 233-240

Comment:
**`packed: None` serializes as `"packed": null`**

`#[serde(default)]` only affects deserialization (missing field → `None`). When `IdlRepr` is serialized (e.g., when writing back a processed stack JSON or running round-trip tests), every struct that does not explicitly set `packed` will emit `"packed": null` in the output, which may break external Anchor IDL consumers that don't expect this key.

Add `skip_serializing_if` to suppress the field when absent:

```suggestion
    #[serde(default)]
    #[serde(skip_serializing_if = "Option::is_none")]
    pub packed: Option<bool>,
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: interpreter/src/typescript.rs
Line: 1688-1720

Comment:
**PascalCase/raw-name mismatch may cause duplicate enum emission**

`extract_idl_enum_type_names` collects the raw enum type names from the IDL JSON (e.g., `"direction_kind"`). However, `generate_idl_enum_schemas` converts each type name to PascalCase before emitting the schema:

```rust
let interface_name = to_pascal_case(type_name);   // "direction_kind" → "DirectionKind"
// emits: export const DirectionKindSchema = z.enum([...])
```

`extract_emitted_enum_type_names` extracts `"DirectionKind"` from the generated output and then checks:

```rust
if idl_enum_names.contains(schema_name)  // contains("DirectionKind") → false
```

…because `idl_enum_names` holds the raw name `"direction_kind"`, not `"DirectionKind"`. The type is therefore **not recorded as emitted**, and the next entity in the stack will re-emit it, producing a duplicate `DirectionKindSchema` declaration in the TypeScript output.

Typical Anchor IDL type names are already PascalCase so this only triggers for snake_case enum names, but the function should be resilient regardless. One fix is to normalise when inserting into `idl_enum_names`:

```rust
names.insert(to_pascal_case(type_name));
```

Or alternatively, normalise `schema_name` when comparing:
```rust
if idl_enum_names.contains(schema_name)
    || idl_enum_names.contains(&to_pascal_case(schema_name))
```

How can I resolve this? If you propose a fix, please make it concise.

_{Last reviewed commit: d626ac1}