Conversation
Introduce a CPU-only Visualizer package that inspects Hugging Face configs and modeling code via AST to render architecture diagrams, including detailed block trees, hybrid layer variants, and fact sheets with loop-derived layer repeat info.
Derive attention and output-gate labels from AST kernel metadata so KDA shows delta-rule computation instead of QKᵀV, document g_proj’s gated-norm role, and add a measure-reflow-validate pipeline to prevent overlapping layout in detailed SVGs. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Align shared tee routing, shrinkwrap, and layout compaction so input fan-out leaves vertically with matched branch tees, and add runtime checks plus tests for the routing, clamp, and per-leg bus behavior. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…xits Feeders stacked in a target's own column competed for the same top-entry port, so one leg cut across the merge bus of the block it passed. Ports are now assigned deterministically by source column and proximity: the nearest feeder keeps the center port, and the ones above it bypass on a port beside the blocking tile, teeing below that tile's bus. The old port order depended on dict iteration order, which under some hash seeds gave the adjacent feeder the offset port and produced a horizontal jog inside its exit stub. Exit-stub validation now inspects every segment rather than just the path endpoints, and sub-epsilon horizontal offsets snap to vertical so they no longer render as slanted. Co-authored-by: Cursor <cursoragent@cursor.com>
Parse config-aware gate math instead of fabricated router stages, classify methods like moe_infer from structure, and compact SparseMoeBlock names without doubling an existing Moe stem. Co-authored-by: Cursor <cursoragent@cursor.com>
…ing space Detailed diagrams were losing content and drifting wider on every layout pass. Kernels imported from a checkpoint's own directory never resolved, so blocks like DeepSeek sparse_attn stayed opaque; norms named attn_norm/ffn_norm were classified as attention/ffn, leaving the transformer block with no pairs to render. Frames travelling with a cluster were also used to constrain it, and side columns were averaged into frame centres, ratcheting widths up each pass. Layout now re-hangs section content from its anchor after ports are re-docked, so a vacated port row no longer opens dead space under the section title, and spine connectors reach frame boundaries and merge glyph edges. Co-authored-by: Cursor <cursoragent@cursor.com>
These fill uncovered branches in TraceLens Agent Analysis, PerfModel extensions, and Reporting generators using fixtures and mocks. Co-authored-by: Cursor <cursoragent@cursor.com>
Combine operations were special-cased as circular tiles that data could
enter from the side, and operation tiles could caption the learned
parameters feeding them. Both made diagrams inconsistent with the rest of
the graph, so operations are now ordinary named rectangles ("Add",
"Multiply") whose inputs always arrive at the top edge.
Dropping the symbolic node labels also removed the layout code that keyed
off them: the MoE router spine no longer matches the first "+" it finds,
which had been selecting an unrelated gate-frame Add and shifting the
sparse MoE section further right on every validation pass.
Co-authored-by: Cursor <cursoragent@cursor.com>
Modeling code binds its activation through a registry lookup (`self.act_fn = ACT2FN[config.hidden_act]`) rather than constructing it, and the init parser only derived class names from constructor calls. The attribute therefore had no class at all, so Qwen3 drew a tile labelled "act fn" and its dense MLP dropped the activation entirely. The checkpoint config already names the activation, so the lookup now resolves through it. A registry lookup is also the fallback arm of a config switch whose other arm constructs a real module, as in Kimi's SituAndMul, so it never displaces a constructed class. Because a registered activation is pointwise and carries no gating, it no longer counts as a composite child that would stop a forward from contributing its own tensor math. Co-authored-by: Cursor <cursoragent@cursor.com>
Side-fed modules that expand into visible steps attached every extra call argument to the frame head, so DeepSeek's MoE drew the router's gating weights as a second input to the expert's first Linear. Operations now record which forward parameter they read, letting each feed dock on the step that actually consumes it, with a new route for entering a frame member below the head. Merge buses were also inferred from coincident coordinates, which turned TopK's two independent operands into a shared trunk. Bus membership now requires the link to own that source/target bus, and same-side merge legs get channels reserved from the graph before routing. Co-authored-by: Cursor <cursoragent@cursor.com>
…r real source Connector anchors were widened to the enclosing inline frame, so the router placed entry and exit ports across the frame rather than the tile. Ports landed in empty space beside a tile and drew as connectors joined to nothing, which is what made the MiniMax dense MLP and experts blocks look half-wired. An anchor is now the tile as drawn; a frame is an obstacle, not a docking surface. Operand resolution also missed calls that become their own node, so an operation reading such a result resolved no source and fell back to a link from the previous step in source order -- a dataflow edge the forward never performs. The indexer's MatMul now reads the rope call it actually consumes instead of the pad amount, and an operation whose operands come from a forward parameter no longer chains to the step above it. Unanchored branches rank from their consumer so they sit beside the step that reads them instead of floating up beside the input. Finally, a submodule invoked on the untouched forward input now branches at the input instead of running in series behind the previous call. The MoE block's experts read the block input rather than the shared expert's output, and both paths reach the output, which is the combine the forward performs. Co-authored-by: Cursor <cursoragent@cursor.com>
…t_tracediff The export gains a fact-sheet builder and merge wires it into the viewer bundle. test_tracediff.py had merge conflict markers committed into it, so the module did not parse at all. Both sides of every marker block were empty, so removing the markers restores the file without choosing between them or losing anything. Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate the bundled graph after the single-block fact sheet change, and check in the CLI export output plus the Kimi operator introspection artifact. Co-authored-by: Cursor <cursoragent@cursor.com>
Fix L2Norm q/k hidden-state inputs and kernel operand edges, use readable purple tiles, add the fact-sheet panel, rename the CLI entry point, and refresh the Kimi-K3 export JSON. Co-authored-by: Cursor <cursoragent@cursor.com>
…odes. Replace per-kernel label tables with generic AST-derived naming, wire multi-port pipeline inputs through labeled edges and inputsMetadata, and default the viewer to show port_label on edges. Co-authored-by: Cursor <cursoragent@cursor.com>
…ibling chain) Closes the two false cycles the vision-tower fidelity work exposed, keeping the export acyclic while removing spurious RMSNorm input edges. merge.py (_inject_group_outputs): a nested boundary (loop body whose @input nodes live at an ancestor id-prefix) could derive an @output id already owned by an ancestor section's real boundary. The viewer fuses same-id nodes, merging their edge sets into a false cycle. Disambiguate the derived id by namespace only when it is already taken by a node in another namespace (prefix != namespace is pervasive/normal, so the guard is collision-based, not mismatch-based). computation_graph.py: - Scope a module's first-op resolution by node identity, not shared attr_name, so section-1b boundary args land on the right instance. Same source line (Glm5NextRMSNorm) is reused by post_layernorm and every block norm; the flat attr_last_index kept only the last, dumping a block's boundary args onto post_layernorm's Cast (5 inputs) and the decoder kv_a_layernorm Cast (2). Both now collect exactly one. - Skip section-1b side-arg edges onto a module's first op when that op provably does not read the arg (_first_op_entry_params): the block-head norm takes only hidden_states; binding cu_seqlens/position_embeddings there both misrepresents the norm and, since the kernel producing cu_seqlens runs later, closed a cycle. - Feed each inlined submodule from its real predecessor (_submodule_chain_input) instead of the previous sibling. Parallel siblings that share an upstream (attention q_norm/k_norm both read the qkv unbind) were chained sequentially, manufacturing a spurious q_norm->k_norm edge that, with the correct k_norm<-unbind edge, closed a cycle. Export is acyclic (0 non-trivial SCCs, no duplicate ids); post_layernorm, kv_a_layernorm and each vision norm Cast have one input; merger proj has one. Full suite: 1018 passed, 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The vision tower conflated its patch axis with the LLM sequence, stamping the text B*S / B*S/4 symbols and text hidden 4096 onto every vision activation. Introduce an "active activation geometry" on ShapeInferencer: while a vision-scoped section is inferred, the (B,S,H) fallback stamps and the -1 reshape flatten switch to the single patch axis (Pv,) + vision hidden 1024. Seed the tower @input with [Pv, C*T*P*P] so inner boundaries inherit the vision axis by conservation; the text/decoder path is untouched (active geometry defaults to (B,S) + text hidden). Result: patch-embed [Pv,1176]->[Pv,3,2,14,14]->Conv3d [Pv,1024,1,1,1]-> [Pv,1024]; merger @output [Pv/4,4096]; zero B*S in the visual namespace; combine node stays [B,S,4096]. Graph remains acyclic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Issue C — restore the rotary path: propagate param aliases through tuple/name unpacks (cos, sin = position_embeddings) so _param_refs recognises the unpacked names and apply_rotary_pos_emb is no longer dropped. Issue B — vision @input / patch-embed hidden_states now [Pv, 1176]: guard infer_model_graph's subgraph merge so a parent-seeded concrete vision spec is not clobbered by a child's root-less default, and stamp the synthetic @vision_input boundary node with the real patch-flat shape. Issue E — a collapsed module's output_shape line shows the slot name only when the namespace has more than one output port (two-pass, arity-based group_boundary_shapes), not for single-output modules. Issue A — enforce concrete-vs-concrete edge equality: after endpoint reconciliation, fail the export when both ends of a wire have fully-concrete but disagreeing shapes, naming both nodes/ports/shapes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…A–G) Improve the fidelity of the Model Explorer graph export so every computation is visible and correctly shaped, with no cyclic graph. All fixes are general graph machinery, not model-specific hacks. - A: enforce concrete-vs-concrete equality on both ends of a wire; a real dim mismatch now fails the export instead of shipping silently. - B: vision @input / patch-embed hidden_states resolve to the raw flat patch dim [Pv,1176] instead of a clobbered root-less default. - C: restore the rotary path — tuple-unpacked `cos, sin = position_embeddings` aliases now propagate so the op is no longer dropped. - D: multi-output split/unbind. Tuple-unpacked `.split([4,4,16])` / `.unbind(0)` fan out into one output-port node per named slice, each carrying its real slice width (`[B,S,4]/[B,S,4]/[B,S,16]`). A multi-output op whose sole operand is a module parameter (`self.base.split(...)`, `self.scale.unbind(0)`) is no longer false-chained to the previous sibling, and its slices size from the parameter shape. - E: a collapsed module shows an output slot name only when it has >1 output. - F/G: expert loop flattens into Glm5NextTextMoE as a single count-labeled `Loop_288_iterations` block with one carried in/out pair; the spurious duplicate MoE-level loop and its mis-resolved carried shape are gone. All loops use the uniform `Loop_{N}_iterations` convention. Adds a loop-carried-pair well-formedness regression test and updates the expert-loop namespace assertions for the MoE flatten. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ging path The --backend option (and its torch/ast split) was removed when the AST pipeline became the sole graph builder; the README still documented a torch default backend and non-existent --seq-len/--batch-size flags. - Replace the two-backends framing with the actual static-analysis + meta-device-shape pipeline; document the two-stage model->JSON->HTML flow. - Refresh the options table: drop --backend, add --from-payload, --no-inline-expansion, --meta-shapes/--no-meta-shapes, --torch-module, --input-shape; fix the stale '(ast only)' note on --github. - Add a 'Debugging: standalone nn.Module introspection' subsection covering --torch-module (torch_introspect), kept intentionally as a debugging aid separate from the HF checkpoint pipeline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A forward that dispatches attention through ALL_ATTENTION_FUNCTIONS[config._attn_implementation] leaves the AST holding only the local variable name (attention_interface), and attention_kernel_details() then suppresses that synthetic name so the exported node shows no kernel at all. GLM-5.3-Flash ships _attn_implementation unset, so the vision (and full text) attention rendered with no indication of which kernel runs. Default the resolved implementation to "sdpa" (the transformers default when the checkpoint leaves _attn_implementation unset) so a dispatched-attention step names the kernel that actually runs. Surface it as a dedicated attn_implementation node attr on AttentionOp nodes, in addition to the existing details line. General: applies to any dispatched-attention model; custom kernels (e.g. GLM's recurrent_kimi_delta_attention linear attention) are untouched since only steps whose kernel is a dispatch variable get resolved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fect 2) Glm5NextVisionAttention.forward branches on `if is_flash_attention_requested(self.config):` between a fused flash kernel and a per-chunk torch.cat fallback. is_flash_attention_requested is opaque to _config_value, so the extractor walked BOTH branches: the export then carried a duplicated attention kernel plus a phantom Concat from the branch that never runs, an output reshape with two inputs, and a dangling Concat self-loop. Resolve the predicate from the checkpoint config in the forward extractor. A new _resolve_flash_request_predicate reads config._attn_implementation (defaulting to the transformers "sdpa" when unset), maps it against the set of flash implementation names, and honours a leading `not`. The undecidable-if then collapses to the branch that actually runs (sdpa -> not flash -> the per-chunk fallback), so exactly one attention kernel and one real Concat remain and the output reshape reads a single producer. Thread the class config through _forward_operations_from_forward into the extractor at the three visit_ClassDef call sites via _config_for_class. General: keys off the shared transformers predicate name (is_flash_attention_requested), not any model or class, and falls back to the existing _config_value path for every other conditional. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sion defect 1) A forward that unpacks `q, k, v = self.qkv(h)...unbind(0)` and then feeds two submodule calls off distinct slots — `self.q_norm(query_states)` (ordinal 0), `self.k_norm(key_states)` (ordinal 1) — lost the consumed output ordinal on the module-call wiring path. The inline-op path already threads it (operation_predecessor_ports), but forward_step_predecessors carried only the producer, so both norms docked onto slot 0: key_states never appeared as its own port and k_norm read query_states. Capture the consumed output ordinal alongside the arg->producer map in the forward extractor (step_predecessor_ordinals), reusing the existing _read_output_ports slot resolver so it fires for any unpacked unbind/split/chunk local, not any particular class. Thread it through ForwardAnalysis -> ClassStructure -> BlockNode (forward_step_predecessor_ordinals) and stamp graph.link_output_ports in computation_graph section 1b, mirroring the inline-op ordinal handling in section 1. The producer then fans out into one named @split_out port per consumed slot. Result for GLM-5.3-Flash vision attention: three distinct query_states / key_states / value_states ports off the qkv unbind, q_norm->query_states, k_norm->key_states, value_states on its own slot, no q_norm->k_norm edge. General: keys off the already-recorded var_output_ordinal (Split/Chunk/Unbind), threaded through the one wiring path that dropped it; no class-name checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ion defect 3) Glm5NextVisionMLP computes gate = gate_proj(hidden_state) and up = up_proj(hidden_state) — two leaf linears in parallel off the same block input (clamp-based swiglu). _add_linear_pipeline_chain feeds each leaf step from the previous sibling's tail by default, so up_proj rendered as up_proj <- gate_proj: two linears in a row, misrepresenting the swiglu as a chain. Consult the AST-recorded predecessor for leaf steps too. Generalize _submodule_chain_input (previously composite-only, and it skipped the forward input) to map a recorded FORWARD_METHOD_INPUT predecessor to the chain input index when the caller supplies one, and call it from the leaf branch. up_proj records reading @method_input, so it now sources the block input; a named-sibling predecessor still resolves to that sibling; everything else falls back to the previous-sibling chain, so genuine sequential pipelines are untouched. The composite call site passes no chain_input_index, so its behavior is unchanged. General: prefers the AST-recorded true predecessor over source-order position; independent of fused-silu vs clamp-swiglu and leaves _situ_gated_mlp untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…roundwork) Vision export dropped every module-level free-function call (get_vision_position_ids, get_vision_attention_seqlens, apply_mask_to_ padding_states). Their results defaulted to whatever fed the call, so the rotary/seqlen side-paths dangled or mis-docked to hidden_states. - ast_analyze: emit unrecognized free-function calls as @fn_* synthetic nodes that bind their own output producer (skip_free_fn gates them off inside conditionals so a dropped attention branch can't resurrect dead ops); seed a secondary forward input as the method boundary only when a traced free-function actually consumes it, and only outside conditionals. - block_tree: render @fn_* synthetics as FunctionOp leaf nodes. - tests: absorb the resulting seq-index / split-ordinal shifts and assert a computation path (not a direct edge) where a newly-visible op now sits between the block input and the forget gate. Fixes Bug 1 (get_vision_position_ids now feeds rotary_pos_emb). Bug 2 (get_vision_attention_seqlens) now renders as a node; its tuple-output ordinals and role classification are addressed in follow-ups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A dispatched-attention module declares packed-attention metadata by keyword (cu_seqlens, max_seqlen) that only the flash code path reads; the default sdpa/eager path ignores some of it. Once is_flash_attention_requested resolves to the non-flash branch, such a parameter is a genuine part of the module's interface yet dead in *this* implementation. Rather than fabricate a live data edge for it, classify forward-signature params against the resolved vs dropped branches: params referenced only in a dropped branch (max_seqlen under sdpa) are removed from the wired kernel inputs and surfaced as an `unused_interface_inputs` detail, while params the surviving branch consumes (cu_seqlens) stay declared kernel inputs. General: keys off the shared transformers flash-request predicate and the forward signature, no model or parameter names baked in. Also: give traced free-function synthetics (@fn_*) a neutral role so an incidental "attention" token in the callee name no longer misclassifies them as attention kernels (which the graph would fold away), and pass their recorded output names through to their leaf nodes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The block-loop attention receives cu_seqlens by keyword from get_vision_attention_seqlens(), which runs before the loop. That producer (@fn_l1840) was emitted but stripped as a dangling leaf: section-1b's module-call wiring skipped the cu_seqlens edge because the kernel had no dedicated param entry, so the guard dumped nothing onto the block head norm (correctly avoiding a cycle) and left the producer unconsumed. Fix, in two general pieces: - block_tree: a boundary attention input (empty provenance chain, e.g. cu_seqlens) becomes a param_inputs entry on the @attention kernel node. _build_module_param_entries then registers it, giving the caller's predecessor edge a dedicated docking point on the kernel instead of the head norm — so the edge survives the guard and the producer stays live. - merge: name a group's input boundary after the @kernel_port_in label of the port it feeds, so the crossing renders as ``cu_seqlens`` rather than a generic ``hidden_states_2`` fallback. Result: fn_l1840 -> @input_mirror:cu_seqlens -> @input:cu_seqlens -> @kernel_in:cu_seqlens -> @attention, acyclic. max_seqlen (impl-dead) stays correctly unwired and flagged. Full suite green (1027 passed, 3 skipped). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… loop (Bug 2C / C4c) The vision rotary path severed at the block boundary: apply_rotary_pos_emb_vision reads `cos, sin = position_embeddings`, but the positional synthetic declared no forward-param input, so its operands defaulted to hidden_states and the pre-loop `rotary_pos_emb` producer was orphaned. Add a general `step_boundary_params` mechanism: an own-step synthetic records the forward parameters it reads straight from the module boundary, resolving unpacked locals (`cos`/`sin`) back to their caller-visible origin (`position_embeddings`) via a param-alias origin map. The positional leaf then carries those as `param_inputs`, so the cross-module predecessor pass docks the caller's producer onto the deep consumer — the same routing `cu_seqlens` uses to reach the kernel. Also fix an over-broad shape heuristic in `fill_missing_node_shapes`: the `"embedding"` substring matched the `position_embeddings` boundary/mirror and stamped it with the language (B, S, H) activation shape. A connected `@input`/ `@input_mirror` now stays pending so edge-propagation inherits the vision producer's real (Pv, ...) shape instead. Result: get_vision_position_ids -> rotary_pos_emb -> position_embeddings mirror -> @input:position_embeddings -> apply_rotary, acyclic, carrying [Pv, 1176]. Adds test_glm53_vision_rotary_position_embeddings_wired_across_loop. Full suite 1028 passed, 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An inline op whose only operands are module parameters/buffers reads no chain producer, so it rendered with no incoming edge and its values appeared to come from nowhere. The canonical case is the mHC mapping's `pre_b, post_b, comb_b = self.base.split(...)` and `pre_scale, post_scale, comb_scale = self.scale.unbind(0)`: a Split/Unbind over a raw learned parameter. The AST already records the parameter as an `external_input`; `_add_module_parameter_inputs` now materializes it as a local `@tensor:external:<param>` operand node docked into the consuming op, so the computation shows where its data originates. General and additive: gated to ops that are otherwise sourceless (no incoming edge), so chain steps that read a weight *and* continue the spine (`x = x * self.weight`) are untouched — matching the guardrail in `_reads_only_a_side_parameter`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A tuple-unpacked split/chunk/unbind (mHC ``pre_w, post_w, comb_w =
F.linear(...).split([4, 4, 16])`` and vision ``q, k, v = qkv(h)...unbind(0)``)
previously fanned out into synthetic ``@split_out:`` port NODES — one per slice.
Those tiles rendered as gray "operation: unknown" boxes because ``@split_port_out``
was not a classifiable synthetic, even though they were correctly wired.
Per the owner's decision, drop the synthetic tiles entirely and give the split node
itself one real named output port per unpacked slice. Consumers were already wired to
the right ordinal (``link_output_ports`` from the AST unpack), so each ``Multiply`` /
``Add`` / ``View`` now reads the split's ordinal directly and shows its own operation.
Each port carries its per-slice shape — the edge feeding ``view`` reads ``[B, S, 16]``,
not the undivided ``[B, S, 24]``.
Mechanism (general, keyed off ``block.output_names``; no class-name checks):
- computation_graph.py: remove ``_add_split_output_port_nodes`` (and its
``@split_port_out`` constant) — the split node keeps its ordinal-threaded edges.
- shape_inference.py: ``infer_model_graph`` publishes one slice spec per ordinal under
a reserved ``f"{id}{PORT_SPEC_SEP}{ordinal}"`` key (via ``_multi_output_slice_shape``);
drop the dead ``@split_port_out`` inference branch.
- adapter.py: emit an ``output_names`` attr on multi-output nodes.
- shapes.py: ``annotate_nodes_with_shapes`` stamps per-ordinal ``outputsMetadata``
(label + slice shape) when reserved port specs are present.
Full suite: 1029 passed, 3 skipped; export acyclic. Payload drops ~67 dummy tiles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Constants and learned weights are never drawn: F.linear shows its activation input and hides self.weight. Two mHC mapping ops slipped past that rule because the weight was their only operand, fanned out: pre_b, post_b, comb_b = self.base.split(...) pre_scale, post_scale, comb_scale = self.scale.unbind(0) self.base and self.scale are nn.Parameter (confirmed via ast.unparse of the analyzed Glm5NextTextHyperConnection.__init__), so no activation flows through these ops -- they are constants and must not appear. Replace _add_module_parameter_inputs (which docked a visible @tensor:external:<param> operand for such ops) with _prune_weight_only_ops: roots are sourceless multi-output forward ops reading only a self.<attr> external (mirroring the conservative multi-output guard in _reads_only_a_side_parameter); weight-only-ness then propagates to any op every operand of which is itself weight-only (e.g. comb_b.view(hc, hc)). Pruned via the existing _prune_computation_nodes. Consumers that mix the result with a real activation (pre_w * pre_scale + pre_b) stay and simply lose the hidden operand edge. Payload 982->952 nodes; 0 @tensor:external operands remain; acyclic. Full suite 1029 passed, 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ask #34) A checkpoint records the transformers version it was exported with in config.transformers_version. When that differs from the version installed alongside TraceLens, the installed modeling file can be the wrong revision for the checkpoint. Prefer the upstream modeling file at the declared release tag; fall back to the installed file (then the pinned SHA) when no version is declared, it matches installed, or the tag is not published. source.py: new _installed_transformers_version, _fetch_versioned_transformers_file (lru_cached incl. the negative result so a missing tag does not re-hit the network on every load_model_spec), and _transformers_versioned_modeling_file; wired into resolve_source_files ahead of the installed-file lookup for transformers-native models. Tries both vX.Y.Z and bare X.Y.Z tag spellings. For zai-org/GLM-5.3-Flash (declares 5.16.0, installed 5.16.1, and the 5.16.0 tag is absent upstream) this falls back to the installed file -> no graph or op-id change. Adds coverage in test_source_kernel_coverage.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…iring (Task #33 defect A) Expand the traced free-function apply_rotary_pos_emb_vision (and nested rotate_half) into a visible op sub-pipeline in the Glm5NextVisionAttention frame, and wire its cos/sin inputs to the correct position_embeddings tuple ports. Consumer side: the position_embeddings boundary now fans out one port per tuple slot (cos = port 0, sin = port 1) to its two consuming unsqueeze ops, via a new ordinal-aware module-param entry lookup in computation_graph (_build_module_param_ordinal_entries / _lookup_ordinal_entries), replacing the single-entry setdefault that dropped ordinals for tuple params. Producer side: when several return slots trace to the same producer op (cos, sin = self.recomposition_frequencies(...)), _inject_group_outputs now disambiguates by source output ordinal so the rotary boundary emits distinct @output:cos / @output:sin nodes instead of colliding both onto one id/label (a hard Model Explorer validity bug). _resolve_slot_names_for_prefix returns an ordered per-producer slot list; the consumer tolerates a scalar mapping too. Verified on zai-org/GLM-5.3-Flash: 971 nodes, 0 duplicate ids, export acyclic. Full suite green (1032 passed, 3 skipped). Updated vision-rotary topology tests and added a shared-producer ordinal-slot coverage test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…il edge (Task #33 defect A) A consumer reading a specific return slot of an inline-expanded tuple-returning free function must dock onto that slot's internal producer, not the frame's last op. ``q_embed, k_embed = apply_rotary_pos_emb_vision(...)``: q_embed is the ordinal-0 internal cast, k_embed the frame's last op. The source-order chain in _add_chain fed the q-path transpose from the frame tail (k_embed), so it read both slots and q_embed was orphaned. Thread the helper's own return_order/return_slots (ordinal -> internal producer attr) as forward_step_return_producers through ClassStructure and BlockNode, and in the section-1 predecessor wiring redirect the consumer to the matching per-ordinal producer, removing the stale frame-tail chain edge. General: derived from the helper's return metadata, no class-name checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Glm5NextTextLinearAttention calls q_proj/k_proj/v_proj/b_proj as parallel projections on the shared apply_mask_to_padding_states(hidden_states, ...) result. The SeqSegment sequential-source fallback in build_computation_graph chained consecutive submodule calls in source order, fabricating a spurious q_proj -> k_proj edge and rendering a "Linear" node with two tensor inputs. Guard the fallback so a submodule call with AST-recorded predecessors (forward_step_predecessors) is left to section-1b predecessor wiring rather than spine-chained onto the previous call. Parallel siblings that read the block input keep their single real producer; only genuinely sequential steps still fall back to source order. General across models -- honors the recorded true predecessors instead of "previous call in source order". Verified: q/k/v/b_proj each in=1 from the masked-input op, no sibling edges; graph-wide zero submodule-call Linears with >=2 inputs; export acyclic; full suite 1034 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An expandable module (collapsed namespace) surfaces its boundary shapes as layer attributes so edges ending on the group still read a shape. Those attributes come from group_boundary_shapes, which reads each @output boundary port's `shape` metadata field. annotate_nodes_with_shapes rebuilds that port metadata from the producer feeding the port, but built the `shape` value with format_shape (dims only) while every other shape-application path (_apply_shape_attrs, _apply_multi_port_shape_attrs) uses format_shape_with_dtype. Result: 15 collapsed text/vision modules (input_layernorm, LinearAttention, MoE, MLP, post_attention_layernorm, visual) showed output shape without its type. Use format_shape_with_dtype for the boundary port `shape` too, matching the other paths. All 134 module boundary shape attrs now carry dtype (was 119/15); full suite 1035 passed / 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
F.linear(input, weight) records both operands, so hyper-connection, MoE- experts and router linears each carried a second tensor input -- the learned weight (a self.fn.float() cast, a per-expert gate_up_proj gather, a self.weight.type(float32) cast). Like the absorbed weight of an nn.Linear submodule, that operand must not be drawn: the owner's rule is a Linear has a single input, the activation. Add _prune_linear_weight_operands after _prune_weight_only_ops: for each forward _linear op, drop the edge from any operation_predecessors[1:] producer that reads a module parameter (non-empty external_inputs), then strip the now- dangling weight producer. Removing the edge (not bridging across the producer) keeps a gathered weight's shared routing-index predecessor from being rewired into the linear. All 55 Linear nodes in the GLM-5.3-Flash export now render with exactly one input (was 12 with two); graph stays acyclic; the only remaining non-boundary leaves are the Task #39 vision unsqueezes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#39 defect B) Glm5NextVisionAttention runs the eager/sdpa path as an else-branch comprehension the provenance walk never descends into, and value_states never passes through a submodule call. The kernel therefore wired only query_states/cu_seqlens, dropping key_states/value_states and leaving their unsqueeze ops as orphan leaves. Add _ForwardOperationExtractor._reconstruct_attention_step: when @attention is a forward call with no recorded predecessors, recover them from the first attention-interface call in source order, resolving each positional tensor operand through var_producer (which tracks inline ops and non-module tensors alike) and naming each so the kernel port is labelled query_states/key_states/value_states. General: no branch or comprehension shape is assumed. Kernel now reads query_states/key_states/value_states/cu_seqlens, each labelled and sourced from its own producer, no orphaned unsqueezes, acyclic. Full suite green (1037+1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
#41 D2) The inline-expanded rope-helper frame took its namespace segment from the raw synthetic call attr (`@positional_l1615_apply_rotary_pos_emb_vision` -> `_positional_l1615_apply_rotary_pos_emb_vision`). inline_block_frame_label now returns the clean source function name for positional/function synthetic frames, so the frame reads like any module call (`apply_rotary_pos_emb_vision`). Node ids keep the raw attr, so the rename is id-stable; _is_free_function_frame_namespace recognizes the renamed segment via the recovered function name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…41 D4) A tensor produced in one child group and consumed in a sibling child under the same parent rendered as two identically named mirror tiles adjacent in the parent scope: an `@output_mirror` feeding an input-styled `@input_mirror`. In the MoE, this left a `topk_weights` input node that went into no block. When an `@input_mirror`'s incoming edges all come from a single same-namespace `@output_mirror`, the two describe the same tensor at the same scope: _collapse_mirror_boundary_passthroughs drops the `@input_mirror` and repoints its consumers onto the `@output_mirror`. Dataflow is unchanged (experts still gather topk_weights, one-hot topk_indices); one redundant tile per crossing disappears. A multi-source tuple fan-in (vision position_embeddings gathering cos+sin) has >1 source and is left intact. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…on (Task #D5) get_vision_position_ids and get_vision_attention_seqlens run host-side index building (.tolist()/.item()). Detect this generally by AST-walking the traced callable for host-materialisation idioms, following cross-file imports (via importlib find_spec + ast.parse, no execution) and recursing into callees. Flagged nodes render a device: cpu attr; pure-tensor rope helpers stay clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…vive (Task #42 D1) Glm5NextVisionRotaryEmbedding.forward calls the same child twice (cos = self.recomposition_frequencies(cos); sin = self.recomposition_frequencies(sin)) and returns (cos, sin). A submodule call's step key was the bare child attr, so call #2 overwrote call #1: the cos branch was dead-code-eliminated, the real Cosine op orphaned, and both return slots aliased onto the single sin producer. Analysis: give a child that is called more than once in a forward an @l{lineno} call-site suffix (submodule_callsite_attr / base_submodule_attr), so the two calls become two distinct steps each keeping its own predecessor and var_producer. Detection is path-aware: a child called once in each of two mutually-exclusive if/else arms is NOT repeated, so single-call and branch-exclusive forwards stay byte-identical. Wiring: in _wire_all_predecessor_edges pass 1b, resolve each tuple-unpacked side arg (position_embeddings -> cos/sin) to its own return-slot producer via _resolve_return_slot_source instead of aliasing every ordinal onto the last slot's tail. Genuine single-producer tuple ops still split by ordinal port. Result: distinct live Cosine and Sine ops, two recomposition_frequencies frames, @output:cos / @output:sin backed by different producers, acyclic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…(Task #43 D3) A traced free-function call (a rope helper apply_rotary_pos_emb_vision, get_vision_position_ids) was special-cased to skip boundary injection: its ops docked straight onto external producers, so the frame had no @input/@output tiles and did not read like a module. Owner rule: a free-function call must render exactly like any other module call, with real boundaries. Drop the free-function branch from _skip_nested_inline_frame_input (and remove the now-dead _is_free_function_frame_namespace helper + its unused imports). The tuple fan-out that motivated the skip (cos/sin position_embeddings[0]/[1]) is preserved by the same per-ordinal machinery a multi-return module already uses: _group_entry_buckets keys a bucket by the producer node, so both ordinal ports land on one @input tile that re-exposes them; _inject_group_outputs emits one @output per tuple slot. _inject_group_outputs only gives an @output to namespaces that already have an @input, so output_namespaces stays a subset of input_namespaces. Result: the apply_rotary frame renders @input:q / @input:k / @input:position_embeddings (the tuple stays one logical input, fanning ports 0/1 = cos/sin to the two unsqueezes) and one @output per return slot feeding the q/k transposes per ordinal; acyclic. Rewrote the two rotary tests that asserted the absence of a boundary to assert the new module-like boundary, and added test_glm53_vision_rotary_frame_has_module_like_boundaries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rotary block's (cos, sin) tuple arrived at each vision block as the forward parameter position_embeddings and collapsed onto one @input tile fanning ports 0/1 (the sin port had no shape, rendering `?`). q_norm and k_norm likewise showed identical `hidden_states` @input tiles though they read distinct qkv-unbind slices, and the unbind's per-ordinal port shapes dropped a dim ([Pv, 1024] -> [1024]). Prevent the merge at its source rather than merging then splitting: - merge.py: name a boundary input after the producer slice it reads. A new _return_slot_label threads the child's forward_return_slots into _inject_group_inputs (symmetric to the output side's _resolve_slot_names_for_prefix), so cos/sin stay two tiles named after the rotary return slots. _source_slice_label additionally recognises named qkv-unbind slices (output_names) and sibling @input/@output slice tiles, so nested consumers and q_norm/k_norm boundaries keep the slice name. Fires only for genuinely multi-output producers (bounded blast radius). - shape_inference.py: an unbind slice keeps every non-split axis. The node spec is already one materialised slice, so only drop the unbind axis when the source still carries it (axis size == output count); otherwise return the slice unchanged. Fixes [Pv, 1024] -> [1024]. Result: distinct @input:cos / @input:sin (shaped, no `?`) at every level; q_norm/k_norm boundaries labelled query_states/key_states; unbind ports report [Pv, 1024]. Acyclic; full suite 1044 passed + 3 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… shape) The rotary embedding returns (cos, sin) where each slot traces to a distinct single-output op (cos, sin = self.recomposition_frequencies(...)). The ordinal-1 (sin) slot's producer coincides with the frame tail, so the fan-out tagged its edge with the tuple ordinal "1" -- but that op has only output port "0". The dangling sourceNodeOutputId left the viewer unable to resolve the sin shape, rendering it as "?". When slots trace to distinct producers, read port "0" instead of the ordinal. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Visualizer,visualize_model_in_explorer).B,S,H, …) by default, keep nested diagrams in their tile slots, and label collapsed groups with boundary shapes so Model Explorer no longer shows?on those edges.Test plan
cd Visualizer && PYTHONPATH=. pytest tests/(install[Visualizer]extra if needed)visualize_model_in_explorer zai-org/GLM-5.3-Flashand confirm shapes on edges, indexer nested in attention, hyperconnection collapse atB x S x 4096Made with Cursor