[ttl] Pipe verification; implement recv post -> send post -> send wait -> recv wait protocol (#608)

[brnorris03]

Problem description

The PipeNet spec currently describes pipe transfer as if data is held in a logical in-transit buffer between the sender and receiver. The runtime protocol does not have such a separate buffer: the sender must write directly into a receiver-owned DFB slot. That means the receiver has to reserve a destination slot and publish its address before the sender can perform the transfer.

The previous lowering did not make that synchronization contract explicit. It treated pipe receives as a sender-side implementation detail, which let the compiler accept syntactically valid same-thread schedules that could deadlock at runtime. In addition, the previous implementation also regressed true unicast loops by adding synthetic DFB producer operations that were not balanced with the user's reserve/wait structure; this breaks the tt-lang contract with users that users are completely responsible for the data movement associated with pipes.

What's changed

• Pipe receives were previously modeled as a sender-side implementation detail. They now use the user's reserved destination DFB slot: ttl.copy(pipe, dst) lowers to the receive post/address publication, and waiting on that copy waits for transfer completion.
• Pipe sends previously assumed the transfer target could be derived without an explicit receiver post. They now respect the receiver-posted protocol for both unicast and multicast, including repeated transfers in loops.
• Pipe synchronization state was previously shared too broadly across pipes and data-movement threads. It is now scoped so independent pipes and concurrent data-movement threads do not interfere with each other. The detailed lowering protocol is documented in docs/development/PipeNets.md.
• Receive completion previously used state that did not advance correctly for repeated receives. It now uses per-PipeNet runtime counters, so repeated unicast and multicast receives in loops advance correctly across iterations.
• The PipeNet verifier previously accepted same-thread pipe schedules that could deadlock. It now rejects those schedules and emits detailed diagnostics for common mistakes:
  • waiting for a receive before the send that completes it;
  • sending before the receiver has posted ttl.copy(pipe, dst).
• Pipe receive waits were previously omitted from the wait-for graph when their guard could not be analyzed. They now have the same static guard requirement as the receive post, so ttl.wait on a pipe receive handle under an unanalyzable coordinate-dependent guard is rejected.
• Internal receive lowering and verification previously allowed invalid pipe receive IR to fail later with less specific diagnostics. They now reject invalid receive IR earlier with clearer diagnostics.
• Per-destination multicast receive addresses are tracked as future work in issue Support per-destination pipe receive addresses for multicast lowering #617.
• Pipe tests previously encoded schedules that did not consistently publish receiver-owned destinations before sends. They now express valid receiver-posted schedules. Existing loopback multicast cases that did not need loopback were changed to exclude the source from the destination range.
• Test coverage previously missed invalid same-thread schedules, unanalyzable receive-wait guards, internal pipe receive op verifier errors, true unicast loops, multicast loops, loopback multicast, unicast forwarding-chain patterns, and mesh-SPMD pipe execution. The PR adds negative pytest/lit coverage for the invalid cases and positive coverage for the valid cases, including the issue [bug] PipeLowering sender-lockstep reserve_back on pipe dst CB hangs under iteration #608 reproducer.

Tests

+--------------------------------------+----------------------+------------------+-----------------------------+
| Pattern / test area                  | Pipe type            | Size shape       | Control-flow covered        |
+--------------------------------------+----------------------+------------------+-----------------------------+
| true unicast split loop              | unicast 1 -> 1       | 1 tile           | 20-iter loop, same DM thread|
| gather                               | N unicast -> 1 dst   | 1 tile           | per-core if, receiver sum   |
| gather multiblock                    | N unicast -> 1 dst   | 1 x 2 tiles      | multi-tile DFB slots        |
| forward ring                         | unicast ring         | 1 tile           | separate DM send/recv       |
| row rings, grid=full                 | many unicast rings   | 1 tile           | full-grid guards            |
| pipe conv chain                      | unicast chain        | 1 x 2 tiles      | conditional send, pipeline  |
| row/col forwarding chains            | unicast chains       | 1 x 2 tiles      | 5-iter loops, row+col chain |
| scatter                              | multicast 1 -> N     | 1 tile           | non-loopback multicast      |
| broadcast_2d                         | multicast 1 -> 2D    | 1 tile           | loopback, same DM ordered   |
| all-to-all 1D/2D                     | multicast N -> N     | 1 tile           | loopback, reductions        |
| overlapping multicast                | multicast overlap    | 1 tile           | multiple senders per recv   |
| partial overlap                      | multicast overlap    | 1 x 2 tiles      | asymmetric recv counts      |
| gather + broadcast loop              | unicast + multicast  | multi-row block  | stripe loops, two PipeNets  |
| rendezvous state                     | unicast/multicast    | 1 tile           | same receiver/source cases  |
| cross-DFB loopback multicast         | multicast loopback   | 1 tile           | 2-iter loop, src/dst DFBs   |
| mcast matmul                         | row/col multicast    | 8 x 8 blocks     | K loops, large grid         |
| minimal matmul mirror                | row/col multicast    | 8 x 8 blocks     | K loops, tt-metal shape     |
| mesh SPMD unicast                    | unicast 1 -> 1       | per-device shard | multi-device mesh tensors   |
| captured/module/mixed-scope nets     | mixed PipeNet usage  | 1 tile           | Python scope variants       |
| invalid schedule tests               | unicast/multicast    | 1 tile           | wait-before-send errors     |
| invalid receive-wait guard           | unicast              | 1 tile           | unknown coord guard reject  |
+--------------------------------------+----------------------+------------------+-----------------------------+

Fixes #608.

[zoecarver]

Happy to see the compiler-inserted reserves going away :)

[zoecarver]

Should this be adaptor.getDst?

[brnorris03]

Good catch! Updated both PipeRecvPostLowering and PipeRecvWaitLowering.

[brnorris03]

Actually, we do need op here -- adaptor no longer has the attributes which I still need -- ttl.cb_reserve, ttl.attach_cb, and slice offset.

[zoecarver]

Filter pip copies like findPipeReceiveCopy does?

[brnorris03]

Agreed! Extracted the filter and reused it, also made the helper return optional.