[Bug] rope_kv_cache精度问题

[Likai-19]

Component

PTO Dialect / ODS (include/PTO/IR)

Description

测试 kernel：rope_kv_cache.pto（已加 pto.kernel）
测试方法：run.sh 全流程（ptoas → bisheng → NPU → compare）

后端	sync 模式	ptoas	编译	NPU 运行	compare 精度
EmitC	--enable-insert-sync	✅	✅	✅	✅ 通过
EmitC	--enable-inject-barrier-all-sync	✅	✅	✅	❌ ULP 30K+
VPTO	--enable-insert-sync	✅	✅	✅	❌ ULP 30K+
VPTO	--enable-inject-barrier-all-sync	✅	—	—	未测试全流程
VPTO（原 run.sh）	两者同时	❌	—	—	互斥报错

关键发现：EmitC + insert-sync 是唯一全过的组合。其他组合 NPU 能跑不崩但结果不对。

需要解决的是VPTO路径下的精度问题，目前VPTO路径下用--enable-insert-sync或--enable-inject-barrier-all-sync精度都不对。

issue0625.zip

Reproduction (minimal)

- 复现场景：`test_for_ptoas`（`source env_remote.sh` 配置环境，里面的路径根据自己环境修改 `bash setup_all.sh` 初始化，`run_rope_kv_cache/`）
- 目标 kernel：`rope_kv_cache.pto`（A5 vec kernel，92 个 tile op）

Expected behavior

compare golden成功

Actual behavior / error logs

comapre golden失败

Git commit

0

Host platform

None

Target Ascend arch (if relevant)

None

PTOAS build level (if relevant)

None

[TelGome]

使用camodel跑 VPTO --enable-insert-sync ，报错

[warning] [00003082] cur_instr RV_VLDI (id:817 pc:0x10d0d9b0 vloop_id:1 vloop_pc:0x10d0d9ac) overlaps with pre_instr RV_VSTI (id:652 pc:0x10d0d948 vloop_id:1 vloop_pc:0x10d0d940)
[warning] [00003084] cur_instr RV_VLDI (id:822 pc:0x10d0d9b0 vloop_id:1 vloop_pc:0x10d0d9ac) overlaps with pre_instr RV_VSTI (id:655 pc:0x10d0d948 vloop_id:1 vloop_pc:0x10d0d940)
[error] [core0.veccore0] [mte_gdma_read_overflow_t0] (PC: 0x10d0d220) MTE2     : (Binary: 0x74866066) (ID: 000336) MOV_SRC_TO_DST_ALIGNv2 Src:OUT, Dst:UB, XD:X3=0x400, XN:X6=0xffffdfd4aa4de00, XM:X0=0x200000010, XT:X25=0x1000000001000, Dtype:B32, Id:336
[error] [core0.veccore0] [mte_gdma_read_overflow_t0] (PC: 0x10d0d230) MTE2     : (Binary: 0x74869066) (ID: 000340) MOV_SRC_TO_DST_ALIGNv2 Src:OUT, Dst:UB, XD:X3=0x500, XN:X9=0xffffdfd4aa4df00, XM:X0=0x200000010, XT:X25=0x1000000001000, Dtype:B32, Id:340
[error] [core0.veccore0] [mte_gdma_write_overflow_t0] (PC: 0x10d0d2dc) MTE3     : (Binary: 0x74c7cff4) (ID: 000183) MOV_SRC_TO_DST_ALIGNv2 Src:UB, Dst:OUT, XD:X3=0xefe91736297080, XN:X28=0x800, XM:X31=0x100000010, XT:X29=0x800000000100, Dtype:B8, Id:183
[error] [core0.veccore0] [mte_gdma_write_overflow_t0] (PC: 0x10d0d33c) MTE3     : (Binary: 0x74ce3060) (ID: 000207) MOV_SRC_TO_DST_ALIGNv2 Src:UB, Dst:OUT, XD:X7=0xefe9173e297200, XN:X3=0xa80, XM:X0=0x200000010, XT:X24=0x1000000000100, Dtype:B8, Id:207
[error] [core0.veccore0] [mte_gdma_read_overflow_t0] (PC: 0x10d0d2f8) MTE2     : (Binary: 0x74aa520e) (ID: 000390) MOV_SRC_TO_DST_ALIGNv2 Src:OUT, Dst:UB, XD:X21=0x880, XN:X5=0xffffdfd4aa5e800, XM:X4=0x400000010, XT:X3=0x2000000001000, Dtype:B32, Id:390

用agent定位问题，结论如下：
V pipe 内部 RAW hazard

ptoas的Insertsync插了sync,但插在 tile op展开之前;展开把一个tileop拆成一串低层DMA/向量指令后,原先按tileop边界插的sync粒度太粗,覆盖不到展开后细粒度的读写依赖,导致camodel仿真出现 Load/store冒险、输出精度错误。

[TelGome]

一、问题代码

lib/PTO/Transforms/InsertSync/SyncCodegen.cpp(CreateBarrierOp,if 在 288-290):

// A5: PIPE_V intra-pipe ordering is guaranteed by hardware; do not emit
// explicit vector barrier (it is also rejected by backend checks).
if (isTargetArchA5(func_.getOperation()) &&
    sync->GetActualSrcPipe() == PipelineType::PIPE_V) {
  return;
}

CreateBarrierOp 是 --enable-insert-sync 自动同步框架插屏障的最终生成点。目标架构为 A5、且源管线为 PIPE_V(向量管线)时,直接 return——不生成任何屏障。来源:commit 0dd9fcd7(2026-03-09),理由写在代码注释里。

二、注释的假设

注释认为:A5 硬件保证同一条 V 管线内部指令按程序序执行(保序)→ V→V 之间不需要插屏障 → 后端校验也会拒绝 V 管线的 barrier。

三、现象:假设不成立

3.1 RAW 冲突的产生机制

VPTO 把 tile op(pto.tcolexpandmul、pto.tsub、pto.tadd 等)展开成 TileLang 模板。从 IR 演变看,tcolexpandmul/tsub 原本各自带内层循环,经过 PTOInlineLibCall → SCCP → Canonicalizer 后,内层循环被折叠,vmul/vsub 合并进同一个外层循环。

循环演变(逐 pass dump)

pass (a.log dump 行)	外层 0..8	各 tile op 的内层循环	vmul/vsub 位置
ExpandTileOp (6440)	1	无(tile op 是 func.call 调用模板函数,循环藏在被调函数里)	主函数不可见
PTOInlineLibCall (7552)	1	每个 tile op 各带一个 scf.for %arg13 = %c0_45 to %131 step %c1_46(边界还是变量),体里 vlds→vmul/vsub→vsts + scf.yield	各自在独立的内层 for 里
SCCP (12337)	1	变量边界被常量传播成单次循环:scf.for %arg13 = 0 to 1 套 scf.for %arg14 = 0 to 64 step 64(共 16 个,每个迭代 1 次)	仍在各自的单次内层 for 里
Canonicalizer (13238)	1	16 个单次内层 for 全部折叠 inline → 0 个	平铺在外层 0..8 循环体里,vmul/vsub 同层相邻

演变的关键两步:

1. SCCP 把变量边界常量化。内联进来的内层循环边界是 %c0_45 to %131 step %c1_46(模板里按 tile shape 算出来的运行时值),SCCP 常量传播后变成 %c0 to %c64 step %c64——只迭代 1 次的单次循环。

2. Canonicalizer 折叠单次循环。scf.for 0 to 64 step 64 只跑 1 次,Canonicalizer 把循环体 inline 进外层、消掉 for。tcolexpandmul、tsub、tcolexpandmul、tadd 各自的单次内层 for 依次被折叠后,它们的 vlds→vmul/vsub→vsts 就平铺到了同一个外层 scf.for %arg12 = 0 to 8 循环体里(见 a.log:13338/13350/13361…,vmul、vmul、vsub 同层相邻)。

不是 tsub 不进循环,而是 tsub 原本自带的内层单次循环被 Canonicalizer 折叠掉了,它的 vsub 直接落到了外层逐行循环里,和前后 tcolexpandmul 的 vmul 成了同层相邻指令——这正是RAW 产生的位置。

复现命令(不带 --enable-v-membar,输出最终 IR 到 a_test.log):

ptoas --pto-arch=a5 --pto-backend=vpto --pto-level=level3 --emit-vpto \
      --enable-insert-sync \
      /home/chendongyan/PTOAS/issue0625/test_for_ptoas/rope_kv_cache.pto \
      > a_test.log 2>&1

a_test.log 全文 pto.mem_bar 出现 0 次。外层 scf.for %arg12 = 0 to 8 循环体第一段(pto.vecscope,行 98–125)vmul 和 vsub 在同一作用域里。各 UB 指针的地址来自 pto.castptr %cNNNN_i64:%25=@1536、%75=@1792。

典型 RAW:%75(@1792)先 vsts 后立刻 vlds

tcolexpandmul #2 的输出写到 %75(@1792),紧接着 tsub 把 %75 当输入读回——同地址、紧邻的 V→V RAW,中间没有任何 pto.mem_bar:

// a_test.log:99-107 (vecscope 内
ptoas --pto-arch=a5 --pto-backend=vpto --pto-level=level3 --emit-vpto --enable-insert-sync /home/chendongyan/PTOAS/issue0625/test_for_ptoas/rope_kv_cache.pto  > a_test.log 2>&1
)
%75 = pto.castptr %c1792_i64 ...                // %75 = UB 指针 @1792
...
%76 = pto.vmul %result, %result_0, %mask        // tcolexpandmul : 乘
pto.vsts %76, %75[%c0], %mask                   //   写 %75 = @1792   ← store
%result_1 = pto.vlds %25[%c0]                   //   读 %25 = @1536
%result_2 = pto.vlds %75[%c0]                   //   读 %75 = @1792   ← 立刻 load 回,RAW
%77 = pto.vsub %result_1, %result_2, %mask      // tsub: 减

冲突点:pto.vsts %76, %75 写 @1792 → 下一条 pto.vlds %75 读 @1792。A5 V 管线乱序下,这条 vlds 可能在 vsts 落地前就发射,读到旧值或半写值。--enable-insert-sync 本应在此插屏障,但 CreateBarrierOp 在 A5+V 直接 return。

注:%75 在本段是 castptr @1792(tcolexpandmul 的中间结果 tile);同文件其它作用域里 %75 会被重定义为别的指针(如行 196 的 arith.muli 行偏移、行 309 的 @14208),SSA 名复用但作用域独立,不影响本段分析。

3.2 --enable-v-membar 把 mem_bar 插在哪

--enable-v-membar 在同一外层循环体内部、相邻 tile op 展开出的指令之间插 pto.mem_bar,挡在"写中间 tile X 的 vsts"和"读中间 tile X 的 vlds"之间:

scf.for %arg12 = %c0 to %c8 step %c1 {          // 同一个外层循环
  ...vlds→vmul→vsts...                          //   tcolexpandmul #1
  ...vlds→vmul→vsts...                          //   tcolexpandmul #2
  pto.mem_bar "VST_VLD"                         //   ← mem_bar 在循环体内,两段向量指令之间
  ...vlds→vsub→vsts...                          //   tsub(读前面 vsts 写的中间 tile)
  pto.mem_bar "VLD_VST"
  ...vlds→vmul→vsts...                          //   tcolexpandmul #3
  ...
}

mem_bar 在 tile op 层面(展开前)插入:PTOInsertVMemBar pass 注册在 ExpandTileOp 之前(tools/ptoas/ptoas.cpp:1573-1574):

kernelModulePM.addPass(pto::createPTOInsertVMemBarPass());      // 先插 mem_bar(tile op 层面)
kernelModulePM.addPass(pto::createExpandTileOpPass(expandOpts)); // 后展开成 vmul/vsub/scf.for

即 mem_bar 是在 IR 还是 pto.tcolexpandmul/pto.tsub 形态时、按 tile op 的 UB 地址依赖插的;之后 ExpandTileOp+Canonicalizer 把 tile op 展开成向量指令、把内层单次循环折叠进外层循环时,mem_bar 跟着落到了同一循环体内相邻指令之间。

3.3 A5 V 管线实际乱序

真机 A5 与 CAModel 仿真都表明:A5 V 管线内部并不保序。同一循环体内 vsts→vlds 同地址 RAW 依赖,若没有 mem_bar 保护,V 管线乱序执行,vlds 可能在 vsts 完成前发射,读到旧数据。--enable-insert-sync 框架本应在此插屏障,但 CreateBarrierOp 在 A5+V 直接 return(见第一节),导致这些 RAW 依赖裸奔——这正是 a.log(不带 v-membar,0 个 mem_bar)呈现的状态。CAModel 表现为 RV_VLDI overlaps RV_VSTI(V 管 load 与之前的 V 管 store 执行重叠),真机同样复现。

3.4 现象小结

A5 + V 管线下,--enable-insert-sync 框架对 V→V 依赖完全不插屏障(CreateBarrierOp 直接 return),导致:

• V 管线内部 vsts/vlds 乱序执行;
• 同一循环体内跨 tile op 的 UB spill(同地址 RAW)被错误读回;
• 计算结果精度偏差(issue 866 的 v1/v2 偏差)。

[TelGome]

pto微指令的同步需求场景，当前ptoas在产生这种代码不会自动插入membar导致精度问题

// Copyright (c) 2026 Huawei Technologies Co., Ltd.
// Minimal repro: tcolexpandmul#2/tsub/#3/#4/tsub fuse into ONE pto.vecscope
// (vmul+vsub+vmul+vmul+vsub adjacent, V->V RAW unguarded w/o --enable-v-membar).
// NOTE: 3 ops (2 tcolexpandmul+1 tsub) get split into separate vecscopes by
// --enable-insert-sync; 6 ops are required for fusion.
// Trimmed from rope_kv_cache.pto L1-90.
// RUN: ptoas --pto-arch=a5 --pto-backend=vpto --pto-level=level3 --emit-vpto --enable-insert-sync %s -o - 2>/dev/null

!t1f = !pto.tile_buf<loc=vec, dtype=f32, rows=1, cols=64, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>
!t1b = !pto.tile_buf<loc=vec, dtype=bf16, rows=1, cols=64, v_row=?, v_col=?, blayout=row_major, slayout=none_box, fractal=512, pad=0>

module attributes {pto.target_arch = "a5", pto.kernel_kind = #pto.kernel_kind<vector>} {
  func.func @min_vecscope_repro(%k_cache : !pto.ptr<bf16>, %k_proj : !pto.ptr<f32>, %cos_lo : !pto.ptr<f32>, %sin_lo : !pto.ptr<f32>, %cos_hi : !pto.ptr<f32>, %sin_hi : !pto.ptr<f32>, %arg10 : index, %arg11 : index) attributes {pto.kernel_kind = #pto.kernel_kind<vector>} {
    %c0_i64 = arith.constant 0 : i64
    %c256_i64 = arith.constant 256 : i64
    %c512_i64 = arith.constant 512 : i64
    %c768_i64 = arith.constant 768 : i64
    %c1024_i64 = arith.constant 1024 : i64
    %c1280_i64 = arith.constant 1280 : i64
    %c1536_i64 = arith.constant 1536 : i64
    %c1792_i64 = arith.constant 1792 : i64
    %c2048_i64 = arith.constant 2048 : i64
    %c32768_index = arith.constant 32768 : index
    %c4096_index = arith.constant 4096 : index
    %c0_index = arith.constant 0 : index
    %c1_index = arith.constant 1 : index
    %c8_index = arith.constant 8 : index
    %c64_index = arith.constant 64 : index
    %c128_index = arith.constant 128 : index
    %c524288_index = arith.constant 524288 : index
    %c16_index = arith.constant 16 : index
    %c1024_index = arith.constant 1024 : index

    %k_cache_v = pto.make_tensor_view %k_cache, shape = [%c524288_index, %c128_index], strides = [%c128_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xbf16>
    %k_proj_v  = pto.make_tensor_view %k_proj,  shape = [%c16_index, %c1024_index],   strides = [%c1024_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
    %cos_lo_v  = pto.make_tensor_view %cos_lo,  shape = [%c1_index, %c64_index], strides = [%c128_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
    %sin_lo_v  = pto.make_tensor_view %sin_lo,  shape = [%c1_index, %c64_index], strides = [%c128_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
    %cos_hi_v  = pto.make_tensor_view %cos_hi,  shape = [%c1_index, %c64_index], strides = [%c128_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>
    %sin_hi_v  = pto.make_tensor_view %sin_hi,  shape = [%c1_index, %c64_index], strides = [%c128_index, %c1_index] {layout = #pto.layout<nd>} : !pto.tensor_view<?x?xf32>

    %cos_lo_pv = pto.partition_view %cos_lo_v, offsets = [%c0_index, %c0_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
    %cos_lo_t  = pto.alloc_tile addr = %c0_i64   valid_row = %c1_index valid_col = %c64_index : !t1f
    pto.tload ins(%cos_lo_pv : !pto.partition_tensor_view<1x64xf32>) outs(%cos_lo_t : !t1f)
    %sin_lo_pv = pto.partition_view %sin_lo_v, offsets = [%c0_index, %c0_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
    %sin_lo_t  = pto.alloc_tile addr = %c256_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
    pto.tload ins(%sin_lo_pv : !pto.partition_tensor_view<1x64xf32>) outs(%sin_lo_t : !t1f)
    %cos_hi_pv = pto.partition_view %cos_hi_v, offsets = [%c0_index, %c0_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
    %cos_hi_t  = pto.alloc_tile addr = %c512_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
    pto.tload ins(%cos_hi_pv : !pto.partition_tensor_view<1x64xf32>) outs(%cos_hi_t : !t1f)
    %sin_hi_pv = pto.partition_view %sin_hi_v, offsets = [%c0_index, %c0_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
    %sin_hi_t  = pto.alloc_tile addr = %c768_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
    pto.tload ins(%sin_hi_pv : !pto.partition_tensor_view<1x64xf32>) outs(%sin_hi_t : !t1f)

    scf.for %ki = %c0_index to %c8_index step %c1_index {
      %off = arith.muli %ki, %c128_index : index
      %k_lo_pv = pto.partition_view %k_proj_v, offsets = [%arg10, %off], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
      %k_lo_t  = pto.alloc_tile addr = %c1024_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tload ins(%k_lo_pv : !pto.partition_tensor_view<1x64xf32>) outs(%k_lo_t : !t1f)
      %off64 = arith.addi %off, %c64_index : index
      %k_hi_pv = pto.partition_view %k_proj_v, offsets = [%arg10, %off64], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xf32> -> !pto.partition_tensor_view<1x64xf32>
      %k_hi_t  = pto.alloc_tile addr = %c1280_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tload ins(%k_hi_pv : !pto.partition_tensor_view<1x64xf32>) outs(%k_hi_t : !t1f)
      %t_t   = pto.alloc_tile addr = %c1536_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tcolexpandmul ins(%k_lo_t, %cos_lo_t : !t1f, !t1f) outs(%t_t : !t1f)
      %u_t   = pto.alloc_tile addr = %c1792_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tcolexpandmul ins(%k_hi_t, %sin_lo_t : !t1f, !t1f) outs(%u_t : !t1f)
      %rot_lo_t = pto.alloc_tile addr = %c1536_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tsub ins(%t_t, %u_t : !t1f, !t1f) outs(%rot_lo_t : !t1f)
      %a_t   = pto.alloc_tile addr = %c1280_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tcolexpandmul ins(%k_hi_t, %cos_hi_t : !t1f, !t1f) outs(%a_t : !t1f)
      %b_t   = pto.alloc_tile addr = %c1024_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tcolexpandmul ins(%k_lo_t, %sin_hi_t : !t1f, !t1f) outs(%b_t : !t1f)
      %rot_hi_t = pto.alloc_tile addr = %c1024_i64 valid_row = %c1_index valid_col = %c64_index : !t1f
      pto.tsub ins(%a_t, %b_t : !t1f, !t1f) outs(%rot_hi_t : !t1f)
      %base = arith.addi %arg11, %off : index
      %o0 = pto.partition_view %k_cache_v, offsets = [%base, %c0_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xbf16> -> !pto.partition_tensor_view<1x64xbf16>
      %lo_b = pto.alloc_tile addr = %c2048_i64 valid_row = %c1_index valid_col = %c64_index : !t1b
      pto.tcvt ins(%rot_lo_t {rmode = #pto<round_mode ROUND>} : !t1f) outs(%lo_b : !t1b)
      pto.tstore ins(%lo_b : !t1b) outs(%o0 : !pto.partition_tensor_view<1x64xbf16>)
      %o1 = pto.partition_view %k_cache_v, offsets = [%base, %c64_index], sizes = [%c1_index, %c64_index] : !pto.tensor_view<?x?xbf16> -> !pto.partition_tensor_view<1x64xbf16>
      %hi_b = pto.alloc_tile addr = %c2048_i64 valid_row = %c1_index valid_col = %c64_index : !t1b
      pto.tcvt ins(%rot_hi_t {rmode = #pto<round_mode ROUND>} : !t1f) outs(%hi_b : !t1b)
      pto.tstore ins(%hi_b : !t1b) outs(%o1 : !pto.partition_tensor_view<1x64xbf16>)
    }
    return
  }
}

输出问题ir（%35，%16）

        pto.vecscope {
          %35 = pto.castptr %c1792_i64 : i64 -> !pto.ptr<f32, ub>
          %result = pto.vlds %22[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %result_0 = pto.vlds %3[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %mask, %scalar_out = pto.plt_b32 %c64_i32 : i32 -> !pto.mask<b32>, i32
          %36 = pto.vmul %result, %result_0, %mask : !pto.vreg<64xf32>, !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<64xf32>
          pto.vsts %36, %35[%c0], %mask : !pto.vreg<64xf32>, !pto.ptr<f32, ub>, !pto.mask<b32>
          %result_1 = pto.vlds %25[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %result_2 = pto.vlds %35[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %37 = pto.vsub %result_1, %result_2, %mask : !pto.vreg<64xf32>, !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<64xf32>
          pto.vsts %37, %25[%c0], %mask : !pto.vreg<64xf32>, !pto.ptr<f32, ub>, !pto.mask<b32>
          %result_3 = pto.vlds %22[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %result_4 = pto.vlds %6[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %38 = pto.vmul %result_3, %result_4, %mask : !pto.vreg<64xf32>, !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<64xf32>
          pto.vsts %38, %22[%c0], %mask : !pto.vreg<64xf32>, !pto.ptr<f32, ub>, !pto.mask<b32>
          %result_5 = pto.vlds %16[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %result_6 = pto.vlds %9[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %39 = pto.vmul %result_5, %result_6, %mask : !pto.vreg<64xf32>, !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<64xf32>
          pto.vsts %39, %16[%c0], %mask : !pto.vreg<64xf32>, !pto.ptr<f32, ub>, !pto.mask<b32>
          %result_7 = pto.vlds %22[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %result_8 = pto.vlds %16[%c0] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %40 = pto.vsub %result_7, %result_8, %mask : !pto.vreg<64xf32>, !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<64xf32>
          pto.vsts %40, %16[%c0], %mask : !pto.vreg<64xf32>, !pto.ptr<f32, ub>, !pto.mask<b32>
        }