disable kv cache broadcast for better performance

[nazanin-beheshti]

Details:

For MSFT Orca model, it has different num_head for query and key/value. It leads to a broadcast behavior in the graph which introduce a big overhead for the performance. This ticket is created to eliminate the broadcast and get good conformance for Orca pipeline.

Tickets:

• CVS-186733

AI Assistance:

• AI assistance used: no

[Copilot]

Pull request overview

This PR aims to improve performance for models using Grouped-Query Attention (e.g., MSFT Orca) by avoiding KV-cache/head broadcast behavior in the generated OpenVINO graph, and adjusting ScaledDotProductAttention (SDPA) shape inference/tests accordingly.

Changes:

• Removes KV head “broadcast via concat/reshape” logic from GroupQueryAttentionDecomposition to avoid expanding KV tensors to Query head count.
• Updates SDPA shape inference/tests to stop propagating Key/Value batch dims into the output shape (reducing broadcast-driven shape effects).
• Updates SDPA type-prop expectations/error-message assertions to match the new shape inference behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/core/tests/type_prop/scaled_dot_product_attention.cpp	Updates SDPA type-prop expected output shapes and thrown-message substrings to match new inference behavior.
src/core/shape_inference/include/scaled_dot_product_attention_shape_inference.hpp	Modifies SDPA shape validation/inference to avoid broadcast-merging Key/Value leading dims into output; currently contains a Value-shape validation bug.
src/common/transformations/src/transformations/op_conversions/group_query_attention_decomposition.cpp	Removes explicit KV broadcasting in GQA decomposition to avoid performance overhead from KV expansion.

Comments suppressed due to low confidence (1)

src/core/shape_inference/include/scaled_dot_product_attention_shape_inference.hpp:50

• [HIGH] After removing broadcast_merge_into(...) from Key/Value validation, the shape inference no longer checks that Key/Value leading dimensions are compatible with Query at all (e.g., mismatched batch dimension can now silently pass). If the intent is to support GQA without explicit broadcast nodes, consider reintroducing non-broadcasting compatibility checks for the true batch dims and a GQA head-dimension rule (Q heads divisible by KV heads), similar to FlashAttentionTile shape inference (src/plugins/intel_npu/src/ops/src/intel_npu/ops/flash_attention_tile.cpp:163-215). This would keep type inference rejecting genuinely incompatible shapes while still avoiding KV broadcast.

    const auto& key = input_shapes[1];
    const auto& key_rank = key.rank();
    if (key_rank.is_static()) {
        const bool& key_input_correctness =
            key_rank.get_length() >= 3 && DimType::merge(e_dim, e_dim, *(key.end() - 1));
        NODE_SHAPE_INFER_CHECK(op,
                               input_shapes,
                               key_input_correctness,
                               "Key input shape not compatible with other inputs.");
        s_dim = *(key.end() - 2);

[Kotomi-Du]

How broadcast behavior impacts the shape which has range?

[Kotomi-Du]

build_jenkins

[Kotomi-Du]

it seems you add some unintentional change which is incorrect.