Skip to content

T4a deletes the keep-quant provider gates that #2782 landed, dropping coverage from ten encodings to three #2938

Description

@localai-org-maint-bot

Row: GFX1100-TG200

#2782 lands the kROCM keep-quant GEMM providers and gates them with
tests/vt/test_rocm_quant_dot.cpp: a ten-row encoding table
(IQ2_XXS, IQ3_XXS, IQ2_S, IQ1_S, IQ1_XXXS, Q2_K, Q3_K, Q4_K,
Q5_K, Q6_K) and three cases over it.

51f5222dc in #2790 (T4a) rewrites that file, 497 insertions(+), 231 deletions(-), and the rewrite removes all three:

  • ROCm keep-quant GEMM == CPU reference and f64 dequant (Q8_K family)
  • ROCm grouped keep-quant GEMM == CPU grouped golden and it WRITES the output
  • ROCm keep-quant registers the native kROCM providers

The table goes with them, replaced by kKQuantCases with Q4_K, Q5_K and
Q6_K. The two pull request bodies record the drop without naming it: #2782
says "132,094 assertions green across all ten encodings", #2790 says
"6/6 cases, 719 assertions".

The providers themselves stay. IsRocmKeepQuantSupported still answers yes for
all ten, and rocm_quant_dot.hip still instantiates LaunchGemm and
LaunchGroupedGemm for IQ2_XXS, IQ3_XXS, IQ2_S, IQ1_S, IQ1_XXXS,
Q2_K and Q3_K. After T4a nothing exercises any of them, and nothing
exercises the grouped provider or the registration at all. That is roughly a
thousand lines of kernel with no gate, on the pull request whose own purpose was
to gate them.

Found while gating the external-contributor ROCm stack. Repaired in the same
flow on the landed merge rather than on #2790's branch: #2790 sits under
nineteen later branches that all edit this file, so a commit there costs
nineteen conflict resolutions and this costs none.

The restoration is additive: WeightCase regains its nmse_ref_max override,
kMaxNmseErr comes back, the ten-row table returns as kCases beside T4a's
narrower kKQuantCases, and the three cases return on the
vt::rocm::DeviceAvailable() guard the file now uses. T4a's own cases are
untouched.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions