[X86] `is_ascii` optimizes poorly with avx-512

[folkertdev]

https://godbolt.org/z/6EzTa5crM (edit, reduced further to https://godbolt.org/z/68d8es9Tv)

It looks like a LoopFullUnrollPass that is never re-rolled (or should not be unrolled in the first place). The unrolled operations are eventually emitted as

        kshiftrd        k7, k6, 12
        add     r10b, r9b
        and     dl, 1
        kmovd   r9d, k7

but this is probably just a symptom of an earlier missed optimization. The idea of is_ascii is to build a mask of which bytes set their highest bit, so I guess some avx-512 pattern for that is missing.

cc rust-lang/rust#151259

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: Folkert de Vries (folkertdev)

Details

https://godbolt.org/z/6EzTa5crM

It looks like a LoopFullUnrollPass that is never re-rolled (or should not be unrolled in the first place). The unrolled operations are eventually emitted as

        kshiftrd        k7, k6, 12
        add     r10b, r9b
        and     dl, 1
        kmovd   r9d, k7

but this is probably just a symptom of an earlier missed optimization. The idea of is_ascii is to build a mask of which bytes set their highest bit, so I guess some avx-512 pattern for that is missing.

cc rust-lang/rust#151259

[nikic]

I don't think this is really related to unrolling. The loop is unrolled in both cases, it's just that SLPVectorizer produces different output depending on subtarget. That may be an issue either in SLPVectorizer or in the X86 cost model.

[nikic]

Here is an SLP-only reproducer: https://llvm.godbolt.org/z/zW7nczY3r (Though this isn't entirely representative of the original case which has loads rather than extracts.)

[alexey-bataev]

TTI returns 295 as the cost of the reduction operation, x86 TTI cost model issue