boulder: Disable forced 128bit vector width by default

[joebonrichie]

Allow clang/gcc cost models to automatically choose whatever width they feel is appropriate for auto-vectorization cases.

The toolchains are mature enough to automatically emit vzeroupper instructions to avoid the large 256bit -> 128bit latency cost.

This will improve performance in some cases where the toolchain deicides that bigger-width vectorization is worth it.

[joebonrichie]

Quick test:

// vw-test.c
float saxpy(float *c, const float *a, const float *b, float s, int n)
{
    float acc = 0.0f;
    for (int i = 0; i < n; i++) {
        c[i] = a[i] + b[i] * s;
        acc += c[i];
    }
    return acc;
}

for cc in gcc clang; do command -v "$cc" >/dev/null 2>&1 || continue; for m in x86-64-v3 x86-64-v4; do for p in none 128 256 512; do pf=""; [ "$p" = none ] || pf="-mprefer-vector-width=$p"; printf 'float f(float*c,const float*a,const float*b,float s,int n){float acc=0;for(int i=0;i<n;i++){c[i]=a[i]+b[i]*s;acc+=c[i];}return acc;}' | "$cc" -O3 -ffast-math -march="$m" $pf -S -x c - -o - 2>/dev/null | awk -v cc="$cc" -v m="$m" -v p="$p" '/%xmm/{x++} /%ymm/{y++} /%zmm/{z++} /vzeroupper/{v++} END{printf "%-6s %-10s pref=%-4s xmm=%-4d ymm=%-4d zmm=%-4d vzeroupper=%d\n", cc, m, p, x, y, z, v}'; done; done; done
gcc    x86-64-v3  pref=none xmm=37   ymm=6    zmm=0    vzeroupper=3
gcc    x86-64-v3  pref=128  xmm=33   ymm=0    zmm=0    vzeroupper=0
gcc    x86-64-v3  pref=256  xmm=37   ymm=6    zmm=0    vzeroupper=3
gcc    x86-64-v3  pref=512  xmm=37   ymm=6    zmm=0    vzeroupper=3
gcc    x86-64-v4  pref=none xmm=51   ymm=9    zmm=6    vzeroupper=2
gcc    x86-64-v4  pref=128  xmm=33   ymm=0    zmm=0    vzeroupper=0
gcc    x86-64-v4  pref=256  xmm=37   ymm=6    zmm=0    vzeroupper=3
gcc    x86-64-v4  pref=512  xmm=51   ymm=9    zmm=6    vzeroupper=2
clang  x86-64-v3  pref=none xmm=45   ymm=21   zmm=0    vzeroupper=1
clang  x86-64-v3  pref=128  xmm=62   ymm=0    zmm=0    vzeroupper=0
clang  x86-64-v3  pref=256  xmm=45   ymm=21   zmm=0    vzeroupper=1
clang  x86-64-v3  pref=512  xmm=45   ymm=21   zmm=0    vzeroupper=1
clang  x86-64-v4  pref=none xmm=45   ymm=21   zmm=0    vzeroupper=1
clang  x86-64-v4  pref=128  xmm=62   ymm=0    zmm=0    vzeroupper=0
clang  x86-64-v4  pref=256  xmm=45   ymm=21   zmm=0    vzeroupper=1
clang  x86-64-v4  pref=512  xmm=43   ymm=8    zmm=22   vzeroupper=1

[ermo]

IIUC, the reason this was originally added was power efficiency FWIW.

[joebonrichie]

I asked a LLM to come up with a benchmark that maximises the amount of ymm registers used and ran it against perf. Note this is ran on a zen2 cpu which double-pumps 128bit registers.

This is for gcc 15 and clang 21. I do remember that clang 21 has some sort of bug which heavily affected auto-vectorization which may scew the results.

#define ALEN 1024
static float x[4][ALEN];

static float kernel(int npass)
{
    float acc[8] = {0};
    for (int it = 0; it < npass; it++) 
        for (int i = 0; i < ALEN; i++) {
            float x0 = x[0][i], x1 = x[1][i];
            float x2 = x[2][i], x3 = x[3][i];
            acc[0] += x0 * x1;                
            acc[1] += x0 * x2;               
            acc[2] += x0 * x3;                
            acc[3] += x1 * x2;
            acc[4] += x1 * x3;
            acc[5] += x2 * x3;
            acc[6] += x0 * x0;
            acc[7] += x1 * x1;
        }
    return acc[0] + ... + acc[7];        
}

cc     march      pref      time    energy   watts    ipc
gcc    x86-64-v3  none    4.287s  206.890J  48.26W   3.08
gcc    x86-64-v3  128     3.922s  192.760J  49.15W   4.26
gcc    x86-64-v3  256     4.287s  209.410J  48.85W   3.08
gcc    x86-64-v3  512     4.296s  208.910J  48.63W   3.08
clang  x86-64-v3  none    4.436s  225.260J  50.77W   2.39
clang  x86-64-v3  128     4.298s  212.110J  49.35W   2.57
clang  x86-64-v3  256     4.453s  224.650W  50.45W   2.39
clang  x86-64-v3  512     4.446s  226.510J  50.95W   2.39

[ermo]

Unless I am misreading the numbers, don't they reinforce the point I was making...?

[joebonrichie]

Yep pretty much

I vaguely remember sunny's main point was the instruction penalty before and wasn't aware of the power consumption argument.

Feel free to close this, my intuition is that may actually regress things on modern CPUs going forward.

[joebonrichie]

Also ran against saxpy

cc     march      pref      time    energy   watts    ipc
gcc    x86-64-v3  none    4.359s  218.280J  50.07W   0.69
gcc    x86-64-v3  128     3.871s  207.410J  53.58W   1.28
gcc    x86-64-v3  256     3.850s  233.150J  49.90W  0.69
gcc    x86-64-v3  512     4.355s  219.200J  50.34W   0.69
clang  x86-64-v3  none    4.522s  242.110J  53.54W   0.35
clang  x86-64-v3  128     4.521s  243.710J  53.90W   0.67
clang  x86-64-v3  256     4.531s  242.470J  53.51W   0.35
clang  x86-64-v3  512     4.512s  241.810J  53.59W   0.35

seems like a mixed bag