Skip to content

optimize SSE path by using more optimized instruction to registers un… - #151

Closed
IRainman wants to merge 2 commits into
vitaut:mainfrom
IRainman:memmove_sse_unload_fix
Closed

IRainman wants to merge 2 commits into
vitaut:mainfrom
IRainman:memmove_sse_unload_fix

Conversation

@IRainman

Copy link
Copy Markdown
Contributor

…load and remove memmove from hot patch.

@IRainman IRainman left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is

Comment thread zmij.cc

ZMIJ_INLINE void write_digits(char* buffer, uint64_t digits,
bool drop_leading_zero, const data&) noexcept {
digits = digits >> (drop_leading_zero * sizeof(digits));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, you want to shift by 8 bits.
digits >>= drop_leading_zero * 8;

Comment thread zmij.cc
unsigned lo, int num_digits,
int dec_exp) noexcept -> char* {
#if ZMIJ_USE_SSE || ZMIJ_USE_SSE4_1
_mm_storeu_si128(reinterpret_cast<__m128i*>(buffer + 1), digits);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 16 byte memcpy call is replaced by the appropriate sse-move operation by the compiler.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread zmij.cc
Comment on lines +1175 to +1177
#elif ZMIJ_USE_SSE
_mm_storeu_si128(reinterpret_cast<__m128i*>(buffer + drop_leading_zero),
digits);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is currently unreachable for SSE2 because the if (!ZMIJ_USE_NEON && !ZMIJ_USE_SSE4_1) above returns first. Also, if made reachable, storing at buffer + drop_leading_zero doesn't seem equivalent to memmove(buffer, buffer + drop_leading_zero, ...): it shifts the output in the opposite direction.

Comment thread zmij.cc
unsigned lo, int num_digits,
int dec_exp) noexcept -> char* {
#if ZMIJ_USE_SSE || ZMIJ_USE_SSE4_1
_mm_storeu_si128(reinterpret_cast<__m128i*>(buffer + 1), digits);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread zmij.cc
Comment on lines 1181 to 1185
ZMIJ_INLINE void write_digits(char* buffer, uint64_t digits,
bool drop_leading_zero, const data&) noexcept {
digits = digits >> (drop_leading_zero * sizeof(digits));
memcpy(buffer, &digits, sizeof(digits));
memmove(buffer, buffer + drop_leading_zero, sizeof(digits));
}

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is reasonable but it should be 8, not sizeof(digits) and you also need to handle big endian, something like

unsigned shift = unsigned(drop_leading_zero) * 8;
digits = is_big_endian ? digits << shift : digits >> shift;
memcpy(buffer, &digits, sizeof(digits));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSVC only optimizes memcpu in O2, not in O1 https://godbolt.org/z/Mbnr33K8M

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then whoever cares about perf should compile with O2. I don't think we need to do any heroics for specific MSVC configurations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MSVC only optimizes memcpu in O2, not in O1 https://godbolt.org/z/Mbnr33K8M

To be more precise it is /Oi that is implied with /O2, and it can be also be overridden with #pragma intrinsic(memcpy) to optimize in any mode.

Still you should just compile with /O2 if you care about perf.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just compile with /O2 if you care about perf.

I always compile code with maximum optimizations, but it's not the case here.

@TobiSchluter

Copy link
Copy Markdown
Contributor

It would be nice to see benchmark results for claimed performance improvements.

vitaut added a commit that referenced this pull request Sep 13, 2026
Dropping the leading '0' on the float fixed path stored eight digit bytes
and then memmoved them back over themselves; shifting before the store
does it in the register.

ftoa-benchmark, Apple M5 Max, fixed-notation floats: 4.25ns -> 4.20ns
(-1.3%, p = 0.0003). The mixed benchmark is unchanged, as only ~15% of
its values format as fixed.

Based on #151.
@vitaut

vitaut commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Merged a fixed version of the write_digits optimization in a24bf7f, thanks.

@vitaut vitaut closed this Sep 13, 2026
@IRainman
IRainman deleted the memmove_sse_unload_fix branch September 13, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants