Problem Description
/v1/embeddings output for embed-gemma:300m (EmbeddingGemma-300M, loaded with flm serve <chat_tag> --embed 1) changed between v0.9.39 and v1.0.4 in a way that is not a numerically-different-but-equivalent re-encoding: it is statistically uncorrelated with the correct embedding for the same input text.
Identical on-disk model weights were used for both versions (the weights cache is shared across binary versions; both binaries resolve the same installed weight files). Only the binary differs.
Expected
Cosine similarity close to 1.0 between the v0.9.39 and v1.0.4 embeddings of the same sentence, if v1.0.4 is a correct (even if numerically different) implementation of the same model.
Actual
Mean cosine similarity across 20 sentence pairs: -0.0228 (range approx. -0.10 to +0.05, n=20). For 768-dimensional unit vectors this is indistinguishable from two unrelated random vectors (expected cosine ~0, stddev ~1/sqrt(768) ~= 0.036).
Two further, independent checks show that v1.0.4's output, not v0.9.39's, is the wrong one:
- v0.9.39's embeddings for the same sentences matched embeddings recorded from an earlier v0.9.39 run of the identical model and prompts: mean cosine 0.9996.
- A from-scratch reference implementation of
embed-gemma:300m (mean-token pooling -> two Dense re-projection layers -> L2 normalize, per the model's published config) disagreed with live v1.0.4 output at mean cosine -0.0378, again statistically random.
Workaround
Pin to v0.9.39 for embed-gemma:300m embedding use; v1.0.4's embedding output for this model should not be trusted until fixed.
Operating System
Microsoft Windows 11 Education, version 10.0.26200
CPU
AMD Ryzen AI MAX+ 395 w/ Radeon 8060S (Framework Desktop)
GPU
AMD Radeon 8060S Graphics (integrated). The issue is on the NPU path: "NPU Compute Accelerator Device" (XDNA), which is what FastFlowLM's --embed 1 runs on.
ROCm Version
Not applicable: Windows, FastFlowLM binary release running on the NPU; no ROCm installed.
Installation Method
Other (describe in Additional Information)
Installed ROCm Packages / Versions
Installed ROCm packages / versions
None (no ROCm; FastFlowLM Windows binaries only: v1.0.4 and v0.9.39).
ROCm Component
No response
Steps to Reproduce
No response
(Optional for Linux users) Output of rocminfo --support
No response
Additional Information
Prebuilt Windows release zip from this repository's GitHub Releases: fastflowlm_1.0.4_windows_amd64.zip (bad) and fastflowlm_0.9.39_windows_amd64.zip (good), sha256-verified, extracted to separate directories.
Identical on-disk weights for both versions; only the binary differs.
Problem Description
/v1/embeddingsoutput forembed-gemma:300m(EmbeddingGemma-300M, loaded withflm serve <chat_tag> --embed 1) changed between v0.9.39 and v1.0.4 in a way that is not a numerically-different-but-equivalent re-encoding: it is statistically uncorrelated with the correct embedding for the same input text.Identical on-disk model weights were used for both versions (the weights cache is shared across binary versions; both binaries resolve the same installed weight files). Only the binary differs.
Expected
Cosine similarity close to 1.0 between the v0.9.39 and v1.0.4 embeddings of the same sentence, if v1.0.4 is a correct (even if numerically different) implementation of the same model.
Actual
Mean cosine similarity across 20 sentence pairs: -0.0228 (range approx. -0.10 to +0.05, n=20). For 768-dimensional unit vectors this is indistinguishable from two unrelated random vectors (expected cosine ~0, stddev ~1/sqrt(768) ~= 0.036).
Two further, independent checks show that v1.0.4's output, not v0.9.39's, is the wrong one:
embed-gemma:300m(mean-token pooling -> two Dense re-projection layers -> L2 normalize, per the model's published config) disagreed with live v1.0.4 output at mean cosine -0.0378, again statistically random.Workaround
Pin to v0.9.39 for
embed-gemma:300membedding use; v1.0.4's embedding output for this model should not be trusted until fixed.Operating System
Microsoft Windows 11 Education, version 10.0.26200
CPU
AMD Ryzen AI MAX+ 395 w/ Radeon 8060S (Framework Desktop)
GPU
AMD Radeon 8060S Graphics (integrated). The issue is on the NPU path: "NPU Compute Accelerator Device" (XDNA), which is what FastFlowLM's --embed 1 runs on.
ROCm Version
Not applicable: Windows, FastFlowLM binary release running on the NPU; no ROCm installed.
Installation Method
Other (describe in Additional Information)
Installed ROCm Packages / Versions
Installed ROCm packages / versions
ROCm Component
No response
Steps to Reproduce
No response
(Optional for Linux users) Output of rocminfo --support
No response
Additional Information
Prebuilt Windows release zip from this repository's GitHub Releases: fastflowlm_1.0.4_windows_amd64.zip (bad) and fastflowlm_0.9.39_windows_amd64.zip (good), sha256-verified, extracted to separate directories.
Identical on-disk weights for both versions; only the binary differs.