Compatibility between fast-plaid and next-plaid indices

[pau-mensa]

I've been trying to reuse indices that I created with fast-plaid but I haven't had success so far. The main README of next-plaid seems to imply that the indices are compatible, but here's what I've been able to find so far:

• Residuals in fast-plaid are stored in little endian (<u1), while in next-plaid they are stored in |u1. This should be compatible (they are unsigned 8-bits after all), but seems like the memory mapping you built only accepts |u1.
• Centroids in fast-plaid are stored in fp16, and in next-plaid they are stored in fp32.
• next-plaid expects a config.json. fast-plaid never creates a config.json.
• fast-plaid stores the embeddings.npy as a pickled object, while next-plaid uses npy.
• next-plaid expects an embeddings_lengths.json (related to the previous item). Since fast-plaid uses pickled objects it never creates the embeddings_lengths.json file

Question

Do you expect compatibility between fast-plaid and next-plaid at some point or should I just stick with next-plaid and ditch fast-plaid altogether? If you don't want to support compatibility between the two libraries (which is fair in my opinion) I think you should make it a bit more clear in the README, especially since there is already a NextPlaid vs FastPlaid section.

Thanks!

Note

I did this comparison with next-plaid==1.1.3 and fast-plaid=1.4.3

[raphaelsty]

Hi @pau-mensa, actually I do plan to integrate next-plaid within fast-plaid for cpu user in the following week, we will stick to fast-plaid when cuda is enabled. I think fast-plaid backend gpu is faster than next-plaid right now. Next-Plaid on cpu is much faster than fast-plaid on cpu. There are a few conversion to make like you mentionned

I had drafted something but with various recent updates I did not have time to assert both a still compatible.

But the goal is to get best of both world into fast-plaid by default (next-plaid for cpu, pytorch for gpu).

I want the compatibility for PyLate user and for FastPlaid users to perform actual inference on cpu at a decent speed rate. The idea would be to be able to build an index in offline with fast-plaid on gpu and serve it with next-plaid as well

[pau-mensa]

That sounds amazing @raphaelsty. It has been a bit hard to move from fast-plaid to next-plaid, mostly because of the index creation (although I am on a stable state now), so when this parity you mention is ready I'll be very happy to use it!

On the other hand and unrelated to this, I think there is an issue with how you handle memory in fastkmeans-rs. I had to fork next-plaid and add the possibility of creating an index with pre-computed centroids (using the fastkmeans library). I haven't had time to investigate properly, so I don't have enough to open an issue, but I was getting OOM errors on an L40S with 900 documents of 900 tokens, and after moving to fastkmeans I wasn't anymore. Just letting you know...