[Currently in development!]

Embeddings-Division

Python script for dividing Embedding layer of LLM.

The solution was tested on meta-llama/Llama-3.2-1B.

Installation

Installation from PyPI:

pip install embdiv

Requirements

• Python 3.9+
• cachetools>=5.5.0,
• torch>=2.5.1,
• transformers>=4.45.2,

Docs

https://embeddings-division.readthedocs.io/en/latest/