Take a look at BigMoeOnEdge

[Markus40]

There is this project called BigMoeOnEdge which uses llamacpp but streamlines the loading of MoE weights from an SSD on mobile devices. It works much better than letting llamacpp and the OS handle it with file caching alone. Bigger models become somewhat usable. The project is still in its infancy, but if it has staying power, it could be an interesting alternative backend.