LM Studio support?

[TofPlay]

LM Studio is a really interesting tool for being able to load different models locally.
With AnyLanguageModel it's possible to use the APIs they offer => LM Studio REST API?

[mattt]

Thanks for suggesting this feature, @TofPlay.

It looks like AnyLanguageModel can already interact with LM Studio through its OpenAI and Anthropic compatible endpoints:

In addition to our native API, we provide OpenAI-compatible endpoints (learn more) and Anthropic-compatible endpoints (learn more).

Of these, OpenAI Responses seems the most capable, so I'd recommend OpenAILanguageModel.

If you need more control over model management, it should be pretty straightforward to write your own LanguageModel subclass. I reckon GPT 5.3 Codex or Opus 4.6 could one-shot that if you fed in OpenAILanguageModel.swift and the LM Studio API docs.

As for bundling that into AnyLanguageModel itself? I'm not sure. I'd like to see some more demand or a real need articulated. There's no shortage of inference providers these days, so it's mostly a line-drawing exercise to see what's included.

[TofPlay]

As for bundling that into AnyLanguageModel itself? I'm not sure. I'd like to see some more demand or a real need articulated.

For American companies, using AI in the Cloud is not a problem. However, it is for European companies (most of them don't want to send sensitive data to AI in the Cloud). Solutions like LM Studio or Ollama are therefore preferred. AnyLanguageModel is interesting because it allows us to directly load MLX and llama.cpp models. But in practice, we have also to use the AI already installed via LM Studio or Ollama.

[mattt]

@TofPlay I'm 100% in agreement with you about the merit of running open-source models locally. It's an explicit goal of this project to support that use case.

By "real need", I meant that in the technical sense: whether the existing APIs are insufficient for using LM Studio with AnyLanguageModel. If the existing OpenAI-compatible API is sufficient (as is the case for Hugging Face Inference Providers, Open Router, Z.AI, and other providers), then we can point users to that. If it's not possible to use LM Studio inference without programmatic model management, then that's a strong case for creating a separate LanguageModel subclass. Then it's a matter of deciding whether enough consumers of AnyLanguageModel would benefit from having that as a built-in option.

Does that make sense?