Skip to content

Refresh recommended LLM models from OpenHands Index #560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
all-hands-bot wants to merge 1 commit into
base: main
Choose a base branch
from
+12 −12
Open

Refresh recommended LLM models from OpenHands Index #560

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions openhands/usage/llms/llms.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
---

<Note>
This section is for users who want to connect OpenHands to different LLMs.

Check warning on line 7 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L7 

Did you really mean 'LLMs'?
</Note>

<Info>
Expand All @@ -25,23 +25,23 @@

### Best Cloud Models by Family

| Family | Recommended Model | Model String | OpenHands Index Average | Notes |
|--------|-------------------|--------------|-------------------------|-------|
| Claude | [Claude Opus 4.7](https://github.com/OpenHands/openhands-index-results/tree/main/results/claude-opus-4-7) | `anthropic/claude-opus-4-7` | 68.2 | Best Claude-series result in the OpenHands Index. Use it for complex, long-running software work. Claude Opus 4.6 is close behind at 66.7. |
| GPT | [GPT-5.5](https://github.com/OpenHands/openhands-index-results/tree/main/results/GPT-5.5) | `openai/gpt-5.5` | 65.9 | Best GPT-series result in the OpenHands Index. GPT-5.4 is close behind at 64.3. |
| Gemini | [Gemini 3.1 Pro](https://github.com/OpenHands/openhands-index-results/tree/main/results/Gemini-3.1-Pro) | `gemini/gemini-3.1-pro-preview` | 57.0 | Best Gemini-series result in the OpenHands Index. Use Gemini 3 Flash when cost or latency is more important than top accuracy. |
| Family | Recommended Model | Model String | OpenHands Index Average |
|--------|-------------------|--------------|-------------------------|
| Claude | [claude-opus-4-8](https://github.com/OpenHands/openhands-index-results/tree/main/results/claude-opus-4-8) | Not yet listed | 71.9 |
| GPT | [GPT-5.5](https://github.com/OpenHands/openhands-index-results/tree/main/results/GPT-5.5) | `openai/gpt-5.5` | 65.9 |
| Gemini | [Gemini-3.1-Pro](https://github.com/OpenHands/openhands-index-results/tree/main/results/Gemini-3.1-Pro) | `gemini/gemini-3.1-pro-preview` | 57.0 |

### Strong Open / Open-Weight Models

These open or open-weight models have good OpenHands Index scores or are recommended for local OpenHands setups:

| Model | Suggested Model String | OpenHands Index Average | Notes |
|-------|------------------------|-------------------------|-------|
| [GLM-5.1](https://github.com/OpenHands/openhands-index-results/tree/main/results/GLM-5.1) | `openrouter/z-ai/glm-5.1` | 58.2 | Strongest open-weight result currently listed in the OpenHands Index. |
| [Kimi-K2.6](https://github.com/OpenHands/openhands-index-results/tree/main/results/Kimi-K2.6) | `openrouter/moonshotai/kimi-k2.6` | 57.1 | Strong open-weight option, especially for coding and information-gathering tasks. |
| [DeepSeek-V4-Pro](https://github.com/OpenHands/openhands-index-results/tree/main/results/DeepSeek-V4-Pro) | `openrouter/deepseek/deepseek-v4-pro` | 51.3 | Strong coding and test-generation scores; current Index entry covers three benchmarks. |
| [MiniMax-M2.7](https://github.com/OpenHands/openhands-index-results/tree/main/results/MiniMax-M2.7) | `openrouter/minimax/minimax-m2.7` | 43.4 | Recommended as a lower-cost open-weight option with strong SWE-bench and SWT-bench scores. Also available from MiniMax-compatible OpenAI endpoints as `openai/MiniMax-M2.7`. |
| [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | `openai/Qwen3.6-35B-A3B` for local OpenAI-compatible servers, or `openrouter/qwen/qwen3.6-35b-a3b` through OpenRouter | Not yet listed | Recommended local / self-hosted model for OpenHands. It is open-weight, supports a large context window, and is featured in the [local LLM guide](/openhands/usage/llms/local-llms). |
| Model | Suggested Model String | OpenHands Index Average |
|-------|------------------------|-------------------------|
| [GLM-5.1](https://github.com/OpenHands/openhands-index-results/tree/main/results/GLM-5.1) | `openrouter/z-ai/glm-5.1` | 58.2 |
| [Kimi-K2.6](https://github.com/OpenHands/openhands-index-results/tree/main/results/Kimi-K2.6) | `openrouter/moonshotai/kimi-k2.6` | 57.1 |
| [GLM-5](https://github.com/OpenHands/openhands-index-results/tree/main/results/GLM-5) | `openrouter/z-ai/glm-5` | 49.4 |
| [Kimi-K2.5](https://github.com/OpenHands/openhands-index-results/tree/main/results/Kimi-K2.5) | `openrouter/moonshotai/kimi-k2.5` | 49.2 |
| [DeepSeek-V3.2-Reasoner](https://github.com/OpenHands/openhands-index-results/tree/main/results/DeepSeek-V3.2-Reasoner) | `openrouter/deepseek/deepseek-v3.2-reasoner` | 45.7 |

<Note>
Hosted model strings can vary by provider and region. If a model string is not accepted, check the provider console and
Expand All @@ -55,7 +55,7 @@
[litellm documentation](https://docs.litellm.ai/docs/providers).

<Warning>
OpenHands will issue many prompts to the LLM you configure. Most of these LLMs cost money, so be sure to set spending

Check warning on line 58 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L58 

Did you really mean 'LLMs'?
limits and monitor usage.
</Warning>

Expand All @@ -63,7 +63,7 @@

For local and self-hosted usage, start with
[Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B). See the
[local LLM guide](/openhands/usage/llms/local-llms) for LM Studio, Ollama, SGLang, and vLLM setup examples.

Check warning on line 66 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L66 

Did you really mean 'Ollama'?

Check warning on line 66 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L66 

Did you really mean 'SGLang'?

### Known Issues

Expand Down Expand Up @@ -115,7 +115,7 @@

LLM providers have specific settings that can be customized to optimize their performance with OpenHands, such as:

- **Custom Tokenizers**: For specialized models, you can add a suitable tokenizer.

Check warning on line 118 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L118 

Did you really mean 'Tokenizers'?

Check warning on line 118 in openhands/usage/llms/llms.mdx

View check run for this annotation

Mintlify / Mintlify Validation (allhandsai) - vale-spellcheck

openhands/usage/llms/llms.mdx#L118 

Did you really mean 'tokenizer'?
- **Native Tool Calling**: Toggle native function/tool calling capabilities.

For detailed information about model customization, see
Expand Down
Loading