Skip to content

[Bug]: Support max_completion_tokens for newer OpenAI models #153

Description

@ochnios

Bug Description

Ask O11y currently sends max_tokens in requests to the Grafana LLM plugin:

  • pkg/agent/types.go serializes MaxTokens as max_tokens
  • pkg/agent/loop.go sets this field for every agent LLM request

Grafana LLM app seems to reject this parameter for newer models and require max_completion_tokens instead. This causes agent runs to fail with this error:

Something went wrong
Grafana LLM app returned an internal error. Ask an admin to check grafana-llm-app provider configuration and backend logs.

and Grafana LLM app logging:

logger=context userId=20 orgId=1 uname=<redacted> t=<redacted> level=error msg="Request Completed" method=POST path=/api/plugins/grafana-llm-app/resources/openai/v1/chat/completions status=500 remote_addr=<redacted> time_ms=21 duration=21.5068ms size=81 referer= handler=/api/plugins/:pluginId/resources/* status_source=downstream
logger=plugin.grafana-llm-app t=<redacted> level=error msg="error establishing stream" err="this model is not supported MaxTokens, please use MaxCompletionTokens"

Steps to Reproduce

  1. Navigate to Administration > Plugins and data > Plugins > LLM > Configuration
  2. Choose "Use OpenAI-compatible API" with "Azure OpenAI"
  3. Configure API keys and model mappings
  4. Compare results for some older and newer model (e.g. gpt-4.1 vs gpt-5.6-sol) - it works for the first one but fails for the second one

Expected Behavior

Ask O11y should send max_completion_tokens instead of max_tokens and not fail with newer OpenAI models

Actual Behavior

Ask O11y send max_tokens and Grafana LLM plugin requests fail

Grafana Version

v13.0.1

Plugin Version

0.2.36

User Role

Admin

Browser

Edge

Operating System

Windows 11

Browser Console Errors

Grafana Server Logs

logger=context userId=20 orgId=1 uname=<redacted> t=<redacted> level=error msg="Request Completed" method=POST path=/api/plugins/grafana-llm-app/resources/openai/v1/chat/completions status=500 remote_addr=<redacted> time_ms=21 duration=21.5068ms size=81 referer= handler=/api/plugins/:pluginId/resources/* status_source=downstream
logger=plugin.grafana-llm-app t=<redacted> level=error msg="error establishing stream" err="this model is not supported MaxTokens, please use MaxCompletionTokens"

Configuration

No response

Screenshots

Image

Additional Context

No response

Pre-submission Checklist

  • I have searched existing issues to ensure this is not a duplicate
  • I have provided all required information above
  • I have included browser console errors (if applicable)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions