Simplify LLM settings around profiles and remove raw LLM config from settings surfaces

[enyst]

Summary

Simplify LLM configuration so there is one persistent/user-facing source of truth: LLM profiles. agent_settings, generic settings REST payloads, conversation-start payloads, Canvas/CLI settings state, and automations should not carry raw llm settings or API keys.

Proposed invariant:

• LLM profiles store model/base URL/API key and related LLM options.
• Agent settings reference a profile, or use the active/default profile, instead of embedding llm.
• Python API raw construction still works (LLM(...), Agent(llm=...)). When a raw Python-created LLM/Agent crosses a persistence/REST boundary, materialize it as a default profile immediately and persist only the profile reference.
• Runtime model switching uses profile references (switch_profile) rather than raw LLM JSON.

Repos investigated:

• OpenHands/software-agent-sdk
• OpenHands/agent-canvas
• OpenHands/automation
• OpenHands/OpenHands
• OpenHands/OpenHands-CLI

This issue was created by an AI agent (OpenHands) on behalf of the user.

Findings with source links and snippets

1. SDK / agent-server: agent_settings.llm is still the canonical runtime config

OpenHandsAgentSettings embeds an LLM and create_agent() passes it directly into Agent.

Source:

software-agent-sdk/openhands-sdk/openhands/sdk/settings/model.py

Lines 766 to 801 in d594db8

	class OpenHandsAgentSettings(AgentSettingsBase):
	"""Settings for a standard LLM-backed :class:`Agent`.
	This is the long-standing ``AgentSettings`` shape; fields here build
	the default ``Agent`` (LLM + tools + MCP + condenser + critic).
	"""
	agent_kind: Literal["openhands"] = Field(
	default="openhands",
	description=(
	"Discriminator for the ``AgentSettings`` union. ``'openhands'`` selects "
	"the standard built-in OpenHands agent."
	),
	)
	agent: str = Field(
	default="CodeActAgent",
	description="Agent class to use.",
	json_schema_extra={
	SETTINGS_METADATA_KEY: SettingsFieldMetadata(
	label="Agent",
	prominence=SettingProminence.MAJOR,
	variant="openhands",
	).model_dump()
	},
	)
	llm: LLM = Field(
	default_factory=_default_llm_settings,
	description="LLM settings for the agent.",
	json_schema_extra={
	SETTINGS_SECTION_METADATA_KEY: SettingsSectionMetadata(
	key="llm",
	label="LLM",
	variant="openhands",
	).model_dump()
	},
	)

class OpenHandsAgentSettings(AgentSettingsBase):
    ...
    llm: LLM = Field(
        default_factory=_default_llm_settings,
        description="LLM settings for the agent.",
        json_schema_extra={...},
    )

Source:

software-agent-sdk/openhands-sdk/openhands/sdk/settings/model.py

Lines 895 to 927 in d594db8

	def create_agent(self) -> Agent:
	"""Build an :class:`Agent` purely from these settings.
	Example::
	settings = OpenHandsAgentSettings(
	llm=LLM(model="m", api_key="k"),
	tools=[Tool(name="TerminalTool")],
	)
	agent = settings.create_agent()
	"""
	from openhands.sdk.agent import Agent
	from openhands.sdk.tool.builtins import BUILT_IN_TOOLS, SwitchLLMTool
	# Bypass ``_serialize_mcp_config``: MCP servers need real env/headers.
	mcp_config = (
	self.mcp_config.model_dump(exclude_none=True, exclude_defaults=True)
	if self.mcp_config is not None
	else {}
	)
	include_default_tools = [tool.__name__ for tool in BUILT_IN_TOOLS]
	if self.enable_switch_llm_tool:
	include_default_tools.append(SwitchLLMTool.__name__)
	return Agent(
	llm=self.llm,
	tools=self.tools,
	mcp_config=mcp_config,
	include_default_tools=include_default_tools,
	agent_context=self.agent_context,
	condenser=self.build_condenser(self.llm),
	critic=self.build_critic(),
	)

return Agent(
    llm=self.llm,
    tools=self.tools,
    mcp_config=mcp_config,
    include_default_tools=include_default_tools,
    agent_context=self.agent_context,
    condenser=self.build_condenser(self.llm),
    critic=self.build_critic(),
)

Agent-server persisted settings store both agent_settings and active_profile; llm_api_key_is_set is derived from agent_settings.llm.api_key.

Source:

software-agent-sdk/openhands-agent-server/openhands/agent_server/persistence/models.py

Lines 104 to 139 in d594db8

	"""Persisted settings for agent server.
	Agent settings (LLM config, MCP config, condenser) live in ``agent_settings``.
	Conversation settings (max_iterations, confirmation_mode) live in
	``conversation_settings``.
	The ``active_profile`` field tracks which LLM profile was last activated,
	allowing frontends to display which profile is currently in use.
	"""
	schema_version: int = Field(
	default=PERSISTED_SETTINGS_SCHEMA_VERSION,
	description="Persisted settings file schema version.",
	)
	agent_settings: AgentSettingsConfig = Field(default_factory=default_agent_settings)
	conversation_settings: ConversationSettings = Field(
	default_factory=ConversationSettings
	)
	active_profile: str | None = Field(
	default=None,
	description="Name of the currently active LLM profile.",
	)
	model_config = ConfigDict(populate_by_name=True)
	@property
	def llm_api_key_is_set(self) -> bool:
	"""Check if an LLM API key is configured."""
	raw = self.agent_settings.llm.api_key
	if raw is None:
	return False
	secret_value = (
	raw.get_secret_value() if isinstance(raw, SecretStr) else str(raw)
	)
	return bool(secret_value and secret_value.strip())

agent_settings: AgentSettingsConfig = Field(default_factory=default_agent_settings)
conversation_settings: ConversationSettings = Field(default_factory=ConversationSettings)
active_profile: str | None = Field(default=None)

@property
def llm_api_key_is_set(self) -> bool:
    raw = self.agent_settings.llm.api_key

2. Agent-server REST settings/profile APIs expose and mutate raw LLM settings

GET /api/settings can return encrypted/plaintext LLM secrets via X-Expose-Secrets; PATCH /api/settings accepts agent_settings_diff, including nested llm.

Source:

software-agent-sdk/openhands-agent-server/openhands/agent_server/settings_router.py

Lines 103 to 163 in d594db8

	@settings_router.get(SETTINGS_PATH, response_model=SettingsResponse)
	async def get_settings(request: Request) -> SettingsResponse:
	"""Get current settings.
	Returns the persisted settings including agent configuration,
	conversation settings, and whether an LLM API key is configured.
	Use the ``X-Expose-Secrets`` header to control secret exposure:
	- ``encrypted``: Returns cipher-encrypted values (safe for frontend clients)
	- ``plaintext``: Returns raw secret values (backend clients only!)
	- (absent): Returns redacted values ("**********")
	Security:
	When the server is configured with ``session_api_keys``, all endpoints
	under ``/api`` (including this one) require the ``X-Session-API-Key``
	header. When no session API keys are configured, endpoints are open.
	**Trust model:** All authenticated clients are treated as equally
	trusted. There is no role-based authorization for ``X-Expose-Secrets``
	modes—any authenticated client can request ``plaintext`` or
	``encrypted`` exposure. This design assumes:
	- All clients sharing session API keys operate in the same trust domain
	- Network-level controls (firewalls, VPCs) restrict access to trusted
	clients only
	- Production deployments use session API keys to prevent anonymous access
	The ``plaintext`` mode exists for backend-to-backend communication
	(e.g., RemoteWorkspace). Frontend clients should prefer ``encrypted``
	mode for round-tripping secrets, or omit the header to receive redacted
	values.
	"""
	expose_mode = parse_expose_secrets_header(request)
	config = get_config(request)
	store = get_settings_store(config)
	settings = store.load() or PersistedSettings()
	# Audit log all settings access for security visibility
	# Use WARNING level for plaintext mode to highlight security-sensitive operations
	client_host = request.client.host if request.client else "unknown"
	log_extra = {
	"client_host": client_host,
	"expose_mode": expose_mode or "redacted",
	"has_llm_api_key": settings.llm_api_key_is_set,
	}
	if expose_mode == "plaintext":
	logger.warning("Settings accessed with PLAINTEXT secrets", extra=log_extra)
	else:
	logger.info("Settings accessed", extra=log_extra)
	context = build_expose_context(expose_mode, config.cipher)
	with translate_missing_cipher():
	return SettingsResponse(
	agent_settings=settings.agent_settings.model_dump(
	mode="json", context=context
	),
	conversation_settings=settings.conversation_settings.model_dump(
	mode="json"
	),
	llm_api_key_is_set=settings.llm_api_key_is_set,
	)

@settings_router.get(SETTINGS_PATH, response_model=SettingsResponse)
async def get_settings(request: Request) -> SettingsResponse:
    expose_mode = parse_expose_secrets_header(request)
    ...
    return SettingsResponse(
        agent_settings=settings.agent_settings.model_dump(mode="json", context=context),
        conversation_settings=settings.conversation_settings.model_dump(mode="json"),
        llm_api_key_is_set=settings.llm_api_key_is_set,
    )

Profile save stores raw LLM objects, and activation copies the profile back into agent_settings.llm, preserving duplication.

Source:

software-agent-sdk/openhands-agent-server/openhands/agent_server/profiles_router.py

Lines 68 to 73 in d594db8

	class SaveProfileRequest(BaseModel):
	llm: LLM
	include_secrets: bool = Field(
	default=True,
	description="Whether to persist the API key with the profile.",
	)

class SaveProfileRequest(BaseModel):
    llm: LLM
    include_secrets: bool = Field(default=True)

Source:

software-agent-sdk/openhands-agent-server/openhands/agent_server/profiles_router.py

Lines 280 to 341 in d594db8

	@profiles_router.post("/{name}/activate", response_model=ActivateProfileResponse)
	async def activate_profile(
	request: Request, name: ProfileName
	) -> ActivateProfileResponse:
	"""Activate a saved LLM profile.
	This endpoint:
	1. Loads the named profile's LLM configuration
	2. Applies it to the current agent settings (updates ``agent_settings.llm``)
	3. Records the profile name as the active profile for frontend tracking
	Returns 404 if the profile does not exist.
	Use ``GET /api/profiles`` to see which profile is currently active via
	the ``active_profile`` field.
	"""
	cipher = get_cipher(request)
	config = get_config(request)
	# Load the profile
	profile_store = LLMProfileStore()
	try:
	with _store_errors():
	llm = profile_store.load(name, cipher=cipher)
	except FileNotFoundError:
	raise HTTPException(
	status_code=status.HTTP_404_NOT_FOUND,
	detail=f"Profile '{name}' not found",
	)
	# Apply the LLM config to settings and record active profile
	settings_store = get_settings_store(config)
	def apply_profile(settings: PersistedSettings) -> PersistedSettings:
	# Update the LLM configuration
	llm_dict = llm.model_dump(mode="json", context={"expose_secrets": "plaintext"})
	settings.update(
	{
	"agent_settings_diff": {"llm": llm_dict},
	"active_profile": name,
	}
	)
	return settings
	try:
	settings_store.update(apply_profile)
	except (OSError, PermissionError):
	logger.error("Failed to activate profile - file I/O error")
	raise HTTPException(status_code=500, detail="Failed to activate profile")
	except RuntimeError as e:
	logger.error(f"Failed to activate profile: {e}")
	raise HTTPException(
	status_code=status.HTTP_409_CONFLICT,
	detail="Settings file is corrupted or encrypted with a different key",
	)
	logger.info(f"Activated profile '{name}'")
	return ActivateProfileResponse(
	name=name,
	message=f"Profile '{name}' activated and applied to current settings",
	llm_applied=True,
	)

def apply_profile(settings: PersistedSettings) -> PersistedSettings:
    llm_dict = llm.model_dump(mode="json", context={"expose_secrets": "plaintext"})
    settings.update({
        "agent_settings_diff": {"llm": llm_dict},
        "active_profile": name,
    })

3. Conversation start can be driven by raw agent_settings

StartConversationRequest accepts agent_settings and constructs the agent in a validator. This is the main Canvas/REST path to replace with profile-reference startup.

Source:

software-agent-sdk/openhands-sdk/openhands/sdk/conversation/request.py

Lines 202 to 239 in d594db8

	agent_settings: dict[str, Any] | None = Field(
	default=None,
	exclude=True,
	description=(
	"Optional agent settings payload. If `agent` is omitted, this is "
	"validated with the AgentSettingsBase `agent_kind` discriminator and "
	"used to construct the concrete agent."
	),
	)
	agent: AgentBase = Field(default=cast(AgentBase, None))
	@model_validator(mode="before")
	@classmethod
	def _populate_agent_from_settings(cls, data: Any) -> Any:
	if not isinstance(data, dict):
	return data
	payload = dict(data)
	if payload.get("agent") is None and payload.get("agent_settings") is not None:
	from openhands.sdk.settings.model import validate_agent_settings
	try:
	payload["agent"] = validate_agent_settings(
	payload["agent_settings"]
	).create_agent()
	except (TypeError, ValueError) as exc:
	raise ValueError(str(exc)) from exc
	elif isinstance(payload.get("agent"), dict):
	agent_payload = dict(payload["agent"])
	if "kind" not in agent_payload and "llm" in agent_payload:
	agent_payload["kind"] = "Agent"
	payload["agent"] = agent_payload
	return payload
	@model_validator(mode="after")
	def _require_agent(self) -> StartConversationRequest:
	if self.agent is None:
	raise ValueError("Either `agent` or `agent_settings` must be provided")
	return self

agent_settings: dict[str, Any] | None = Field(default=None, exclude=True)
agent: AgentBase = Field(default=cast(AgentBase, None))

@model_validator(mode="before")
def _populate_agent_from_settings(cls, data: Any) -> Any:
    if payload.get("agent") is None and payload.get("agent_settings") is not None:
        payload["agent"] = validate_agent_settings(
            payload["agent_settings"]
        ).create_agent()

4. Agent-server still has raw switch_llm

There is already profile-based switching, but a raw LLM endpoint remains for app-server compatibility.

Source:

software-agent-sdk/openhands-agent-server/openhands/agent_server/conversation_router.py

Lines 339 to 394 in d594db8

	@conversation_router.post(
	"/{conversation_id}/switch_profile",
	responses={
	400: {"description": "Invalid or corrupted profile"},
	404: {"description": "Conversation or profile not found"},
	},
	)
	async def switch_conversation_profile(
	conversation_id: UUID,
	profile_name: str = Body(..., embed=True),
	conversation_service: ConversationService = Depends(get_conversation_service),
	) -> Success:
	"""Switch the conversation's LLM profile to a named profile."""
	event_service = await conversation_service.get_event_service(conversation_id)
	if event_service is None:
	raise HTTPException(status.HTTP_404_NOT_FOUND)
	conversation = event_service.get_conversation()
	try:
	conversation.switch_profile(profile_name)
	except FileNotFoundError:
	raise HTTPException(
	status_code=status.HTTP_404_NOT_FOUND,
	detail=f"Profile '{profile_name}' not found",
	)
	except ValueError as e:
	raise HTTPException(
	status_code=status.HTTP_400_BAD_REQUEST,
	detail=str(e),
	)
	return Success()
	@conversation_router.post(
	"/{conversation_id}/switch_llm",
	responses={404: {"description": "Conversation not found"}},
	)
	async def switch_conversation_llm(
	request: Request,
	conversation_id: UUID,
	llm: LLM = Body(..., embed=True), # noqa: B008
	conversation_service: ConversationService = Depends(get_conversation_service),
	) -> Success:
	"""Swap the conversation's LLM to a caller-supplied object.
	Used by app-servers that own the LLM directly and don't push profiles
	to the agent-server's filesystem (see #3017).
	"""
	event_service = await conversation_service.get_event_service(conversation_id)
	if event_service is None:
	raise HTTPException(status.HTTP_404_NOT_FOUND)
	conversation = event_service.get_conversation()
	cipher = get_cipher(request)
	if cipher is not None:
	llm = decrypt_incoming_llm_secrets(llm, cipher)
	conversation.switch_llm(llm)
	return Success()

@conversation_router.post("/{conversation_id}/switch_profile")
async def switch_conversation_profile(...):
    conversation.switch_profile(profile_name)

@conversation_router.post("/{conversation_id}/switch_llm")
async def switch_conversation_llm(..., llm: LLM = Body(..., embed=True)) -> Success:
    conversation.switch_llm(llm)

5. SDK workspace helpers fetch plaintext LLM settings

RemoteWorkspace.get_llm() reads /api/settings with X-Expose-Secrets: plaintext and returns settings.llm.

Source:

software-agent-sdk/openhands-sdk/openhands/sdk/workspace/remote/base.py

Lines 341 to 424 in d594db8

	def _fetch_agent_settings(
	self,
	) -> "OpenHandsAgentSettings | LLMAgentSettings | ACPAgentSettings":
	"""Call ``GET /api/settings`` and return a validated settings model.
	Uses ``X-Expose-Secrets: plaintext`` so secret fields (e.g. LLM
	api_key) are returned as plain strings. The outer response is
	validated via :class:`SettingsResponse`, then the ``agent_settings``
	dict is validated through :meth:`SettingsResponse.get_agent_settings`,
	which applies the persisted settings migration entry point before
	picking the correct discriminated-union variant
	(``OpenHandsAgentSettings`` or ``ACPAgentSettings``).
	"""
	headers = dict(self._headers)
	headers["X-Expose-Secrets"] = "plaintext"
	response = self.client.get("/api/settings", headers=headers)
	response.raise_for_status()
	data = SettingsResponse.model_validate(response.json())
	return data.get_agent_settings()
	def _fetch_llm_profile_config(self, profile_name: str) -> dict[str, Any]:
	"""Call ``GET /api/profiles/{name}`` and return plaintext LLM config."""
	headers = dict(self._headers)
	headers["X-Expose-Secrets"] = "plaintext"
	response = self.client.get(
	f"/api/profiles/{quote(profile_name, safe='')}",
	headers=headers,
	)
	if response.status_code == 404:
	raise FileNotFoundError(f"LLM profile '{profile_name}' not found")
	response.raise_for_status()
	config = response.json().get("config")
	if not isinstance(config, dict):
	raise ValueError(f"LLM profile '{profile_name}' has invalid config")
	return dict(config)
	@tenacity.retry(
	stop=tenacity.stop_after_attempt(_MAX_RETRIES),
	wait=tenacity.wait_exponential(multiplier=1, min=1, max=5),
	retry=tenacity.retry_if_exception(_is_retryable_error),
	reraise=True,
	)
	def get_llm(self, profile_name: str | None = None, **llm_kwargs: Any) -> "LLM":
	"""Fetch LLM settings from persisted settings or a named profile.
	Args:
	profile_name: Optional LLM profile name. When provided, loads that
	named profile instead of the active persisted LLM settings.
	**llm_kwargs: Additional keyword arguments that override persisted
	or profile values (e.g., ``model``, ``temperature``).
	Returns:
	An LLM instance configured with the persisted settings or profile.
	Raises:
	FileNotFoundError: If ``profile_name`` does not exist.
	httpx.HTTPStatusError: If the API request fails.
	RuntimeError: If the workspace host is not set.
	Example:
	>>> with DockerWorkspace(...) as workspace:
	... llm = workspace.get_llm(profile_name="fast")
	... agent = Agent(llm=llm, tools=get_default_tools())
	"""
	from openhands.sdk.llm.llm import LLM
	if not self.host or self.host == "undefined":
	raise RuntimeError("Workspace host is not set")
	if profile_name:
	llm_data = self._fetch_llm_profile_config(profile_name)
	llm_data["usage_id"] = f"profile:{profile_name}"
	else:
	settings = self._fetch_agent_settings()
	if not llm_kwargs:
	return settings.llm
	llm_data = settings.llm.model_dump(context={"expose_secrets": "plaintext"})
	llm_data.update(llm_kwargs)
	return LLM(**llm_data)

headers = dict(self._headers)
headers["X-Expose-Secrets"] = "plaintext"
response = self.client.get("/api/settings", headers=headers)
...
settings = self._fetch_agent_settings()
return settings.llm

OpenHandsCloudWorkspace.get_llm() reads SaaS user settings with expose_secrets=true and falls back to top-level llm_model / llm_api_key / llm_base_url.

Source:

software-agent-sdk/openhands-workspace/openhands/workspace/cloud/workspace.py

Lines 572 to 639 in d594db8

	def get_llm(self, profile_name: str | None = None, **llm_kwargs: Any) -> LLM:
	"""Fetch LLM settings from the user's SaaS account and return an LLM.
	Calls ``GET /api/v1/users/me?expose_secrets=true`` to retrieve the
	user's LLM configuration or a named LLM profile and returns a fully
	usable ``LLM`` instance. Retries up to 3 times on transient errors
	(network issues, server 5xx).
	Args:
	profile_name: Optional LLM profile name. When provided, loads that
	named profile instead of the default SaaS LLM fields.
	**llm_kwargs: Additional keyword arguments passed to the LLM
	constructor, allowing overrides of any LLM parameter
	(e.g. ``model``, ``temperature``).
	Returns:
	An LLM instance configured with the user's SaaS credentials.
	Raises:
	FileNotFoundError: If ``profile_name`` does not exist.
	httpx.HTTPStatusError: If the API request fails.
	RuntimeError: If the sandbox is not running.
	Example:
	>>> with OpenHandsCloudWorkspace(...) as workspace:
	... llm = workspace.get_llm(profile_name="fast")
	... agent = Agent(llm=llm, tools=get_default_tools())
	"""
	from openhands.sdk.llm.llm import LLM
	if not self._sandbox_id:
	raise RuntimeError("Sandbox is not running")
	resp = self._send_api_request(
	"GET",
	f"{self.cloud_api_url}/api/v1/users/me",
	params={"expose_secrets": "true"},
	headers={"X-Session-API-Key": self._session_api_key or ""},
	)
	data = resp.json()
	if profile_name:
	profiles_payload = data.get("llm_profiles") or {}
	profiles = (
	profiles_payload.get("profiles")
	if isinstance(profiles_payload, dict)
	else None
	)
	profile_config = (
	profiles.get(profile_name) if isinstance(profiles, dict) else None
	)
	if not isinstance(profile_config, dict):
	raise FileNotFoundError(f"LLM profile '{profile_name}' not found")
	kwargs = dict(profile_config)
	kwargs["usage_id"] = f"profile:{profile_name}"
	else:
	kwargs = {}
	if data.get("llm_model"):
	kwargs["model"] = data["llm_model"]
	if data.get("llm_api_key"):
	kwargs["api_key"] = data["llm_api_key"]
	if data.get("llm_base_url"):
	kwargs["base_url"] = data["llm_base_url"]
	# User-provided kwargs take precedence
	kwargs.update(llm_kwargs)
	return LLM(**kwargs)

resp = self._send_api_request(
    "GET",
    f"{self.cloud_api_url}/api/v1/users/me",
    params={"expose_secrets": "true"},
    headers={"X-Session-API-Key": self._session_api_key or ""},
)
...
if data.get("llm_api_key"):
    kwargs["api_key"] = data["llm_api_key"]
return LLM(**kwargs)

6. Agent Canvas sends encrypted agent_settings.llm to start conversations

Canvas keeps legacy top-level LLM compatibility fields and an embedded agent_settings.llm default.

Source: https://github.com/OpenHands/agent-canvas/blob/6cac0a0501e79df230c8046181f1d7577be26ef8/src/services/settings.ts#L5-L62

export const DEFAULT_SETTINGS: Settings = {
  llm_model: "openhands/minimax-m2.7",
  llm_base_url: "",
  llm_api_key: null,
  llm_api_key_set: false,
  ...
  agent_settings: {
    agent_kind: "openhands",
    agent: "CodeActAgent",
    llm: { model: "openhands/minimax-m2.7" },
  },
};

It fetches encrypted settings just to start conversations.

Source: https://github.com/OpenHands/agent-canvas/blob/6cac0a0501e79df230c8046181f1d7577be26ef8/src/api/settings-service/settings-service.api.ts#L302-L327

const response = await this.fetchSettingsFromApi("encrypted");
return {
  agentSettings: response.agent_settings,
  conversationSettings: response.conversation_settings,
  secretsEncrypted: true,
};

Then it passes those settings into agent_settings on the start request.

Source: https://github.com/OpenHands/agent-canvas/blob/6cac0a0501e79df230c8046181f1d7577be26ef8/src/api/agent-server-adapter.ts#L729-L778

const sourceAgentSettings = options.encryptedAgentSettings
  ? { ...options.settings, agent_settings: options.encryptedAgentSettings }
  : options.settings;
const agentSettings = buildConfiguredAgentSettings(sourceAgentSettings);

const payload: StartConversationPayload = {
  agent_settings: agentSettings,
  workspace: conversationSettings.workspace,
  ...
};

7. Automation is mostly profile-aligned, but generated scripts still pull raw LLM objects

Automation persists model profile names, which is good.

Source: https://github.com/OpenHands/automation/blob/074755a85a5d416fe0ef9359467ac53909b3cddf/openhands/automation/utils/model_profiles.py#L27-L39

def resolve_model_profile_for_user(requested_profile: str | None, user: AuthenticatedUser) -> str | None:
    """Automations store model profile names, not raw LLM settings."""
    model_profile = requested_profile or user.active_model_profile_name
    validate_model_profile_for_user(model_profile, user)
    return model_profile

Generated scripts still call workspace.get_llm() and build an agent from that raw LLM.

Source: https://github.com/OpenHands/automation/blob/074755a85a5d416fe0ef9359467ac53909b3cddf/openhands/automation/presets/prompt/sdk_main.py#L257-L299

llm = workspace.get_llm(profile_name=model_profile)
...
agent = get_default_agent(llm=llm, cli_mode=True)

8. Legacy OpenHands app-server already has profiles but duplicates them into settings and sends raw LLM to agent-server

OpenHands seeds profiles from old agent_settings.llm, but still persists both llm_profiles and agent_settings.

Source: https://github.com/OpenHands/OpenHands/blob/459b00986b28608bb4cd3b02469ae66b76c6a691/openhands/app_server/settings/file_settings_store.py#L17-L31

if 'llm_profiles' not in kwargs:
    legacy_llm = (kwargs.get('agent_settings') or {}).get('llm')
    if isinstance(legacy_llm, dict) and legacy_llm.get('model'):
        kwargs['llm_profiles'] = {
            'profiles': {'Default': legacy_llm},
            'active': 'Default',
        }

Source: https://github.com/OpenHands/OpenHands/blob/459b00986b28608bb4cd3b02469ae66b76c6a691/openhands/app_server/settings/settings_models.py#L308-L322

def switch_to_profile(self, name: str) -> None:
    llm = self.llm_profiles.require(name)
    self.agent_settings = self.agent_settings.model_copy(
        update={'llm': llm.model_copy()}
    )
    self.llm_profiles.active = name

Running cloud conversations switch models by POSTing raw llm JSON to the runtime agent-server.

Source: https://github.com/OpenHands/OpenHands/blob/459b00986b28608bb4cd3b02469ae66b76c6a691/openhands/app_server/app_conversation/app_conversation_router.py#L689-L700

llm_payload = profile_llm.model_dump(
    mode='json',
    exclude_none=True,
    context={'expose_secrets': True},
)
switch_response = await httpx_client.post(
    f'{ctx.agent_server_url}/api/conversations/{conversation_id}/switch_llm',
    json={'llm': llm_payload},
    headers=headers,
)

9. CLI persists full Agent JSON with LLM/API key and supports env-var LLM overrides

CLI stores serialized Agent JSON, including LLM config/secrets, in agent_settings.json.

Source: https://github.com/OpenHands/OpenHands-CLI/blob/3ca17446c5d9c1e35e054803478a3501ec251ecf/openhands_cli/stores/agent_store.py#L262-L275

def load_from_disk(self) -> Agent | None:
    str_spec = self.file_store.read(AGENT_SETTINGS_PATH)
    return Agent.model_validate_json(str_spec)

Source: https://github.com/OpenHands/OpenHands-CLI/blob/3ca17446c5d9c1e35e054803478a3501ec251ecf/openhands_cli/stores/agent_store.py#L473-L475

def save(self, agent: Agent) -> None:
    serialized_spec = agent.model_dump_json(context={"expose_secrets": True})
    self.file_store.write(AGENT_SETTINGS_PATH, serialized_spec)

Direct env override mode creates an LLM from LLM_API_KEY, LLM_MODEL, and LLM_BASE_URL.

Source: https://github.com/OpenHands/OpenHands-CLI/blob/3ca17446c5d9c1e35e054803478a3501ec251ecf/openhands_cli/stores/agent_store.py#L131-L153

ENV_LLM_API_KEY = "LLM_API_KEY"
ENV_LLM_BASE_URL = "LLM_BASE_URL"
ENV_LLM_MODEL = "LLM_MODEL"

Source: https://github.com/OpenHands/OpenHands-CLI/blob/3ca17446c5d9c1e35e054803478a3501ec251ecf/openhands_cli/stores/agent_store.py#L286-L301

llm = LLM(
    model=overrides.model,
    api_key=overrides.api_key.get_secret_value(),
    base_url=overrides.base_url,
    usage_id="agent",
)
return get_default_cli_agent(llm)

Proposed end state

Persistent data model

• LLMProfileStore / LLMProfiles is the only persistent location for model/base_url/API key/LLM options.
• PersistedSettings.agent_settings no longer contains llm.
• PersistedSettings contains active_profile (or default_profile) and only non-LLM agent config: agent kind, tools, MCP, condenser settings, verification, ACP settings, skills/context.
• llm_api_key_is_set becomes active/default-profile-derived compatibility state.
• Migrations move legacy agent_settings.llm into a Default profile and set it active. After migration, saved settings should not contain agent_settings.llm.

Python API

Raw Python construction remains supported:

llm = LLM(model="...", api_key="...")
agent = Agent(llm=llm, tools=[...])

When raw Python-created LLM/Agent data is persisted, sent through a remote server, or converted to settings, create/update a default profile immediately and persist/profile-reference that instead of saving llm under agent_settings.

REST/API

• Add profile-reference based conversation start, e.g. profile_name?: string or agent_profile?: { name: string } on StartConversationRequest.
• If omitted, server uses active/default profile.
• Start requests should not require Canvas/automation to fetch encrypted LLM settings.
• Keep /api/profiles as the profile-management surface, but stop copying activated profile data into agent_settings.llm.
• Deprecate GET/PATCH /api/settings handling of agent_settings.llm and X-Expose-Secrets for LLM fields.
• Deprecate raw POST /api/conversations/{id}/switch_llm in favor of switch_profile. For app-server/cloud compatibility, either sync/push named profiles into the runtime agent-server before switch or let runtime agent-server resolve profiles via app-server/cloud APIs.

Estimated work

Phase 1 — SDK + agent-server core (5–8 engineering days)

1. Add a profile reference to settings/conversation start.
2. Keep OpenHandsAgentSettings.llm accepted but deprecated during a compatibility window.
3. Add migration helpers: agent_settings.llm → LLMProfileStore.save("Default", llm, include_secrets=True).
4. Change server-side settings-to-agent construction to resolve the active/default profile at creation time. The pure Python create_agent() path may need an optional resolver/store or a separate server-only builder to avoid unexpected IO in Pydantic models.
5. Update /api/settings schemas to remove or hide the LLM section for non-Python clients.
6. Update profile activation to set active_profile only, not deep-merge agent_settings_diff.llm.
7. Deprecate switch_llm and add compatibility tests.

Risk: public SDK and REST compatibility. The repo guidance says REST contract breaks need a 5-minor-release runway, so this likely needs deprecations plus dual-read support before removal.

Phase 2 — Agent Canvas (3–5 engineering days)

1. Remove top-level LLM compatibility fields from local-mode runtime state, or make them derived display-only from active profile summaries.
2. Stop calling getSettingsForConversation() for encrypted LLM settings.
3. Change start-conversation builder to send profile reference + non-LLM agent/conversation settings only.
4. Update LLM settings pages to become profile-management pages only; generic settings pages should not send agent_settings_diff.llm.
5. Update configured/unconfigured checks to use active/default profile api_key_set only.

Risk: onboarding and local backend UX currently expect “enter API key in settings then start conversation”. That should become “create/update Default profile”.

Phase 3 — Automation (1–2 engineering days)

Automation already stores model profile names. Remaining work:

1. Change preset scripts to start conversations with a profile reference and let agent-server construct the default agent.
2. Avoid workspace.get_llm() for normal automation runs.
3. Keep AUTOMATION_MODEL as a profile name.
4. Update tests that assert generated scripts print or inspect llm.api_key.

Risk: low-medium.

Phase 4 — Legacy OpenHands app-server / cloud compatibility (3–6 engineering days)

1. Migrate Settings.llm_profiles + agent_settings.llm duplication to profile-only state.
2. Change llm_api_key_is_set to derive from active/default profile.
3. Replace cloud conversation switching via raw /switch_llm with profile synchronization + /switch_profile, or add a profile-resolving switch endpoint that avoids sending raw LLM JSON from app-server to runtime agent-server.
4. Keep older settings responses compatible while Canvas migration lands.

Risk: medium-high because this code mediates cloud/enterprise settings and running-conversation model switching.

Phase 5 — CLI (2–4 engineering days)

1. Replace agent_settings.json full-Agent persistence with profile store + non-LLM CLI/agent settings.
2. Migrate existing agent_settings.json by extracting agent.llm into Default profile and rewriting remaining agent config without embedded LLM.
3. Replace first-run settings save with “create/update Default profile”.
4. Revisit --override-with-envs: either convert env vars into a temporary/default profile immediately or deprecate direct env overrides.

Risk: medium.

Phase 6 — Docs/tests/release migration (2–4 engineering days)

1. Update SDK docs for Agent Settings, LLM Profile Store, Remote Agent Server, Automations, and CLI settings.
2. Add migration tests for persisted settings and CLI files.
3. Add REST contract tests proving agent_settings.llm is no longer emitted to new clients and is migrated from old clients.
4. Add Canvas/automation tests for profile-reference conversation starts.

Total estimate

Approximate total: 16–29 engineering days (3–6 engineer-weeks) depending on how strict the REST compatibility/deprecation runway needs to be.

A low-risk rollout would be:

1. Add profile-reference paths and migrations while still accepting old agent_settings.llm.
2. Update Canvas and automation to stop using old paths.
3. Update CLI.
4. Mark old settings fields and switch_llm deprecated.
5. Remove after the SDK/REST compatibility window.

Open questions

1. Should /api/profiles remain a REST surface that accepts model/API key values, or is the desired rule stricter: only Python API may ever submit raw LLM config, with UI profile creation routed through a separate secret/profile provisioning flow?
   • Yes
2. For ACP agents, should ACPAgentSettings.llm become a profile reference used only for accounting/provider env derivation, or should ACP credential/model handling be separated completely from LLM profiles?
   • Not sure about ACP
3. Should condenser and critic use the active agent profile by default, or separate profile references (condenser_profile, critic_profile) for advanced users?
   • Irrelevant
4. In cloud mode, should profiles be copied into each sandbox agent-server, or should runtime agent-server resolve profiles by calling back to app-server/cloud APIs?
   • it should work the same as agent-canvas, which is server side.

[enyst]

Implementation plan, starting minimally:

1. Expose the active LLM profile in the settings API

   • Add active_profile to GET /api/settings and PATCH /api/settings.
   • Keep this additive and compatible: no raw agent_settings.llm behavior changes yet.
   • First PR: Expose active LLM profile in settings API #3552

2. Stop profile activation from duplicating profile state into settings internals

   • Change profile activation so it records the active profile/profile reference instead of copying the profile's raw LLM config into agent_settings.llm.
   • Keep legacy consumers working while making the server-owned source of truth profile-first.

3. Add profile-reference conversation start support

   • Allow conversation start payloads to specify a profile reference, or default to the active profile.
   • Resolve the profile to a concrete runtime LLM only at the agent/conversation creation boundary.
   • Keep raw Python/REST agent payloads as compatibility paths during the rollout.

4. Materialize legacy agent_settings.llm into a default profile

   • On persisted settings load/update, detect legacy raw agent_settings.llm and create/update a default profile when safe.
   • Persist the profile reference going forward rather than continuing to duplicate raw LLM config in settings.

5. Gradually deprecate raw agent_settings.llm in REST/server-owned settings

   • After profile references are supported by settings, activation, and conversation start, mark raw REST settings paths as legacy/deprecated.
   • Keep the required compatibility runway before removing or making profile references mandatory.

I reviewed the ordering above against the current code paths. The main correction is that we should keep concrete LLM objects at runtime; the migration should target persistence and REST/app boundaries first, resolving profile references to LLM only when an agent actually runs.

This issue comment was created by an AI agent (OpenHands) on behalf of the user.