Fix: read LLM config from nao_config.yaml as a fallback

[d-axel-b]

Summary

• The UI backend was silently ignoring the llm section in
  nao_config.yaml, causing "The selected model could not be resolved"
  errors whenever the internal SQLite DB was reset (e.g. after rebasing
  from upstream)
• Users were forced to re-enter their API key through the UI settings
  even though it was already declared in nao_config.yaml

What changed

apps/backend/src/utils/llm.ts — added nao_config.yaml as a third
fallback in the credential resolution chain:

Priority: DB config → env vars → nao_config.yaml

The three affected functions are:

• resolveProviderSettings — used by web search tools
• resolveProviderModel — used by the agent on every chat message
• getProjectAvailableModels — used to populate the model selector in the UI

apps/backend/tests/llm.test.ts — 13 unit tests covering:

• Returns null when no source is configured
• Picks up credentials from nao_config.yaml
• Ignores nao_config.yaml when provider doesn't match
• Respects priority: DB > env > yaml
• No duplication in getProjectAvailableModels when provider is already covered

Test plan

• npx vitest run tests/llm.test.ts — 13 tests pass
• npm run lint — no errors
• Start the app without DB or env keys, with only llm in
  nao_config.yaml — model resolves correctly and chat works

[cubic-dev-ai]

1 issue found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/backend/src/utils/llm.ts">

<violation number="1" location="apps/backend/src/utils/llm.ts:198">
P1: Unvalidated provider value from nao_config.yaml can cause runtime errors when deriving model metadata. Add runtime validation to ensure the provider value is a valid LlmProvider before using it in getDefaultModelId/getModelName.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[Bl3f]

Just to be sure, how did you launch the UI that it ignored the settings? Was it in dev or prod?

Currently we decided to make the UI/backend unaware of the nao_config.yaml so it's more flexible for later, so im not sure this is a good idea to do this PR.

[d-axel-b]

Hello @Bl3f - I was running it through npm run dev

What's is disturbing is that I did the configuration of the databases / datawarehouses throught nao init& nao sync (nao_config.yaml), and it works. I also did the LLM configuration during this process but it was not taken into account.

Just to be sure to understand. What are you planning to keep in nao_config and what do you prefer to keep in the nao db ?

[d-axel-b]

ANy update on this one :)

[Bl3f]

Sorry @d-axel-b, forgot to answer!

So yes, when running through npm run dev you will need to put the LLM keys in the .env to be picked up.

The reason here is to keep the context lib + nao_config.yaml and the agentic loop separated and only ENV variables are doing the bridge.

When it comes to what goes into nao config vs. db actually nao config should contain everything that is related to getting the context whereas the db is for everything related to the agentic loop.

In this case LLM config is a bit in both place because to pull context and use ai annotations or use nao test you will need a LLM key.

Anyway. Would love to chat with you about this if you want esp. as it will change a bit when going multi project (I'm working on it)

Your PR could be merged but it I'm not sure it simplifies the understanding of where to put LLM keys when working locally.