Validate provider model, auth, and endpoint before save by kgp0213 · Pull Request #943 · ValueCell-ai/ClawX

kgp0213 · 2026-04-30T02:10:29Z

Validate provider model, auth, and endpoint before save

Summary

This PR adds a lightweight pre-save model check to both the Add Provider and Edit Provider flows.

Before saving a provider, users can run a small real request to verify that:

the endpoint is reachable
authentication works
the selected model is accepted by the provider

The goal is to reduce trial and error during provider setup and surface configuration mistakes earlier.

Problem

Right now, provider setup failures are often discovered only later, when the provider is actually used.

Common cases include:

a valid endpoint with an invalid model ID
a valid model with a bad API key
a custom or self-hosted endpoint that is unreachable or misconfigured

From a user perspective, these all look like: save succeeded, but usage failed later.

That creates avoidable setup friction, especially for:

custom-compatible providers
self-hosted gateways
users switching between multiple vendors or protocols

Why this belongs in the settings flow

This check is intentionally placed in Add/Edit Provider, not in a later runtime-only path, because the UX problem happens at configuration time.

Providing feedback before save helps users fix the exact field they are editing, instead of discovering the problem only after the provider is already stored and selected.

What this PR adds

UI

Adds a model check action to:

Add Provider dialog
Edit Provider panel

Also adds inline result feedback for:

success
degraded or slow response
model not found
auth failure
timeout
unsupported protocol

Backend

Adds a dedicated endpoint:

/api/provider-model-check

And a small service that performs a real validation request for supported protocols.

Frontend

Adds a small client wrapper and wires the result into ProvidersSettings.tsx.

Design choices

This implementation is intentionally lightweight.

The check:

uses a tiny prompt (Hi)
uses small output or token limits
uses streaming requests
treats the first returned chunk as proof that the model is usable

This keeps the request cheap while still validating the most important setup conditions.

Why first-chunk success is sufficient here

This is not intended to be a full semantic correctness test or a long health check.

It is a configuration validation step.

If the request reaches the provider, authenticates successfully, and the requested model starts returning streamed output, then the setup is already validated well enough for this UI purpose.

That gives fast feedback without turning the check into a heavier probe.

Supported protocols in this PR

This PR currently supports:

openai-completions
openai-responses
anthropic-messages

That scope is intentional:

these cover the main protocol families already exposed in provider configuration
keeping the first version narrow makes the change easier to review and safer to merge
additional protocol families can be added later if needed

Scope and non-goals

This PR intentionally does not include unrelated changes:

no branding changes
no provider list policy changes
no advanced settings redesign
no packaging or release pipeline changes
no behavior change to the normal save flow when users choose not to run the check

The goal is to keep the patch small, reviewable, and upstream-friendly.

Verification

Validated locally on an upstream-based working tree with only the minimal feature patch applied.

Commands used:

pnpm run ext:bridge
pnpm exec tsc --noEmit
pnpm exec vite build

All passed successfully.

Screenshot

Tested UI screenshot from the validation build:

This screenshot is included in the branch at:

docs/pr/provider-model-check-test.png

Files added or touched

electron/api/routes/providers.ts
electron/services/providers/provider-model-check.ts
src/components/settings/ProvidersSettings.tsx
src/lib/provider-model-check.ts

Why I think this is merge-worthy

This change addresses a real setup pain point with a relatively small surface area:

clear user benefit
low conceptual complexity
isolated implementation
explicit protocol scope
verified build and typecheck

If maintainers prefer different UI wording or button placement, I am happy to adjust that while keeping the validation behavior minimal.

kgp0213 · 2026-04-30T02:37:02Z

A few implementation notes that may help review:

The check is intentionally lightweight and only targets configuration-time validation.
It does not change the normal save flow unless the user explicitly runs the check.
The first streamed chunk is treated as success on purpose: for this UX step, the main goal is to confirm endpoint reachability, auth validity, and model acceptance with minimal latency/cost.
Protocol support is intentionally limited in this first version to keep the patch small and reviewable.

If you prefer different UI wording or button placement, I can adjust that separately while keeping the validation behavior minimal.

kgp0213 · 2026-04-30T02:53:46Z

One final clarification on intent:

This PR is meant to reduce setup-time friction, not to introduce a heavyweight provider health-check system.

The benefit is mainly that users can catch the three most common configuration problems earlier:

wrong model ID
invalid credentials
unreachable or misconfigured endpoint

That is why the implementation stays intentionally narrow, cheap, and opt-in.

hazeone · 2026-04-30T10:43:45Z

@@ -0,0 +1,96 @@
+# Add provider model-check support


Remove this file. (docs/pr/* change is useless)

hazeone

Please fix the issue and submit again

hazeone · 2026-04-30T10:44:02Z

hazeone · 2026-04-30T10:44:44Z

+  if (!result) return '';
+  const suffix = result.responseTimeMs ? ` · ${result.responseTimeMs} ms` : '';
+  if (result.success && result.status === 'operational') return `检测成功${suffix}`;
+  if (result.success && result.status === 'degraded') return `模型可用，但响应较慢${suffix}`;


Change test language to English version

hazeone · 2026-04-30T10:44:54Z

+              <div className="flex items-center justify-between gap-3">
+                <div>
+                  <p className={currentSectionLabelClasses}>模型检测</p>
+                  <p className="text-[12px] text-muted-foreground">用当前配置发起一次真实流式请求，只要首包返回即判定通过。</p>


hazeone · 2026-04-30T10:45:42Z

+                  <div className="rounded-xl border border-black/10 dark:border-white/10 p-4 bg-white/60 dark:bg-white/[0.03] space-y-2">
+                    <div className="flex items-center justify-between gap-3">
+                      <div>
+                        <p className={labelClasses}>模型检测</p>


You can set this function by i18n component

Vis-X Team added 2 commits April 30, 2026 09:23

feat(providers): add model check for add/edit provider

0349cc0

docs(pr): add model-check verification notes and screenshot

2ddc109

kgp0213 changed the title ~~Add provider model-check support~~ Validate provider model / auth / endpoint before save Apr 30, 2026

kgp0213 changed the title ~~Validate provider model / auth / endpoint before save~~ Validate provider model, auth, and endpoint before save Apr 30, 2026

hazeone reviewed Apr 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Validate provider model, auth, and endpoint before save#943

Validate provider model, auth, and endpoint before save#943
kgp0213 wants to merge 2 commits into
ValueCell-ai:mainfrom
kgp0213:feat/provider-model-check

kgp0213 commented Apr 30, 2026 •

edited

Loading

Uh oh!

kgp0213 commented Apr 30, 2026

Uh oh!

kgp0213 commented Apr 30, 2026

Uh oh!

hazeone Apr 30, 2026

Uh oh!

hazeone left a comment

Uh oh!

hazeone Apr 30, 2026

Uh oh!

hazeone Apr 30, 2026

Uh oh!

hazeone Apr 30, 2026

Uh oh!

hazeone Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

kgp0213 commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Validate provider model, auth, and endpoint before save

Summary

Problem

Why this belongs in the settings flow

What this PR adds

UI

Backend

Frontend

Design choices

Why first-chunk success is sufficient here

Supported protocols in this PR

Scope and non-goals

Verification

Screenshot

Files added or touched

Why I think this is merge-worthy

Uh oh!

kgp0213 commented Apr 30, 2026

Uh oh!

kgp0213 commented Apr 30, 2026

Uh oh!

hazeone Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

hazeone left a comment

Choose a reason for hiding this comment

Uh oh!

hazeone Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

hazeone Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

hazeone Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

hazeone Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kgp0213 commented Apr 30, 2026 •

edited

Loading