Add OpenAI Images provider support#53
Conversation
|
CodeAnt AI is reviewing your PR. Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request adds support for the OpenAI Images API, enabling users to generate images using OpenAI's dedicated endpoint. The changes include backend implementation for the new provider, updated configuration management to handle provider-specific parameters like size and quality, and corresponding frontend updates to provide a seamless user experience when switching between different image generation providers. Highlights
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
codeant-ai
Bot
added
the
size:XL
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 28 |
| Duplication | 15 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes. Give us feedback
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the openai-image provider, integrating OpenAI's image generation capabilities into the application. Key updates include the backend OpenAIImageProvider implementation, database seeding for the new provider, and frontend enhancements to support native size and quality settings while conditionally disabling reference image uploads. The review feedback highlights a security policy violation concerning a hardcoded API key placeholder in the seed script and identifies several redundant conditional checks in the ReferenceImageUpload component that should be removed for better code clarity.
| ProviderName: "openai-image", | ||
| DisplayName: "OpenAI Images", | ||
| APIBase: "https://api.openai.com/v1", | ||
| APIKey: "YOUR_API_KEY_HERE", |
There was a problem hiding this comment.
Hardcoded API Key found in seed script. This violates the security policy of not committing secrets to version control.
References
- 禁止将密钥提交到版本控制
| className={cn( | ||
| "flex items-center justify-between rounded-xl transition-all", | ||
| showDragOver && "bg-blue-50 ring-2 ring-blue-400 ring-dashed" | ||
| interactive && showDragOver && "bg-blue-50 ring-2 ring-blue-400 ring-dashed" |
There was a problem hiding this comment.
The variable interactive is already checked in the if block at line 1607, making the interactive && check here redundant.
| interactive && showDragOver && "bg-blue-50 ring-2 ring-blue-400 ring-dashed" | |
| showDragOver && "bg-blue-50 ring-2 ring-blue-400 ring-dashed" |
| </div> | ||
| <div className="flex items-center gap-2"> | ||
| {showDragOver && ( | ||
| {interactive && showDragOver && ( |
There was a problem hiding this comment.
The interactive && check is redundant here as the entire component returns early if !interactive (via !allowReferenceImages).
| {interactive && showDragOver && ( | |
| {showDragOver && ( |
| </span> | ||
| )} | ||
| {refFiles.length > 0 && !showDragOver && ( | ||
| {interactive && refFiles.length > 0 && !showDragOver && ( |
There was a problem hiding this comment.
The interactive && check is redundant here as the entire component returns early if !interactive (via !allowReferenceImages).
| {interactive && refFiles.length > 0 && !showDragOver && ( | |
| {refFiles.length > 0 && !showDragOver && ( |
|
|
||
| {/* 上传按钮/区域 */} | ||
| {refFiles.length === 0 && refFiles.length < 10 && ( | ||
| {interactive && refFiles.length === 0 && refFiles.length < 10 && ( |
There was a problem hiding this comment.
The interactive && check is redundant here as the entire component returns early if !interactive (via !allowReferenceImages).
| {interactive && refFiles.length === 0 && refFiles.length < 10 && ( | |
| {refFiles.length === 0 && refFiles.length < 10 && ( |
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
| return "gpt-image-1" | ||
| } | ||
| if name == "openai" { | ||
| return "" |
There was a problem hiding this comment.
Suggestion: Returning an empty default model for this provider makes model resolution fail whenever callers omit model_id and no DB default exists, which leads to runtime task failures (缺少 model_id 参数) in the provider execution path. Use a non-empty fallback default model. [possible bug]
Severity Level: Major ⚠️
- ❌ OpenAI provider calls without model_id fail at runtime.
- ⚠️ Workflows relying on default model resolution break.| return "" | |
| return "gemini-3-pro-image-preview" |
Steps of Reproduction ✅
1. Start the backend built from this PR so that
`backend/internal/provider/model_resolver.go` is in use (function
`defaultModelForProvider` at lines 85-97 is compiled into the provider resolution flow).
2. Trigger an image or model-based operation that uses the generic `"openai"` provider
name without explicitly setting a `model_id` in the request (this causes the system to
call `defaultModelForProvider("openai", purpose)` to resolve a model).
3. When the provider name is `"openai"`, the branch at
`backend/internal/provider/model_resolver.go:93-95` executes and returns an empty string
`""` as the default model ID.
4. Downstream, the provider execution path expects a non-empty model identifier; an empty
model causes resolution to fail and results in a runtime task failure indicating a missing
`model_id` parameter (as described in the suggestion text: `缺少 model_id 参数`).Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** backend/internal/provider/model_resolver.go
**Line:** 94:94
**Comment:**
*Possible Bug: Returning an empty default model for this provider makes model resolution fail whenever callers omit `model_id` and no DB default exists, which leads to runtime task failures (`缺少 model_id 参数`) in the provider execution path. Use a non-empty fallback default model.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| setRefFiles([]); | ||
| setRefImageEntries([]); | ||
| toast.info(t('refImage.unsupportedModel')); | ||
| }, [allowReferenceImages, refFiles.length, setRefFiles, setRefImageEntries, t]); |
There was a problem hiding this comment.
Suggestion: This effect permanently clears refFiles and persisted entries when switching to an unsupported provider, which causes user data loss and can delete persisted reference-image metadata/files even if the user switches back later. Keep the files in state/storage and only show the unsupported notice. [logic error]
Severity Level: Critical 🚨
- ❌ Reference-image uploads lost when switching to openai-image provider.
- ⚠️ Image-to-image generation unusable after switching providers back.
- ⚠️ Local ref image files deleted from AppData directory.| setRefFiles([]); | |
| setRefImageEntries([]); | |
| toast.info(t('refImage.unsupportedModel')); | |
| }, [allowReferenceImages, refFiles.length, setRefFiles, setRefImageEntries, t]); | |
| toast.info(t('refImage.unsupportedModel')); | |
| }, [allowReferenceImages, refFiles.length, t]); |
Steps of Reproduction ✅
1. Start the desktop app and ensure the image provider is a reference-image-capable
provider (default 'gemini' per `desktop/src/store/configStore.ts:142-146` where
`imageProvider` defaults to `'gemini'`).
2. In the main config panel (`desktop/src/components/ConfigPanel/index.tsx:25-27`), use
the `<ReferenceImageUpload />` component to add one or more reference images; this
populates `refFiles` in the config store and persists entries via `syncPersistedRefFiles`
in `desktop/src/components/ConfigPanel/ReferenceImageUpload.tsx:407-552` (writing files
under `REF_IMAGE_DIR` and updating `refImageEntries`).
3. Open the settings modal (`desktop/src/components/Settings/SettingsModal.tsx:150-200`),
change `imageProvider` to `'openai-image'` using `setImageProvider` (wired via
`useConfigStore` at `SettingsModal.tsx:152-47`), which causes `supportsReferenceImages`
(`desktop/src/store/configStore.ts:29`) to return `false` and re-render
`ReferenceImageUpload` with `allowReferenceImages = false`.
4. After that re-render, the effect at
`desktop/src/components/ConfigPanel/ReferenceImageUpload.tsx:175-182` runs: since
`allowReferenceImages` is false and `refFiles.length > 0`, it calls `setRefFiles([])` and
`setRefImageEntries([])`, which triggers `syncPersistedRefFiles`
(`ReferenceImageUpload.tsx:561-563` and `407-552`) to see an empty `refFiles` list and
remove all previously managed persisted images from disk (`remove(...)` in lines 539-544).
Later, switching the provider back to a supporting provider leaves `refFiles` and
`refImageEntries` empty, and `restorePersistedRefFiles`
(`ReferenceImageUpload.tsx:252-332`) finds no entries to restore, so all previously
uploaded reference images are permanently lost and cannot be used again.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** desktop/src/components/ConfigPanel/ReferenceImageUpload.tsx
**Line:** 179:182
**Comment:**
*Logic Error: This effect permanently clears `refFiles` and persisted entries when switching to an unsupported provider, which causes user data loss and can delete persisted reference-image metadata/files even if the user switches back later. Keep the files in state/storage and only show the unsupported notice.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| next = { | ||
| ...next, | ||
| imageModel: next.imageModel ?? DEFAULT_IMAGE_MODEL |
There was a problem hiding this comment.
Suggestion: This migration uses nullish coalescing only, so an existing persisted empty string for imageModel is preserved instead of being repaired, which later sends an empty model and can break generation. Normalize empty-string values to a valid default during migration. [logic error]
Severity Level: Major ⚠️
- ⚠️ Migrated configs can retain invalid empty imageModel values.
- ❌ Image generation requests may use empty, unsupported model identifiers.
- ⚠️ Affected users see image-generation failures after upgrading.| next = { | |
| ...next, | |
| imageModel: next.imageModel ?? DEFAULT_IMAGE_MODEL | |
| const migratedImageModel = typeof next.imageModel === 'string' ? next.imageModel.trim() : ''; | |
| next = { | |
| ...next, | |
| imageModel: migratedImageModel || DEFAULT_IMAGE_MODEL |
Steps of Reproduction ✅
1. Run a desktop build from before this PR that persists `useConfigStore` to
`localStorage` under the key `app-config-storage` (persistence configured in
`desktop/src/store/configStore.ts:295-303`).
2. Using that older build, get into a state where `imageModel` in the persisted store is
an empty string (e.g., via a prior bug or manual tampering of localStorage) while the
stored `version` is `< 17`.
3. Upgrade to a build that includes this PR; on startup, Zustand persistence calls the
`migrate(persistedState, version)` function in `desktop/src/store/configStore.ts:308-422`
with the old persisted state and `version` set to `18`.
4. During migration, the block at `desktop/src/store/configStore.ts:407-412` executes:
`imageModel` is `''`, so `next.imageModel ?? DEFAULT_IMAGE_MODEL` leaves it as `''`
instead of repairing it, and the final migrated state still has `imageModel: ''`, which is
not one of the defined `GEMINI_IMAGE_MODEL_OPTIONS`/`OPENAI_IMAGE_MODEL_OPTIONS` and will
later be used as an invalid model identifier when constructing image generation requests.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** desktop/src/store/configStore.ts
**Line:** 408:410
**Comment:**
*Logic Error: This migration uses nullish coalescing only, so an existing persisted empty string for `imageModel` is preserved instead of being repaired, which later sends an empty model and can break generation. Normalize empty-string values to a valid default during migration.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
| if (typeof aspectRatio === 'string' && aspectRatio.trim()) { | ||
| return aspectRatio.trim(); | ||
| } | ||
| return '—'; |
There was a problem hiding this comment.
Suggestion: Native-size snapshots often don't include an explicit aspect ratio, and the current logic returns — even when size has enough information (like 1024x1536) to infer the ratio. This causes incorrect task metadata display for OpenAI Images failures. [logic error]
Severity Level: Major ⚠️
- ❌ History panel omits aspect ratio for many OpenAI failures.
- ⚠️ Users lose metadata when debugging failed image generations.| return '—'; | |
| if (typeof nativeSize === 'string') { | |
| const m = nativeSize.trim().match(/^(\d+)\s*x\s*(\d+)$/i); | |
| if (m) { | |
| return formatAspectRatioLabel(Number(m[1]), Number(m[2])); | |
| } | |
| } |
Steps of Reproduction ✅
1. In the desktop app, trigger an OpenAI Images generation using a non-square native size
(e.g. `1024x1536`) so that the request snapshot stores `"size": "1024x1536"` but no
`aspectRatio`/`aspect_ratio`/`aspect` fields in the `options` JSON attached to the task
(`GenerationTask.options` from `desktop/src/types`).
2. Cause this generation to fail (for example by disconnecting the network) so that a
failed task is persisted in the history store; `useHistoryStore` in
`desktop/src/components/HistoryPanel/FailedTaskCard.tsx:6` provides this `GenerationTask`
(including its `options` string) to the `FailedTaskCard` component.
3. Open the History panel so it renders a `FailedTaskCard` for this failed task; inside
that component, `snapshotLabels` is computed via `React.useMemo` at
`desktop/src/components/HistoryPanel/FailedTaskCard.tsx:123-157`, where `parsed.size` is
read into `nativeSize` at line 133 and `aspectRatio` remains `undefined` because no aspect
fields exist in the snapshot.
4. Observe that `aspectRatioLabel` defined at
`desktop/src/components/HistoryPanel/FailedTaskCard.tsx:145-150` only checks the
`aspectRatio` variable and ignores `nativeSize`, so it returns `'—'`; the JSX that
displays `snapshotLabels.aspectRatioLabel` on the failed task card therefore shows `—`
instead of the correct aspect ratio that could be derived from `"size": "1024x1536"`.Fix in Cursor | Fix in VSCode Claude
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** desktop/src/components/HistoryPanel/FailedTaskCard.tsx
**Line:** 149:149
**Comment:**
*Logic Error: Native-size snapshots often don't include an explicit aspect ratio, and the current logic returns `—` even when `size` has enough information (like `1024x1536`) to infer the ratio. This causes incorrect task metadata display for OpenAI Images failures.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
|
CodeAnt AI finished reviewing your PR. |
Add OpenAI Images provider support
1 participant
User description
Summary\n- remove the previous DALL-E-specific image workflow cleanup in a dedicated commit\n- add a new openai-image provider for /v1/images/generations\n- add frontend provider-specific controls for size and quality\n\n## Testing\n- cd backend && go test ./...\n- cd desktop && npm run build
CodeAnt-AI Description
Add OpenAI Images provider support with native size and quality controls
What Changed
/v1/images/generationsflowImpact
✅ OpenAI image generation works without reference-image uploads✅ Fewer invalid image requests for OpenAI Images✅ Clearer generation settings for OpenAI users🔄 Retrigger CodeAnt AI Review
Details
💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.