[AI-3] Let administrators add, edit and delete models by hand by tangopium · Pull Request #24884 · opf/openproject

tangopium · 2026-08-22T14:55:08Z

Ticket

What are you trying to accomplish?

PR 6 of 11 in the AI-3 stack. Not every OpenAI-compatible server exposes a model list: a gateway may route /v1/chat/completions and nothing else. This part lets administrators add, edit and delete models by hand, including capability assertions stored as admin-sourced verdicts and an administrator-supplied context window that takes precedence over reported figures. A manual model's identifier can be corrected later; the rename carries every verdict and the connection defaults along. Discovered models cannot be renamed or deleted here, and a catalogue refresh never withdraws a manual entry.

Merge checklist

Added/updated tests
Behind the llm_connection feature flag

Stacked on #24883.

github-actions · 2026-08-22T14:57:22Z

Caution

The provided work package version does not match the core version

Details:

Work package URL: https://community.openproject.org/work_packages/66020
Work package version: not set
Core version: 17.9.0

Please make sure that:

The work package version OR your pull request target branch is correct

github-actions · 2026-08-22T14:57:23Z

Caution

The Enterprise plan field is not set on the work package

Details:

Work package URL: https://community.openproject.org/work_packages/66020

Please make sure that:

The work package Enterprise plan field is set

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e6d4db2e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Not every OpenAI-compatible server exposes a model list: a gateway may route /v1/chat/completions and nothing else, in which case the operator knows the model name and OpenProject cannot discover it. This adds hand-entered LlmModel rows with their own form, including capability assertions stored as admin-sourced verdicts that survive re-detection, and an administrator-supplied context window that takes precedence over reported figures. A manual model's identifier may be corrected after the fact; the rename carries every verdict and the connection defaults along, because everything references a model by its identifier string. Discovered models are named by the server and cannot be renamed or deleted here, and a catalogue refresh never withdraws a manual entry. Part 6 of the AI-3 stack. https://community.openproject.org/work_packages/66020

github-actions · 2026-08-22T16:13:45Z

Warning

Flaky specs

rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:1]
rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]

🤖 Ask Copilot to investigate

Copy the prompt below into a new comment on this PR to delegate the investigation to GitHub Copilot. It will look into the flakiness and open a separate pull request with you as reviewer.

@copilot The following spec(s) are flaky in CI (first seen on PR #24884, linked for reference only):

- `rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:1]`
- `rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]`

Treat this as a standalone task, unrelated to PR #24884. Create a new branch from origin/dev and open a new pull request targeting dev — do not stack it on PR #24884 or reuse that branch.

Follow the playbook in docs/development/testing/handling-flaky-tests/README.md to find the root cause and fix the underlying race — do not skip, delete, or weaken the spec to make it pass; disabling is a last resort per the playbook, and only with a bug ticket. Verify the fix by running the spec(s) repeatedly (e.g. `script/bulk_run_rspec --run-count 10`).

If you cannot reproduce the flake or are not confident in a fix after reasonable investigation, do not fabricate a change or skip the spec to force CI green. Instead, leave the pull request in draft and document what you tried, the suspected cause, and any leads in its description, then assign @tangopium to take over.

Once the fix is verified, title the PR after the spec(s) it fixes, and use the PR description to explain the root cause, how the change resolves it, and the before/after results. Label the PR `flaky-spec`, assign @tangopium, and request a review from @tangopium.
On every commit, set @tangopium as the sole co-author with a `Co-authored-by:` trailer (use their GitHub no-reply email so it links to their account), so it is traceable who dispatched the fix.

github-actions · 2026-08-22T16:41:57Z

Warning

Flaky specs

rspec ./spec/features/work_packages/details/inplace_editor/version_editor_spec.rb[1:1:2]
rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:1]
rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]

🤖 Ask Copilot to investigate

Copy the prompt below into a new comment on this PR to delegate the investigation to GitHub Copilot. It will look into the flakiness and open a separate pull request with you as reviewer.

@copilot The following spec(s) are flaky in CI (first seen on PR #24884, linked for reference only):

- `rspec ./spec/features/work_packages/details/inplace_editor/version_editor_spec.rb[1:1:2]`
- `rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:1]`
- `rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]`

Treat this as a standalone task, unrelated to PR #24884. Create a new branch from origin/dev and open a new pull request targeting dev — do not stack it on PR #24884 or reuse that branch.

Follow the playbook in docs/development/testing/handling-flaky-tests/README.md to find the root cause and fix the underlying race — do not skip, delete, or weaken the spec to make it pass; disabling is a last resort per the playbook, and only with a bug ticket. Verify the fix by running the spec(s) repeatedly (e.g. `script/bulk_run_rspec --run-count 10`).

If you cannot reproduce the flake or are not confident in a fix after reasonable investigation, do not fabricate a change or skip the spec to force CI green. Instead, leave the pull request in draft and document what you tried, the suspected cause, and any leads in its description, then assign @tangopium to take over.

Once the fix is verified, title the PR after the spec(s) it fixes, and use the PR description to explain the root cause, how the change resolves it, and the before/after results. Label the PR `flaky-spec`, assign @tangopium, and request a review from @tangopium.
On every commit, set @tangopium as the sole co-author with a `Co-authored-by:` trailer (use their GitHub no-reply email so it links to their account), so it is traceable who dispatched the fix.

thykel

👍 As usual, leaving you with a mixture of my own comments + robot ones (that look legible to me)

thykel · 2026-09-05T23:04:02Z

+        context_window: "Context window"
+        context_window_caption: "How many tokens this model accepts. Leave blank if you do not know."
+        context_window_known: "Leave blank to use %{value}, %{source}."
+        create_submit: "Add model"


Fits our usual convention better:

Suggested change

create_submit: "Add model"

create_submit: "+ Model"

thykel · 2026-09-05T23:11:32Z

+      edit:
+        description: "Edit how OpenProject uses this model."
+      form:
+        capabilities_description: "Leave a capability unspecified to let OpenProject determine it. Marking Embeddings as supported makes this an embedding model, offered only to features that need one."


Ghost key? 👀 Not seeing any references even in the future PRs.

thykel · 2026-09-05T23:14:14Z

+      @llm_model = @connection.models.new
+    end
+
+    def edit


Please rescue & add a spec for an invalid PATCH request.

thykel · 2026-09-05T23:22:06Z

+
+    # Verdicts are keyed by the identifier string, not by foreign key, so they
+    # would silently apply to a future model re-added under the same name.
+    def destroy_with_verdicts(llm_model)


🤖 #Logic 🚨 Deleting a model leaves the connection defaults pointing at it

cascade_rename! goes to real trouble to keep references intact on a rename: it moves the verdicts and rewrites default_chat_model_id / default_embedding_model_id. This path deletes the verdicts but leaves the defaults untouched.

Confirmed on this branch: set default_chat_model_id to a manual model, delete it, and the column still holds the deleted identifier.

The dialog does warn through affected_defaults, so this may be a deliberate "the administrator was told". If so, could we say that in a comment? As it stands one method treats the defaults as references to maintain and the other does not, and it is not obvious which behaviour is intended.

thykel · 2026-09-05T23:22:07Z

+      end
+
+      f.text_field(
+        name: :display_name,


🤖 #Logic ⚠️ An edited display name does not survive the next catalogue refresh

This PR makes display_name editable, which makes an existing behaviour in SyncModelsService#upsert reachable: it assigns display_name: card[:display_name] unconditionally, and the OpenAI adapter never supplies one.

Measured on this stack:

manual model the server does not report - name survives

manual model the server also reports - name lost

discovered model whose id is in the registry - name silently replaced by the vendor's

discovered model not in the registry - name blanked

merged_metadata already protects admin_context_window from exactly this. Would it make sense to give display_name the same treatment, so the sync keeps the stored value when the card carries none?

thykel · 2026-09-05T23:22:09Z

+    # administrator knows things about their deployment that neither a published
+    # registry nor a probe can determine.
+    def apply_capabilities(llm_model)
+      submitted = llm_model_params


🤖 #Design ℹ️ apply_capabilities re-parses the request parameters

Both callers have already built this hash before calling in. Could apply_capabilities take it as an argument instead of running params.expect a second time per request?

thykel · 2026-09-05T23:22:10Z


+  <% subheader.with_action_button(
+       scheme: :primary,
+       label: t("admin.llm_connections.show.add_model_submit"),


🤖 #Style ℹ️ This key names a link, not a submit, and duplicates create_submit

Two separate points on the same key, both worth a look alongside the suggestion already left on create_submit:

add_model_submit labels this sub-header link, while the form's actual submit button uses admin.llm_models.form.create_submit. Both currently read "Add model", so the same text appears on two different controls and the _submit suffix sits on the one that is not a submit. Would it be clearer to rename this one (for example add_model) and leave create_submit naming the action the form performs?

On the label itself: the closest analogues in the repo use a bare noun for a primary sub-header button with leading_icon: :plus - + Portfolio in portfolios/index_sub_header_component.html.erb and + Item in admin/custom_fields/hierarchy/items_component.html.erb. The other convention is a bare verb, + Add, used by the projects index. Since the icon already supplies the plus, the literal "+" would not be needed in the string either.

Separately, admin.llm_connections.show.add_model_description looks like a second ghost key in this area - unreferenced since it was added, and superseded by admin.llm_models.new.description.

This was referenced Aug 22, 2026

[AI-3] Let administrators curate which models are offered #24885

Open

[#66020] Add LLM connection settings to admin AI configuration #24851

Closed

chatgpt-codex-connector Bot reviewed Aug 22, 2026

View reviewed changes

Comment thread app/models/llm_model.rb

Comment thread app/controllers/admin/llm_models_controller.rb Outdated

Comment thread app/components/llm_connections/delete_model_dialog_component.html.erb Outdated

tangopium force-pushed the feature/66020-pr06-manual-model-entries branch from 0e6d4db to cc8a279 Compare August 22, 2026 15:05

tangopium changed the title ~~[#66020] Let administrators add, edit and delete models by hand~~ [AI-3] Let administrators add, edit and delete models by hand Aug 22, 2026

tangopium force-pushed the feature/66020-pr06-manual-model-entries branch from cc8a279 to 1e9e5a9 Compare August 22, 2026 15:26

tangopium force-pushed the feature/66020-pr06-manual-model-entries branch from 1e9e5a9 to 7b53618 Compare August 22, 2026 15:51

tangopium force-pushed the feature/66020-pr06-manual-model-entries branch from 7b53618 to 75ff06e Compare August 22, 2026 16:16

tangopium self-assigned this Aug 22, 2026

tangopium requested a review from thykel August 22, 2026 21:39

thykel reviewed Sep 5, 2026

View reviewed changes

Uh oh!

Conversation

tangopium commented Aug 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ticket

What are you trying to accomplish?

Merge checklist

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

thykel left a comment

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

thykel Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

tangopium commented Aug 22, 2026 •

edited

Loading