Skip to content

fix(browser): report verified model labels in CLI output - #318

Merged
steipete merged 5 commits into
steipete:mainfrom
DragonFSKY:codex/browser-model-provenance
Aug 2, 2026
Merged

fix(browser): report verified model labels in CLI output#318
steipete merged 5 commits into
steipete:mainfrom
DragonFSKY:codex/browser-model-provenance

Conversation

@DragonFSKY

@DragonFSKY DragonFSKY commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Distinguish browser picker targets from requested CLI model keys before selection.
  • Use exact picker labels in CLI and TUI session displays only when persisted evidence is verified.
  • Preserve the generic Pro picker label without inferring a server-side GPT version.
  • Keep raw model keys for API output, model filtering, per-model log routing, and non-primary model runs.

Fixes #317.

Display contract

Browser model identity has three separate sources:

  • requestedKey: the Oracle model key, such as gpt-5.5-pro, gpt-5.6, or gpt-5.6-sol.
  • target: the label Oracle asks the ChatGPT picker to select, such as Pro or GPT-5.6 Sol.
  • resolvedLabel: an observed picker/model label after selection. It becomes the effective display label only when it is non-empty and verified=true.

Before selection, launch and dry-run output reports intent rather than observed state:

Launching browser mode (target=GPT-5.6 Sol; requested=gpt-5.6) ...
Launching browser mode (picker=current; requested=gpt-5.5-pro) ...

After selection, compact status and completion output uses the exact verified label, otherwise it falls back to the requested key. Detailed CLI and TUI views retain both when they differ:

Pro (requested gpt-5.5-pro)

The two Sol aliases retain distinct requested keys even though they currently share the GPT-5.6 Sol target. A bare Pro result remains Pro; no path expands it to GPT-5.6 Pro.

Evidence hardening

  • A picker target is never copied into resolvedLabel. If the UI exposes only effort text or no usable label, the stored value is null.
  • verified=yes now means that a non-empty observed label was available after an explicit selection strategy; a successful selection without observable label evidence remains verified=no.
  • The live browser finish line uses the same verified-label display helper as stored-session summaries, status tables, and TUI headers.
  • The browser status still records switched or already-selected separately, so the lack of a displayable label does not overwrite selection state.

Implementation notes

  • Shared browser-layer formatters avoid a browser-to-CLI dependency.
  • Live and persisted evidence use one stable field order: requestedKey, target, resolvedLabel, status, strategy, verified, source, and capturedAt.
  • Session-level picker evidence applies only to the primary metadata.model; other model runs keep their raw keys.
  • Legacy browser sessions using options.mode receive the same display behavior.
  • TUI action values continue to use raw keys so per-model log lookup is unchanged.
  • API-mode output and routing are unchanged.

Validation

  • pnpm check
  • pnpm build
  • Focused picker, runner, display, status-table, TUI, and integration tests: 181 passed
  • env -u ANTHROPIC_BASE_URL pnpm vitest run: 1,570 passed, 44 skipped across 164 files

The focused coverage includes the two provenance regressions fixed here:

  • a successful picker operation without an observable label cannot produce resolvedLabel=Pro; verified=yes from the requested target;
  • a verified Pro label appears in the live browser finish line, while unverified evidence retains the requested key.

Real persisted-browser evidence

A completed signed-in browser session stored requestedKey=gpt-5.5-pro, target=Pro, resolvedLabel=Pro, and verified=yes. Replaying that real metadata through the PR produced:

Recent Sessions
Status    Model
completed Pro

Models:
- Pro (requested gpt-5.5-pro) — completed
Browser evidence:
- model requestedKey=gpt-5.5-pro; target=Pro; resolvedLabel=Pro; status=already-selected; strategy=select; verified=yes; source=chatgpt-model-picker

The interactive TUI for the same persisted session showed:

Model: Pro
Models:
 - Pro (requested gpt-5.5-pro) — completed

The TUI log action still used gpt-5.5-pro, proving display formatting does not alter log routing. Local paths, conversation identifiers, and browser profile details are omitted.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. labels Jul 12, 2026
@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed August 2, 2026, 1:19 PM ET / 17:19 UTC.

ClawSweeper review

What this changes

The PR separates requested Oracle browser model keys from ChatGPT picker targets and displays a picker label in CLI and TUI output only when the label was observed and verified.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

This PR remains necessary: current main still promotes an unobserved picker target into a resolved browser-model label and marks non-current selection as verified. The PR replaces that behavior with observed-only evidence and consistent presentation across browser launch, completion, session tables, and the TUI; its head is reconciled with current main, and an owner has reviewed the final repair and recommended landing after checks complete.

Priority: P2
Reviewed head: c1f7debd82744ce0f5ab375230d45e41c9c17e3e

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) The patch has a clear provenance boundary, broad focused coverage, reconciled current-main ancestry, and useful real persisted-browser output; the only remaining gate is the still-running platform check.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The PR body includes redacted after-fix terminal/TUI output replaying persisted signed-in browser evidence and confirms that raw model keys still drive TUI log actions.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The PR body includes redacted after-fix terminal/TUI output replaying persisted signed-in browser evidence and confirms that raw model keys still drive TUI log actions.
Evidence reviewed 5 items Current-main defect: Current main falls back from a missing observed picker label to desiredModel and sets verified for every non-current strategy, so a picker target can be presented as verified selection evidence.
Narrow evidence boundary on PR head: The PR stores only a non-empty observed label as resolvedLabel, validates only positive observed labels, and requires both an explicit strategy and observed label before setting verified=true.
Display integration: The new browser-layer formatter uses verified evidence for a display name and otherwise retains the requested key, including the explicit Pro (requested gpt-5.5-pro) form when the two differ.
Findings None None.
Security None None.

How this fits together

Oracle’s browser engine maps a requested CLI model key to a ChatGPT picker target, then records selection evidence in session metadata. CLI completion output, session listings, and the interactive TUI consume that metadata, so they must distinguish intended selection from an observed ChatGPT label without changing execution or log-routing keys.

flowchart LR
  A[CLI model key] --> B[Browser configuration]
  B --> C[ChatGPT picker target]
  C --> D[Observed picker result]
  D --> E[Verified selection evidence]
  E --> F[Model display formatter]
  F --> G[CLI session and TUI output]
Loading

Before merge

  • Resolve merge risk (P1) - This deliberately changes human-facing browser CLI/TUI text; scripts that scrape those strings may need to handle target=…; requested=… before selection and verified picker labels afterward.
  • Resolve merge risk (P1) - The macOS build check was still in progress in the supplied PR state, so landing should wait for that exact head’s required checks to become terminal.
  • Complete next step (P2) - No code repair is needed; wait for the pending macOS check on the reconciled head, then follow the owner’s explicit landing recommendation.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch scope 17 files affected; 669 added, 73 removed The change reaches every browser-model presentation boundary plus focused regression coverage, so review should treat it as a coordinated display-contract update rather than a one-line logging tweak.
Focused validation 149 focused tests reported after final reconciliation The owner’s final review reports picker, display, session, and build coverage after repairing the unlabeled-success path.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #317
Summary: This PR is the active, evidence-backed candidate fix for the browser-model provenance problem tracked by the linked issue; the separately merged durable-conversation-URL work is adjacent but distinct.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Land the explicit display contract (recommended)
    Accept the intentional human-output change after the remaining required check completes, because routing and raw model keys remain unchanged and the new formatter has focused regression coverage.
  2. Pause for a compatibility commitment
    Hold the PR only if Oracle intends browser CLI text to be a stable machine-readable interface, then define and document that compatibility contract before changing these strings.

Technical review

Best possible solution:

Land the evidence-backed display contract after the remaining required check completes, preserving raw requested keys for routing while exposing observed browser labels only when verification is recorded.

Do we have a high-confidence way to reproduce the issue?

Yes, from current source: a non-current browser selection with no observed label falls back to the requested picker target and is marked verified in src/browser/actions/modelSelection.ts. The supplied PR evidence also shows the desired after-fix persisted-session output, but this review did not run a signed-in browser session.

Is this the best way to solve the issue?

Yes. A shared browser-layer formatter plus observed-only selection evidence is the narrow maintainable fix because it keeps browser intent, observed UI state, and raw execution keys separate at every display boundary.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 421b3bfc1a55.

Labels

Label justifications:

  • P2: This is a bounded browser-mode display and provenance correction with meaningful but non-emergency user impact.
  • merge-risk: 🚨 compatibility: Merging changes established CLI and TUI browser-model text, which can affect users or scripts that parse those human-facing outputs.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes redacted after-fix terminal/TUI output replaying persisted signed-in browser evidence and confirms that raw model keys still drive TUI log actions.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted after-fix terminal/TUI output replaying persisted signed-in browser evidence and confirms that raw model keys still drive TUI log actions.

Evidence

What I checked:

  • Current-main defect: Current main falls back from a missing observed picker label to desiredModel and sets verified for every non-current strategy, so a picker target can be presented as verified selection evidence. (src/browser/actions/modelSelection.ts:70, 421b3bfc1a55)
  • Narrow evidence boundary on PR head: The PR stores only a non-empty observed label as resolvedLabel, validates only positive observed labels, and requires both an explicit strategy and observed label before setting verified=true. (src/browser/actions/modelSelection.ts:68, c1f7debd8274)
  • Display integration: The new browser-layer formatter uses verified evidence for a display name and otherwise retains the requested key, including the explicit Pro (requested gpt-5.5-pro) form when the two differ. (src/browser/modelDisplay.ts:31, c1f7debd8274)
  • Reconciled branch and maintainer provenance: The PR head is a merge commit whose second parent is current main; its preceding maintainer repair commit f1d12438 specifically fixed the unlabeled-success path. The supplied owner comments report focused tests, build validation, and recommend landing. (src/browser/actions/modelSelection.ts:68, c1f7debd8274)
  • Release/main status: No local release tag contains the PR head, and current main has no src/browser/modelDisplay.ts; this change is neither shipped in v0.16.1 nor already implemented on main. (src/browser/modelDisplay.ts:1, c1f7debd8274)

Likely related people:

  • steipete: Peter Steinberger authored the targeted unlabeled-selection repair and both reconciliation commits, and supplied the owner review recommending landing. (role: recent area contributor and merger; confidence: high; commits: f1d12438d8f2, ea47e7ce2c64, c1f7debd8274; files: src/browser/actions/modelSelection.ts, CHANGELOG.md)
  • DragonFSKY: The repository history identifies DragonFSKY as the author of the GPT-5.6 Sol browser alias work, and this PR continues that model-label provenance surface. (role: browser model-alias contributor; confidence: high; commits: f2f4a6c35605, f19dc537dd34, ec844ca3c056; files: src/browser/actions/modelSelection.ts, src/browser/modelDisplay.ts)
  • Dongwoo Jeong: Recent main history shows several fixes separating GPT-5.6 Sol from Pro effort and picker behavior, directly adjacent to this display-provenance contract. (role: recent adjacent browser-model contributor; confidence: medium; commits: ea8b1b57f140, aa1efd32daf3, 52afe0850ecb; files: src/browser/actions/modelSelection.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Wait for the macOS build check on c1f7debd82744ce0f5ab375230d45e41c9c17e3e to complete successfully before landing.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (27 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-01T18:19:42.997Z sha ec844ca :: needs maintainer review before merge. :: none
  • reviewed 2026-08-01T19:46:33.128Z sha ec844ca :: needs maintainer review before merge. :: none
  • reviewed 2026-08-01T23:23:07.718Z sha ec844ca :: needs changes before merge. :: [P2] Allow unlabeled GPT-5.6 Sol selections to remain unverified
  • reviewed 2026-08-02T05:33:16.367Z sha f1d1243 :: needs maintainer review before merge. :: none
  • reviewed 2026-08-02T06:00:08.085Z sha ea47e7c :: needs maintainer review before merge. :: none
  • reviewed 2026-08-02T10:20:48.770Z sha ea47e7c :: needs maintainer review before merge. :: none
  • reviewed 2026-08-02T12:20:53.324Z sha ea47e7c :: needs maintainer review before merge. :: none
  • reviewed 2026-08-02T16:19:30.168Z sha ea47e7c :: needs maintainer review before merge. :: none

@DragonFSKY

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Addressed the TUI consistency finding, added strategy-aware pre-selection provenance, prevented session evidence from leaking across model runs, unified live/persisted evidence formatting, and added redacted real persisted-browser replay output to the PR description.

@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@DragonFSKY
DragonFSKY marked this pull request as ready for review July 12, 2026 04:47
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 12, 2026
@DragonFSKY

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Addressed the evidence-provenance and live finish-line findings in 72a17e83a88df138ed1f9dcbf5424d70320e2be7:

  • resolvedLabel is now observed-only; picker targets and effort-only labels cannot become verified display evidence.
  • The live browser finish line now uses the verified display label and falls back to the requested key for unverified evidence.
  • Added focused regressions for both boundaries.

Validation: pnpm check, pnpm build, focused 181 tests, and full suite 1,547 passed / 44 skipped. All GitHub CI and GitGuardian checks are green.

@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. and removed rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. labels Jul 12, 2026
@DragonFSKY
DragonFSKY force-pushed the codex/browser-model-provenance branch from 72a17e8 to ec844ca Compare July 13, 2026 02:53
@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 13, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Jul 30, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 31, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Aug 2, 2026
@steipete

steipete commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Maintainer repair pushed in f1d12438.

The no-label success path now records GPT-5.6 Sol as explicitly unverified instead of running strict label validation against an empty string and throwing. Positive observed labels remain strictly validated.

Proof:

  • 144 focused picker/display/session tests passed
  • format, typecheck, lint, and build passed
  • built-library invocation returned status=already-selected, resolvedLabel=null, and verified=false for gpt-5.6-sol
  • independent Codex autoreview found no actionable defect

Recommendation: LAND once refreshed checks complete.

@steipete

steipete commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Final reconciliation pushed in ea47e7ce: current main is merged, and the #317 release note now sits under 0.16.2 — Unreleased with @DragonFSKY credit instead of the shipped 0.16.0 section. The refreshed branch passes 149 focused tests, check, build, and autoreview.

@steipete
steipete merged commit 332cc86 into steipete:main Aug 2, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Browser: distinguish requested model keys from verified picker labels

2 participants