Conversation
Add a generic FireworksV1CompletionsClient that handles local tokenization via HuggingFace transformers and calls the /v1/completions endpoint with token-in/token-out. Tool-call parsing is pluggable via a callback rather than hardcoded to any specific domain. Also add TokenDebugView component for the evaluation dashboard with: - Text view: readable colored text with mask overlay (prompt vs completion) - Episode view: per-token visualization with mask or logprob coloring - Turns view: per-turn breakdown of prompt/completion tokens - Token ID chips with hover tooltips showing detokenized text and logprob - Smooth gradient logprob coloring (green=high confidence, red=low) Made-with: Cursor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cd7e0937e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Rewrite tests to match the generic client API (remove references to old domain-specific methods like _parse_tool_call_with_optional_fallback) - Fix TokenDebugSection guard to also check extra?.full_episode - Fix zero reward styled as red/negative — now uses neutral gray - Fix tools=[] vs None: explicit empty list no longer falls back to default_tools Made-with: Cursor
| showIds={showIds} | ||
| /> | ||
| )) | ||
| )} |
There was a problem hiding this comment.
View selector falls through silently without fullEpisode
Medium Severity
The ternary chain for view selection uses viewLevel === "text" && fullEpisode and viewLevel === "episode" && fullEpisode, so when fullEpisode is null but the user has selected "text" or "episode" mode, the condition silently falls through to rendering tokenTurnTraces (the "turns" view). The UI shows the "text" or "episode" button as active/selected while displaying turns content, which is misleading.
…issing fullEpisode - build_assistant_turn_token_ids now passes _thinking_kwargs() to apply_chat_template, consistent with _build_prompt_token_ids and build_tool_response_suffix_token_ids - View selector no longer silently renders nothing when fullEpisode is null and user selected text/episode view — falls back to turns view or shows a placeholder message Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
…ams overriding core fields - Add second None check in _normalize_token_id_sequence after extracting from a Mapping, since .get() can return None even when the key exists - Move request_params spread before explicit keys (model, prompt, temperature, max_tokens, logprobs) so they cannot be silently overridden by stray entries in request_params Made-with: Cursor
benjibc
deleted the
bchen/generic-v1-completions-client-and-token-debug-view
branch
1 participant
Summary
FireworksV1CompletionsClient— a generic, domain-agnostic client for the/v1/completionsendpoint with local tokenization via HuggingFacetransformers. Tool-call parsing is pluggable via atool_call_parsercallback.TokenDebugViewReact component for the evaluation dashboard with three view modes:ParsedToolCallandto_openai_tool_callsas generic typesTest plan
test_fireworks_v1_completions_client.pyunit testsMade with Cursor
Note
Medium Risk
Introduces new model-calling and retry/tokenization logic plus a sizable new UI debug surface; risk is mainly around API/response shape assumptions and tokenizer/template edge cases rather than core auth/data integrity.
Overview
Adds
FireworksV1CompletionsClient, a generic Fireworks/v1/completionswrapper that locally tokenizes chat messages (with fallbacks when chat templates/tools fail), retries transient API errors, extracts token IDs and logprobs, and optionally injects OpenAI-styletool_callsvia a pluggabletool_call_parser(exportingParsedToolCall/to_openai_tool_calls).Updates the Vite evaluation dashboard to show a new
TokenDebugViewwhenexecution_metadata.extraincludestoken_turn_tracesorfull_episode, providing per-token visualization (mask-by-turn or logprob coloring) and per-turn breakdowns; adds unit tests covering token normalization, template retry behavior, thinking flag passthrough, and parser integration.Written by Cursor Bugbot for commit c23e31d. This will update automatically on new commits. Configure here.