fix: eliminate repeated JSONL delta reconstruction that starves extension host#565
Open
tianzheng-zhou wants to merge 2 commits intorajbos:mainfrom
Open
Conversation
…sion host Root cause: the extension host's single-threaded event loop was blocked by repeated synchronous split+JSON.parse+applyDelta loops on the same large delta-based JSONL files across multiple analysis helpers, triggering the VS Code unresponsive watchdog and crash-restart loop. Three fixes: 1. usageAnalysis.ts: the delta-based JSONL early-return branch in analyzeSessionUsage now computes model switching inline from the already-reconstructed sessionState instead of calling calculateModelSwitching (which re-read the file and called getModelUsageFromSession for yet another re-read). The non-delta JSONL and regular JSON paths now pass preloadedContent through to calculateModelSwitching and trackEnhancedMetrics to avoid re-reads. 2. extension.ts: removed the hidden pre-warm of calculateUsageAnalysisStats that ran even when the analysis panel was not open. This triggered workspace customization scans and JSONL processing on every 5-minute timer tick, amplifying the event-loop starvation on startup. 3. extension.ts: replaced hand-rolled synchronous applyDelta loops in the session details and log viewer paths with reconstructJsonlStateAsync, a new helper in tokenEstimation.ts that yields to the event loop every 500 lines to prevent blocking.
tianzheng-zhou
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses extension-host unresponsiveness/crash loops caused by repeatedly re-reading and synchronously reconstructing large delta-based JSONL Copilot session logs, especially during periodic background updates and usage analysis.
Changes:
- Avoid redundant JSONL file reads/re-parses in usage analysis by passing preloaded content through and computing delta-based model switching from already-reconstructed state.
- Stop pre-warming usage analysis stats when the analysis panel isn’t open (prevents heavy background work every timer tick).
- Introduce an async delta-state reconstruction helper that periodically yields to the event loop, and use it in session details/log viewer paths.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
vscode-extension/src/usageAnalysis.ts |
Thread preloaded content through model switching/enhanced metrics and avoid re-reading delta JSONL by computing switching from reconstructed state. |
vscode-extension/src/tokenEstimation.ts |
Add reconstructJsonlStateAsync() to reconstruct delta-based JSONL while yielding to the event loop. |
vscode-extension/src/extension.ts |
Remove hidden usage analysis pre-warm; use async JSONL reconstruction in session details/log viewer to reduce event-loop starvation. |
Comment on lines
+1286
to
+1296
| const models: string[] = []; | ||
| for (const req of requests) { | ||
| if (!req || !req.requestId) { continue; } | ||
| let reqModel = 'gpt-4o'; | ||
| if (req.modelId) { | ||
| reqModel = req.modelId.replace(/^copilot\//, ''); | ||
| } else if (req.result?.metadata?.modelId) { | ||
| reqModel = req.result.metadata.modelId.replace(/^copilot\//, ''); | ||
| } else if (req.result?.details) { | ||
| reqModel = getModelFromRequest(req, deps.modelPricing); | ||
| } |
There was a problem hiding this comment.
In the delta-based JSONL branch, the inline model-switching logic defaults to 'gpt-4o' when a request lacks modelId/result.metadata.modelId. Reconstructed requests can legitimately omit modelId (the logviewer path already falls back to sessionState.inputState.selectedModel), so this can misattribute models and under/over-count switches/tiers for sessions that used a different selected model. Consider deriving a defaultModel from sessionState.inputState?.selectedModel (identifier/metadata.id) and using it as the fallback before hard-coding a default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #566
Root cause: the extension host's single-threaded event loop was blocked by repeated synchronous split+JSON.parse+applyDelta loops on the same large delta-based JSONL files across multiple analysis helpers, triggering the VS Code unresponsive watchdog and crash-restart loop.
Three fixes:
usageAnalysis.ts: the delta-based JSONL early-return branch in analyzeSessionUsage now computes model switching inline from the already-reconstructed sessionState instead of calling calculateModelSwitching (which re-read the file and called getModelUsageFromSession for yet another re-read). The non-delta JSONL and regular JSON paths now pass preloadedContent through to calculateModelSwitching and trackEnhancedMetrics to avoid re-reads.
extension.ts: removed the hidden pre-warm of calculateUsageAnalysisStats that ran even when the analysis panel was not open. This triggered workspace customization scans and JSONL processing on every 5-minute timer tick, amplifying the event-loop starvation on startup.
extension.ts: replaced hand-rolled synchronous applyDelta loops in the session details and log viewer paths with reconstructJsonlStateAsync, a new helper in tokenEstimation.ts that yields to the event loop every 500 lines to prevent blocking.