refactor(auto-mode): Enhance revision prompt customization #714
Conversation
…ormat validation - Updated the revision prompt generation to utilize a customizable template, allowing for dynamic insertion of plan version, previous plan content, user feedback, and task format examples. - Added validation to ensure the presence of a tasks block in the revised specification, with clear instructions on the required format to prevent execution issues. - Introduced logging for scenarios where no tasks are found in the revised plan, warning about potential fallback to single-agent execution.
Summary of ChangesHello @Shironex, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly improves the robustness and flexibility of the auto-mode's plan revision process. By introducing a templated approach for revision prompts, it allows for more dynamic and context-aware communication with the AI. Furthermore, it strengthens the system's ability to ensure AI-generated plans adhere to required formats, specifically by validating the presence and structure of the 'tasks' block, and providing clear warnings when these critical elements are missing. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
📝 WalkthroughWalkthroughThe pull request replaces hard-coded plan revision prompt generation with a dynamic, customizable template system. It retrieves revision templates via configuration, applies placeholder substitutions based on planning mode, and adds runtime guards that compute and validate task extraction from revised plans while emitting observability events for zero-task scenarios. Changes
Sequence DiagramsequenceDiagram
participant AutoModeService as Auto-Mode Service
participant SettingsService as Settings Service
participant PromptSystem as Prompt System
participant PlanningEngine as Planning Engine
participant TaskValidator as Task Validator
AutoModeService->>SettingsService: getPromptCustomization('[AutoMode]')
SettingsService-->>AutoModeService: revision prompt config
AutoModeService->>PromptSystem: retrieve planRevisionTemplate
PromptSystem-->>AutoModeService: template with placeholders
AutoModeService->>AutoModeService: substitute placeholders<br/>(planVersion, feedback, mode, taskFormat)
AutoModeService->>PlanningEngine: generate revised plan with<br/>customized prompt
PlanningEngine-->>AutoModeService: currentPlanContent
AutoModeService->>TaskValidator: extract tasks from<br/>revised plan
TaskValidator-->>AutoModeService: revisedTasks
alt spec/full mode & no tasks
AutoModeService->>AutoModeService: log warning
AutoModeService->>AutoModeService: emit plan_revision_warning event
end
AutoModeService->>AutoModeService: update planSpec & emit progress
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly enhances the revision prompt customization by introducing a templating mechanism, allowing for dynamic insertion of plan version, previous plan content, user feedback, and task format examples. Additionally, it adds crucial validation to ensure the presence of a tasks block in revised specifications, preventing potential execution issues and providing clear warnings when tasks are missing. These changes improve the system's flexibility, robustness, and the quality of AI-generated plans.
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{planVersion\}\}/g, | ||
| String(planVersion - 1) | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{previousPlan\}\}/g, | ||
| hasEdits | ||
| ? approvalResult.editedPlan || currentPlanContent | ||
| : currentPlanContent | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{userFeedback\}\}/g, | ||
| approvalResult.feedback || | ||
| 'Please revise the plan based on the edits above.' | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{planningMode\}\}/g, | ||
| planningMode | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{taskFormatExample\}\}/g, | ||
| taskFormatExample | ||
| ); |
There was a problem hiding this comment.
The repetitive use of .replace() for each placeholder can make the code less readable and harder to maintain as the number of placeholders increases. Consider encapsulating this logic within a helper function or using a more declarative approach for string templating to make the code cleaner and more scalable.
| logger.warn( | ||
| `WARNING: Revised plan in ${planningMode} mode has no tasks! ` + | ||
| `This will cause fallback to single-agent execution. ` + | ||
| `The AI may have omitted the required \`\`\`tasks block.` |
There was a problem hiding this comment.
The warning message uses string concatenation, which can be less readable for multi-line strings or when interpolating variables. Using a template literal (``) would improve clarity and maintainability.
`WARNING: Revised plan in ${planningMode} mode has no tasks!\nThis will cause fallback to single-agent execution.\nThe AI may have omitted the required \`\`\`tasks block.`![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@apps/server/src/services/auto-mode-service.ts`:
- Around line 4600-4647: The current sequence of revisionPrompt =
revisionPrompt.replace(..., replacementString) risks special "$" token expansion
when replacementString contains patterns like "$1" or "$&"; update the
replacements that use user/content variables (specifically the replacements for
{{previousPlan}} and {{userFeedback}} which use approvalResult.editedPlan,
currentPlanContent, and approvalResult.feedback) to pass a replacer function
instead of a raw string (e.g., .replace(regex, () => String(...))) so the
replacement is treated as literal text; keep the other replacements
(planVersion, planningMode, taskFormatExample) consistent but you can also
switch them to replacer functions for uniformity; locate and modify uses around
revisionPrompt and revisionPrompts.taskExecution.planRevisionTemplate.
| // Build revision prompt using customizable template | ||
| const revisionPrompts = await getPromptCustomization( | ||
| this.settingsService, | ||
| '[AutoMode]' | ||
| ); | ||
|
|
||
| ## Instructions | ||
| Please regenerate the specification incorporating the user's feedback. | ||
| Keep the same format with the \`\`\`tasks block for task definitions. | ||
| After generating the revised spec, output: | ||
| "[SPEC_GENERATED] Please review the revised specification above." | ||
| `; | ||
| // Get task format example based on planning mode | ||
| const taskFormatExample = | ||
| planningMode === 'full' | ||
| ? `\`\`\`tasks | ||
| ## Phase 1: Foundation | ||
| - [ ] T001: [Description] | File: [path/to/file] | ||
| - [ ] T002: [Description] | File: [path/to/file] | ||
| ## Phase 2: Core Implementation | ||
| - [ ] T003: [Description] | File: [path/to/file] | ||
| - [ ] T004: [Description] | File: [path/to/file] | ||
| \`\`\`` | ||
| : `\`\`\`tasks | ||
| - [ ] T001: [Description] | File: [path/to/file] | ||
| - [ ] T002: [Description] | File: [path/to/file] | ||
| - [ ] T003: [Description] | File: [path/to/file] | ||
| \`\`\``; | ||
|
|
||
| let revisionPrompt = revisionPrompts.taskExecution.planRevisionTemplate; | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{planVersion\}\}/g, | ||
| String(planVersion - 1) | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{previousPlan\}\}/g, | ||
| hasEdits | ||
| ? approvalResult.editedPlan || currentPlanContent | ||
| : currentPlanContent | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{userFeedback\}\}/g, | ||
| approvalResult.feedback || | ||
| 'Please revise the plan based on the edits above.' | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{planningMode\}\}/g, | ||
| planningMode | ||
| ); | ||
| revisionPrompt = revisionPrompt.replace( | ||
| /\{\{taskFormatExample\}\}/g, | ||
| taskFormatExample | ||
| ); |
There was a problem hiding this comment.
Prevent $ token expansion in revision prompt substitutions.
Using regex .replace() with raw plan/feedback can mangle content containing $1, $&, etc. (common in shell/regex examples). Use a replacer function to keep literal text.
🔧 Suggested safe replacement
- revisionPrompt = revisionPrompt.replace(
- /\{\{planVersion\}\}/g,
- String(planVersion - 1)
- );
+ revisionPrompt = revisionPrompt.replace(
+ /\{\{planVersion\}\}/g,
+ () => String(planVersion - 1)
+ );
revisionPrompt = revisionPrompt.replace(
/\{\{previousPlan\}\}/g,
- hasEdits
- ? approvalResult.editedPlan || currentPlanContent
- : currentPlanContent
+ () =>
+ hasEdits
+ ? approvalResult.editedPlan || currentPlanContent
+ : currentPlanContent
);
revisionPrompt = revisionPrompt.replace(
/\{\{userFeedback\}\}/g,
- approvalResult.feedback ||
- 'Please revise the plan based on the edits above.'
+ () =>
+ approvalResult.feedback ||
+ 'Please revise the plan based on the edits above.'
);
revisionPrompt = revisionPrompt.replace(
/\{\{planningMode\}\}/g,
- planningMode
+ () => planningMode
);
revisionPrompt = revisionPrompt.replace(
/\{\{taskFormatExample\}\}/g,
- taskFormatExample
+ () => taskFormatExample
);🤖 Prompt for AI Agents
In `@apps/server/src/services/auto-mode-service.ts` around lines 4600 - 4647, The
current sequence of revisionPrompt = revisionPrompt.replace(...,
replacementString) risks special "$" token expansion when replacementString
contains patterns like "$1" or "$&"; update the replacements that use
user/content variables (specifically the replacements for {{previousPlan}} and
{{userFeedback}} which use approvalResult.editedPlan, currentPlanContent, and
approvalResult.feedback) to pass a replacer function instead of a raw string
(e.g., .replace(regex, () => String(...))) so the replacement is treated as
literal text; keep the other replacements (planVersion, planningMode,
taskFormatExample) consistent but you can also switch them to replacer functions
for uniformity; locate and modify uses around revisionPrompt and
revisionPrompts.taskExecution.planRevisionTemplate.
Shironex
deleted the
feature/bug-request-changes-on-plan-mode-is-not-proceedin-8xpd
branch
…ormat validation
Updated the revision prompt generation to utilize a customizable template, allowing for dynamic insertion of plan version, previous plan content, user feedback, and task format examples.
Added validation to ensure the presence of a tasks block in the revised specification, with clear instructions on the required format to prevent execution issues.
Introduced logging for scenarios where no tasks are found in the revised plan, warning about potential fallback to single-agent execution.
Closes [Bug]: request changes on plan mode is not proceeding #712
Summary by CodeRabbit
New Features
Improvements
✏️ Tip: You can customize this high-level summary in your review settings.