fix: prompt_score scope scoring no longer rewards length over clarity - #248
Open
TerminalGravity wants to merge 1 commit into
Open
fix: prompt_score scope scoring no longer rewards length over clarity#248TerminalGravity wants to merge 1 commit into
TerminalGravity wants to merge 1 commit into
Conversation
Previously, any prompt >100 chars got full scope points (25/25) regardless of whether scope was actually clear. A rambling 150-char prompt with no bounding language scored the same as a tightly scoped one. Now scope scoring is purely semantic: - Bounding words (only, just, specific, this) → 25 pts - Broad words (all, every, entire) → 10 pts (with feedback) - Neither → 15 pts (with suggestion to scope) Also exports scorePrompt for direct testing and adds 13 unit tests covering all scoring dimensions and the regression case.
TerminalGravity
commented
Mar 13, 2026
TerminalGravity
left a comment
TerminalGravity
left a comment
Collaborator
Author
There was a problem hiding this comment.
Good catch — text.length > 100 was sneaking full scope points to verbose-but-vague prompts. The bounding-vs-broad logic is much more intentional. One thought: the fallback score went 10→15 for unclear scope — might be worth a brief comment explaining the reasoning so future contributors don't second-guess it. ✅
TerminalGravity
commented
Mar 16, 2026
TerminalGravity
left a comment
TerminalGravity
left a comment
Collaborator
Author
There was a problem hiding this comment.
Good fix. The old text.length > 100 giving full scope points was rewarding verbosity, not clarity. New bounding-word logic is better, and the regression test nails it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
prompt_scoregave full scope points (25/25) to any prompt longer than 100 characters, regardless of whether scope was actually defined. A rambling prompt like "I want you to look at the codebase and think about what could be improved and then maybe do some things" scored the same as "Fix only the login validation in auth.ts".Fix
Removed the
text.length > 100shortcut. Scope is now scored purely on semantic signals:only,just,specific,this) → 25 ptsall,every,entire) → 10 pts + feedbackAlso bumped the default 'neither' case from 10→15 (no scope indicators isn't as bad as actively broad).
Tests
Added 13 unit tests for
scorePromptcovering all dimensions + the regression case. Exported the function for direct testing.