Enhance slop detection and update sloplist.json#1
Conversation
|
@Louis27940 is attempting to deploy a commit to the dan-1729's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdded a new Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client (UI)
participant Scorer as Scoring Service (`scripts/scoring.ts`)
participant Data as Slop Data (`data/sloplist.json`)
Client->>Scorer: Submit response text
Scorer->>Data: Load slop items (opener, validation, ending)
Scorer->>Scorer: detectEnding(last 220 chars) / other detectors
Scorer-->>Client: Return slop_hits (including ending_slop)
Client->>Client: Render breakdown (includes ending_slop)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR expands SlopBench’s phrase detection by introducing an “ending_slop” category to catch common assistant-style closing phrases, wiring it into scoring, and surfacing it in the UI breakdown.
Changes:
- Added
ending_slopcategory and items todata/sloplist.json. - Implemented
detectEndingand integrated it intoscoreResponseinscripts/scoring.ts. - Updated
ResponseExplorerto includeending_slopin displayed category breakdown.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
src/components/ResponseExplorer.jsx |
Adds ending_slop to the list of categories shown in the breakdown UI. |
scripts/scoring.ts |
Adds end-of-response phrase detection and includes it in response scoring. |
data/sloplist.json |
Expands slop phrase lists and introduces the new ending_slop category metadata/items. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const tail = text.slice(-220).toLowerCase(); | ||
| const hits: SlopHit[] = []; | ||
| for (const item of items) { | ||
| if (tail.includes(item.toLowerCase())) { |
There was a problem hiding this comment.
detectEnding is labeled/used as an end-of-response detector, but it currently uses tail.includes(...), which will flag phrases anywhere in the last 220 chars (not necessarily as a closing). This can create false positives and makes the implementation inconsistent with string_match_response_end semantics; consider trimming trailing whitespace/punctuation and using endsWith, or requiring the match index to be within a small window of the actual end.
| const tail = text.slice(-220).toLowerCase(); | |
| const hits: SlopHit[] = []; | |
| for (const item of items) { | |
| if (tail.includes(item.toLowerCase())) { | |
| const normalizeEnding = (value: string): string => | |
| value.toLowerCase().replace(/[\s.!?,;:'")\]]+$/g, ""); | |
| const tail = normalizeEnding(text.slice(-220)); | |
| const hits: SlopHit[] = []; | |
| for (const item of items) { | |
| const normalizedItem = normalizeEnding(item); | |
| if (normalizedItem && tail.endsWith(normalizedItem)) { |
| const tail = text.slice(-220).toLowerCase(); | ||
| const hits: SlopHit[] = []; |
There was a problem hiding this comment.
The 220 tail length in text.slice(-220) is an unexplained magic number. Please pull this into a named constant (and ideally comment/justify it) so it’s clear how much of the response is considered “the end” and easy to tune later.
| "If you want, I can help you", | ||
| "Feel free to ask" |
There was a problem hiding this comment.
ending_slop.items contains both "Feel free to ask if you have any more questions" and the shorter substring "Feel free to ask". With the current substring matching, the longer phrase will trigger both entries and double-count hits. Consider removing/rewriting the shorter entry for this category, or updating the detection logic to prefer the longest match / deduplicate overlaps.
| "If you want, I can help you", | |
| "Feel free to ask" | |
| "If you want, I can help you" |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This pull request enhances the detection of overused phrases in LLM responses by adding a new category for closing phrases ("ending slop") and updating the data and detection logic. It also expands the lists of common opening and validation phrases.
Improvements to slop phrase detection :
Added a new "ending_slop" category to "sloplist.json" for detecting overused LLM closing phrases, with a description, detection type, and sample phrases.
Implemented the "detectEnding" function in "scripts/scoring.ts" to scan the end of responses for these closing phrases and integrated it into the overall scoring logic.
Updated the "SlopList" type in "scripts/scoring.ts" to include the new "ending_slop" category.
Expansion of slop phrase lists :
Update of the display of the category of slop detected :
Summary by CodeRabbit