Add 2 slopwords as well as a new prompt#2
Conversation
Added new multi-word phrases to the sloplist.
|
@Antix5 is attempting to deploy a commit to the dan-1729's projects Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughThe pull request adds a new technical prompt entry and two corresponding phrase-level slop detection entries to the data configuration files, expanding the detection library with content about rate limiting and related terminology. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
data/sloplist.json (1)
134-134: "smoking gun" may not be AI-specific enough."Smoking gun" is a common English idiom meaning "conclusive evidence" that predates LLMs and is regularly used by humans in both technical and non-technical contexts. While it's possible that some LLMs overuse it, the existing phrases in this list are more distinctly AI fingerprints (e.g., "actionable insights", "a testament to", "delve into").
Consider whether this phrase truly represents AI slop or just standard English usage.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@data/sloplist.json` at line 134, The phrase "smoking gun" in the sloplist is not distinctly AI-origin and should be removed or replaced; locate the "smoking gun" entry and either delete it from the list or swap it for a more AI-specific slop phrase (e.g., a suggested replacement), then run any list-based tests or linters to ensure the JSON remains valid.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@data/prompts.json`:
- Around line 387-394: Prompt id 54 conflicts with the sloplist: it expects a
correct technical diagnosis ("thundering herd") but that phrase was added to the
sloplist, causing valid responses to be flagged; fix by either removing the
"thundering herd" entry from the sloplist (the entry you added in
data/sloplist.json) so the prompt can validly elicit that term, or reword prompt
id 54 to avoid prompting for the specific term (describe bursty retry behavior
and ask for mitigation strategies without naming "thundering herd"); update
whichever you choose and run tests to ensure no other prompts rely on that
sloplist entry.
- Line 390: Update the JSON entry whose "prompt" value contains the typos:
change "fell" to "feel" and "comming" to "coming" so the prompt reads correctly
(e.g., "I'm currently working on a project where I have bursts of requests to my
server, I have a rate limiter but I feel like all the requests are coming back
at the same time"); edit the "prompt" string in data/prompts.json (the "prompt"
key shown in the diff) to make these spelling fixes.
- Line 393: Update the "notes" field in the JSON entry so it no longer claims
humans "rarely use" the term "thundering herd"; instead either remove that
comparative claim or replace it with an accurate neutral statement (e.g., "Some
LLMs like the term 'thundering herd' — it is also a well-established term in
systems programming and CS literature"). Locate and edit the "notes" property in
the same prompt object (the JSON key "notes") to reflect the corrected wording.
---
Nitpick comments:
In `@data/sloplist.json`:
- Line 134: The phrase "smoking gun" in the sloplist is not distinctly AI-origin
and should be removed or replaced; locate the "smoking gun" entry and either
delete it from the list or swap it for a more AI-specific slop phrase (e.g., a
suggested replacement), then run any list-based tests or linters to ensure the
JSON remains valid.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c1321d09-b6e9-46d1-a269-06978962d97e
📒 Files selected for processing (2)
data/prompts.jsondata/sloplist.json
| }, | ||
| { | ||
| "id": 54, | ||
| "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I fell like all the request are comming back at the same time", | ||
| "topic": "tech", | ||
| "primary_slop_target": "phrase_level", | ||
| "notes": "Some LLM love the term thundering herd even if human rarely use that term to describe bursty patterns" | ||
| } |
There was a problem hiding this comment.
This prompt contradicts the sloplist—it elicits a correct technical answer that will be flagged as slop.
The prompt describes a classic "thundering herd" problem: burst requests hitting a rate limiter, then all retrying simultaneously. "Thundering herd" is the correct technical term for this scenario. However, because "thundering herd" was added to the sloplist in this same PR, any model that correctly diagnoses the problem will be penalized.
This creates a contradiction: the prompt is designed to test whether models use the term "thundering herd," but using that term correctly should not be considered slop.
Recommendation: Either remove "thundering herd" from the sloplist (see comment on data/sloplist.json:135), or redesign this prompt to test for a different slop pattern.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/prompts.json` around lines 387 - 394, Prompt id 54 conflicts with the
sloplist: it expects a correct technical diagnosis ("thundering herd") but that
phrase was added to the sloplist, causing valid responses to be flagged; fix by
either removing the "thundering herd" entry from the sloplist (the entry you
added in data/sloplist.json) so the prompt can validly elicit that term, or
reword prompt id 54 to avoid prompting for the specific term (describe bursty
retry behavior and ask for mitigation strategies without naming "thundering
herd"); update whichever you choose and run tests to ensure no other prompts
rely on that sloplist entry.
| }, | ||
| { | ||
| "id": 54, | ||
| "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I fell like all the request are comming back at the same time", |
There was a problem hiding this comment.
Fix typos in the prompt text.
The prompt contains two typos:
- "comming" → "coming"
- "fell" → "feel"
📝 Proposed fix
- "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I fell like all the request are comming back at the same time",
+ "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I feel like all the request are coming back at the same time",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I fell like all the request are comming back at the same time", | |
| "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I feel like all the request are coming back at the same time", |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/prompts.json` at line 390, Update the JSON entry whose "prompt" value
contains the typos: change "fell" to "feel" and "comming" to "coming" so the
prompt reads correctly (e.g., "I'm currently working on a project where I have
bursts of requests to my server, I have a rate limiter but I feel like all the
requests are coming back at the same time"); edit the "prompt" string in
data/prompts.json (the "prompt" key shown in the diff) to make these spelling
fixes.
| "prompt": "I'm currently working on a project where I have burst of request to my server, I have a rate limiter but I fell like all the request are comming back at the same time", | ||
| "topic": "tech", | ||
| "primary_slop_target": "phrase_level", | ||
| "notes": "Some LLM love the term thundering herd even if human rarely use that term to describe bursty patterns" |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
How commonly is the term "thundering herd" used in computer science and systems programming literature? Is it a standard term that human engineers use?
💡 Result:
“Yes—thundering herd (more often “thundering herd problem”) is a widely used, human-engineer term in computer systems,” especially in operating systems, networking, and distributed systems/performance contexts.
- It appears in long-standing systems research and practitioner venues. A USENIX paper on Linux server scalability (circa 2000) explicitly describes unnecessary wakeups in
accept()as “commonly referred to as a thundering herd problem.” [1] - It’s used in OS/kernel literature and teaching materials. OS/kernel texts and references describe wake-queue behavior specifically in terms of avoiding the “thundering herd.” [2][3]
- It continues to appear in modern systems/networking papers. Recent conference papers still use the term when discussing overload bursts and mitigation strategies. [4]
- It’s also a common term in engineering practice. The phenomenon is broadly defined and discussed in general computing references, and in industry usage it’s closely related to (and sometimes overlaps with) “cache stampede.” [5]
Is it “standard”? Not a formal standards-body term (like an RFC keyword), but it is a de facto standard term of art that engineers routinely use and will generally understand in systems programming discussions. [1][5]
Sources: [1] [2] [3] [4] [5]
Remove or correct the claim about "thundering herd" usage.
The notes incorrectly state that humans rarely use the term "thundering herd." This term is a well-established technical term in systems programming and CS literature—it appears in peer-reviewed papers, OS documentation, kernel references, and technical discussions. Engineers commonly use it when discussing concurrency patterns and overload scenarios.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@data/prompts.json` at line 393, Update the "notes" field in the JSON entry so
it no longer claims humans "rarely use" the term "thundering herd"; instead
either remove that comparative claim or replace it with an accurate neutral
statement (e.g., "Some LLMs like the term 'thundering herd' — it is also a
well-established term in systems programming and CS literature"). Locate and
edit the "notes" property in the same prompt object (the JSON key "notes") to
reflect the corrected wording.
1 participant
I noticed that when talking with LLM, they love to use words like "thundering herd" when talking about what a human would simply call a burst of request. I also noticed they love to use the words "smoking gun" when they think they found the source of a problem.
In this PR I therefore add the 2 slop words and added a new prompt more targeted toward the detection of the "thundering heard" pattern
Summary by CodeRabbit