Summary
A question that contains another operation's id noun can push the correct operation out of search results entirely — not merely rank it lower. Adding one common word to a question ("Stellar") drops the intended op from rank 4 to absent, while the ops that stay are the ones whose names contain a word from the question.
This is separate from #109 (bare project-name queries reaching no directory op), though both point at how much weight the operation id carries.
Reproduction
Two search calls that differ by one word:
query: "top projects by GitHub activity"
1. searchProjects
2. searchResearch
3. searchRepos
4. getLeaderboard <- the intended answer, present
5. getBuilders
query: "top Stellar projects by GitHub activity"
1. searchProjects
2. searchResearch
3. searchRepos
getLeaderboard <- absent from the result set
Why this looks like a ranking issue rather than a card issue
getLeaderboard's own description begins:
"Ranked list of the top / most active Stellar projects by GitHub activity …"
The failing query is very nearly that sentence. The card also carries activity, most active, top projects, commits, stars, leaderboard in x-routing.keywords. So the words are present in both description and keywords, at the weights the docs describe, and the op is still excluded.
What the surviving ops have in common is that a noun from the question appears in their operation id — projects → searchProjects, repos → searchRepos. Our reading is that id matches dominate strongly enough that an op cannot compete on description or keywords when another op's id contains one of the question's nouns; adding "Stellar" widens the candidate field and pushes the correct op past the result cut.
We can confirm the index is current, so this is not staleness: keywords we published recently do work on direct probes — "ranked by commits" surfaces getLeaderboard, and "in the code" ranks explainRepo first.
Other cases with the same shape
- "jobs, bounties and freelance work for Stellar contributors" →
getRfps absent, though "rfps" alone ranks it first.
- "how does the Blend lending pool calculate interest" →
explainRepo absent, though "in the code" ranks it first.
Why we're reporting rather than patching
We've been closing routing misses by adding vocabulary to our cards, and this class does not respond to that: there is no keyword that outweighs another op's id on that id's own noun. If the intent is that a well-described op should be reachable by a natural question, this seems worth a look at the scorer — perhaps capping id weight, or guaranteeing that an op matching a high proportion of query terms survives the result cut.
Happy to run any probe you'd like against the live endpoint and report what comes back.
Summary
A question that contains another operation's id noun can push the correct operation out of
searchresults entirely — not merely rank it lower. Adding one common word to a question ("Stellar") drops the intended op from rank 4 to absent, while the ops that stay are the ones whose names contain a word from the question.This is separate from #109 (bare project-name queries reaching no directory op), though both point at how much weight the operation id carries.
Reproduction
Two
searchcalls that differ by one word:Why this looks like a ranking issue rather than a card issue
getLeaderboard's own description begins:The failing query is very nearly that sentence. The card also carries
activity,most active,top projects,commits,stars,leaderboardinx-routing.keywords. So the words are present in both description and keywords, at the weights the docs describe, and the op is still excluded.What the surviving ops have in common is that a noun from the question appears in their operation id —
projects→searchProjects,repos→searchRepos. Our reading is that id matches dominate strongly enough that an op cannot compete on description or keywords when another op's id contains one of the question's nouns; adding "Stellar" widens the candidate field and pushes the correct op past the result cut.We can confirm the index is current, so this is not staleness: keywords we published recently do work on direct probes — "ranked by commits" surfaces
getLeaderboard, and "in the code" ranksexplainRepofirst.Other cases with the same shape
getRfpsabsent, though "rfps" alone ranks it first.explainRepoabsent, though "in the code" ranks it first.Why we're reporting rather than patching
We've been closing routing misses by adding vocabulary to our cards, and this class does not respond to that: there is no keyword that outweighs another op's id on that id's own noun. If the intent is that a well-described op should be reachable by a natural question, this seems worth a look at the scorer — perhaps capping id weight, or guaranteeing that an op matching a high proportion of query terms survives the result cut.
Happy to run any probe you'd like against the live endpoint and report what comes back.