Skip to content

Content audit: remove 20 entries, correct 12, add 8 - #1

Merged
Jonathan-Improving merged 7 commits into
jbrinkman:fix/ai-libraries-595from
Jonathan-Improving:audit/aea-691-catalog-decisions
Sep 15, 2026
Merged

Jonathan-Improving merged 7 commits into
jbrinkman:fix/ai-libraries-595from
Jonathan-Improving:audit/aea-691-catalog-decisions

Conversation

@Jonathan-Improving

@Jonathan-Improving Jonathan-Improving commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Content audit: remove 20 entries, correct 12, add 8

Follow-up to valkey-io#602, targeting fix/ai-libraries-595 rather than main so it lands inside the existing PR.

This applies the outcome of a content audit (AEA-691) over every entry on this branch, ahead of the handoff to the Valkey project.

🚧 Still a draft — but no longer just removals

The first commit here was data-only removals. Since then the audit finished, and this PR now carries the field corrections, the descriptions, the new entries and one submission-guide fix. The one open question for you is server modules.

What this PR does now

Entries removed 20 — 1 client, 19 integrations
Entries corrected 12 — licences, repository URLs, install commands, feature flags, descriptions
Entries added 8 — all functionally verified
Guide docsNote documented properly; the self-negating licence rule reworded
Net branch goes 48 → 34 entries (12 clients · 22 integrations)

Commits are ordered so each is reviewable on its own: removals, then field corrections + additions, then the final edits.

Methodology

Every entry started disqualified and had to earn re-inclusion. A remove-on-suspicion audit keeps whatever nobody happens to challenge; deny-by-default forces a positive case for each entry.

1. Functional verification against a live server

32 verification runs, one project per run, each isolated: cloned fresh, built from source where required, exercised against a real Valkey server in its own container on its own port (valkey/valkey:latestvalkey_version 9.1.2; valkey/valkey-bundle where modules were needed).

"It connects" was not a pass. The bar was set per entry type:

  • Queue libraries — a job had to be enqueued and executed by a worker, proven by the job writing a value that was read back. In one case execution was confirmed in a forked work-horse child by comparing PIDs.
  • Vector stores — an embedding written and retrieved by similarity search with correct ranking. For valkey-search itself, distances were hand-computed and matched to float32 precision rather than merely observed to be non-erroring.
  • Clients — the advertised surface, not the handshake. Where a claim was about protocol behaviour, MONITOR captured the actual wire traffic; that is how one client's CLIENT CAPA REDIRECT claim was confirmed rather than assumed.

Every result was cross-checked server-side rather than trusting the client: KEYS, DBSIZE, TYPE, FT._LIST, FT.INFO, INFO server.

Results were graded by confidence — functional proof, reproducible workaround, or static analysis only — with static positives treated as weak and static decisive negatives as strong. Two entries are retained on non-Tier-1 evidence and both say so in the trail, rather than being rounded up.

2. Source inspection, not README reading

An early pass checked READMEs and published API docs and produced false negatives — projects wrongly assessed as having no Valkey support. The method was corrected to: clone, grep the entire tree for every spelling of Valkey, read the dependency manifests directly, and search history for support that was renamed or removed.

That correction changed real outcomes, five times. README absence turned out to carry no information at all:

Project README mentions Actual footprint
litellm 0 379 hits / 56 files, plus e2e CI on real TLS cluster-mode Valkey
localai 0 292 hits, a 1,746-line dedicated backend
langbot 0 176 hits / 12 files, an 830-line backend
semantic-router 0 1,753 hits / 124 files, 3,181 lines of implementation

One of them explains why: its README never enumerates backends at all. A README is a marketing surface, not a dependency manifest.

3. Human ratification of every decision

No entry was removed or retained on tooling judgment. Verification produced verbatim audit trails — roughly 23,000 lines across the 32 runs, recording actual commands and actual output rather than summaries.

Every client and integration trail was read individually, looking for the markers that separate a real integration from an incidental one: which client library is actually imported, whether Valkey is named in the project's own manifests and docs, whether Valkey-specific code, tests or CI exist, whether a version floor is published, and whether the functional proof exercised the project's own Valkey code path rather than something adjacent. Where a trail's conclusion did not hold up against its own raw output, the conclusion was overturned — which is how the reinstatements were caught, and how several claims in our own catalog were found wrong.

Tooling gathered and graded evidence; the inclusion decision was mine.

Classification

Three outcomes rather than two, because "remove" was hiding two different situations:

  • Qualified — stays. Any remaining defects are in our copy, ours to fix.
  • Remediable — removed now, documented route back. Genuine first-class Valkey integration, actively maintained, but the advertised support is broken as shipped. The bar is deliberately usability, not severity: a released fix should bring it back.
  • Disqualified — nothing to fix, nobody home to fix it, or out of scope.

Removals (20)

Disqualified (16) — no route back

No first-class Valkey integration to fix (8). These work only because Valkey speaks RESP: they depend on a Redis client, name Valkey nowhere, and have no Valkey CI, docs, or changelog entry. Adding support would be a feature request upstream, not a correction.

celery · kombu · resque · hangfire · redli · webdis · openmemory-javascript-sdk · trino-connector

redli was re-verified after the finding was challenged: full history unshallowed, all case variants, pickaxe over every commit — zero Valkey references have ever existed in the repo.

Whole project stale or abandoned (5). Not merely the Valkey parts — no maintainer is present to merge a fix: tinywebdis (~10 yr) · medis (~2.5 yr) · predixy (~2.5 yr) · php-resque (~37 mo) · walrus (~7.9 mo, and no Valkey story)

Audience (2). Both serve people developing Valkey itself: valkey-test-framework · valkey-perf-benchmark

Licence (1). phpredis — PHP-3.01 is not OSI-approved, and the guide requires an OSI licence.

vllm — removed, and worth explaining

Removed, but the original reasoning was wrong and has been replaced. The first finding was made against vllm-project/vllm and concluded there was no Valkey support. That is true of that repository — but the Valkey work exists, in the sibling project vllm-project/semantic-router, which we then audited properly.

The cut still stands, on corrected grounds: vllm-project/vllm genuinely has no Valkey support (code search: 0 hits), so an entry named vllm would point readers somewhere that cannot do what the entry claims. The two are separate products — not a fork (fork:false, parent:null), Go vs Python, own domain, own release train.

So semantic-router is added below as its own entry instead. Same conclusion for the reader, arrived at honestly.

Remediable (4) — removed now, reinstatement documented

Genuine first-class integrations on official Valkey clients, actively maintained, but the advertised support is broken as shipped. We intend to raise each upstream.

Entry Integration Defect
recall official @valkey/valkey-glide + dedicated adapter README install command at line 294 installs nothing; the working commands already exist at lines 313 and 494
db-gpt valkey-glide, 870-line vector store 3 undeclared runtime dependencies prevent import dbgpt.core after a documented install
langchain-aws real ValkeyVectorStore on GLIDE from_texts() raises — check_index_exists catches the wrong exception type; unfixed in a released version ~6 months
langchain4j-valkey io.valkey:valkey-glide directly published pom declares glide with an unresolvable ${os.detected.classifier}, so consumers get the native-less jar and fail on first call

Deliberately excluded from these findings: anything that is our problem (catalog wording, install fields) and anything the project documents by design.

One further removal, on a rule rather than a defect

localai — its Valkey support is real and first class (go.mod requires valkey-io/valkey-go), MIT, actively maintained. Nothing is wrong with the project. But its Valkey backend is not in the default image: it is a separately published gallery artifact, so using it is a two-step operation, and installCommand holds one string. Every candidate value was tested and none delivers the advertised capability — docker run localai/localai gets the product without the backend. Rather than add a schema field or ship a misleading one-liner, the entry comes out. It qualifies again if the backend ships in the default image.

Additions (8)

All functionally verified; each has a trail.

Entry Evidence Integration
litellm Tier 1 — FT._LIST → its index, FT.INFO → VECTOR dim 8 COSINE 2nd class, and deliberately so: its own docstring explains that RedisVL cannot drive valkey-search, so they wrote a Valkey-specific backend that bypasses it
langbot Tier 1 — KNN ordering verified arithmetically 1st class, glide's native ft namespace
agno Tier 1 on both surfaces, tested on two servers 1st class, valkey-glide-sync
langflow Tier 1 — components driven from the published wheel mixed: vector store 1st class via glide, chat memory 2nd class via redis-py
semantic-router Tier 2 — FT.CREATE vector lifted verbatim from source, accepted by a live server 1st class, valkey-glide/go/v2
valkey-mcp-server operator-attested 1st class, built on GLIDE
valkey-message-queue-starter operator-attested 1st class, sole datastore dep is @valkey/valkey-glide

semantic-router is named vLLM Semantic Router, not "vLLM" — see above.

Corrections to retained entries (12)

Each was found by testing, not by reading:

  • sidekiq — description said "for Rails applications". Upstream says any Ruby app, and this was disproved functionally by running a worker with no Rails present. Repository also moved mperham/sidekiqsidekiq/sidekiq.
  • valkey-py — feature flags 110000001111000101. Two false negatives proven against a live server: pubsub state restoration and client-side caching both work.
  • valkey-admin — described as a "desktop app". It is web-based.
  • rq — wrong SPDX licence identifier; repository and description corrected.
  • spring-data-valkey — had no installCommand. Its coordinate is a three-segment vendor namespace (io.valkey.springframework.data) that could not have been guessed.
  • Install commands on 5 entries — each tested from a clean environment. Several installed the project but left it unable to reach Valkey at all; one silently omitted the components entirely, giving no error to search for.
  • valkey-glide-c.jsonvalkey-glide-csharp.json — the # was lost in slugification, so the slug read as the language C.

Module requirements are now disclosed, and scoped correctly. Where an entry has two Valkey surfaces and only one needs a module, the qualifier sits mid-sentence next to that surface rather than at the end of the sentence where it would claim both. Two entries were wrong in exactly that way and previously contradicted their own upstream docs, which correctly say plain Valkey is enough.

Question for @jbrinkman — server modules

This branch contains no entries for the Valkey server modules: valkey-search, valkey-json, valkey-bloom, valkey-ldap, valkey-bundle, valkeymodule-rs. They were present in the source data this catalog was built from, and this PR does not add them back — we'd rather ask than assume.

Was that deliberate scoping, or a side effect of the clients + integrations restructure? A server module is neither a client nor an integration, so it may simply have had nowhere to go.

Flagging it because the modules are load-bearing for this very page: during verification, most AI/RAG entries would not function at all on a stock Valkey server — they need valkey-search and/or valkey-json, which is what valkey-bundle provides. Several failed silently, returning 0 or [] rather than erroring.

So the catalog lists integrations that depend on components the catalog itself doesn't mention. Options: add the modules here (a third category, or inside integrations), keep the disclosure in entry descriptions only, cover it in page copy, or point elsewhere if modules are catalogued somewhere better. Happy to follow whichever you prefer.

Two smaller things in the same area, both left for you:

  • The guide's tag list has drifted from the data. Three documented tags now have no membersProxies & gateways, Data movement, Observability & testing — because this audit removed their only occupants. Two tags in active use are undocumented: Clients and Server modules, and the guide states that clients do not use tags while all 12 client entries carry one.
  • Two entries advertise documentation that is broken upstream: one URL 404s, and one project's GitHub homepage field points at a parked for-sale domain. Both are theirs to fix; flagging in case you'd rather report them together.

Verification

manifest.json parses, and its entry list matches the files on disk exactly — 34 = 12 clients + 22 integrations — with no orphaned references in either direction. Every entry file parses. Every description is within the guide's 200-character limit. isFirstParty is present on all 34.

Applies the outcome of content audit AEA-691 to the entry data on this
branch. Data-only: no template, style, or page-copy changes.

Disqualified (17):
- No first-class Valkey integration to fix (9): celery, kombu, resque,
  hangfire, redli, webdis, vllm, openmemory-javascript-sdk,
  trino-connector. These work only via RESP compatibility, depend on a
  Redis client, and name Valkey nowhere in code, CI, docs or changelog.
- Whole project stale or abandoned (5): tinywebdis, medis, predixy,
  php-resque, walrus. Not just the Valkey parts, so no fix could land.
- Audience (2): valkey-test-framework, valkey-perf-benchmark. Both serve
  people developing Valkey itself rather than people building on it.
- Licence (1): phpredis. PHP-3.01 is not OSI-approved.

Remediable (4) - removed now, reinstatement documented:
- recall: README install command at line 294 installs nothing
- db-gpt: 3 undeclared runtime deps break `import dbgpt.core`
- langchain-aws: from_texts() raises; wrong exception type caught
- langchain4j-valkey: published pom's unresolvable
  ${os.detected.classifier} yields a native-less glide jar
These have genuine first-class Valkey integrations on official Valkey
clients and are actively maintained, but the support they advertise is
broken or incomplete as shipped. Each should return once a fix ships in
a released version.

Verification: 19 functional runs against live Valkey in isolated
containers, cross-checked server-side; full-tree source inspection with
`git log -S` history search; every decision ratified by hand against the
raw audit trails.

manifest.json updated to match; 48 -> 27 entries (12 clients, 15
integrations), validated against files on disk in both directions.
@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 97052dd4-2f05-48cf-baab-6ac8c04be629

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

`Only open-source licenses are accepted` was immediately followed by a
clause admitting proprietary and source-available licences case by case,
so the first sentence claimed an absoluteness the second withdrew.

Reframed as a requirement with a bounded exception, which is the intent:
an open-source licence is required, and other licences are considered
only by exception where the tool provides significant community value.

No change to the substantive policy or to the community-value bar.

Found by content audit AEA-691 (recorded as P7).
Follow-up to the deletion commit on this branch. Two kinds of change:

FIELD CORRECTIONS to 11 existing entries, each backed by a functional
audit trail:

- Sidekiq: description said "for Rails applications" (wrong - upstream
  says any Ruby app, and this was disproved functionally by running a
  worker without Rails); repository moved mperham/sidekiq -> sidekiq/sidekiq
- Rq: licence was the wrong SPDX id; repository and description corrected
- valkey-py features: 110000001 -> 111000101 (two false negatives proven
  against a live server - pubsub_state_restoration and client_side_caching)
- iovalkey features: corrected after direct measurement
- huey, Valkey Admin, Cognee, Haystack, PraisonAI: description fixes
- Valkey Admin: "desktop app" -> "web interface" (upstream is web-based)
- CocoIndex, PraisonAI, Spring Data Valkey: installCommand corrected;
  Spring Data Valkey had none, and its coordinate is a three-segment
  vendor namespace (io.valkey.springframework.data)

7 NEW ENTRIES added to integrations, all functionally verified:

- LiteLLM, LangBot, Agno, Langflow, LocalAI - requested by @jbrinkman.
  Four are Tier 1 (exercised against a live Valkey); LocalAI is Tier 3
  (first-class valkey-go dependency proven from go.mod, runtime not
  executed - its Go build failed on an unrelated dependency).
- Valkey MCP Server, Valkey Message Queue Starter - operator-attested.

Each install command was tested from a clean environment; several were
wrong in ways that left users with no Valkey support at all.

NOT included, deliberately: the 6 server-module entries (valkey-search,
valkey-json, valkey-bloom, valkey-ldap, valkey-bundle, valkeymodule-rs).
They are absent from this branch entirely and we could not tell whether
that was deliberate scoping in the clients+integrations restructure or a
side effect. Flagging rather than re-adding them. 27 + 7 = 34 here; our
audit set is 40, and those 6 are the difference.

vLLM remains cut. The original basis was wrong (it was audited against
vllm-project/vllm), but the correct finding supports the same outcome:
the Valkey work lives in vllm-project/semantic-router, a separate Go
product with its own release train, so an entry named "vLLM" would point
readers at a repository that genuinely has no Valkey support.
Field edits, each decided on evidence recorded in the audit trail:

- huey installCommand -> "pip install huey valkey-glide-sync". huey
  reaches Valkey two ways: valkey-glide-sync (first class, but
  undocumented upstream and with real gaps - blocking ops rejected,
  priorities NotImplementedError) and huey[redis] (documented, and
  upstream's install doc even names Valkey). Chose the GLIDE route
  because this entry's description says it is "built on the official
  Valkey GLIDE client", so a redis-py install line would contradict
  the copy.

- LiteLLM and LocalAI retagged to "Inference & serving". LiteLLM is an
  LLM API gateway that was filed under Agent frameworks; LocalAI is a
  self-hosted inference server that was filed under RAG & retrieval.

- valkey-glide-c.json renamed to valkey-glide-csharp.json. The '#' was
  lost in slugification, so the slug read as the language C. Pure
  rename, content hash unchanged; the manifest was the only reference.

Also fixes a mistake of mine from the previous commit (992f368): the
7 entries added there were missing "isFirstParty", which all 27
pre-existing entries carry. All 7 are third-party, so all are set to
false, and the field is now present on 34/34 entries. Separately, a
key-reordering step in this change had dropped isFirstParty from
huey.json; that is restored, and entries are now rewritten preserving
their original key order.

Not included: valkey-search and valkey-ldap "documentation" URLs were
also set (https://valkey.io/topics/search/ and /topics/ldap/, both
verified 200), but those two entries are among the 6 server-module
entries absent from this branch, so the change has nowhere to land here.
valkey-json and valkey-bloom stay unset - their topics pages 404.
Ruling: no new fields on entry JSON, and an entry that can't be
described within the existing schema doesn't ship.

LocalAI advertises a Valkey vector backend that is not part of the
default image. backend/index.yaml publishes it as a separate OCI
artifact resolved through LocalAI's backend gallery, so using it is a
two-step operation. Every candidate value for installCommand was
checked and none of them delivers the advertised capability:

- "docker run localai/localai" installs the product but not the
  valkey-store backend, so it points users at something that won't
  work as described
- the backend image itself is not a command a user runs; it is pulled
  by the gallery

installCommand can hold one string, so there is nowhere truthful to put
"then install the valkey-store backend". Rather than add a field or ship
a misleading one-liner, the entry comes out.

Every other entry with a null installCommand was re-checked against the
same test - does the null hide a requirement, or is there simply nothing
to install? All the others are honest nulls: valkey-cli ships with the
server, valkey-container is itself an image, valkey-helm and
valkey-operator are deployed rather than installed, Valkey Admin is a
console you run, valkey-skills has no runtime artifact. LocalAI was the
only one concealing a requirement.

Valkey Message Queue Starter was reviewed under the same rule and stays:
its "pnpm create next-app --example ..." line is long but is a single
valid command, which is a different thing from unexpressible.

Nothing is wrong with LocalAI itself. Its Valkey support is first class
and proven from source (go.mod requires valkey-io/valkey-go), it is MIT
and actively maintained. It qualifies again if the backend ever ships in
the default image, or if a prerequisites field is introduced later.

Integrations 22 -> 21.
The field was described in one sentence, which left three things
unstated that reviewers were having to infer:

- it only makes sense next to "documentation": null; a note beside a
  populated URL explains nothing
- a useful note says what was looked for, what exists instead, and when
  it was checked, in the form "(checked YYYY-MM-DD)". Without a date the
  note stops being trustworthy once docs sites move
- an unverified candidate URL must be named as unverified and must stay
  out of the documentation field

That last point is not hypothetical. One entry's note recorded an
unconfirmed <project>.io reference; the domain turned out to be a parked
for-sale page with no documentation on it. Saying so in docsNote is what
kept it out of documentation, which is exactly what the field is for.

Also states plainly that docsNote does not satisfy the documentation
expectation in the review checklist - it records a gap honestly so a
reviewer can tell a researched null from a skipped one - and that the
field is never rendered on the card, so it can be candid. Verified: no
template references it.

Examples are taken from entries already in the catalog.
The counterpart to removing vllm. vllm-project/vllm has no Valkey
support, but the sibling project vllm-project/semantic-router has three
first-class Valkey backends, so it gets its own entry rather than the
removal quietly losing a real integration.

Named "vLLM Semantic Router", not "vLLM". Upstream self-brands it that
way, and the shorter name is what caused the original audit error: a
reader would go to vllm-project/vllm and find nothing. They are separate
products - not a fork, Go rather than Python, own domain, own release
train.

Verified at Tier 2. Tier 1 was not reachable honestly: both
Valkey-bearing packages fail to link without the project's Rust
candle-binding, and its own Valkey integration test needs real model
embeddings. What was proven instead: the FT.CREATE argument vector taken
verbatim from pkg/memory/valkey_store.go was accepted by a live server,
confirmed with FT._LIST and FT.INFO on Valkey 9.1.2.

First class, decisively: src/semantic-router/go.mod requires
valkey-io/valkey-glide/go/v2, and every Valkey file imports glide. There
is a go-redis dependency too, but it serves a separate Redis backend.
The Valkey work is substantial - 124 files, 3,181 lines of non-test
implementation, five merged PRs.

Needs the search module; failure without it is loud, not silent.
installCommand is null because the router is deployed rather than
installed, the same as the operator and Helm entries. A PyPI package
exists but at 135 KB cannot be the Go router, and its purpose is
unverified, so it is not asserted as the install.

Apache-2.0, confirmed from the LICENSE file. Integrations 21 -> 22.
@Jonathan-Improving Jonathan-Improving changed the title Content audit: remove 21 catalog entries that don't meet the inclusion bar Content audit: remove 20 entries, correct 12, add 8 Sep 15, 2026
@Jonathan-Improving
Jonathan-Improving marked this pull request as ready for review September 15, 2026 02:20
@Jonathan-Improving

Copy link
Copy Markdown
Collaborator Author

These changes are meant to be incorporated into valkey-io#602

@Jonathan-Improving
Jonathan-Improving merged commit 1bed776 into jbrinkman:fix/ai-libraries-595 Sep 15, 2026
1 check passed
@Jonathan-Improving
Jonathan-Improving deleted the audit/aea-691-catalog-decisions branch September 15, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant