Composable Performance Feedback Middleware for AI Agents

A plug-and-play, event-sourced middleware for capturing, interpreting, and operationalizing feedback to improve LLM and agent performance.

Status: 0.x. APIs may change between minor releases. The classifier, event schema, and ports are stable; reference projections and capture adapters are still hardening.

A provider-agnostic framework for capturing, classifying, storing, and dispatching feedback signals from humans and systems that review AI-generated artifacts. Built on event sourcing with ports-and-adapters isolation, an optional middleware pipeline for cross-cutting concerns, and an optional RxJS stream wrapper for composable subscribers.

Why this exists

Most production AI systems throw away most of the feedback they receive. The systems that capture feedback at all capture it as a dead-end log. This framework treats feedback as a first-class event stream:

• A 2x2 model (explicit/implicit x positive/negative) plus a whitelist/blacklist/observe inference dimension that the deterministic classifier produces on every event.
• An immutable event log as the source of truth, queryable by partition or by filter.
• Pluggable transport (in-memory, Redis pub/sub, Kafka, NATS, cloud queues; bring your own).
• Pluggable storage (Postgres today, EventStoreDB or custom tomorrow).
• Extension points for consumer-specific actions, artifact types, and middleware.
• No LLM in the feedback path. Classification, inference, and edit-diff labeling are pure deterministic functions.

Install

pnpm add @ai-feedback-middleware/core @ai-feedback-middleware/in-memory
# for production storage:
pnpm add @ai-feedback-middleware/postgres pg
# for cross-process dispatch:
pnpm add @ai-feedback-middleware/redis-pubsub ioredis
# optional, for stream operators:
pnpm add @ai-feedback-middleware/streams rxjs
# optional, for reference projections + capture adapters:
pnpm add @ai-feedback-middleware/reference

ESM only. Targets Node 18+.

Quickstart

import { createFeedback, DEFAULT_ACTIONS } from "@ai-feedback-middleware/core";
import {
  createInMemoryEventStore,
  createInMemoryProjectionStore,
} from "@ai-feedback-middleware/in-memory";

const feedback = createFeedback({
  eventStore: createInMemoryEventStore(),
  projectionStore: createInMemoryProjectionStore(),
  actions: DEFAULT_ACTIONS,
  artifactTypes: [{ name: "draft" }],
});

await feedback.capture({
  action: "approve",
  artifact_type: "draft",
  artifact_id: "draft-123",
  artifact_version: 1,
  producer: "secretary-agent",
  task_type: "email_draft:warm",
  payload: { artifact_hash: "sha256:..." },
});

for await (const event of feedback.readStream("draft-123")) {
  console.log(event.event_id, event.action, event.inference);
}

For production wiring (Postgres + Redis + outbox + middleware) see examples/postgres-redis/ and the Postgres setup guide.

Packages

Package	Status	Purpose
@ai-feedback-middleware/core	F0	Types, classifier, registry, ports, middleware, upcasters, edit-diff labeler
@ai-feedback-middleware/in-memory	F0	In-memory adapters for tests and small deployments
@ai-feedback-middleware/postgres	F1	Postgres event store + projection store + outbox + dedupe + inference rules + migrations
@ai-feedback-middleware/redis-pubsub	F2	Redis pub/sub event bus
@ai-feedback-middleware/streams	F3	Optional RxJS-based stream wrapper (peer dep)
@ai-feedback-middleware/reference	F3	Reference projections (approval-rate, whitelist-examples, blacklist-phrases) and capture adapters (HTTP button, signed webhook)
@ai-feedback-middleware/adapter-conformance	F1+	Conformance test suites for adapter implementers

Adapters are independent: install only the ones you use.

Documentation

• Getting started
• Concepts: the 2x2 framework
• Concepts: event sourcing basics
• Concepts: ports and adapters
• Concepts: middleware pipeline
• Adapters: Postgres setup
• Adapters: choosing a bus
• Adapters: writing a custom adapter
• Cookbook: threshold crystallization
• Framework specification - full design rationale
• Article: Building a Learning Loop - Medium-style introduction

Schemas

JSON Schema (Draft 7) and protobuf (proto3) artifacts for the canonical FeedbackEvent live in schemas/ for non-Node consumers (Go, Rust, Python, Java).

Examples

Runnable examples with end-to-end output:

• minimal-nodejs - in-memory only, no infra.
• postgres-only - durable storage, sync projection, rebuild.
• postgres-redis - full stack with outbox + middleware + inference rule.
• threshold-crystallization - streams + groupBy + bufferCount.

Conformance

If you write a new adapter, drop the suites from @ai-feedback-middleware/adapter-conformance into your tests. If they pass, your adapter is interchangeable with the in-memory and Postgres reference implementations.

Contributing

See CONTRIBUTING.md for the workflow, CODE_OF_CONDUCT.md for community norms, and CLA.md for the contributor license. Security issues go through SECURITY.md, not the public issue tracker.

License

Apache License 2.0. See LICENSE and NOTICE.

Copyright 2026 Babak Abbaschian.