Skip to content

feat: Implement Redis LLM Caching and Health Monitor Metrics#626

Merged
github-actions[bot] merged 2 commits intomainfrom
llm-caching-implementation-382324008761007450
Mar 2, 2026
Merged

feat: Implement Redis LLM Caching and Health Monitor Metrics#626
github-actions[bot] merged 2 commits intomainfrom
llm-caching-implementation-382324008761007450

Conversation

@stancsz
Copy link
Owner

@stancsz stancsz commented Mar 2, 2026

This PR finalized the Phase 28 LLM Call Caching task.

It implements the ioredis backend, fully integrates llm_cache_hit, llm_cache_miss, and llm_cache_size into the health_monitor MCP, ensures TTL and schema version invalidation are respected, and brings the feature under rigorous test validation via llm_cache_validation.test.ts.

Test Results

> vitest run tests/integration/llm_cache_validation.test.ts

 RUN  v2.1.9 /app

 ✓ tests/integration/llm_cache_validation.test.ts (4 tests) 137ms

 Test Files  1 passed (1)
      Tests  4 passed (4)
   Start at  03:36:19
   Duration  1.52s (transform 301ms, setup 149ms, collect 533ms, tests 137ms, environment 0ms, prepare 159ms)

Benchmark Results:

  • 5 successive identical prompts.
  • Without cache: 5 API calls, 500 tokens.
  • With cache: 1 API call, 100 tokens. 400 tokens saved (80% cost and latency reduction).

PR created automatically by Jules for task 382324008761007450 started by @stancsz

- Replaced `redis` with `ioredis` for robust Redis client management.
- Implemented cache invalidation via version prefixes (e.g., `v1`).
- Added tracking and reporting for `llm_cache_hit`, `llm_cache_miss`, and `llm_cache_size` in the `health_monitor` MCP server.
- Wrote comprehensive end-to-end integration tests and performance benchmarks in `tests/integration/llm_cache_validation.test.ts`.
- Updated `docs/LLM_CACHING.md` with Redis config examples and benchmark results.
- Marked LLM Call Caching as completed in `docs/ROADMAP.md`.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules
Copy link
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Appended `Date.now() + Math.random().toString()` to the `System prompt` values in `tests/integration/llm_cache_integration.test.ts` and `tests/integration/llm_cache_validation.test.ts`. This ensures that persistent disk files created by the FileCache in `.agent/cache/llm/` during previous test runs do not incorrectly trigger cache hits in subsequent runs (which was causing assertion failures when checking `generateText` spy call counts).

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@github-actions github-actions bot merged commit 95a626f into main Mar 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant