Skip to content

chore(ci): measure remote cache phase timings - #1275

Open
jdx wants to merge 6 commits into
mainfrom
agent/measure-mise-remote-cache
Open

chore(ci): measure remote cache phase timings#1275
jdx wants to merge 6 commits into
mainfrom
agent/measure-mise-remote-cache

Conversation

@jdx

@jdx jdx commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • run the production cache qualification with mise 2026.8.8
  • consume mise's versioned JSON report instead of parsing human-readable cache output
  • require real cache hits and, on pull requests, remote prefetch and downloaded bytes
  • publish result and phase-timing tables for remote and post-clean local passes
  • retain raw cache reports as workflow artifacts for 30 days
  • compare against the existing GitHub Actions archive-cache control

Why

The previous qualification established correctness and basic restore behavior, but it could not explain where the remaining time went. The structured report separates session, prefetch, manifest lookup, action lookup, blob transfer, local CAS write, and materialization time while recording request counts and restored volume.

That data will show whether the next optimization belongs in the client, server, transfer protocol, or local materialization path. In particular, the per-blob request count provides the baseline for evaluating packed blob transfers.

Validation

  • rebased onto current main
  • hk run check --no-stage .github/workflows/ci.yml
  • actionlint .github/workflows/ci.yml
  • git diff --check

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.


Note

Low Risk
Changes are limited to CI workflow configuration and observability; they do not affect application code or production runtime behavior.

Overview
CI now measures mise’s production Rust action cache using versioned JSON stats (MISE_TASK_CACHE_STATS_REPORT) instead of scraping log lines, bumps the qualification job to mise 2026.8.9, and sets rust_cache = true on the qualify task (dropping { verify = true }).

The cache-qualification step validates report shape with jq, requires cache hits, and on pull requests still requires remote prefetch and non-zero downloaded bytes; non-PR runs keep a post-cargo clean local pass for a second report. It writes hit/miss and phase-timing tables to the job summary and uploads raw JSON as 30-day artifacts. A new github-cache job on ubuntu-latest runs a standard actions/cache Rust build and records exact hit vs build time for comparison, and final now depends on that job.

Reviewed by Cursor Bugbot for commit b9f15c7. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: b03454f7-290c-45a3-b2fd-ec49420cc5c8

📥 Commits

Reviewing files that changed from the base of the PR and between 8cb0fcc and f564323.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/ci.yml

📝 Walkthrough

Walkthrough

The PR changes the Rust action-cache setting and updates CI qualification. CI checks cache activity, repeats qualification after cleaning on non-pull-request runs, and adds a hosted github-cache job to the final gate.

Changes

Rust action-cache qualification

Layer / File(s) Summary
Update Rust cache setting
.github/mise-cache-qualification.toml, .github/workflows/ci.yml
rust_cache now uses the boolean setting true. The workflow comment describes the experimental Rust action-cache exercise.
Revise CI cache validation and hosted coverage
.github/workflows/ci.yml
The workflow upgrades mise to 2026.8.6, parses cache hit, miss, prefetch, and download results, and fails on cache warnings. It requires a cache hit, checks pull-request remote activity, repeats non-PR qualification after cleaning the target directory, and adds the hosted github-cache job to the final gate.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to f5643

This PR changes CI-only cache measurement and gating behavior without changing application or production runtime behavior; no actionable merge-blocking risk remains after normal checks and review.

Possibly related PRs

  • jdx/aube#1167: Both PRs modify CI workflow caching behavior.
  • jdx/aube#1270: This PR updates the Rust cache qualification flow introduced there.
  • jdx/aube#1274: Both PRs validate remote Rust cache activity.

Poem

A rabbit checks each cache hit,
Then counts downloads bit by bit.
The target cleans and builds anew,
Hosted cache joins the queue,
The final gate turns green too.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary CI change: measuring remote cache phase timings during cache qualification.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR updates CI cache qualification to consume mise’s structured statistics reports and adds an archive-cache control for comparative timing.

  • Records cache-hit, transfer, and phase-timing measurements in the workflow summary.
  • Uploads raw cache reports as retained workflow artifacts.
  • Adds a GitHub Actions Rust cache control job and includes it in final CI aggregation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
.github/mise-cache-qualification.toml Enables the Rust action cache without the former qualification mode so the workflow can collect structured cache statistics.
.github/workflows/ci.yml Upgrades mise, validates and publishes structured cache reports, retains report artifacts, and adds a GitHub Actions cache timing control.

Reviews (8): Last reviewed commit: "ci: isolate GitHub cache control keys" | Re-trigger Greptile

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026

jdx commented Aug 11, 2026

Copy link
Copy Markdown
Owner Author

Live normal-hit result on commit 632a0453:

  • 382 cache hits and 0 misses
  • 218 actions prefetched
  • 910.0 MiB downloaded and 0 B uploaded
  • Cargo build: 7m37s
  • qualification job: 8m17s

Comparator from the same workflow run using Namespace caching:

  • mise run build: 27s
  • Linux build job: 1m04s

The normal mise cache path is correct, but this is not acceptable rollout performance. This PR should remain diagnostic and should not replace Namespace caching until transfer/materialization is optimized. The local cache shape is 3,063 files totaling about 963 MiB; 2,040 files are smaller than 4 KiB, which makes per-blob request overhead a leading suspect.

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.

@github-actions

github-actions Bot commented Aug 11, 2026

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▁▇▇▇▇▇▇▇▇███▆▆▆▆▆ 18,359,997 → 18,360,760 +0.00% 4.72 → 4.91ms +4.06%
install ▃▁▁▂▂▂▃▃▃▅▆█▃▃▆▅▇ 114,936,518 → 114,943,367 +0.01% 27.12 → 27.09ms -0.12%
startup ▄▁▁▁▃▃▄▂▃███▆▆█▆▇ 7,488,172 → 7,489,371 +0.02% 3.41 → 3.26ms -4.35%
tree ▅▄▃▃▂▂▃▃▁██▆▆▅▆▆▆ 17,759,511 → 17,760,569 +0.01% 4.66 → 4.60ms -1.28%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

b9f15c708f4e vs f88a72940c1e · measured on this runner, not pushed to the history.

@greptile-apps
greptile-apps Bot dismissed their stale review August 11, 2026 19:33

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026
@greptile-apps
greptile-apps Bot dismissed their stale review August 11, 2026 19:39

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

jdx commented Aug 11, 2026

Copy link
Copy Markdown
Owner Author

Additional cache-shape comparisons:

  • GitHub-hosted actions/cache exact hit: 819 MiB archive restored in about 13.5s, Cargo finished in 40.78s, and the whole job took 1m05s.
  • Namespace attached warm target volume: Cargo finished in 26.64s in the prior comparison run, with no cache upload/download phase.
  • mise local CAS with an empty target: 382 hits in 25.40s with zero network traffic. This used the same 953 MiB aube action corpus on a local machine, so hardware differs and the number is directional rather than an apples-to-apples CI result.

This changes the rollout bar: the optimized remote cache should be compared primarily with the roughly one-minute GitHub-hosted archive-cache setup, while Namespace remains the attached-volume upper bound. It also validates a tiered deployment model: mount MISE_TASK_CACHE_DIR as a local L1 cache where a persistent volume exists, and retain the remote service as L2/fallback. PR #1275 remains diagnostic until mise PR #11905 is merged, released, and rerun against the production service.

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 11, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 188-196: Update the workflow steps using jdx/mise-action and
actions/cache: set the mise action’s cache input to false, add id cargo-cache to
the explicit actions/cache step, and include github.event.pull_request.number ||
github.ref_name in that step’s key to isolate caches by pull request or branch.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 96e77dc5-0504-4ff7-bc9d-85ac8ab46ac2

📥 Commits

Reviewing files that changed from the base of the PR and between 632a045 and b470b02.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Comment thread .github/workflows/ci.yml Outdated
@greptile-apps
greptile-apps Bot dismissed their stale review August 12, 2026 12:19

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 12, 2026
Comment thread .github/workflows/ci.yml Outdated

jdx commented Aug 12, 2026

Copy link
Copy Markdown
Owner Author

mise 2026.8.5 production qualification

Workflow run

Metric Result
Action-cache hits 382
Action-cache misses 0
Prefetched actions 368
Downloaded 910.0 MiB
Cargo build 2m40s
Qualification job 2m51s

Compared with mise 2026.8.4, Cargo improved from 7m37s to 2m40s (about 65%), while the complete job improved from 8m17s to 2m51s (about 66%).

The GitHub Actions-cache comparator in this run had a cold miss because the updated base changed the Cargo.lock key, so its 2m23s build is a cold baseline. The prior exact-hit baseline remains 40.78s for Cargo and about 1m05s for the job.

The remote cache is substantially improved, but it is not yet competitive with the warm archive cache. This PR should remain open.

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.

@jdx
jdx force-pushed the agent/measure-mise-remote-cache branch from 8cb0fcc to f564323 Compare August 15, 2026 18:35
@greptile-apps
greptile-apps Bot dismissed their stale review August 15, 2026 18:35

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 15, 2026

jdx commented Aug 15, 2026

Copy link
Copy Markdown
Owner Author

mise 2026.8.6 production qualification

Workflow run

Metric Result
Action-cache hits 382
Action-cache misses 0
Prefetched actions 371
Downloaded 910.6 MiB
Cargo build 1m48s
Qualification job 1m57s

Compared with mise 2026.8.5, Cargo improved from 2m40s to 1m48s (about 33%), while the complete job improved from 2m51s to 1m57s (about 32%).

The same-commit GitHub Actions-cache control was rerun after seeding its new Cargo.lock key. Its 821 MiB archive restored in about 16 seconds, Cargo finished in 45.95s, and the whole job took 1m19s.

The reflink materialization release is a substantial improvement, but the remote action cache remains about 62 seconds slower at Cargo and 38 seconds slower end to end than the warm archive control. This PR should remain diagnostic rather than replacing the current Namespace cache. The next material improvement should target packed or bundled remote artifact transfer instead of additional per-file materialization tuning.

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.

@jdx
jdx force-pushed the agent/measure-mise-remote-cache branch from f564323 to b8a382b Compare August 17, 2026 18:11
@greptile-apps
greptile-apps Bot dismissed their stale review August 17, 2026 18:11

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@jdx jdx changed the title chore(ci): measure remote cache hits chore(ci): measure remote cache phase timings Aug 17, 2026
greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 17, 2026

jdx commented Aug 17, 2026

Copy link
Copy Markdown
Owner Author

mise 2026.8.8 phase qualification

Workflow run · raw report

Metric Result
Hits / misses 382 / 0
Prefetched actions 419
Downloaded 1.08 GiB
Restored outputs 1,144 files / 908.69 MiB
Remote action lookups 440
Remote blob requests 3,076
Cache session 527.48s
Prefetch 506.20s
Cumulative action lookup 630.49s
Cumulative blob transfer 23,778.74s
Cumulative local CAS writes 4.27s
Cumulative materialization 2.12s

The report isolates the bottleneck: prefetch occupies almost the entire cache session, and 3,076 independent blob requests accumulate 6.6 hours of concurrent transfer time. In contrast, the report attributes only 4.27s to local CAS writes and 2.12s to materialization. The next cache performance PR should implement packed/bundled blob retrieval; more local materialization tuning is unlikely to move the result materially.

The Namespace production build completed in about 44s on this run; the GitHub Actions cache control completed its build step in about 2m28s. The remote action-cache Cargo build took 8m46s on the current base, so it remains diagnostic rather than ready to replace the production cache.

AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.

@greptile-apps
greptile-apps Bot dismissed their stale review August 19, 2026 21:22

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@jdx
jdx force-pushed the agent/measure-mise-remote-cache branch from 0634c55 to 44e9653 Compare August 19, 2026 21:23
greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 19, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 44e9653. Configure here.

Comment thread .github/workflows/ci.yml Outdated
@greptile-apps
greptile-apps Bot dismissed their stale review August 19, 2026 21:36

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant