Skip to content

Speed up blink check-sources via cached repo trees - #203

Merged
TomasVotruba merged 1 commit into
mainfrom
faster-check-sources
Oct 10, 2026
Merged

TomasVotruba merged 1 commit into
mainfrom
faster-check-sources

Conversation

@TomasVotruba

Copy link
Copy Markdown
Member

Why

The Blink Tests / fixer_sources job ran check-sources as 164 per-fixer GitHub HEAD requests, each with a 150ms throttle sleep plus retry backoff. That took minutes and was prone to GitHub rate-limiting (503/429).

What

  • Fetch the recursive git tree of each of the 3 source repos (PHP-CS-Fixer, symplify/coding-standard, ecsphp/ecs-src) in a single GitHub API call, then verify every fixer path locally against that set: 3 requests instead of 164.
  • Commit the last known-good trees as sources_cache.json (embedded), used as a fallback when GitHub is unreachable, so the job no longer fails on a transient outage.
  • Add -refresh to regenerate the committed cache from live data.

Still catches a renamed or removed upstream rule, since the live tree is the default source of truth.

Local run drops from minutes to ~0.5s. Workflow YAML unchanged.

Replace 164 per-fixer GitHub HEAD requests with 3 recursive git-tree
fetches checked locally, plus a committed sources_cache.json fallback
for when GitHub is unreachable. Add -refresh to regenerate the cache.
@TomasVotruba
TomasVotruba merged commit e5950a9 into main Oct 10, 2026
20 checks passed
@TomasVotruba
TomasVotruba deleted the faster-check-sources branch October 10, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant