docs(env-vars): raise the SCAN_WORKERS default to 4 - #131
Merged
Conversation
The reference lists SCAN_WORKERS as 1, which measured 2.2x slower than 4 concurrent workers when hashing a real library. rommapp/romm#4386 changes the env.template line this table is generated from. Regenerated from that branch rather than hand-written: the Scans & Tasks section here is byte-identical to the generator's output, so the next release bump reproduces it. sources.toml stays pinned at 5.2.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gantoine
approved these changes
Sep 8, 2026
gantoine
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Updates the
SCAN_WORKERSrow in the environment-variables reference, which currently shows a default of1. Hashing a real 1.96 GiB library measured 2.2x faster at 4 concurrent workers than at 1, and a single worker is a large part of why first scans feel slow.Live page: https://docs.romm.app/latest/reference/environment-variables/
14Why
SCAN_WORKERSis anasyncio.Semaphorearound per-ROM identification. Inside it, hashing is offloaded withasyncio.to_threadandhashlib/zlibrelease the GIL, so more workers genuinely use more cores. 67 real ROM files, 1.96 GiB, warm page cache, core counts below 16 pinned withtaskset:4 is the peak at both 16 and 4 cores and still 1.6x at 2 cores, so it does not assume a large host.
The added half-sentence is the part users most need: raising this mainly speeds up hashing. Metadata providers cap themselves independently (IGDB 4/s, RetroAchievements 4/s, MobyGames 1/s, Steam 0.6/s, and ScreenScraper via a concurrency limiter set to the account's advertised thread count), so extra workers queue there rather than scraping faster.
It deliberately does not name a ceiling. The practical limit today is
FILE_READ_CHUNK_SIZE = 8 KiB, which rommapp/romm#4387 raises to 256 KiB; past 4 workers the 8 KiB chunk turns added concurrency into GIL contention, and at 256 KiB throughput instead scales to 666 MiB/s at 8 workers and stays flat out to 20. Wording the row around what the setting buys rather than where it stops keeps it correct before and after that change.About editing a generated file
env-vars.mdcarries theAUTOGENERATED by scripts/gen_env_vars.py: do not editheader, so this needs a word of explanation.The real fix is rommapp/romm#4386, which changes the
env.templateline this table is generated from. That is the source of truth and should be merged first. Butscripts/sources.tomlpinsref = "5.2.0", so the published table keeps showing1until the next release triggersromm-release-bump.yml. This PR closes that window.To make it safe, the edit is not free-hand. I ran
scripts.gen_env_varsagainst theenv.templatefrom rommapp/romm#4386, formatted the result, and the Scans & Tasks section is byte-identical to what is committed here, so the next bump reproduces this exact content rather than fighting it:I confirmed the two versions of that section differ only in the
SCAN_WORKERSrow, so nothing else in it drifts in from unreleased master. The rest of the diff is only Prettier re-padding the table to the new column width. No other section is touched, andsources.tomlis deliberately left at5.2.0(re-pinning to master would pull in env vars that are not in a release yet).If you would rather not carry a manual edit at all, feel free to close this and let the bump handle it after rommapp/romm#4386 merges. The fix is complete without this PR, just delayed by one release.
Verification
uv run mkdocs build --strictpassesuv run python -m scripts.check_redirectspasses (all 46 targets exist)trunk fmt/trunk checkcleansite/reference/environment-variables/index.htmlrenders4and the new descriptionRelated: rommapp/romm#4386 (the source-of-truth change), rommapp/romm#4387 (the read-chunk fix that moves the practical ceiling)
AI disclosure
The benchmarking, analysis and wording here were produced with Claude Code. I reviewed and verified every claim against the source before submitting.
🤖 Generated with Claude Code