✨ feat(mq-lang): add mq.lock integrity check for HTTP module imports#1977
Merged
Conversation
Records each fetched module URL's SHA-256 hash on first fetch, and rejects a later fetch of the same URL whose content no longer matches unless --refresh-modules is passed (which also updates the recorded hash) — making mutable-ref HTTP imports (HEAD/branch) reproducible across machines and CI, the same way package-lock.json/ deno.lock work. - New --lockfile <PATH> / --no-lockfile CLI flags, and matching Engine/ModuleLoader/DefaultModuleResolver API. - --clear-cache/--refresh-modules also prune the corresponding mq.lock entries (all / mutable-ref only), independent of whether the check is currently enabled. - Concurrent access to mq.lock (e.g. mq-run's parallel file processing) is serialized via an OS-level advisory lock, and writes go through a temp-file-then-rename for atomicity. - Ported to the WASM/OPFS build (WasmFetcher), sharing the same ModuleLock data model and compute_hash implementation as the native fetcher so both backends produce identical hashes.
check_lock cloned the entire ModuleLock map on each call just to read it; operate on the mutex-guarded lock directly instead.
harehare
force-pushed
the
feat/import-lockfile
branch
2 times, most recently
from
1e9e157 to
9099d41
Compare
Previously the mq.lock check only ran on network fetches, so a disk cache hit bypassed it entirely. Since the module cache is shared per machine while mq.lock is per project, a project whose lock file expected different content than the local cache held would silently run the cached version, and a project without an mq.lock never got one created while modules were served from the cache. - UreqFetcher: check the (sidecar-verified) cached hash against mq.lock on both cache-hit paths, recording new entries as on fetch. - WasmFetcher: same check on OPFS cache hits, extracted into a shared check_opfs_lock helper. - Mismatch errors now suggest the recovery that actually works per URL kind: --refresh-modules for mutable refs, --clear-cache for versioned (tagged) URLs whose cache --refresh-modules doesn't touch (clearHttpCache()/clearAllHttpCache() on WASM).
…he result
Keys the advisory lock file by the SHA-256 hash of mq.lock's absolute
path under mq-{hash}.lock in the system temp dir instead of a .flock
sidecar next to mq.lock, so distinct lock file paths never collide and
no stray file is left in the user's project. Also replaces the
(String, String) tuple returned by try_read_cache with a named
CachedModule struct.
Call lockfile::compute_hash directly instead of delegating through UreqFetcher::compute_hash, and simplify super:: path references now that http_import and lockfile are imported directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Records each fetched module URL's SHA-256 hash on first fetch, and rejects a later fetch of the same URL whose content no longer matches unless --refresh-modules is passed (which also updates the recorded hash) — making mutable-ref HTTP imports (HEAD/branch) reproducible across machines and CI, the same way package-lock.json/ deno.lock work.
Type of Change
Checklist
cargo fmtandcargo clippyand addressed any warningsjust test-alland all tests pass/docs, crateREADME.md) if neededAdditional Context