Feat/build catalog - #5
Conversation
Replaces the placeholder script with a robust Python builder that: - Executes concurrent HTTP requests against registry.npmjs.org - Flattens all valid versions preserving integrity, tarball URLs, and peerDependencies - Strictly adheres to the spec generating choysum module boundaries
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces static configuration files for routing and headers, and replaces a placeholder build script with a robust, concurrent catalog builder (scripts/build_catalog.py) that fetches package metadata from NPM. The review feedback focuses on enhancing the reliability of the build script by adding network timeouts, URL-encoding scoped package names, and implementing defensive type checks for parsed JSON data and NPM registry responses.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Overrides the legacy dashboard settings to automatically point to the new python scripts/build_catalog.py tool.
Deploying modules-directory with
|
| Latest commit: |
93c559e
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a0742f5b.modules-directory.pages.dev |
| Branch Preview URL: | https://feat-build-catalog.modules-directory.pages.dev |
Cloudflare Pages does not support the [build] configuration block in wrangler.toml. The build command must be updated directly in the Cloudflare Dashboard.
Restores actions/checkout@v6, setup-python@v6, and upload-artifact@v7 in the build catalog workflow to match project baseline.
Incorporates gemini-code-assist review feedback: - Adds a 10s URL timeout to prevent fetch hangs - URL-encodes scoped package names properly - Defensively asserts type dictionary instances for registry payloads
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a basic bootstrapping script with a robust Python script (scripts/build_catalog.py) that concurrently fetches metadata from the NPM registry to construct a static catalog. It also introduces static _headers and _redirects files for deployment. The review feedback suggests enhancing error handling in the NPM metadata fetcher to catch JSON parsing errors, and adding stricter type validation for the depends and peerDependencies fields to ensure downstream clients do not receive malformed data.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Resolves github-code-quality flagging unused imports by cleanly grouping urllib library components at the top level instead of doing an inline import.
Incorporates the second round of gemini-code-assist review feedback: - Catches json.JSONDecodeError from NPM registry outages to prevent unhandled crashes - Explicitly enforces list/dict typing for depends/peerDependencies attributes
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a new script (scripts/build_catalog.py) that concurrently fetches metadata from NPM to build static catalog artifacts, and adds static _headers and _redirects files. The review feedback suggests several key improvements: implementing retry logic and better error handling for NPM metadata requests, skipping package versions that lack choysum metadata, aggregating errors during concurrent module processing instead of failing on the first exception, and cleaning the dist directory before building to prevent stale file accumulation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Incorporates the third round of gemini-code-assist review feedback: - Erases the dist/ target folder before building to prevent leftover bloated snapshots - Collects ThreadPool concurrent exceptions instead of short-circuiting so bad module inputs are reported collectively
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the bootstrapping build script with a more robust scripts/build_catalog.py that concurrently fetches metadata from the NPM registry to build the static catalog directory, while also adding static _headers and _redirects configuration files. Feedback on these changes highlights three key areas for improvement: first, detecting duplicate module IDs across different trust tiers to prevent silent catalog corruption; second, implementing a retry mechanism with exponential backoff for NPM registry requests to handle transient network failures; and third, reordering the build steps to collect modules before deleting the dist directory to ensure the build fails fast without leaving the local disk in a broken state.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Add configurable NPM fetch retry/backoff/timeout via environment variables. - Detect duplicate module IDs across trust tiers and fail with explicit conflict details. - Collect module metadata before deleting dist to avoid destructive cleanup on fetch failures.
|
/gemini review |
- Add CHOYSUM_NPM_FETCH_TIMEOUT_SECONDS for request timeout. - Add CHOYSUM_NPM_FETCH_MAX_RETRIES for transient failure retries. - Add CHOYSUM_NPM_FETCH_BACKOFF_SECONDS for exponential backoff base delay.
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a robust scripts/build_catalog.py script that fetches package metadata from NPM in parallel, processes module configurations, and generates static catalog artifacts. It also adds a static _headers file to configure CORS and caching. The review feedback suggests two improvements: using dist_meta.get("integrity") or "" to safely handle explicit None values for integrity, and adding ensure_ascii=False to json.dumps to preserve UTF-8 characters in the generated catalog index.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Use "dist_meta.get('integrity') or ''" to avoid emitting null integrity values.
- Set ensure_ascii=False for canonical index JSON to preserve UTF-8 characters.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a robust scripts/build_catalog.py that fetches package metadata from the NPM registry in parallel using a thread pool, and manages static configuration files (_headers and _redirects) directly. The review feedback suggests failing immediately on non-retryable HTTP errors (400, 401, 403) during metadata fetching, defensively verifying that the retrieved versions data is a dictionary to avoid an AttributeError, and simplifying the UTC datetime formatting using strftime.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Fail fast for non-retryable client HTTP errors during NPM fetches. - Validate that npm_data.versions is a dict before iterating items. - Use strftime for UTC timestamp serialization with Z suffix.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a placeholder build script with a new scripts/build_catalog.py script that fetches package metadata from NPM to construct a static catalog directory. It also adds static _headers and _redirects configuration files. The review feedback suggests simplifying redundant HTTP status code checks and broadening exception handling during NPM metadata fetching to ensure transient network errors are correctly retried.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Remove redundant client-status check while preserving non-retryable 4xx fail-fast behavior. - Expand retry handling with explicit network/transport exception types instead of broad catch-all logic.
|
/gemini review |
Deploying choysum-modules-directory with
|
| Latest commit: |
5de5267
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://0d45ea55.modules-directory.pages.dev |
| Branch Preview URL: | https://feat-build-catalog.modules-directory.pages.dev |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a fully featured scripts/build_catalog.py that concurrently fetches package metadata from NPM, processes module entries, and generates static catalog artifacts. It also introduces static _headers and _redirects configuration files. The review feedback highlights a critical hash mismatch bug due to trailing newlines, suggests safer directory cleanup handling for DIST_ROOT, and recommends making the build concurrency configurable via an environment variable.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Hash the exact newline-terminated index payload written to disk to keep indexHash and file checksums consistent. - Make DIST_ROOT cleanup safe for directory, file, and symlink cases. - Add CHOYSUM_BUILD_CONCURRENCY to configure ThreadPoolExecutor worker count.
Expose CHOYSUM_BUILD_CONCURRENCY in build-catalog workflow so CI can control ThreadPoolExecutor workers explicitly.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a robust build_catalog.py script that concurrently fetches metadata from NPM to build static catalog artifacts, and adds static _headers and _redirects files. The review feedback suggests two important improvements: ensuring package integrity by falling back to the shasum field if integrity is missing, and raising an error if an NPM package contains no valid versions.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Enforce non-empty per-version integrity with shasum fallback conversion to sha1 SRI. - Fail module processing when no valid versions are found. - Require _headers and _redirects static files during dist generation. - Run scripts/build_catalog.py in validate workflow with explicit CHOYSUM_* env settings.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a placeholder build script with a fully functional catalog builder (scripts/build_catalog.py) that fetches metadata from the NPM registry, processes package versions and dependencies, and generates static catalog artifacts. It also adds static _headers and _redirects files. The feedback suggests wrapping the integrity resolution logic in a try-except block to prevent a single malformed package version from crashing the entire build process.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a new scripts/build_catalog.py script that builds a static catalog by fetching metadata from NPM concurrently. It also introduces static _headers and _redirects configuration files. Feedback on the changes suggests wrapping the resolve_integrity call in a try-except block to handle missing or malformed integrity hashes gracefully for individual package versions, preventing a single external data anomaly from crashing the entire catalog build process.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a placeholder build script with a more robust scripts/build_catalog.py script that concurrently fetches NPM metadata to build a static catalog directory, alongside adding static _headers and _redirects files. The review feedback suggests improving build resilience by catching exceptions when resolving package integrity to skip problematic versions rather than failing the entire build, and optimizing memory usage in generate_checksums by reading files in chunks instead of loading them entirely into memory.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Replace full-file read_bytes hashing with chunked hashing in generate_checksums. - Keep checksum output format and ordering unchanged while reducing peak memory usage.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a placeholder build script with a robust build_catalog.py script that fetches package metadata from the NPM registry, processes versions, and generates static catalog artifacts along with _headers and _redirects files. Feedback suggests wrapping the resolve_integrity call in a try-except block to prevent a single version's missing or invalid integrity metadata from crashing the entire build process.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces a placeholder build script with a fully functional 'scripts/build_catalog.py' script that builds a static catalog by fetching metadata from NPM. It also introduces static configuration files '_headers' and '_redirects'. Feedback on the changes suggests validating that the SHA-1 hash ('shasum') is exactly 40 hex characters before processing to prevent invalid SRI hashes.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Enforce 40-character SHA-1 hex length before converting shasum to SRI. - Keep fail-fast behavior for malformed integrity metadata.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request replaces the placeholder build script with a concurrent catalog builder (scripts/build_catalog.py) that fetches package metadata from NPM, resolves integrity hashes, and generates static catalog artifacts. It also adds static routing and header configurations via _headers and _redirects. The feedback suggests improving build robustness by catching integrity resolution errors to avoid failing the entire build, and ensuring deterministic checksum generation across different operating systems by sorting files by their relative POSIX path strings.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- Make checksum file ordering deterministic across operating systems. - Sort paths using DIST_ROOT-relative POSIX string keys in generate_checksums.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request adds static configuration files (_headers and _redirects) and replaces the placeholder build script with a new scripts/build_catalog.py script that fetches package metadata from the NPM registry to construct the static catalog. Feedback suggests wrapping the integrity resolution logic in a try-except block to gracefully skip malformed package versions instead of failing the entire build process.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
1 participant
Module Submission / Update
Description:
Checklist
Please confirm the following before submitting your PR (refer to CONTRIBUTING.md / 中文指南 for guidance):
*.jsonfile structure strictly followsschemas/catalog-entry.schema.json(contains onlypackage,trust, andmaintainers; no redundant data like version numbers).official/verified/community).choysum.moduleNamespecified in the npm metadata.