diff --git a/docs/superpowers/plans/2026-05-18-pyg-uv-flat-index.md b/docs/superpowers/plans/2026-05-18-pyg-uv-flat-index.md new file mode 100644 index 0000000..a4d2eb9 --- /dev/null +++ b/docs/superpowers/plans/2026-05-18-pyg-uv-flat-index.md @@ -0,0 +1,117 @@ +# PyG uv Flat Index Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make generated uv-based projects with PyTorch Geometric resolve PyG wheel pages correctly by emitting a flat uv index and binding PyG extension packages to that explicit source. + +**Architecture:** Keep the current template structure intact and change only the generated uv configuration for PyG: mark the wheel page as a flat index and map PyG extension packages to that explicit index. Validate by regenerating projects that enable PyG and rerunning dependency resolution. + +**Tech Stack:** Copier, Jinja2, uv, PyTorch Geometric + +--- + +### Task 1: Add flat index metadata and source mappings for PyG uv packages + +**Files:** +- Modify: `pyproject.toml.jinja` +- Test: generated `pyproject.toml` in a temporary copied project + +- [ ] **Step 1: Write the failing test** + +Use the existing reproduction as the failing test: generate a project with uv + PyG enabled and run `uv lock`, expecting failure before the template change. + +```bash +uvx copier copy --trust --defaults \ + --data project_name="tmp-pyg-flat" \ + --data package_name="tmp_pyg_flat" \ + --data python_version="3.12" \ + --data package_manager="uv" \ + --data pytorch_cuda_preset="pytorch-2.8.0-cuda-12.6" \ + --data use_pytorch_geometric=true \ + --data use_lightning=true \ + --data use_hydra=true \ + --data logger_choice="none" \ + . /tmp/tmp-pyg-flat +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```bash +rtk uv lock --project /tmp/tmp-pyg-flat +``` + +Expected: dependency resolution fails while probing the PyG wheel page or while resolving PyG extension wheels, often by falling back to a source build for `torch-cluster`. + +- [ ] **Step 3: Write minimal implementation** + +Update the PyG-specific uv configuration in `pyproject.toml.jinja`. + +```toml +[[tool.uv.index]] +name = "pytorch-geometric" +url = "https://data.pyg.org/whl/torch-{{ pytorch_version }}+cu{{ cuda_version | replace('.', '') }}.html" +format = "flat" +explicit = true + +[tool.uv.sources] +torch-scatter = { index = "pytorch-geometric" } +torch-sparse = { index = "pytorch-geometric" } +torch-cluster = { index = "pytorch-geometric" } +``` + +- [ ] **Step 4: Run test to verify it passes** + +Regenerate the same project, then run: + +```bash +rtk uv lock --project /tmp/tmp-pyg-flat +``` + +Expected: `uv lock` succeeds and resolves PyG dependencies. + +- [ ] **Step 5: Commit** + +```bash +git add pyproject.toml.jinja +git commit -m "fix: mark PyG uv source as flat index" +``` + +### Task 2: Sanity-check generated configuration + +**Files:** +- Modify: none +- Test: generated `/tmp/tmp-pyg-flat/pyproject.toml` + +- [ ] **Step 1: Write the failing test** + +Check that the generated file lacks the required flat format and source mappings before the change. + +```bash +grep -n 'format = "flat"' /tmp/tmp-pyg-flat/pyproject.toml +grep -n '\[tool\.uv\.sources\]\|torch-scatter\|torch-sparse\|torch-cluster' /tmp/tmp-pyg-flat/pyproject.toml +``` + +- [ ] **Step 2: Run test to verify it fails** + +Expected: no matching lines before the template fix. + +- [ ] **Step 3: Write minimal implementation** + +No additional implementation beyond Task 1. + +- [ ] **Step 4: Run test to verify it passes** + +Regenerate the project and run: + +```bash +grep -n 'format = "flat"' /tmp/tmp-pyg-flat/pyproject.toml +grep -n '\[tool\.uv\.sources\]\|torch-scatter\|torch-sparse\|torch-cluster' /tmp/tmp-pyg-flat/pyproject.toml +``` + +Expected: the generated PyG uv configuration contains `format = "flat"` and the three source mappings. + +- [ ] **Step 5: Commit** + +No additional commit beyond Task 1 unless follow-up changes are needed. diff --git a/docs/superpowers/specs/2026-05-18-pyg-uv-flat-index-design.md b/docs/superpowers/specs/2026-05-18-pyg-uv-flat-index-design.md new file mode 100644 index 0000000..a453af9 --- /dev/null +++ b/docs/superpowers/specs/2026-05-18-pyg-uv-flat-index-design.md @@ -0,0 +1,36 @@ +# PyG uv Flat Index Design + +## Goal +Fix generated uv-based projects that enable PyTorch Geometric so `uv lock` can resolve PyG extension wheels from `data.pyg.org` without failing on index probing or falling back to source builds. + +## Problem +The template currently emits a PyG-specific `[[tool.uv.index]]` entry pointing at a `data.pyg.org` HTML wheel listing, but it does not mark that source as a flat index and it does not bind PyG extension packages to that explicit index. `uv` treats normal indexes as Simple API-style repositories unless told otherwise, and indexes marked `explicit = true` are only used for packages explicitly mapped to them. As a result, generated projects can fail during `uv lock` with 403 errors while probing the PyG URL or by attempting to build packages like `torch-cluster` from source instead of resolving wheels. + +## Chosen Approach +Update the generated uv configuration for PyG by adding `format = "flat"` to the `pytorch-geometric` `[[tool.uv.index]]` block in `pyproject.toml.jinja` and add `[tool.uv.sources]` mappings for `torch-scatter`, `torch-sparse`, and `torch-cluster` so those packages resolve from the explicit PyG index. + +## Why This Approach +- It matches `uv`'s documented handling of find-links style package pages. +- It matches a known-good working configuration already validated in another project. +- It is the smallest possible template change that addresses the root cause. +- It preserves the current package-selection model and does not require changing Copier questions or post-generation installation steps. + +## Scope +### In scope +- Add `format = "flat"` to the generated PyG uv index block. +- Add `[tool.uv.sources]` mappings for `torch-scatter`, `torch-sparse`, and `torch-cluster`. +- Regenerate uv + PyG projects and verify `uv lock` succeeds. + +### Out of scope +- Changing available PyTorch/CUDA presets. +- Reworking Pixi behavior. +- Replacing the uv index approach with custom install commands. + +## Testing +1. Generate projects with `package_manager=uv` and `use_pytorch_geometric=true`. +2. Confirm the generated `pyproject.toml` includes both `format = "flat"` and `[tool.uv.sources]` mappings for the PyG extension packages. +3. Run `uv lock` in the generated projects. +4. Confirm dependency resolution succeeds for PyG packages such as `torch-cluster` without source build fallback. + +## Risks +The only meaningful risk is that some PyG wheel pages may still have upstream availability gaps for specific torch/CUDA combinations. That would be a separate compatibility issue, not a misconfiguration of `uv` index format. diff --git a/pyproject.toml.jinja b/pyproject.toml.jinja index 8378aab..4145ac4 100755 --- a/pyproject.toml.jinja +++ b/pyproject.toml.jinja @@ -98,7 +98,13 @@ explicit = true [[tool.uv.index]] name = "pytorch-geometric" url = "https://data.pyg.org/whl/torch-{{ pytorch_version }}+cu{{ cuda_version | replace('.', '') }}.html" +format = "flat" explicit = true + +[tool.uv.sources] +torch-scatter = { index = "pytorch-geometric" } +torch-sparse = { index = "pytorch-geometric" } +torch-cluster = { index = "pytorch-geometric" } {% endif -%} # ============================================================================