Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .claude/skills/ac_correct_cti.md
1 change: 1 addition & 0 deletions .claude/skills/ac_fit_cti_model.md
1 change: 1 addition & 0 deletions .claude/skills/ac_load_results.md
1 change: 1 addition & 0 deletions .claude/skills/ac_mask_and_extract.md
1 change: 1 addition & 0 deletions .claude/skills/ac_plot.md
1 change: 1 addition & 0 deletions .claude/skills/ac_setup_environment.md
1 change: 1 addition & 0 deletions .claude/skills/ac_simulate_dataset_1d.md
1 change: 1 addition & 0 deletions .claude/skills/ac_simulate_imaging_ci.md
1 change: 1 addition & 0 deletions .claude/skills/ac_workspace_navigation.md
4 changes: 0 additions & 4 deletions PENDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ wiki page or dataset, in the reference's shape.
- [ ] `.claude/skills/ac_hierarchicac_inference.md` — regenerate for this domain (reference: `.claude/skills/al_hierarchical_inference.md`)
- [ ] `.claude/skills/ac_inspect_results_mcp.md` — regenerate for this domain (reference: `.claude/skills/al_inspect_results_mcp.md`)
- [ ] `.claude/skills/ac_inspect_source_reconstruction.md` — regenerate for this domain (reference: `.claude/skills/al_inspect_source_reconstruction.md`)
- [ ] `.claude/skills/ac_load_results.md` — regenerate for this domain (reference: `.claude/skills/al_load_results.md`)
- [ ] `.claude/skills/ac_mge_decomposition.md` — regenerate for this domain (reference: `.claude/skills/al_mge_decomposition.md`)
- [ ] `.claude/skills/ac_multi_dataset.md` — regenerate for this domain (reference: `.claude/skills/al_multi_dataset.md`)
- [ ] `.claude/skills/ac_plot_fit_residuals.md` — regenerate for this domain (reference: `.claude/skills/al_plot_fit_residuals.md`)
Expand All @@ -32,7 +31,6 @@ wiki page or dataset, in the reference's shape.
- [ ] `.claude/skills/ac_run_search.md` — regenerate for this domain (reference: `.claude/skills/al_run_search.md`)
- [ ] `.claude/skills/ac_run_slam_pipeline.md` — regenerate for this domain (reference: `.claude/skills/al_run_slam_pipeline.md`)
- [ ] `.claude/skills/ac_sensitivity_mapping.md` — regenerate for this domain (reference: `.claude/skills/al_sensitivity_mapping.md`)
- [ ] `.claude/skills/ac_setup_environment.md` — regenerate for this domain (reference: `.claude/skills/al_setup_environment.md`)
- [ ] `.claude/skills/ac_simulate_dataset.md` — regenerate for this domain (reference: `.claude/skills/al_simulate_dataset.md`)
- [ ] `.claude/skills/ac_subhalo_detect.md` — regenerate for this domain (reference: `.claude/skills/al_subhalo_detect.md`)
- [ ] `.claude/skills/ac_time_delay_cosmography.md` — regenerate for this domain (reference: `.claude/skills/al_time_delay_cosmography.md`)
Expand Down Expand Up @@ -113,7 +111,6 @@ wiki page or dataset, in the reference's shape.
- [ ] `skills/ac_ingest_paper.md` — regenerate for this domain (reference: `skills/al_ingest_paper.md`)
- [ ] `skills/ac_inspect_results_mcp.md` — regenerate for this domain (reference: `skills/al_inspect_results_mcp.md`)
- [ ] `skills/ac_inspect_source_reconstruction.md` — regenerate for this domain (reference: `skills/al_inspect_source_reconstruction.md`)
- [ ] `skills/ac_load_results.md` — regenerate for this domain (reference: `skills/al_load_results.md`)
- [ ] `skills/ac_mge_decomposition.md` — regenerate for this domain (reference: `skills/al_mge_decomposition.md`)
- [ ] `skills/ac_multi_dataset.md` — regenerate for this domain (reference: `skills/al_multi_dataset.md`)
- [ ] `skills/ac_plot_fit_residuals.md` — regenerate for this domain (reference: `skills/al_plot_fit_residuals.md`)
Expand All @@ -124,7 +121,6 @@ wiki page or dataset, in the reference's shape.
- [ ] `skills/ac_run_search.md` — regenerate for this domain (reference: `skills/al_run_search.md`)
- [ ] `skills/ac_run_slam_pipeline.md` — regenerate for this domain (reference: `skills/al_run_slam_pipeline.md`)
- [ ] `skills/ac_sensitivity_mapping.md` — regenerate for this domain (reference: `skills/al_sensitivity_mapping.md`)
- [ ] `skills/ac_setup_environment.md` — regenerate for this domain (reference: `skills/al_setup_environment.md`)
- [ ] `skills/ac_simulate_dataset.md` — regenerate for this domain (reference: `skills/al_simulate_dataset.md`)
- [ ] `skills/ac_subhalo_detect.md` — regenerate for this domain (reference: `skills/al_subhalo_detect.md`)
- [ ] `skills/ac_time_delay_cosmography.md` — regenerate for this domain (reference: `skills/al_time_delay_cosmography.md`)
Expand Down
234 changes: 65 additions & 169 deletions skills/README.md
Original file line number Diff line number Diff line change
@@ -1,185 +1,81 @@
# skills/

Procedural how-to-do-X skills for the PyAuto\* lensing stack. Each skill is a single
Markdown file with YAML frontmatter; the body teaches an agent (and through them, the
user) how to write Python that accomplishes one lensing task.
Procedural how-to-do-X skills for the PyAutoCTI stack. Each skill is a single
Markdown file with YAML frontmatter; the body teaches an agent (and through them,
the user) how to write Python that accomplishes one CTI calibration task.

Skills are also exposed at `.claude/skills/` (Claude Code) and `~/.codex/skills/` (when
configured) via symlinks; the canonical files live here.
Skills are also exposed at `.claude/skills/` (Claude Code) and `~/.codex/skills/`
(when configured) via symlinks; the canonical files live here.

## Conventions

- File names use the `ac_<task>` convention for lensing-API skills, e.g. `ac_run_search.md`.
- Project-workflow skills (repo-level operations, template manipulation) use a plain
kebab-case name, e.g. `init-slam.md`, `start-new-project.md`.
- Domain-mode skills that pair an external pipeline repo to the assistant use that
domain as prefix — `euclid_<task>.md` for the Euclid pipeline (euclid mode).
- File names use the `ac_<task>` convention for CTI-API skills, e.g.
`ac_fit_cti_model.md`.
- Project-workflow skills (repo-level operations, template manipulation) use a
plain kebab-case name, e.g. `start-new-project.md`.
- Meta-skills (writing guide, bootstrap protocol) start with `_`.
- Every lensing-API skill is **python-first**: the deliverable is a runnable `.py` script
+ the understanding to evolve it. Project-workflow skills may instead drive `rsync`,
`cp`, or other repo-level operations.
- Source citations use the project-name + repo-relative-path form,
e.g. `PyAutoFit:autofit/non_linear/search/nest/nautilus/`, resolved via
[`../sources.yaml`](../sources.yaml).
- Wiki references use workspace-relative paths,
e.g. `wiki/core/concepts/non_linear_search.md`.
- Every CTI-API skill is **python-first**: the deliverable is a runnable `.py`
script + the understanding to evolve it, grounded in a validated
`autocti_workspace` example — never API memory.
- Source citations use the project-name + repo-relative-path form, e.g.
`PyAutoCTI:autocti/clocker/`, resolved via [`../sources.yaml`](../sources.yaml).
- Wiki references use workspace-relative paths, e.g.
`wiki/core/concepts/charge_transfer_inefficiency.md`.

## Index

Every skill below is a **complete recipe** unless marked `(stub)` — the stubs are gathered
under "Pending — stubbed" at the end, with a queue of catalogued-but-unstubbed topics after
them.
The recipes below are the assistant's current CTI capability. The full growth
queue — skills, wiki pages and datasets still to be authored — is
[`../PENDING.md`](../PENDING.md); this cell was born as a lightweight seed and is
grown in use.

### Meta

- [`_style.md`](./_style.md) — writing guide every skill is authored against. Read first
before adding or editing any skill.
- [`_bootstrap_skill.md`](./_bootstrap_skill.md) — protocol for authoring a new skill on
demand when a user requests a capability not yet covered.

### Setup & maintenance

- [`ac_setup_environment.md`](./ac_setup_environment.md) — detect absent or broken PyAuto\*
environments, install via pip or editable clones when needed, configure caches, verify imports.
- [`ac_update_wiki.md`](./ac_update_wiki.md) — refresh `wiki/core/` pages whose pinned
source commits have moved; surface new public APIs for review.
- [`ac_audit_skill_apis.md`](./ac_audit_skill_apis.md) — verify every PyAuto\* symbol
cited in `skills/` and `wiki/core/api+stack/` resolves in the installed stack;
report stale references with suggested replacements.
- [`ac_refresh_api_docs.md`](./ac_refresh_api_docs.md) — orchestrate a full maintenance
sweep across skill recipes, wiki API pages, and pinned-source drift after a PyAuto\*
upgrade or source refresh.
- [`ac_ingest_paper.md`](./ac_ingest_paper.md) — add a strong-lensing paper (local PDF
or arxiv URL): project-local `wiki/project/bibliography.md` by default inside a science
project; shared `wiki/literature/` in the assistant clone or on explicit promotion.
- [`_style.md`](./_style.md) — writing guide every skill is authored against.
Read first before adding or editing any skill.
- [`_bootstrap_skill.md`](./_bootstrap_skill.md) — protocol for authoring a new
skill on demand when a user requests a capability not yet covered.

### Setup

- [`ac_setup_environment.md`](./ac_setup_environment.md) — install and verify the
PyAutoCTI stack, **including arcticpy** (the C++ arctic clocking code `import
autocti` needs but pip won't install). The #1 setup failure; covers the
libgsl-dev + numpy/cython + arcticpy recipe and the numpy-downgrade trap.

### The CTI calibration pipeline

Grounded per-skill in the matching `autocti_workspace` stage; navigate the whole
catalogue with `ac_workspace_navigation`.

- [`ac_workspace_navigation.md`](./ac_workspace_navigation.md) — the map: the two
geometries (`dataset_1d`, `imaging_ci`), their shared stage pipeline, and which
`start_here.py` to read for any task.
- [`ac_simulate_dataset_1d.md`](./ac_simulate_dataset_1d.md) — simulate a 1D CTI
dataset (charge lines + a known trap model) to generate ground truth.
- [`ac_simulate_imaging_ci.md`](./ac_simulate_imaging_ci.md) — simulate a 2D
charge-injection image (the Euclid VIS / HST ACS geometry; parallel/serial CTI,
`Clocker2D`).
- [`ac_mask_and_extract.md`](./ac_mask_and_extract.md) — mask the FPR/EPER before
fitting, and extract FPR/EPER regions as arrays (`layout.extract.fpr/eper`).
- [`ac_fit_cti_model.md`](./ac_fit_cti_model.md) — compose a CTI model and fit it
with a non-linear search, including the **factor graph** that fits several
charge lines jointly. The core capability.
- [`ac_correct_cti.md`](./ac_correct_cti.md) — remove CTI from data with
`clocker.remove_cti`, to correct science data or diagnose a calibration.
- [`ac_load_results.md`](./ac_load_results.md) — inspect a fit's `result`, and
aggregate across many fits (`Aggregator`, `ac.agg.CTIAgg` and its generators).

### Plotting

- [`ac_plot.md`](./ac_plot.md) — the `autocti.plot` **function API**
(the `subplot_*` / `figure_*` families, `aplt.plot_yx`, `aplt.plot_array`, `aplt.plot_cti_1d`, the
`*_list` combined subplots). No MatPlot/Output/Visuals object.

### Project workflow

- [`start-new-project.md`](./start-new-project.md) — the single bridge to a standalone
**science project** and its full lifecycle (Create → Work → Collaborate → Publish):
scaffold a lean repo that copies the reproducible science and refers back to the assistant
for skills/wiki, run modelling with reproducibility manifests + the `wiki/project/` journal,
build collaborator summaries, and harden for an open-science release (CITATION/license/Zenodo).
Optional HPC folder.
- [`contribute-upstream.md`](./contribute-upstream.md) — prepare a scoped change,
push it either to your collaborator branch on `PyAutoLabs/autocti_assistant`
or to your fork, and open a draft PR into `PyAutoLabs/autocti_assistant`.
- [`init-slam.md`](./init-slam.md) — populate an empty `scripts/` folder with SLaM
pipeline script(s) copied from `autocti_workspace` and tailored to the chosen data
type.

### Euclid mode (pipeline-paired)

Skills pairing the collaboration's
[`euclid_strong_lens_modeling_pipeline`](https://github.com/PyAutoLabs/euclid_strong_lens_modeling_pipeline)
to the assistant; the paired literature context is the dedicated
[`wiki/euclid/`](../wiki/euclid/index.md) sub-wiki.

- [`euclid_setup_pipeline.md`](./euclid_setup_pipeline.md) — clone/install the pipeline,
the `dataset/<sample>/<dataset>/` layout, and the black-box `start_here.py` run.
- [`euclid_prepare_data.md`](./euclid_prepare_data.md) — segmentation validation, binary
mask tuning, extra-galaxy masks/centres via the GUI tools, hand-assembling a dataset
folder.
- [`euclid_model_lens.md`](./euclid_model_lens.md) — choose and run the staged
pipelines: initial MGE+SIE, Sersic photometry, lens-only subtraction, multi-waveband,
full SLaM.
- [`euclid_workflow_products.md`](./euclid_workflow_products.md) — aggregate many fits
into .csv catalogues, .fits stacks, and one-line .png summaries via `workflow/`.
- [`euclid_hpc_runs.md`](./euclid_hpc_runs.md) — `hpc/sync` configuration and the
SLURM batch-array templates for sample-scale runs.

### Data preparation

- [`ac_prepare_imaging_data.md`](./ac_prepare_imaging_data.md) — load and preprocess
FITS imaging, decide masking for real data, measure noise, prepare PSF.
- [`ac_simulate_dataset.md`](./ac_simulate_dataset.md) — synthesise a lens dataset
(imaging or interferometer) from a ground-truth model.

### Model building

- [`ac_build_imaging_model.md`](./ac_build_imaging_model.md) — compose a `Tracer` from
light + mass profiles and wrap it in an `AnalysisImaging`.
- [`ac_build_interferometer_model.md`](./ac_build_interferometer_model.md) — same, but
for visibility-plane data.
- [`ac_custom_profile.md`](./ac_custom_profile.md) — write a new light or mass profile
subclass and register it for use in models.

### Fitting

- [`ac_configure_search.md`](./ac_configure_search.md) — pick and tune a non-linear
search (Nautilus, Dynesty, Emcee, Zeus, …) for your problem.
- [`ac_run_search.md`](./ac_run_search.md) — execute `search.fit(model=..., analysis=...)`
and monitor convergence.
- [`ac_chain_searches.md`](./ac_chain_searches.md) — sequence searches so a later phase
inherits priors from an earlier one.
- [`ac_run_slam_pipeline.md`](./ac_run_slam_pipeline.md) — run a Source-Light-Mass
pipeline (the canonical automated lensing workflow).
- [`ac_debug_fit_failure.md`](./ac_debug_fit_failure.md) — diagnose a fit that didn't
converge or produced unphysical results.

### Results & visualisation

- [`ac_load_results.md`](./ac_load_results.md) — load a completed fit's `Tracer`,
`Samples`, dataset and FITS products from its output folder.
- [`ac_inspect_results_mcp.md`](./ac_inspect_results_mcp.md) — the read-only
results-inspector MCP server: browse fits, summaries, result images and bulk
subplot/FITS extraction from chat harnesses without code execution (Claude
Desktop first).
- [`ac_plot_tracer.md`](./ac_plot_tracer.md) — plot ray tracing, critical curves,
caustics, magnification maps.
- [`ac_plot_fit_residuals.md`](./ac_plot_fit_residuals.md) — plot model image,
residuals, normalised residuals, chi-squared map.
- [`ac_inspect_source_reconstruction.md`](./ac_inspect_source_reconstruction.md) —
inspect a pixelised inversion: regularisation, source-plane image, reconstruction
diagnostics.
- [`ac_to_notebook.md`](./ac_to_notebook.md) — convert a generated narrative-docstring
script to a Jupyter notebook (docstrings → markdown cells, code → code cells) via the
stdlib-only `autoassistant/to_notebook.py`.

### Pending — stubbed (need full recipes)

Drafted as scaffolds during the 2026-05-22 coverage audit against
`autocti_workspace/scripts/`. Each has frontmatter + Orient/Ask/Branch/Combine
structure + `Further reading`; the `Branch` recipes are TODO markers. Fill in
one at a time, paired with their companion wiki/core stub.

**Data types and regimes**

- [`ac_point_source.md`](./ac_point_source.md) (stub) — quasar / multi-image
position fits, flux ratios, point-source deblending.
- [`ac_time_delay_cosmography.md`](./ac_time_delay_cosmography.md) (stub) — H0
from time-delay strong lenses.
- [`ac_group_lensing.md`](./ac_group_lensing.md) (stub) — extra galaxies,
scaling-relation members.
- [`ac_cluster_csv_api.md`](./ac_cluster_csv_api.md) (stub) — cluster-scale
CSV-driven model composition.
- [`ac_multi_dataset.md`](./ac_multi_dataset.md) (stub) — joint imaging +
interferometer, multi-band, wavelength-dependent sources.
- [`ac_weak_lensing.md`](./ac_weak_lensing.md) (stub) — shear catalogue fits
(`WeakDataset` / `AnalysisWeak`).
- [`ac_datacube_modeling.md`](./ac_datacube_modeling.md) (stub) — interferometer
spectral cubes.

**Dark-matter substructure**

- [`ac_subhalo_detect.md`](./ac_subhalo_detect.md) (stub) — Bayesian-evidence
grid search for perturbing subhaloes.
- [`ac_sensitivity_mapping.md`](./ac_sensitivity_mapping.md) (stub) — quantitative
detectability calibration.

**Advanced techniques**

- [`ac_hierarchicac_inference.md`](./ac_hierarchicac_inference.md) (stub) —
population-level / graphical models, expectation propagation.
- [`ac_aggregator_bulk_analysis.md`](./ac_aggregator_bulk_analysis.md) (stub) —
bulk operations across many completed fits, optional result database.
- [`ac_adaptive_pixelization.md`](./ac_adaptive_pixelization.md) (stub) — adaptive
mesh + adaptive regularisation source reconstructions.
- [`ac_mge_decomposition.md`](./ac_mge_decomposition.md) (stub) — Multi-Gaussian
Expansion workflows for lens / source.
- [`ac_custom_analysis.md`](./ac_custom_analysis.md) (stub) — subclassing
`Analysis` to add custom likelihood terms.

**Queue (catalogued, not yet stubbed):** `ac_multi_plane`, `ac_los_halos`,
`ac_over_sampling`, `ac_workflow_outputs`, `ac_data_prep_interactive`,
`ac_bayesian_model_comparison`.
- [`start-new-project.md`](./start-new-project.md) — bridge to a standalone
**science project** repo and its lifecycle (Create → Work → Collaborate →
Publish).
- [`contribute-upstream.md`](./contribute-upstream.md) — prepare a scoped change
and open a draft PR into `PyAutoLabs/autocti_assistant`.
2 changes: 1 addition & 1 deletion skills/_style.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ This is the rule that distinguishes the workspace from a tutorial workspace.
Every Python script the agent saves — whether to `scripts/` or `scripts/scratch/` — follows the
PyAutoCTI **workspace** style, not ad-hoc banner comments. It is the same style used by
every script in `autocti_workspace/scripts/` (canonical example:
`autocti_workspace:scripts/imaging/start_here.py`), and it exists for two reasons: it
`autocti_workspace:scripts/dataset_1d/modeling/start_here.py`), and it exists for two reasons: it
keeps the science and inference narrative inline with the code, and it makes the script
mechanically convertible to a Jupyter notebook — each top-level `"""..."""` block becomes a
markdown cell and the code between blocks becomes a code cell.
Expand Down
Loading
Loading