fix(howtolens): should_simulate every dataset load; run under SMALL_DATASETS

Make every HowToLens tutorial run under PYAUTO_SMALL_DATASETS=1 (16x16) so
smoke/CI stays fast, instead of relying on a dead unset-workaround.

- Migrate all 22 raw `if not dataset_path.exists():` guards to
  `al.util.dataset.should_simulate(str(dataset_path))` — the drop-in idiom that
  ALSO deletes + re-simulates stale data under the cap (so on-disk full-res
  data can't mismatch the 16x16 masks/grids).
- Add missing guards: tutorial_0_visualization (simple__no_lens_light) and
  tutorial_7_fitting's second load (simple__no_lens_light__mass_sis).
- Fix tutorial_7_fitting's first-load guard: it pointed at the wrong simulator
  (no_lens_light__mass_sis) for the `howtolens` dataset, which is produced by
  tutorial_6_data.py — a latent bug the migration exposed.
- Add guard for tutorial_6_lens_modeling's second load (lens_sersic).
- Remove the dead `howtolens/` + `guides/` PYAUTO_SMALL_DATASETS-unset overrides
  from config/build/env_vars.yaml (they matched zero files here).
- Regenerate notebooks + navigator catalogue.

Verified: every chapter passes on a FRESH dataset/ dir (cloud shard isolation)
under SMALL_DATASETS=1 — ch1 9/9, ch2 8/8, ch3 6/6, ch4 all green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: 2 people

config/build/env_vars.yaml

@@ -30,15 +30,10 @@ overrides:
     unset: [PYAUTO_SMALL_DATASETS]
   - pattern: "multi/start_here"
     unset: [PYAUTO_SMALL_DATASETS]
-  # Non-simulator scripts that load committed FITS data need full-size
-  # datasets to avoid shape mismatch with pre-existing 100x100 data.
-  # Simulators run first and regenerate data at the capped size, but
-  # scripts using the old `if not dataset_path.exists()` pattern skip
-  # re-simulation when data already exists.
-  - pattern: "howtolens/"
-    unset: [PYAUTO_SMALL_DATASETS]
-  - pattern: "guides/"
-    unset: [PYAUTO_SMALL_DATASETS]
+  # (Removed the dead `howtolens/` + `guides/` PYAUTO_SMALL_DATASETS-unset
+  # workarounds: they matched no files here, and the tutorials now use
+  # `al.util.dataset.should_simulate` which deletes + re-simulates stale
+  # data under the cap, so the chapters run correctly at 16x16.)
   # guides/results/start_here.py must produce real samples so the example
   # scripts that read from `output/results_folder` afterwards
   # (data_fitting, queries, models, samples_via_aggregator) find a

notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb

@@ -116,6 +116,32 @@
    "outputs": [],
    "execution_count": null
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "__Dataset Auto-Simulation__\n",
+    "\n",
+    "If the dataset does not already exist on your system, it is created by running the corresponding\n",
+    "simulator script. This ensures every example script can be run without manually simulating data first."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
+    "    import subprocess\n",
+    "    import sys\n",
+    "\n",
+    "    subprocess.run(\n",
+    "        [sys.executable, \"scripts/simulator/no_lens_light.py\"],\n",
+    "        check=True,\n",
+    "    )"
+   ],
+   "outputs": [],
+   "execution_count": null
+  },
   {
    "cell_type": "markdown",
    "metadata": {},

notebooks/chapter_1_introduction/tutorial_7_fitting.ipynb

@@ -129,20 +129,21 @@
    "source": [
     "__Dataset Auto-Simulation__\n",
     "\n",
-    "If the dataset does not already exist on your system, it will be created by running the corresponding\n",
-    "simulator script. This ensures that all example scripts can be run without manually simulating data first."
+    "The `howtolens` dataset is the one built up and saved in tutorial 6 (`tutorial_6_data.py`). If it does\n",
+    "not already exist on your system, it is created by running that script. This ensures every example\n",
+    "script can be run without manually simulating data first."
    ]
   },
   {
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",
     "    subprocess.run(\n",
-    "        [sys.executable, \"scripts/simulator/no_lens_light__mass_sis.py\"],\n",
+    "        [sys.executable, \"scripts/chapter_1_introduction/tutorial_6_data.py\"],\n",
     "        check=True,\n",
     "    )\n",
     "\n",
@@ -1025,6 +1026,15 @@
     "dataset_name = \"simple__no_lens_light__mass_sis\"\n",
     "dataset_path = Path(\"dataset\") / \"imaging\" / dataset_name\n",
     "\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
+    "    import subprocess\n",
+    "    import sys\n",
+    "\n",
+    "    subprocess.run(\n",
+    "        [sys.executable, \"scripts/simulator/no_lens_light__mass_sis.py\"],\n",
+    "        check=True,\n",
+    "    )\n",
+    "\n",
     "dataset = al.Imaging.from_fits(\n",
     "    data_path=dataset_path / \"data.fits\",\n",
     "    psf_path=dataset_path / \"psf.fits\",\n",

notebooks/chapter_2_lens_modeling/tutorial_1_non_linear_search.ipynb

@@ -281,7 +281,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_2_practicalities.ipynb

@@ -191,7 +191,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_3_realism_and_complexity.ipynb

@@ -129,7 +129,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_4_dealing_with_failure.ipynb

@@ -125,7 +125,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_5_linear_profiles.ipynb

@@ -139,7 +139,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_6_masking_and_positions.ipynb

@@ -112,7 +112,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",

notebooks/chapter_2_lens_modeling/tutorial_7_results.ipynb

@@ -107,7 +107,7 @@
    "cell_type": "code",
    "metadata": {},
    "source": [
-    "if not dataset_path.exists():\n",
+    "if al.util.dataset.should_simulate(str(dataset_path)):\n",
     "    import subprocess\n",
     "    import sys\n",
     "\n",