Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ Items with a **→ details** link have a full write-up in [`backlog/`](backlog/)
- [ ] **Bloom perf — instrument first** `needs-design` — bloom is ~5 ms / 23% on solar-system; THREE levers now measured-dead (5→3, bloom0 1/3, fold-into-tonemap — the last a wash on a clean interleaved A/B, spikes 2026-07-22), and the whole pyramid is one timing slot so the cost is unlocalised. Split the slot per-sub-pass before any more attempts. → [details](backlog/2026-07-21-bloom-mip-count-perf.md)
- [ ] **Fold star-upsample into hdr→swap** `needs-design` — delete the standalone fullscreen composite (~1.0 ms real) by sampling the aggregate target in the tonemap shader; bloom-ordering question open. → [details](backlog/2026-07-21-fold-star-upsample-into-tonemap.md)
- [ ] **Real star apparent magnitudes from Earth** `needs-design` — relative photometry is already physical; calibrate the display mapping so the Earth vantage matches the real night sky (interacts with bloom + the brightness slider). → [details](backlog/2026-07-22-star-apparent-magnitude-realism.md)
- [ ] **Physically-honest galaxy surface brightness** `needs-design` — `galaxySbAmp` divides a catalog-relative luminosity by an absolute 30 kpc size reference; Famous is fudged with `sbBoost 0.45` and GLADE's SB is Tully-derived from its own B mag (no real information). → [details](backlog/2026-07-24-galaxy-surface-brightness-model.md)
- [ ] **Per-source colour-gradient spread** `needs-design` — the shared `DISK_TINT_SPREAD` ramp-space constant renders a different physical core-to-rim gradient per catalog (≈0.04–0.30 mag); derive it per source instead. → [details](backlog/2026-07-24-per-source-colour-gradient-spread.md)
- [ ] **Famous-seed redshift-distance fallback breaks on infall members** `needs-design` — M90 bakes at 1.47 Mpc via `v3k/70`, a class bug wherever peculiar velocity swamps Hubble flow. → [details](backlog/2026-07-24-famous-seed-redshift-distance-fallback.md)
- [ ] **Bright star clump at ~5.9 kpc** `deferred` — flux verified conserved; residual over-exposure is display policy (mid-anchor slider + summed knee shipped; retune or tone-map shoulder next). → [details](backlog/2026-07-17-star-clump-brightness-5-9kpc.md)
- [ ] **Foreground body draw/drawPick share a per-frame resolved set** `deferred` — mirrored partition/cull invocations can desync under future edits; star partition runs up to 4×/frame at deep zoom. → [details](backlog/2026-07-17-foreground-body-resolved-set.md)
- [ ] **Star drawBudget small-tier mobile cap + iOS device pass** `deferred` — lower `hardCap` for `tier === 'small'` in `gaia-stars.ts`, tuned on a real device; verify the new vertex-stage storage bindings under WebKit's stricter WebGPU in the same pass.
Expand Down
131 changes: 131 additions & 0 deletions docs/backlog/2026-07-24-famous-seed-redshift-distance-fallback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# Famous-seed redshift-distance fallback breaks on cluster infall members

**Status:** needs-design (2026-07-24)

## Ask

M90 (NGC 4569) renders at `distanceMpc: 1.4714285714285715`, inside the Local
Group, when it is a Virgo Cluster member at roughly 16-17 Mpc. Every other
Virgo member in the famous seed sits at 11-20 Mpc. The wrong distance also
drags the wrong `diameterKpc` with it, so the point mispositions and
mis-sizes at once. Decide how the famous-seed pipeline should handle a galaxy
whose peculiar velocity swamps its Hubble-flow signal, since M90 is the
visible instance of a class, not a one-off.

## Current state

`tools/famous/expandFamousFromCatalogs.ts:384-392` (`distanceMpcFromHyperLeda`)
chains two HyperLEDA fields:

1. `mod0` (true distance modulus) if `e_mod0 < MAX_MOD0_ERROR` (`0.3`,
`expandFamousFromCatalogs.ts:112`): `d_Mpc = 10^((mod0-25)/5)`.
2. else `v3k` (CMB-frame velocity) if positive: `d_Mpc = v3k / H0_KM_S_MPC`
(`H0_KM_S_MPC = 70`, `expandFamousFromCatalogs.ts:106`).

The cached HyperLEDA row for NGC4569 (`data/raw/hyperleda/hyperleda_famous_cache.tsv:70`)
carries `mod0 = 30.37`, `e_mod0 = 0.35`, `v3k = 103`. `0.35 > 0.3` rejects
`mod0`, so the chain falls to `v3k / 70 = 103 / 70 = 1.4714285714285714…`,
which lands in the seed verbatim
(`data/seeds/famous_galaxies.seed.json:1275`, alongside `diameterKpc:
3.9035999332040086`). M89 and M91, the seed's immediate neighbours, sit at
`distanceMpc: 15.68…` and `15.62…` (`famous_galaxies.seed.json:1259,1292`),
both of which cleared the `mod0` gate.

The rejected `mod0` is worth pricing: `10^((30.37-25)/5) = 11.9` Mpc. So the
error gate discards a measurement that misses its threshold by 0.05 mag and
would have placed M90 within a factor of 1.4 of the cluster, in favour of an
estimator that misses by a factor of 11. The gate scores each estimator's
own stated uncertainty in isolation and never compares the two candidates
against each other, so it cannot notice that its fallback disagrees with the
value it just rejected by an order of magnitude.

The chain's own doc comment (`expandFamousFromCatalogs.ts:378-382`) already
names the failure mode for the opposite sign: Local Group galaxies with
`v3k < 0` are excluded by the `> 0` check because they're the ones falling
_toward_ us, and the comment assumes they always have a good `mod0` instead.
M90 is the counter-example the comment doesn't cover: it's a Virgo _infall_
member, close enough that its CMB-frame velocity (103 km/s, mostly peculiar
motion toward the cluster, not Hubble flow) is small and positive, so it
clears the `> 0` check and reads as a 1.5 Mpc Local Group object instead of a
16 Mpc cluster member with a large negative peculiar velocity relative to the
Hubble flow at its true distance. `v3k / H0` is invalid precisely where
peculiar velocity is comparable to Hubble-flow velocity: the Virgo infall
region generally, not something specific to this one galaxy.

**Blast radius beyond position.** `diameterKpc` is angular size times
distance, so the wrong distance also produces a diameter about 10x too
small. Both then feed the galaxy surface-brightness model
(`docs/backlog/2026-07-24-galaxy-surface-brightness-model.md`):
`src/utils/galaxy/galaxySbAmp.ts:36-41` divides by `(diameterKpc /
SB_REF_DIAMETER_KPC)^2`, and the absolute magnitude driving it is
`absoluteFromApparent(magB, distanceMpc)`
(`src/services/engine/bake/buildPointInterleavedBuffer.ts:308`, using
`magB` off the seed entry). Distance also sets the raw Cartesian position:
`tools/famous/buildFamous.ts:45-51` (`entryToXyz`) converts `(ra, dec,
distanceMpc)` directly to `(x, y, z)`. M90 currently renders in the wrong
place, undersized, and at the wrong brightness, all from the one bad
distance.

**Re-running the pipeline reproduces it, so a hand-edit doesn't stick.**
`npm run famous-seed-from-leda -- NGC4569` (`tools/famous/famousSeedFromHyperleda.ts`)
calls the same `mergeIntoFamousEntry` (`expandFamousFromCatalogs.ts:445`) that
`expand-famous` uses, and its merge rule explicitly overwrites
`distanceMpc`/`diameterKpc`/`axisRatio`/`positionAngleDeg`/mags on every run,
preserving only `id`/`names`/`description`
(`expandFamousFromCatalogs.ts:434-440`). A hand-corrected `distanceMpc` in
`famous_galaxies.seed.json` is reverted the next time `expand-famous` runs
over the seed.

**A precedent mechanism exists, but only on the survey path.**
`data/seeds/local_volume_distances.seed.json` is a curated
redshift-independent distance override for exactly this failure class:
entries already carry `method: "Tully-Fisher, Cosmicflows-3 (cz sign is bad
catalog data)"` and `method: "Virgo cluster mean"`
(`local_volume_distances.seed.json:24,36`). It's keyed by 2MASS `massId`
(not PGC, per its own header comment, `tools/catalog/loadLocalVolumeDistanceSeed.ts:15-17`)
and loaded into the survey bin build via
`loadLocalVolumeDistanceSeed()` (`tools/catalog/buildAllBins.ts:637`). The
famous seed path has no equivalent override, which is why M90 has nowhere
to be corrected that survives a re-run.

A related but narrower mechanism is already documented for a different
sub-case: `.claude/skills/add-famous/SKILL.md:151-166` tells a human curator
to co-locate an _interacting pair_ at a CF4 group distance
(`table3.dat.gz` `DMav`) when the pair's members land at mismatched HyperLEDA
depths. That procedure is scoped to pairs sharing a tidal bridge; it doesn't
address a lone infall member like M90, though the CF4 lookup it describes is
the same kind of redshift-independent source a general fix would need.

## Options

- **A distance-override seed for the famous path**, mirroring
`local_volume_distances.seed.json`: a small curated file (keyed by PGC or
NGC id) that `expand-famous`/`mergeIntoFamousEntry` consults before falling
to `v3k`, and refuses to overwrite. Reuses a proven pattern, keeps the
correction auditable with a `method` string per entry. Costs a second
override file plus a merge-priority rule in the builder.
- **A guard in `distanceMpcFromHyperLeda`** that rejects the `v3k` fallback
below some velocity floor and falls through to a CF4 group distance
instead of `null`. Fixes the whole class, not one row, and the
`add-famous` skill already half-describes the CF4 `table3` `DMav` lookup
for pairs, so the machinery to reuse is partly built. Needs a defensible
velocity floor (Virgo's velocity dispersion is a few hundred km/s, so a
fixed cut has to clear that without also rejecting genuinely nearby, slow
Hubble-flow galaxies) and a CF4-by-PGC lookup on a path (famous) that
doesn't have one today.
- **Make the chain comparative rather than sequential.** Where both `mod0`
and `v3k` resolve, take the `mod0` value even past its error gate when the
two disagree by more than the gate could explain, on the reasoning that a
noisy direct measurement beats a systematically invalid proxy. Smallest
change of the three and it needs no new data source, but it only narrows
the failure rather than closing it: a galaxy with no `mod0` at all still
falls through to `v3k`.
- **Accept per-entry manual curation and document the trap.** Cheapest, but
as shown above it doesn't survive the next `expand-famous` run unless
paired with some preservation rule for the corrected field; without that
it's not really an option, just a temporary state.

Landing any of these still requires `npm run build-famous`
(`tools/famous/buildFamous.ts`) and `npm run sync-r2-secure`: `famous.bin`
(`buildFamous.ts:220`) reaches production through R2, not git
(`docs/DEPLOY.md:16`).
57 changes: 57 additions & 0 deletions docs/backlog/2026-07-24-galaxy-surface-brightness-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Physically-honest galaxy surface-brightness model

**Status:** needs-design (2026-07-24)

## Ask

The galaxy point/disk passes are driven by a physical surface-brightness amplitude (`sbAmp`, shipped in the 2026-07 look spike). The _form_ is right but the _normalisation_ is not: it divides a catalog-**relative** luminosity by an **absolute** size reference. Replace it with a physically defensible model, or decide deliberately to keep the current tuning and stop calling it physical.

## Current state

The amplitude is one shared helper consumed by both passes, so any fix lands in one place:

- `src/utils/galaxy/galaxySbAmp.ts:36-41` — `lumRel = 10^(-0.4·(absMag − medianAbsMag))`, `raw = lumRel / (diameterKpc/30)²`.
- `src/utils/galaxy/galaxyMedianAbsMag.ts:43-71` — per-catalog median absolute magnitude, `-20.5` fallback.
- Point pass: baked into slot 13 (`buildPointInterleavedBuffer.ts:177,351`), consumed at `shaders/galaxyCatalog/points/vertex.wesl:223-234`.
- Disk pass: recomputed CPU-side per frame (`proceduralDiskSubsystem.ts:127-132`) and packed into `extras.w`.
- Live knobs: `DEFAULT_GALAXY_SB_SCALE = 5.0`, `SB_MAX = 30.0`, `FALLOFF_STRENGTH = 0.7` (`src/data/defaults.ts:184,193,203`).

**What is already physical:** the `L/D²` form. Surface brightness genuinely is distance-independent — `SB = M + 5·log₁₀(D) + const`, the distance cancels exactly. Keep this.

## Three defects

### 1. `medianAbsMag` conflates a calibration constant with real physics

Subtracting each catalog's own median absolute magnitude erases two different things at once:

- the **photometric band zero-point** (SDSS g / 2MRS J / GLADE B / Famous B) — a genuine per-catalog constant that _should_ be corrected;
- the catalog's **real luminosity distribution** — a genuine physical difference that should _not_ be erased.

Famous galaxies really are more luminous and higher surface brightness. Normalising that away and then re-introducing an absolute 30 kpc size term is what manufactures the mismatch. Secondary effect: the zero-point is a central statistic in log space, then exponentiated, so `lumRel` has median ≈ 1 but mean > 1 (Jensen) — a bright-tail skew that disappears with a fixed reference.

### 2. `sbBoost` is the right shape, derived the wrong way

A per-source multiplicative constant is _exactly_ how a band zero-point offset should be absorbed, so the knob sits in the correct slot. But `famous-galaxy.ts:55` is `0.45` because it cancels an observed median (measured `raw` median 2.14 vs nominal 1.0, tail to 11.7 for NGC 4449), not because of any photometric transformation. Famous's median diameter is 25.4 kpc against the fixed 30 kpc reference, so the size term alone inflates every row by `1/0.847² = 1.39×`. Note the `sbMax` ceiling never engaged — 0 of 80 rows reached it.

### 3. GLADE's surface brightness is synthetic — this caps how physical the model can get

GLADE carries no measured size. `tools/parsers/glade.ts:384-404` derives the diameter from the B magnitude via Tully (1988), implemented at `src/utils/math/galaxyDiameterKpc.ts:45-47`:

```
logR = −0.249·(M_B + 21) + 1.366 ⇒ D ∝ L^0.62 ⇒ SB = L/D² ∝ L^−0.245
```

So for the largest catalog, `sbAmp` is a deterministic re-parameterisation of the B magnitude carrying **no independent surface-brightness information** — and with an inverted slope, where more luminous galaxies render slightly _dimmer_ per pixel. A 40× luminosity range yields only ~2.5× SB variation.

Catalogs with genuinely measured sizes, where SB is real: SDSS (`petroR50_r`, `sdssCsv.ts:278`), 2MRS (`twoMrs.ts:311`), Famous (HyperLEDA `logd25`, `expandFamousFromCatalogs.ts:459`).

## What needs decided

- **Band unification.** Replace `medianAbsMag` with a per-catalog **band zero-point constant** plus a single **global absolute reference magnitude**. Needs a real transformation per source (B→g, J→g, …), each requiring a colour term — Famous has B−V, 2MRS has J−K, GLADE is B-only. Decide the target band and what to do where the colour term is missing.
- **What to do about GLADE.** Options: flag its SB as synthetic and render it at flat surface brightness; source measured diameters (HyperLEDA `logd25` by PGC — coverage is partial, see `project_hyperleda_partial_cache`); or accept the degeneracy and document it.
- **Re-tuning cost.** Dropping the per-catalog median shifts _every_ catalog's brightness, so this needs a full visual pass and probably new `sbScale` defaults. Famous would become legitimately brighter than the field, at which point "too bright" is an exposure/tone-map question, not a data one — it interacts with the HDR bloom threshold and the brightness slider the same way [star apparent-magnitude realism](2026-07-22-star-apparent-magnitude-realism.md) does.
- **Whether it's worth it.** A defensible alternative is to keep the current tuning, delete the "physically grounded" claim from the comments, and treat `sbAmp` as a legibility heuristic. Cheaper and honest.

## Validation

Compare rendered SB against published values for galaxies with real measurements (M31, M87, Sombrero sit around 20–22 mag/arcsec² for disks) from a fixed pose; confirm the point↔disk crossfade still holds constant brightness (`galaxyImpostorBaseline.test.ts` hashes the disk instances); check Milliquas and the DESI cones, which have their own `sbBoost` lifts, don't invert.
49 changes: 49 additions & 0 deletions docs/backlog/2026-07-24-per-source-colour-gradient-spread.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Per-source colour-gradient spread

**Status:** needs-design (2026-07-24)

## Ask

`DISK_TINT_SPREAD` is one shared ramp-space constant driving the warm-core /
cool-rim colour gradient for every galaxy catalog. Because the ramp-units-per-
magnitude conversion is per-source, the same constant produces a different
_physical_ colour gradient in each catalog. Derive the spread per source
instead of sharing one constant.

## Current state

- `src/services/gpu/shaders/lib/colorIndex.wesl:177,189-191` — `rampRadial(t, r)`
calls `ramp(t + DISK_TINT_SPREAD * (0.5 - r))` with `const DISK_TINT_SPREAD =
0.2`, shared by both galaxy passes (points, procedural-disk).
- `src/data/galaxyCatalog/colourIndex.ts:66` — `pickColourIndex` remaps each
source's raw colour onto the ramp's 0..2 range via
`((raw - rangeMin) / (rangeMax - rangeMin)) * 2`, so one magnitude of real
colour is worth `2 / (rangeMax - rangeMin)` ramp units, and that factor is
fixed per source by its own `colourSpec`.
- Per-source factors, read from each `colourSpec` in `src/data/sources/*.ts`:
2MRS 5.00 (`twomrs.ts:20`, range 0.7–1.1), the three DESI sources 2.86
(`desiDeep.ts:47`, `desiSgw.ts:43`, `desiWedge.ts:43`, range 0.35–1.05), SDSS
and Famous 1.33 (`sdss.ts:18`, `famous-galaxy.ts:26`, range 0.5–2.0),
Milliquas 1.00 (`milliquas.ts:35`, range 0.0–2.0), GLADE 0.67
(`glade.ts:20`, range 0.5–3.5, the widest).
- The 0.2 value rests on a measured centre-to-edge gradient of ≈0.15
mag (SDSS-DR4 spiral disk gradients + early-type d(g−r)/d log r + bulge−disc
colour offset), converted through the median per-source factor (1.33):
0.15 × 1.33 ≈ 0.2. One shared constant applied against a per-source factor
means the effective physical gradient this produces today ranges from
≈0.04 mag for 2MRS up to ≈0.30 mag for GLADE — a 7.5× spread in the actual
colour shift a viewer sees, despite every catalog using the same knob.

## What needs decided

- Replace the shared `DISK_TINT_SPREAD` with a per-source spread, derived
from each `colourSpec`'s own `rangeMax − rangeMin` and the measured
gradient for that source, so the physical gradient each catalog renders is
consistent rather than the ramp-space constant.
- The 0.15 mag figure is a g−r gradient. Applying it uniformly to 2MRS's J−K
or SDSS's u−g is itself an approximation — a per-band measured gradient
(where the literature has one) is the more honest input than reusing the
single g−r figure across every band.
- Where a per-band gradient isn't available (2MRS's J−K, GLADE's B−J,
Milliquas's B−R), decide whether to fall back to the g−r figure explicitly,
or flag those catalogs' gradients as approximate.
18 changes: 18 additions & 0 deletions src/@types/data/galaxyCatalog/GalaxyCatalog.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,24 @@ export type GalaxyCatalog = {
*/
parentSurveyByte: Uint8Array;

/**
* Per-catalog MEDIAN absolute magnitude — the surface-brightness
* zero-point `galaxySbAmp` normalises against (see
* `utils/galaxy/galaxySbAmp.ts` and `utils/galaxy/galaxyMedianAbsMag.ts`).
*
* Populated by every runtime construction path: `decodeGalaxyCatalog`,
* `generateSyntheticCloud`, `emptyGalaxyCatalog`, and
* `cloneGalaxyCatalogForTransfer`. Optional ONLY so lightweight test
* fixtures may omit it — consumers that need a value fall back
* themselves (the point bake recomputes via `galaxyMedianAbsMag`; the
* disk planner falls back to -20.5).
*
* Derived, NOT stored in the `.bin` — `decodeGalaxyCatalog` recomputes
* it from the decoded `magG` + `positions` on every load, so the field
* costs no binary format version.
*/
medianAbsMag?: number;

/**
* Per-galaxy "orientation is a deterministic fallback" flag — length ===
* count. 1 means the (axisRatio, positionAngleDeg) pair was synthesised by
Expand Down
Loading
Loading