Look-Elsewhere Effect in CMB Anomaly Searches

A small, reproducible case study of why an apparent "N-σ anomaly" in the cosmic microwave background (CMB) can be a statistical artifact — and how to test such a claim honestly. The worked example is the "Hawking points" proposed in the context of Penrose's conformal cyclic cosmology (CCC).

This repository is teaching material on the look-elsewhere effect (the trials factor / multiple-comparisons problem), not a new scientific result. It independently reproduces the published conclusion of Jow & Scott (2020).

Summary

• A single-test significance ("the most extreme point is 5σ") is meaningless if the location and scale of that point were chosen after looking at the data.
• The honest quantity is the probability-to-exceed (PTE): how often the same search, applied to pure noise, produces something at least as extreme.
• A data-free synthetic demo (lookelsewhere_demo.py) shows that pure Gaussian skies — containing no signal at all — routinely manufacture 4–5σ "Hawking points" once you scan over position and ring scale. The very same test still flags a genuinely injected signal at overwhelming significance, so it is a calibrated referee, not a debunking tool.
• Running the identical pipeline on the real Planck SMICA map (hawking_point_real.py) gives PTE ≈ 0.14 — fully consistent with Gaussian noise — reproducing Jow & Scott (2020). The headline significance dissolves.

Background

An, Meissner, Nurowski & Penrose (2020) reported numerous anomalous circular spots of raised temperature (angular diameters ≈ 3–4°) in the Planck and WMAP CMB maps — "Hawking points," interpreted as the conformally compressed remnants of supermassive-black-hole evaporation in a previous aeon — and quoted confidence levels as high as 99.98%. Jow & Scott (2020) re-analysed the claim and showed that, once the freedom to choose the ring scale is properly marginalised over, the excess drops to roughly 1σ (≈87% confidence): no statistically significant evidence. In their analysis, 13% of pure-ΛCDM simulations produced a more extreme "Hawking point" than the real data at some scale.

The look-elsewhere effect, briefly

If you search a large map over many positions and many scales and report the single most extreme feature, its significance must be assessed against the distribution of the most extreme feature found by the same search in pure noise — not against a single-trial Gaussian. Skipping this inflates apparent significance dramatically. The remedy is always to build a null distribution that includes the full search.

Contents

File	What it does	Needs data?
lookelsewhere_demo.py	Synthetic 2-D Gaussian fields; shows noise manufacturing 4–5σ "anomalies" and the same test confirming a real planted signal. Saves lookelsewhere_demo.png.	No
hawking_point_real.py	The honest core of the Jow & Scott test on a real Planck SMICA map: scans the radial ring-gradient statistic over position and scale, builds a Gaussian null with an identical pipeline, reports the PTE.	Yes (healpy + map)

Quick start

Synthetic demo (no data, no healpy):

pip install numpy scipy matplotlib
python lookelsewhere_demo.py

Real-data analysis (requires healpy, most easily installed via conda-forge — on Windows, native pip usually fails):

conda install -c conda-forge healpy        # or, on Linux/macOS: pip install healpy
python hawking_point_real.py --map COM_CMB_IQU-smica_2048_R3.00_full.fits --nsims 50

The Planck 2018 SMICA temperature map (COM_CMB_IQU-smica_2048_R3.00_full.fits, ~1–2 GB) is available from the Planck Legacy Archive or IRSA. It is not included in this repository.

Example output

Synthetic demo (seed 7):

Null from 400 PURE-NOISE skies: best-point significance median = 4.14 sigma (5-95%: 3.78-4.72)
A pure-noise sky as "data":      4.27 sigma  ->  PTE = 0.318  -> NOT significant (look-elsewhere illusion)
A sky with a real planted signal: 21.69 sigma -> PTE = 0.0000 -> correctly CONFIRMED

Real Planck SMICA map (|b| > 20° mask, 50 sims):

DATA most-significant Hawking point : 5.41 sigma
NULL best point (sims): median = 5.16  95th pct = 5.62
PROPER PTE (scale + position marginalised): 0.14
VERDICT: NOT significant -> consistent with Gaussian noise.

The single-test 5.41σ looks impressive, but pure Gaussian skies produce a median best point of 5.16σ under the same scan; the data sits inside that null (PTE ≈ 0.14), in agreement with Jow & Scott (2020).

Scope and honest caveats

• This reproduces an existing result; it is not a new analysis. Credit for the null result belongs to Jow & Scott (2020).
• hawking_point_real.py implements the correct logic, not a bit-exact replica of the published pipeline: the null uses the data's own (fsky-corrected) power spectrum rather than a theory C_ℓ, the default mask is a simple galactic cut, and there is no beam or noise model. Increase --nsims and supply the official common mask via --mask for a more rigorous number.
• Because such imperfections, if anything, inflate apparent significance, a "not significant" verdict is conservative. If the script ever reports "significant," suspect the fidelity of the null (mask/foreground leakage) before believing new physics.

References

• D. An, K. A. Meissner, P. Nurowski, R. Penrose, Apparent evidence for Hawking points in the CMB Sky, MNRAS 495, 3403 (2020). arXiv:1808.01740
• D. L. Jow, D. Scott, Re-evaluating evidence for Hawking points in the CMB, JCAP 03, 021 (2020). arXiv:1909.09672

License

MIT — see LICENSE.