diff --git a/reports/Astrophotography stacking techniques since 2022.md b/reports/Astrophotography stacking techniques since 2022.md new file mode 100644 index 0000000..bba331c --- /dev/null +++ b/reports/Astrophotography stacking techniques since 2022.md @@ -0,0 +1,31 @@ +# Stacking software converges on splines, CFA drizzle, AI plugins + +Astrophotography stacking software has advanced substantially since 2022, but along much narrower lines than the pace of AI hype would suggest: production tools converged on **spline-based distortion models** for registration (PixInsight's DDM engine, December 2024; Siril's polynomial astrometric registration, beta April 2025), on **true Bayer/CFA-native drizzle** as a first-class feature (Siril 1.4.0, December 2025; AstroPixelProcessor's "Bayer Drizzle"), and on **proprietary neural-network plugins** for denoising/deconvolution/star removal (RC-Astro's BlurXTerminator/NoiseXTerminator, GraXpert's AI models, Seti Astro's Cosmic Clarity) that amateurs now treat as standard workflow steps despite having essentially no independent quantitative validation. Meanwhile, the rejection/combine mathematics that actually decides which pixels survive into a stack — sigma-clip, Winsorized sigma-clip, ESD, linear-fit clipping — has not changed at all since 2022 in any tool surveyed; the one genuine 2022 advance there was PixInsight's PSF Signal Weight frame-weighting scheme, not a new rejection test. Deep learning has made real, peer-reviewed inroads into professional survey pipelines — a transformer denoiser published in *Science* in 2026, a self-supervised method in *Nature Astronomy* in 2025, ML-based transient triage deployed operationally at ZTF since 2023 — but none of this machinery has crossed into shipped amateur stacking software, which remains almost entirely classical/geometric under the hood. Measured against this landscape, OriginStack's own more unusual capabilities — robust-PCA calibration masters, ZOGY transient detection, a physics-based sky model, Monte Carlo uncertainty propagation with per-pixel confidence maps, PSF-matched and native-CFA drizzle, and an in-house multi-task ONNX frame classifier — have no close published or shipped counterpart in the surveyed literature or competing tools; several of them (robust PCA on calibration frames, an amateur-tool confidence map, a 7-task frame-quality classifier trained specifically on deep-sky data) appear to be near-unique, while a few areas (true Gaia-spectrum-based color calibration, ML-based transient triage layered on differencing output, explicit multi-panel mosaic stitching) represent real, citable gaps worth tracking. + +## Registration converged on splines; nobody uses ML for star matching + +Every actively developed desktop tool shipped a dated registration upgrade in this window, and they landed on the same mathematical family. PixInsight 1.9.0 "Lockhart" (December 20, 2024) reimplemented its astrometry engine around a Domain Decomposition Method using thin-plate splines and radial basis functions supporting up to 25,000 control points, and carried the same spline engine into StarAlignment itself for "a significant improvement in its ability to model arbitrary distortions" ([DIYPhotography](https://www.diyphotography.net/pixinsight-1-9-1-lockhart-released-major-upgrades-and-improvements/)). Siril's 1.4 line (previewed December 2024, beta April 2025) answered with up-to-fifth-order polynomial astrometric distortion correction and, on top of it, genuine mosaic stacking with edge feathering — explicitly aimed at frames "with significantly different center points" ([Siril, "Twelve Days of Siril"](https://siril.org/2024/12/the-twelve-days-of-siril/)). Both tools treat field rotation on alt-az mounts the same way: as one term of whatever multi-degree-of-freedom transform (homography, affine, rigid) each frame is fit against, not as a separately modeled phenomenon — true optical de-rotation for deep-sky work remains a hardware problem (mechanical field de-rotator plus off-axis guider), distinct from the ephemeris-driven derotation planetary tools like AutoStakkert and WinJUPOS perform in a different imaging regime ([Cloudy Nights](https://www.cloudynights.com/topic/764538-field-rotation-alt-az-tracking-software-to-de-rotate/)). + +Cross-session, unknown-rotation registration followed the same pattern: rather than publishing a named "blind rotation matching" algorithm, the industry's practical answer has been to route multi-night registration through plate-solving/WCS astrometry (Siril 1.4's mosaic engine) rather than frame-to-frame blind matching. That makes OriginStack's own approach — a blind rigid star-pattern match (`src/blind_match.py`) used for both `--merge` and hierarchical multi-session combining, requiring no plate solve — a genuinely different design point from what shipped tools do, not an inferior one; it works without a WCS at all, where Siril's newer mosaic path depends on one. Machine learning is conspicuously absent from this whole area: the only located 2022+ ML work on star centroiding, matching, or transform estimation targets HST/WFPC2 photometric calibration, CubeSat star-tracker attitude determination, or historical photographic-plate digitization ([arXiv:2404.16995](https://arxiv.org/html/2404.16995); [arXiv:2404.19108](https://arxiv.org/html/2404.19108v1); [arXiv:2604.04714](https://arxiv.org/pdf/2604.04714)) — no amateur deep-sky stacking tool has adopted a learned star detector or matcher. OriginStack's `registration_stars` thin-catalog rescue and its elastic/local-displacement-field registration combined with a fitted radial distortion model sit squarely inside the same spline/local-regression family the two flagship tools converged on in late 2024 — OriginStack already had this machinery before either PixInsight's DDM or Siril's polynomial model shipped in their current form, and its analytic `RadialModel` plus `fit_displacement_field` covers single-session optical distortion that neither competitor's registration engine models as a *global instrument correction* the way OriginStack's persistent-per-instrument radial fit does. The one real gap: OriginStack has no dedicated multi-panel mosaic-stitching feature comparable to Siril 1.4's edge-feathered mosaic stacking or the wide-field-specific control-point density PixInsight's DDM engine targets — its hierarchical combine handles multiple *sessions of the same field*, not multiple *pointings* tiled into a larger frame. + +## Rejection math is settled; PSF-aware weighting is the real 2022 news + +No tool surveyed — Siril, PixInsight, AstroPixelProcessor, or DeepSkyStacker — has shipped a structurally new pixel-rejection statistic since 2022. Sigma-clip, Winsorized sigma-clip, the Generalized Extreme Studentized Deviate test (Rosner 1983), and linear-fit clipping remain the complete rejection toolkit across every product, and Siril's current documentation lists exactly these with no dated post-2022 addition ([Siril stacking docs](https://siril.readthedocs.io/en/stable/preprocessing/stacking.html)). The one dated, concrete 2022 advance is on the *weighting* side: PixInsight 1.8.9 (March 13, 2022) introduced PSF Signal Weight and PSF SNR, which derive a frame's integration weight from per-star PSF/aperture photometry rather than whole-frame statistics, and are explicitly resistant to being fooled by a stray artifact like an airplane trail contaminating one frame's global SNR ([PixInsight documentation](https://pixinsight.com/doc/docs/ImageWeighting/ImageWeighting.html)). That is a real, useful idea — but it is still a single scalar weight per frame, not a spatially-varying weight map, which makes it strictly less granular than OriginStack's own patch-weighted quality scoring and inverse-variance-weighted combine (`ivw_combine`, which derives weight from each frame's own measured Phase 1 noise sigma, a real Gauss-Markov optimum rather than PSFSW's ad hoc photometric proxy) and its native `patch_weighted_sigma_combine` kernel, which fuses sigma-clip rejection with quality weighting at sub-frame granularity. Nothing found in this research suggests PixInsight, Siril, or APP weight *below* the whole-frame level the way OriginStack's patch-weighted path already does. + +Drizzle saw the most concrete rewrites of any combine-adjacent feature in this window, converging on two axes: kernel flexibility and native CFA operation. PixInsight's "Fast Drizzle" (December 2024) added circular, Gaussian, and variable-shape kernels at substantially higher speed than classical square-kernel drizzle ([DIYPhotography](https://www.diyphotography.net/pixinsight-1-9-lockhart-released-major-upgrades-and-improvements/)), while Siril spent "more than two years" replacing its older upscale-then-stack "simplified drizzle" with a genuine HST-style Variable-Pixel Linear Reconstruction implementation in version 1.4.0 (December 5, 2025), shipping five kernel choices (square, point, turbo, Gaussian, Lanczos2/3) and full native CFA/Bayer drizzle where "the CFA color of the current droplet sets which channel of the output image it lands on" ([Siril drizzle docs](https://siril.readthedocs.io/en/stable/preprocessing/drizzle.html)). AstroPixelProcessor's parallel "Bayer Drizzle" feature is well-established in community use, though no dated official changelog entry for it could be located. This is exactly the ground OriginStack already occupies and, in places, exceeds: its PSF-matched drizzle kernel (`--drizzle-kernel psf`, a Wiener-regularized inverse filter of the session's own measured PSF used as a matched-filter drizzle drop, rather than a fixed circular/Gaussian shape) is a more targeted idea than PixInsight's "variable-shape" kernel language suggests, and `src/cfa_drizzle.py`'s native Bayer-aware drizzle — measured at 15.7x native speedup and validated against a numpy mirror — predates Siril's December 2025 true-drizzle rewrite. The one place the field has moved past raw classical drizzle that OriginStack should note as a live community practice rather than a shipped algorithm: pairing drizzle's low-contrast recovered fine detail with a commercial AI deconvolver (BlurXTerminator) is described by its vendor as already-working production practice on real data, not a research proposal, needing well-dithered, low-noise data for the deconvolution step to have real information to sharpen ([RC Astro](https://www.rc-astro.com/drizzle-plus-deconvolution-a-killer-combination/)) — a two-stage workflow OriginStack's own IBP-on-drizzle path does natively and without a proprietary black box. + +## AI denoising plugins dominate the amateur market with no independent benchmarks + +The commercial denoising/deconvolution/star-removal ecosystem is the single most active area surveyed since 2022, and it is now dominated by opaque, actively-updated neural-network plugins rather than open, published methods. BlurXTerminator's AI4 release (December 14, 2023) was the first version to process linear data directly rather than through an intermediate stretch, which the vendor says cut flux-conservation error from ~75% (AI2) to 2–5% ([RC Astro](https://www.rc-astro.com/blurxterminator-2-0-ai4-release/)) — but that number, like essentially every performance claim in this space, comes only from the vendor's own release notes. GraXpert added AI denoising in v3.0.0 (April 2024), AI object-deconvolution in v3.1.0rc1 (November 2024), and a separate AI *stellar* deconvolution model in v3.1.0rc2 (January 2025) — deliberately splitting stellar and non-stellar structure into two networks so neither modifies the other's target class ([GraXpert releases](https://github.com/Steffenhir/GraXpert/releases)), the same "process stars and structure separately" philosophy behind OriginStack's own `--starless-process`. Siril's 2024–2025 strategy has been to *integrate* these third-party engines (GraXpert's background extraction, denoising, and deconvolution; Seti Astro's Cosmic Clarity) rather than build a competing in-house model ([Siril GraXpert docs](https://siril.readthedocs.io/en/latest/processing/graxpert.html)). Independent review confirms these tools work but flags real tradeoffs — Cloudy Nights found NoiseXTerminator's default settings "more aggressive" than PixInsight's classical wavelet/median-transform hybrid noise reduction and capable of producing "heterogeneous blobs" under stretching ([Cloudy Nights](https://www.cloudynights.com/articles/astro-gear-today/reviews/software/noise-be-gone33-testing-rc-astro-noisexterminator-r4568/)) — but no controlled, third-party PSNR/SSIM/flux-recovery benchmark against classical wavelet or Richardson-Lucy methods exists anywhere in the sources found for this report. The absence of independent validation is itself the finding: OriginStack's own documented, test-covered numbers for its curvelet-inspired wavelet denoiser and native deconvolution kernels are more rigorously characterized than the commercial tools amateurs currently treat as best-in-class. + +The academic literature tells a different story, and it has genuinely moved fast: self-supervised and diffusion-based denoising crossed into top-tier journals in 2025–2026. ASTERIS, a self-supervised transformer that jointly denoises across multiple raw exposures using their correlated noise structure (rather than denoising a single combined frame), was published in *Science* and reports a full magnitude improvement in detection limit at 90% completeness and three times more z>9 galaxy candidates on JWST/Subaru data ([arXiv:2602.17205](https://arxiv.org/pdf/2602.17205)). The TDR (Train-Denoise-Restore) method, built on Self2Self self-supervised training plus an explicit error-correction restoration pass, published in *Nature Astronomy* in 2025, cut solar magnetogram noise from ~8 Gauss to ~2 Gauss with quantitative control over deviation — addressing what its authors call a real gap in prior deep-learning denoisers, which "lack quantitative control of the deviation or error" ([arXiv:2502.16807](https://arxiv.org/abs/2502.16807)). Diffusion-model deconvolution (2023–2024) reframes PSF deconvolution as posterior sampling rather than a single point estimate, explicitly to capture reconstruction uncertainty ([arXiv:2307.11122](https://arxiv.org/abs/2307.11122); [arXiv:2411.19158](https://arxiv.org/html/2411.19158v2)) — conceptually the closest academic analogue to OriginStack's own Monte Carlo uncertainty propagation, though aimed at a different stage of the chain. None of this — transformers, diffusion posterior sampling, self-supervised training — has reached any amateur stacking tool. And on the specific question of curvelet/shearlet-style directional denoising with structure protection, the search found essentially no astronomy-specific 2022+ literature at all; the active 2022+ work on that transform family is in hyperspectral remote sensing and seismic denoising, not astronomy, suggesting the field's research attention has simply moved to the deep-learning methods above rather than any documented result showing curvelet/shearlet approaches underperformed. That means OriginStack's `directional_wavelet_denoise` is not behind any missed classical-transform advance — there isn't one to be behind on — and the meaningful comparison class for it is the commercial AI denoisers, against which it is arguably better characterized, not the academic literature. + +## OriginStack's rarer capabilities have little or no shipped competition + +Several of OriginStack's more distinctive features turn out to have essentially no counterpart anywhere in the surveyed 2022+ literature or competing tools, which is worth stating plainly rather than hedging. Robust PCA / low-rank-plus-sparse decomposition applied to bias/dark/flat calibration masters returned no hits at all outside this project's own release notes; the nearest precedent, low-rank+sparse decomposition for exoplanet direct-imaging PSF subtraction (the LLSG algorithm), is an eight-year-old technique applied to a different problem (separating a static PSF from a sparse companion across science frames, not building a calibration master from many independent exposures), and every mainstream calibration-frame guide found — Celestron's, Siril's, PixInsight's — recommends only median or sigma-clipped-mean combination for masters ([Celestron guide](https://www.celestron.com/blogs/knowledgebase/the-ultimate-guide-to-calibration-frames-for-astrophotography)). A dedicated, amateur-stacking-specific academic paper on per-pixel confidence/error maps likewise does not exist; the closest precedent is Rubin/LSST's per-pixel variance-plane coadd product, which is linear-stage inverse-variance propagation, not a Monte Carlo pass through a nonlinear post-processing chain the way OriginStack's `propagate_uncertainty` works — and that hybrid pattern (analytic where the chain is linear, Monte Carlo where it is adaptive and nonlinear) is exactly what both Rubin's own noise-image-through-pipeline practice for weak-lensing calibration and the 2024–2025 conformal-prediction literature independently converged on, which is a genuine validation of the design choice even though no one has published it for a stacking pipeline specifically ([Rubin DP2 docs](https://dp2.lsst.io/products/images/deep_coadd.html); [arXiv:2502.19194](https://arxiv.org/pdf/2502.19194)). OriginStack's physics-based sky background model, likewise, has no located competitor in any surveyed tool — GraXpert and PixInsight's own background-extraction tools are both purely statistical/mesh-based, not physically parameterized against airglow, moonlight, and zodiacal geometry. + +The frame-quality/defect-classification picture is more mixed but still favorable. On the professional side, ML frame-quality screening is real and deployed — GWAC's autoencoder quality screen, ZTF's BTSbot (a multimodal CNN achieving 94.1% test accuracy, autonomously confirming 609 genuine transients between December 2023 and May 2024) ([ApJ 2024](https://iopscience.iop.org/article/10.3847/1538-4357/ad5666)) — but every located instance operates at survey scale on difference-imaging alert streams, sitting *downstream* of classical differencing (ZOGY, Alard-Lupton) as a triage layer, never replacing the differencing mathematics itself. No amateur stacking tool has shipped anything comparable for either per-subframe quality scoring or transient candidate triage; the nearest amateur-adjacent analogue, a 2023 ResNet50/AutoML image-quality study on Stellina smart-telescope stacks, is a single exploratory paper, not a shipped feature ([arXiv:2311.10617](https://arxiv.org/pdf/2311.10617)). That makes `originvision` — a from-scratch, seven-task multi-head CNN trained specifically on amateur deep-sky frames and deployed via a native ONNX runtime — genuinely ahead of anything documented for the amateur space, while simultaneously exposing OriginStack's one real, citable ML gap: it has no BTSbot/O'TRAIN-style learned triage layer sitting on top of its own ZOGY transient candidates, which is precisely the pattern professional pipelines have converged on since 2022 to manage false-positive volume ([O'TRAIN, A&A 664, A81](https://www.aanda.org/articles/aa/full_html/2022/08/aa42952-21/aa42952-21.html)). The other clear, dated gap is photometric color calibration: PixInsight's Spectrophotometric Color Calibration (November 2022) integrates each matched star's actual Gaia DR3 spectrum against the user's real filter/sensor response curves, claiming a 300–400% reduction in color-calibration uncertainty versus older photometry-based methods, and Siril has since adopted the same approach as a replacement for its older PCC ([PixInsight SPCC docs](https://pixinsight.com/doc/docs/SPCC/SPCC.html)). OriginStack's `spcc` method approximates this with a blackbody spectrum fit to each star's Gaia `Teff`, not the star's actual measured spectrum — a real, acknowledged methodological shortcut relative to what is now close to an industry standard among the two most sophisticated competing tools. Finally, on infrastructure: GPU adoption across amateur tools remains genuinely uneven (AstroPixelProcessor has explicitly declined CUDA support over cross-platform dependency concerns) and Rust-based rewrites of stacking engines are embryonic — the only located example, AetherStack, has zero releases and 27 commits — which means OriginStack's breadth of native Rust kernel coverage across nearly every hot path in the pipeline has no comparably mature open-source precedent at all. + +## Conclusion + +The clearest pattern across all five research threads is a split between a *classical core that has stopped changing* and a *commercial AI periphery that is changing fast but is barely measured*. Rejection algorithms, the actual arithmetic that decides which pixels survive a stack, have not moved since 2022 anywhere; what has moved is distortion modeling (splines), drizzle (CFA-native, flexible kernels), and everything downstream of the linear stack (denoise, deconvolve, star-removal), where proprietary neural plugins now dominate amateur workflows on the strength of vendor marketing rather than independent benchmarking. OriginStack sits in an unusual position relative to this landscape: its classical/statistical machinery (IVW, patch-weighted native combine, PSF-matched and CFA-native drizzle, elastic registration with a fitted radial distortion model) already matches or exceeds what shipped competitors reached only in their most recent 2024–2025 releases, while several of its more idiosyncratic capabilities — robust-PCA masters, a physics-based sky model, full-chain Monte Carlo uncertainty propagation with confidence maps, and a purpose-trained multi-task frame classifier — appear to have no real precedent in either the published literature or competing products, which is a stronger competitive position than "keeping pace" implies. The honest gaps are narrow and specific rather than broad: a true spectral (not blackbody-approximated) color calibration step, a learned triage layer downstream of ZOGY analogous to what ZTF and Rubin now run operationally, and an explicit multi-panel mosaic-stitching mode. None of those gaps require chasing the AI-plugin arms race that dominates current amateur discourse — they are targeted, well-precedented extensions of capabilities OriginStack has already built the surrounding infrastructure for. diff --git a/research_notes/Astrophotography stacking techniques since 2022/combine_rejection_drizzle_superres.md b/research_notes/Astrophotography stacking techniques since 2022/combine_rejection_drizzle_superres.md new file mode 100644 index 0000000..d5c86c2 --- /dev/null +++ b/research_notes/Astrophotography stacking techniques since 2022/combine_rejection_drizzle_superres.md @@ -0,0 +1,116 @@ +# Combine/Rejection Algorithms, Drizzle Variants, and Super-Resolution in Astro Stacking (2022+) + +## What new rejection/combine algorithms (beyond classic sigma-clip, median, Winsorized sigma clip, linear-fit clipping) have been published or shipped since 2022? + +### Takeaway +No fundamentally new *rejection* algorithm (i.e., a new statistical test for discarding outlier pixels across the frame stack) was found published or shipped since 2022 in the major amateur tools or in astro-ph.IM — sigma-clip, Winsorized sigma-clip, ESD (Rosner 1983), and linear-fit clipping remain the state of the art and predate 2022 in all the tools surveyed. The clearest 2022+ advance is on the *weighting* side of combination rather than rejection: PixInsight's PSF Signal Weight / PSF SNR estimators (introduced in PixInsight 1.8.9, March 13, 2022), which replace whole-frame SNR statistics with per-star PSF/aperture photometry as the basis for a frame's integration weight. + +### Cited Findings +- PixInsight 1.8.9 was released March 13, 2022, per a PixInsight Forum thread — [PixInsight 1.8.9 Released](https://pixinsight.com/forum/index.php?threads/pixinsight-1-8-9-released.18148/) +- PixInsight's official documentation states PSF Signal Weight (PSFSW) and PSF SNR "since version 1.8.9" are new image weighting algorithms for `ImageIntegration`: PSFSW is "a new image quality estimation algorithm based on a hybrid PSF/aperture photometry measurement methodology," combining "the sum of PSF flux estimates" (total signal) and "the sum of mean PSF flux estimates" (signal concentration in star cores) — [PixInsight Reference Documentation: New Image Weighting Algorithms in PixInsight](https://pixinsight.com/doc/docs/ImageWeighting/ImageWeighting.html) +- The same documentation demonstrates PSFSW is resistant to being fooled by non-astronomical artifacts (it specifically calls out resistance to "the airplane trail" contaminating a frame), unlike legacy global-statistic SNR weighting — [PixInsight Reference Documentation: New Image Weighting Algorithms in PixInsight](https://pixinsight.com/doc/docs/ImageWeighting/ImageWeighting.html) +- Siril's documentation (current, i.e. Siril 1.4.x/1.5.0-era, 2025) describes its rejection methods as Sigma Clipping, Median Sigma Clipping, Winsorized Sigma Clipping, Linear Fit Clipping, and Generalized Extreme Studentized Deviate (ESD) Test (Rosner 1983) — none dated as new since 2022; a Cloudy Nights community thread discusses Linear Fit vs. ESD as an established choice, not a new feature — [Stacking — Siril 1.4.4 documentation](https://siril.readthedocs.io/en/stable/preprocessing/stacking.html); [Siril Linear Fit vs. Generalized Extreme Studentized Deviate Test](https://www.cloudynights.com/topic/911059-siril-linear-fit-vs-generalized-extreme-studentized-deviate-test/) +- A search surfaced an arXiv preprint titled "Robust Heteroskedastic Matrix Factorization: A Generalization of PCA that Flags Outliers and Handles Missing Data" (arXiv:2607.08081) that by title appears to generalize PCA-style decomposition with built-in outlier flagging — this was **not independently verified** (only the title/abstract snippet was seen, the paper was not fetched in full, and it is not confirmed to be astronomy-specific or applied to frame-stack rejection) — [Robust Heteroskedastic Matrix Factorization](https://arxiv.org/pdf/2607.08081) + +### Inferences +- The rejection-algorithm space appears mature/saturated for the amateur-tooling ecosystem: vendors are investing 2022+ engineering effort in *weighting* (which frames count more) and *drizzle performance/kernels* (see next section) rather than inventing new per-pixel outlier tests. +- PSFSW's approach (deriving a combine weight from per-star PSF photometry rather than global frame statistics) is conceptually adjacent to — but distinct from — "patch/local adaptive weighting" schemes that weight sub-regions of a frame differently; PSFSW is still a single scalar weight per whole frame, not a spatially-varying weight map. + +### Gaps +- No source found describing a *new* per-pixel rejection statistic (i.e., something structurally different from sigma-clip/Winsorized/ESD/linear-fit) shipped in Siril, PixInsight, AstroPixelProcessor, or DeepSkyStacker between 2022 and 2026. +- The arXiv:2607.08081 "Robust Heteroskedastic Matrix Factorization" paper's relevance to astronomical frame-stack rejection could not be confirmed — it was not fetched or read in full; flagging it here as an unverified lead only. +- AstroPixelProcessor's own changelog/release notes were not directly located (searches surfaced only community forum/tutorial pages, not a dated official changelog), so any 2022+ rejection-algorithm changes specific to that tool could not be confirmed either way. + +## What's new in drizzle algorithm implementations/variants since 2022 (e.g. PSF-matched drizzle kernels, alternatives to Lanczos/square-pixel drizzle, Bayer-aware/CFA-native drizzle)? + +### Takeaway +Two major amateur tools shipped substantial, dated drizzle rewrites in this window: PixInsight's "Fast Drizzle" algorithm (v1.9.0 "Lockhart," December 20, 2024) added circular, Gaussian, and variable-shape drizzle kernels at much higher speed than the classical square-kernel drizzle; Siril replaced its older "simplified drizzle" (an upscale-before-stack approximation) with a true Variable-Pixel Linear Reconstruction implementation in v1.4.0 (December 5, 2025), adding five kernel choices and full native CFA/Bayer drizzle. Separately, the STScI reference `drizzle` Python package (used by HST/JWST/Roman pipelines, and the basis several of these tools' algorithms trace back to) had several 2022–2023 releases improving numerical stability, error propagation, and square-kernel performance — this is upstream research-grade software, not an amateur tool, but it is the closest thing to a maintained "reference implementation" and shows the algorithm itself is still being refined. + +### Cited Findings +- **PixInsight 1.9.0 "Lockhart"** was released December 20, 2024 (with 1.9.1 and 1.9.2 patch releases following on December 23 and December 28, 2024) — [PixInsight 1.9.0 Lockhart Released — Cloudy Nights thread summary via search](https://www.cloudynights.com/topic/948329-pixinsight-190-lockhart-released/) +- DrizzleIntegration in PixInsight 1.9 "Lockhart" now uses the newly built "Fast Drizzle" algorithm designed by Roberto Sartori; compared to the standard square-kernel classical drizzle, Fast Drizzle "significantly speeds up the execution of drizzle integration and enables alternative drizzle kernel functions (circular, Gaussian, and variable-shape kernels) at higher speeds," and it is active by default in both the `DrizzleIntegration` tool and the `WeightedBatchPreprocessing`/`FastBatchPreprocessing` scripts — [Pixinsight 1.9 Lockhart released: Major upgrades and improvements](https://www.diyphotography.net/pixinsight-1-9-lockhart-released-major-upgrades-and-improvements/) (content retrieved via search synthesis; the article page itself could not be fully fetched to confirm verbatim wording) +- **Siril 1.4.0** was released December 5, 2025; its release notes describe a "Drizzle algorithm: implementation of the algorithm used for the Hubble Space Telescope, allowing to increase the effective resolution of images" — [Siril 1.4.0](https://siril.org/download/2025-12-05-siril-1-4-0/) +- Per a search-result synthesis of Siril community/documentation sources, prior Siril versions only had a "simplified Drizzle" that was "little more than a glorified upscale applied to each frame prior to stacking," and after "more than two years of intense development that began in 2023," the "true Drizzle algorithm" (Variable-Pixel Linear Reconstruction, i.e. the genuine HST-style algorithm) shipped as part of Siril 1.4.0 — [search synthesis citing Siril 1.4 release/community pages](https://buymeacoffee.com/deepspaceastro/new-drizzle-function-siril-1-4) +- Siril's current documentation (site reflects the 1.4.x/1.5.0 era) describes five drizzle kernels: **Square** (default, "mathematically flux preserving by construction," works at all scales/pixfracs), **Point** (lowest correlated noise, but leaves gaps without heavy dithering), **Turbo** (a simplified square-kernel approximation assuming negligible rotation, for speed), **Gaussian** (droplet modeled as a Gaussian of FWHM = pixfrac, recommended for point-source/photometry work), and **Lanczos2/Lanczos3** (restricted to same-WCS-scale resampling, i.e. scale == pixfrac == 1.0) — [Drizzle — Siril 1.4.4 documentation](https://siril.readthedocs.io/en/stable/preprocessing/drizzle.html) +- Siril's drizzle documentation also confirms full CFA/Bayer-native drizzle: "the CFA color of the current droplet sets which channel of the output image it lands on," letting users skip conventional debayering and instead drizzle the raw Bayer samples directly at scale = pixfrac = 1.0 to "restore resolution in each color channel" — [Drizzle — Siril 1.4.4 documentation](https://siril.readthedocs.io/en/stable/preprocessing/drizzle.html) +- AstroPixelProcessor markets an equivalent feature under the name "Bayer Drizzle," where only the native CFA pixels are drizzled (no debayer-before-drizzle), and its own guidance is that CFA/X-Trans droplet sizes should be roughly doubled relative to already-debayered-RGB drizzle to get comparable sharpness/noise trade-offs — [Bayer Drizzle - Tutorials & Workflows, AstroPixelProcessor community](https://www.astropixelprocessor.com/community/tutorials-workflows/bayer-drizzle/); a parallel explainer independent of any vendor is at [CFA Drizzle Stacking "Bayer Drizzle": Introduction - MyPetStars](https://mypetstars.com/tutorials/pixinsight/processes/drizzleintegration/cfa-drizzle). No dated official APP changelog entry pinning this feature to a specific 2022+ release could be located (forum/tutorial activity discussing it is from 2023 onward). +- The STScI reference `drizzle` Python package (used across HST/JWST/Roman pipelines) had the following dated releases relevant to the algorithm itself: **v2.1.0 (June 2022)** — performance work on the square-kernel path; **v2.1.1 (August 2022)** — fixed "a numerical instability in the new and old 'boxer' algorithm" (the polygon-overlap computation core to drizzle) and resolved polygon-intersection issues for nearly-collinear edges; **v2.2.0 (January 2023)** — added error and data-quality (DQ) flag propagation, including "support for co-adding with squared weights (can be used for standard error propagation)," plus bug fixes to Lanczos resampling and decoupling of previously-conflated fill-value parameters — [spacetelescope/drizzle releases, synthesized from GitHub](https://github.com/spacetelescope/drizzle/releases) +- On PSF-matched kernels generally (not dated 2022+ specifically, general background): comparative studies note the Lanczos kernel "is very good in preserving the PSF" (a Lanczos3-drizzled image measured ~5.4% sharper PSF width than square-kernel drizzling) but "introduces strong artifacts around bad pixels and cosmic rays," while a Gaussian kernel at fine pixel scale "reduces aliasing in ellipticity measurements compared to the square kernel image" — these are general drizzle-kernel trade-offs, not new since 2022, but form the technical backdrop for why PixInsight's 2024 Fast Drizzle work added exactly circular/Gaussian/variable-shape alternatives — [search synthesis, HST/JWST PSF-matching literature] +- A shipped (not experimental) 2023 amateur technique pairs drizzle with AI-based deconvolution: RC Astro's article (published April 26, 2023) describes combining drizzle's 2x upsampled detail recovery (which arrives at low contrast — drizzle-only recovered detail measured at "about 12.4%" contrast, described as "washed out to a muddy grey") with BlurXTerminator (a commercial deep-learning deconvolution tool) to restore contrast in the recovered fine detail; the article frames this as already-working production practice on real data (Iris Nebula), not a research proposal, and states it needs low noise (more/longer exposures) and well-dithered subs for drizzle's inter-pixel recovery to have real information for deconvolution to sharpen — [Drizzle plus deconvolution: a killer combination – RC Astro](https://www.rc-astro.com/drizzle-plus-deconvolution-a-killer-combination/) + +### Inferences +- The dominant 2022–2026 drizzle story in amateur tooling is **CFA/Bayer-native drizzle becoming a first-class, well-documented feature** (Siril's full rewrite, APP's established "Bayer Drizzle") rather than a new resampling mathematics — the underlying Variable-Pixel Linear Reconstruction algorithm itself is unchanged from its HST-era formulation; what changed is (a) speed (PixInsight's Fast Drizzle) and (b) kernel choice/flexibility (both PixInsight and Siril now expose non-square kernels), plus (c) doing it on raw sensor-native CFA samples instead of post-debayer RGB. +- Siril explicitly abandoning its earlier "simplified drizzle" (interpreted here as effectively pre-upsampling each frame before stacking, i.e. not true flux-conserving variable-pixel reconstruction) for the "true" algorithm in 2025 suggests that, prior to this rewrite, at least one popular open-source tool's "drizzle" was not the real HST algorithm — worth flagging for anyone benchmarking historical Siril output against a true-drizzle reference. +- The RC Astro drizzle+deconvolution pairing is best understood as an amateur-community-adopted *workflow* combining two already-shipped tools (drizzle + a commercial AI deconvolver), not a new combine/rejection algorithm in the stacking engine itself — but it is directly relevant to the "super-resolution" theme of this report since it is explicitly marketed as resolution recovery beyond what either technique achieves alone. + +### Gaps +- Exact release date/version for when AstroPixelProcessor's "Bayer Drizzle" feature was introduced or last updated could not be pinned down from available sources (no official dated changelog surfaced in search). +- Full technical detail of PixInsight's Fast Drizzle algorithm (Roberto Sartori) — e.g., whether it changes the underlying overlap-area math, uses a lookup-table approach, or is purely an implementation-level (SIMD/threading) speedup — was not available beyond the marketing-level description found; the primary PixInsight documentation/technical article describing it was not successfully fetched (only search-engine synthesis of secondary coverage was obtained). +- No source was found describing "Magic Kernel" drizzle (Costella's Magic Kernel family) being adopted in any of Siril/PixInsight/APP/DeepSkyStacker's shipped drizzle implementations — the Magic Kernel does not appear in any of the surveyed release notes or documentation for these four tools. +- DeepSkyStacker's drizzle implementation was not found to have received any specific 2022+ documented change (its 5.1.x-era release notes found in search cover Qt portability and UI theming, not drizzle algorithm work). + +## Has iterative back-projection (Irani-Peleg-style) or other super-resolution reconstruction been applied to amateur/professional astrophotography stacking since 2022? Any deep-learning super-resolution approaches for multi-frame astro stacking? + +### Takeaway +Yes, but the confirmed 2022+ work found is concentrated in research papers (astro-ph/GAN and deep-image-prior approaches for solar and star-field imaging) rather than in shipped amateur stacking software; the clearest deep-learning-based multi-frame reconstruction result is a 2025 solar-physics paper using an Instrument-to-Instrument GAN framework, and a 2025 "DIPLI" paper explicitly revives Irani-Peleg-style back-projection (fused with deep image priors and optical flow) for lucky imaging, needing only 7–13 frames versus the thousands classical lucky imaging needs. No independent (non-OriginStack) source was found describing classical Irani-Peleg IBP shipped in a mainstream amateur deep-sky stacking tool (Siril/PixInsight/APP/DSS) since 2022. + +### Cited Findings +- **DIPLI (Deep Image Prior Lucky Imaging)**, arXiv:2503.15984 (latest revision dated May 4, 2026 per the fetched page metadata), extends single-image Deep Image Prior to multi-frame "unordered frame sets" via a **back-projection loss** combined with optical-flow estimation (TVNet) for frame alignment without assuming temporal coherence, plus **Stochastic Gradient Langevin Dynamics (SGLD)** as a Bayesian regularizer replacing "fragile early-stopping heuristics" with Monte Carlo averaging, and a confidence-weighting scheme based on optical-flow reliability — [DIPLI: Deep Image Prior Lucky Imaging for Blind Astronomical Image Restoration](https://arxiv.org/html/2503.15984v3) +- DIPLI's headline comparison against classical Lucky Imaging: it requires only **7–13 frames** vs. "thousands" for classical Lucky Imaging, and reports best perceptual-fidelity scores (LPIPS best on 12/12 test scenes, DISTS best on 10/12) among compared methods; its stated limitation is that it is "limited to resolved objects with sufficient texture" — [DIPLI: Deep Image Prior Lucky Imaging for Blind Astronomical Image Restoration](https://arxiv.org/html/2503.15984v3) +- **Deep learning image burst stacking for ground-based solar observations**, arXiv:2506.04781 (dated June 5, 2025; also published in Astronomy & Astrophysics), reconstructs 100 short-exposure frames into one high-resolution image via an **unpaired image-to-image translation** approach built on the **Instrument-to-Instrument (ITI)** framework (a GAN with two generators — U-Net with skip connections — and two multi-scale discriminators, each composed of three sub-networks at different scales), trained using speckle-reconstructed observations as the target/reference domain — [Deep learning image burst stacking to reconstruct high-resolution ground-based solar observations](https://arxiv.org/html/2506.04781) / [A&A published version](https://www.aanda.org/articles/aa/full_html/2025/01/aa51850-24/aa51850-24.html) +- That same paper reports the deep-learning approach runs at real-time speed (~0.48 seconds per image burst on an NVIDIA A100 GPU) versus "computationally expensive and time-consuming" traditional speckle reconstruction, and shows "increased robustness in terms of perceptual quality, especially when speckle reconstruction methods show artifacts" — [Deep learning image burst stacking to reconstruct high-resolution ground-based solar observations](https://arxiv.org/html/2506.04781) +- **STAR** (arXiv:2507.16385, dated July 22, 2025, authors Guocheng Wu, Guohang Zhuang, Jinyang Huang, Xiang Zhang, Wanli Ouyang, Yan Lu) is a large-scale benchmark (54,738 flux-consistent image pairs from HST observations) for astronomical star-field super-resolution; it evaluates general-purpose SR networks (HAT, SwinIR, RCAN, EDSR, RealESRGAN) alongside a proposed **Flux-Invariant Super Resolution (FISR)** method that outperformed prior methods by 24.84% on a novel Flux Error (FE) metric — this is explicitly **single-image** SR (enhancing one already-combined/observed image), not multi-frame stacking, per the paper's own framing — [STAR: A Benchmark for Astronomical Star Fields Super-Resolution](https://arxiv.org/html/2507.16385v1) +- A shipped (2023, described above under drizzle) amateur-community technique pairs drizzle (which does the actual multi-frame resolution recovery) with deep-learning deconvolution (BlurXTerminator) as a two-stage super-resolution pipeline in production use — [Drizzle plus deconvolution: a killer combination – RC Astro](https://www.rc-astro.com/drizzle-plus-deconvolution-a-killer-combination/) +- General background (not dated 2022+, included for context only): a broader 2023-era literature summary found via search states deep-learning super-resolution "typically requires large labeled training datasets, which are rarely available in astrophotography" — explaining why DIPLI's approach (deep image *prior*, needing no training set, rather than a pretrained network) and the solar ITI paper's *unpaired* translation approach (also avoiding paired ground-truth training data) are both explicitly designed around that constraint — [search synthesis of deep-learning-for-astronomy SR literature] + +### Inferences +- The two strongest 2022+ deep-learning multi-frame results (DIPLI, solar ITI-GAN) both deliberately avoid needing large paired training datasets — an implicit acknowledgment that a labeled-astrophotography-frame-stack dataset large enough for supervised training does not exist, which is likely why neither is yet shipped in mainstream amateur stacking software. +- DIPLI's design (back-projection + optical flow + Bayesian/MC regularization in place of early stopping) is a direct methodological descendant of Irani-Peleg IBP, updated with modern deep-image-prior and diffusion-adjacent (SGLD) machinery — so the honest characterization is "Irani-Peleg-style back-projection has been *revived and modernized*, not merely reapplied unchanged," and this modernized version is what's actually new since 2022. +- None of the three research papers found (DIPLI, solar ITI-GAN, STAR/FISR) targets deep-sky (faint extended nebulosity/galaxy) amateur imaging specifically — DIPLI targets "resolved objects with sufficient texture" (more akin to planetary/lucky-imaging targets), the solar paper is solar-physics-specific, and STAR is HST-derived star-field imagery. This suggests deep-sky nebula/galaxy stacking specifically remains under-served by 2022+ published multi-frame deep-learning SR work relative to point-source/planetary/solar targets. + +### Gaps +- No independent, non-OriginStack source was found describing classical Irani-Peleg iterative back-projection shipped as a stacking-pipeline feature in any of Siril, PixInsight, AstroPixelProcessor, or DeepSkyStacker since 2022 — search results for this specific combination returned no hits; this appears to be either unpublished/unmarketed in those tools, or genuinely absent. (Note: a codebase in this research's own working directory implements IBP super-resolution as an opt-in feature, but per the assignment's sourcing rules this is not treated as an external, independently-verified finding and is excluded from cited findings above.) +- DIPLI's exact publication venue (peer-reviewed journal vs. arXiv-only) could not be confirmed from the fetched content. +- No source quantified how DIPLI or the solar ITI-GAN approach would perform on typical amateur equipment (small-aperture, high-noise, few-hundred-frame OSC/DSLR sessions) as opposed to the professional/large-telescope contexts each paper was evaluated in. + +## Has robust PCA (low-rank + sparse decomposition), or other advanced matrix-factorization methods, been used for calibration-frame (bias/dark/flat) master generation since 2022? + +### Takeaway +No independent (non-OriginStack) source was found describing robust PCA or low-rank+sparse matrix decomposition being applied specifically to bias/dark/flat calibration-frame master generation, in either shipped amateur tools or the astro-ph.IM literature, since 2022 or at any earlier date. The closest precedent found is low-rank+sparse decomposition applied to *science*-image sequences (not calibration frames) in high-contrast direct-imaging PSF subtraction, and that precedent predates 2022. + +### Cited Findings +- A search for "low-rank sparse decomposition dark frame denoising" returned no astronomy-calibration-frame results; the only space-adjacent hit was "Low-rank plus sparse decomposition for exoplanet detection in direct-imaging ADI sequences: The LLSG algorithm" (arXiv:1602.08381) — this applies RPCA-style low-rank+sparse+noise decomposition to angular-differential-imaging (ADI) *science* frame sequences for exoplanet/disk detection (separating a smoothly-varying stellar PSF "low-rank" component from sparse companion/disk signal), not to bias/dark/flat calibration masters, and it is a 2016 paper (predates the 2022+ window) — [Low-rank plus sparse decomposition for exoplanet detection in direct-imaging ADI sequences. The LLSG algorithm](https://arxiv.org/pdf/1602.08381) +- A separate high-contrast-imaging search likewise surfaced only classical PCA (not robust/low-rank+sparse PCA) being used to model coronagraphic PSFs for speckle subtraction, again on science frames, not calibration frames — [Astrometric and photometric accuracies in high contrast imaging: The SPHERE speckle calibration tool (SpeCal)](https://arxiv.org/pdf/1805.04854) +- Standard/mainstream calibration-frame guidance found in the search (Celestron, Optical Mechanics, general astrophotography tutorials) uniformly recommends simple robust averaging (median or sigma/kappa-sigma-clipped mean) for building bias/dark/flat masters, with no mention of PCA-based or matrix-factorization-based master generation — [The Ultimate Guide to Calibration Frames for Astrophotography – Celestron](https://www.celestron.com/blogs/knowledgebase/the-ultimate-guide-to-calibration-frames-for-astrophotography); [Calibration Frames for Deep‑Sky Astrophotography](https://www.opticalmechanics.com/calibration-frames-for-deep%E2%80%91sky-astrophotography/) +- One search result surfaced was this project's own (OriginStack) GitHub release notes describing its own `--master-method robust_pca` feature and its own auto-upgrade thresholds/performance numbers — this is explicitly the codebase this research supports, not independent third-party evidence, and is excluded from being treated as a finding about the broader field — [Release OriginStack 2.2.6 · hd152/originstack](https://github.com/hd152/originstack/releases/tag/v2.2.6) + +### Inferences +- Robust PCA / low-rank+sparse decomposition is an established technique in the *adjacent* astronomical sub-field of high-contrast direct imaging (separating a static PSF/speckle pattern from a sparse companion signal across science frames), but that adjacent use case is >8 years old (LLSG, 2016) and targets a different data type (multi-epoch science frames of the same static field) than calibration-frame master generation (many independent bias/dark/flat exposures of a featureless target). No evidence was found that anyone has published or shipped the specific idea of applying this machinery to bias/dark/flat masters other than in this project's own codebase. +- This is a genuine, well-supported gap: the mainstream calibration-frame literature and all four surveyed amateur tools' documentation describe only classical order-statistic combination (median, sigma/kappa-sigma clip) for masters, and no 2022+ paper or product changelog was found proposing or shipping anything more sophisticated for that specific step. + +### Gaps +- This is close to a total gap: no 2022+ (or any-date) independent source was found applying robust PCA / low-rank+sparse decomposition to bias/dark/flat calibration-frame stacking. If such prior art exists, it was not surfaced by the search queries used (`robust PCA calibration frames astrophotography`, `low-rank sparse decomposition dark frame denoising arxiv`), which may indicate the terminology used in any such work differs from what was searched (e.g., "matrix completion," "low-rank denoising," or a specific instrument-team internal report not indexed by general web search). +- The unverified arXiv:2607.08081 "Robust Heteroskedastic Matrix Factorization" paper noted under Key Question 1 was not read in full and its applicability (if any) to calibration-frame stacking is unknown — flagged here again as a lead for further investigation, not a finding. +- No information was found on whether any professional observatory pipeline (as opposed to amateur tools or the direct-imaging literature) uses matrix-factorization-based calibration-frame generation; this line of inquiry was not pursued given the amateur-deep-sky focus of the assignment and the tool-call budget. + +## What do major stacking tools (Siril, PixInsight, AstroPixelProcessor, DeepSkyStacker) advertise as new combine/rejection/drizzle features in 2022-2026 release notes? + +### Takeaway +PixInsight and Siril both shipped dated, substantial combine/drizzle features in this window (PixInsight: PSFSW/PSF-SNR weighting in March 2022 and Fast Drizzle in December 2024; Siril: a from-scratch true-drizzle rewrite with CFA support in December 2025). DeepSkyStacker's dated 2023 release notes found in search cover cross-platform/Qt portability and UI work rather than new combine or rejection algorithms. AstroPixelProcessor's CFA/"Bayer Drizzle" feature is well-documented and actively used by the community but no dated official changelog entry pinning a 2022+ introduction or update to it could be located. + +### Cited Findings +- **PixInsight 1.8.9** (March 13, 2022): introduced PSF Signal Weight (PSFSW) and PSF SNR as new `ImageIntegration` weighting estimators (see Key Question 1 for detail) — [PixInsight 1.8.9 Released](https://pixinsight.com/forum/index.php?threads/pixinsight-1-8-9-released.18148/); [PixInsight Reference Documentation: New Image Weighting Algorithms](https://pixinsight.com/doc/docs/ImageWeighting/ImageWeighting.html) +- **PixInsight 1.9.0 "Lockhart"** (December 20, 2024, with 1.9.1/1.9.2 patches December 23/28, 2024): shipped the Fast Drizzle algorithm (circular/Gaussian/variable-shape kernels, higher speed, on by default in `DrizzleIntegration`/`WeightedBatchPreprocessing`/`FastBatchPreprocessing`) alongside an unrelated but headline feature, MultiscaleGradientCorrection (MGC) — [PixInsight 1.9.0 Lockhart Released, community-thread summary](https://www.cloudynights.com/topic/948329-pixinsight-190-lockhart-released/); [Pixinsight 1.9 Lockhart released: Major upgrades and improvements](https://www.diyphotography.net/pixinsight-1-9-lockhart-released-major-upgrades-and-improvements/) +- **Siril 1.2.0** (September 15, 2023): per its release page, this version added no functional changes over its release candidate but included roughly 30 bug fixes, one of which was stacking-adjacent — "Fixed output_norm behavior for stacking to ignore border values (#1159)" — a bug fix, not a new algorithm — [Siril 1.2.0](https://siril.org/download/2023-09-15-siril-1.2.0/) +- **Siril 1.4.0** (December 5, 2025): headline release-notes item "Drizzle algorithm: implementation of the algorithm used for the Hubble Space Telescope, allowing to increase the effective resolution of images," representing (per community/documentation synthesis) the replacement of a long-standing "simplified drizzle" approximation with the true algorithm, developed over "more than two years" starting in 2023 — [Siril 1.4.0](https://siril.org/download/2025-12-05-siril-1-4-0/) +- **DeepSkyStacker 5.1.0** (March 2023): described in coverage as "the start of the process of converting the code to Qt so that it can be ported to platforms other than Windows" — a portability milestone, not a new combine/rejection algorithm — [DeepSkyStacker Update 5.1.0 | Astronomy Technology Today](https://astronomytechnologytoday.com/2023/03/06/deepskystacker-update-5-1-0/) +- **DeepSkyStacker 5.1.4/5.1.5** (2023): added Windows Light/Dark theme support and "many other improvements" per a forum summary, again UI-level rather than algorithmic — [DeepSkyStacker 5.1.5 is now available - Stargazers Lounge](https://stargazerslounge.com/topic/415898-deepskystacker-515-is-now-available/) +- **AstroPixelProcessor "Bayer Drizzle"**: actively documented and discussed in the tool's own community tutorials/forum (guidance on droplet-size settings, workflow steps) — [Bayer Drizzle - Tutorials & Workflows](https://www.astropixelprocessor.com/community/tutorials-workflows/bayer-drizzle/); [Drizzle settings guidance](https://www.astropixelprocessor.com/community/main-forum/drizzle-settings-guidance/) — but no dated official changelog entry for its introduction or any 2022+ revision was located in this research. + +### Inferences +- Across the four tools, the pattern in dated release notes is: PixInsight ships incremental, well-documented, dated algorithmic features on a roughly semiannual-to-annual cadence with named authorship (e.g., Roberto Sartori for Fast Drizzle); Siril made one large, multi-year (2023–2025) architectural investment specifically in fixing drizzle to be the "true" algorithm; DeepSkyStacker's 2023 public-facing changes were about codebase modernization (Qt/cross-platform) rather than new imaging algorithms, suggesting its stacking-algorithm surface has been comparatively stable since before 2022; AstroPixelProcessor's CFA drizzle capability, while real and actively used, is not well-documented in a dated, citable changelog format from what this research could locate. + +### Gaps +- No official AstroPixelProcessor changelog page was successfully retrieved in this research; all APP findings come from community forum/tutorial pages, which are undated relative to specific software version numbers. +- DeepSkyStacker's official GitHub `Releases` page was identified (`https://github.com/deepskystacker/DSS/releases`) but not fetched directly in this research pass, so more granular 2024–2026 combine/rejection-specific entries (if any exist beyond what search snippets surfaced) were not checked. +- PixInsight's own official news/changelog pages (as opposed to third-party coverage) could not be fetched directly during this research (attempts to fetch pixinsight.net articles returned only navigation/listing content, not full article bodies) — the Fast Drizzle and PSFSW descriptions above rely on third-party summaries and PixInsight's own reference documentation page (for PSFSW specifically), not a directly-fetched official release-notes article for Fast Drizzle. diff --git a/research_notes/Astrophotography stacking techniques since 2022/denoising_psf_deconvolution.md b/research_notes/Astrophotography stacking techniques since 2022/denoising_psf_deconvolution.md new file mode 100644 index 0000000..8558d55 --- /dev/null +++ b/research_notes/Astrophotography stacking techniques since 2022/denoising_psf_deconvolution.md @@ -0,0 +1,119 @@ +# Denoising, PSF Estimation, and Deconvolution Advances for Astrophotography (2022+) + +## What deep-learning-based denoising and deconvolution tools/models have emerged since 2022 specifically for astrophotography, and how do they claim to compare to classical wavelet/RL methods? + +### Takeaway +The dominant production tools are commercial/community neural-network plugins built specifically for amateur deep-sky processing — RC-Astro's BlurXTerminator (deconvolution/sharpening) and NoiseXTerminator (denoising), Seti Astro's free Cosmic Clarity suite, and GraXpert's AI denoise/deconvolution models — all of which now ship model updates or entirely new capabilities dated 2023–2025, and Siril 1.4 (in development through 2024–2025) integrates several of these rather than building its own. Vendor claims (flux conservation, artifact reduction, "outperforms classical tools") are largely marketing/changelog statements from the vendors themselves; independent, quantitative third-party benchmarking against classical wavelet/RL methods is thin, consisting mostly of qualitative visual reviews on hobbyist forums. + +### Cited Findings +- **BlurXTerminator 2.0 / AI4** (RC-Astro) was released December 14, 2023, and is the first version of the tool to process **linear image data directly** rather than applying an intermediate stretch before/after the neural network — the vendor says this eliminated flux-conservation errors, quantifying ~2–5% flux deviation for AI4 vs. ~75% flux reduction measured in AI2 on the same test — [RC Astro](https://www.rc-astro.com/blurxterminator-2-0-ai4-release/). +- AI4 was trained on an expanded range of optical aberrations (first/second-order coma and astigmatism, trefoil, defocus, field curvature, longitudinal/lateral chromatic aberration, guiding-error motion blur, per-channel seeing/scatter, 2x drizzle artifacts) and processes all color channels jointly to preserve inter-channel registration — [RC Astro](https://www.rc-astro.com/blurxterminator-2-0-ai4-release/). +- The vendor explicitly states "there is no substitute for proper equipment tuning" and that AI4 must only be run on linear data — [RC Astro](https://www.rc-astro.com/blurxterminator-2-0-ai4-release/). +- RC-Astro's BlurXTerminator, StarXTerminator and NoiseXTerminator are now also distributed as a **stand-alone CLI tool**, decoupled from PixInsight/Photoshop, and are integrated into Siril and Seti Astro Suite Pro; a "unified PixInsight RC-Astro Suite" installs the same core engine used by the CLI — [RC Astro](https://www.rc-astro.com/stand-alone-rc-astro-tools/), [RC Astro](https://www.rc-astro.com/software/). +- **GraXpert** (open-source, Steffenhir/GraXpert on GitHub) added an **AI denoising model** in v3.0.0 (April 17, 2024, redesigned UI, ONNX-format cross-platform GPU models), refined it in v3.0.2 (May 3, 2024, hardware-acceleration toggle, cancel button, prep for denoise model v2.0.0), then added an **AI object-deconvolution model** in v3.1.0rc1 (November 10, 2024, "deconvolving structures in nebulae or galaxies," stars unaffected) and an **AI stellar-deconvolution model** in v3.1.0rc2 (January 1, 2025, deconvolves stars only, objects unaffected) — [GraXpert Releases](https://github.com/Steffenhir/GraXpert/releases). +- GraXpert's two deconvolution models are explicitly split by target class (non-stellar "object" vs. "stellar") so that each network only ever modifies one kind of structure — [GraXpert Releases](https://github.com/Steffenhir/GraXpert/releases), corroborated by [Dark-Matters-Astro/graxpert-ai-models releases](https://github.com/Dark-Matters-Astro/graxpert-ai-models/releases) (Object Deconvolution model v1.0.1, compatible with GraXpert ≥ 3.1.x). +- **Siril 1.4** (beta expected early 2025 per the project's December 2024 "Twelve Days of Siril" post) integrates GraXpert's background extraction, denoising, and "the new deconvolution operations, including all the AI modes" directly into Siril's own workflow, plus a Region-of-Interest preview system for tuning compute-heavy operations like deconvolution before committing to a full-frame run — [Siril blog](https://siril.org/2024/12/the-twelve-days-of-siril/), [Siril GraXpert docs](https://siril.readthedocs.io/en/latest/processing/graxpert.html). This indicates Siril's 2024–2025 AI strategy is integration of third-party engines (GraXpert, and per workflow documentation, Seti Astro's Cosmic Clarity) rather than an in-house model — [FAS37 Siril Workflow PDF, Dec 2025](https://www.fas37.org/wp/wp-content/uploads/2025/12/Siril-Workflow-Detail.pdf). +- **Seti Astro's Cosmic Clarity** is a free, stand-alone (no PixInsight/Photoshop required) AI suite offering separate Stellar/Non-Stellar/Both sharpening (deconvolution-like), Full or Luminance-only denoising, and satellite-trail removal, with non-CUDA GPU acceleration support for Windows added later in its development — [Seti Astro](https://www.setiastro.com/cosmic-clarity), [Stargazers Lounge thread](https://stargazerslounge.com/topic/425968-seti-astro-has-just-released-a-new-stand-alone-ai-based-sharpeningdeconvolution-tool-called-cosmic-clarity/). +- Independent review (Cloudy Nights) of NoiseXTerminator found it eliminated "the worst of digital noise while retaining and enhancing faint stars" and "smoothed noise while also sharpening tiny stars," but by comparison to PixInsight's other denoising tools NXT's **default settings are more aggressive** and can produce "easily visible artifacts that create heterogeneous blobs during stretching" — [Cloudy Nights](https://www.cloudynights.com/articles/astro-gear-today/reviews/software/noise-be-gone33-testing-rc-astro-noisexterminator-r4568/). +- PixInsight's own official comparison describes its "next generation" wavelet-based noise reduction as combining the à trous wavelet transform (for smooth regions) with a multiscale median transform (for significant structures), i.e. a hybrid classical approach still actively maintained alongside the AI plugins rather than replaced by them — [PixInsight tutorial](https://pixinsight.com/tutorials/nr-comparison/). +- Comparative hobbyist reviews of star-removal tools (StarNet++ vs. StarXTerminator, both actively updated across 2022–2024) found no universal winner: StarXTerminator was rated better on large stars, galaxy-adjacent removal, and residual-artifact handling, while StarNet++ (also continuing to be updated, e.g. StarNet2) was rated better specifically on small stars, attributed to differences in each network's training data — [Evan Tsai comparison](https://www.astroimagetw.com/en/tutorials/starless-tools-comparison/), [AstroBin forum](https://www.astrobin.com/forum/c/equipment-forums/russell-croman-astrophotography-starxterminator/starnet2-vs-starx-and-the-winner-is/). + +### Inferences +- The commercial/community AI tool ecosystem (RC-Astro, GraXpert, Cosmic Clarity) is iterating faster and more visibly than the open academic literature on the *exact same problems* (denoise, deconvolve, star-separate) this repository's `src/denoising.py`, `src/psf_deconvolution.py`, and `src/star_removal.py` address — but nearly all public evidence is vendor changelogs, marketing pages, or informal forum comparisons rather than controlled quantitative benchmarks; there does not appear to be a rigorous, independently-reproduced numeric comparison (e.g., PSNR/SSIM on a common dataset) between BXT/NXT/GraXpert-AI and classical wavelet/RL methods in the sources found. +- The GraXpert "split by target class" deconvolution design (separate stellar vs. non-stellar networks) mirrors this project's own `star_removal.py` `--starless-process` philosophy of processing stars and non-stellar structure separately, and its documented finding that recombining separately-stretched layers causes visible seams is analogous to a failure mode a from-scratch implementation should watch for. +- BXT AI4's move to processing **linear** data directly (rather than stretch/process/unstretch) mirrors this project's own precedent of running its post-processing chain on stretched vs. linear data carefully (e.g., `sky_pedestal` ordering, `--transient-detect` requiring `RAWSTACK`) — flux-conservation failures from stretch-domain processing appear to be a known, recurring pitfall in this space, not unique to any one codebase. + +### Gaps +- No independent, quantitative (not just qualitative/visual) benchmark comparing BlurXTerminator/NoiseXTerminator/GraXpert-AI against classical Richardson-Lucy or wavelet denoising on a shared, objective metric (PSNR, SSIM, star FWHM, flux recovery) was found; all vendor performance numbers (e.g., BXT's 2–5% vs. 75% flux deviation) come from the vendor's own release notes, not a third party. +- Version/release history and specific technical claims for **StarXTerminator** and **NoiseXTerminator**'s own major-version changes (analogous to the BXT 2.0/AI4 writeup) were not directly retrieved beyond the "NoiseXTerminator 2/AI3" manual title found in search — the exact AI3 changelog content was not fetched. +- Topaz (Topaz Photo AI / DeNoise / Sharpen AI) astrophotography-specific claims/updates since 2022 were not directly researched in this pass (out of tool-call budget); it is named in the assignment's key questions but no dedicated source was retrieved. + +## What academic papers (2022+) address astronomical image denoising via self-supervised learning, diffusion models, or transformer architectures? + +### Takeaway +There is now a substantive 2022–2026 academic literature applying self-supervised learning (Noise2Self/Noise2Void/Self2Self/Noise2Noise-style methods), diffusion models, and transformer architectures to astronomical image denoising and deconvolution — spanning solar magnetograms, HST/JWST galaxy imaging, and radio interferometry — with the most recent (2026) work published in *Science* and *Nature Astronomy*, indicating this has moved beyond a niche technique into mainstream professional-astronomy tooling, though it remains research/pipeline-grade rather than adopted in amateur stacking software. + +### Cited Findings +- **Noise2Astro** (Sep 2022): a self-supervised neural-network denoiser for astronomical images; the method "can well recover the flux for Poisson noise and for Gaussian noise when image data has a smooth signal profile" — [arXiv:2209.07071](https://arxiv.org/abs/2209.07071). +- **"Astronomical image denoising by self-supervised deep learning and restoration processes"** (submitted Feb 24, 2025 to *Nature Astronomy*; arXiv:2502.16807), led by Tie Liu with 11 co-authors, introduces the **TDR method** (Train, Denoise, Restore): Self2Self is trained once on a representative image, then applied to denoise many similar images (useful for large surveys), followed by a restoration step that corrects pixels whose deviation exceeds a predefined threshold — addressing the stated gap that "existing deep learning methods lack quantitative control of the deviation or error on denoised images." Measured results: solar magnetogram noise reduced from ~8 Gauss to ~2 Gauss; HST galaxy images showed enhanced faint structure — [arXiv:2502.16807](https://arxiv.org/abs/2502.16807). +- **AstroSURE** (submitted April 2026; arXiv:2604.16793) is a training pipeline explicitly built to denoise astronomical images "without ground truth," adapting and comparing **Noise2Noise, Stein's Unbiased Risk Estimator (SURE), and blind-spot-based methods** (the Noise2Self/Noise2Void family) on synthetic data plus real HST and CFHT observations. Evaluated via object-detection metrics (correct-detection rate, false-alarm rate) plus image/pixel-distribution diagnostics; found the methods "improve faint-source detectability relative to the original noisy images," with "encouraging gains on HST data after domain-consistent initialization, while transfer to CFHT data is more limited" — [arXiv:2604.16793](https://arxiv.org/abs/2604.16793). +- **ASTERIS** ("astronomical self-supervised transformer-based denoising algorithm," Guo, Zhang, Li, et al., submitted Feb 19 2026, revised Apr 30 2026, published in *Science*) is a self-supervised **transformer** that integrates spatiotemporal information across multiple exposures (leveraging correlated noise structure across adjacent pixels and exposures) rather than denoising single frames independently. Reported results: **1.0-magnitude improvement in detection limit at 90% completeness/purity**, preserved PSF and photometric accuracy, discovery of previously undetectable low-surface-brightness galaxy structures and gravitationally-lensed arcs in JWST/Subaru data, and **3× more high-redshift (z > 9) galaxy candidates** detected, ~1.0 mag fainter in rest-frame UV, than prior methods — [arXiv:2602.17205 abstract](https://arxiv.org/abs/2602.17205). +- **Diffusion models for deconvolution**: "Diffusion Models for Probabilistic Deconvolution of Galaxy Images" (Xue, Li, Patel, Regier; ICML 2023 Workshop on ML for Astrophysics, July 20, 2023; arXiv:2307.11122) proposes a **classifier-free conditional diffusion model** for PSF deconvolution of galaxy images, framing deconvolution as sampling from a posterior over possible deblurred images rather than producing one point estimate. Claimed advantage over classical deep generative baselines: it "captures a greater diversity of possible deconvolutions compared to a conditional VAE," addressing VAEs'/GANs' "inadequate sample diversity" — [arXiv:2307.11122](https://arxiv.org/abs/2307.11122). +- **"Bayesian Deconvolution of Astronomical Images with Diffusion Models: Quantifying Prior-Driven Features in Reconstructions"** (Nov 2024; arXiv:2411.19158) applies score-based diffusion models trained on high-resolution cosmological simulations, via Diffusion Posterior Sampling (DPS), to treat astronomical deconvolution as a Bayesian inverse problem, with an explicit focus on quantifying which reconstructed features are driven by the learned prior vs. the actual data — [arXiv:2411.19158](https://arxiv.org/html/2411.19158v2). +- Radio astronomy: Wang et al. (2023) "demonstrated for the first time that DDPM can be used for image reconstruction for radio astronomical images," trained on simulated Event Horizon Telescope-style data, cited within "Radio-Astronomical Image Reconstruction with Conditional Denoising Diffusion Model" (2024) — [arXiv:2402.10204](https://arxiv.org/html/2402.10204v2). +- **IRIS** (Jan 2025; arXiv:2501.02473) is a Bayesian radio-interferometric image-reconstruction approach using expressive score-based (diffusion) priors as "plug-and-play" priors — [arXiv:2501.02473](https://arxiv.org/pdf/2501.02473). +- **Transformer-based galaxy restoration**: "Deeper, Sharper, Faster: Application of Efficient Transformer to Galaxy Image Restoration" (Park, Jo, Kang, Kim, Jee; *The Astrophysical Journal* 972, published Aug 23, 2024) adapts **Restormer** (Zamir et al. 2022's linear-complexity, feature-space-attention efficient Transformer) to simultaneously **denoise and deconvolve**, restoring HST-quality images toward JWST quality. Trained first on synthetic GalSim (Sérsic-profile) galaxies, then fine-tuned on real JWST data. Quantitative results: isophotal photometry scatter reduced 4.4×, Sérsic index scatter reduced 3.6×, half-light radius scatter reduced 4.7×, Pearson correlation with ground truth "approaching unity," and combined synthetic+real training gave an SSIM improvement of ~4.67% over training on JWST data alone — [IOPscience](https://iopscience.iop.org/article/10.3847/1538-4357/ad5954). +- A 2023 pedagogical review, "Transformers for scientific data: a pedagogical review for astronomers" (arXiv:2310.12069), and a 2022 review "Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy" (arXiv:2211.03796) both situate this transformer/diffusion adoption within a broader 2022+ shift toward these architectures across astronomical ML generally, without being denoising-specific papers themselves — [arXiv:2310.12069](https://arxiv.org/pdf/2310.12069), [arXiv:2211.03796](https://arxiv.org/pdf/2211.03796). +- A British Astronomical Association article by Scott Sloka, "Image denoising in astrophotography — an approach using recent network denoising models" (published Aug 10, 2023), specifically targets amateur astrophotography: it evaluates CNN denoisers (DnCNN, VDNet, FFDNet cited) and transformer-based architectures against **BM3D** as the classical baseline, states that "implementations of CNNs have been shown to outperform state-of-the-art methods such as BM3D, WNNM, and TNRD," and — notably for amateur applicability — trains/tests using **stacked amateur astronomical images rather than idealized ground truth**, finding the approach practically viable without perfect reference data — [BAA Journal](https://britastro.org/journal_contents_ite/image-denoising-in-astrophotography-an-approach-using-recent-network-denoising-models). + +### Inferences +- The self-supervised/diffusion/transformer literature has bifurcated: one branch (Noise2Astro, TDR/Self2Self, AstroSURE, ASTERIS) targets professional survey/telescope pipelines (solar magnetograms, HST/JWST/Subaru, CFHT) where no ground truth exists and detection-limit gains matter most; the other (Restormer galaxy restoration, diffusion deconvolution) targets simultaneous deconvolution+denoising as a joint inverse problem. Neither branch is currently packaged as amateur-facing software — the amateur-facing tools (Q1) remain proprietary/opaque networks (BXT/NXT/GraXpert-AI) whose training methodology is not published academically, so there is a real gap between the rigorously-published self-supervised/diffusion methods and what amateur stacking pipelines actually ship. +- ASTERIS (Science, 2026) and the TDR method (Nature Astronomy, 2025) both being published in top-tier general-science/astronomy journals in the same ~18-month window suggests self-supervised astronomical denoising has crossed a credibility threshold since 2022 that it previously lacked (Noise2Astro in 2022 was a more modest, single-author-team arXiv-only contribution by comparison). +- Diffusion-based deconvolution's core claimed advantage (posterior diversity vs. deterministic point estimates) is conceptually the closest academic analogue to this project's own `src/uncertainty.py` Monte Carlo uncertainty propagation — both are, in different ways, responses to the fact that a single deterministic restored image cannot represent reconstruction uncertainty. + +### Gaps +- No paper was found that specifically validates diffusion-model deconvolution or transformer denoising against **ground-based amateur one-shot-color (OSC) DSLR/CMOS deep-sky data** (as opposed to professional survey/space-telescope imagery) — all the strongest quantitative diffusion/transformer results (ASTERIS, Restormer) are on JWST/HST/Subaru/CFHT data, and it is unclear how well these transfer to amateur SNR regimes and Bayer-sensor artifacts. +- The full text of the AstroSURE and ASTERIS arXiv PDFs could not be fully retrieved (size/format limits); only abstract-level summaries were obtained, so architecture-level detail (e.g., exact network depth, training compute) is not confirmed here beyond what the abstracts state. + +## What's new in Richardson-Lucy deconvolution variants (spatially-variant PSF, GPU acceleration, regularization) since 2022? + +### Takeaway +The clearest, best-documented 2022+ academic advance found is a spatially-variant Richardson-Lucy (RL_sv) formulation applied to Chandra X-ray imaging of Cassiopeia A (2023), which also contributes a principled iteration-count stopping rule and per-pixel uncertainty propagation — methodologically relevant to this project's own `richardson_lucy_svpsf` even though the application domain (X-ray) differs from optical deep-sky imaging; beyond that, most of the visible RL-adjacent innovation for amateur astrophotography is happening inside the closed-source AI tools covered in Q1 rather than in openly-published classical-RL research. + +### Cited Findings +- **"Richardson–Lucy Deconvolution with a Spatially Variant Point-spread Function of Chandra: Supernova Remnant Cassiopeia A as an Example"** (Sakai, Yamada, Sato et al.; submitted arXiv June 23, 2023 as arXiv:2306.13355; published *The Astrophysical Journal* 951, 59, July 2023) develops "RL_sv," which incorporates the PSF's positional dependence directly into the RL update (rather than assuming one global PSF), applied to Chandra observations of Cassiopeia A — [arXiv:2306.13355](https://arxiv.org/abs/2306.13355). +- The paper's iteration-count selection is **statistically principled**: it predicts the "appropriate number of iterations by using statistical fluctuation of the observed images," rather than a fixed or manually-tuned iteration count — [arXiv:2306.13355](https://arxiv.org/abs/2306.13355). +- The method also propagates **uncertainty estimates** via "error propagation from the last iteration," phenomenologically validated against the observed data — giving RL_sv output pixels an associated error bar, not just a point estimate — [arXiv:2306.13355](https://arxiv.org/abs/2306.13355). +- Applying RL_sv "enables us to uncover the smeared features in the forward/backward shocks and jet-like structures" of Cas A that a spatially-invariant PSF assumption would blur together — [arXiv:2306.13355](https://arxiv.org/abs/2306.13355). +- Diffusion-based deconvolution (Q2 above) is explicitly framed by its authors as an alternative to classical iterative methods like Richardson-Lucy for capturing a *distribution* over plausible deconvolutions rather than one MAP/ML estimate — [arXiv:2307.11122](https://arxiv.org/abs/2307.11122). +- The Restormer-based galaxy restoration paper (Q2) folds deconvolution into a joint learned denoise+deconvolve transformer rather than an explicit RL iteration, and reports it "outperformed" single-dataset training baselines on photometric-scatter metrics — [IOPscience](https://iopscience.iop.org/article/10.3847/1538-4357/ad5954) — though this is a supervised transformer replacing RL's *whole role*, not an RL variant per se. + +### Inferences +- The Cas A RL_sv paper's two contributions — (1) a data-driven stopping criterion based on statistical fluctuation, and (2) formal per-pixel error propagation from the last RL iteration — are both generically applicable improvements to *any* iterative RL implementation (including an optical spatially-variant RL like this project's `richardson_lucy_svpsf`), independent of the X-ray-specific application; a from-scratch adoption would need to re-derive the fluctuation-based stopping rule for optical CCD/CMOS Poisson+Gaussian noise rather than X-ray photon-counting statistics, since the noise model differs. +- No 2022+ paper was found proposing a fundamentally new *regularization* scheme for RL (e.g., a new TV or wavelet-domain prior) distinct from what already existed pre-2022 (TV-regularized RL, entropy-regularized RL) — the visible 2022+ RL-adjacent innovation is concentrated on (a) spatial-variance handling (Cas A paper) and (b) wholesale replacement by learned/diffusion models (Q2), not incremental regularization improvements to classical RL itself. +- GPU acceleration of RL specifically for amateur astrophotography was not found as a distinct published advance in this research pass; this project's own README already documents its RL implementation running via cupy FFT on GPU as a project-internal fact, not something traced to an external 2022+ publication. + +### Gaps +- No paper specifically addressing GPU-accelerated Richardson-Lucy for optical (not X-ray) deep-sky amateur imaging since 2022 was located; it's possible this is discussed only in vendor documentation (e.g. PixInsight's own RL/dynamic PSF tools) rather than the academic literature, which this pass did not check due to tool-call budget. +- No dedicated 2022+ paper on **blind deconvolution** (jointly estimating the PSF and the deconvolved image, as opposed to a pre-estimated/measured PSF) for amateur deep-sky data was found; searches surfaced only an older (2018) semi-blind microscopy CNN-PSF-estimation paper as tangentially related prior art, not a 2022+ astro-specific instance. +- PixInsight's built-in classical deconvolution tool (non-AI) and any 2022+ changelog improvements to it (e.g., updated regularization, "Dynamic PSF" tool changes) were not directly researched in this pass. + +## Are there new approaches to curvelet/shearlet/directional wavelet denoising with structure protection published since 2022? + +### Takeaway +Search results turned up essentially no astronomy-specific 2022+ literature on curvelet/shearlet/directional-wavelet denoising with structure protection; the active 2022+ research on curvelet/shearlet-domain denoising with structure protection is concentrated in **adjacent remote-sensing/geophysics domains** (hyperspectral imagery, seismic data), not astronomical imaging — the astronomy-specific curvelet/starlet denoising literature the search surfaced (e.g., curvelet transform for astronomical image representation) predates 2022 and was not found to have a clearly-dated 2022+ successor. + +### Cited Findings +- A 2024 paper in the *International Journal of Remote Sensing* on "Multiscale reweighted smoothing regularization in curvelet domain for hyperspectral image denoising" reports improvements of "at least 1.5 dB in PSNR and reducing ERGAS error by more than 35%" versus prior state-of-the-art — but this is a **hyperspectral remote-sensing** application, not astronomical imaging — [Tandfonline](https://www.tandfonline.com/doi/abs/10.1080/01431161.2024.2357836). +- A 2024 paper (*Frontiers in Earth Science*) combines the **non-subsampled shearlet transform (NSST)** with an improved FFDNet for **seismic random-noise suppression**, finding NSST "preserves more edge details while suppressing high frequency noise" compared with wavelet, curvelet, and contourlet transforms — again a geophysics, not astronomy, application, but methodologically relevant as a 2022+ instance of combining a directional multiscale transform with a learned denoiser for structure protection — [Frontiers](https://www.frontiersin.org/journals/earth-science/articles/10.3389/feart.2024.1408317/full). +- Pre-2022 astronomy-specific curvelet work (e.g., "Astronomical image representation by the curvelet transform," "Astronomical image denoising using curvelet and starlet transform") was found in search results but carries no 2022+ update or successor paper in the results retrieved — [A&A](https://www.aanda.org/articles/aa/full/2003/05/aa2574/aa2574.right.html), [IEEE Xplore](https://ieeexplore.ieee.org/document/6530927). +- General background: shearlets are noted as better than traditional wavelets at representing distributed discontinuities (edges), which is the generic rationale for using them for structure-preserving denoising — but the specific citation found for this claim is not dated 2022+ — [search summary, unattributed general claim]. + +### Inferences +- The absence of astronomy-specific 2022+ curvelet/shearlet papers, combined with the presence of active 2022+ curvelet/shearlet work in adjacent imaging domains (hyperspectral, seismic), suggests that recent astronomical directional-multiscale denoising research effort has shifted toward the deep-learning/self-supervised/diffusion/transformer methods covered in Q2 instead of continuing to refine classical curvelet/shearlet transforms — i.e., the field's attention moved on rather than curvelet/shearlet methods being disproven or displaced by a documented head-to-head result. +- This project's own `directional_wavelet_denoise` (structure-tensor-coherence-gated BayesShrink, described as "curvelet/shearlet-*inspired*, not an actual ridgelet/shearlet transform") is therefore not obviously behind any specific *published* 2022+ astronomical curvelet/shearlet advance, because no such advance was found — the comparison class Claude Code should worry about is the deep-learning denoisers (Q1/Q2), not a missed classical-transform improvement. + +### Gaps +- This is the weakest-covered key question: no astronomy-specific, 2022-or-later curvelet/shearlet/directional-wavelet paper with an explicit structure-protection mechanism was located despite multiple query variations. It's possible such work exists in venues not well-indexed by general web search (e.g., SPIE proceedings, national radio-astronomy technical reports) that a deeper literature-database search (ADS, arXiv astro-ph.IM direct browse) might surface; this pass relied on general web search rather than a direct arXiv category listing search, which is a methodological limitation worth flagging to the report writer. + +## What new techniques exist for star removal/repair, saturated-core repair, or halo/star reduction since 2022? + +### Takeaway +Star removal has become a mature, actively-competing neural-network product category since 2022 (StarXTerminator vs. continually-updated StarNet2, plus Cosmic Clarity's stellar/non-stellar split), while saturated-core repair remains dominated by pre-2022 PixInsight scripting techniques (Repaired HSV Separation) with only incremental, tool-specific 2022+ refinements (e.g., StarNet2 highlight-protection fixes) rather than a new published algorithm. + +### Cited Findings +- **StarXTerminator** (debuted Sept 2021, so just pre-2022, but under continuous update through the assignment's window) and **StarNet++/StarNet2** remain the two dominant star-removal tools, with head-to-head hobbyist comparisons finding no absolute winner: StarXTerminator does better "at filling in the space where a star was removed" and performs "better on star removal around galaxies and on large stars, as well as in handling of residual artifacts," while StarNet++ "does better on small stars," attributed to differences in training data — [Evan Tsai](https://www.astroimagetw.com/en/tutorials/starless-tools-comparison/), [AstroBin](https://www.astrobin.com/forum/c/equipment-forums/russell-croman-astrophotography-starxterminator/starnet2-vs-starx-and-the-winner-is/). +- **StarNet2** (an actively updated successor line, per its own release notes) shipped a fix specifically described as improving "highlight protection in very bright image regions" and fixing "unscreen star-layer artifacts in saturated or near-saturated regions" — i.e., a direct, dated (within the tool's release-notes page) improvement to saturated-star handling — [StarNet release notes](https://starnetastro.com/release-notes/). +- **Seti Astro Cosmic Clarity** (free, stand-alone, ongoing development) offers separate Stellar / Non-Stellar / Both sharpening modes plus satellite-trail removal, explicitly separating star-halo/sharpening treatment from background/nebulosity treatment — the same "treat stars and non-stellar structure as separate problems" design philosophy seen in GraXpert's split deconvolution models (Q1) — [Seti Astro](https://www.setiastro.com/cosmic-clarity). +- For saturated star cores specifically, the dominant documented technique remains PixInsight's pre-existing **"Repaired HSV Separation"** script workflow — apply it to a clone of the *linear* image, setting "Clip Shadows" and "Repair" levels, then use MaskedStretch (which itself protects saturated cores from further brightening as periphery brightness increases) — this is presented as an established (not newly-2022+) technique in the source retrieved, with no clear 2022+ dated revision found — [PixInsight.com.ar](https://pixinsight.com.ar/fr/processing-examples/maskedstretch-stars-sores-28.html). +- **StarTools**' "Dark Anomaly Filter" (within its Develop module) is offered as an alternative saturated-core-handling tool, though user reports note limitations combining it with masks during development — [StarTools forum](https://forum.startools.org/viewtopic.php?f=10&t=886). +- BlurXTerminator AI4 (Q1) is documented to have specifically improved dense/crowded star-field handling (Omega Centauri test case) "without artifact generation," which is a form of star-repair-adjacent improvement (better resolving/reconstructing stars in crowded fields) dated to its Dec 2023 release — [RC Astro](https://www.rc-astro.com/blurxterminator-2-0-ai4-release/). + +### Inferences +- Saturated-core repair appears to be the least-innovated sub-area within this whole research assignment: unlike star removal (a genuine multi-vendor AI arms race since 2022) or denoising/deconvolution (both academic diffusion/transformer papers and commercial AI tools), saturated-core repair since 2022 looks like incremental patching of existing (pre-2022) classical or tool-specific techniques rather than a new algorithm or published method — this is consistent with this project's own `src/star_repair.py` approach (a from-scratch Moffat-wing-fit refill) not having an obvious 2022+ external technique it should be benchmarked against beyond the general AI star/deconvolution tools in Q1. +- The recurring "split stars from non-stellar structure into separate processing paths" pattern (GraXpert's two deconvolution models, Cosmic Clarity's Stellar/Non-Stellar sharpening modes, this project's own `--starless-process`) suggests this is close to an industry-wide convergent design choice for AI-era astrophotography tools since 2022, not something unique to any one implementation. + +### Gaps +- No comparative, quantitative (as opposed to qualitative forum-review) benchmark of star-removal quality (e.g., residual flux left behind, background reconstruction fidelity) across StarXTerminator/StarNet2/Cosmic Clarity was found. +- No 2022+ **academic** (arXiv/journal) paper specifically on saturated-stellar-core reconstruction or star-halo reduction was located — everything found in this sub-area is vendor/tool documentation and hobbyist forum discussion, not peer-reviewed research; this itself is worth reporting as a finding (the gap is real, not a search failure) given how much peer-reviewed material exists for the adjacent denoising/deconvolution questions. +- **Chroma-noise / color-artifact reduction specific to one-shot-color (Bayer) sensors** (an item in the assignment's broader "Cover" list) turned up no dedicated 2022+ source: general Bayer-artifact/debayering background pages were found, but no specific 2022+ paper or tool feature addressing OSC-specific chroma-noise reduction as a distinct topic from general RGB denoising. The closest tool-level evidence is NoiseXTerminator's stated ability to reduce "both colored chrominance noise and gritty luminance noise" ([Cloudy Nights](https://www.cloudynights.com/articles/astro-gear-today/reviews/software/noise-be-gone33-testing-rc-astro-noisexterminator-r4568/)), but this is a general chroma-denoising claim, not documented as OSC/Bayer-specific. diff --git a/research_notes/Astrophotography stacking techniques since 2022/ml_deep_learning_stacking.md b/research_notes/Astrophotography stacking techniques since 2022/ml_deep_learning_stacking.md new file mode 100644 index 0000000..94bf80d --- /dev/null +++ b/research_notes/Astrophotography stacking techniques since 2022/ml_deep_learning_stacking.md @@ -0,0 +1,131 @@ +# ML/Deep Learning Approaches Applied to the Astrophotography Stacking/Calibration Pipeline (2022+) + +## What published/deployed systems use ML to score individual sub-frame quality or classify defects (satellite trails, clouds, tracking errors, coma, etc.) since 2022? + +### Takeaway +Real deployed ML frame-quality/defect systems exist mainly on the *professional survey* side (GWAC's autoencoder quality screen, all-sky-camera cloud classifiers) and as narrow academic defect detectors (satellite-trail segmentation networks); on the *amateur* side the only concrete, citable example is a small 2023 AutoML/ResNet50 image-quality-assessment study on Stellina smart-telescope stacks, not a per-subframe rejection system integrated into mainstream amateur stacking software (Siril, PixInsight, APP, DeepSkyStacker all still gate on classical FWHM/eccentricity/background metrics as far as documented). + +### Cited Findings +- Parisot, Bruneau & Hitzelberger, "Astronomical Images Quality Assessment with Automated Machine Learning" (submitted arXiv 17 Nov 2023, accepted at DATA2024): applies Image Quality Assessment (IQA) + AutoML to *stacked* live images from a Stellina electronically-assisted-astronomy (EAA) telescope, built from 10-second sub-frames over 20–30 minute integrations; a ResNet50 IQA model splits each high-resolution image into 256×256 RGB patches and averages per-patch ratings into an overall score, and was evaluated across different observational setups to see whether the rating tracked equipment/quality differences — [arXiv:2311.10617](https://arxiv.org/pdf/2311.10617) +- The GWAC (Ground-based Wide Angle Camera) time-domain survey pipeline uses a self-supervised **autoencoder** for frame quality screening (attributed to Jia et al., 2024, in a 2025 preprocessing review): the encoder learns features of high-quality images, and a larger image-reconstruction error at inference flags a low-quality frame; described as successfully screening low-quality images "with acceptable accuracy" and integrated into the GWAC operational/prototype pipeline — [arXiv:2502.10783](https://arxiv.org/html/2502.10783v1) +- All-sky-camera cloud/precipitation classification for observatory safety: a ResNet-based classifier reached ~85% accuracy detecting clouds but needed heavy compute, while a gradient-boosted-tree model (LightGBM) on extracted image features reached ~95% accuracy with only ~1,000 training images and modest compute — [Cloud Identification from All-sky Camera Data with Machine Learning](https://arxiv.org/abs/2003.11109) (note: base paper is 2020, pre-2022; the comparative ResNet-vs-LightGBM figures surfaced in search summaries citing continued/updated work — treat the exact publication date of the 85%/95% comparison as unconfirmed; see Gaps) +- Newer (2025) deep-learning work specifically on precipitation-cloud identification from all-sky camera data for observatory safety exists in ScienceDirect's *Astronomy and Computing* — [Deep learning-based identification of precipitation clouds from all-sky camera data for observatory safety](https://www.sciencedirect.com/science/article/pii/S2666827025000234) +- Satellite-trail defect detection (a direct analogue of this pipeline's own `trail_reject.py`) has multiple 2024–2025 deep-learning papers: a U-Net + Hough-transform hybrid ("ASTA", *A&A*, published Dec 2024, arXiv submitted Jul 2024) — [arXiv:2407.19461](https://arxiv.org/abs/2407.19461); a U-Net + Line Segment Detector hybrid (Sept 2025) — [arXiv:2509.16771](https://arxiv.org/html/2509.16771); an ASA-U-Net variant adding atrous spatial pyramid pooling + channel attention for sparse trail features; a multi-band photometric deep-learning trail identifier (Sept 2025) — [arXiv:2509.04081](https://arxiv.org/pdf/2509.04081); and a YOLOv12-based detector for satellite/debris trails in all-sky imagery +- O'TRAIN (real/bogus point-source classifier, CNN, *A&A* published 2022 — Aug 2022 issue, arXiv submitted Dec 2021) reports 93–98% real/bogus classification accuracy across four different telescopes and two different transient-detection pipelines, i.e., it generalizes across pixel scales/observing conditions rather than being telescope-specific — [A&A 664, A81 (2022)](https://www.aanda.org/articles/aa/full_html/2022/08/aa42952-21/aa42952-21.html); [arXiv:2112.10280](https://arxiv.org/abs/2112.10280) + +### Inferences +- The professional/survey and amateur worlds are on different tracks: survey pipelines (GWAC, ZTF-adjacent real/bogus work, all-sky cameras) have deployed learned quality/defect classifiers because they operate at a scale (thousands–millions of frames/alerts nightly) where manual/heuristic screening doesn't scale; amateur stacking tools have not needed to cross that threshold (a session is dozens to low hundreds of subs) and still rely on classical per-frame statistics (FWHM, eccentricity, background level, star count) — consistent with this project's own `quality.py`/`frame_processor.py` design. +- The Stellina/AutoML IQA paper (2311.10617) is evidence that amateur/EAA-adjacent quality scoring is at the *proof-of-concept* stage (evaluating whether a generic ResNet50 IQA model tracks quality across setups) rather than a shipped subframe-rejection feature. + +### Gaps +- No source found describing a **shipped amateur stacking tool** (Siril, PixInsight, DeepSkyStacker, Astro Pixel Processor, Nebulosity) that uses a trained ML model — as opposed to classical statistics — to score or reject individual light-frames for stacking. Searches on Siril's and APP's documentation surfaced only classical quality/weighting metrics (star-based Gaussian PSF fitting, quality score from registration, sigma-clip rejection), not neural quality scoring. +- Could not confirm the exact publication date/venue for the specific "85% ResNet / 95% LightGBM" cloud-classification comparison figures beyond the aggregator summary; the only directly verifiable arXiv record for that topic (2003.11109) predates the 2022 cutoff, so it is cited for context only, not as an in-scope (2022+) result. +- Could not locate the primary bibliographic record for "Jia et al. 2024" (the GWAC autoencoder quality paper) independently of the review article that cites it as reference [39]; only the secondary description from the review (arXiv:2502.10783) is confirmed. +- No evidence found of an ML classifier specifically for **tracking-error** or **coma** defect classification (as distinct from the star-shape/eccentricity heuristics already common); this may be an unaddressed niche versus satellite-trail and cloud detection, which are betterrepresented. + +## Has anyone built or published a neural-network-based frame combination/stacking method (learned weighting or learned fusion of multiple exposures) for astronomical imaging since 2022, in academic literature or commercial tools? + +### Takeaway +Yes, at least one strong, recent, peer-reviewed example exists — ASTERIS, a transformer-based self-supervised network published in *Science* (2026) that jointly processes multiple raw exposures (not a post-stack denoiser) and outperforms classical stacking+denoising for faint-source detection on JWST/Subaru data — but this is a research-grade professional-survey system, not a commercial/amateur tool; no evidence was found of a shipped amateur product that replaces classical mean/median/sigma-clip/ivw combination with a trained neural combiner. + +### Cited Findings +- **ASTERIS** ("Deeper detection limits in astronomical imaging using self-supervised spatiotemporal denoising"), Guo, Zhang, Li, Yu, Wu, Hao, Huang, Liang, Lin, Li, Wu, Cai & Dai — submitted to arXiv 19 Feb 2026, revised 30 Apr 2026, published in *Science* — is described as "an astronomical self-supervised transformer-based denoising algorithm" that "integrates spatiotemporal information across multiple exposures," processing multiple exposures **jointly** (i.e., operating across the frame stack, not on an already-combined image) by exploiting correlated noise structure across frames — [arXiv:2602.17205](https://arxiv.org/pdf/2602.17205); [Science DOI](https://www.science.org/doi/10.1126/science.ady9404) +- ASTERIS reported results: detection-limit improvement of **1.0 magnitude at 90% completeness and purity**, preserving PSF shape and photometric accuracy; on deep JWST imaging it recovered **3× more redshift > 9 galaxy candidate** than prior methods, at rest-frame UV luminosity 1.0 mag fainter than previously detectable; validated on JWST and Subaru data plus mock/simulated benchmarks — [arXiv:2602.17205](https://arxiv.org/pdf/2602.17205) +- Separately, a solar-physics deep-learning "image burst stacking" method reconstructs high-resolution ground-based solar observations from bursts of frames using an encoder–decoder / recurrent architecture with a multi-input design and a convolutional block attention module (CBAM) to focus on salient spatial/channel features when combining multiple frames — published in *Astronomy & Astrophysics*, 2025 (arXiv version June 2025) — [A&A 2025](https://www.aanda.org/articles/aa/full_html/2025/01/aa51850-24/aa51850-24.html); [arXiv:2506.04781](https://arxiv.org/html/2506.04781) +- By contrast, the Vera C. Rubin Observatory (LSST) production coaddition pipeline, as of its Data Preview 1 documentation, still combines exposures with a **classical inverse-variance-weighted mean** for deep coadds — i.e., the flagship next-generation professional survey has not (as of the DP1-era documentation reviewed) replaced its core stacking/coaddition statistic with a learned combiner — [Rubin DP1: Deep coadd](https://dp1.lsst.io/products/images/deep_coadd.html) + +### Inferences +- The one clear "neural combination of multiple raw exposures" result (ASTERIS) is framed and evaluated as a *denoising/detection-limit* problem (recovering faint sources), not as a general-purpose replacement for classical mean/median/sigma-clip stacking metrics (noise validation, drizzle, rejection of outliers/satellite trails) — it complements rather than obsoletes classical combination logic, and the published work does not claim to handle cosmic-ray/satellite-trail/outlier rejection the way sigma-clip or ESD combination does. +- No commercial amateur tool (Siril, PixInsight, DeepSkyStacker, Astro Pixel Processor, ASTAP) appears to have shipped a trained neural network as its core combine/stacking statistic; this remains, as far as the search surfaced, exclusively a professional-research-pipeline capability as of the 2026 publication date, and even there it is a single (very recent) instance rather than an established practice. + +### Gaps +- No evidence of any commercial/amateur astrophotography product marketing "AI stacking" or "neural combine" as of this research (2026-09); searches for "learned"/"neural" frame combination/coadd weighting in 2024–2025 mostly returned generic reviews or unrelated ML-in-astronomy surveys, not a second concrete learned-combination system to cross-validate ASTERIS's approach. +- ASTERIS's paper details (loss function specifics, exact network depth/parameter count, training-data provenance beyond "JWST and Subaru plus mock data") could not be extracted from the fetched excerpt; only high-level architecture/results language was available via the tool's summarization, not the full paper text — a report author wanting exact architecture diagrams/hyperparameters should treat this as unconfirmed and consult the primary Science/arXiv PDF directly. +- Because ASTERIS was submitted Feb 2026 (very recent relative to this research's Sept 2026 date), independent replications, citations, or critical commentary were not found — it should be flagged as a single-source, very new finding pending community validation. + +## What ML approaches exist for automatic sky-object/target classification to drive per-target processing parameter selection (auto-advisors) in astro tools since 2022? + +### Takeaway +No evidence was found of a shipped or published system that uses a trained ML classifier to drive *per-target automatic processing-parameter selection* in an astrophotography stacking/processing tool since 2022 — published ML target/object classification work in this window (nebula/galaxy/planetary-nebula classifiers, star–galaxy separation, low-surface-brightness-galaxy-vs-artifact classifiers) targets scientific cataloguing and survey artifact rejection, not driving a consumer tool's stacking/stretch/denoise parameter choices; auto-advisor features in amateur tools (where documented) appear to remain heuristic/rule-based. + +### Cited Findings +- Deep transfer learning (ImageNet-pretrained backbone) was applied to classify true planetary nebulae vs. other object types, reported as achieving high classification success "even without any parameter tuning" (originally an MDPI 2019/2021-adjacent line of work; most recent arXiv revision touching this remains pre-2022 in its core publication, so it is noted for context rather than as an in-scope 2022+ primary result) — [arXiv:2101.12628](https://arxiv.org/pdf/2101.12628) +- DeepShadows (arXiv, low surface brightness galaxies vs. imaging artifacts, deep learning classifier) — separates genuine low-surface-brightness galaxy detections from artifacts, i.e., a defect/genuine-object classifier for a *survey* pipeline rather than a per-target parameter advisor for a stacking tool — [arXiv:2011.12437](https://arxiv.org/pdf/2011.12437) (note: base submission predates 2022; included only as the closest match found to "classify then decide what to do with the object," not as evidence of a 2022+ deployed system) +- Broad-survey engagement with AutoML in astronomy is confirmed generically ("AutoML has already been used in astronomy... applications to galaxy morphological classification and other astronomical tasks") but no specific 2022+ system tying an ML target classification output to *stacking/processing parameter selection* (e.g., automatically choosing background-extraction method, stretch curve, or denoiser strength based on a learned target-type classification) was located in the sources retrieved. + +### Inferences +- This project's own `src/auto_settings.py` design — an explicit, documented heuristic (inverse-distance-weighted blending across hand-defined target-type anchor points, `_classify()` for a single printed label, SIMBAD/header-based prior-type lookup) rather than a trained classifier — appears consistent with the state of the art found in the wider ecosystem: nothing found in this research suggests any competing amateur tool has moved this decision to a learned model instead of heuristics/metadata lookup (SIMBAD, catalog names, header keywords). +- Where ML target/object classification research does exist since 2022, it is oriented toward large-survey science goals (identifying real astrophysical sources, morphological classification for catalogs) rather than the amateur workflow problem of "what processing recipe should I apply to this personally-imaged nebula/galaxy." + +### Gaps +- This is the weakest-evidenced key question of the assignment: despite multiple query phrasings ("deep learning target classification astrophotography auto parameter selection," "AutoML astronomy classification"), no primary source describing an ML-driven per-target parameter advisor (the functional equivalent of `--auto` in this codebase, but ML-based) was found, in either academic literature or commercial tool documentation, published 2022 or later. This should be reported to the writer as a confirmed absence/gap rather than guessed at. +- It remains possible such a feature exists in a commercial tool's marketing copy that wasn't surfaced by search (e.g., inside PixInsight's WBPP, ImageSolver, or a plugin ecosystem) — this was not confirmed either way and should be treated as unresolved, not negative evidence beyond what was searched. + +## What's new in ML-based transient detection / difference imaging (beyond classical ZOGY) since 2022, in both professional survey pipelines (e.g. Rubin/LSST, ZTF) and amateur tools? + +### Takeaway +Professional survey pipelines have shipped real ML systems since 2022 — most notably ZTF's BTSbot (deployed Sept 2023) for automated real/bogus-and-classification triage of difference-imaging alerts, plus a family of CNN/ViT real-bogus classifiers (ATLAS, TransientViT/KATS, DES, Mini-SiTian) — and Rubin/LSST is explicitly building its whole 10-year, ~10-million-alert/night operation around Difference Image Analysis with ML-based alert brokers/classifiers downstream; on the amateur side, the closest analogue found is O'TRAIN (2022, generalizes across four telescopes), not a widely shipped consumer feature. No 2022+ work was found proposing to *replace* ZOGY's statistical formalism itself with a learned differencing method for the amateur/this-pipeline use case — the ML layer sits **downstream** of classical (ZOGY/Alard-Lupton/HOTPANTS-style) differencing, filtering its candidate list. + +### Cited Findings +- **BTSbot** (ZTF Bright Transient Survey III), deployed 2023-09-29: a multimodal CNN (image triplets of science/reference/difference cutouts at 63×63×3, fused with a 25-feature metadata branch) trained on 608,943 alerts from 19,558 sources; achieves 94.1%±0.28% test accuracy, ROC-AUC 0.984±0.001; in production use Dec 2023–May 2024 it autonomously saved 609 sources with 96.1%/96.2% confirmed as genuine extragalactic transients, and produced the first fully automatic end-to-end transient discovery-and-classification, SN 2023tyk — [ApJ 2024](https://iopscience.iop.org/article/10.3847/1538-4357/ad5666) +- ATLAS survey published a 2024 retrained CNN for real/bogus classification (peer-reviewed in *RAS Techniques and Instruments*), tested on 27,174 human-verified real transients and 20,000 recent bogus objects, finding it "maintains competitive performance while maintaining adaptability to new data trends" — [RASTI 2024](https://academic.oup.com/rasti/article/3/1/385/7713043) +- **TransientViT** (Sept 2023) — a hybrid CNN–Vision-Transformer real/bogus classifier for the Kilodegree Automatic Transient Survey (KATS), reporting AUC 0.97 and accuracy 99.44% — [arXiv:2309.09937](https://arxiv.org/abs/2309.09937) +- Dark Energy Survey applied CNNs to identify transients (2022 arXiv submission) — [arXiv:2203.09908](https://arxiv.org/pdf/2203.09908) +- Mini-SiTian Array published a deep-learning real/bogus classifier (2025) — [arXiv:2504.01608](https://arxiv.org/pdf/2504.01608) +- Rubin/LSST: full operations expected to begin late 2025, streaming up to **10 million transient alerts nightly**, with alerts generated via Difference Image Analysis comparing new observations to deep reference templates; Data Preview 1 (commissioning data, Nov 2023–Dec 2024) work on identifying/photometrically classifying extragalactic transients has been published — [arXiv:2507.22864](https://arxiv.org/pdf/2507.22864); pipeline emphasis on ML/broker-based early transient-bogus discrimination "to suppress spurious detections" at LSST's data scale is discussed in — [arXiv:2512.11959](https://arxiv.org/pdf/2512.11959) +- A DECam-based effort explicitly aims to enable earlier transient discovery ahead of/alongside LSST via difference imaging (2025) — [arXiv:2507.22156](https://arxiv.org/html/2507.22156v2) +- The amateur-adjacent, telescope-agnostic O'TRAIN CNN classifier (2022, *A&A*) achieves 93–98% real/bogus accuracy across four different telescopes/two different pipelines, making it the most plausible existing building block for an amateur-scale ML transient triage layer, though it is a research tool (requires human-labeled training cutouts) rather than a packaged amateur-software feature — [A&A 664, A81](https://www.aanda.org/articles/aa/full_html/2022/08/aa42952-21/aa42952-21.html) +- A 2025 paper on transfer learning explicitly bridges ZTF-trained real/bogus models to LSST-style data and to small-field optical survey telescopes, i.e., addressing the domain-gap problem for reusing survey-trained classifiers on lower-resource instruments — [Transfer learning for transient classification: from simulations to real data and ZTF to LSST](https://www.researchgate.net/publication/393507894_Transfer_learning_for_transient_classification_from_simulations_to_real_data_and_ZTF_to_LSST); [Transfer learning for transient search with small-field optical survey telescopes, arXiv:2606.15705](https://arxiv.org/pdf/2606.15705) +- Active/semi-supervised learning has been proposed (published *A&A*, Jan 2025) to reduce the human-labeling burden for real/bogus training sets — [A&A 2025](https://www.aanda.org/articles/aa/full_html/2025/01/aa48581-23/aa48581-23.html) +- A 2026 paper on human-label-free, interpretable deep learning for real/bogus classification with uncertainty quantification pushes further toward removing manual labeling entirely — [arXiv:2607.05393](https://arxiv.org/pdf/2607.05393) + +### Inferences +- The dominant 2022+ ML trend in transient detection is **not** a replacement for ZOGY/Alard-Lupton-style differencing itself, but a learned *post-hoc triage/classification* layer (real/bogus + source classification) sitting on top of classical difference-imaging output, driven by the sheer alert volume of modern surveys (ZTF, and especially the ~10M/night scale anticipated at Rubin/LSST) — this matches this project's own `difference_imaging.py` docstring characterization of ZOGY as the differencing formalism, with no mention of an ML layer, suggesting the pipeline currently has no analogue to BTSbot/O'TRAIN-style candidate triage. +- Architecture trend within this space: CNN-only (2022–2023, ATLAS/DES/BTSbot) giving way to CNN-ViT hybrids (TransientViT, 2023) and then to reducing labeling cost (active/semi-supervised 2025, "human-label-free" 2026) as the frontier concern, rather than further differencing-algorithm changes. +- Given O'TRAIN's demonstrated cross-telescope generalization (93–98% across 4 telescopes), a real/bogus CNN triage layer is plausibly transferable to an amateur single-target pipeline's `--transient-detect` output, but no one has published doing so for an amateur-stacking-software context specifically. + +### Gaps +- No source found describing an amateur astrophotography tool (as opposed to a research group's custom pipeline) shipping an ML-based transient/candidate classifier as a packaged feature since 2022. +- Could not confirm whether any of the survey pipelines (ZTF, ATLAS, Rubin) have modified the *differencing* mathematics itself (i.e., a learned alternative to ZOGY's cross-convolution formalism) rather than only adding a downstream classifier; all evidence found points to ML being applied after classical differencing, which should be reported as the state of the art rather than assumed to be a temporary gap. + +## Are there published vision-transformer or CNN classifiers specifically trained on amateur deep-sky astrophotography frames (for quality/category/defect scoring) since 2022, and what architectures/datasets do they use? + +### Takeaway +Evidence is thin and mostly indirect: the clearest match is the 2023 ResNet50 AutoML/IQA study trained on Stellina (a consumer/EAA smart-telescope) live-stack outputs, and a hybrid CNN–ViT real/bogus classifier (TransientViT) exists for a robotic *survey* telescope (KATS) rather than amateur imaging; a dedicated smartphone-astrophotography dataset (MobilTelesco) surfaced but its exact ML task/architecture could not be confirmed from available summaries. No large, well-known, purpose-built "amateur deep-sky frame quality/category/defect" CNN/ViT benchmark akin to ImageNet-for-astrophotography was found. + +### Cited Findings +- Parisot, Bruneau & Hitzelberger (Nov 2023): ResNet50-based IQA model applied in 256×256-patch mode to live-stacked outputs from a Stellina smart telescope (a consumer EAA device), the closest confirmed match to "CNN trained on amateur deep-sky frames for quality scoring" in the 2022+ window — [arXiv:2311.10617](https://arxiv.org/pdf/2311.10617) +- **MobilTelesco**, described as "a smartphone based astrophotography dataset," surfaced in search results as directly relevant to amateur astrophotography imagery, but the tool could not retrieve architecture/task specifics (classification target, model type, publication venue/date) beyond the dataset's existence — flagged in Gaps below, not to be treated as a confirmed finding beyond "a dataset with this name exists." +- **TransientViT** (Sept 2023) is a genuine CNN-ViT hybrid for astronomical frame classification, but it is trained on KATS (a robotic wide-field *survey* instrument's) difference-image cutouts, not amateur deep-sky imaging, and its task is real/bogus transient discrimination, not general frame quality/category/defect scoring — [arXiv:2309.09937](https://arxiv.org/abs/2309.09937) +- Stellar-ViT: a Vision Transformer applied to SDSS photometric images for stellar spectral classification (2024, MDPI *Universe*) is a ViT applied to real astronomical imagery, but SDSS is a professional survey, not amateur astrophotography, and the task is stellar classification, not frame quality/defect scoring — [doi:10.3390/universe10050214](https://doi.org/10.3390/universe10050214) +- A benchmarking paper explicitly compares deep-learning object-detection models on a "feature deficient astrophotography imagery dataset" (2025) — potentially relevant to amateur-style low-SNR data but the tool's summary did not surface the specific architectures/results in enough detail to cite beyond the title — [arXiv:2508.06537](https://arxiv.org/html/2508.06537v1) + +### Inferences +- Given how central `--originvision` already is to this project (a bespoke, from-scratch multi-task CNN — 7 trained heads incl. reject/quality/category/exposure/sky_brightness — trained specifically on amateur deep-sky frames and shipped inside the product), and how little independent published work turned up matching that exact profile, it's plausible this project's `originvision` model is *unusually far ahead* of publicly documented academic/commercial work specifically targeting "quality/category/defect scoring trained on amateur deep-sky astrophotography frames" — the closest published analogues (Stellina IQA study) are narrower in scope (single IQA score, not a 7-task multi-head classifier) and still at the exploratory-study stage rather than a shipped, versioned model. +- This suggests a real gap in the public literature that a write-up of `originvision`'s own approach (multi-task CNN, no-SSL, epoch-selected checkpoint, ONNX/Rust deployment) would be filling rather than following if ever published/described externally. + +### Gaps +- Could not confirm architecture, training set size, or classification targets for MobilTelesco beyond its existence as "a smartphone-based astrophotography dataset" — worth a dedicated follow-up search/fetch of the primary source if the report needs it. +- No purpose-built, named "amateur deep-sky quality/category/defect" CNN or ViT benchmark (with public dataset) comparable in scope to `originvision`'s 7-task design was found; this should be reported as an apparent gap in the public literature, not assumed away. +- Could not verify whether any hobbyist/enthusiast community project (e.g., a GitHub repo or forum tool) trains such a classifier outside formal publication — this research was scoped to arXiv/academic + major commercial-tool documentation per the assignment's suggested sources, so informal/forum-only projects may exist unindexed by these searches. + +## What ML-based background extraction / gradient removal tools exist (e.g. GraXpert's AI background extraction) and how do they claim to compare to classical mesh/DBE methods? + +### Takeaway +GraXpert is the clear, concrete, shipped example: since v2.0 it added an AI (CNN/U-Net-style) background-extraction mode requiring no user-selected sample points, later (v3.0.x/3.1.0-rc) extended to AI denoising and stellar/object deconvolution, and it has been integrated as an interoperable tool inside both Siril and PixInsight workflows; however, no rigorously quantified, apples-to-apples published benchmark of GraXpert's AI mode vs. classical mesh/DBE-style background extraction (in the way this project documents its own DBE improvements) was found — comparisons found were qualitative/community-sourced ("markedly improved," "significantly enhances the final results") rather than measured. + +### Cited Findings +- GraXpert 2.0 introduced single-click AI-based background extraction that "does not require any user input" (no manual sample-point placement, unlike classical DBE-style tools), described as based on an AI model that "significantly enhances the final results" versus the prior non-AI approach — [AstroWorldCreations, GraXpert 2.0 blog](https://www.astroworldcreations.com/blog/graxpert-20-single-click-background-extraction-right-within-pixinsight); [GraXpert GitHub](https://github.com/Steffenhir/GraXpert/) +- Mechanism as documented in Siril's own integration docs: GraXpert's AI mode computes a background model of the image; a user-adjustable "smoothing strength" (0.0–1.0) applies a Gaussian blur to that background model before subtracting it from the image (default) or dividing by it; the result is renormalized — [Siril docs, GraXpert Interface](https://siril.readthedocs.io/en/latest/processing/graxpert.html) +- GraXpert's AI models are user-selectable by task: "background-extraction" (default) or "denoising"; deconvolution models, and the denoising/background-extraction models, have been available "since version 3.0.0" (stellar and object deconvolution added by v3.1.0-rc per community documentation) — [PyPI graxpert](https://pypi.org/project/graxpert/); [GitHub releases](https://github.com/Steffenhir/GraXpert/releases) +- During an earlier beta phase, GraXpert crowdsourced "difficult" user-submitted images to further train/improve the AI model, with community reports describing performance as "quite impressive and markedly improved over earlier versions" — [starfieldview.com, GraXpert AI-Powered Background Extraction](http://starfieldview.com/imaging-and-processing/graxpert-ai-powered-background-extraction/); [The Suffolk Sky, background extraction/noise reduction using AI for free](http://www.suffolksky.com/2024/04/25/background-extraction-and-noise-reduction-using-ai-for-free/) +- General background: CNN-based pixel-level foreground/background separation models (of the type GraXpert is built on) are described generically as supporting "high accuracy in complex scenes" — this is a generic ML-vision framing rather than a GraXpert-specific measured claim, and should be read as background context, not a GraXpert benchmark + +### Inferences +- GraXpert's positioning is explicitly against the manual-sample-point-based classical background-extraction workflow common to PixInsight's DynamicBackgroundExtraction and Siril's own background extraction (the "no user input needed" framing is the comparison point), rather than against a specific automated statistical method like this project's own DBE/mesh approach — i.e., GraXpert's claimed advantage is workflow automation (no manual point placement, no per-image tuning beyond one smoothing slider) more than a claimed *numerical accuracy* superiority over automated classical methods. +- The lack of any rigorous published quantitative benchmark (e.g., synthetic-gradient recovery error, edge-artifact measurements, or corner-gradient residuals of the kind this project measures for its own DBE work) suggests GraXpert's AI background-extraction claims rest on qualitative community consensus and vendor description rather than peer-reviewed or even blog-published head-to-head numeric comparison against classical mesh/DBE methods — a report should flag this as an evidence-quality gap rather than repeat "AI background extraction is better" as an established fact. + +### Gaps +- No quantitative, methodical benchmark (error metrics, residual-gradient measurements, or a controlled synthetic test) comparing GraXpert's AI mode against classical DBE/mesh background extraction was found in any source retrieved — all comparative language found was qualitative/anecdotal from blogs and community documentation, not a paper or vendor-published benchmark with numbers. +- GraXpert's underlying architecture (confirmed to be a trained neural network, generically consistent with U-Net-style encoder–decoder segmentation designs used for background/foreground separation) could not be confirmed at the level of an official architecture diagram, parameter count, or training-set description from the sources retrieved — the U-Net characterization here is inferred from general background-extraction ML literature, not a GraXpert-specific technical disclosure, and should be presented with that caveat. +- No other named ML-based background-extraction/gradient-removal *tool* (beyond GraXpert) was found in this research; PixInsight and Siril's own native tools remain classical (DBE/ABE-style, or this project's own mesh/DBE) as far as documented. If other tools exist, they were not surfaced by the queries run. diff --git a/research_notes/Astrophotography stacking techniques since 2022/registration_alignment.md b/research_notes/Astrophotography stacking techniques since 2022/registration_alignment.md new file mode 100644 index 0000000..071c930 --- /dev/null +++ b/research_notes/Astrophotography stacking techniques since 2022/registration_alignment.md @@ -0,0 +1,123 @@ +# Astrophotography Stacking: Registration/Alignment Advances Since 2022 + +## What new star detection/matching algorithms or sub-pixel correlation methods (phase correlation variants, deep-learning star detectors) have appeared since 2022? + +### Takeaway +Production stacking tools have not adopted deep-learning star detectors; the concrete, dated 2022+ advances are classical/algorithmic — Siril's new KOMBAT shift-registration algorithm (2023) and its refactored "2-pass" global registration (2023) — while deep-learning centroiding and CNN star detection remain confined to academic papers on space-telescope (HST) and star-tracker imagery, not amateur deep-sky stacking pipelines. + +### Cited Findings +- Siril 1.2.0-beta1 (released 2023-02-24) implemented a new "KOMBAT" algorithm for registration and removed the deprecated ECC (Enhanced Correlation Coefficient) method; KOMBAT is a fast, selection-based, shift-only registration method, primarily intended for planetary alignment but usable on deep-sky images too, faster than the DFT method but less robust when patterns differ significantly between channels — [Siril 1.2.0-beta1 ChangeLog](https://gitlab.com/free-astro/siril/-/raw/1.2.0-beta1/ChangeLog); [Siril registration docs](https://siril.readthedocs.io/en/1.2/preprocessing/registration.html) +- The same Siril 1.2 release refactored global registration into new "2pass" and "Apply Existing" methods, consolidated 1-star and 2/3-star registration into one unified approach, refactored 3-star registration to run star analysis sequentially, and added the ability to specify a maximum number of stars used for registration — [Siril 1.2.0-beta1 ChangeLog](https://gitlab.com/free-astro/siril/-/raw/1.2.0-beta1/ChangeLog) +- Siril's global star alignment (as described in the peer-reviewed JOSS paper on Siril, submitted 2024-08-02) matches common stars between frames via a triangle-similarity method, then computes a linear transform with RANSAC-based outlier rejection to build the projection matrix — [arXiv:2408.03346](https://arxiv.org/abs/2408.03346) (published in Journal of Open Source Software, 2024) +- "Star-Image Centering with Deep Learning" (Paper I, arXiv:2303.03346, submitted March 2023) and its follow-up "Star-Image Centering with Deep Learning II: HST/WFPC2 Full Field of View" (arXiv:2404.16995, submitted April 2024, published in PASP) developed a deep-learning centroiding model for HST/WFPC2 images trained on >600 dithered exposures of globular cluster 47 Tuc in two filters; the DL model corrects for PSF variation across the full chip and nonlinear magnitude/charge-transfer-efficiency effects, and eliminates "pixel-phase bias" that a classic centroiding algorithm exhibits at up to ~40 milli-pixels amplitude — [arXiv:2404.16995](https://arxiv.org/html/2404.16995) +- "Real-Time Convolutional Neural Network-Based Star Detection and Centroiding Method for CubeSat Star Tracker" (arXiv:2404.19108, 2024) applies a CNN to star detection/centroiding for spacecraft star-tracker attitude determination, not deep-sky astrophotography stacking — [arXiv:2404.19108](https://arxiv.org/html/2404.19108v1) +- "Enhancing astrometric registration of Chinese historical Astronomical Digital Plates with deep learning" (arXiv:2604.04714, submitted 2026) uses a Swin Transformer vision-transformer architecture to automate registration of digitized historical photographic plates to celestial coordinates; explicitly targets archival plate imagery, not modern CCD/CMOS amateur deep-sky data — [arXiv:2604.04714](https://arxiv.org/pdf/2604.04714) +- Python's `astroalign` library (originally described in a pre-2022 paper, arXiv:1909.02946) — which matches 3-point asterisms/triangles between frames rather than using WCS — continued to receive updates in the 2022+ window: v2.4.2 (2023-02-21), v2.5.0 and v2.5.1 (both 2023-10-14), v2.6.0 (2024-10-13), v2.6.1 (2024-11-15) — [astroalign PyPI](https://pypi.org/project/astroalign/); [astroalign docs](https://astroalign.quatrope.org/) + +### Inferences +- The dominant "new" 2022+ registration algorithm actually shipped in a mainstream tool is Siril's KOMBAT (2023) — a fast shift-only matcher — layered onto an unchanged triangle-similarity + RANSAC star-matching core that predates 2022. +- Deep-learning star centroiding is real and published (2023-2024), but every located instance targets a specialized, high-precision context (HST WFPC2 photometry, spacecraft star trackers, historical plate digitization) rather than amateur multi-frame deep-sky stacking, where classical matched-filter/Gaussian-fit centroiding remains standard. +- astroalign's continued 2023-2024 release cadence suggests incremental maintenance/compatibility fixes rather than a new registration algorithm; no changelog detail on algorithmic changes was found (see Gaps). + +### Gaps +- No changelog/release-notes detail was retrievable for what specifically changed algorithmically in astroalign v2.5.0/2.5.1/2.6.0/2.6.1 (2023-2024) versus bug fixes or API changes — the underlying asterism-matching algorithm itself appears unchanged since the original 2020 paper. +- No evidence was found of any production astrophotography stacking tool (Siril, PixInsight, APP, DeepSkyStacker, ASTAP) shipping a deep-learning-based star detector or matcher as of this research; this should be treated as an absence of evidence within the time available, not a confirmed universal negative. +- Phase-correlation-variant advances specific to amateur stacking (e.g., new sub-pixel-refinement schemes) were not found in the 2022+ window beyond what is already documented in this project's own codebase notes; no external 2022+ published phase-correlation innovation was located. + +--- + +## What advances exist in modeling optical distortion (radial, elastic/local displacement fields) for stacking, and how do real tools handle field rotation on alt-az mounts? + +### Takeaway +The two flagship desktop tools each shipped a materially new distortion model in the 2022+ window — PixInsight's DDM (Domain Decomposition Method) thin-plate-spline/radial-basis-function engine (Dec 2024) and Siril's up-to-5th-order polynomial astrometric distortion correction (previewed Dec 2024 for the 1.4 line) — both explicitly aimed at mosaics and wide/dithered fields; field rotation itself is handled everywhere by fitting a rotation-inclusive transform (homography/affine/rigid) per frame rather than by any dedicated "de-rotation" algorithm, and true optical de-rotation on alt-az mounts is still primarily a hardware (mechanical field de-rotator) problem outside the stacking software's scope. + +### Cited Findings +- PixInsight 1.9.0 "Lockhart" (released 2024-12-20) reimplemented its astrometry engine with a spline-based approach using DDM (Domain Decomposition Method) radial basis functions and surface splines, supporting up to 25,000 control points/reference stars, and the ImageSolver script now defaults to 4,000 points with DDM thin-plate splines for faster, higher-accuracy solves — [DIYPhotography coverage](https://www.diyphotography.net/pixinsight-1-9-1-lockhart-released-major-upgrades-and-improvements/); [PixInsight forum](https://pixinsight.com/forum/index.php?threads%2Fpixinsight-1-9-3-lockhart-released.25260%2F=) +- The same DDM thin-plate-spline implementation was carried into StarAlignment itself, "resulting in a significant improvement in its ability to model arbitrary distortions during image registration" — [DIYPhotography / PixInsight 1.9.0 coverage](https://www.diyphotography.net/pixinsight-1-9-1-lockhart-released-major-upgrades-and-improvements/) +- PixInsight's underlying thin-plate-spline distortion-correction machinery in StarAlignment (originally introduced mid-2013, per the tool's own tutorial) uses a "successive approximations" scheme: an initial projective model from matched stars is iteratively refined by fitting thin-plate splines and re-validating matches with RANSAC at progressively tighter tolerances; post-2013 versions use *approximating* (not interpolating) surface splines with a smoothness parameter (default 0.25) to avoid overfitting star-position measurement noise. This is the mechanism the 2024 DDM work extends, not replaces — [PixInsight distortion tutorial](https://www.pixinsight.com/tutorials/sa-distortion/index.html) (note: base algorithm pre-2022; the DDM extension is dated Dec 2024, see above) +- PixInsight also added (per forum-documented, non-dated-precisely 2023-era release notes) a "local sparsity-based adaptive outlier rejection" algorithm for StarAlignment with tunable Rejection radius/lower-limit/sigma parameters, and automatically falls back to **rigid transformations** (3-star-pair minimum) instead of full projective transforms (6-star-pair minimum) when fewer than 6 star-pair matches are found, explicitly to make RANSAC more robust "in difficult image registration cases" and "large proportions of outliers" — [PixInsight 1.8.9-2 forum notes](https://pixinsight.com/forum/index.php?threads/pixinsight-1-8-9-2-build-1593-released.22195/) +- Siril's documentation states that "since version 1.3, Siril can account for distortions for some of the registration methods" — [Siril registration docs, 1.4.x](https://siril.readthedocs.io/en/stable/preprocessing/registration.html) +- Siril 1.4 (previewed 2024-12, beta released 2025-04-26) introduces "astrometric registration with up to fifth order polynomial distortion correction," explicitly to let frames "with significantly different center points" (i.e., mosaic panels or widely dithered/multi-night frames) register accurately, and to improve star-position correspondence feeding into PCC/SPCC (photometric colour calibration) — [Siril "Twelve Days of Siril" post, 2024-12](https://siril.org/2024/12/the-twelve-days-of-siril/); [Siril 1.4.0 Beta 1 announcement](https://siril.org/download/2025-04-26-siril-1-4-0-beta1/) +- Siril 1.4 also debuted **mosaic stacking** ("the single most-requested feature"), built directly on the new astrometric registration, aligning full-mosaic frames and using edge feathering to minimize panel-blend artifacts — [Siril "Twelve Days of Siril," 2024-12](https://siril.org/2024/12/the-twelve-days-of-siril/) +- Siril's default global-registration transform is **homography** (an 8-degree-of-freedom transform, needing ≥4 matched star pairs), explicitly recommended for wide-field images and described as capable of warping images onto the reference frame including rotation — i.e., field rotation between frames is handled as one term of the fitted homography, not a separate step — [Siril registration docs](https://siril.readthedocs.io/en/latest/preprocessing/registration.html) +- On hardware/mechanical field rotation for alt-az mounts specifically: a genuine field de-rotator requires an off-axis guider and physically counter-rotates the camera; without one, "field derotation will not work," per discussion referencing amateur alt-az imaging practice — [Cloudy Nights: Field Rotation - Alt Az tracking](https://www.cloudynights.com/topic/764538-field-rotation-alt-az-tracking-software-to-de-rotate/); [GitHub: A Field DeRotator for Alt-Az Telescopes](https://github.com/cytan299/field_derotator) +- For planetary/lucky-imaging alt-az work (a related but distinct regime from deep-sky stacking), AutoStakkert can compensate for field rotation given latitude, the target's alt/az, and recording duration, and WinJUPOS/Registax can correct rotation via derotation stacking of many short sub-sequences — [Cloudy Nights field-rotation software thread](https://www.cloudynights.com/topic/764538-field-rotation-alt-az-tracking-software-to-de-rotate/) +- "Non rigid geometric distortions correction – Application to atmospheric turbulence stabilization" (Yu Mao & Jerome Gilles, arXiv:2411.01788, submitted 2024-11-04) models atmospheric-turbulence-induced *local* geometric distortion and corrects it with an elastic registration algorithm based on diffeomorphic mappings — a general imaging paper, not astronomy-specific, but directly applicable to the "local displacement field" style of distortion correction — [arXiv:2411.01788](https://arxiv.org/abs/2411.01788) + +### Inferences +- Both major closed/open desktop tools converged on the same mathematical family (thin-plate/radial-basis spline local-displacement models) for distortion correction, and both shipped their major upgrade to it in the same window (PixInsight Dec 2024; Siril previewed Dec 2024 / beta April 2025) — suggesting the amateur/prosumer field converged on splines as the practical answer to per-panel/per-session optical + field distortion around the same time, likely responding to the same driving use case (mosaics and multi-session/dithered wide-field registration). +- No tool was found to model field rotation on alt-az mounts as a distinct physical/geometric phenomenon requiring special handling in the *registration* transform; it is absorbed as the rotational component of whatever multi-DOF transform (homography, affine, rigid) each frame already gets fit against the reference. True field de-rotation for alt-az mounts remains predominantly a hardware concern (mechanical derotator + off-axis guider) for deep-sky work, distinct from planetary/lucky-imaging software derotation (AutoStakkert/WinJUPOS), which operates in a different regime (short sub-sequences, known ephemeris-derived rotation rate). + +### Gaps +- No published academic paper specifically modeling alt-az field rotation *within a deep-sky stacking registration pipeline* (as opposed to planetary lucky-imaging derotation) was found; this appears to be handled purely as a byproduct of general affine/homography fitting in every tool surveyed, with no dedicated literature. +- Exact release date for PixInsight's "local sparsity-based adaptive outlier rejection" / rigid-transformation-fallback RANSAC robustness improvements could not be pinned down precisely (the source discusses version 1.8.9-2, whose date was not confirmed as 2022+ in the available snippet) — flagged as a possible pre-2022 feature; treat with caution and verify before citing as a "new since 2022" item. + +--- + +## Are there published methods (papers, blog posts, changelogs) for cross-session/multi-night registration with unknown rotation? + +### Takeaway +The concrete, dated 2022+ mechanism for cross-session/multi-night, unknown-rotation registration in production tooling is Siril 1.4's mosaic-stacking + up-to-5th-order polynomial astrometric registration (previewed Dec 2024, beta April 2025), which explicitly targets frames "with significantly different center points"; AstroPixelProcessor separately announced a wholesale rewrite of its registration engine (beta39, Nov 2025) aimed at multi-panel mosaics; no arXiv paper specifically addressing amateur multi-night deep-sky registration under unknown field rotation (as distinct from single-session RANSAC/homography fitting) was located. + +### Cited Findings +- Siril 1.4's new astrometric registration (WCS/plate-solve-based, up to 5th-order polynomial distortion) is the feature explicitly cited as enabling frames "with significantly different center points" — i.e., different pointings/sessions/nights — to register accurately against each other, and it underlies the new mosaic-stacking feature ("edge feathering to minimize blending artifacts") — [Siril "Twelve Days of Siril," 2024-12](https://siril.org/2024/12/the-twelve-days-of-siril/) +- AstroPixelProcessor 2.0.0-beta39 (released 2025-11-28) shipped "a new and much improved registration engine," and the subsequent beta46 (2026-06-24) and forthcoming beta47 notes specifically call out mosaic registration speed as a target, stating "mosaics can be even 10x faster than before because registration received a major boost compared to beta46" — [AstroPixelProcessor release notes](https://www.astropixelprocessor.com/) ; [APP release-information forum](https://www.astropixelprocessor.com/community/release-information/) +- `astroalign`'s asterism (3-point-triangle) matching approach is rotation- and scale-agnostic by construction (it does not rely on WCS and estimates the affine transform directly from matched triangles), making it a candidate building block for unknown-rotation cross-night registration, though this project's own codebase notes (not this research) already independently implement an equivalent blind rigid-match approach; astroalign's own continued 2023-2024 releases (see above) did not describe any rotation-handling algorithm change — [astroalign docs](https://astroalign.quatrope.org/) +- Siril's WCS-based astrometric registration only became distortion-aware "since version 1.3" (per Siril's own docs) and gained the up-to-5th-order polynomial model in the 1.4 line (beta, 2025) — before that, cross-session registration in Siril relied on the same star-triangle+RANSAC global registration used for single-session stacking, without an explicit "unknown rotation" framing in available documentation — [Siril registration docs](https://siril.readthedocs.io/en/stable/preprocessing/registration.html) + +### Inferences +- The industry's practical answer to "cross-session/multi-night registration with unknown rotation" in the 2022+ window has been to route it through **plate-solving / WCS astrometric registration** (Siril 1.4) rather than through a specialized blind-rotation star-matching algorithm published as its own technique — i.e., the rotation is resolved by solving each frame's absolute sky orientation independently (astrometry), then registering in a common WCS frame, rather than by matching frame-to-frame without external calibration. +- No production tool's changelog in the surveyed window described a *published, named* algorithm for "blind" (non-WCS) unknown-rotation cross-session matching; this appears to be treated as an internal implementation detail (e.g., "improved registration engine" in APP) rather than a documented, citable method. + +### Gaps +- No arXiv or conference paper specifically titled or focused on "multi-night" or "cross-session" astronomical image registration under unknown rotation (for amateur deep-sky stacking) was found in this search pass; it is possible such work exists in the SPIE/software-conference literature suggested by the assignment but was not surfaced by the queries run (time/tool-call budget did not allow direct SPIE Digital Library search). +- Exact technical description of APP's beta39 "new and much improved registration engine" (algorithm family, whether it changed from feature-matching to a different approach) was not available beyond the changelog's plain announcement — AstroPixelProcessor does not appear to publish detailed algorithmic changelogs comparable to Siril's. +- DeepSkyStacker's 2023-2024 releases (5.1.0 March 2023 through 5.1.6 June 2024) showed no evidence of new cross-session/mosaic registration capability — the located changes were UI/porting-focused (Qt port, "blink" frame comparison, 64-bit-only support), suggesting DSS has not modernized its registration algorithm in this window; this is an absence-of-evidence finding, not a confirmed lack of any change, since a full DSS GitHub release-notes review was not completed within budget. + +--- + +## Has anyone applied machine learning to star matching, transform estimation, or sub-pixel registration in astrophotography specifically? + +### Takeaway +Yes, but only in specialized academic/professional contexts (HST photometric centroiding, CubeSat star trackers, historical plate digitization) — no evidence was found of machine learning being applied to star matching, transform estimation, or sub-pixel registration inside any mainstream amateur deep-sky stacking tool (Siril, PixInsight, APP, DeepSkyStacker, ASTAP) as of this research. + +### Cited Findings +- "Star-Image Centering with Deep Learning" I & II (arXiv:2303.03346, March 2023; arXiv:2404.16995, April 2024, published in PASP) train a deep-learning model on HST/WFPC2 dithered exposures to predict sub-pixel star centers, explicitly correcting PSF-variation-across-chip and magnitude-dependent (charge-transfer-efficiency) biases that classical centroiding does not model, and eliminate a pixel-phase bias of up to ~40 milli-pixels seen with classic algorithms — [arXiv:2404.16995](https://arxiv.org/html/2404.16995) +- "Real-Time Convolutional Neural Network-Based Star Detection and Centroiding Method for CubeSat Star Tracker" (arXiv:2404.19108, 2024) is a CNN applied to star detection and centroiding for spacecraft attitude determination — a form of "star matching" adjacent to astrophotography but in the star-tracker/attitude-determination domain — [arXiv:2404.19108](https://arxiv.org/html/2404.19108v1) +- "Enhancing astrometric registration of Chinese historical Astronomical Digital Plates with deep learning" (arXiv:2604.04714, 2026) applies a Swin Transformer to automate registration (effectively transform estimation) of historical photographic plates to celestial coordinates, replacing manual/semi-manual registration — [arXiv:2604.04714](https://arxiv.org/pdf/2604.04714) +- No changelog, forum announcement, or paper was found describing ML-based star matching, transform estimation, or sub-pixel registration being integrated into Siril, PixInsight, AstroPixelProcessor, DeepSkyStacker, or ASTAP; all documented improvements in these tools (KOMBAT, DDM splines, RANSAC robustness tuning, database/solver speed) are classical/geometric algorithms — [Siril ChangeLog](https://gitlab.com/free-astro/siril/-/raw/1.2.0-beta1/ChangeLog); [PixInsight coverage](https://www.diyphotography.net/pixinsight-1-9-1-lockhart-released-major-upgrades-and-improvements/); [ASTAP history](https://www.hnsky.org/history_astap.htm) + +### Inferences +- There is a clear split: ML-for-registration research (2023-2026) targets domains with either (a) a fixed, well-characterized instrument and enormous calibration datasets (HST/WFPC2, where a single well-studied PSF/detector justifies training a dedicated network), or (b) archival/legacy data lacking machine-readable metadata (historical plates), or (c) real-time/embedded constraints (CubeSat trackers) — none of which match the amateur deep-sky stacking use case (heterogeneous, amateur-grade optics/sensors, no large labeled training corpus per rig). This plausibly explains why no amateur stacking tool has adopted ML for this purpose yet. +- Given the absence of any located production adoption, ML-based registration in the amateur stacking space should currently be characterized as **entirely unproven/research-only**, and confined to adjacent professional domains rather than "coming soon" to consumer tools based on available evidence. + +### Gaps +- No direct evidence either confirms or rules out unpublished/informal experiments (e.g., a Cloudy Nights or AstroBin forum thread by a hobbyist trying an ML-based star matcher) — the searches run did not surface such a thread within budget, and this remains an open gap rather than a confirmed absence. +- GraXpert (explicitly named in the assignment's suggested sources) was checked and confirmed to be an AI-based tool for gradient/background *extraction* and (per one search result) image deconvolution, not registration/alignment — it has no bearing on this key question and is noted here only to close out that source per the assignment's list — [GraXpert GitHub releases](https://github.com/Steffenhir/GraXpert/releases) + +--- + +## What do major open-source/commercial stacking tools list as their newest registration features in 2022-2026 changelogs/release notes? + +### Takeaway +Every actively developed tool surveyed (Siril, PixInsight, AstroPixelProcessor, ASTAP) shipped a dated, named registration/plate-solving improvement between 2022 and 2026; DeepSkyStacker's activity in the same window was comparatively modest and UI/porting-focused rather than algorithmic; N.I.N.A.'s relevant changes were on the plate-solving/polar-alignment side (a capture-time function) rather than post-capture stacking registration. + +### Cited Findings +- **Siril**: v1.0.6 (2022-10-18) fixed star-detection failures for large stars near image borders; v1.2.0-beta1 (2023-02-24) added the KOMBAT registration algorithm, refactored global registration into "2pass"/"Apply Existing" methods, unified 1-/2-/3-star registration, added max-star-count limiting for registration, added homography-based star tracking to `seqpsf`, and shipped Starnet++ integration on sequences; documentation states distortion-aware registration arrived "since version 1.3"; the 1.4 line (previewed Dec 2024, beta 2025-04-26) adds up-to-5th-order polynomial astrometric distortion correction, a true HST-style Variable-Pixel Linear Reconstruction ("drizzle") algorithm that works on Bayer-patterned data, and mosaic stacking with edge feathering — [Siril 1.2.0-beta1 ChangeLog](https://gitlab.com/free-astro/siril/-/raw/1.2.0-beta1/ChangeLog); [Siril "Twelve Days of Siril," 2024-12](https://siril.org/2024/12/the-twelve-days-of-siril/); [Siril 1.4.0 Beta 1](https://siril.org/download/2025-04-26-siril-1-4-0-beta1/); [Siril registration docs](https://siril.readthedocs.io/en/stable/preprocessing/registration.html) +- **PixInsight**: version 1.9.0 "Lockhart" (2024-12-20, with bugfix builds 1.9.1 on 2024-12-23 and 1.9.2 on 2024-12-28) reimplemented the astrometry engine with DDM (Domain Decomposition Method) radial-basis-function/thin-plate splines supporting up to 25,000 control points, carried the same DDM spline model into StarAlignment for arbitrary distortion modeling, and defaulted ImageSolver to 4,000 points for faster/more accurate solves — [DIYPhotography PixInsight 1.9.0 coverage](https://www.diyphotography.net/pixinsight-1-9-1-lockhart-released-major-upgrades-and-improvements/); [PixInsight forum, 1.9.3](https://pixinsight.com/forum/index.php?threads%2Fpixinsight-1-9-3-lockhart-released.25260%2F=) +- **AstroPixelProcessor**: 2.0.0-beta39 (2025-11-28) — "new and much improved registration engine" plus "much better Multi-Narrowband processing"; 2.0.0-beta46 (2026-06-24) — lower memory usage, startup-bug fix; upcoming beta47 promises workflows "more than 2x faster" generally and mosaic workflows "even 10x faster" due to a further registration-engine boost — [AstroPixelProcessor site](https://www.astropixelprocessor.com/); [APP release-information forum](https://www.astropixelprocessor.com/community/release-information/) +- **ASTAP**: 2023-12-30 improved the solver's behavior near the celestial pole and fixed annotation correctness for images at ±90° declination; 2023-12-12 fixed a major ephemerides-alignment bug; star databases (H18/H17 replacements) rolled out roughly a year before March 2024, improving both accuracy and plate-solving speed; 2024-11-06 fixed a major bug in astrometric aligned stacking that had been present since 2024-03-11; 2024-11-09 fixed a major bug affecting dark/flat analysis — [ASTAP history page](https://www.hnsky.org/history_astap.htm) +- **DeepSkyStacker**: 5.1.0 (March 2023) began a port of the codebase to Qt for cross-platform support; subsequent 5.1.x releases through 5.1.6 (June 2024) were primarily bug-fix releases, with 5.1.6 adding a UI "blink" comparison feature (caching the last twenty displayed stacking-panel images) and later versions dropping 32-bit/pre-Windows-10 support; no new registration *algorithm* was identified in the located release notes for this window — [Cloudy Nights DSS 5.1.6 announcement](https://www.cloudynights.com/forums/topic/926275-deepskystacker-516-is-now-available/); [DSS GitHub releases](https://github.com/deepskystacker/DSS/releases) +- **N.I.N.A.** (capture-time, not a stacking tool per se, but relevant to the pipeline's front end): plate-solving was updated to feed solvers an unstretched FITS image (instead of a JPEG) for quicker, more reliable solves, and to allow configuring camera gain/binning specifically for automated plate-solve exposures; the community "Three Point Polar Alignment" plugin uses repeated plate-solves across three RA-axis positions to align an alt-az/equatorial mount without a clear pole view — [N.I.N.A. plate-solving docs](https://nighttime-imaging.eu/docs/master/site/advanced/platesolving/); [Astrobasics N.I.N.A. plate-solving guide](https://astrobasics.de/en/proceeding/alignment-2/aligning-with-n-i-n-a-using-plate-solving/) +- **GraXpert**: confirmed to be a background-gradient-extraction (and, per one source, AI deconvolution) tool, not a registration/stacking tool; no registration-relevant changelog entries exist for it — [GraXpert GitHub releases](https://github.com/Steffenhir/GraXpert/releases) +- **Plate-solving ecosystem beyond astrometry.net**: Watney Astrometry Engine (a .NET/C# solver library with an astrometry.net-compatible API, usable as a local substitute for the nova.astrometry.net web service) and PlateSolve3 (an "improved PlateSolve3.80" specifically tuned for longer focal lengths and small fields of view, solving quickly even with few stars) were both cited as current alternatives in the amateur ecosystem — exact version/release dates were not confirmed as 2022+ in the available snippets — [Watney Astrometry GitHub](https://github.com/Jusas/WatneyAstrometry); [N.I.N.A. plate-solving docs](https://nighttime-imaging.eu/docs/master/site/advanced/platesolving/) + +### Inferences +- Registration/plate-solving is one of the most actively developed areas across every surveyed tool in 2022-2026, but the *type* of advance clusters into two buckets: (1) distortion-modeling upgrades for mosaics/wide fields (PixInsight DDM splines, Siril polynomial astrometric registration) concentrated at the very end of 2024, and (2) incremental robustness/performance/database improvements (ASTAP solver fixes, APP registration-engine speed) spread more evenly across 2022-2026. +- DeepSkyStacker's comparative inactivity on the registration-algorithm front (vs. active UI/porting work) is a reasonable signal that it has fallen behind Siril/PixInsight/APP specifically in registration sophistication during this window, though this is an inference from an absence of found evidence, not a stated admission by the DSS team. + +### Gaps +- Precise release dates for Watney Astrometry Engine and PlateSolve3.80 updates could not be confirmed as falling within 2022-2026 from the sources retrieved; both are plausibly relevant but should be re-verified before being cited with a specific date. +- A full/complete Siril GitLab ChangeLog for versions between 1.2.0-beta1 (Feb 2023) and the 1.4 line (Dec 2024 preview / April 2025 beta) was not retrieved line-by-line (only the beta1 excerpt and the 1.4 preview blog post were fetched) — there may be additional dated 1.2.x/1.3.x registration entries (e.g., the exact version/date "distortion-aware registration since 1.3" first shipped) not captured here. +- No SPIE conference paper on astronomical image registration/stacking (2022+) was located; the assignment's suggested SPIE source category was not covered due to tool-call budget, and should be treated as an open gap rather than "nothing exists." + diff --git a/research_notes/Astrophotography stacking techniques since 2022/uncertainty_photometry_performance.md b/research_notes/Astrophotography stacking techniques since 2022/uncertainty_photometry_performance.md new file mode 100644 index 0000000..fd12b84 --- /dev/null +++ b/research_notes/Astrophotography stacking techniques since 2022/uncertainty_photometry_performance.md @@ -0,0 +1,128 @@ +# Uncertainty Quantification, Photometric Calibration, and Performance/GPU Acceleration in Astrophotography Stacking (2022+) + +## What published methods (2022+) exist for propagating pixel-level uncertainty through a full nonlinear astronomical image processing chain (denoising, deconvolution, stretch)? Monte Carlo vs analytic/Jacobian approaches? + +### Takeaway +Survey/professional pipelines (Rubin/LSST) propagate uncertainty analytically through the *linear* parts of their chain (variance planes, inverse-variance weighting) and switch to explicit Monte Carlo noise-realization injection specifically at the point where a step becomes too nonlinear/adaptive for a closed-form Jacobian (e.g. shear calibration); in the broader inverse-problems/deep-learning literature (2024–2025), the newest tool for bounding uncertainty on reconstructions from nonlinear, learned, or iterative pipelines is self-supervised conformal prediction, not classical error propagation — both lines of evidence support the general design choice of "Monte Carlo when the chain has adaptive/nonlinear steps with no tractable Jacobian." + +### Cited Findings +- Rubin Observatory's LSST Science Pipelines coadd images carry an explicit variance plane (units nJy²) that is propagated analytically through calibration and coaddition, and coaddition itself uses inverse-variance weighting (weighted by the median of the variance-image plane) — an analytic, linear-algebra propagation, not Monte Carlo, for this stage of the chain — [Deep coadd — DP2, Rubin Observatory docs](https://dp2.lsst.io/products/images/deep_coadd.html). +- The LSST Science Pipelines documentation notes that "interpolation and warping modify the noise properties of the image," and that for accurate `METACALIBRATION` noise corrections used in the Rubin weak-lensing (Metadetection) pipeline, "we must also run the noise image through the same procedures" as the science image — i.e., an explicit correlated-noise-realization (Monte Carlo-style) pass through the *same* nonlinear resampling pipeline, once linear error propagation through warping/interpolation is no longer tractable — [Rubin Observatory LSST DRP / DP0.2 processing docs](https://dp0-2.lsst.io/data-products-dp0-2/data-processing.html) (this same noise-image-through-pipeline approach underlies the Metadetection Weak Lensing method for Rubin, arXiv:2303.03947, though the PDF could not be parsed to extract exact wording in this session — see Gaps). +- A 2023 arXiv paper, "Efficient Propagation of Uncertainty via Reordering Monte Carlo Samples" (arXiv:2302.04945), presents a general (not astronomy-specific) method for making per-sample Monte Carlo uncertainty propagation cheaper by reordering samples, applicable in principle to any nonlinear forward model too complex for a closed-form Jacobian — [arXiv:2302.04945](https://arxiv.org/abs/2302.04945). +- In deconvolution specifically, 2024 work applies diffusion-model-based Bayesian posterior sampling (Diffusion Posterior Sampling) to astronomical image deconvolution, explicitly framing the problem as computing a posterior distribution (hence supporting uncertainty/credible-interval estimates) rather than a single point estimate, "quantifying prior-driven features in reconstructions" — [Bayesian Deconvolution of Astronomical Images with Diffusion Models, arXiv:2411.19158](https://arxiv.org/pdf/2411.19158). +- For radio-interferometric imaging, 2024 work on "augmented equivariant bootstrap" provides fast uncertainty quantification for learned/fast reconstruction methods as an alternative to full posterior MCMC sampling — [arXiv:2410.23178](https://arxiv.org/pdf/2410.23178). +- Self-supervised conformal prediction (2024–2025) is proposed as a distribution-free way to wrap uncertainty intervals around image restoration outputs (denoising, deblurring) *without* needing ground truth for calibration, using Stein's Unbiased Risk Estimator (SURE) to self-calibrate from the noisy measurements themselves — explicitly framed for "any linear imaging inverse problem that is ill-conditioned" — [Self-supervised conformal prediction for uncertainty quantification in Poisson imaging problems, arXiv:2502.19194 / arXiv:2502.05127](https://arxiv.org/pdf/2502.19194). +- A related 2025/2026 line extends this to non-ground-truth, "equivariant bootstrapping" conformal prediction specifically for reconstruction uncertainty quantification — [Self-Supervised Conformal Prediction with Equivariant Bootstrapping, arXiv:2605.18655](https://arxiv.org/pdf/2605.18655) (note: this arXiv ID is dated later than the current session date context and should be treated cautiously — see Gaps). +- A plug-and-play approach with fast uncertainty quantification specifically for weak-lensing mass mapping was published in Astronomy & Astrophysics — a domain-specific (not generic denoiser-agnostic) analytic/plug-and-play hybrid — [A&A, DOI 10.1051/0004-6361/202557652](https://doi.org/10.1051/0004-6361/202557652). + +### Inferences +- The dominant pattern across both survey pipelines and academic inverse-problems research since 2022 is a **hybrid**, not a single winner: keep uncertainty analytic (variance-plane propagation, inverse-variance weighting) wherever the operation is linear or has a tractable Jacobian, and fall back to Monte Carlo / posterior-sampling / conformal methods only at the specific nonlinear or adaptive steps that break the analytic chain. This matches this pipeline's own architecture (Phase 3's analytic IVW sigma map, Phase 4's Monte-Carlo `propagate_uncertainty` for the nonlinear post-processing chain) rather than being an unusual design choice. +- Conformal prediction's appeal for a stacking-pipeline use case is that it doesn't require a differentiable/known forward model of every denoiser — relevant if a pipeline wanted calibrated per-pixel error bars without re-running the full chain K times, though as of the search results found, no work explicitly applies conformal prediction to ground-based amateur stacking (only to general imaging inverse problems and radio-interferometric/weak-lensing reconstruction). + +### Gaps +- Could not extract full text of the primary Rubin Metadetection paper (arXiv:2303.03947) — WebFetch returned only raw/corrupted PDF content, so the exact noise-injection methodology (number of noise realizations, whether it's literally "Monte Carlo" in the paper's own terminology) is inferred from secondary LSST documentation, not the paper itself. +- No 2022+ source was found that directly compares Monte Carlo vs. analytic/Jacobian uncertainty propagation cost/accuracy specifically for a *stacking* (rather than single-exposure or interferometric-imaging) pipeline; the "MC is slightly biased low for adaptive steps" type of finding (relevant to any MC-based propagation through denoisers that estimate parameters from the noisy data) was not corroborated or contradicted by any external 2022+ source found in this search — treat as an open, pipeline-specific empirical question rather than a documented general result. +- One arXiv ID surfaced (2605.18655) carries a date stamp inconsistent with a 2024–2025 publication window relative to the current date; flagging this as a possible dataset/indexing artifact rather than a verified claim. + +## What's new in photometric/spectrophotometric colour calibration for astrophotography since 2022 (e.g. PixInsight's SPCC)? + +### Takeaway +PixInsight's Spectrophotometric Color Calibration (SPCC), released November 21, 2022, is the clear headline advance: it replaced broadband-photometry-based color calibration (PCC, using APASS) with full per-star Gaia DR3 spectral integration against the user's actual sensor QE and filter transmission curves, and PixInsight's own testing claims a ~400%/300% reduction in R/B color-calibration uncertainty versus the older PCC method; the technique has since been adopted (with its own from-scratch Gaia DR3 spectral catalog extraction) by Siril, which now considers its own older PCC "obsolete" in favor of SPCC. + +### Cited Findings +- PixInsight's SPCC (Spectrophotometric Color Calibration) tool was published November 21, 2022 (last updated December 13, 2022 per the documentation page), described by PixInsight as "the culmination of nearly ten years of work" — [PixInsight SPCC documentation](https://pixinsight.com/doc/docs/SPCC/SPCC.html). +- SPCC uses Gaia DR3's mean BP/RP spectra, covering 219,165,266 point sources sampled from 336–1020 nm at 2 nm intervals (343 spectral values per star) — [PixInsight SPCC documentation](https://pixinsight.com/doc/docs/SPCC/SPCC.html). +- The algorithm: (1) detects stars separately per RGB channel and fits an elliptical PSF (Gaussian or Moffat) via Levenberg-Marquardt per channel; (2) evaluates PSF flux within the fitted ellipse out to the FWTM (full width at tenth maximum); (3) cross-matches image stars to Gaia catalog entries; (4) computes catalog R/G and B/G flux ratios by numerically integrating each user-specified filter transmission curve (and, for mono cameras, sensor QE curve) against each star's actual Gaia spectrum; (5) fits a robust (repeated-median) regression — tolerant of up to 50% outliers — between catalog and image flux ratios to derive the white-balance correction function — [PixInsight SPCC documentation](https://pixinsight.com/doc/docs/SPCC/SPCC.html). +- For one-shot-color (OSC) cameras, PixInsight recommends the "Ideal QE curve" option since the Bayer-filter response is already embedded in the per-channel measurement; genuine QE curves are only needed for monochrome + filter setups — [PixInsight SPCC documentation](https://pixinsight.com/doc/docs/SPCC/SPCC.html). +- PixInsight's own validation across 21 broadband test images found SPCC reduced color-calibration uncertainty by roughly 400% (red channel) and 300% (blue channel) relative to the older APASS-photometry-based PCC method — [PixInsight SPCC documentation](https://pixinsight.com/doc/docs/SPCC/SPCC.html). +- SPCC was conceived by PixInsight team member Vicent Peris and developed with Roberto Sartori, Edoardo Luca Radice, and Alicia Lozano — [PixInsight Forum, "New Tool Released: SpectrophotometricColorCalibration (SPCC)"](https://pixinsight.com/forum/index.php?threads/new-tool-released-spectrophotometriccolorcalibration-spcc.19599/). +- Siril has since implemented its own SPCC, describing it as making the older PCC "obsolete": "SPCC is a more accurate version of PCC ... SPCC takes your setup's sensor and filters into account. As a result, the color produced is closer to 'reality'" — [Siril 1.5.0 SPCC documentation](https://siril.readthedocs.io/en/latest/processing/color-calibration/spcc.html). +- Siril's SPCC implementation uses the Gaia DR3 `xp_sampled` datalink product (spectral data for the brightest ~127 sources per HEALPix level-8 pixel, typically down to Gaia magnitude ~17.6), requires user-specified filter transmittance or sensor QE curves ideally spanning 300–1100 nm (minimum 380–700 nm), and Siril has published its own extracted/re-hosted offline Gaia DR3 spectrophotometric catalog for this purpose (uncompressed random-access version released via Zenodo) — [Siril SPCC docs](https://siril.readthedocs.io/en/latest/processing/color-calibration/spcc.html); [Siril Spectrophotometric Catalog on Zenodo](https://zenodo.org/records/17988559); [Siril merge request !611 "Spectrophotometric Color Calibration plus color managed Photometric Color Calibration"](https://gitlab.com/free-astro/siril/-/merge_requests/611). +- Siril also ships a separate Gaia DR3 astrometry-only extract for plate solving, distinct from the spectrophotometric one — [Siril Astrometry Catalogue on Zenodo](https://zenodo.org/records/14692304). + +### Inferences +- SPCC's core methodological advance over plain Gaia-color-index-formula approaches (like this project's `colorindex` method) is per-star **spectral integration against the actual instrument response** rather than a single fixed color-index-to-color-correction formula; a fixed formula (BP-RP → B-V, as used by "colorindex" methods generically) is filter/sensor-agnostic by construction and therefore cannot capture a specific camera+filter's true color response the way SPCC's per-instrument integration can. +- The rapid cross-adoption (PixInsight Nov 2022 → Siril's own SPCC, with its own Gaia DR3 spectral extract shipped for offline use) indicates SPCC-style calibration is now considered close to a de facto standard among the two most technically sophisticated open/commercial amateur processing tools, not a PixInsight-only proprietary technique. + +### Gaps +- No public academic/independent (non-vendor) validation of SPCC's claimed 400%/300% uncertainty-reduction figures was found; the only source for this number is PixInsight's own documentation, so it should be treated as a vendor-reported claim, not an independently peer-reviewed result. +- No release date could be confirmed for Siril's own SPCC feature (only that it postdates Siril 1.4.0/1.4.1); the exact version/date it shipped in Siril was not found in the sources retrieved. +- No source was found describing the Teff/blackbody-integration style of color calibration (as opposed to Gaia color-index or full spectral integration) being adopted by any other tool besides what a project might build in-house; this project's own `spcc` method (blackbody spectrum from Gaia `teff_gspphot` × generic per-channel Gaussian response) is a lighter-weight approximation of true SPCC (which uses the star's actual measured Gaia BP/RP spectrum, not a blackbody fit to its temperature) — worth flagging as a real methodological difference from PixInsight/Siril's SPCC, not merely a naming coincidence. + +## What advances exist in dark-current temperature modeling, photon transfer curve (PTC) gain/read-noise estimation, or per-pixel noise validation (odd/even split methods) for stacking pipelines since 2022? + +### Takeaway +The most concrete 2022+ advance found is in sensor-hardware dark-current *compensation* (in-pixel temperature sensors driving a synthetic dark-frame subtraction, ~80% median dark-signal reduction), and in PTC methodology a 2022+ line of work moves from the classical two-flat-frame variance-vs-signal method toward full-likelihood modeling of up-the-ramp reads for detectors with many non-destructive reads (relevant to scientific CMOS/IR arrays, not typical consumer CMOS); no published external source was found specifically describing odd/even half-stack noise-consistency validation for amateur stacking pipelines, which appears to be a technique this project uses but that isn't independently documented elsewhere in the literature found. + +### Cited Findings +- A 2023 paper (published in *Sensors*, presented at the 2023 International Image Sensors Workshop) describes CMOS dark-current compensation using in-pixel temperature sensors (IPTS): an artificial dark reference frame is synthesized from per-pixel temperature readings plus a pre-calibrated dark-current-vs-temperature model, then subtracted from the live image — achieving ~80% median dark-signal reduction and ~55% reduction in non-uniformity over a −40 °C to 90 °C range — [Harvest Imaging / MDPI Sensors 2023, "A CMOS Image Sensor Dark Current Compensation Using In-Pixel Temperature Sensors"](https://www.harvestimaging.com/pubdocs/271_Sensors_MDPI_IISW2023.pdf); [PMC10674984](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10674984/). +- The same paper documents that dark current's temperature dependence is not a single exponential across the full range: at lower temperatures dark current increases ~1.08× per 5 °C (depletion-current dominated), while at higher temperatures it increases ~1.8× per 5 °C (diffusion-current dominated) — a two-regime, not single-slope, temperature model — [PMC10674984](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10674984/). +- In PTC methodology, a paper on "Computing the Electronic Gain for Detectors Read Out Up-The-Ramp" presents an alternative to the classical photon transfer curve for detectors with many non-destructive reads per exposure (typical of modern astronomical IR/CMOS arrays), using the full likelihood function of all reads rather than only endpoint-difference variance, explicitly framed as overcoming classical-PTC limitations for this readout mode — [arXiv:2512.09131](https://arxiv.org/pdf/2512.09131) (note: this arXiv ID/date appears to be from a period after the stated "January 2026" knowledge cutoff window referenced in this environment — flagged for the report writer to verify currency independently; see Gaps). +- General PTC methodology write-ups (not dated 2022+, used only as background) describe deriving conversion gain (e⁻/DN) from the slope of variance vs. signal in the photon-noise-dominated region, and note the PTC's continued role as "a unifying tool for analyzing sensor noise," including read noise, shot noise and PRNU characterization — [Adimec, "How to measure the Photon Transfer Curve for CCD or CMOS cameras"](https://www.adimec.com/how-to-measure-the-photon-transfer-curve-for-ccd-or-cmos-cameras/); [Jim Kasson blog, "Photon Transfer Curves: a unifying tool for analyzing sensor noise"](https://blog.kasson.com/the-last-word/photon-transfer-curves-a-unifying-tool-for-sensor-noise/). +- A "3D noise" extension of the classical PTC (adding a third dimension of noise parameters beyond the standard 2D variance-vs-signal plot) was found referenced in an Optica (formerly OSA) publication listing, though full methodological detail could not be retrieved in this session — [Optica, "3D noise photon transfer curve"](https://opg.optica.org/ao/upcoming_pdf.cfm?id=452166). + +### Inferences +- The hardware-level (in-pixel temperature sensor) dark-current compensation research is targeted at sensor/ASIC design, not post-hoc software calibration-frame libraries — it is not directly applicable to a software stacking pipeline working from a fixed dark-frame library and header-reported sensor temperature, but it does validate the general principle (used by temperature-interpolated dark models in software) that dark current vs. temperature is non-linear/multi-regime rather than a simple single-slope relationship, reinforcing why a polynomial (rather than linear) fit across the library's temperature range is the more defensible modeling choice. +- The shift toward full-likelihood up-the-ramp gain estimation is a scientific-detector (JWST-class, many-non-destructive-read) methodology; consumer CMOS astrophotography cameras (the dominant target of amateur stacking pipelines) still use simple two-frame/two-flat difference PTC methods, so the classical Janesick-style two-frame PTC approach remains the applicable state of the art for this use case, not superseded by up-the-ramp likelihood methods. + +### Gaps +- No 2022+ source was found describing "odd/even split-half" or similar per-pixel noise-consistency validation (comparing two independently-combined half-stacks to derive an empirical noise map, or a correlation-based consistency check) in any amateur or professional pipeline context — general web search returned only generic "stacking reduces noise by sqrt(N)" explainer content, not this specific validation technique. This appears to be either a niche/undocumented technique or one primarily developed in-house by specific pipelines (including possibly this one) rather than published or widely blogged about; flagged as a genuine literature gap rather than an omission of search effort. +- No 2022+ academic or vendor source was found on PTC gain estimation specifically as applied within an *amateur* astrophotography stacking tool's auto-calibration flow (e.g., automatically running PTC from bias+flat pairs as part of a stacking pipeline, as opposed to a standalone sensor-characterization exercise) — the sources found are all either sensor-engineering papers or generic photography/DSLR PTC tutorials, not stacking-software integration case studies. +- The exact currency of arXiv:2512.09131 relative to the "January 2026" cutoff context could not be independently verified within this search session; treat its publication date with caution and re-verify before citing it as a hard 2022+ reference in the final report. + +## What's new in real-time/live stacking and online (streaming, single-pass, O(1)-memory) sigma-clip or combination algorithms since 2022? + +### Takeaway +Live/real-time stacking has become mainstream and vendor-supported since 2022 (Siril 1.2.0's experimental live-stacking mode watching a directory for new subs, N.I.N.A.'s dedicated Live Stacking plugin in nightly builds, ZWO ASIAIR's continued live-stack feature with firmware updates through 2023–2024), but no source found in this search describes the *specific algorithmic internals* (e.g., a formally single-pass/O(1)-memory Welford-style online sigma-clip accumulator) of any of these tools' live-stacking combine step — the public documentation for all three describes the user-facing feature and workflow, not the underlying streaming statistics algorithm. + +### Cited Findings +- Siril 1.2.0 added live stacking as an explicitly labeled "experimental" feature: it monitors a target directory in real time and stacks images as they arrive — [Siril Livestacking documentation, 1.5.0/1.4.4](https://siril.readthedocs.io/en/latest/Livestack.html); [Cloudy Nights, "Live stacking with Siril"](https://www.cloudynights.com/forums/topic/866258-live-stacking-with-siril/). +- Siril's general stacking documentation (current, 1.2.6/1.5.0) confirms Sigma Clipping (median-based, iterative, low/high sigma thresholds) and MAD Clipping (Median Absolute Deviation estimator, aimed at noisy infrared-style data) as available rejection algorithms, alongside a changelog note about "a new parallelized stacking algorithm for all sequences, including all SER formats," configurable maximum memory use, and threading of the heaviest processing steps to keep the GUI responsive — [Siril 1.2.6 Stacking docs](https://siril.readthedocs.io/en/1.2/preprocessing/stacking.html); [Siril GitHub mirror ChangeLog](https://github.com/gnthibault/siril/blob/master/ChangeLog). +- N.I.N.A. (Nighttime Imaging 'N' Astronomy) added a dedicated Live Stacking plugin (available in nightly builds as of the sources found, with active user discussion through December 2024) that performs calibration and live stacking without requiring a third-party application, with sequencer instructions for "Start Live Stacking" / "Take Many Exposures" / "Stop Live Stacking" — [Cloudy Nights, "NINA Live Stacking Plugin"](https://www.cloudynights.com/forums/topic/949494-nina-live-stacking-plugin/); [Cloudy Nights, "N.I.N.A. Livestack Plug-in Setup"](https://www.cloudynights.com/forums/topic/991855-nina-livestack-plug-in-setup/). +- ZWO's ASIAIR (Pro/Plus) continues to ship Live Stacking as a core feature (build on faster onboard processor/IO), with 2023–2024-era firmware/user reports covering related but adjacent updates (e.g., EAF focuser firmware max-step-count increase in Feb 2023) and known issues (missing `OBJCTRA`/`OBJCTDEC` header fields in live-mode light frames reported since March 2024, and satellite trails persisting visibly in the live-stacked result, implying no trail-specific rejection in that mode) — [Cloudy Nights ASIAIR forum threads](https://www.cloudynights.com/forums/topic/893295-asiair-live-stacking-issues/); [ZWO User Forum, "Satellite Tracks in ASIAIR Live Stack"](https://bbs.zwoastro.com/d/14259-satellite-tracks-in-asiair-live-stack/). + +### Inferences +- The live-stacking feature landscape genuinely expanded since 2022 across all three major amateur ecosystems (Siril, N.I.N.A., ZWO/ASIAIR), which supports treating live/streaming stacking as now a standard expected capability of a modern pipeline rather than a niche add-on — but the *reported issues* (ASIAIR live mode dropping WCS-relevant header fields, satellite trails surviving into the live-stacked result) suggest that in practice, shipped live-stacking implementations trade off robustness/completeness (metadata fidelity, outlier/trail rejection) for the real-time constraint, which is a useful competitive data point: a streaming combiner that still performs full per-frame outlier/trail rejection (rather than a simple running mean/clip) would be differentiated from at least the ASIAIR implementation as documented in user reports. +- None of the three tools' public documentation exposes whether their live-stacking combine is a true single-pass/O(1)-memory statistical accumulator (e.g., Welford's algorithm generalized to sigma-clipping) or simply a bounded/rolling in-memory buffer of recent frames — this is a genuine architectural detail none of the vendors publish, likely because it's treated as an implementation detail rather than a marketed feature. + +### Gaps +- No 2022+ academic paper or engineering blog post was found describing a formally single-pass (Welford-style) online sigma-clipping algorithm specifically for astronomical frame stacking — the search surfaced only user-facing feature documentation for the three tools above, not algorithmic internals or complexity analysis. If such a description exists (e.g., in a GitHub issue/PR discussion, a conference poster, or a niche blog), it was not surfaced by the queries run in this session. +- No comparison was found of live-stacking memory/CPU characteristics across Siril vs. N.I.N.A. vs. ASIAIR (e.g., whether any of them keep the full per-frame stack in memory rather than an accumulator) — this remains an open question for the report writer. + +## What trends exist in using native/compiled code (Rust, C++, GPU/CUDA) to accelerate astronomical stacking pipelines since 2022, and what performance gains are typically reported? + +### Takeaway +GPU/CUDA acceleration remains uneven and contested among mainstream amateur stacking tools as of 2022–2024 (AstroPixelProcessor has explicitly declined to add CUDA support citing system-dependency issues, while expressing future interest in broader GPU support), whereas Rust as a systems-programming choice for new astrophotography stacking engines is an emerging but still very early-stage trend — one concrete example (AetherStack) is a from-scratch Rust rewrite explicitly prioritizing a deterministic f64 CPU reference implementation as the numerical "oracle" before any GPU optimization is attempted, and it is pre-release (0 releases, 27 commits, effectively unadopted) as found in this search. Larger-scale professional/survey radio-astronomy imaging pipelines (not stacking in the amateur DSO sense, but a closely related image-synthesis domain) are reporting real, measured GPU speedups on top-tier supercomputing hardware (e.g., the Leonardo pre-exascale system) since 2023–2024. + +### Cited Findings +- AstroPixelProcessor's developers have stated CUDA support "is not planned due to system dependency issues," while also acknowledging interest in adding more GPU-accelerated processing "at a later date" — i.e., as of the sources found, APP has not shipped full CUDA acceleration for its stacking/registration core — [AstroPixelProcessor community forum, "processing acceleration by CUDA/GPU"](https://www.astropixelprocessor.com/community/main-forum/processing-acceleration-by-cuda-gpu/). +- AetherStack (GitHub: albedo83/AetherStack) is a from-scratch Rust "precision-first astrophotography calibration and stacking engine" whose stated design goals are "numerical accuracy, reproducible results, explainable decisions, and bounded-memory processing" of large FITS/SER sessions; its current milestone establishes a "strict CPU reference implementation" using "the deterministic f64 CPU path" as "the numerical oracle" that any future optimized CPU or GPU implementation must match via differential testing against documented tolerances — [GitHub: albedo83/AetherStack](https://github.com/albedo83/AetherStack). As of the retrieval in this session, the repository shows 27 commits, 0 releases, 0 stars/forks — i.e., a very early-stage, unreleased, and effectively unadopted project, not a proven production tool. +- On the professional/radio-astronomy side (a related but distinct image-synthesis domain, not amateur DSO stacking), a 2024 paper on "RICK" (radio imaging) reports GPU-accelerated imaging results run on the Leonardo pre-exascale supercomputer (ranked 7th on the TOP500 list as of June 2024), with head-to-head CPU vs. GPU tests on LOFAR-representative SKA-pathfinder data using the same codebase — [arXiv:2411.07321, "Accelerating radio astronomy imaging with RICK"](https://arxiv.org/pdf/2411.07321); a follow-up, "Green computing toward SKA era with RICK," extends this to power/efficiency framing — [arXiv:2504.00959](https://arxiv.org/pdf/2504.00959). +- A GPU-accelerated pipeline for Fast Radio Burst (FRB) searches with low-frequency radio telescopes is explicitly designed to "minimise I/O operations and process the data inside GPU memory" — a design principle (keep data resident on-GPU across pipeline stages rather than round-tripping through host memory per stage) directly transferable to any GPU-accelerated stacking pipeline design — [arXiv:2405.13478, "High-Time Resolution GPU Imager for FRB searches at low radio frequencies"](https://arxiv.org/pdf/2405.13478). + +### Inferences +- The overall picture for *amateur DSO stacking specifically* is that native/compiled acceleration trends since 2022 are still led by CPU-side optimization and cautious, incremental GPU adoption (APP's stated hesitation over cross-platform CUDA dependency management) rather than a wholesale industry shift to GPU-first pipelines — consistent with this project's own measured finding that `--use-gpu` is currently *slower* than CPU-only on a real consumer GPU for realistic session sizes, since a partial-coverage GPU dispatch (only some pipeline stages ported) can lose more from reduced CPU worker parallelism than it gains from the ported stages. +- Rust adoption for a full stacking-engine rewrite (AetherStack) is real but nascent — its explicit "CPU f64 oracle before any optimization" methodology is a defensible engineering practice (numerically validate before optimizing) that parallels a "numpy mirror + native kernel, parity-tested" pattern, but as an unreleased, unstarred project it should not be cited as an adopted or proven approach, only as an early, single-example data point that some developers in this space are experimenting with Rust as of the current period. +- The strongest evidence of real, measured (not merely claimed) GPU acceleration gains since 2022 in image-synthesis-adjacent astronomy work comes from *survey/radio-astronomy* imaging pipelines run on national supercomputing infrastructure, not from amateur DSO stacking tools — the performance-gain regime (many-node, many-GPU HPC clusters processing SKA-pathfinder-scale data) is not representative of what a single-workstation amateur stacking tool with one consumer GPU could expect to reproduce. + +### Gaps +- No specific, quantified GPU vs. CPU speedup numbers (e.g., "Nx faster") were retrieved from the RICK/FRB-imager papers in this session — WebSearch snippets described the existence and hardware context of the benchmarks but not the actual reported multiplier; the report writer should treat "GPU acceleration reported" as confirmed but the magnitude as not yet sourced from this research pass. +- No 2022+ source was found benchmarking DeepSkyStacker's or Siril's own native/SIMD (as opposed to GPU) acceleration work specifically (e.g., AVX2/AVX-512 usage, multi-threading changes) beyond the generic Siril changelog note about a "new parallelized stacking algorithm" and configurable memory limits — exact before/after performance numbers for Siril's own native-code optimizations since 2022 were not found. +- No independent, adopted (non-experimental, released) Rust-based amateur astrophotography stacking tool other than AetherStack was found in this search; if one exists it was not surfaced by the queries run, and this should be treated as a real gap rather than evidence that none exists. + +## Any notable academic work (2022+) on error/confidence maps for amateur or survey-pipeline stacked images? + +### Takeaway +Confirmed academic-grade error/confidence-map work since 2022 is concentrated in survey pipelines (Rubin/LSST's per-pixel variance-plane coadd product) and in general computational-imaging uncertainty-quantification research (conformal prediction, diffusion-based posterior sampling) rather than in any paper specifically targeting *amateur* stacked-image confidence maps — no academic (peer-reviewed or arXiv) paper focused specifically on amateur/consumer-camera stacking confidence maps was found in this search. + +### Cited Findings +- Rubin/LSST's Deep Coadd data product ships a per-pixel variance plane (in nJy²) as a first-class output of the stacking (coaddition) pipeline, propagated via inverse-variance-weighted mean stacking — the clearest example of a survey pipeline treating per-pixel error/confidence mapping as a core deliverable, not an add-on — [Deep coadd — DP1](https://dp1.lsst.io/products/images/deep_coadd.html); [Deep coadd — DP2](https://dp2.lsst.io/products/images/deep_coadd.html). +- The broader 2023–2025 computational-imaging uncertainty-quantification literature (conformal prediction for imaging inverse problems, diffusion-model posterior sampling for deconvolution, equivariant-bootstrap uncertainty for fast/learned reconstructions — all cited in the first section above) represents the closest generalizable academic methodology that could, in principle, be adapted to produce confidence maps for amateur stacked images, but none of the papers found explicitly target that use case — [arXiv:2502.19194](https://arxiv.org/pdf/2502.19194); [arXiv:2411.19158](https://arxiv.org/pdf/2411.19158); [arXiv:2410.23178](https://arxiv.org/pdf/2410.23178). +- A 2025 A&A paper applies fast, plug-and-play uncertainty quantification specifically to weak-lensing *mass mapping* (a downstream science product derived from stacked/coadded imaging, not the stacked image itself) — an example of confidence-mapping methodology one step downstream of stacking in a survey context — [A&A, DOI 10.1051/0004-6361/202557652](https://doi.org/10.1051/0004-6361/202557652). + +### Inferences +- The complete absence of amateur-stacking-specific academic literature on confidence/error maps (as distinct from survey-pipeline and generic computational-imaging work) suggests this remains a genuinely underexplored niche — any pipeline that ships a per-pixel confidence map for a consumer-camera stacked/post-processed image (as opposed to a raw linear stack) would be doing something not directly precedented in the academic literature found, though it can draw on the *methodological* precedent of survey pipelines' variance-plane propagation (linear stage) plus Monte Carlo / conformal methods (nonlinear stage) as a hybrid template. + +### Gaps +- No dedicated academic paper (2022+) on error/confidence maps for *amateur* astrophotography specifically was found; this appears to be a genuine gap in the published literature rather than a search-coverage failure, since searches specifically targeting this combination returned only generic stacking-tutorial content or the survey-pipeline/computational-imaging papers already cited elsewhere in these notes. +- Could not verify whether any conference (e.g., ADASS — Astronomical Data Analysis Software and Systems) has published proceedings specifically on this topic since 2022; an ADASS 2023 calendar page surfaced in search results but was not fetched for content in this session, so its relevance (if any) to error/confidence mapping is unconfirmed — [ADASS 2023 Calendar (unverified relevance)](https://adass2023.lpl.arizona.edu/sites/adass2023.lpl.arizona.edu/files/2023-11/ADASS_2023_Calendar_Wednesday.pdf).