POTPyRI v1.0.18

Overview

This release completes the merge of dev into main (PR #84). It adds configurable per-frame background subtraction, fixes photometry FWHM estimation for undersampled and crowded fields, improves raw-file discovery across instruments, and restores CI documentation builds.

If you are upgrading from v1.0.16 or earlier, also review the v1.0.17 changes (photutils 3.x compatibility, photometry error handling, WCS/alignment options, relative calibration).

---

New features

Per-frame background subtraction (--bkg-sub)

Choose how optical science frames are sky-subtracted before stacking:

Mode	Flag	Behavior
Local (default)	--bkg-sub local	Spatially varying 2D mesh (Background2D)
Constant	--bkg-sub constant	Single sigma-clipped median subtracted from every pixel
None	--bkg-sub none or --skip-skysub	No per-frame subtraction

Constant mode is useful for fields with extended sources (e.g. galaxies) where the default 2D background model can subtract source flux. Each frame records BKGSUB and SKYBKG in the header.

Raw file discovery

• --proc fits (or .fits / uncompressed) now works across all instruments via the base Instrument class.
• File sorting logs search paths and glob patterns when discovering raw data.

Instrument registry

• instrument_getter is registry-driven from __all__ (single source of truth for supported instruments).
• Aliases centralized (BINO→BINOSPEC, MMIR→MMIRS).
• New helpers: resolve_instrument_name(), supported_instruments(), UnknownInstrumentError.

---

Bug fixes

Photometry FWHM

Fixed a feedback loop that inflated FWHM estimates (e.g. reporting ~125 px when the true PSF is ~5 px):

• FWHM is measured in a fixed compact aperture instead of an aperture that grows with the measured FWHM.
• ePSF cutout size is capped relative to fwhm_init.
• Moffat fitting uses the PSF core only; unreliable ePSF fits fall back to the catalog FWHM.

CI / packaging

• Stop tracking potpyri/_version.py in git (generated by setuptools_scm at install time). Fixes Documentation and pip install failures on GitHub Actions.

photutils 3.x (included in v1.0.17; relevant if upgrading from older releases)

• Normalize DAOStarFinder / ApertureStats column names (x_centroid vs xcentroid).
• photloop raises PhotometryError instead of continuing to zeropoint without APPPHOT.
• absphot.find_zeropoint handles missing photometry gracefully.

---

WCS, alignment & stacking (v1.0.17)

• --skip-external-astrometry: use existing header WCS only.
• --fine-align-catalog: Gaia, Pan-STARRS, SDSS, Legacy, 2MASS, SkyMapper.
• --relative-calibration: SExtractor + RA/Dec cross-match for relative flux scaling before stack combine.
• Improved Gaia alignment fallback and WCS header sanitization for calibration frames.

---

Breaking / behavior changes

• Failed photometry now stops the pipeline before zeropoint calculation (no silent KeyError on missing APPPHOT).
• --skip-skysub is equivalent to --bkg-sub none.
• potpyri/_version.py is no longer in the repository; run pip install -e . (or pip install .) after cloning to generate it.

---

Example usage

# Constant sky level per frame (extended sources)
main_pipeline GMOS /path/to/data --bkg-sub constant

# Uncompressed FITS ingestion (all instruments)
main_pipeline GMOS /path/to/data --proc fits

# Tighter star finding for undersampled PSF
main_pipeline GMOS /path/to/data --fwhm-init 3

POTPyRI 1.0.17 – WCS/catalog refactor, photometry reliability, and zeropoint robustness

POTPyRI 1.0.17 – WCS/catalog refactor, photometry reliability, and zeropoint robustness

This release merges the dev branch into main (PR #83). It hardens WCS handling and catalog access, makes PSF photometry and zeropoint calibration robust across photutils 3.x, and prevents the pipeline from continuing to flux calibration when photometry fails.

WCS, astrometry, and catalogs

• Centralized catalog utilities in potpyri/utils/catalogs.py for Vizier queries, fine alignment, and zeropoint fitting; shared metadata for Gaia, Pan-STARRS, SDSS, Legacy, 2MASS, etc.
• Fine alignment refactor: --fine-align-catalog (e.g. gaia, panstarrs, sdss, legacy, 2mass, skymapper); improved fallback when primary catalogs are unavailable.
• Skip external astrometry when a valid celestial WCS is already present: new --skip-external-astrometry option and validate_existing_wcs_header() checks.
• FITS WCS/time header normalization via fix_deprecated_wcs_header_cards() (RADECSYS → RADESYS, sync/remove MJD-OBS vs DATE-OBS) to avoid astropy WCS warnings; applied on GMOS extension load and calibration reads.
• Sky-frame stacking improvements for small frame counts and degenerate sigma-clipping edge cases.

Photometry (photutils 3.x compatibility)

• PhotometryError and clearer logging in photloop / do_phot when star detection or PSF photometry fails.
• photloop now raises if all S/N attempts fail (no silent exit leaving stacks without APPPHOT).
• DAOStarFinder column normalization (x_centroid / y_centroid → legacy names used internally) for photutils 3.x.
• ApertureStats API updates for photutils 3.x (x_centroid, covariance_xx, semimajor_axis, etc.) with fallbacks for older photutils.
• PSF photometry/residual image compatibility across photutils 2.x/3.x (PSFPhotometry, include_local_bkg).

Absolute photometry (zeropoint)

• find_zeropoint returns bool and handles missing or incomplete APPPHOT extensions without raising KeyError.
• Actionable log messages when photometry was not run, extensions are missing, or catalog matching fails.

Testing and CI

• Regression tests for production failure modes (DAOStarFinder None, photutils 3 centroid columns, exhausted photloop, missing APPPHOT + zeropoint).
• main_pipeline test ensuring zeropoint is not attempted when photometry raises.
• Optional BINOSPEC FRB stack integration test (POTPYRI_FRB_STACK_TEST; skipped in CI when data absent).
• CI fix: remove top-level EPSFModel import in tests (removed in photutils 3.x).
• .gitignore updates for .coverage and local examples/ content.

Documentation / housekeeping

• WCS alignment fallback improvements (PR #78).
• Paper/bibliography updates (PRs #76–#79).

---

One-paragraph summary (for PyPI or short announcement)

POTPyRI 1.0.17 refactors WCS alignment and catalog handling, adds options to skip external astrometry when headers already contain a usable WCS, and normalizes legacy FITS WCS/time keywords. PSF photometry is updated for photutils 3.x (DAOStarFinder centroids, ApertureStats, PSF fitting), with explicit PhotometryError reporting when photometry cannot complete. Zeropoint fitting no longer crashes on missing APPPHOT tables and logs clear failure modes. New regression tests guard against the silent photometry failures that previously led to KeyError: "Extension 'APPPHOT' not found." downstream.

---

Install / upgrade

pip install --upgrade potpyri

POTPyRI 1.0.16 – Sky masking, relative calibration, and calibration/WCS fixes

POTPyRI 1.0.16 – Sky masking, relative calibration, and calibration/WCS fixes

Sky image creation and masking

• Iterative sigma clipping when building sky frames and master sky: configurable maxiters and sigma limits so the sky level is estimated from progressively cleaned pixels.
• Bright-source and low-outlier masking in each sky frame and in the combined master sky (pixels above/below median ± Nσ are masked; masked pixels set to 1.0 in normalized sky).
• Configurable parameters for create_sky: sky_sigma_upper/lower, sky_maxiters, sky_n_sigma_high/low, msky_n_sigma_high/low, msky_maxiters (defaults remain backward compatible).
• Robust scale when applying sky in process_science: sigma-clipped median of the science frame (with maxiters) is used to scale the master sky so bright objects don’t bias the sky level.
• CCDData mask handling: initialize mask when None before setting masked pixels so combination and downstream steps work correctly.

Relative calibration before stacking

• Optional relative flux calibration before stacking: run Source Extractor on each aligned frame, cross-match sources in RA/Dec, and use relative source fluxes to set per-frame scale factors for ccdproc.combine.
• Inverse-variance weighting using FLUXERR_AUTO when available; otherwise median of flux ratios.
• CLI flag --relative-calibration to enable this step; scales are used as-is in stack_data (no division by exposure time when relative scales are provided).
• Fallback for failed frames: if a frame has too few matches, its scale is set to 1.0/exptime for that frame instead of aborting relative calibration; exptimes is passed into compute_relative_scales.
• Unit tests for compute_relative_scales (mocked SExtractor): single image, exptimes mismatch, SExtractor failure, catalog too small, two-frame flux ratio, failed-frame exptime fallback, FLUXERR weighting, no-FLUXERR median path.

Calibration and WCS robustness

• Calibration FITS (master bias/dark/flat/sky) no longer carry WCS keywords that can trigger invalid transforms (e.g. DEC=-100): headers are sanitized before write and calibration frames are loaded with _read_calibration_ccd (no WCS parsing).
• Sky subtraction units: fixed UnitConversionError when subtracting normalized (dimensionless) sky from electron science frame by scaling sky with the science median in electron units before subtract.
• find_catalog (utilities): all code paths now return a 6-tuple (catalog, catalog_ID, ra, dec, mag, err); 2MASS K-band mapping for K, Ks, Kspec.

Testing and CI

• Unit tests for create_sky (masking bright sources, default and custom sigma params), compute_relative_scales (eight cases above), instruments, utilities, and image_procs.
• Integration tests marked with @pytest.mark.integration and skipped in CI; CI runs pytest -m "not integration".

Documentation

• README and CONTRIBUTING updated (testing, docs, contributing).
• Installation and index docs updated for current setup.

---

One-paragraph summary (for PyPI “Description” or short announcment)

This release improves sky frame creation with iterative sigma clipping and configurable masking of bright sources and defects, and adds an optional relative flux calibration step before stacking: Source Extractor catalogs are cross-matched in RA/Dec and relative source fluxes (with optional FLUXERR_AUTO weighting) define per-frame scale factors for the stack combine. Calibration frames no longer write or parse WCS keywords that could cause invalid transform errors; sky subtraction uses a sigma-clipped science median for scaling and correct units. Additional unit tests cover sky creation, relative calibration, and calibration loading; integration tests are marked and skipped in CI.

---

Version suggestion

• Current dev version in the repo: 1.0.15.dev10.
• For a patch release (fixes only): 1.0.15 (tag and build from the commit you want).
• For a minor release (new features: relative calibration, sky improvements): 1.0.16 (bump then tag, or use setuptools-scm and tag e.g. v1.0.16 so the built package gets that version).

After tagging, build and upload to PyPI:

pip install build twine
python -m build
twine upload dist/*

v1.0.14

What's Changed

• Update to files for POTPyRI paper. by @KerryPaterson in #50
• Fast forward dev-cdk by @charliekilpatrick in #51
• Fixes for paper. by @KerryPaterson in #52
• Updates to astrometry methods by @charliekilpatrick in #54
• Changes for POTPyRI v1.1 to make code more modular and closer to Keck DRP by @charliekilpatrick in #57
• Adding error test and fixes to MOSFIRE methods by @charliekilpatrick in #59
• Merging recent updates to dev branch by @charliekilpatrick in #63

Full Changelog: v1.0.13...v1.0.14

v1.0.13

Merge pull request #49 from CIERA-Transients/dev-cdk

Updating with new WCS test

v1.0.11

Merge pull request #39 from CIERA-Transients/dev-cdk

Dev cdk

v1.0.9

Reverting to previous publish.yml version.

v1.0.8

Fixing issue with publish.yml and pypi upload.

v1.0.7

Adding new --keep-all-astro options, fixing issues with static mask references.

v1.0.6

Updating with no local version for pypi.