Skip to content

Reprocess STAC catalogs with morton indices (mbox/mpolygon) #78

@espg

Description

@espg

Summary

We've updated our STAC catalog code to encode morton indices (opr:mbox per item, opr:mpolygon per collection) via the OPR STAC Extension (#77). Now we need to reprocess all existing STAC catalogs so that the deployed parquet files on source.coop include this information.

Catalogs are prioritized by bedmap overlap — we want morton indices on catalogs that overlap with bedmap data first, so we can use them for bedmap-to-frame matching (#69, #72).

The links column fix and thumbnailsthumbnail rename from #73 should also be applied during reprocessing.

Update 2026-04-22: Bulk reprocessing complete. All 51 deployed catalogs on source.coop now have valid opr:mbox (4 cells per item), opr:provider, and the schema fixes from #73. Verified programmatically against every parquet file.


Tier 1 — Bedmap 2 Overlap (CReSIS, Antarctica)

Highest priority. These seasons have direct overlap with Bedmap 2 campaign data and are the first targets for bedmap-to-frame matching.

OPR Catalog Bedmap 2 File
2004_Antarctica_P3chile NASA_2004_ICEBRIDGE_AIR_BM2
2009_Antarctica_DC8 NASA_2009_ICEBRIDGE_AIR_BM2
2010_Antarctica_DC8 NASA_2010_ICEBRIDGE_AIR_BM2
2011_Antarctica_DC8 NASA_2011_ICEBRIDGE_AIR_BM2
2012_Antarctica_DC8 NASA_2012_ICEBRIDGE_AIR_BM2
  • 2004_Antarctica_P3chile
  • 2009_Antarctica_DC8
  • 2010_Antarctica_DC8
  • 2011_Antarctica_DC8
  • 2012_Antarctica_DC8

Known issue: 2002_Antarctica_P3chile also overlaps Bedmap 2 (NASA_2002_ICEBRIDGE_AIR_BM2) but has geometry jumps in the raw radar data (see #69 discussion). Now deployed with morton indices but source data quality issues remain.


Tier 2 — Bedmap 3 Overlap (CReSIS, Antarctica)

These seasons overlap with Bedmap 3 campaign data.

OPR Catalog Bedmap 3 File
2009_Antarctica_TO_Gambit CRESIS_2009_AntarcticaTO_AIR_BM3
2009_Antarctica_TO CRESIS_2009_Thwaites_AIR_BM3
2013_Antarctica_Basler CRESIS_2013_Siple-Coast_AIR_BM3
2013_Antarctica_P3 NASA_2013_ICEBRIDGE_AIR_BM3
2014_Antarctica_DC8 NASA_2014_ICEBRIDGE_AIR_BM3
2016_Antarctica_DC8 NASA_2016_ICEBRIDGE_AIR_BM3
2017_Antarctica_Basler NASA_2017_ICEBRIDGE_AIR_BM3
2018_Antarctica_DC8 NASA_2018_ICEBRIDGE_AIR_BM3
2019_Antarctica_GV NASA_2019_ICEBRIDGE_AIR_BM3
  • 2009_Antarctica_TO_Gambit
  • 2009_Antarctica_TO
  • 2013_Antarctica_Basler
  • 2013_Antarctica_P3
  • 2014_Antarctica_DC8
  • 2016_Antarctica_DC8
  • 2017_Antarctica_Basler
  • 2018_Antarctica_DC8
  • 2019_Antarctica_GV

Tier 3 — Bedmap 1 Overlap

Bedmap 1 is a single CSV without campaign identifiers, confined to pre-2000 data. OPR seasons that could overlap:

  • 1993_Greenland_P3
  • 1995_Greenland_P3
  • 1996_Greenland_P3
  • 1997_Greenland_P3
  • 1998_Greenland_P3
  • 1999_Greenland_P3

Note: Bedmap 1 matching is significantly harder — no campaign IDs, incomplete variables, and purely spatial matching required. Catalogs are now reprocessed with morton indices; matching logic is a separate effort.


Tier 4 — UTIG Catalogs (Antarctica)

UTIG-operated Basler campaigns (BaslerJKB = HiCARS radar, BaslerMKB = later MKB radar). Several of these have bedmap overlap, annotated below.

Existing catalogs (reprocess)

OPR Catalog Bedmap Overlap
2008_Antarctica_BaslerJKB Bedmap 2: UTIG_2008_ICECAP_AIR_BM2
2016_Antarctica_BaslerJKB Bedmap 3: UTIG_2016_OLDICE_AIR_BM3
2017_Antarctica_BaslerJKB
2018_Antarctica_BaslerJKB
2022_Antarctica_BaslerMKB
2023_Antarctica_BaslerMKB
  • 2008_Antarctica_BaslerJKB (bedmap 2 overlap)
  • 2016_Antarctica_BaslerJKB (bedmap 3 overlap)
  • 2017_Antarctica_BaslerJKB
  • 2018_Antarctica_BaslerJKB
  • 2022_Antarctica_BaslerMKB
  • 2023_Antarctica_BaslerMKB

Future catalogs (pending initial processing)

These are known UTIG seasons that don't have STAC catalogs yet. They need to be processed before they can be reprocessed with morton indices.

Season Bedmap Overlap
2009_Antarctica_BaslerJKB Bedmap 3: UTIG_2009_Darwin-Hatherton_AIR_BM3
2010_Antarctica_BaslerJKB Bedmap 3: UTIG_2010_ICECAP_AIR_BM3
2011_Antarctica_BaslerJKB
2012_Antarctica_BaslerJKB
2014_Antarctica_BaslerMKB
2015_Antarctica_BaslerJKB Bedmap 3: UTIG_2015_EAGLE_AIR_BM3
  • 2009_Antarctica_BaslerJKB (pending processing; bedmap 3 overlap)
  • 2010_Antarctica_BaslerJKB (pending processing; bedmap 3 overlap)
  • 2011_Antarctica_BaslerJKB (pending processing)
  • 2012_Antarctica_BaslerJKB (pending processing)
  • 2014_Antarctica_BaslerMKB (pending processing)
  • 2015_Antarctica_BaslerJKB (pending processing; bedmap 3 overlap)

Tier 5 — Remaining CReSIS (Antarctica, no bedmap overlap)

These CReSIS Antarctic seasons don't have known bedmap overlap:

  • 2002_Antarctica_P3chile
  • 2011_Antarctica_TO
  • 2017_Antarctica_P3
  • 2018_Antarctica_Ground
  • 2024_Antarctica_GroundGHOST2

Tier 6 — Northern Hemisphere (CReSIS, AWI, DTU)

All Greenland catalogs. Bedmap is Antarctica-only, so no overlap.

CReSIS:

  • 2001_Greenland_P3
  • 2002_Greenland_P3
  • 2003_Greenland_P3
  • 2005_Greenland_TO
  • 2006_Greenland_TO
  • 2008_Greenland_TO
  • 2009_Greenland_TO
  • 2010_Greenland_DC8
  • 2011_Greenland_P3
  • 2012_Greenland_P3
  • 2013_Greenland_P3
  • 2014_Greenland_P3
  • 2015_Greenland_C130
  • 2016_Greenland_G1XB
  • 2016_Greenland_P3
  • 2017_Greenland_P3
  • 2018_Greenland_P3
  • 2019_Greenland_P3

AWI:

  • 2016_Greenland_Polar6

DTU:

  • 2016_Greenland_TOdtu

Related Issues

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions