From 84d7ac3780dc3eb7b5ae325e581658d20ad86158 Mon Sep 17 00:00:00 2001 From: Mason Yates Date: Wed, 26 Aug 2026 15:00:24 -0400 Subject: [PATCH 1/5] CUMULUS-5256 CHANGELOG update --- CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index fedfba64584..8e88b4f72a5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - **CUMULUS-5256** - Add orphan file cleanup to Iceberg tables + - NOTE: As of Aug 25, 2026 Iceberg replication is running in Prod 1, but the tables are out of sync for two reasons: + Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication + The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup. + After the PR is merged and a release is created subsequent PRs for the consolidated-cumulus repository and the CIRRUS-core repository will be needed as usual before a deployment is done. + + Following the deployment, the Iceberg tables will need to be destroyed using the `remove_iceberg_tables.py` script in the cumulus operators script repository, and the replication services restarted so that the tables get recreated and bootstrapped from the current contents of the RDS database. James Norton and Chris Durbin should provide support for this. After that the replication should keep up. - **CUMULUS-5012** - Adding a SECURITY.md file that contains NASA's official guidance on reporting vulnerabilities From 27e9d0586a46a2057aaf7abbf82d0a434d52c7db Mon Sep 17 00:00:00 2001 From: Mason Yates Date: Wed, 26 Aug 2026 15:02:53 -0400 Subject: [PATCH 2/5] CUMULUS-5256 formatting --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e88b4f72a5..08d56c22c28 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - **CUMULUS-5256** - Add orphan file cleanup to Iceberg tables - NOTE: As of Aug 25, 2026 Iceberg replication is running in Prod 1, but the tables are out of sync for two reasons: - Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication + Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication. The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup. After the PR is merged and a release is created subsequent PRs for the consolidated-cumulus repository and the CIRRUS-core repository will be needed as usual before a deployment is done. From 7507116698c7247dd652f88c7476ed3072cb70e9 Mon Sep 17 00:00:00 2001 From: Mason Yates Date: Wed, 26 Aug 2026 15:05:13 -0400 Subject: [PATCH 3/5] CUMULUS-5256 spacing --- CHANGELOG.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 08d56c22c28..88fb3bc4c7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,9 +11,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - **CUMULUS-5256** - Add orphan file cleanup to Iceberg tables - NOTE: As of Aug 25, 2026 Iceberg replication is running in Prod 1, but the tables are out of sync for two reasons: - Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication. - The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup. - After the PR is merged and a release is created subsequent PRs for the consolidated-cumulus repository and the CIRRUS-core repository will be needed as usual before a deployment is done. + Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing + block of time in the replication. + The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that + should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup. + After the PR is merged and a release is created subsequent PRs for the consolidated-cumulus repository and the CIRRUS-core + repository will be needed as usual before a deployment is done. Following the deployment, the Iceberg tables will need to be destroyed using the `remove_iceberg_tables.py` script in the cumulus operators script repository, and the replication services restarted so that the tables get recreated and bootstrapped from the current contents of the RDS database. James Norton and Chris Durbin should provide support for this. After that the replication should keep up. From 60f5a2e17d4033ee756280cd3f2e174bf4cb9628 Mon Sep 17 00:00:00 2001 From: Mason Yates Date: Wed, 26 Aug 2026 15:07:05 -0400 Subject: [PATCH 4/5] CUMULUS-5256 --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 88fb3bc4c7e..303963b5d15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - **CUMULUS-5256** - Add orphan file cleanup to Iceberg tables - NOTE: As of Aug 25, 2026 Iceberg replication is running in Prod 1, but the tables are out of sync for two reasons: - Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing + Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication. The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup. From e109696b49ca662091f99b9a8cc99cf0b45cab3c Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 26 Aug 2026 19:07:39 +0000 Subject: [PATCH 5/5] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 303963b5d15..88fb3bc4c7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/). - **CUMULUS-5256** - Add orphan file cleanup to Iceberg tables - NOTE: As of Aug 25, 2026 Iceberg replication is running in Prod 1, but the tables are out of sync for two reasons: - Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing + Iceberg replication was disabled for a while after the initial rollout due to conflicts with migrations, so there is a missing block of time in the replication. The batch size for the replication was initially set too low to keep up with traffic. This is corrected in an open PR that should be merged soon https://github.com/nasa/cumulus/pull/4572 . This PR also enables orphan file cleanup.