Skip to content

Notification updates, fix get_globus_url, small updates#1236

Merged
jpuerto-psc merged 16 commits intodevelfrom
phillips/notify_updates
Apr 30, 2026
Merged

Notification updates, fix get_globus_url, small updates#1236
jpuerto-psc merged 16 commits intodevelfrom
phillips/notify_updates

Conversation

@gesinaphillips
Copy link
Copy Markdown
Collaborator

@gesinaphillips gesinaphillips commented Apr 1, 2026

Major changes

  • Notification updates:
    • Clarified error email language for when pipeline fails before derived dataset is created; added pipeline name
    • Added pipeline name to Slack messages
    • Use test Slack channel & config override values for email contacts automatically when not on PROD
  • Fix get_globus_url:
    • Switched to appropriate API endpoint to retrieve Globus URL

Bugfixes/updates

  • Made getting log directory safer by checking for run_id first
  • Updated EntityUpdater reindex param to default to reindex-priority=3 but also allow setting reindex=<1, 2> or reindex=True (transformed into reindex-priority=1*)
    * Is this the right call?

Refactoring

  • Removed Union / Optional typing
  • Added some missing return typing
  • utils updates:
    • EntityUpdater._enums_to_lowercase -> utils.enums_to_lowercase
    • status_utils.get_env -> utils.get_env
    • Added utils.format_multiline to help with formatting long messages
  • Rearranged status_utils.py
    • Consolidated Statuses.valid_str into get_status_enum
  • Added additional docstrings

@gesinaphillips gesinaphillips force-pushed the phillips/notify_updates branch from b249dde to 29aa6d0 Compare April 27, 2026 20:55
Comment thread src/ingest-pipeline/airflow/dags/status_change/status_manager.py Outdated
return self.status_str.title()

@staticmethod
def valid_str(status: Union[str, Statuses]) -> str:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consolidated this logic into get_status_enum

return response.json()


def get_globus_url(entity_data: dict, token: str) -> str:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New get_globus_url func

Comment thread src/ingest-pipeline/airflow/dags/utils.py
@gesinaphillips gesinaphillips marked this pull request as ready for review April 28, 2026 18:42
).update()

@staticmethod
def _enums_to_lowercase(data: Any) -> Any:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to utils.py



def get_status_enum(entity_type: str, status: Statuses | str, uuid: str) -> Statuses:
def get_status_enum(entity_type: str, status: Statuses | str) -> Statuses:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated func, incorporates validation logic from Statuses.valid_str

SENNET = ("sennet", "SenNet")


globus_dirs = {
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deprecated by new status_utils.get_globus_url

@jpuerto-psc jpuerto-psc merged commit 27d748e into devel Apr 30, 2026
2 checks passed
@jpuerto-psc jpuerto-psc deleted the phillips/notify_updates branch April 30, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants