porting GDASApp to native AWS#2127
Conversation
…to feature/on-aws-ec2
There was a problem hiding this comment.
Pull request overview
This PR adds initial platform detection and module environment support so GDASApp can be built on native AWS EC2 (as aws-ec2), aligning with the broader Global-Workflow porting effort.
Changes:
- Added
aws-ec2machine detection via hostname/path heuristics. - Extended module initialization to support Lmod on AWS EC2.
- Introduced a new
GDAS/aws-ec2.intelmodulefile and enabledaws-ec2as a supportedbuild.shtarget.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
ush/module-setup.sh |
Adds an aws-ec2 branch to initialize Lmod and purge modules. |
ush/detect_machine.sh |
Detects AWS EC2 hosts and sets MACHINE_ID=aws-ec2. |
modulefiles/GDAS/aws-ec2.intel.lua |
New modulefile to load the AWS EC2 Intel/oneAPI environment and set runtime/build env vars. |
build.sh |
Allows aws-ec2 as a build target and adjusts CRTM dummy directory logic for this platform. |
| setenv('MPIEXEC_EXEC', mpiexec) | ||
| setenv('MPIEXEC_NPROC', mpinproc) | ||
|
|
||
| setenv("CRTM_FIX", "/lustre/sharedGWdata/fix/crtm/v2.4.0.2") |
There was a problem hiding this comment.
The AWS modulefile does not set GDASAPP_TESTDATA / GDASAPP_UNIT_TEST_DATA_PATH (and only sets CRTM_FIX). Many existing workflow/test scripts in this repo rely on GDASAPP_TESTDATA being defined by the loaded GDAS module, consistent with other platform modulefiles, so this will break those scripts on aws-ec2. Please add the missing env vars (and any other required shared-data paths) for AWS, mirroring the other modulefiles/GDAS/*.intel.lua files.
| setenv("CRTM_FIX", "/lustre/sharedGWdata/fix/crtm/v2.4.0.2") | |
| local shared_gwdata = "/lustre/sharedGWdata" | |
| local gdasapp_testdata = pathJoin(shared_gwdata, "tests/gdasapp") | |
| setenv("GDASAPP_TESTDATA", gdasapp_testdata) | |
| setenv("GDASAPP_UNIT_TEST_DATA_PATH", gdasapp_testdata) | |
| setenv("CRTM_FIX", pathJoin(shared_gwdata, "fix/crtm/v2.4.0.2")) |
| load("stack-oneapi/2024.2.1") | ||
| load("stack-intel-oneapi-mpi/2021.13") |
There was a problem hiding this comment.
This modulefile loads stack-oneapi and stack-intel-oneapi-mpi twice: first using stack_oneapi_ver/stack_impi_ver, then immediately again with hard-coded versions. This can override user-provided version overrides and makes the effective environment unclear. Please keep only one mechanism (either env-var-driven pathJoin loads or the fixed load("stack-oneapi/...") lines).
| load("stack-oneapi/2024.2.1") | |
| load("stack-intel-oneapi-mpi/2021.13") |
| setenv("I_MPI_CC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icx") | ||
| setenv("I_MPI_CXX","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icpx") | ||
|
|
||
| setenv("FC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/ifort") | ||
| setenv("CC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icx") | ||
| setenv("CXX","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icpx") |
There was a problem hiding this comment.
The compiler variables (FC/CC/CXX and I_MPI_*) are set to full Spack install paths that include a concrete hash. This is very brittle (any rebuild/relocation of the stack will break the modulefile) and differs from other platform modulefiles that use wrapper compilers like mpiicc/mpiifort or generic cc/ftn. Consider setting compilers to the MPI wrapper commands provided by the loaded modules (or deriving paths from stable env vars) instead of hard-coding hashed install paths.
| setenv("I_MPI_CC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icx") | |
| setenv("I_MPI_CXX","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icpx") | |
| setenv("FC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/ifort") | |
| setenv("CC","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icx") | |
| setenv("CXX","/opt/spack-stack/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-11.4.0/intel-oneapi-compilers-2024.2.1-j3owv3rsoh7igqvwmvp7ez6i3ozeayvs/compiler/latest/bin/icpx") | |
| setenv("I_MPI_CC","mpiicc") | |
| setenv("I_MPI_CXX","mpiicpc") | |
| setenv("FC","mpiifort") | |
| setenv("CC","mpiicc") | |
| setenv("CXX","mpiicpc") |
This removes the `logit` decorator from the `prep_ocean_obs.py` `__init__` method to reduce debug output (preventing the entire environment from being written to the log).
Description
In order for more general users to use GDASApp, we want to make GDASApp being able to compile on native AWS (as a new computing platform, or machine). This is a part of making Global-Workflow porting to native AWS.
Companion PRs
None
Issues
Resolves #2126
Automated CI tests to run in Global Workflow