Conversation
pipelines and test config
updating with fallback values
new setup structure
adding test sugar
pairing down sfts
| @@ -0,0 +1,166 @@ | |||
| name: Build, Test and Publish Pipeline | |||
|
|
|||
There was a problem hiding this comment.
I don't think we're going to need this pipeline. We don't want anything to run automatically and I don't think publishing and testing will be done in one swoop and we don't want to do it automatically.
| runs-on: ubuntu-latest | ||
| # Requires a "pypi-review" environment configured in GitHub repo settings | ||
| # with required reviewers — the job pauses here until a reviewer approves. | ||
| environment: pypi-review |
There was a problem hiding this comment.
I don't think we're going to need this. There's a very small number of people who will have access to this and I don't want to not be able to publish because I can't find one of the other folks that can help me approve - Kurt mentioned that you can work around that, but I don't think we need to have this in the first place.
| description: 'Additional regression_test.py command-line options' | ||
| required: false | ||
| default: '--scons --use-dlls' | ||
| copy_binaries: |
There was a problem hiding this comment.
default should include --all-outputs (it's new and wasn't in previous scripts)
| inputs: | ||
| regression_test_cores: | ||
| description: 'Number of cores to run regression tests with' | ||
| required: false |
There was a problem hiding this comment.
maybe add something along the lines of "4 is max for gha" so folks aren't requesting anything higher.
There was a problem hiding this comment.
I don't know that we need that statement. You can ask for 8 cores on a 4 core machine. EMOD will just create 8 processes. It will run slower, but it should still run.
| type: boolean | ||
| default: false | ||
| run_generic: | ||
| description: 'Run Generic suite' |
There was a problem hiding this comment.
ooohhhh are these checkboxes? Beautiful. Thank you.
| ../build/x64/Release/Eradication/Eradication \ | ||
| --hidegraphs --config-constraints Num_Cores:${{ inputs.regression_test_cores || '4' }} \ | ||
| ${{ inputs.regression_test_options || '--scons --use-dlls' }} \ | ||
| --local \ |
There was a problem hiding this comment.
--add-outputs in addition to --scons --use-dlls
.github/workflows/sft_tests.yml
Outdated
| required: false | ||
| default: '4' | ||
| regression_test_options: | ||
| description: 'Additional regression_test.py command-line options' |
There was a problem hiding this comment.
"Additional SFT tests command-line options" (this one doesn't need --all-outputs ever)
Bridenbecker
left a comment
Bridenbecker
left a comment
There was a problem hiding this comment.
This is great!! Thank you guys for working on this.
- Test Artifacts - Love the regression test output folders being zipped up as artifacts. This allows us to get the data, compare it to what we had before, and see if the difference is expected or not.
- emod-all-build - I may not understand this, but I don't think we need an artifact that contains an All Disease build that is separate from the binaries that went with the tests.
- No On Push to Main - For the foreseeable future, I do not think we want any workflow process to start automatically based on a commit or PR. We to select want is run manually.
- No Pypi-Review - I don't think we want the review. We have it on another package and it seems more annoying than helpful.
Again, this is great. Thanks.
I believe I've made all the requested changes here, but please let me know if there's anything I missed or can make better.
|
4 participants
Introduces three GitHub Actions pipelines for building, testing, and publishing EMOD, along with a workflow README.
Build, Test and Publish Pipeline (e2e_parallel_pipeline.yml)
The primary integration pipeline. Triggers automatically on push to main or manually via workflow_dispatch. Builds EMOD for all disease types, then runs 5 regression test suites (Generic, HIV, Malaria, STI, Vector) in parallel. On success, packages a versioned release tarball and publishes emod-common, emod-hiv, and emod-malaria to PyPI — each gated behind a manual reviewer approval step using GitHub environment protection rules.
Regression Tests (regression_tests.yml)
Manual pipeline for targeted regression testing. Supports selecting any subset of the 5 disease suites via boolean inputs, with an option to skip the build step and reuse the most recent artifact. Test results and optionally the compiled binaries are uploaded as artifacts.
Science Feature Tests (sft_tests.yml)
Manual pipeline for running science feature test (SFT) suites. Always performs a fresh build with --TestSugar enabled. Supports selecting any subset of 5 SFT suites (Generic, HIV, Malaria, STI, Vector Science) via boolean inputs.
Workflow README (.github/workflows/README.md)
Adds a reference document covering all pipelines: job graphs, inputs, artifact outputs, suite selection behavior, and setup requirements for the PyPI publishing environments.