-
Notifications
You must be signed in to change notification settings - Fork 40
CI: test cache, bump tested version, global config #591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
451d8ac
update cache
d-chambers 14e7237
update caching action
d-chambers 1ab7e63
make actions global var
d-chambers 606a064
Merge branch 'master' into cache_fix
d-chambers d39bd0e
try to fix doc build, define matricies in one place
d-chambers b085688
ensure pyyaml is installed
d-chambers d2e19fd
add py3.14
d-chambers File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: "Cache Test Data" | ||
| description: "Caches DASCore test data using Pooch cache directory" | ||
| inputs: | ||
| cache-number: | ||
| description: "Cache number. Use != 1 to reset data cache" | ||
| required: false | ||
| default: "1" | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: verify prerequisites | ||
| shell: bash -el {0} | ||
| run: | | ||
| if ! command -v python &> /dev/null; then | ||
| echo "❌ Error: Python is not installed or not in PATH" | ||
| echo "This action requires Python to be installed first." | ||
| exit 1 | ||
| fi | ||
| if ! python -c "import pooch" 2>/dev/null; then | ||
| echo "❌ Error: Pooch is not installed" | ||
| echo "This action requires Pooch to be installed (pip install pooch or include in dependencies)." | ||
| exit 1 | ||
| fi | ||
| echo "✅ Prerequisites verified: Python and Pooch are available" | ||
|
|
||
| - name: get data registry hash | ||
| shell: bash -el {0} | ||
| run: | | ||
| echo "DATA_REGISTRY_HASH=$(python -c "import pooch; print(pooch.file_hash('dascore/data_registry.txt'))")" >> $GITHUB_ENV | ||
|
|
||
| - name: get data cache path | ||
| shell: bash -el {0} | ||
| run: | | ||
| echo "DATA_CACHE_PATH=$(python -c "import pooch; print(pooch.os_cache('dascore'))")" >> $GITHUB_ENV | ||
|
|
||
| - name: cache test data | ||
| uses: actions/cache@v4 | ||
| id: cache-test-data | ||
| with: | ||
| enableCrossOsArchive: true | ||
| path: ${{ env.DATA_CACHE_PATH }} | ||
| key: data-${{ env.DATA_REGISTRY_HASH }}-${{ inputs.cache-number }} | ||
| restore-keys: | | ||
| data-${{ env.DATA_REGISTRY_HASH }}- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: "Load Shared Variables" | ||
| description: "Loads shared variables from .github/shared-vars.yml and sets them as environment variables and outputs" | ||
|
|
||
| outputs: | ||
| test-matrix: | ||
| description: "Python version matrix for full tests" | ||
| value: ${{ steps.load.outputs.test-matrix }} | ||
| min-deps-matrix: | ||
| description: "Python version matrix for minimum dependency tests" | ||
| value: ${{ steps.load.outputs.min-deps-matrix }} | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: install pyyaml | ||
| shell: bash | ||
| run: python -m pip install --quiet pyyaml | ||
|
|
||
| - name: load shared variables | ||
| id: load | ||
| shell: bash | ||
| run: | | ||
| # Read shared-vars.yml and set environment variables | ||
| python << 'EOF' | ||
| import yaml | ||
| import os | ||
| import json | ||
|
|
||
| def flatten_dict(d, parent_key='', sep='_'): | ||
| """Recursively flatten nested dictionary into env var format.""" | ||
| items = [] | ||
| for k, v in d.items(): | ||
| new_key = f"{parent_key}{sep}{k}".upper() if parent_key else k.upper() | ||
| if isinstance(v, dict): | ||
| items.extend(flatten_dict(v, new_key, sep=sep).items()) | ||
| elif isinstance(v, list): | ||
| # Convert lists to JSON strings for use in workflows | ||
| items.append((new_key, json.dumps(v))) | ||
| else: | ||
| items.append((new_key, str(v))) | ||
| return dict(items) | ||
|
|
||
| # Load shared variables file | ||
| with open('.github/shared-vars.yml', 'r') as f: | ||
| config = yaml.safe_load(f) | ||
|
|
||
| # Flatten all variables | ||
| env_vars = flatten_dict(config) | ||
|
|
||
| # Write to GITHUB_ENV | ||
| with open(os.environ['GITHUB_ENV'], 'a') as env_file: | ||
| for key, value in sorted(env_vars.items()): | ||
| env_file.write(f"{key}={value}\n") | ||
| print(f"✅ Set {key}={value}") | ||
|
|
||
| # Also output specific matrices for job outputs | ||
| python_config = config.get('python', {}) | ||
| test_matrix = python_config.get('test_matrix', []) | ||
| min_deps_matrix = python_config.get('min_deps_matrix', []) | ||
|
|
||
| # Write to GITHUB_OUTPUT for action outputs | ||
| with open(os.environ['GITHUB_OUTPUT'], 'a') as output_file: | ||
| output_file.write(f"test-matrix={json.dumps(test_matrix)}\n") | ||
| output_file.write(f"min-deps-matrix={json.dumps(min_deps_matrix)}\n") | ||
|
|
||
| print(f"📋 Output test-matrix: {test_matrix}") | ||
| print(f"📋 Output min-deps-matrix: {min_deps_matrix}") | ||
| EOF | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,7 +2,6 @@ name: dascore | |
| channels: | ||
| - conda-forge | ||
| dependencies: | ||
| - python=3.12 | ||
| - pytest | ||
| - pydantic>2.0 | ||
| - pip | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Shared variables used across all GitHub Actions workflows | ||
| # Update these values to propagate changes to all workflows | ||
|
|
||
| python: | ||
| # Default Python version for most workflows (docs, linting, coverage, etc.) | ||
| default: "3.13" | ||
|
|
||
| # Full test matrix versions (runtests.yml) | ||
| test_matrix: | ||
| - "3.11" | ||
| - "3.12" | ||
| - "3.13" | ||
| - "3.14" | ||
|
|
||
| # Minimum dependency test matrix versions (run_min_dep_tests.yml) | ||
| min_deps_matrix: | ||
| - "3.13" | ||
| - "3.14" | ||
|
|
||
| # Add other shared variables here as needed | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.