|
| 1 | +# Pre-Release Validation Checklist |
| 2 | + |
| 3 | +This checklist ensures SDK releases meet quality standards. **Complete ALL items before publishing to PyPI.** |
| 4 | + |
| 5 | +## Automated Checks (Run Script) |
| 6 | + |
| 7 | +```bash |
| 8 | +./scripts/pre-release-validation.sh |
| 9 | +``` |
| 10 | + |
| 11 | +This script runs all automated validations and reports pass/fail status. |
| 12 | + |
| 13 | +## Manual Checklist |
| 14 | + |
| 15 | +### 1. Version Management ✅ |
| 16 | +- [ ] Version bumped in `pyproject.toml` |
| 17 | +- [ ] Version bumped in `oilpriceapi/__init__.py` |
| 18 | +- [ ] Version updated in `CHANGELOG.md` |
| 19 | +- [ ] CHANGELOG has comprehensive release notes |
| 20 | +- [ ] No `UNRELEASED` sections in CHANGELOG |
| 21 | + |
| 22 | +### 2. Code Quality ✅ |
| 23 | +- [ ] All unit tests pass (`pytest tests/unit -v`) |
| 24 | +- [ ] All integration tests pass (`pytest tests/integration -v`) |
| 25 | +- [ ] Test coverage ≥ 80% (`pytest --cov`) |
| 26 | +- [ ] No linting errors (`ruff check .`) |
| 27 | +- [ ] No type errors (`mypy oilpriceapi`) |
| 28 | +- [ ] Code formatted (`black --check .`) |
| 29 | + |
| 30 | +### 3. Integration Validation ✅ |
| 31 | +- [ ] Historical endpoint tests pass (catches timeout bug) |
| 32 | +- [ ] Performance baselines met: |
| 33 | + - 1-week queries: <30s |
| 34 | + - 1-month queries: <60s |
| 35 | + - 1-year queries: <120s |
| 36 | +- [ ] All commodities tested |
| 37 | +- [ ] Error handling verified |
| 38 | + |
| 39 | +### 4. Documentation ✅ |
| 40 | +- [ ] README.md updated with new features |
| 41 | +- [ ] API documentation current |
| 42 | +- [ ] Code examples work |
| 43 | +- [ ] Migration guide included (if breaking changes) |
| 44 | +- [ ] Docstrings updated for new/changed functions |
| 45 | + |
| 46 | +### 5. Build & Package ✅ |
| 47 | +- [ ] Clean build: `rm -rf dist/ build/ *.egg-info` |
| 48 | +- [ ] Build succeeds: `python -m build` |
| 49 | +- [ ] Wheel created: `ls dist/*.whl` |
| 50 | +- [ ] Source distribution created: `ls dist/*.tar.gz` |
| 51 | +- [ ] Package installs locally: `pip install dist/*.whl` |
| 52 | +- [ ] Imports work: `python -c "import oilpriceapi; print(oilpriceapi.__version__)"` |
| 53 | + |
| 54 | +### 6. Backwards Compatibility ✅ |
| 55 | +- [ ] No breaking changes (or documented in CHANGELOG) |
| 56 | +- [ ] Existing code samples still work |
| 57 | +- [ ] Deprecations properly warned |
| 58 | +- [ ] Migration guide provided (if needed) |
| 59 | + |
| 60 | +### 7. Security ✅ |
| 61 | +- [ ] No hardcoded credentials |
| 62 | +- [ ] No secrets in code or tests |
| 63 | +- [ ] Dependencies scanned: `pip-audit` |
| 64 | +- [ ] SECURITY.md reviewed and current |
| 65 | + |
| 66 | +### 8. Git & GitHub ✅ |
| 67 | +- [ ] All changes committed |
| 68 | +- [ ] Commit message follows convention |
| 69 | +- [ ] Git tag created: `git tag v1.X.Y` |
| 70 | +- [ ] Tag pushed: `git push --tags` |
| 71 | +- [ ] No uncommitted changes |
| 72 | + |
| 73 | +### 9. PyPI Publishing ✅ |
| 74 | +- [ ] Test PyPI upload works: `twine upload --repository testpypi dist/*` |
| 75 | +- [ ] Test installation from TestPyPI |
| 76 | +- [ ] Production PyPI upload: `twine upload dist/*` |
| 77 | +- [ ] Verify on PyPI: https://pypi.org/project/oilpriceapi/ |
| 78 | +- [ ] Installation works: `pip install --upgrade oilpriceapi` |
| 79 | + |
| 80 | +### 10. Post-Release ✅ |
| 81 | +- [ ] GitHub release created with notes |
| 82 | +- [ ] Documentation site updated |
| 83 | +- [ ] Announcement prepared (if major release) |
| 84 | +- [ ] Monitor error tracking for 24 hours |
| 85 | +- [ ] Check PyPI download stats |
| 86 | + |
| 87 | +## What Would Have Caught the v1.4.1 Bug? |
| 88 | + |
| 89 | +The historical timeout bug (reported by idan@comity.ai) would have been caught by: |
| 90 | + |
| 91 | +1. ✅ **Integration Tests** (`tests/integration/test_historical_endpoints.py`) |
| 92 | + - `test_7_day_query_uses_past_week_endpoint` - Would fail (67s timeout) |
| 93 | + - `test_365_day_query_uses_past_year_endpoint` - Would fail (30s timeout) |
| 94 | + |
| 95 | +2. ✅ **Performance Baselines** (`TestHistoricalPerformanceBaselines`) |
| 96 | + - All tests would fail with timeouts |
| 97 | + |
| 98 | +3. ✅ **Pre-Release Script** (`scripts/pre-release-validation.sh`) |
| 99 | + - Integration tests would fail |
| 100 | + - Script would prevent release |
| 101 | + |
| 102 | +## Automation Script |
| 103 | + |
| 104 | +The `pre-release-validation.sh` script automates items 1-7: |
| 105 | + |
| 106 | +```bash |
| 107 | +# Run full validation |
| 108 | +./scripts/pre-release-validation.sh |
| 109 | + |
| 110 | +# Run with verbose output |
| 111 | +./scripts/pre-release-validation.sh --verbose |
| 112 | + |
| 113 | +# Skip slow tests (for quick checks) |
| 114 | +./scripts/pre-release-validation.sh --skip-slow |
| 115 | +``` |
| 116 | + |
| 117 | +**Exit Codes:** |
| 118 | +- `0` - All checks passed, ready to release |
| 119 | +- `1` - One or more checks failed, DO NOT release |
| 120 | + |
| 121 | +## Emergency Release Procedure |
| 122 | + |
| 123 | +If critical bug requires immediate release: |
| 124 | + |
| 125 | +1. Run minimum validation: |
| 126 | + ```bash |
| 127 | + pytest tests/unit -v --tb=short |
| 128 | + pytest tests/integration/test_historical_endpoints.py -v |
| 129 | + ``` |
| 130 | + |
| 131 | +2. Verify the specific fix works |
| 132 | + |
| 133 | +3. Document in CHANGELOG as emergency release |
| 134 | + |
| 135 | +4. **Still run full validation after emergency release** |
| 136 | + |
| 137 | +## Failed Validation - What to Do |
| 138 | + |
| 139 | +### Tests Failed |
| 140 | +1. Fix failing tests |
| 141 | +2. Re-run full validation |
| 142 | +3. Update CHANGELOG if fixes required code changes |
| 143 | + |
| 144 | +### Performance Regression |
| 145 | +1. Investigate using profiling |
| 146 | +2. Fix performance issue |
| 147 | +3. Re-establish baseline |
| 148 | + |
| 149 | +### Documentation Missing |
| 150 | +1. Update documentation |
| 151 | +2. Add code examples |
| 152 | +3. Test examples actually work |
| 153 | + |
| 154 | +### Build Failed |
| 155 | +1. Check `pyproject.toml` for errors |
| 156 | +2. Verify all files included in manifest |
| 157 | +3. Test clean build: `rm -rf dist/ && python -m build` |
| 158 | + |
| 159 | +## Version History |
| 160 | + |
| 161 | +| Version | Date | Validator | Result | Notes | |
| 162 | +|---------|------|-----------|--------|-------| |
| 163 | +| v1.4.2 | 2025-12-16 | Manual | ✅ Pass | Fixed historical timeout bug | |
| 164 | +| v1.4.1 | 2025-12-15 | None | ❌ Fail | Historical timeout bug shipped | |
| 165 | + |
| 166 | +*Note: v1.4.1 did not use this checklist, which is why the bug reached production.* |
| 167 | + |
| 168 | +## Integration with CI/CD |
| 169 | + |
| 170 | +### GitHub Actions (Recommended) |
| 171 | + |
| 172 | +```yaml |
| 173 | +name: Pre-Release Validation |
| 174 | + |
| 175 | +on: |
| 176 | + push: |
| 177 | + tags: |
| 178 | + - 'v*' |
| 179 | + |
| 180 | +jobs: |
| 181 | + validate: |
| 182 | + runs-on: ubuntu-latest |
| 183 | + steps: |
| 184 | + - uses: actions/checkout@v3 |
| 185 | + |
| 186 | + - name: Set up Python |
| 187 | + uses: actions/setup-python@v4 |
| 188 | + with: |
| 189 | + python-version: '3.11' |
| 190 | + |
| 191 | + - name: Install dependencies |
| 192 | + run: | |
| 193 | + pip install -e ".[dev]" |
| 194 | + pip install twine pip-audit |
| 195 | +
|
| 196 | + - name: Run pre-release validation |
| 197 | + env: |
| 198 | + OILPRICEAPI_KEY: ${{ secrets.OILPRICEAPI_KEY }} |
| 199 | + run: ./scripts/pre-release-validation.sh |
| 200 | + |
| 201 | + - name: Build package |
| 202 | + if: success() |
| 203 | + run: python -m build |
| 204 | + |
| 205 | + - name: Publish to PyPI |
| 206 | + if: success() |
| 207 | + env: |
| 208 | + TWINE_USERNAME: __token__ |
| 209 | + TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} |
| 210 | + run: twine upload dist/* |
| 211 | +``` |
| 212 | +
|
| 213 | +## Contact |
| 214 | +
|
| 215 | +Questions about the validation process: |
| 216 | +- GitHub Issues: https://github.com/OilpriceAPI/python-sdk/issues |
| 217 | +- Email: support@oilpriceapi.com |
| 218 | +
|
| 219 | +## Related Issues |
| 220 | +
|
| 221 | +- [#20](https://github.com/OilpriceAPI/python-sdk/issues/20) - Integration tests |
| 222 | +- [#21](https://github.com/OilpriceAPI/python-sdk/issues/21) - Performance baselines |
| 223 | +- [#22](https://github.com/OilpriceAPI/python-sdk/issues/22) - Pre-release validation (this document) |
0 commit comments