Add StreamFlow benchmark suite for remotepath module#963
Add StreamFlow benchmark suite for remotepath module#963GlassOfWhiskey wants to merge 2 commits intomasterfrom
remotepath module#963Conversation
|
|
||
|
|
||
| @pytest_asyncio.fixture(scope="session") | ||
| async def location(context: StreamFlowContext, deployment: str) -> ExecutionLocation: |
Check notice
Code scanning / CodeQL
Explicit returns mixed with implicit (fall through) returns Note test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 month ago
In general, to fix “explicit returns mixed with implicit returns” in a function that is supposed to return a value, ensure that every control-flow path ends with an explicit return, even if it returns None. Here, the location fixture has one code path (the try block) that returns an ExecutionLocation, while the except block only calls pytest.skip and implicitly falls off the end. Although pytest.skip raises, static analysis expects an explicit return.
The best fix without changing existing functionality is to keep the function signature as-is and add an explicit return statement in the except block after pytest.skip. Because pytest.skip raises, the new return is not executed at runtime but makes it clear to tools and readers that this branch does not yield a meaningful ExecutionLocation. To keep type checking satisfied without altering the return type annotation, we can use cast(ExecutionLocation, None) when returning, leveraging the already imported cast and ExecutionLocation. Concretely, in benchmark/conftest.py, within the location fixture’s except block (lines 125–128), add return cast(ExecutionLocation, None) on the next line. No new imports or other code changes are needed.
| @@ -126,8 +126,8 @@ | ||
| return await get_location(context, deployment) | ||
| except Exception as e: | ||
| pytest.skip(f"Deployment {deployment} not available: {e}") | ||
| return cast(ExecutionLocation, None) | ||
|
|
||
|
|
||
| @pytest.fixture(scope="session") | ||
| def connector_type(context: StreamFlowContext, location: ExecutionLocation) -> str: | ||
| """ |
1a36223 to
da71f83
Compare
remotepath module
da71f83 to
236f4c2
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #963 +/- ##
==========================================
+ Coverage 74.13% 74.14% +0.01%
==========================================
Files 91 91
Lines 12471 12471
Branches 2180 2180
==========================================
+ Hits 9245 9247 +2
+ Misses 2712 2711 -1
+ Partials 514 513 -1 ☔ View full report in Codecov by Sentry. |
1e088e3 to
c0c9df6
Compare
65dea68 to
c0c9df6
Compare
This commit adds and `AGENTS.md` file to instruct AI agents when implementing new StreamFlow features.
Add comprehensive benchmark suite to monitor performance of StreamFlow remotepath operations across different deployment connectors (local, Docker, Kubernetes, Singularity, SSH). This enables tracking performance improvements and regressions in critical code paths. Features: - 16 benchmark tests covering file queries, I/O, filesystem operations, and directory traversal - Support for all StreamFlow connectors with deployment selection flags (`--local`, `--remote`, `--deploys`) - Proper async handling with setup/teardown isolation - Resource leak prevention using root directory cleanup pattern - GitHub Actions CI integration for automated benchmarking - Clean benchmark output grouped by deployment type New files: - `benchmark/conftest.py`: pytest configuration with deployment parametrization and fixture management - `benchmark/remotepath.py`: 16 benchmark tests for StreamFlowPath operations (exists, mkdir, read/write, glob, walk, etc.) - `benchmark/utils.py`: Helper functions for async benchmarking and test file generation Code changes: - `streamflow/data/remotepath.py`: Add `_make_child_relpath()` method to support path joining operations needed by benchmarks
c0c9df6 to
0fa7792
Compare
This commit adds comprehensive benchmark suite to monitor performance of StreamFlow remotepath operations across different deployment connectors (local, Docker, Kubernetes, Singularity, SSH). This enables tracking performance improvements and regressions in critical code paths.
Features:
--local,--remote,--deploys)New files:
benchmark/conftest.py: pytest configuration with deployment parametrization and fixture managementbenchmark/remotepath.py: 16 benchmark tests for StreamFlowPath operations (exists, mkdir, read/write, glob, walk, etc.)benchmark/utils.py: Helper functions for async benchmarking and test file generationCode changes:
streamflow/data/remotepath.py: Add_make_child_relpath()method to support path joining operations needed by benchmarks