Skip to content

Add StreamFlow benchmark suite for remotepath module#963

Draft
GlassOfWhiskey wants to merge 2 commits intomasterfrom
streamflow-benchmarks
Draft

Add StreamFlow benchmark suite for remotepath module#963
GlassOfWhiskey wants to merge 2 commits intomasterfrom
streamflow-benchmarks

Conversation

@GlassOfWhiskey
Copy link
Copy Markdown
Member

@GlassOfWhiskey GlassOfWhiskey commented Feb 22, 2026

This commit adds comprehensive benchmark suite to monitor performance of StreamFlow remotepath operations across different deployment connectors (local, Docker, Kubernetes, Singularity, SSH). This enables tracking performance improvements and regressions in critical code paths.

Features:

  • 16 benchmark tests covering file queries, I/O, filesystem operations, and directory traversal
  • Support for all StreamFlow connectors with deployment selection flags (--local, --remote, --deploys)
  • Proper async handling with setup/teardown isolation
  • Resource leak prevention using root directory cleanup pattern
  • GitHub Actions CI integration for automated benchmarking
  • Clean benchmark output grouped by deployment type

New files:

  • benchmark/conftest.py: pytest configuration with deployment parametrization and fixture management
  • benchmark/remotepath.py: 16 benchmark tests for StreamFlowPath operations (exists, mkdir, read/write, glob, walk, etc.)
  • benchmark/utils.py: Helper functions for async benchmarking and test file generation

Code changes:

  • streamflow/data/remotepath.py: Add _make_child_relpath() method to support path joining operations needed by benchmarks

@GlassOfWhiskey GlassOfWhiskey changed the title Streamflow benchmarks Added StreamFlow benchmark suite Feb 22, 2026


@pytest_asyncio.fixture(scope="session")
async def location(context: StreamFlowContext, deployment: str) -> ExecutionLocation:

Check notice

Code scanning / CodeQL

Explicit returns mixed with implicit (fall through) returns Note test

Mixing implicit and explicit returns may indicate an error, as implicit returns always return None.

Copilot Autofix

AI about 1 month ago

In general, to fix “explicit returns mixed with implicit returns” in a function that is supposed to return a value, ensure that every control-flow path ends with an explicit return, even if it returns None. Here, the location fixture has one code path (the try block) that returns an ExecutionLocation, while the except block only calls pytest.skip and implicitly falls off the end. Although pytest.skip raises, static analysis expects an explicit return.

The best fix without changing existing functionality is to keep the function signature as-is and add an explicit return statement in the except block after pytest.skip. Because pytest.skip raises, the new return is not executed at runtime but makes it clear to tools and readers that this branch does not yield a meaningful ExecutionLocation. To keep type checking satisfied without altering the return type annotation, we can use cast(ExecutionLocation, None) when returning, leveraging the already imported cast and ExecutionLocation. Concretely, in benchmark/conftest.py, within the location fixture’s except block (lines 125–128), add return cast(ExecutionLocation, None) on the next line. No new imports or other code changes are needed.

Suggested changeset 1
benchmark/conftest.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/benchmark/conftest.py b/benchmark/conftest.py
--- a/benchmark/conftest.py
+++ b/benchmark/conftest.py
@@ -126,8 +126,8 @@
         return await get_location(context, deployment)
     except Exception as e:
         pytest.skip(f"Deployment {deployment} not available: {e}")
+        return cast(ExecutionLocation, None)
 
-
 @pytest.fixture(scope="session")
 def connector_type(context: StreamFlowContext, location: ExecutionLocation) -> str:
     """
EOF
@@ -126,8 +126,8 @@
return await get_location(context, deployment)
except Exception as e:
pytest.skip(f"Deployment {deployment} not available: {e}")
return cast(ExecutionLocation, None)


@pytest.fixture(scope="session")
def connector_type(context: StreamFlowContext, location: ExecutionLocation) -> str:
"""
Copilot is powered by AI and may make mistakes. Always verify output.
@GlassOfWhiskey GlassOfWhiskey changed the title Added StreamFlow benchmark suite Add StreamFlow benchmark suite for remotepath module Feb 22, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.14%. Comparing base (b963a1e) to head (0fa7792).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #963      +/-   ##
==========================================
+ Coverage   74.13%   74.14%   +0.01%     
==========================================
  Files          91       91              
  Lines       12471    12471              
  Branches     2180     2180              
==========================================
+ Hits         9245     9247       +2     
+ Misses       2712     2711       -1     
+ Partials      514      513       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@GlassOfWhiskey GlassOfWhiskey force-pushed the streamflow-benchmarks branch 7 times, most recently from 1e088e3 to c0c9df6 Compare March 1, 2026 08:06
@GlassOfWhiskey GlassOfWhiskey force-pushed the streamflow-benchmarks branch from 65dea68 to c0c9df6 Compare March 24, 2026 10:44
This commit adds and `AGENTS.md` file to instruct AI agents when
implementing new StreamFlow features.
Add comprehensive benchmark suite to monitor performance of StreamFlow
remotepath operations across different deployment connectors (local,
Docker, Kubernetes, Singularity, SSH). This enables tracking performance
improvements and regressions in critical code paths.

Features:
- 16 benchmark tests covering file queries, I/O, filesystem operations,
  and directory traversal
- Support for all StreamFlow connectors with deployment selection flags
  (`--local`, `--remote`, `--deploys`)
- Proper async handling with setup/teardown isolation
- Resource leak prevention using root directory cleanup pattern
- GitHub Actions CI integration for automated benchmarking
- Clean benchmark output grouped by deployment type

New files:
- `benchmark/conftest.py`: pytest configuration with deployment
  parametrization and fixture management
- `benchmark/remotepath.py`: 16 benchmark tests for StreamFlowPath
  operations (exists, mkdir, read/write, glob, walk, etc.)
- `benchmark/utils.py`: Helper functions for async benchmarking and test
  file generation

Code changes:
- `streamflow/data/remotepath.py`: Add `_make_child_relpath()` method to
  support path joining operations needed by benchmarks
@GlassOfWhiskey GlassOfWhiskey force-pushed the streamflow-benchmarks branch from c0c9df6 to 0fa7792 Compare March 24, 2026 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants