pipe node decorator#97
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a pipeline_node decorator intended to wrap regular Python functions so they can be executed as pipeline nodes (reading inputs from a datablock and writing outputs back), addressing issue #96.
Changes:
- Introduces
pipeline_nodedecorator inpipeline.pyusingget_dict/set_dictdatablock helpers. - Fixes
Pipeline.add_nodeto avoid a mutable default argument (params={}→params=None). - Updates pipeline tests and package exports to reflect the new decorator/import patterns.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
agrifoodpy/pipeline/pipeline.py |
Adds pipeline_node, imports get_dict/set_dict, and fixes add_node default params handling. |
agrifoodpy/pipeline/tests/test_pipeline.py |
Adds unit tests covering pipeline_node behavior (single/multiple inputs, return key, external function, error cases) and adjusts imports. |
agrifoodpy/pipeline/__init__.py |
Changes re-exports (drops dict_utils wildcard export; keeps pipeline exports). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| pipeline_kwargs = dict(bound.arguments) | ||
| for key in input_keys: | ||
| pipeline_kwargs[key] = get_dict(datablock, | ||
| pipeline_kwargs[key]) | ||
| result = func(**pipeline_kwargs) |
There was a problem hiding this comment.
In pipeline mode the wrapper calls func(**pipeline_kwargs). This will fail for arbitrary functions that have positional-only parameters (common in builtins and some C-extension APIs), which undermines the stated goal of supporting external functions. A more robust approach is to update the existing bound.arguments values for input_keys and then invoke func(*bound.args, **bound.kwargs) so positional-only parameters are passed correctly.
| pipeline_kwargs = dict(bound.arguments) | |
| for key in input_keys: | |
| pipeline_kwargs[key] = get_dict(datablock, | |
| pipeline_kwargs[key]) | |
| result = func(**pipeline_kwargs) | |
| for key in input_keys: | |
| bound.arguments[key] = get_dict(datablock, | |
| bound.arguments[key]) | |
| result = func(*bound.args, **bound.kwargs) |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Description
This PR addresses #96 by adding a decorator which converts arbitrary functions into node functions which can work inside a pipeline
Checklist