Skip to content

Feat/rpc system#1

Merged
mesudip merged 25 commits into
masterfrom
feat/rpc-system
Feb 8, 2026
Merged

Feat/rpc system#1
mesudip merged 25 commits into
masterfrom
feat/rpc-system

Conversation

@mesudip
Copy link
Copy Markdown
Owner

@mesudip mesudip commented Feb 7, 2026

No description provided.

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello @mesudip, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive overhaul and expansion of the 'cbor-rpc' library's internal architecture. It focuses on establishing a more modular and robust foundation for asynchronous communication by refining the pipe and transformer mechanisms. The changes aim to provide greater flexibility in handling data streams, support diverse transport protocols, and formalize the RPC interfaces, making the system more adaptable and easier to extend. This refactoring also brings improvements to event handling and error reporting, ensuring a more stable and debuggable RPC framework.

Highlights

  • Core Architecture Refactoring: Introduced a new hierarchy for pipes ('EventPipe', 'Pipe', 'AioPipe') and transformers ('Transformer', 'AsyncTransformer', 'EventTransformerPipe', 'TransformerPipe') to provide more flexible and explicit control over data flow and event handling.
  • Enhanced Event Emission: The 'AbstractEmitter's '_emit' method is now synchronous, offloading coroutine handlers to background tasks, and includes improved error logging with 'traceback' and 'warnings'.
  • New Transport Layers: Added 'SshPipe' for SSH-based communication and 'StdioPipe' for standard I/O and subprocess integration, significantly expanding the library's connectivity options.
  • RPC Interface Formalization: Introduced abstract base classes ('RpcClient', 'RpcServer', 'RpcAuthorizedClient') to define clear interfaces for RPC components, improving extensibility and maintainability.
  • Improved Promise Handling: Renamed 'DeferredPromise' to 'TimedPromise' and added 'await' for better integration with 'asyncio', along with explicit 'sys.stderr' logging for RPC errors.
  • Modularized Codebase: The project structure has been reorganized into subpackages ('event', 'pipe', 'rpc', 'tcp', 'ssh', 'stdio', 'transformer') for better organization and separation of concerns.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • .vscode/settings.json
    • Added pytest configuration for 'tests' directory.
  • README.md
    • Added a basic project title.
  • cbor_rpc/init.py
    • Updated imports and 'all' to reflect the new module structure, replacing old pipe and promise classes with new ones.
  • cbor_rpc/async_pipe.py
    • Removed, its functionality is now distributed across 'cbor_rpc/pipe/event_pipe.py' and 'cbor_rpc/pipe/pipe.py'.
  • cbor_rpc/event/init.py
    • New module for event-related classes.
  • cbor_rpc/event/emitter.py
    • Renamed from 'cbor_rpc/emitter.py'.
    • Modified '_emit' to be synchronous and use background tasks for coroutine handlers, improving error logging.
  • cbor_rpc/json_transformer.py
    • Removed, replaced by 'cbor_rpc/transformer/json_transformer.py'.
  • cbor_rpc/pipe/init.py
    • New module for pipe-related classes, introducing 'EventPipe', 'Pipe', and 'AioPipe'.
  • cbor_rpc/pipe/aio_pipe.py
    • New abstract base class for asyncio stream-based pipes.
  • cbor_rpc/pipe/event_pipe.py
    • New abstract base class for event-driven pipes, replacing the core logic of the old 'async_pipe'.
  • cbor_rpc/pipe/pipe.py
    • New abstract base class for explicit read/write pipes, including in-memory pair creation and conversion to 'EventPipe'.
  • cbor_rpc/rpc/init.py
    • New module for RPC-related classes.
  • cbor_rpc/rpc/rpc_base.py
    • New file defining abstract base classes for 'RpcClient', 'RpcAuthorizedClient', and 'RpcServer'.
  • cbor_rpc/rpc/rpc_server.py
    • Renamed from 'cbor_rpc/server.py'.
    • 'RpcV1Server' constructor updated to accept a 'Server' instance, and 'add_connection' now expects an 'EventPipe'.
  • cbor_rpc/rpc/rpc_v1.py
    • Renamed from 'cbor_rpc/client.py'.
    • Updated to use 'EventPipe' and 'TimedPromise', with improved error logging to 'sys.stderr'.
  • cbor_rpc/rpc/server_base.py
    • Renamed from 'cbor_rpc/server_base.py'.
    • Generic type 'P' now bound to 'EventPipe', and an abstract 'accept' method added for connection filtering.
  • cbor_rpc/ssh/init.py
    • New module for SSH pipe implementation.
  • cbor_rpc/ssh/ssh_pipe.py
    • New implementation of 'AioPipe' for SSH connections using 'asyncssh'.
  • cbor_rpc/stdio/stdio_pipe.py
    • New implementation of 'AioPipe' for standard I/O and subprocess communication.
  • cbor_rpc/sync_pipe.py
    • Removed, synchronous pipe concept deprecated.
  • cbor_rpc/tcp/init.py
    • New module for TCP pipe and server implementations.
  • cbor_rpc/tcp/tcp.py
    • Renamed from 'cbor_rpc/tcp.py'.
    • 'TcpPipe' now inherits from 'AioPipe', and 'TcpServer' includes an abstract 'accept' method.
  • cbor_rpc/timed_promise.py
    • Renamed from 'cbor_rpc/promise.py'.
    • 'DeferredPromise' renamed to 'TimedPromise' and gained 'await' support.
  • cbor_rpc/transformer.py
    • Removed, replaced by new transformer base classes.
  • cbor_rpc/transformer/init.py
    • Updated imports for new transformer structure.
  • cbor_rpc/transformer/base/init.py
    • New module for transformer base classes.
  • cbor_rpc/transformer/base/base_exception.py
    • New file defining 'NeedsMoreDataException'.
  • cbor_rpc/transformer/base/event_transformer_pipe.py
    • New class to wrap 'EventPipe' with a transformer.
  • cbor_rpc/transformer/base/transformer_base.py
    • New abstract base classes for synchronous and asynchronous transformers.
  • cbor_rpc/transformer/base/transformer_pipe.py
    • New class to wrap 'Pipe' with a transformer.
  • cbor_rpc/transformer/json_transformer.py
    • New location for 'JsonTransformer', now inheriting from the new 'Transformer' base class.
  • examples/fs_rpc/init.py
    • Renamed from 'examples/rpc_backends_example.py'.
  • examples/fs_rpc/filesystem_client.py
    • New example demonstrating a filesystem RPC client.
  • examples/fs_rpc/filesystem_server.py
    • New example demonstrating a filesystem RPC server.
  • examples/json_transformer_example.py
    • Removed, examples refactored.
  • examples/tcp_rpc_example.py
    • Removed, examples refactored.
  • pyproject.toml
    • Added 'asyncssh' and 'bcrypt' to dependencies.
  • setup.py
    • New file for package distribution.
  • tests/docker/sshd-python/Dockerfile
    • New Dockerfile for SSH testing environment.
  • tests/docker/sshd-python/binary_emitter.py
    • New script for binary data emission in SSH tests.
  • tests/docker/sshd-python/echo_back.py
    • New script for echo-back functionality in SSH tests.
  • tests/helpers/simple_pipe.py
    • Updated to use 'EventPipe' and its event emission methods.
  • tests/helpers/simple_tcp_server.py
    • New helper class for simplified TCP server testing.
  • tests/helpers/stdio_test_script.py
    • New script for 'StdioPipe' testing.
  • tests/test_async_pipe.py
    • Removed, replaced by new pipe tests.
  • tests/test_event_emitter.py
    • Updated imports and '_emit' calls.
  • tests/test_event_pipe.py
    • New tests for 'EventPipe'.
  • tests/test_json_transformer.py
    • Rewritten to test the new 'JsonTransformer' and its integration with 'EventTransformerPipe'.
  • tests/test_pipe.py
    • New tests for 'Pipe' (explicit read/write).
  • tests/test_rpc_v1.py
    • Updated imports to use 'EventPipe' and 'TimedPromise'.
  • tests/test_server_generics.py
    • Removed, server testing refactored.
  • tests/test_ssh_docker_pipe.py
    • New comprehensive tests for 'SshPipe' including various authentication methods and data types.
  • tests/test_stdio_rpc.py
    • New tests for 'StdioPipe'.
  • tests/test_sync_pipe.py
    • Removed, synchronous pipes deprecated.
Activity
  • The author 'mesudip' initiated this pull request.
  • The pull request involves a significant refactoring of the 'cbor-rpc' library.
  • New modules and abstract base classes have been introduced for pipes, transformers, and RPC components.
  • Existing core components like 'emitter', 'client', 'server', 'server_base', 'tcp', and 'promise' have been renamed, moved, and updated to fit the new architecture.
  • New transport layers for SSH and standard I/O have been added.
  • Extensive new test files have been added to cover the new pipe types, SSH, and standard I/O functionalities.
  • Old test files and examples have been removed or refactored to align with the new design.
  • Dependencies 'asyncssh' and 'bcrypt' were added to 'pyproject.toml' to support the new SSH functionality.
  • A 'setup.py' file was added for package distribution.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a major refactoring of the RPC system, improving its structure and modularity by introducing new abstractions for pipes and transformers. The changes are extensive and well-organized, moving towards a more robust and extensible architecture. I've identified several issues, including a critical packaging bug in setup.py, some logic errors in the new transformer and pipe implementations, and a few inconsistencies in error handling and method signatures. Additionally, one of the new examples appears to be broken. Addressing these points will significantly improve the quality and correctness of this new RPC system.

Comment thread setup.py Outdated
Comment thread cbor_rpc/stdio/stdio_pipe.py Outdated
Comment thread cbor_rpc/transformer/base/transformer_base.py Outdated
Comment on lines +17 to +22
def __init__(self, pipe: Pipe[Any, Any], transformer: 'Optional[Transformer[T1, T2]]' ):
super().__init__()
self.pipe = pipe

self.encode = transformer.encode
self.decode = transformer.decode
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The __init__ method's type hint for transformer is Optional, but the implementation directly accesses transformer.encode and transformer.decode without checking if transformer is None. This will raise an AttributeError if the pipe is initialized with transformer=None. You should add a check to ensure a transformer is provided.

    def __init__(self, pipe: Pipe[Any, Any], transformer: 'Transformer[T1, T2]' ):
        super().__init__()
        self.pipe = pipe

        if not transformer:
            raise ValueError("A transformer must be provided.")

        self.encode = transformer.encode
        self.decode = transformer.decode

Comment thread examples/fs_rpc/filesystem_server.py Outdated
Comment thread cbor_rpc/rpc/rpc_server.py Outdated
Comment thread cbor_rpc/rpc/rpc_v1.py Outdated
Comment thread cbor_rpc/rpc/server_base.py
Comment thread cbor_rpc/timed_promise.py
Comment thread cbor_rpc/transformer/base/transformer_base.py Outdated
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 8, 2026

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@mesudip mesudip merged commit 4e28a57 into master Feb 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants