misc: add arca-networking code#33
Conversation
There was a problem hiding this comment.
I think we could use some existing crate (like embedded-io) that provides no_std io traits so that we are not redoing some of the things...
|
|
||
| --- | ||
|
|
||
| ## 3. Stream layer (`data-pipe`) |
There was a problem hiding this comment.
I feel like the more ergonomic way of handling this asyncness would be that for each direction of the write ---> read of the pipe, there needs to be an event fd like file descriptor for the writer to notify the reader that there are new data readable, instead of doing the current yield_now.
This file descriptor would be set up differently depends on whether the pipe is going into Arca or leaving Arca, but I could imagine the Linux monitor that relays from Arca is implemented as a bunch of tokio::spawn of loops triggered by the ready event fd.
There was a problem hiding this comment.
For sending notifications into Arca the Linux end can write to an EventFd to trigger an interrupt in Arca (which will reschedule any kernel thread that had called kthread::wifi() to deschedule itself). For sending notifications out of Arca the Arca end can make a hypercall, which could notify a condition variable in Rust.
| } | ||
|
|
||
| /// Bytes available to write. Called by the producer. | ||
| pub fn free_space(&self, capacity: u64) -> u64 { |
There was a problem hiding this comment.
I would change "used_space ==> readable_len" and "free_space ==> writable_len"
| --- | ||
|
|
||
| ## 4. Message catalog | ||
|
|
There was a problem hiding this comment.
I feel like we should put in more layers of abstractions here. The control pipe messaging channel should be only about that the Arca side would like to create a new pair of bidirectional pipes, and then there is a separate protocol for negotiating what this newly-created pair of pipes is for (e.g. this could be by the first frame/message sent from the Arca side on this pipe). So it could be something like
-
Control Pipe: Arca -> Linux: NewStreamRequest; Linux -> Arca: DataStreamInfo
-
Data Pipe:
- First Frame from Arca to Linux: Protocol (0 = FileSystem, 1= TCPListener, 2 = TCPConnection, etc.). We can also piggy back the actual protocol payload here (like the Endpoint for TCP Listener)
- If it's a pipe for file system: Arca -> Linux: ReadFileRequest { path }; Linux -> Arca: FileData { payload }
- If it's a pipe for listening: Arca first creates a new stream via control pipe; Arca -> Linux: AcceptConnection (with the new pipe id as payload); Linux -> Arca: IncomingConnection (also setup the relaying on the monitor side)
Then, on the monitor side, the implementation would be something like for each kind of pipe, we implement the logic for handling the pipe-specific protocol.
| per-connection byte relay: | ||
|
|
||
| ```rust | ||
| loop { |
There was a problem hiding this comment.
I think pump_once and the relaying should probably be independent tokio::spawn if possible...
| └── PROTOCOL.md # this file | ||
| ``` | ||
|
|
||
| The Arca-facing public surface is intentionally small: |
There was a problem hiding this comment.
If we do split the control frame into more layers of abstractions, Arca kernel should only care about open/read/write/close of a stream. The actual protocol implementation would be part of the user procedures.
| │ └── src/ | ||
| │ ├── protocol.rs # wire types + payload encodings | ||
| │ ├── message.rs # ControlRequest / ControlReply enums + to_frame / try_from | ||
| │ ├── codec.rs # read_frame / write_frame / FrameReadBuf |
There was a problem hiding this comment.
Everything under control/ could be made no_std if we move codec.rs to somewhere else. I think codec.rs should be together with monitor as the Linux code, and control should just be a no_std crate shared by Arca side/user procedures and Linux side that handles messages.
2 participants
Draft PR for commenting/discussing purpose for now.