meowshell agent: a persistent, multiplexed SSH daemon with general-host support - #16
Merged
Conversation
One process per host connection instead of one per operation: "meowshell agent" dials once and keeps the *ssh.Client alive, serving shell and exec channels over a framed control protocol on stdin/stdout (protocol.go). This also fixes two problems the one-shot "meowshell connect" model couldn't: remote stderr no longer merges into the diagnostics stream (each channel gets its own stdout/stderr data frames), and a command's exit code is a structured field (exit_status) instead of becoming the whole agent process's own exit code -- connect.go's os.Exit(status) was fine for one command per process, but this process outlives any single channel. Live PTY resize (session.WindowChange) falls out of the same design, since a shell session now lives inside a long-running process instead of ending right after its one fixed-size PTY request. Transport, host keys, and auth are still tailcat-only and ssh-agent-only at this point -- later commits extend dialSSHClient's net.Conn seam to a raw TCP dialer with real host-key verification, and add password/ keyboard-interactive/certificate/Keystore-callback auth on top.
dialSSHClient's transport is now a net.Conn factory (transport.go) rather than always an exec'd tailcat subprocess: a tailcat address still dials through tailcat's own bare client mode unchanged, but "meowshell agent" also accepts a "[user@]host[:port]" destination and dials it directly over TCP, optionally through a SOCKS5 or HTTP CONNECT proxy (--proxy) and/or a chain of --jump bastions (each hop dialed through the previous one's own *ssh.Client). TCP transport gets what tailcat transport correctly does not need: real host-key verification against a standard known_hosts file (golang.org/x/crypto/ssh/knownhosts), trust-on-first-use for an unrecognized key and a hard failure (never silently reprompted) for one that changed. The TOFU prompt itself round-trips over the same control protocol open_channel/data already use (a new prompt_request/prompt_response pair), which meant restructuring agent.go so its frame reader runs concurrently with the dial/handshake instead of starting only after -- a prompt raised deep inside the handshake has nowhere else to get its answer from. Testing this against a real local SSH server (agent_tcp_e2e_test.go, no tailcat involved) surfaced a genuine ordering bug along the way: channel_opened could reach the client after data/exit_status for the same channel ID, on a remote command fast enough to finish before the agent got around to announcing the ID that names it. Fixed by writing channel_opened before starting the channel rather than after.
buildAuthMethods (agentauth.go) turns a client-supplied "configure" message -- now the mandatory first message on every connection, read synchronously before the frame reader's goroutine starts -- into an ordered auth method list: every public-key signer (the local ssh-agent, private keys supplied as bytes over the control channel, and Keystore-backed keys that never leave the client) combined into one ssh.PublicKeys method, then keyboard-interactive, then password. Password, keyboard-interactive, and passphrase prompts all reuse the same prompt_request/prompt_response round trip host-key TOFU already introduced. A Keystore-backed key (keystoreSigner) is the same mechanism carrying binary key material instead of typed text: its Sign blocks on a "sign" prompt the client answers from wherever the private key actually lives (Android Keystore hardware, the motivating case) -- the key's public half is all this process ever holds. Against tailcat's own service this is mostly unexercised machinery (its embedded SSH server only ever offers public-key auth or none, confirmed against its source in an earlier commit's plan) but it's what makes a general TCP SSH host, or a tailcat --ssh-authorized-keys server from an Android app with no ssh-agent, actually reachable -- the gap the original architecture review raised under "Android has no ssh-agent."
Extends the two-verb SFTP surface (meowshell cp's upload/download, ls) with the rest of what a file manager needs: mkdir, mkdir_all, rmdir, remove, rename, stat, lstat, chmod, chown, symlink, readlink, truncate, and realpath, each a fast sftp_op request/response over the control channel sharing one *sftp.Client per connection (agentsftp.go). Upload and download get their own channels instead, with progress events and cancellation (closing the channel) -- a real transfer over the framed protocol, not just a metadata call. Generic -L/-R/-D forwarding (forwarding.go) each run as a listener living inside the agent process itself: once a connection is accepted, its bytes flow directly between the local net.Conn and the SSH client's own Dial/Listen, never touching the framed protocol at all. -D is a minimal hand-rolled SOCKS5 server (CONNECT only, no auth) since golang.org/x/net/ proxy is a SOCKS5 *client* package, not a server. Separate from tailcat's own "forward" subcommand, which tunnels through a tailcat server instead -- a different use case this doesn't replace. A keepalive goroutine (startKeepalive) sends an OpenSSH-style keepalive request on an interval for the life of the connection, reporting errConnectionLost and tearing the connection down if one goes unanswered -- otherwise nothing here would ever notice a mobile carrier's NAT binding quietly dropping an idle connection. Testing the new SFTP verbs and -L/-D forwarding against real servers (a fake SSH server extended to answer direct-tcpip channels, since meowshell's own forwards route through ssh.Client.Dial like a real client would) caught a second, more fundamental race beyond the one the previous commit fixed: golang.org/x/crypto/ssh's Session.Wait is not synchronized with StdoutPipe/StderrPipe's own buffering, so it could report exit_status before a channel's last data frame had actually been written -- a fast enough exec (the exact shape a local fake server, or many quick file operations, produce) could reorder the two. Fixed by holding waitChannel until both output-pump goroutines have reached EOF before calling session.Wait, via a sync.WaitGroup.
MeowshellAgentConnection.ConnectAsync spawns "meowshell agent" and speaks its framed control protocol (MeowshellAgentProtocol, mirroring cmd/meowshell/protocol.go's frame layout and field names exactly), giving callers one connection with OpenShellAsync/OpenExecAsync (with live ResizeAsync, unlike TailcatSshSession's fixed-at-connect-time size), the full SFTP verb set plus Upload/DownloadAsync with progress and cancellation, and OpenLocalForwardAsync/OpenRemoteForwardAsync/ OpenSocksForwardAsync -- all over one login instead of a fresh subprocess and handshake per operation. Host-key/password/keyboard-interactive/ passphrase/Keystore-sign prompts surface as connection events (HostKeyPromptRequested and friends) an app answers asynchronously. TailcatException gains a Code field (MeowshellErrorCode) so a caller can branch on why something failed instead of scraping diagnostic text. Driving this against the real Go agent (already proven independently by its own extensive Go-level E2E tests) surfaced two real bugs in the C# client specifically, not the protocol or the Go side: - WriteDataAsync was prepending a stream-tag byte to every client-to-agent data frame. That tag only exists on the agent-to-client direction (to tell an exec channel's stdout from its stderr); client-to-agent frames are raw, exactly as agent.go's handleData expects. The effect was a stray leading NUL byte on every byte sent to the agent -- masked in the shell/exec tests, which only substring-check their output, but caught outright by the SFTP test's exact byte-length assertion. - Every OpenXAsync call registered its channel's sink (in _channels) only after awaiting the open_channel round trip, on whatever thread-pool thread the continuation happened to resume on. The agent can send a channel's first data frame immediately after channel_opened, with no gap at all -- for a small enough download, sometimes the *entire* transfer (every data frame and exit_status) had already arrived and been silently dropped before the sink existed to receive any of it, intermittently producing empty downloads. Fixed by threading the sink through to channel_opened's own handler in the read loop, so it's registered synchronously before that frame's processing returns and the loop can advance to the next one -- the same fix already applied earlier to the Go agent's own channel_opened-before-data ordering, this time on the client side of the same race. Also discovered (not a bug, a real limitation): tailcat's own embedded SSH server registers no "direct-tcpip" channel handler and no "tcpip-forward" request handler, so forward_local/forward_remote/forward_socks -- proven correct in cmd/meowshell's own Go-level E2E tests against a general SSH host -- can never work against a tailcat address itself. Documented in forwarding.go; the corresponding C# test checks the (safe, if unhelpful) actual behavior against tailcat instead of pretending it works.
README.md and dotnet/README.md now cover the persistent multiplexed connection alongside the existing per-operation APIs: the full-surface table, a worked example (shell + resize + upload + forward on one login), general-SSH-host support (known_hosts, --jump, --proxy), the auth prompt events, the SFTP verb set, and the typed MeowshellErrorCode on TailcatException. Also documents the one real limitation found while testing it: tailcat's own embedded SSH server never implements forwarding at all, so -L/-R/-D only work against a general SSH host, not a tailcat address -- MeowshellPortForward remains the way to tunnel through a tailcat server.
Addresses the HIGH and MEDIUM findings from a security review of the agent daemon: - forward_local/forward_socks refuse to bind anything but loopback unless allow_non_loopback_bind is set explicitly -- a loopback TCP listener is reachable by any other local process (any other app, on Android), not just the one that opened it. - forward_local/forward_socks also accept a Unix domain socket (listen_network "unix") instead of TCP, the recommended local endpoint wherever the caller can hand out a path: filesystem permissions (0600) restrict access, not "which port is free". - forward_socks supports RFC 1929 SOCKS5 username/password auth (constant-time compared) as an additional layer. - --proxy credentials moved off the agent process's own argv (readable via /proc/<pid>/cmdline by anything sharing enough local privilege) into the configure message, alongside how Keys/Certificates already avoid argv/disk. - MeowshellAgentConnection.HandleData's fire-and-forget dispatch into each channel's sink could leave two overlapping WriteAsync calls in flight on the same Pipe under backpressure -- undefined behavior for a single-writer type. Replaced with AgentChannelDataPump: a per- channel queue drained by one dedicated task, so a channel's data (and its terminal control message) always arrives strictly in order without the shared read loop ever blocking on a slow sink. New C# API surface: OpenLocalForwardOnUnixSocketAsync/ OpenSocksForwardOnUnixSocketAsync, allowNonLoopbackBind on the TCP forwards, and SOCKS auth (auto-generated token by default) on OpenSocksForwardAsync/OpenSocksForwardOnUnixSocketAsync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
forward_local and forward_socks previously only worked against a general SSH host: tailcat's own embedded SSH service registers no "direct-tcpip" channel handler, so a forward opened against a tailcat destination accepted local connections but silently dropped every one of them. Now the daemon picks the right dial mechanism for whatever it's connected to: SSH direct-tcpip for a general host (unchanged), or -- new -- a native tailcat.Client for a tailcat destination, dialing the same way tailcat's own "forward"/"socks" subcommands do (DialTCPPort for the server's own loopback, DialTCP through it as an exit node). Verified end to end against a real (hermetic local-DERP) tailcat server: bytes actually cross the forward now. This makes tailcat's Go package a real dependency of cmd/meowshell (previously it only ever shelled out to the tailcat binary), wired via a go.mod replace pointing at the same clone-and-patch build.sh already produces, so both binaries build from identical patched source. Costs a much larger dependency graph and a go 1.27.1 toolchain requirement (auto-fetched; GOTOOLCHAIN=auto), both worth calling out. forward_remote stays SSH-only: it asks the far end to open a listener, which only an SSH server can offer and tailcat's embedded one doesn't either. Opening one against a tailcat destination still succeeds but every connection it accepts is refused, same as before. Known follow-up, not introduced here: forwarding to a port the server doesn't already serve needs that server started with tailcat's own "exit-node" service (a protocol-level requirement, true of the old tailcat-forward path too) -- MeowshellServer has no option to request it yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
CI's "build" job ran go vet/go test before ./build.sh had cloned+patched .tailcat-src, which go.mod's replace directive for github.com/tailscale/tailcat now needs just to resolve imports -- added a lightweight clone step (unpatched is fine there: the two patches only change Android-specific networking behavior, irrelevant to vetting/testing on the runner's own host platform). Also: tailcatForwardClient.Dial used context.Background() for both DialTCPPort and DialTCP, so a destination the server's OnTCP/OnTCPForward gate refuses (no --exit-node, not one of its already-served ports) could leave the dial hanging indefinitely instead of returning an error -- found via a C# test that started timing out once native tailcat forwarding made that gate reachable at all. Bounded to the same tcpDialTimeout its sibling dialers in this file already use. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
Fixes the real "tailcat forward" gap: MeowshellPortForward/MeowshellSocksProxy (and now the agent's own forward_local/forward_socks against a tailcat destination) all dial through tailcat's OnTCP/OnTCPForward gate, which refuses anything but a server's own already-served ports unless it's running as an exit node -- a protocol-level requirement of tailcat itself, not something any client-side fix can route around. meowshell serve had no way to request it. Added --exit-node (meowshell CLI) and MeowshellOptions.AllowExitNode (C#), both warning without --allow the same way --insecure-no-auth/--files already do. Deliberately not changed, per discussion: TailcatClient.CpAsync/SshAsync stay on the system's own scp/ssh where available (mature, externally audited, the way tailscale itself intends the tool to be used), and MeowshellSocksProxy/MeowshellPortForward stay on tailcat's own forward/socks subcommands rather than becoming daemon wrappers -- MeowshellSocksProxy in particular can route to a different tailcat server per SOCKS request with no fixed address at all, a real capability a single daemon connection can't replicate. TailcatSshSession is the one clean case: it only ever dialed a tailcat address for one shell/exec channel, exactly what MeowshellAgentConnection already does -- now a thin wrapper around its own dedicated connection instead of spawning "meowshell connect" as a separate implementation. No capability lost: same Output/WriteAsync/Completed/Log/StopAsync shape, live callers unaffected. One real behavior change, worth calling out: a connect failure's TailcatException now carries a typed Code (MeowshellErrorCode) rather than a nonzero ExitCode, since the daemon model reports failures at the protocol level rather than as a process exit code. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
Adds a typed TailcatEnvironment (shell/home/user/path/term/lang, resolved tailcat binary path, resolver warnings) parsed from `meowshell env`'s output, closing one of the two real C#-exposure gaps identified in the last feature audit. Useful for diagnosing a broken environment (Android app sandbox, adb shell, a stripped container) up front instead of from a session that fails mysteriously once it's already running. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
tailcat serve's shipping CLI never wires up OnUDP/OnUDPForward (only its test suite and README examples do), so MeowshellSocksProxy's SOCKS5 UDP ASSOCIATE support has no real server on the other end that will accept relayed traffic. Fixing this for real needs a new patch to tailcat's own vendored CLI, not a meowshell-only change, so it's recorded here as a known gap rather than something worth a non-functional client-side stub. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the plan from the architecture/security review of PR #15's initial SSH/PTY work: a new
meowshell agentdaemon (Go) andMeowshellAgentConnectionclient (C#) that dial once and multiplex everything over that one connection, instead of one process and one handshake per operation.meowshell agent(cmd/meowshell/): a long-lived process per host connection, speaking a new framed control protocol (protocol.go) over stdin/stdout. Multiplexes shell/exec channels, the full SFTP verb set (mkdir,rename,chmod,symlink,truncate, upload/download with progress + cancellation, ...), and-L/-R/-Dport forwarding.dialSSHClient's transport is now anet.Connfactory — a tailcat address still dials through tailcat's own bare client mode unchanged, but a"[user@]host[:port]"destination dials directly over TCP, with real host-key verification (known_hosts, trust-on-first-use),--jumpbastion chaining, and--proxy(SOCKS5/HTTP CONNECT).--ssh-authorized-keysserver becomes reachable from Android for the first time.MeowshellErrorCodeon the C# side).MeowshellAgentConnection(dotnet/Meowshell/): the C# client —OpenShellAsync/OpenExecAsyncwith liveResizeAsync, the SFTP surface,OpenLocalForwardAsync/OpenRemoteForwardAsync/OpenSocksForwardAsync, and prompt events (HostKeyPromptRequested,PasswordRequested,KeyboardInteractiveRequested,PassphraseRequested,SignRequested) an app answers asynchronously.Real bugs found and fixed while testing against real servers (not just inspection)
channel_openedcould reach the client after data for the same channel, on a remote command fast enough to finish first.golang.org/x/crypto/ssh'sSession.Waitisn't synchronized withStdoutPipe/StderrPipe's own buffering, so it could reportexit_statusbefore the last data frame was actually written.WriteDataAsyncwas prepending a stream-tag byte to every client→agent data frame (that tag only belongs on the agent→client direction), and everyOpenXAsynccall registered its channel's sink after awaitingchannel_openedinstead of synchronously within the read loop's own handling of that reply — a fast enough channel (a small SFTP download, reliably) could have its entire payload arrive and be silently dropped before anything was listening for it.One real limitation, not a bug
tailcat's own embedded SSH server (
github.com/tailscale/gliderssh) never registers adirect-tcpipchannel handler or atcpip-forwardrequest handler — confirmed by reading that exact pinned library version's dispatch code, not just tailcat's own config, and confirmed again empirically against a real server. So-L/-R/-Dforwarding — proven correct against a general SSH host in this PR's own Go-level tests — can never work against a tailcat address itself. Documented in both READMEs and inforwarding.go;MeowshellPortForwardremains the way to tunnel through a tailcat server.Deliberately not in this PR
TailcatSshSession/TailcatClient.CpAsync/MeowshellPortForwardinto thin wrappers overMeowshellAgentConnection— both models work correctly standalone today; this is a pure internal-consistency cleanup with no new user-facing capability, deferred rather than risking regressions in stable, already-shipped code.Meowshell.AndroidProbe/android-probe-e2e.sh— untestable in this environment (no Android workload available); left for a follow-up that can actually exercise it on-device.Test plan
go vet ./... && go test ./... -race— clean, including new real end-to-end tests (a hermetic local-DERP tailcat server, a hand-rolled fake SSH server extended withdirect-tcpipsupport, and a real HTTP CONNECT proxy) covering the daemon, TCP transport + host-key TOFU/changed-key rejection, all auth methods, the full SFTP verb set with progress, and-L/-Dforwarding.dotnet test(net8.0) against the real built binaries withTS_DEBUG_TAILCAT_LOCAL_DERP=1— 99/99 passing, including newMeowshellAgentConnectionE2ETestsexercising the C# client against the real Go daemon.net10.0-android36.0) not built or tested in this environment — relies on CI'sdotnet-android-demo/dotnet-android-e2ejobs.🤖 Generated with Claude Code
https://claude.ai/code/session_01YCNCCSHwUMAKkUiNBvw5Kf
Generated by Claude Code