-
Notifications
You must be signed in to change notification settings - Fork 134
🤖 feat: add an experimental native mobile companion #4103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ThomasK33
wants to merge
83
commits into
main
Choose a base branch
from
mobile-app-z0ya
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
83 commits
Select commit
Hold shift + click to select a range
a8d5c39
🤖 feat: build native Xum mobile workspace and chat interface
ThomasK33 1b17b84
🤖 feat: add mobile remote transport and transcript reducer
ThomasK33 65ac4a4
🤖 fix: replace mobile connections on explicit retry
ThomasK33 40fbfe4
🤖 feat: scaffold and validate the Expo Xum mobile companion
ThomasK33 e323034
🤖 fix: complete mobile model discovery and history recovery
ThomasK33 f53be0f
🤖 feat: polish native mobile forms and sheets
ThomasK33 41e39b0
🤖 feat: refine native mobile navigation and conversation UX
ThomasK33 43e839f
🤖 fix: explain empty mobile assistant history
ThomasK33 2f170dc
🤖 feat: refine mobile transcript and tool inspection
ThomasK33 5fc6af6
🤖 feat: refine mobile UI from Claude and Codex references
ThomasK33 cb15511
🤖 fix: keep mobile login copy native-facing
ThomasK33 c6773d5
🤖 feat: use focused native mobile pickers
ThomasK33 41febc6
🤖 fix: search all visible models in the mobile picker
ThomasK33 eed1626
🤖 feat: refine mobile navigation and bottom composer
ThomasK33 8162824
🤖 fix: support native abort signals in mobile transport
ThomasK33 3d8ea5a
🤖 fix(mobile): align workspace and transcript presentation with desktop
ThomasK33 a2e0171
🤖 fix(mobile): measure native keyboard avoidance in window coordinates
ThomasK33 66f6854
🤖 fix(mobile): expose context progress to native and web accessibility
ThomasK33 557c812
🤖 fix(mobile): fetch changes across repositories and validate mobile …
ThomasK33 9ddd05e
🤖 fix(mobile): preserve canonical AI settings and provider preferences
ThomasK33 0792887
🤖 fix: recover mobile sessions and replace sidebar detail routes
ThomasK33 d7bb515
🤖 docs: refresh bundled mobile compatibility guidance
ThomasK33 5c1d1e1
🤖 tests: deduplicate merged mobile icon aliases
ThomasK33 5d88915
🤖 fix(mobile): retain recovery retries and app goal capability
ThomasK33 d65367b
🤖 fix(mobile): support structured questions and searchable subagents
ThomasK33 1467973
🤖 tests: exercise recovery with canonical question choices
ThomasK33 5ea0951
🤖 fix(mobile): enforce live server policy before conversation actions
ThomasK33 6640ae8
🤖 fix(mobile): keep custom question option identities distinct
ThomasK33 1dce2bb
🤖 fix(mobile): pin shared schema runtime for production bundles
ThomasK33 795b5a5
🤖 fix(mobile): preserve active context capacity and deleted file names
ThomasK33 3cf93cc
🤖 fix(mobile): refresh conversation settings from live config changes
ThomasK33 c675587
🤖 fix(mobile): keep live interruption independent of settings readiness
ThomasK33 5153b3f
🤖 fix(mobile): recover saved question answers after replay
ThomasK33 d55fb0b
🤖 fix(stream): pin effective context capacity to provider requests
ThomasK33 7b835b4
🤖 fix(mobile): render active context from the request capacity snapshot
ThomasK33 f12fa5e
🤖 fix(mobile): restore complete queued input after interruption
ThomasK33 7225a8c
🤖 fix(mobile): retain composer cleanup across restored draft sends
ThomasK33 7548f3a
🤖 fix: restore queued reviews before message boundaries are flattened
ThomasK33 81964ab
🤖 tests: align queue restoration fixtures with lint rules
ThomasK33 5ff65d8
🤖 refactor: reconcile mobile branch with current main
ThomasK33 d5d122d
🤖 fix: validate queued review metadata before restoring input
ThomasK33 4a0e9d8
🤖 fix(stream): publish non-destructive fallback model metadata
ThomasK33 41e757c
🤖 fix(stream): replace optional fallback metadata explicitly
ThomasK33 8ff11e1
🤖 fix: validate reviews in live and replayed queue snapshots
ThomasK33 e751c94
🤖 fix(stream): reset live usage at fallback attempt boundaries
ThomasK33 88a2fd3
🤖 tests: expect fallback metadata in compaction event ordering
ThomasK33 f2cd06b
🤖 fix(mobile): gate actions on actual route and authentication availa…
ThomasK33 82a27de
🤖 fix(mobile): recognize effective API keys alongside OpenAI OAuth
ThomasK33 dcdd3ca
🤖 fix(mobile): project native colors from the canonical CSS theme
ThomasK33 a76b293
🤖 fix: reconcile mobile User Stop with monitor scheduling
ThomasK33 c11a2f9
🤖 fix: keep live answers independent of next-turn routing
ThomasK33 e398737
🤖 fix(mobile): share canonical tool icon semantics
ThomasK33 f49d463
🤖 tests: stabilize persisted sidebar split fixture
ThomasK33 a5c4708
🤖 fix(mobile): add focused web composer keyboard shortcuts
ThomasK33 00416d4
🤖 refactor: reconcile mobile branch with current main
ThomasK33 a2cbc9f
🤖 fix(mobile): restore target-agent reasoning mode on mode switches
ThomasK33 232c639
🤖 fix(mobile): respect delegated identity and transcript-only workspaces
ThomasK33 d95c80f
🤖 fix(mobile): initialize structured question prefills
ThomasK33 9bfd4cf
🤖 tests(mobile): cover untrusted question prefill keys
ThomasK33 57498bb
🤖 fix(mobile): refresh project catalogs on config changes
ThomasK33 c5bf5c2
🤖 fix(mobile): submit final fields and gate queued questions
ThomasK33 43828f8
🤖 fix(mobile): preserve durable Stop intent during question recovery
ThomasK33 be7aa44
🤖 tests(mobile): model executing questions and newer recovery turns
ThomasK33 2ec59b4
🤖 fix(mobile): block creation from removed project selections
ThomasK33 bddc6c7
🤖 fix(mobile): require trusted worktree-capable project selections
ThomasK33 6f65c26
🤖 fix(chat): use pinned active context limits in desktop meters
ThomasK33 ccdda28
🤖 fix: isolate mobile drafts and hide model-only transcript rows
ThomasK33 ffda71e
🤖 fix(mobile): render inspectable nested tool calls
ThomasK33 6e61688
🤖 fix(mobile): share legacy nested tool replay reconstruction
ThomasK33 19639e5
🤖 fix(auth): authorize oRPC WebSockets with single-use upgrade tickets
ThomasK33 c257637
🤖 fix(auth): add renderer-safe WebSocket ticket acquisition
ThomasK33 5ff7494
🤖 fix(auth): migrate browser token sockets to single-use tickets
ThomasK33 c7dc714
🤖 fix(mobile): authenticate WebSockets with single-use tickets
ThomasK33 b350322
🤖 fix(mobile): require the negotiated oRPC application protocol
ThomasK33 9106b6e
🤖 docs(mobile): explain secure WebSocket ticket requirements
ThomasK33 d1c1a11
🤖 fix(mobile): require TLS for every non-loopback endpoint
ThomasK33 3c6f3e3
🤖 fix(mobile): refresh agent catalogs with live settings
ThomasK33 e478b03
🤖 fix(mobile): enforce runtime policy during workspace creation
ThomasK33 cd97460
🤖 perf(mobile): coalesce streamed text display updates
ThomasK33 fd0a60c
🤖 fix(mobile): preserve header-like lines inside diff hunks
ThomasK33 900ea0c
🤖 fix(mobile): disable Git changes for transcript-only workspaces
ThomasK33 de171a0
🤖 fix(auth): retry rejected stored tokens without bearer credentials
ThomasK33 20cdf97
🤖 tests(auth): inspect request URLs without implicit object conversion
ThomasK33 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| --- | ||
| title: Mobile companion | ||
| description: Develop the React Native Xum companion and connect it to your server. | ||
| --- | ||
|
|
||
| The experimental mobile companion lives in `packages/mobile`. It uses native React Native views, with React Native Web for browser development—not an embedded copy of the desktop website. | ||
|
|
||
| It connects to your existing Xum server for projects, workspace creation, conversations, agent/model selection, and read-only changes. On phones, a searchable workspace list opens conversations in a native navigation stack; wider screens keep the workspace sidebar visible. Phone workspace search and creation stay in a bottom dock, while wide layouts keep their controls above the list. Conversation headers show project and server context with grouped navigation actions. Drafts and unsent model choices survive returning to the list. The composer stays compact when empty and unfocused, expands for writing, and stays at the bottom with mode/model controls directly above it. Creation uses a sheet with a pinned action. The composer's separate model and mode controls open focused pickers; selections apply immediately, and changing mode preserves the chosen model and effort. Search models directly in the model picker by name, provider, or alias. The grouped list follows model visibility in Settings and includes models available through configured gateway routes; the current selection remains visible even if subsequently hidden. Use Effort to adjust thinking. Custom model IDs require confirmation. Tool activity stays compact in the conversation; tap a tool to inspect its input, output, and status. Provider configuration, terminal/desktop access, and advanced administration remain in the main Xum app. | ||
|
|
||
| ## Connect to a server | ||
|
|
||
| Enable [server access](/config/server-access), or start `xum server`. Use a trusted HTTPS endpoint accessible from the device and enter the server's bearer token separately. Include any reverse-proxy path prefix in the endpoint. A Coder login page or another upstream authentication layer may require additional network access; the Xum token does not authenticate to that outer layer. | ||
|
|
||
| During development, run the mobile client and server from the same branch/revision. Their shared API contract evolves together; for example, the multi-repository changes view requires the server's bulk project-diff endpoint. | ||
|
|
||
| The token grants access to the server, including its code-execution capabilities. Treat it like a password. Native builds save connection details in device secure storage. The web preview keeps them in memory only; refreshing requires entering them again. Disconnect clears the saved native connection. | ||
|
|
||
| Before opening a WebSocket, the companion exchanges the token in an HTTP Authorization header for a short-lived, single-use upgrade ticket. The long-lived token is not included in the WebSocket URL or subprotocols. Older servers without ticket support must be updated; there is no credential-URL fallback. | ||
|
|
||
| All non-loopback endpoints—including private LAN, ULA, and link-local addresses—require HTTPS. HTTP is accepted only for `localhost`, IPv4 loopback (`127.0.0.0/8`), or IPv6 loopback (`[::1]`) for development, with a plaintext-token warning. A phone's `localhost` refers to the phone, not your development computer: use a trusted HTTPS endpoint to reach that computer from a device. | ||
|
|
||
| ## Develop with React Native Web | ||
|
|
||
| Install the repository's Bun dependencies and use Node.js 22.19 or later for Expo and the preview proxy: | ||
|
|
||
| ```bash | ||
| bun install | ||
| make mobile-install | ||
|
|
||
| # Point this at a running Xum instance. Do not put the token in the URL. | ||
| XUM_MOBILE_ENDPOINT=http://127.0.0.1:3000 make mobile-web | ||
| ``` | ||
|
|
||
| Open `http://127.0.0.1:8082` in a current Chromium browser, then enter that configured **server endpoint** and its token. Metro runs on port 8081; use the proxy on 8082, not Metro's direct URL, for API access. The web preview uses CSS content sizing for the composer; native builds use React Native's text measurement. | ||
|
|
||
| With the Message composer focused, Enter sends in desktop-sized windows with a fine pointer; Shift+Enter inserts a newline. Narrow windows and coarse-pointer devices keep Enter for newlines. Ctrl+Enter (or Cmd+Enter on macOS) sends while idle on either layout. Escape uses the active conversation's Stop action. Shortcuts respect disabled actions and composition, do not run from other inputs or modal dialogs, and do not queue messages or stop a turn when Enter is pressed during streaming. | ||
|
|
||
| The preview forwards to exactly one endpoint configured at startup. It checks the request Host and Origin before forwarding, strips preview cookies/forwarded identity, and preserves the upstream path prefix. It does not relax the production server's origin protections. Native builds connect directly and do not need this proxy. | ||
|
|
||
| Optional development settings: | ||
|
|
||
| - `MOBILE_METRO_PORT`: Metro port (Make variable). | ||
| - `XUM_MOBILE_PORT`: preview port, default 8082. | ||
| - `XUM_MOBILE_ORIGIN`: exact public preview origin when forwarding this loopback-bound server; the forwarding proxy must preserve that Host. | ||
| - `XUM_MOBILE_ENDPOINT`: the fixed Xum target; restart the preview to change it. | ||
|
|
||
| Serve a production web export: | ||
|
|
||
| ```bash | ||
| make mobile-export | ||
| XUM_MOBILE_ENDPOINT=http://127.0.0.1:3000 make mobile-preview | ||
| ``` | ||
|
|
||
| The preview is development tooling, not a general-purpose public proxy. It intentionally runs under Node: Bun's Node HTTP compatibility can stall forwarded WebSocket frames. | ||
|
|
||
| ## Native development | ||
|
|
||
| ```bash | ||
| make mobile-native | ||
| ``` | ||
|
|
||
| Use Expo's device/simulator workflow with the installed SDK-compatible client or development build. The native app uses `expo-secure-store` and safe-area/keyboard-aware layouts. Native networking, keyboard behavior, secure storage, and background/resume behavior still need device testing; a successful JavaScript export does not establish that they work on iOS. | ||
|
|
||
| ```bash | ||
| # Compiles the iOS JavaScript/Hermes bundle; does not launch or build a simulator app. | ||
| make mobile-export-ios | ||
| ``` | ||
|
|
||
| Send/Stop keyboard shortcuts currently apply only to React Native Web. Native software-keyboard Enter behavior is unchanged; native hardware-keyboard Send/Stop bindings are not implemented. The installed native TextInput APIs do not expose the modifier/source information needed for that behavior without additional native integration. | ||
|
|
||
| The mobile dependency graph and lockfile are isolated from desktop React. Update SDK-compatible versions together and run `bun x expo install --check` from `packages/mobile` after dependency changes. | ||
|
|
||
| ## Validation and dogfooding | ||
|
|
||
| ```bash | ||
| make mobile-check | ||
| make mobile-export | ||
| make mobile-export-ios | ||
| ``` | ||
|
|
||
| `mobile-check` runs typechecking against the shared API schemas, lint/format checks, and endpoint, transport, transcript, lifecycle, and preview-proxy tests. For the opt-in real-server test, use a **disposable** Xum root with `XUM_MOCK_AI=1`, then run: | ||
|
|
||
| ```bash | ||
| cd packages/mobile | ||
| XUM_MOBILE_TEST_ENDPOINT=http://127.0.0.1:3000 \ | ||
| XUM_MOBILE_TEST_TOKEN=your-disposable-server-token \ | ||
| bun test ./scripts/server.integration.test.ts | ||
| ``` | ||
|
|
||
| That test creates and removes a scratch workspace. It exercises real authentication, persistence, streaming and reconnect/replay; only the model response is deterministic. | ||
|
|
||
| With the production preview running against that same disposable server, run the full-app browser regressions from the repository root: | ||
|
|
||
| ```bash | ||
| # One-time browser installation | ||
| (cd packages/mobile && bun x playwright install chromium) | ||
| XUM_MOBILE_TEST_ENDPOINT=http://127.0.0.1:3000 \ | ||
| XUM_MOBILE_TEST_TOKEN=your-disposable-server-token \ | ||
| make mobile-test-web | ||
| ``` | ||
|
|
||
| These tests pin 375px, 390px, and 1200px viewports, create and remove scratch chats, and check draft/model retention, keyboard focus, and reachable sheet actions. Set `XUM_MOBILE_TEST_WEB_URL` if the preview is not at `http://127.0.0.1:8082`. | ||
|
|
||
| For a browser walkthrough, use the production preview and a phone viewport around 375–390 pixels, then repeat at tablet/desktop width: | ||
|
|
||
| 1. Check invalid URL, wrong token, and successful connection. | ||
| 2. Open the workspace navigator; create a scratch chat and a project workspace. | ||
| 3. Send a message, observe streamed text/tools/reasoning, and interrupt a running turn. | ||
| 4. Change agent/model settings, switch workspaces, and verify conversations do not mix. | ||
| 5. Open Changes and Settings; disconnect and confirm credentials are not retained in browser storage. | ||
| 6. Drop the connection, retry, and verify authoritative history reloads before sending is enabled. | ||
| 7. Capture screenshots and a short recording of the walkthrough, including narrow layouts and any failure/recovery steps. | ||
|
|
||
| ## Can Xum run inside the native JS engine? | ||
|
|
||
| **Not the existing backend unchanged.** Hermes executes JavaScript, but it does not provide Xum's Node filesystem/process APIs, shell/git toolchain, PTY bindings, or database/native addons. The companion runs its UI and client logic locally; agent execution stays on the server. | ||
|
|
||
| A native Node sidecar would be a separate runtime and substantial platform port. It would not make the desktop shell tools available inside an iOS sandbox. This app does not advertise an embedded backend mode. | ||
|
|
||
| Research references: | ||
|
|
||
| - [React Native core/native views](https://reactnative.dev/docs/intro-react-native-components) | ||
| - [Hermes](https://reactnative.dev/docs/hermes) | ||
| - [Expo web support](https://docs.expo.dev/workflow/web/) | ||
| - [SecureStore](https://docs.expo.dev/versions/latest/sdk/securestore/) | ||
| - [Node.js Mobile's separate native runtime](https://nodejs-mobile.github.io/docs/guide/guide-react-native/getting-started/) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| .expo/ | ||
| node_modules/ | ||
| dist/ | ||
| dist-ios/ | ||
| ios/ | ||
| android/ | ||
| *.tsbuildinfo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| .expo/ | ||
| dist/ | ||
| dist-ios/ | ||
| ios/ | ||
| android/ | ||
| node_modules/ |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.