fix(sshproxy): run docker exec as the devcontainer's remoteUser - #45
Merged
Conversation
Running any git command in a VS Code terminal opened against a session
failed with:
fatal: detected dubious ownership in repository at '/workspaces/<repo>'
Devcontainer images routinely declare `USER root` and a separate
`remoteUser`, and DevPod chowns the workspace tree to that remoteUser.
`devpod ssh` — the browser terminal path — honors it, but the SSH proxy
built every `docker exec` without `--user`, so VS Code channels landed
as uid 0 in a uid 1000 tree. Git's safe.directory check compares the
repo owner against the process euid, sees the mismatch, and refuses.
Resolve the user from the container's devcontainer.metadata label
(remoteUser, then containerUser, per the spec's precedence) and pass
`--user` on every exec shape: shells, exec, SFTP, and TCP forwards.
SFTP matters as much as the shell — writing files as root into a
remoteUser-owned tree is the same bug with a slower fuse.
Resolution fails open. No declared user, a docker inspect error, or a
proxy without a workspace manager all fall back to the image's USER
rather than killing the channel: guessing wrong costs a git warning,
erroring out costs the whole editor session.
The label value is attacker-influenceable — a session member controls
the devcontainer.json in their own repo — so it is validated against a
strict user/uid[:group] pattern before reaching argv. Lookups are
memoized per container with a TTL, since VS Code opens many channels
per connection and each resolves the user.
Note `devpod.user` is not this value: it reports root on images that
declare a remoteUser, so keying off it would reproduce the bug.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Running any git command in a VS Code terminal opened against a session fails:
The same command in the in-browser Terminal panel works fine.
Root cause
A UID mismatch between the two paths into the container.
Devcontainer images routinely ship
USER rootand declare a separateremoteUser. DevPod chowns the workspace tree to thatremoteUser. Verified against a live container:docker inspect --format '{{.Config.User}}'rootdevcontainer.metadatalabel{"remoteUser":"vscode"}ls -ld /workspaces/<repo>drwxr-xr-x vscode vscodeThe browser Terminal panel shells out to
devpod ssh, which honorsremoteUserand lands as uid 1000. The SSH proxy shells out todocker exec, anddockerArgsbuilt every invocation without--user— so VS Code channels fell back to the image'sUSERand landed as uid 0.Git ≥2.35.2's
safe.directorycheck compares the repository directory's owner against the process euid. 1000 ≠ 0, so it refuses.Reproduced in the same container, with only the user differing:
Fix
Resolve the exec user from the container's
devcontainer.metadatalabel and pass--useron every exec shape — PTY shell, non-PTY shell, exec, SFTP, and TCP forwards.SFTP matters as much as the interactive shell: writing files as root into a
remoteUser-owned tree is the same bug with a slower fuse.Why
--userrather thansafe.directoryAdding a
safe.directoryexception would silence the error, but it patches one symptom of the UID mismatch. Running as root also means files created in VS Code land asroot:rootin avscode-owned tree,$HOMEresolves to/root(so git config, SSH keys, and tool caches all miss), and theHOME/USERthe proxy forwards from the client no longer describe the actual process user.--userfixes the class.Design notes
Precedence follows the devcontainer spec:
remoteUser, thencontainerUser, then leave the image'sUSERalone. Metadata entries merge with later ones overriding earlier, and an entry that omits the key does not clear a value an earlier entry set.Resolution fails open. No declared user, a
docker inspecterror, or a proxy built without a workspace manager all fall back to the previous behaviour rather than killing the channel. Guessing wrong costs a git ownership warning; erroring out costs the whole editor session.The label value is validated. It is attacker-influenceable — a session member controls the
devcontainer.jsonin their own repository — so it is checked against a strictuser[:group]/ uid pattern before reaching argv. It lands in an argv slot rather than a shell, so this is defence in depth; the leading-character rule is what stops a value from reading as a flag.Lookups are memoized per container with a 5-minute TTL. VS Code Remote-SSH opens many channels per connection and each resolves the user, so an uncached
docker inspectper channel would add real latency to every terminal and port forward. The cache lives on theManagerrather than in package globals so instances and tests don't share it.devpod.useris not this value. It reportsrooton images that declare aremoteUser, so keying off it would reproduce the bug. Called out in the code and in CLAUDE.md so the next person doesn't reach for it.Test plan
Full backend suite passes, including under
-race.go vetandgofmtclean.20 new cases:
sshproxy/docker_test.go—--useris present in all five exec modes and precedes the container name (after it, docker treats it as part of the in-container command); absent when no user resolves, with no empty argv slot; threaded throughbuildExecCmd,buildSFTPCmd, andbuildTCPForwardCmd, the last also asserting the forwarding script survives intact.workspace/container_user_test.go— the real multi-entry DevPod label shape,remoteUseroutrankingcontainerUser, later-entry override, empty values not clobbering earlier ones, bare-object tolerance, numeric uid anduser:groupforms, and rejection of malformed JSON, wrong value types, leading dashes, whitespace, shell metacharacters, and path traversal.Existing argv tests were updated to pass an empty user, preserving their original assertions as the "no declared user" case.
Why existing tests missed this: they assert
dockerArgsargv exactly as written, and a shape assertion cannot detect that the shape produces the wrong UID. Nothing related exec identity to workspace ownership.End-to-end verification:
workspace.ContainerUserwas run against the live container through the real code path and resolvedvscode, and the resulting argv was confirmed to fix git in that container.Post-Deploy Monitoring & Validation
No schema, auth, or data impact. The change is confined to the argv the SSH proxy builds.
Validate after deploy:
git statusin the integrated terminal — it succeeds.idin that terminal reports the devcontainer'sremoteUser, notroot.root.docker: Error response from daemon: unable to find user ...in server logs would mean a container declared a user that does not exist inside it. Look for thecould not resolve container exec userwarning, which indicates resolution fell back to the image default. Rollback is a straight revert.Known residuals / follow-ups
remoteUsernorcontainerUserstill exec as the image'sUSER, so they keep the original behaviour — including this bug if such an image also chowns the workspace elsewhere. Fixing that would mean stat-ing the workspace directory, which diverges from whatdevpod sshdoes.docker execfail outright rather than fall back. That is a broken devcontainer, anddevpod sshwould fail on it too, but the error surfaces later than a validation-time check would.docker execlayer, which is where the bug lives, but the full editor round trip has not been exercised.