feat: run shell in a sandbox

[zaytsev]

Use bubblewrap to spawn devenv shell process in an isolated sandbox. Early PoC just to get some feedback.

Limitations:

• Tested on NixOS only
• Linux Only/No MacOS support

example devenv.local.yaml

sandbox:
  enable: true
  network:
    enable: true
  mounts:
    - path: /nix/store
    - path: /dev
      mode: dev
    - path: /dev/null
      mode: dev
    - path: /dev/random
      mode: dev
    - path: /dev/urandom
      mode: dev
    - path: $NIX_USER_PROFILE_DIR/profile
    - path: $HOME
      mode: overlay

[LorenzBischof]

I did not have time to look at the details yet, but did you see my draft PR? #1783 I opted to try Landlock, which I personally think might be better suited, but to be honest I do not have a lot of experience.

The problem I see when running the whole shell in a sandbox, is that we are basically sandboxing everything the user is running, not just the tools provided by the Nix shell. I quickly tried your implementation to confirm my suspicions that I then do not have zsh with starship configured inside the sandbox. This means we also basically have to whitelist/mount the whole nix store, because we want all other local tools in the profile. It also means that the user cannot access any other files on their filesystem. Basically we are giving the user and the Nix shell binaries the exact same permissions.

Why are we copying the whole industry and building something similar to devcontainers? Lets use Nix and build something transparent and awesome. The goal is to sandbox every binary that is provided by the Nix shell. The binaries themselves are sandboxed, but the shell is not. We basically trust the user, but not the scripts.

This also results in some other nice benefits like scripts only being able to use tools that are defined in the Nix shell.

What do you think?

[zaytsev]

I quickly tried your implementation to confirm my suspicions that I then do not have zsh with starship configured inside the sandbox.

Right. I miss zsh/starship a little bit too. But I believe this is the way devenv shell works, it just spawns bash (don't confuse with direnv, which configures devenv environment inside of a current shell). Also I'm ok to trade some convince stuff (like starship) for a peace of mind, that none of malicious cargo build scripts coming from compromised 3rd-party dependency and/or AI agent gone rogue will be allowed to rm -rf $HOME or exfiltrate my secrets or private keys, etc.

Regarding to Landlock. Yes, I like the idea that app could apply filesystem restrictions on itself (for example, devenv executable could restrict its FS write permissions only to a workspace/project directory) and child processes will inherit this restrictions, though Landlock is not designed to control usern/networking/ipc/etc namespaces which might be handy in some cases. Anyway I started my experiment with Bubblewrap and it worked fine for my use case.

The goal is to sandbox every binary that is provided by the Nix shell.

I'm not sure if I like the idea of creating a wrapper for every executable of every package because while it adds some overhead it doesn't add any meaningful protection, bad actor could still execute original/unwrapped executable, right?

The binaries themselves are sandboxed, but the shell is not.

I'm not sure I understand this statement. Shell is also a binary, and therefore it have to be sandboxed, no?

Anyway... here is my current config

xdg.configFile."devenv/devenv.yaml".source = yamlFormat.generate "devenv-config" {
    sandbox = {
      enable = false;
      network = {
        enable = true;
      };
      mounts = [
        {
          path = "/nix/store";
        }
        {
          path = "/etc/passwd";
        }
        {
          path = "/etc/group";
        }
        {
          path = "/run/current-system/sw";
        }
        {
          path = "/run/current-system/sw/bin";
          dest = "/bin";
        }
        {
          path = "/run/current-system/sw/bin";
          dest = "/usr/bin";
        }
        {
          path = "/dev";
          mode = "dev";
        }
        {
          path = "/dev/null";
          mode = "dev";
        }
        {
          path = "/dev/random";
          mode = "dev";
        }
        {
          path = "/dev/urandom";
          mode = "dev";
        }
        {
          path = "$HOME";
          mode = "tmpfs";
        }
        {
          path = "$NIX_USER_PROFILE_DIR/profile";
          canonicalize = true;
        }
        {
          path = "$HOME/.nix-profile";
          canonicalize = true;
        }
        {
          path = "$HOME/.config/opencode";
        }
        {
          path = config.sops.secrets.opencode_api_key.path;
          canonicalize = true;
        }
        {
          path = config.sops.secrets.zai_api_key.path;
          canonicalize = true;
        }
        {
          path = config.sops.secrets.context7_api_key.path;
          canonicalize = true;
        }
        {
          path = "$HOME/.config/helix";
        }
        {
          path = "$HOME/.config/git";
        }
        {
          path = "$HOME/.local/state/opencode";
          mode = "state-dir";
        }
        {
          path = "$HOME/.cargo";
          mode = "state-dir";
        }
        {
          path = "$XDG_RUNTIME_DIR/podman/podman.sock";
          dest = "/var/run/docker.sock";
        }
        {
          path = "$PWD/target";
          mode = "tmpfs";
          late = true;
        }
      ];
    };
  };

which pretty much provides me with a functional but isolated rust development environment (no zsh/startship yet though 😞)

[LorenzBischof]

Right. I miss zsh/starship a little bit too. But I believe this is the way devenv shell works, it just spawns bash (don't confuse with direnv, which configures devenv environment inside of a current shell).

Of course 😞 Thats right.

I'm not sure if I like the idea of creating a wrapper for every executable of every package because while it adds some overhead it doesn't add any meaningful protection, bad actor could still execute original/unwrapped executable, right?

I think it does. Executables are either executed by the user (trusted, so will not run unwrapped) or by devenv (e.g. processes). A bad actor could run the unwrapped path, but is already restricted by Landlock, so he would not gain anything.

I'm not sure I understand this statement. Shell is also a binary, and therefore it have to be sandboxed, no?

Sorry for the confusion. The shell after running devenv shell or using direnv should NOT be sandboxed. This allows the developer to run any global commands or access all files on the filesystem without restrictions.

Basically everything that is added to PATH by devenv would be wrapped with a small CLI that adds Landlock restrictions.

[zaytsev]

OK. Thanks. I think I see now.

So sandboxing will be applied only to packages installed by devenv. User/system packages will remain unchanged, right?. And it should work seamlessly with direnv too.

And if I execute a shell script inside of a devenv containing something like

#/bin/sh

rm -rf ~

Then it is purely my fault 😅
It would also require to have all packages that could potentially execute code to be declared in devenv packages list (e.g. AI tools like claude-code, opencode, kilo, cline, codex, whatever your teammates might use, also all kind of editors, IDEs?)
And.. if $PATH contents gets modified or simply reordered then sandbox is no more.

I agree your approach with individual sandbox per executable might reduce supply chain attack risks and also prevent AI agents doing bad stuff. But it feels kind of fragile. I'd rather go with bubblewrap

[LorenzBischof]

Thanks for the back and forth. I think both solutions have their merits and flaws.

And if I execute a shell script inside of a devenv containing something like

That's a good point. The script would have to be declared in devenv.nix. I can't think of a way to circumvent that. I wonder if bwrap and landlock could be combined? /bin/sh would be symlinked to the wrapped executable in the nix store. The bwrap sandbox would be quite lax, with additional landlock rules... Then we could mount the whole home directory...

It would also require to have all packages that could potentially execute code to be declared in devenv packages list (e.g. AI tools like claude-code, opencode, kilo, cline, codex, whatever your teammates might use, also all kind of editors, IDEs?)

Yes, but the teammate can also just run claude-code without entering the devenv shell... 😀

Thanks for the brainstorming session. I did not want to hijack your PR 😆

[RafaelKr]

@zaytsev

Landlock is not designed to control usern/networking/ipc/etc namespaces which might be handy in some cases.

I'm not sure, if this may be available sooner or later, looking at their roadmap "In review" section: Landlock