From 2e5218b2d77e20184792d6f0f43ed8713c54aa3d Mon Sep 17 00:00:00 2001 From: ai-anant Date: Sun, 2 Aug 2026 14:40:02 +0530 Subject: [PATCH] feat: shared read-only toolchain volumes (one copy of Go/Rust/JDK for all projects) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go/Rust/Java toolchains were baked into every project image (~1-2GB each, compiled from source for gosec/cargo-audit). Move them to ONE shared named volume (aidc_toolchains), populated once and mounted READ-ONLY into every project container at /opt/toolchains — the 'mount snapshots of pre-verified tools as shared read-only memory' idea. - templates/devcontainer/Dockerfile.toolchain.tmpl (new): builds a store image per toolchain (go1.26.5 + gosec, rustup stable + cargo-audit, Temurin 21 + nothing) into /opt/toolchains-store, then copies into the volume when run. Pinned versions; content-hashed image tag. - lib/aidc.sh: aidc tools manages the volume; ensure_toolchain_volumes auto-populates on up/rebuild/rescan and before tool commands. - Dockerfile.tmpl: go/rust/java arms are now no-ops (volume provides them); PATH + GOROOT/RUSTUP_HOME/JAVA_HOME point at /opt/toolchains. - compose.yaml.tmpl: aidc_toolchains volume mounted read_only at /opt/toolchains. Verified end-to-end: store image builds, volume populates, go1.26.5 + gosec run from the mounted volume, and the read-only mount rejects writes. Refs #9 --- README.md | 5 +- docs/install.md | 4 + lib/aidc.sh | 123 ++++++++++++++++++ templates/devcontainer/Dockerfile.tmpl | 26 ++-- .../devcontainer/Dockerfile.toolchain.tmpl | 72 ++++++++++ templates/devcontainer/compose.yaml.tmpl | 8 ++ 6 files changed, 222 insertions(+), 16 deletions(-) create mode 100644 templates/devcontainer/Dockerfile.toolchain.tmpl diff --git a/README.md b/README.md index f7cb8a9..4407338 100644 --- a/README.md +++ b/README.md @@ -166,8 +166,9 @@ echo "AIDC_ISOLATE_VM=1" >> .ai-container/project.env ## Notes - Generated files are added to `.git/info/exclude` when the target directory is a git repo, so your project stays clean. The seeded project docs (`CHANGELOG.md`, `DETAILED_CHANGELOG.md`, `logs/`) are *not* excluded — they belong to your repo and are meant to be committed. -- Settings can be set host-wide in `~/.config/aidc/config.env` (universal defaults for every project) or per project in `.ai-container/project.env`, which overrides the global default. Both files are sourced for env vars like `AIDC_AUTO_SYNC_SESSIONS`, `AIDC_ENABLE_EGRESS_FIREWALL`, and `AIDC_ISOLATE_VM`. -- Container egress is open by default; set `AIDC_ENABLE_EGRESS_FIREWALL=1` in `.ai-container/project.env` for a default-deny allowlist. See [`docs/security.md`](docs/security.md#optional-egress-firewall). +- Settings can be set host-wide in `~/.config/aidc/config.env` (universal defaults for every project) or per project in `.ai-container/project.env`, which overrides the global default. Both files are sourced for env vars like `AIDC_AUTO_SYNC_SESSIONS`, `AIDC_ENABLE_EGRESS_FIREWALL`, `AIDC_ISOLATE_VM`, and `AIDC_AGENTS`. +- **Projects share one `aidc-base` image.** The common layer (base OS, apt tools, uv/Python, agents, scanners) is built once as `aidc-base:` and reused by every project via `FROM aidc-base` — a fresh project only builds its toolchain delta (minutes, not tens of minutes). `docker images` shows the full logical size per project, but on disk the base layers are shared; N projects cost ~3 GB once + small deltas. Pin a custom base with `AIDC_BASE_IMAGE=` in `.ai-container/project.env`. +- **Toolchains are shared read-only, not baked per project.** `aidc tools install go|rust|java|all` populates one named volume (`aidc_toolchains`) with Go/Rust/JDK once; every project container mounts it **read-only** at `/opt/toolchains`. One copy of Go serves all projects (the "mount pre-verified tools as shared read-only memory" idea), agents can't tamper with it, and revoking a bad toolchain is one command (`docker volume rm aidc_toolchains` + `aidc tools install`). `aidc up` auto-ensures the toolchains a project needs; `aidc tools status` shows what's installed. - The host-clipboard bridge is **off by default** — no host clipboard socket is mounted into the container. Opt in per (re)create with `aidc up --clipboard` / `aidc rebuild --clipboard`, or persist `AIDC_ENABLE_CLIPBOARD=1` in `.ai-container/project.env`. See [`docs/clipboard-bridge.md`](docs/clipboard-bridge.md). - Per-project VM isolation (`--isolate-vm`) is **off by default** due to resource cost. Opt in per (re)create with `aidc up --isolate-vm` / `aidc rebuild --isolate-vm`, or persist `AIDC_ISOLATE_VM=1` in `.ai-container/project.env`. See [Isolation modes](#isolation-modes) above. - Generated Claude alias wrappers are `aidc`-managed and live in `~/.local/bin` by default. diff --git a/docs/install.md b/docs/install.md index c872ecf..d2fddce 100644 --- a/docs/install.md +++ b/docs/install.md @@ -44,6 +44,8 @@ aidc up # build + start container | `aidc down` | stop the container, keep volumes | | `aidc rebuild` | rebuild the image and restart | | `aidc destroy` | remove container + volumes + image (prompts; `-f` to skip) | +| `aidc tools install [go\|rust\|java\|all]` | populate the shared read-only toolchain volume | +| `aidc tools status` | show which shared toolchains are installed | ## What lives where (inside the container) @@ -81,6 +83,8 @@ Node and Python markers don't trigger a language install (the base image already The detected list is passed as a Docker `--build-arg AIDC_TOOLCHAINS=go,rust,...` so it caches per combination — switching between repos doesn't rebuild. +Go, Rust, and Java toolchains are **not baked into the image** — they live in a shared read-only volume (`aidc_toolchains`) populated once by `aidc tools install` and mounted at `/opt/toolchains` in every project container. `aidc up` auto-ensures the toolchains a project needs. This keeps per-project images small (no per-project copy of Go/Rust/JDK) and lets you revoke a bad toolchain everywhere by repopulating the volume. Ruby, PHP, and shell toolchains (and their linters) still install into the image. + **Override** in `.ai-container/project.env`: ```bash diff --git a/lib/aidc.sh b/lib/aidc.sh index c526fd5..573b6c1 100644 --- a/lib/aidc.sh +++ b/lib/aidc.sh @@ -98,6 +98,9 @@ aidc::main() { cursor) aidc::cmd_cursor ;; + tools) + aidc::cmd_tools "$@" + ;; sync-claude-aliases) aidc::cmd_sync_claude_aliases ;; @@ -139,6 +142,7 @@ Usage: aidc sync-claude-aliases aidc sync-config aidc sync-sessions [claude|codex|opencode|grok|all] + aidc tools Notes: - Run commands from the repo root you want to isolate. @@ -153,6 +157,9 @@ Notes: ~/.claude/projects so '/insights' on the host can see them. Sessions also auto-sync on container start, agent exit, 'down', and 'destroy' unless AIDC_AUTO_SYNC_SESSIONS=0. + - aidc tools manages the shared read-only toolchain volume (one copy of + Go/Rust/JDK serves all projects). 'install' populates it; 'status' shows + what's present. - The host-clipboard bridge is off by default. Enable it at (re)create time with 'aidc up --clipboard' or 'aidc rebuild --clipboard'. - Per-project VM isolation is off by default due to resource cost. Enable it @@ -237,6 +244,7 @@ aidc::cmd_up() { aidc::vm_ensure "$workspace" fi + aidc::ensure_toolchain_volumes "$workspace" aidc::compose "$workspace" up -d --build workspace # Catch up the host with any transcripts a prior (possibly ungraceful) session # left in the volume — the recovery case the on-exit hooks can't cover. @@ -254,6 +262,7 @@ aidc::cmd_rebuild() { aidc::vm_ensure "$workspace" fi + aidc::ensure_toolchain_volumes "$workspace" aidc::compose "$workspace" up -d --build --force-recreate workspace aidc::auto_sync_sessions "$workspace" all aidc::log "container rebuilt for $(basename "$workspace")" @@ -282,6 +291,7 @@ aidc::cmd_rescan() { fi # --build picks up the changed AIDC_TOOLCHAINS build arg and reinstalls. + aidc::ensure_toolchain_volumes "$workspace" aidc::compose "$workspace" up -d --build workspace aidc::log "rescan complete for $(basename "$workspace")" } @@ -1529,6 +1539,7 @@ aidc::ensure_container_running() { fi if [[ -z "$(aidc::compose_capture "$workspace" ps -q workspace)" ]]; then + aidc::ensure_toolchain_volumes "$workspace" aidc::compose "$workspace" up -d --build workspace # Only on the down→up transition (not every exec), so we recover prior # transcripts without adding a sync to each command. @@ -1741,6 +1752,118 @@ aidc::compose() { docker compose -f "$workspace/.devcontainer/compose.yaml" "$@" } +# ─── Shared read-only toolchain store (aidc tools) ─── +# +# One named volume (aidc_toolchains) holds Go/Rust/JDK, populated once by +# 'aidc tools install' and mounted READ-ONLY into every project container at +# /opt/toolchains. This is the "one copy of Go serves 10,000 sandboxes" idea: +# projects share the toolchains on disk instead of baking ~2-4GB into each +# image, and revoking a bad toolchain (repopulate the volume) stops it +# everywhere at once. + +AIDC_TOOLCHAIN_VOLUME="${AIDC_TOOLCHAIN_VOLUME:-aidc_toolchains}" +AIDC_TOOLCHAIN_STORE_IMAGE_PREFIX="aidc-toolchain-store" + +aidc::toolchain_image_tag() { + local lang="$1" + local hash cmd + if command -v sha256sum >/dev/null 2>&1; then + cmd="sha256sum" + else + cmd="shasum -a 256" + fi + hash="$($cmd "$AIDC_ROOT/templates/devcontainer/Dockerfile.toolchain.tmpl" 2>/dev/null | awk '{print $1}' | cut -c1-12)" + printf '%s-%s:%s' "$AIDC_TOOLCHAIN_STORE_IMAGE_PREFIX" "$lang" "${hash:-latest}" +} + +# Build the toolchain store image (once per content hash) if missing. +aidc::ensure_toolchain_image() { + local lang="$1" + local tag + tag="$(aidc::toolchain_image_tag "$lang")" + if ! docker image inspect "$tag" >/dev/null 2>&1; then + aidc::log "building toolchain store image $tag (one-time)" + ( cd "$AIDC_ROOT" \ + && docker build \ + -f "templates/devcontainer/Dockerfile.toolchain.tmpl" \ + --build-arg AIDC_TOOLCHAIN="$lang" \ + -t "$tag" templates/devcontainer ) \ + || aidc::die "failed to build toolchain store image for $lang" + aidc::log "toolchain store image $tag ready" + fi +} + +# Populate the shared volume with one toolchain, if not already present. +aidc::ensure_toolchain_volume() { + local lang="$1" + local tag + tag="$(aidc::toolchain_image_tag "$lang")" + docker volume inspect "$AIDC_TOOLCHAIN_VOLUME" >/dev/null 2>&1 \ + || docker volume create "$AIDC_TOOLCHAIN_VOLUME" >/dev/null + aidc::ensure_toolchain_image "$lang" + if ! docker run --rm -v "$AIDC_TOOLCHAIN_VOLUME:/opt/toolchains" \ + --entrypoint test "$tag" -f "/opt/toolchains/$lang/.aidc-ready" >/dev/null 2>&1; then + aidc::log "populating shared $lang toolchain in $AIDC_TOOLCHAIN_VOLUME (one-time)" + docker run --rm -v "$AIDC_TOOLCHAIN_VOLUME:/opt/toolchains" "$tag" \ + || aidc::die "failed to populate $lang toolchain volume" + aidc::log "shared $lang toolchain ready" + fi +} + +# Ensure every toolchain the workspace needs is present in the shared volume. +aidc::ensure_toolchain_volumes() { + local workspace="$1" + aidc::export_compose_env "$workspace" + local tc + for tc in $(echo "${AIDC_TOOLCHAINS:-}" | tr ',' ' '); do + case "$tc" in + go|rust|java) aidc::ensure_toolchain_volume "$tc" ;; + *) : ;; + esac + done +} + +aidc::cmd_tools() { + local sub="${1:-status}" + case "$sub" in + install|ensure) + shift + local lang="${1:-all}" + case "$lang" in + all) aidc::ensure_toolchain_volume go; aidc::ensure_toolchain_volume rust; aidc::ensure_toolchain_volume java ;; + go|rust|java) aidc::ensure_toolchain_volume "$lang" ;; + *) aidc::die "usage: aidc tools install [go|rust|java|all]" ;; + esac + ;; + status) + aidc::tools_status + ;; + *) + aidc::die "usage: aidc tools " + ;; + esac +} + +aidc::tools_status() { + printf 'shared toolchain volume: %s\n' "$AIDC_TOOLCHAIN_VOLUME" + if ! docker volume inspect "$AIDC_TOOLCHAIN_VOLUME" >/dev/null 2>&1; then + # shellcheck disable=SC2016 # backticks are literal text, not command substitution + printf ' (not created yet — run `aidc tools install all`)\n' + return 0 + fi + for lang in go rust java; do + local tag + tag="$(aidc::toolchain_image_tag "$lang")" + if docker image inspect "$tag" >/dev/null 2>&1 \ + && docker run --rm -v "$AIDC_TOOLCHAIN_VOLUME:/opt/toolchains" \ + --entrypoint test "$tag" -f "/opt/toolchains/$lang/.aidc-ready" >/dev/null 2>&1; then + printf ' %-5s installed\n' "$lang" + else + printf ' %-5s not installed\n' "$lang" + fi + done +} + aidc::compose_capture() { local workspace="$1" shift diff --git a/templates/devcontainer/Dockerfile.tmpl b/templates/devcontainer/Dockerfile.tmpl index bae5a6f..deb70ca 100644 --- a/templates/devcontainer/Dockerfile.tmpl +++ b/templates/devcontainer/Dockerfile.tmpl @@ -190,7 +190,15 @@ ENV HOME=/home/vscode # shims, NOT on the shell aliases pmg also writes to ~/.zshrc/~/.bashrc: build # RUN steps and exec'd agent subprocesses never source rc files, so only the # shims (first on PATH at build and runtime) reliably gate npm/pip/uv/etc. -ENV PATH=/home/vscode/.pmg/bin:/home/vscode/.local/bin:/usr/local/bin:/usr/bin:/bin +# +# The shared toolchain volume is mounted read-only at /opt/toolchains by +# compose (one copy of Go/Rust/JDK serves all projects — see +# Dockerfile.toolchain). Its bin dirs are on PATH; empty dirs are harmless +# when the matching toolchain isn't installed. +ENV PATH=/home/vscode/.pmg/bin:/home/vscode/.local/bin:/opt/toolchains/go/bin:/opt/toolchains/rust/bin:/opt/toolchains/java/bin:/usr/local/bin:/usr/bin:/bin +ENV GOROOT=/opt/toolchains/go \ + RUSTUP_HOME=/opt/toolchains/rust/rustup \ + JAVA_HOME=/opt/toolchains/java # Wire pmg in before ANY user-level package install so nothing slips past it. RUN pmg setup install \ @@ -224,25 +232,15 @@ RUN if [ -n "$AIDC_TOOLCHAINS" ]; then \ for tc in $(echo "$AIDC_TOOLCHAINS" | tr ',' ' '); do \ case "$tc" in \ go) \ - sudo apt-get update && \ - sudo apt-get install -y --no-install-recommends golang-go && \ - GOBIN=/home/vscode/.local/bin go install github.com/securego/gosec/v2/cmd/gosec@latest ;; \ + echo "aidc: go toolchain + gosec provided by shared volume (/opt/toolchains/go); run 'aidc tools install go'" ;; \ rust) \ - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \ - | sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path && \ - mkdir -p /home/vscode/.local/bin && \ - ln -sf "$HOME/.cargo/bin/cargo" /home/vscode/.local/bin/cargo && \ - ln -sf "$HOME/.cargo/bin/rustc" /home/vscode/.local/bin/rustc && \ - ln -sf "$HOME/.cargo/bin/rustup" /home/vscode/.local/bin/rustup && \ - "$HOME/.cargo/bin/cargo" install cargo-audit --locked && \ - ln -sf "$HOME/.cargo/bin/cargo-audit" /home/vscode/.local/bin/cargo-audit ;; \ + echo "aidc: rust toolchain + cargo-audit provided by shared volume (/opt/toolchains/rust); run 'aidc tools install rust'" ;; \ ruby) \ sudo apt-get update && \ sudo apt-get install -y --no-install-recommends ruby-full && \ sudo gem install --no-document bundler-audit ;; \ java) \ - sudo apt-get update && \ - sudo apt-get install -y --no-install-recommends default-jdk ;; \ + echo "aidc: java toolchain provided by shared volume (/opt/toolchains/java); run 'aidc tools install java'" ;; \ php) \ sudo apt-get update && \ sudo apt-get install -y --no-install-recommends php-cli ;; \ diff --git a/templates/devcontainer/Dockerfile.toolchain.tmpl b/templates/devcontainer/Dockerfile.toolchain.tmpl new file mode 100644 index 0000000..b95cc47 --- /dev/null +++ b/templates/devcontainer/Dockerfile.toolchain.tmpl @@ -0,0 +1,72 @@ +# aidc-managed +# Shared toolchain store image: builds ONE toolchain (go | rust | java) into +# /opt/toolchains-store/ and, when run with the shared aidc_toolchains +# volume mounted at /opt/toolchains, copies it into the volume. +# +# The volume is then mounted READ-ONLY into every project container at +# /opt/toolchains — one copy of Go/Rust/JDK serves all projects ("mount +# snapshots of pre-verified tools as shared read-only memory"). Revoke a bad +# toolchain once: `docker volume rm aidc_toolchains` + `aidc tools ensure` +# repopulates it everywhere. +# +# Built once per (content hash, lang) by aidc::ensure_toolchain_volume. + +ARG AIDC_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:ubuntu-24.04@sha256:d94c97dd9cacf183d0a6fd12a8e87b526e9e928307674ae9c94139139c0c6eae +FROM ${AIDC_BASE_IMAGE} + +ARG AIDC_TOOLCHAIN +ENV DEBIAN_FRONTEND=noninteractive + +USER root + +RUN set -eux; \ + ARCH="$(dpkg --print-architecture)"; \ + case "$ARCH" in \ + amd64) GOARCH="amd64"; JDKARCH="x64" ;; \ + arm64) GOARCH="arm64"; JDKARCH="aarch64" ;; \ + *) echo "unsupported arch: $ARCH" >&2; exit 1 ;; \ + esac; \ + mkdir -p /opt/toolchains-store; \ + case "$AIDC_TOOLCHAIN" in \ + go) \ + curl -fsSL "https://go.dev/dl/go1.26.5.linux-${GOARCH}.tar.gz" -o /tmp/go.tgz \ + && tar -C /opt/toolchains-store -xzf /tmp/go.tgz \ + && rm /tmp/go.tgz \ + && GOROOT=/opt/toolchains-store/go \ + GOBIN=/opt/toolchains-store/go/bin \ + GOPATH=/tmp/gopath GOCACHE=/tmp/gocache \ + /opt/toolchains-store/go/bin/go install github.com/securego/gosec/v2/cmd/gosec@latest \ + && rm -rf /tmp/gopath /tmp/gocache \ + && touch /opt/toolchains-store/go/.aidc-ready \ + ;; \ + rust) \ + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \ + | RUSTUP_HOME=/opt/toolchains-store/rust/rustup \ + CARGO_HOME=/opt/toolchains-store/rust/cargo \ + sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path \ + && mkdir -p /opt/toolchains-store/rust/bin \ + && for b in cargo rustc rustup; do \ + ln -sf "/opt/toolchains-store/rust/cargo/bin/$b" "/opt/toolchains-store/rust/bin/$b"; \ + done \ + && RUSTUP_HOME=/opt/toolchains-store/rust/rustup \ + CARGO_HOME=/opt/toolchains-store/rust/cargo \ + CARGO_TARGET_DIR=/tmp/cargo-target \ + /opt/toolchains-store/rust/cargo/bin/cargo install cargo-audit --locked \ + && ln -sf "/opt/toolchains-store/rust/cargo/bin/cargo-audit" "/opt/toolchains-store/rust/bin/cargo-audit" \ + && rm -rf /tmp/cargo-target /opt/toolchains-store/rust/cargo/registry \ + && touch /opt/toolchains-store/rust/.aidc-ready \ + ;; \ + java) \ + curl -fsSL "https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.12%2B8/OpenJDK21U-jdk_${JDKARCH}_linux_hotspot_21.0.12_8.tar.gz" -o /tmp/jdk.tgz \ + && mkdir -p /opt/toolchains-store/java \ + && tar -C /opt/toolchains-store/java -xzf /tmp/jdk.tgz --strip-components=1 \ + && rm /tmp/jdk.tgz \ + && touch /opt/toolchains-store/java/.aidc-ready \ + ;; \ + *) \ + echo "unsupported toolchain: $AIDC_TOOLCHAIN" >&2; exit 1 ;; \ + esac; \ + chmod -R a+rX /opt/toolchains-store + +# Populate the mounted shared volume with the built toolchain. +ENTRYPOINT ["sh", "-c", "cp -a /opt/toolchains-store/. /opt/toolchains/"] diff --git a/templates/devcontainer/compose.yaml.tmpl b/templates/devcontainer/compose.yaml.tmpl index cc520fe..85170ff 100644 --- a/templates/devcontainer/compose.yaml.tmpl +++ b/templates/devcontainer/compose.yaml.tmpl @@ -64,6 +64,13 @@ services: - type: volume source: command_history target: /commandhistory + # Shared read-only toolchain store: one copy of Go/Rust/JDK serves all + # projects (populated by 'aidc tools install'). Read-only so agents + # can't tamper with the shared toolchain; revoke by repopulating. + - type: volume + source: aidc_toolchains + target: /opt/toolchains + read_only: true - type: volume source: claude_home target: /home/vscode/.claude @@ -82,6 +89,7 @@ services: volumes: command_history: + aidc_toolchains: claude_home: codex_home: opencode_home: