diff --git a/README.md b/README.md index d7e205f..c66ebfe 100644 --- a/README.md +++ b/README.md @@ -224,6 +224,173 @@ advisory, timestamp, and run number, in the same format as `trace --debug`: uv run python -m code_audit eval --debug ``` +## Dockerizing an advisory's repository + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q +``` + +When the advisory's repository looks like a web application and has no working +Dockerfile or compose file, this generates one, verifies that it actually builds and +runs, and writes it to `./output//`. It is a separate, explicitly opt-in +pipeline: it never runs as part of `trace`, and it requires `git` and Docker on the +machine running it, since it clones and executes real code from the repository being +audited. + +`dockerize` is meant to run once an advisory's commits are already known, not to +re-investigate commit provenance itself: a deterministic-only pass over the advisory's +own references can miss cases the full trace agent resolves. For example, +GHSA-v98v-ff95-f3cp (n8n) references three different commits with no merged pull +request to confirm which one actually fixed it; the deterministic pass correctly +returns no fixing commit rather than guessing among them, and only the full trace +agent's reasoning can pick the right one. So before doing anything else, `dockerize` +needs to know the fixing commit. With none of the three flags below given, it asks +interactively: + +``` +Trace this advisory now to resolve its commits? [y/N]: +``` + +Answering yes runs the same deterministic pass plus full trace agent `trace` uses, and +uses the result directly. Answering no prompts for either a fixing commit SHA pasted +directly, or a path to a trace result JSON file (the same JSON `trace` prints to +stdout); a bare SHA is refetched through the GitHub API for full detail. For +non-interactive use (scripts, CI), three mutually exclusive flags skip the prompt +entirely: + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --trace +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --trace-result trace.json +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --fixing-commit c77b3cb39312b83b053d23a2158b99ac7de44dd3 +``` + +Once the fixing commit is known (however it was resolved), the rest of the pipeline +runs. Because a full run can take minutes with otherwise no other output, every +long-running step prints a short line to stderr as it starts and finishes: resolving +the target commit, tracing (if chosen), cloning, classifying the repository, checking +for existing artifacts, each generation turn, the build-and-start phase, the health +check, and each repair attempt. Concretely, the pipeline: + +1. Resolves the commit to build: the git tag for the advisory's last affected version + when one can be found (this does not depend on the fixing commit at all), otherwise + the parent of the fixing commit resolved above. If neither can be resolved, it exits + with an explanation rather than guessing a commit. +2. Shallow-clones the repository at exactly that commit into a temporary directory. +3. Classifies the repository as an application or a library, in two tiers. Tier 1 is + file-based signals (an entrypoint like `manage.py` or a `scripts.start` in + `package.json` versus a build-backend-only `pyproject.toml` or a `setup.py` with + nothing to run) and is trusted outright once its signals clearly favor one side. + When they do not, a Tier 2 Anthropic call settles it from a compact digest (the + file tree, any root-level manifest file, and a README excerpt) instead of another + hand-added deterministic rule for the next packaging convention that turns up. Most + GHSA advisories target libraries, not applications, so this step exists to skip + those rather than generate a Dockerfile for something that is never run as a + service. +4. Searches the whole checkout for an existing Dockerfile, compose file, `docker/` + directory, or `.devcontainer/`. A compose file only counts as already present when + some service actually builds from the repository or references its image; a compose + file that only defines infrastructure (a database, a cache) does not. +5. If generation is needed, an Anthropic agent reads the repository's files and produces + a Dockerfile (and a compose.yml, only if the application needs more than one service). +6. Builds and runs the result in an isolated, sandboxed compose project. A small helper + container joins the same isolated network and polls the application over HTTP from + inside it until the application answers or a timeout is hit. Whether verification + succeeds or fails, the containers, images, and temporary checkout are always removed + afterward. +7. If verification fails and repair attempts remain (`--repair-attempts`, default 2), + the failure is classified deterministically (the build image failed to pull, + a dependency failed to resolve or install, the build succeeded but the container + never started, or the container started but the health check timed out) and fed + back to the agent together with the previous Dockerfile/compose.yml and the actual + build/run log, asking it to fix that specific failure rather than start over. This + repeats until verification succeeds or repair attempts run out. + +The sandboxing on that last step, in plain terms: + +- **No bind mounts from the host.** Any volume the generated compose declares that + points at a host path is stripped before it runs; only Docker-managed named volumes + are kept. +- **No extra privileges.** Every container runs with `cap_drop: [ALL]` and no added + capabilities, and `privileged`, `cap_add`, and `network_mode: host` are all stripped + if the generated compose sets any of them. +- **No network access once running.** Every service, including the application, joins + one Docker network created with `internal: true` for the verification run, which + blocks all outbound traffic to the internet. This only restricts the *running* + containers: the build phase (`docker build`) still has normal network access, since + installing dependencies needs it. An internal network also has no route from the + host, so a small helper container (not the generated application) joins the same + network and checks over HTTP from inside it; nothing is published to the host. +- **Bounded resources and time.** Each container is capped at 512MB of memory, 1 CPU, + and 256 processes; the build-and-start phase and the health check both have a hard + wall-clock timeout, so a hung or resource-hungry container cannot stall the command + indefinitely. The build-and-start timeout defaults to 300 seconds but is configurable + with `--build-timeout`, since a large monorepo can legitimately need much longer than + a typical small application. +- **No exposed secrets.** The `docker compose` subprocess is given a minimal, + explicitly allowlisted environment (`PATH`, `HOME`, and the handful of `DOCKER_*` + variables the CLI itself needs to find the right daemon), not the full host + environment with a couple of names removed. Nothing else on the host, including + unrelated secrets the tool never touches otherwise, is reachable through the + generated compose file's variable interpolation. + +This is meaningfully more trust than the rest of the tool extends: `trace` only ever +reads data through the GitHub API, while `dockerize` builds and executes code from the +audited repository, which is itself the subject of a security advisory. Treat the +sandboxing above as containment for a real build-and-run step, not as a guarantee that +nothing in the repository can do anything unwanted. + +On success, the command prints the resolved target commit and the output path. On +failure (an unresolvable commit, a repository that does not look like a web +application, existing artifacts already in place, a generation failure, or a +verification failure) it prints the reason and exits non-zero; it never retries or +discards a failed attempt silently. A verification failure also copies the generated +Dockerfile/compose to the output path and writes the full, untruncated build output to +`build.log` there: the terminal-facing summary is only the last 500 lines, which can +hide the actual cause of a failure (a truncated tail once hid a real dependency +install error behind a generic "command failed with exit code 1"). + +Add `--force` to generate and verify even when the repository already has a +Dockerfile or compose file, purely to exercise generation and verification end to +end against a repository you know is already containerized. It still skips a +repository the classifier identifies as a library: `--force` only bypasses the +already-present check, never the application-vs-library judgment. When it does +bypass that check, the command prints a note that the existing artifact is being +overwritten in this run's temporary checkout only, never in the real repository, +since nothing here is pushed upstream: + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --force +``` + +Add `--build-timeout` (seconds, default 300) to raise the build-and-start timeout for a +repository that legitimately needs longer, for example a large monorepo compiling a Go +backend alongside a large frontend bundle in the same image: + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --build-timeout 900 +``` + +Adjust `--repair-attempts` (default 2) to change how many extra generate-and-verify +tries run after a verification failure before giving up. Each attempt is a full +generate-plus-verify cycle: a fresh Anthropic generation call and another sandboxed +build and health check, so cost and time scale with it. Set it to `0` to go back to a +single, one-shot attempt: + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --repair-attempts 0 +``` + +Add `--debug` to record the generation agent's turns as JSON Lines under `debug/`, in +the same format `trace --debug` uses: one line per turn (thinking summary, tool calls, +truncated tool results, stop reason), and on the terminal turn the raw response text +before it is validated, which is useful for seeing exactly what the model returned when +generation fails (for example, an empty or `FROM`-less Dockerfile, which is rejected +rather than left for Docker itself to fail on with a much less specific error): + +```sh +uv run python -m code_audit dockerize GHSA-jfh8-c2jp-5v3q --debug +``` + ## Development ```sh diff --git a/output/GHSA-mqpq-2p68-46fv/Dockerfile b/output/GHSA-mqpq-2p68-46fv/Dockerfile new file mode 100644 index 0000000..a9466a2 --- /dev/null +++ b/output/GHSA-mqpq-2p68-46fv/Dockerfile @@ -0,0 +1,34 @@ +FROM python:3.10-slim-bullseye + +ENV PYTHONUNBUFFERED=1 \ + PYCURL_SSL_LIBRARY=openssl \ + LANG=C.UTF-8 + +# libcurl4-openssl-dev + build tools are required to compile pycurl from source +RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential \ + libcurl4-openssl-dev \ + libssl-dev \ + && rm -rf /var/lib/apt/lists/* + +WORKDIR /app +COPY . . + +# Patch the default config so the web UI binds on all interfaces +# instead of localhost only (which is unreachable from outside the container) +RUN sed -i \ + 's/ip host : "IP address" = localhost/ip host : "IP address" = 0.0.0.0/' \ + src/pyload/core/config/default.cfg + +# Install the core package from local source then pull in the +# companion web-UI package from PyPI (pyload.webui namespace package) +RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \ + pip install --no-cache-dir . && \ + pip install --no-cache-dir pyload-ng-webui + +# Directories used at runtime (config + downloads) +RUN mkdir -p /root/.pyload /downloads + +EXPOSE 8000 + +CMD ["python", "-m", "pyload"] diff --git a/output/GHSA-mqpq-2p68-46fv/build.log b/output/GHSA-mqpq-2p68-46fv/build.log new file mode 100644 index 0000000..640bf99 --- /dev/null +++ b/output/GHSA-mqpq-2p68-46fv/build.log @@ -0,0 +1,617 @@ +#1 [internal] load local bake definitions +#1 reading from stdin 565B done +#1 DONE 0.0s + +#2 [internal] load build definition from Dockerfile +#2 transferring dockerfile: 1.11kB done +#2 DONE 0.0s + +#3 [internal] load metadata for docker.io/library/python:3.10-slim-bullseye +#3 DONE 1.2s + +#4 [internal] load .dockerignore +#4 transferring context: 83B done +#4 DONE 0.0s + +#5 [1/7] FROM docker.io/library/python:3.10-slim-bullseye@sha256:f1fb49e4d5501ac93d0ca519fb7ee6250842245aba8612926a46a0832a1ed089 +#5 resolve docker.io/library/python:3.10-slim-bullseye@sha256:f1fb49e4d5501ac93d0ca519fb7ee6250842245aba8612926a46a0832a1ed089 0.0s done +#5 ... + +#6 [internal] load build context +#6 transferring context: 4.11MB 0.2s done +#6 DONE 0.2s + +#5 [1/7] FROM docker.io/library/python:3.10-slim-bullseye@sha256:f1fb49e4d5501ac93d0ca519fb7ee6250842245aba8612926a46a0832a1ed089 +#5 sha256:7360b233c86094e10bc61af00bf28bd367deb30a63c6a54ad91429cf7db86de1 250B / 250B 0.1s done +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 1.05MB / 14.85MB 0.3s +#5 sha256:5e584f8f28c34b1709b431a08b1ae3892c8ab2c82c8da1f98148f589a2a664f1 0B / 1.08MB 0.2s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 0B / 30.26MB 0.2s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 3.15MB / 14.85MB 0.5s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 5.24MB / 14.85MB 0.6s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 7.34MB / 14.85MB 0.8s +#5 sha256:5e584f8f28c34b1709b431a08b1ae3892c8ab2c82c8da1f98148f589a2a664f1 1.08MB / 1.08MB 0.6s done +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 9.44MB / 14.85MB 0.9s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 11.47MB / 14.85MB 1.1s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 13.63MB / 14.85MB 1.2s +#5 sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 14.85MB / 14.85MB 1.3s done +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 2.10MB / 30.26MB 1.2s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 4.19MB / 30.26MB 1.5s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 7.34MB / 30.26MB 1.8s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 10.49MB / 30.26MB 2.0s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 12.58MB / 30.26MB 2.1s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 14.68MB / 30.26MB 2.3s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 16.78MB / 30.26MB 2.4s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 19.92MB / 30.26MB 2.6s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 22.02MB / 30.26MB 2.7s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 24.12MB / 30.26MB 2.9s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 27.26MB / 30.26MB 3.0s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 30.26MB / 30.26MB 3.2s +#5 sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 30.26MB / 30.26MB 3.2s done +#5 extracting sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d +#5 extracting sha256:ccaf924377f936af2c0396fce237145b7d1ecc0b8196916667fc6d5ff4866e2d 0.6s done +#5 DONE 4.0s + +#5 [1/7] FROM docker.io/library/python:3.10-slim-bullseye@sha256:f1fb49e4d5501ac93d0ca519fb7ee6250842245aba8612926a46a0832a1ed089 +#5 extracting sha256:5e584f8f28c34b1709b431a08b1ae3892c8ab2c82c8da1f98148f589a2a664f1 0.1s done +#5 extracting sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 +#5 extracting sha256:954774345a619d861c3805b1b0100b7319d4eaf3108b2659e6f1bf2e4f095394 0.4s done +#5 DONE 4.5s + +#5 [1/7] FROM docker.io/library/python:3.10-slim-bullseye@sha256:f1fb49e4d5501ac93d0ca519fb7ee6250842245aba8612926a46a0832a1ed089 +#5 extracting sha256:7360b233c86094e10bc61af00bf28bd367deb30a63c6a54ad91429cf7db86de1 0.0s done +#5 DONE 4.5s + +#7 [2/7] RUN apt-get update && apt-get install -y --no-install-recommends build-essential libcurl4-openssl-dev libssl-dev && rm -rf /var/lib/apt/lists/* +#7 0.272 Get:1 http://deb.debian.org/debian bullseye InRelease [75.1 kB] +#7 0.297 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [27.2 kB] +#7 0.298 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.0 kB] +#7 0.470 Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8066 kB] +#7 1.046 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [461 kB] +#7 1.067 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB] +#7 1.806 Fetched 8692 kB in 2s (5528 kB/s) +#7 1.806 Reading package lists... +#7 2.181 Reading package lists... +#7 2.545 Building dependency tree... +#7 2.636 Reading state information... +#7 2.740 The following additional packages will be installed: +#7 2.740 binutils binutils-common binutils-x86-64-linux-gnu bzip2 cpp cpp-10 dpkg-dev +#7 2.740 g++ g++-10 gcc gcc-10 libasan6 libatomic1 libbinutils libbrotli1 +#7 2.740 libc-dev-bin libc6 libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 +#7 2.740 libcurl4 libdpkg-perl libgcc-10-dev libgdbm-compat4 libgomp1 libisl23 +#7 2.740 libitm1 libldap-2.4-2 liblsan0 libmpc3 libmpfr6 libnghttp2-14 libnsl-dev +#7 2.740 libperl5.32 libpsl5 libquadmath0 librtmp1 libsasl2-2 libsasl2-modules-db +#7 2.740 libssh2-1 libssl1.1 libstdc++-10-dev libtirpc-dev libtsan0 libubsan1 +#7 2.740 linux-libc-dev make patch perl perl-base perl-modules-5.32 xz-utils +#7 2.741 Suggested packages: +#7 2.741 binutils-doc bzip2-doc cpp-doc gcc-10-locales debian-keyring g++-multilib +#7 2.741 g++-10-multilib gcc-10-doc gcc-multilib manpages-dev autoconf automake +#7 2.741 libtool flex bison gdb gcc-doc gcc-10-multilib glibc-doc libc-l10n locales +#7 2.741 libcurl4-doc libidn11-dev libkrb5-dev libldap2-dev librtmp-dev libssh2-1-dev +#7 2.741 pkg-config zlib1g-dev gnupg sensible-utils git bzr libssl-doc +#7 2.741 libstdc++-10-doc make-doc ed diffutils-doc perl-doc +#7 2.741 libterm-readline-gnu-perl | libterm-readline-perl-perl +#7 2.741 libtap-harness-archive-perl +#7 2.741 Recommended packages: +#7 2.741 fakeroot gnupg libalgorithm-merge-perl manpages manpages-dev libc-devtools +#7 2.741 libnss-nis libnss-nisplus libfile-fcntllock-perl liblocale-gettext-perl +#7 2.741 libldap-common publicsuffix libsasl2-modules +#7 2.911 The following NEW packages will be installed: +#7 2.911 binutils binutils-common binutils-x86-64-linux-gnu build-essential bzip2 cpp +#7 2.911 cpp-10 dpkg-dev g++ g++-10 gcc gcc-10 libasan6 libatomic1 libbinutils +#7 2.911 libbrotli1 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 +#7 2.911 libctf0 libcurl4 libcurl4-openssl-dev libdpkg-perl libgcc-10-dev +#7 2.911 libgdbm-compat4 libgomp1 libisl23 libitm1 libldap-2.4-2 liblsan0 libmpc3 +#7 2.911 libmpfr6 libnghttp2-14 libnsl-dev libperl5.32 libpsl5 libquadmath0 librtmp1 +#7 2.911 libsasl2-2 libsasl2-modules-db libssh2-1 libssl-dev libstdc++-10-dev +#7 2.911 libtirpc-dev libtsan0 libubsan1 linux-libc-dev make patch perl +#7 2.912 perl-modules-5.32 xz-utils +#7 2.912 The following packages will be upgraded: +#7 2.913 libc6 libssl1.1 perl-base +#7 2.953 3 upgraded, 54 newly installed, 0 to remove and 16 not upgraded. +#7 2.953 Need to get 79.1 MB of archives. +#7 2.953 After this operation, 289 MB of additional disk space will be used. +#7 2.953 Get:1 http://deb.debian.org/debian-security bullseye-security/main amd64 perl-base amd64 5.32.1-4+deb11u5 [1629 kB] +#7 3.044 Get:2 http://deb.debian.org/debian-security bullseye-security/main amd64 perl-modules-5.32 all 5.32.1-4+deb11u5 [2823 kB] +#7 3.199 Get:3 http://deb.debian.org/debian-security bullseye-security/main amd64 libc6 amd64 2.31-13+deb11u14 [2820 kB] +#7 3.354 Get:4 http://deb.debian.org/debian bullseye/main amd64 libgdbm-compat4 amd64 1.19-2 [44.7 kB] +#7 3.356 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 libperl5.32 amd64 5.32.1-4+deb11u5 [4102 kB] +#7 3.593 Get:6 http://deb.debian.org/debian-security bullseye-security/main amd64 perl amd64 5.32.1-4+deb11u5 [294 kB] +#7 3.608 Get:7 http://deb.debian.org/debian-security bullseye-security/main amd64 libssl1.1 amd64 1.1.1w-0+deb11u8 [1566 kB] +#7 3.693 Get:8 http://deb.debian.org/debian bullseye/main amd64 bzip2 amd64 1.0.8-4 [49.3 kB] +#7 3.694 Get:9 http://deb.debian.org/debian bullseye/main amd64 xz-utils amd64 5.2.5-2.1~deb11u1 [220 kB] +#7 3.707 Get:10 http://deb.debian.org/debian bullseye/main amd64 binutils-common amd64 2.35.2-2 [2220 kB] +#7 3.833 Get:11 http://deb.debian.org/debian bullseye/main amd64 libbinutils amd64 2.35.2-2 [570 kB] +#7 3.865 Get:12 http://deb.debian.org/debian bullseye/main amd64 libctf-nobfd0 amd64 2.35.2-2 [110 kB] +#7 3.870 Get:13 http://deb.debian.org/debian bullseye/main amd64 libctf0 amd64 2.35.2-2 [53.2 kB] +#7 3.874 Get:14 http://deb.debian.org/debian bullseye/main amd64 binutils-x86-64-linux-gnu amd64 2.35.2-2 [1809 kB] +#7 3.984 Get:15 http://deb.debian.org/debian bullseye/main amd64 binutils amd64 2.35.2-2 [61.2 kB] +#7 3.987 Get:16 http://deb.debian.org/debian-security bullseye-security/main amd64 libc-dev-bin amd64 2.31-13+deb11u14 [277 kB] +#7 4.008 Get:17 http://deb.debian.org/debian-security bullseye-security/main amd64 linux-libc-dev amd64 5.10.259-1 [1917 kB] +#7 4.110 Get:18 http://deb.debian.org/debian bullseye/main amd64 libcrypt-dev amd64 1:4.4.18-4 [104 kB] +#7 4.117 Get:19 http://deb.debian.org/debian bullseye/main amd64 libtirpc-dev amd64 1.3.1-1+deb11u1 [191 kB] +#7 4.126 Get:20 http://deb.debian.org/debian bullseye/main amd64 libnsl-dev amd64 1.3.0-2 [66.4 kB] +#7 4.136 Get:21 http://deb.debian.org/debian-security bullseye-security/main amd64 libc6-dev amd64 2.31-13+deb11u14 [2360 kB] +#7 4.268 Get:22 http://deb.debian.org/debian bullseye/main amd64 libisl23 amd64 0.23-1 [676 kB] +#7 4.309 Get:23 http://deb.debian.org/debian bullseye/main amd64 libmpfr6 amd64 4.1.0-3 [2012 kB] +#7 4.418 Get:24 http://deb.debian.org/debian bullseye/main amd64 libmpc3 amd64 1.2.0-1 [45.0 kB] +#7 4.420 Get:25 http://deb.debian.org/debian bullseye/main amd64 cpp-10 amd64 10.2.1-6 [8528 kB] +#7 4.927 Get:26 http://deb.debian.org/debian bullseye/main amd64 cpp amd64 4:10.2.1-1 [19.7 kB] +#7 4.933 Get:27 http://deb.debian.org/debian bullseye/main amd64 libcc1-0 amd64 10.2.1-6 [47.0 kB] +#7 4.934 Get:28 http://deb.debian.org/debian bullseye/main amd64 libgomp1 amd64 10.2.1-6 [99.9 kB] +#7 4.943 Get:29 http://deb.debian.org/debian bullseye/main amd64 libitm1 amd64 10.2.1-6 [25.8 kB] +#7 4.944 Get:30 http://deb.debian.org/debian bullseye/main amd64 libatomic1 amd64 10.2.1-6 [9008 B] +#7 4.949 Get:31 http://deb.debian.org/debian bullseye/main amd64 libasan6 amd64 10.2.1-6 [2065 kB] +#7 5.132 Get:32 http://deb.debian.org/debian bullseye/main amd64 liblsan0 amd64 10.2.1-6 [828 kB] +#7 5.190 Get:33 http://deb.debian.org/debian bullseye/main amd64 libtsan0 amd64 10.2.1-6 [2000 kB] +#7 5.304 Get:34 http://deb.debian.org/debian bullseye/main amd64 libubsan1 amd64 10.2.1-6 [777 kB] +#7 5.342 Get:35 http://deb.debian.org/debian bullseye/main amd64 libquadmath0 amd64 10.2.1-6 [145 kB] +#7 5.352 Get:36 http://deb.debian.org/debian bullseye/main amd64 libgcc-10-dev amd64 10.2.1-6 [2328 kB] +#7 5.486 Get:37 http://deb.debian.org/debian bullseye/main amd64 gcc-10 amd64 10.2.1-6 [17.0 MB] +#7 6.444 Get:38 http://deb.debian.org/debian bullseye/main amd64 gcc amd64 4:10.2.1-1 [5192 B] +#7 6.444 Get:39 http://deb.debian.org/debian bullseye/main amd64 libstdc++-10-dev amd64 10.2.1-6 [1741 kB] +#7 6.545 Get:40 http://deb.debian.org/debian bullseye/main amd64 g++-10 amd64 10.2.1-6 [9380 kB] +#7 7.073 Get:41 http://deb.debian.org/debian bullseye/main amd64 g++ amd64 4:10.2.1-1 [1644 B] +#7 7.074 Get:42 http://deb.debian.org/debian bullseye/main amd64 make amd64 4.3-4.1 [396 kB] +#7 7.094 Get:43 http://deb.debian.org/debian bullseye/main amd64 libdpkg-perl all 1.20.13 [1552 kB] +#7 7.177 Get:44 http://deb.debian.org/debian bullseye/main amd64 patch amd64 2.7.6-7 [128 kB] +#7 7.184 Get:45 http://deb.debian.org/debian bullseye/main amd64 dpkg-dev all 1.20.13 [2314 kB] +#7 7.315 Get:46 http://deb.debian.org/debian bullseye/main amd64 build-essential amd64 12.9 [7704 B] +#7 7.315 Get:47 http://deb.debian.org/debian bullseye/main amd64 libbrotli1 amd64 1.0.9-2+b2 [279 kB] +#7 7.331 Get:48 http://deb.debian.org/debian bullseye/main amd64 libsasl2-modules-db amd64 2.1.27+dfsg-2.1+deb11u1 [69.1 kB] +#7 7.335 Get:49 http://deb.debian.org/debian bullseye/main amd64 libsasl2-2 amd64 2.1.27+dfsg-2.1+deb11u1 [106 kB] +#7 7.341 Get:50 http://deb.debian.org/debian bullseye/main amd64 libldap-2.4-2 amd64 2.4.57+dfsg-3+deb11u1 [232 kB] +#7 7.355 Get:51 http://deb.debian.org/debian-security bullseye-security/main amd64 libnghttp2-14 amd64 1.43.0-1+deb11u3 [77.6 kB] +#7 7.359 Get:52 http://deb.debian.org/debian bullseye/main amd64 libpsl5 amd64 0.21.0-1.2 [57.3 kB] +#7 7.363 Get:53 http://deb.debian.org/debian bullseye/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2+b2 [60.8 kB] +#7 7.368 Get:54 http://deb.debian.org/debian bullseye/main amd64 libssh2-1 amd64 1.9.0-2+deb11u1 [156 kB] +#7 7.374 Get:55 http://deb.debian.org/debian-security bullseye-security/main amd64 libcurl4 amd64 7.74.0-1.3+deb11u16 [347 kB] +#7 7.400 Get:56 http://deb.debian.org/debian-security bullseye-security/main amd64 libcurl4-openssl-dev amd64 7.74.0-1.3+deb11u16 [438 kB] +#7 7.424 Get:57 http://deb.debian.org/debian-security bullseye-security/main amd64 libssl-dev amd64 1.1.1w-0+deb11u8 [1822 kB] +#7 7.686 debconf: delaying package configuration, since apt-utils is not installed +#7 7.710 Fetched 79.1 MB in 5s (17.2 MB/s) +#7 7.753 (Reading database ... +(Reading database ... 5% +(Reading database ... 10% +(Reading database ... 15% +(Reading database ... 20% +(Reading database ... 25% +(Reading database ... 30% +(Reading database ... 35% +(Reading database ... 40% +(Reading database ... 45% +(Reading database ... 50% +(Reading database ... 55% +(Reading database ... 60% +(Reading database ... 65% +(Reading database ... 70% +(Reading database ... 75% +(Reading database ... 80% +(Reading database ... 85% +(Reading database ... 90% +(Reading database ... 95% +(Reading database ... 100% +(Reading database ... 7034 files and directories currently installed.) +#7 7.767 Preparing to unpack .../perl-base_5.32.1-4+deb11u5_amd64.deb ... +#7 7.803 Unpacking perl-base (5.32.1-4+deb11u5) over (5.32.1-4+deb11u4) ... +#7 9.931 Setting up perl-base (5.32.1-4+deb11u5) ... +#7 9.989 Selecting previously unselected package perl-modules-5.32. +#7 9.989 (Reading database ... +(Reading database ... 5% +(Reading database ... 10% +(Reading database ... 15% +(Reading database ... 20% +(Reading database ... 25% +(Reading database ... 30% +(Reading database ... 35% +(Reading database ... 40% +(Reading database ... 45% +(Reading database ... 50% +(Reading database ... 55% +(Reading database ... 60% +(Reading database ... 65% +(Reading database ... 70% +(Reading database ... 75% +(Reading database ... 80% +(Reading database ... 85% +(Reading database ... 90% +(Reading database ... 95% +(Reading database ... 100% +(Reading database ... 7035 files and directories currently installed.) +#7 10.000 Preparing to unpack .../perl-modules-5.32_5.32.1-4+deb11u5_all.deb ... +#7 10.00 Unpacking perl-modules-5.32 (5.32.1-4+deb11u5) ... +#7 10.28 Preparing to unpack .../libc6_2.31-13+deb11u14_amd64.deb ... +#7 10.36 debconf: unable to initialize frontend: Dialog +#7 10.36 debconf: (TERM is not set, so the dialog frontend is not usable.) +#7 10.36 debconf: falling back to frontend: Readline +#7 10.43 debconf: unable to initialize frontend: Dialog +#7 10.43 debconf: (TERM is not set, so the dialog frontend is not usable.) +#7 10.43 debconf: falling back to frontend: Readline +#7 10.48 Unpacking libc6:amd64 (2.31-13+deb11u14) over (2.31-13+deb11u13) ... +#7 11.43 Setting up libc6:amd64 (2.31-13+deb11u14) ... +#7 11.50 debconf: unable to initialize frontend: Dialog +#7 11.50 debconf: (TERM is not set, so the dialog frontend is not usable.) +#7 11.50 debconf: falling back to frontend: Readline +#7 12.61 Selecting previously unselected package libgdbm-compat4:amd64. +#7 12.61 (Reading database ... +(Reading database ... 5% +(Reading database ... 10% +(Reading database ... 15% +(Reading database ... 20% +(Reading database ... 25% +(Reading database ... 30% +(Reading database ... 35% +(Reading database ... 40% +(Reading database ... 45% +(Reading database ... 50% +(Reading database ... 55% +(Reading database ... 60% +(Reading database ... 65% +(Reading database ... 70% +(Reading database ... 75% +(Reading database ... 80% +(Reading database ... 85% +(Reading database ... 90% +(Reading database ... 95% +(Reading database ... 100% +(Reading database ... 8429 files and directories currently installed.) +#7 12.62 Preparing to unpack .../libgdbm-compat4_1.19-2_amd64.deb ... +#7 12.62 Unpacking libgdbm-compat4:amd64 (1.19-2) ... +#7 12.68 Selecting previously unselected package libperl5.32:amd64. +#7 12.69 Preparing to unpack .../libperl5.32_5.32.1-4+deb11u5_amd64.deb ... +#7 12.69 Unpacking libperl5.32:amd64 (5.32.1-4+deb11u5) ... +#7 13.02 Selecting previously unselected package perl. +#7 13.03 Preparing to unpack .../perl_5.32.1-4+deb11u5_amd64.deb ... +#7 13.04 Unpacking perl (5.32.1-4+deb11u5) ... +#7 13.13 Preparing to unpack .../libssl1.1_1.1.1w-0+deb11u8_amd64.deb ... +#7 13.15 Unpacking libssl1.1:amd64 (1.1.1w-0+deb11u8) over (1.1.1w-0+deb11u3) ... +#7 13.31 Setting up libssl1.1:amd64 (1.1.1w-0+deb11u8) ... +#7 13.38 debconf: unable to initialize frontend: Dialog +#7 13.38 debconf: (TERM is not set, so the dialog frontend is not usable.) +#7 13.38 debconf: falling back to frontend: Readline +#7 13.44 Selecting previously unselected package bzip2. +#7 13.44 (Reading database ... +(Reading database ... 5% +(Reading database ... 10% +(Reading database ... 15% +(Reading database ... 20% +(Reading database ... 25% +(Reading database ... 30% +(Reading database ... 35% +(Reading database ... 40% +(Reading database ... 45% +(Reading database ... 50% +(Reading database ... 55% +(Reading database ... 60% +(Reading database ... 65% +(Reading database ... 70% +(Reading database ... 75% +(Reading database ... 80% +(Reading database ... 85% +(Reading database ... 90% +(Reading database ... 95% +(Reading database ... 100% +(Reading database ... 9000 files and directories currently installed.) +#7 13.45 Preparing to unpack .../00-bzip2_1.0.8-4_amd64.deb ... +#7 13.46 Unpacking bzip2 (1.0.8-4) ... +#7 13.51 Selecting previously unselected package xz-utils. +#7 13.51 Preparing to unpack .../01-xz-utils_5.2.5-2.1~deb11u1_amd64.deb ... +#7 13.52 Unpacking xz-utils (5.2.5-2.1~deb11u1) ... +#7 13.57 Selecting previously unselected package binutils-common:amd64. +#7 13.57 Preparing to unpack .../02-binutils-common_2.35.2-2_amd64.deb ... +#7 13.57 Unpacking binutils-common:amd64 (2.35.2-2) ... +#7 13.75 Selecting previously unselected package libbinutils:amd64. +#7 13.75 Preparing to unpack .../03-libbinutils_2.35.2-2_amd64.deb ... +#7 13.75 Unpacking libbinutils:amd64 (2.35.2-2) ... +#7 13.82 Selecting previously unselected package libctf-nobfd0:amd64. +#7 13.83 Preparing to unpack .../04-libctf-nobfd0_2.35.2-2_amd64.deb ... +#7 13.83 Unpacking libctf-nobfd0:amd64 (2.35.2-2) ... +#7 13.88 Selecting previously unselected package libctf0:amd64. +#7 13.88 Preparing to unpack .../05-libctf0_2.35.2-2_amd64.deb ... +#7 13.89 Unpacking libctf0:amd64 (2.35.2-2) ... +#7 13.94 Selecting previously unselected package binutils-x86-64-linux-gnu. +#7 13.94 Preparing to unpack .../06-binutils-x86-64-linux-gnu_2.35.2-2_amd64.deb ... +#7 13.94 Unpacking binutils-x86-64-linux-gnu (2.35.2-2) ... +#7 14.10 Selecting previously unselected package binutils. +#7 14.11 Preparing to unpack .../07-binutils_2.35.2-2_amd64.deb ... +#7 14.11 Unpacking binutils (2.35.2-2) ... +#7 14.16 Selecting previously unselected package libc-dev-bin. +#7 14.16 Preparing to unpack .../08-libc-dev-bin_2.31-13+deb11u14_amd64.deb ... +#7 14.17 Unpacking libc-dev-bin (2.31-13+deb11u14) ... +#7 14.23 Selecting previously unselected package linux-libc-dev:amd64. +#7 14.23 Preparing to unpack .../09-linux-libc-dev_5.10.259-1_amd64.deb ... +#7 14.24 Unpacking linux-libc-dev:amd64 (5.10.259-1) ... +#7 14.36 Selecting previously unselected package libcrypt-dev:amd64. +#7 14.36 Preparing to unpack .../10-libcrypt-dev_1%3a4.4.18-4_amd64.deb ... +#7 14.37 Unpacking libcrypt-dev:amd64 (1:4.4.18-4) ... +#7 14.42 Selecting previously unselected package libtirpc-dev:amd64. +#7 14.42 Preparing to unpack .../11-libtirpc-dev_1.3.1-1+deb11u1_amd64.deb ... +#7 14.43 Unpacking libtirpc-dev:amd64 (1.3.1-1+deb11u1) ... +#7 14.48 Selecting previously unselected package libnsl-dev:amd64. +#7 14.48 Preparing to unpack .../12-libnsl-dev_1.3.0-2_amd64.deb ... +#7 14.49 Unpacking libnsl-dev:amd64 (1.3.0-2) ... +#7 14.53 Selecting previously unselected package libc6-dev:amd64. +#7 14.53 Preparing to unpack .../13-libc6-dev_2.31-13+deb11u14_amd64.deb ... +#7 14.54 Unpacking libc6-dev:amd64 (2.31-13+deb11u14) ... +#7 14.76 Selecting previously unselected package libisl23:amd64. +#7 14.76 Preparing to unpack .../14-libisl23_0.23-1_amd64.deb ... +#7 14.76 Unpacking libisl23:amd64 (0.23-1) ... +#7 14.85 Selecting previously unselected package libmpfr6:amd64. +#7 14.85 Preparing to unpack .../15-libmpfr6_4.1.0-3_amd64.deb ... +#7 14.85 Unpacking libmpfr6:amd64 (4.1.0-3) ... +#7 14.97 Selecting previously unselected package libmpc3:amd64. +#7 14.97 Preparing to unpack .../16-libmpc3_1.2.0-1_amd64.deb ... +#7 14.97 Unpacking libmpc3:amd64 (1.2.0-1) ... +#7 15.03 Selecting previously unselected package cpp-10. +#7 15.03 Preparing to unpack .../17-cpp-10_10.2.1-6_amd64.deb ... +#7 15.04 Unpacking cpp-10 (10.2.1-6) ... +#7 15.58 Selecting previously unselected package cpp. +#7 15.58 Preparing to unpack .../18-cpp_4%3a10.2.1-1_amd64.deb ... +#7 15.58 Unpacking cpp (4:10.2.1-1) ... +#7 15.63 Selecting previously unselected package libcc1-0:amd64. +#7 15.64 Preparing to unpack .../19-libcc1-0_10.2.1-6_amd64.deb ... +#7 15.64 Unpacking libcc1-0:amd64 (10.2.1-6) ... +#7 15.70 Selecting previously unselected package libgomp1:amd64. +#7 15.70 Preparing to unpack .../20-libgomp1_10.2.1-6_amd64.deb ... +#7 15.71 Unpacking libgomp1:amd64 (10.2.1-6) ... +#7 15.77 Selecting previously unselected package libitm1:amd64. +#7 15.77 Preparing to unpack .../21-libitm1_10.2.1-6_amd64.deb ... +#7 15.78 Unpacking libitm1:amd64 (10.2.1-6) ... +#7 15.84 Selecting previously unselected package libatomic1:amd64. +#7 15.84 Preparing to unpack .../22-libatomic1_10.2.1-6_amd64.deb ... +#7 15.85 Unpacking libatomic1:amd64 (10.2.1-6) ... +#7 15.90 Selecting previously unselected package libasan6:amd64. +#7 15.91 Preparing to unpack .../23-libasan6_10.2.1-6_amd64.deb ... +#7 15.91 Unpacking libasan6:amd64 (10.2.1-6) ... +#7 16.09 Selecting previously unselected package liblsan0:amd64. +#7 16.09 Preparing to unpack .../24-liblsan0_10.2.1-6_amd64.deb ... +#7 16.10 Unpacking liblsan0:amd64 (10.2.1-6) ... +#7 16.19 Selecting previously unselected package libtsan0:amd64. +#7 16.20 Preparing to unpack .../25-libtsan0_10.2.1-6_amd64.deb ... +#7 16.20 Unpacking libtsan0:amd64 (10.2.1-6) ... +#7 16.38 Selecting previously unselected package libubsan1:amd64. +#7 16.38 Preparing to unpack .../26-libubsan1_10.2.1-6_amd64.deb ... +#7 16.38 Unpacking libubsan1:amd64 (10.2.1-6) ... +#7 16.50 Selecting previously unselected package libquadmath0:amd64. +#7 16.51 Preparing to unpack .../27-libquadmath0_10.2.1-6_amd64.deb ... +#7 16.51 Unpacking libquadmath0:amd64 (10.2.1-6) ... +#7 16.56 Selecting previously unselected package libgcc-10-dev:amd64. +#7 16.57 Preparing to unpack .../28-libgcc-10-dev_10.2.1-6_amd64.deb ... +#7 16.57 Unpacking libgcc-10-dev:amd64 (10.2.1-6) ... +#7 16.73 Selecting previously unselected package gcc-10. +#7 16.74 Preparing to unpack .../29-gcc-10_10.2.1-6_amd64.deb ... +#7 16.74 Unpacking gcc-10 (10.2.1-6) ... +#7 17.72 Selecting previously unselected package gcc. +#7 17.72 Preparing to unpack .../30-gcc_4%3a10.2.1-1_amd64.deb ... +#7 17.73 Unpacking gcc (4:10.2.1-1) ... +#7 17.76 Selecting previously unselected package libstdc++-10-dev:amd64. +#7 17.77 Preparing to unpack .../31-libstdc++-10-dev_10.2.1-6_amd64.deb ... +#7 17.77 Unpacking libstdc++-10-dev:amd64 (10.2.1-6) ... +#7 18.10 Selecting previously unselected package g++-10. +#7 18.10 Preparing to unpack .../32-g++-10_10.2.1-6_amd64.deb ... +#7 18.10 Unpacking g++-10 (10.2.1-6) ... +#7 18.67 Selecting previously unselected package g++. +#7 18.68 Preparing to unpack .../33-g++_4%3a10.2.1-1_amd64.deb ... +#7 18.68 Unpacking g++ (4:10.2.1-1) ... +#7 18.71 Selecting previously unselected package make. +#7 18.71 Preparing to unpack .../34-make_4.3-4.1_amd64.deb ... +#7 18.71 Unpacking make (4.3-4.1) ... +#7 18.77 Selecting previously unselected package libdpkg-perl. +#7 18.77 Preparing to unpack .../35-libdpkg-perl_1.20.13_all.deb ... +#7 18.78 Unpacking libdpkg-perl (1.20.13) ... +#7 18.85 Selecting previously unselected package patch. +#7 18.85 Preparing to unpack .../36-patch_2.7.6-7_amd64.deb ... +#7 18.86 Unpacking patch (2.7.6-7) ... +#7 18.91 Selecting previously unselected package dpkg-dev. +#7 18.91 Preparing to unpack .../37-dpkg-dev_1.20.13_all.deb ... +#7 18.92 Unpacking dpkg-dev (1.20.13) ... +#7 19.04 Selecting previously unselected package build-essential. +#7 19.04 Preparing to unpack .../38-build-essential_12.9_amd64.deb ... +#7 19.05 Unpacking build-essential (12.9) ... +#7 19.09 Selecting previously unselected package libbrotli1:amd64. +#7 19.10 Preparing to unpack .../39-libbrotli1_1.0.9-2+b2_amd64.deb ... +#7 19.10 Unpacking libbrotli1:amd64 (1.0.9-2+b2) ... +#7 19.16 Selecting previously unselected package libsasl2-modules-db:amd64. +#7 19.16 Preparing to unpack .../40-libsasl2-modules-db_2.1.27+dfsg-2.1+deb11u1_amd64.deb ... +#7 19.16 Unpacking libsasl2-modules-db:amd64 (2.1.27+dfsg-2.1+deb11u1) ... +#7 19.20 Selecting previously unselected package libsasl2-2:amd64. +#7 19.20 Preparing to unpack .../41-libsasl2-2_2.1.27+dfsg-2.1+deb11u1_amd64.deb ... +#7 19.20 Unpacking libsasl2-2:amd64 (2.1.27+dfsg-2.1+deb11u1) ... +#7 19.25 Selecting previously unselected package libldap-2.4-2:amd64. +#7 19.25 Preparing to unpack .../42-libldap-2.4-2_2.4.57+dfsg-3+deb11u1_amd64.deb ... +#7 19.26 Unpacking libldap-2.4-2:amd64 (2.4.57+dfsg-3+deb11u1) ... +#7 19.31 Selecting previously unselected package libnghttp2-14:amd64. +#7 19.32 Preparing to unpack .../43-libnghttp2-14_1.43.0-1+deb11u3_amd64.deb ... +#7 19.32 Unpacking libnghttp2-14:amd64 (1.43.0-1+deb11u3) ... +#7 19.38 Selecting previously unselected package libpsl5:amd64. +#7 19.38 Preparing to unpack .../44-libpsl5_0.21.0-1.2_amd64.deb ... +#7 19.39 Unpacking libpsl5:amd64 (0.21.0-1.2) ... +#7 19.45 Selecting previously unselected package librtmp1:amd64. +#7 19.45 Preparing to unpack .../45-librtmp1_2.4+20151223.gitfa8646d.1-2+b2_amd64.deb ... +#7 19.46 Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... +#7 19.52 Selecting previously unselected package libssh2-1:amd64. +#7 19.52 Preparing to unpack .../46-libssh2-1_1.9.0-2+deb11u1_amd64.deb ... +#7 19.52 Unpacking libssh2-1:amd64 (1.9.0-2+deb11u1) ... +#7 19.58 Selecting previously unselected package libcurl4:amd64. +#7 19.59 Preparing to unpack .../47-libcurl4_7.74.0-1.3+deb11u16_amd64.deb ... +#7 19.59 Unpacking libcurl4:amd64 (7.74.0-1.3+deb11u16) ... +#7 19.66 Selecting previously unselected package libcurl4-openssl-dev:amd64. +#7 19.67 Preparing to unpack .../48-libcurl4-openssl-dev_7.74.0-1.3+deb11u16_amd64.deb ... +#7 19.67 Unpacking libcurl4-openssl-dev:amd64 (7.74.0-1.3+deb11u16) ... +#7 19.73 Selecting previously unselected package libssl-dev:amd64. +#7 19.73 Preparing to unpack .../49-libssl-dev_1.1.1w-0+deb11u8_amd64.deb ... +#7 19.74 Unpacking libssl-dev:amd64 (1.1.1w-0+deb11u8) ... +#7 19.89 Setting up libpsl5:amd64 (0.21.0-1.2) ... +#7 19.90 Setting up perl-modules-5.32 (5.32.1-4+deb11u5) ... +#7 19.91 Setting up libbrotli1:amd64 (1.0.9-2+b2) ... +#7 19.92 Setting up binutils-common:amd64 (2.35.2-2) ... +#7 19.93 Setting up libnghttp2-14:amd64 (1.43.0-1+deb11u3) ... +#7 19.95 Setting up linux-libc-dev:amd64 (5.10.259-1) ... +#7 19.96 Setting up libctf-nobfd0:amd64 (2.35.2-2) ... +#7 19.98 Setting up libgomp1:amd64 (10.2.1-6) ... +#7 19.99 Setting up bzip2 (1.0.8-4) ... +#7 20.01 Setting up libasan6:amd64 (10.2.1-6) ... +#7 20.02 Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-2.1+deb11u1) ... +#7 20.03 Setting up libtirpc-dev:amd64 (1.3.1-1+deb11u1) ... +#7 20.04 Setting up make (4.3-4.1) ... +#7 20.06 Setting up libmpfr6:amd64 (4.1.0-3) ... +#7 20.07 Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2+b2) ... +#7 20.09 Setting up xz-utils (5.2.5-2.1~deb11u1) ... +#7 20.11 update-alternatives: using /usr/bin/xz to provide /usr/bin/lzma (lzma) in auto mode +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzma.1.gz because associated file /usr/share/man/man1/xz.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/unlzma.1.gz because associated file /usr/share/man/man1/unxz.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzcat.1.gz because associated file /usr/share/man/man1/xzcat.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzmore.1.gz because associated file /usr/share/man/man1/xzmore.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzless.1.gz because associated file /usr/share/man/man1/xzless.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzdiff.1.gz because associated file /usr/share/man/man1/xzdiff.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzcmp.1.gz because associated file /usr/share/man/man1/xzcmp.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzgrep.1.gz because associated file /usr/share/man/man1/xzgrep.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzegrep.1.gz because associated file /usr/share/man/man1/xzegrep.1.gz (of link group lzma) doesn't exist +#7 20.11 update-alternatives: warning: skip creation of /usr/share/man/man1/lzfgrep.1.gz because associated file /usr/share/man/man1/xzfgrep.1.gz (of link group lzma) doesn't exist +#7 20.12 Setting up libquadmath0:amd64 (10.2.1-6) ... +#7 20.14 Setting up libssl-dev:amd64 (1.1.1w-0+deb11u8) ... +#7 20.15 Setting up libmpc3:amd64 (1.2.0-1) ... +#7 20.16 Setting up libatomic1:amd64 (10.2.1-6) ... +#7 20.18 Setting up patch (2.7.6-7) ... +#7 20.19 Setting up libgdbm-compat4:amd64 (1.19-2) ... +#7 20.20 Setting up libperl5.32:amd64 (5.32.1-4+deb11u5) ... +#7 20.22 Setting up libsasl2-2:amd64 (2.1.27+dfsg-2.1+deb11u1) ... +#7 20.23 Setting up libubsan1:amd64 (10.2.1-6) ... +#7 20.24 Setting up libnsl-dev:amd64 (1.3.0-2) ... +#7 20.25 Setting up libcrypt-dev:amd64 (1:4.4.18-4) ... +#7 20.26 Setting up libssh2-1:amd64 (1.9.0-2+deb11u1) ... +#7 20.27 Setting up libbinutils:amd64 (2.35.2-2) ... +#7 20.28 Setting up libisl23:amd64 (0.23-1) ... +#7 20.29 Setting up libc-dev-bin (2.31-13+deb11u14) ... +#7 20.30 Setting up libcc1-0:amd64 (10.2.1-6) ... +#7 20.31 Setting up liblsan0:amd64 (10.2.1-6) ... +#7 20.33 Setting up cpp-10 (10.2.1-6) ... +#7 20.34 Setting up libitm1:amd64 (10.2.1-6) ... +#7 20.36 Setting up libtsan0:amd64 (10.2.1-6) ... +#7 20.37 Setting up libctf0:amd64 (2.35.2-2) ... +#7 20.38 Setting up libgcc-10-dev:amd64 (10.2.1-6) ... +#7 20.40 Setting up libldap-2.4-2:amd64 (2.4.57+dfsg-3+deb11u1) ... +#7 20.41 Setting up perl (5.32.1-4+deb11u5) ... +#7 20.44 Setting up libdpkg-perl (1.20.13) ... +#7 20.46 Setting up cpp (4:10.2.1-1) ... +#7 20.48 Setting up libcurl4:amd64 (7.74.0-1.3+deb11u16) ... +#7 20.49 Setting up libc6-dev:amd64 (2.31-13+deb11u14) ... +#7 20.51 Setting up binutils-x86-64-linux-gnu (2.35.2-2) ... +#7 20.52 Setting up libstdc++-10-dev:amd64 (10.2.1-6) ... +#7 20.53 Setting up binutils (2.35.2-2) ... +#7 20.55 Setting up dpkg-dev (1.20.13) ... +#7 20.57 Setting up libcurl4-openssl-dev:amd64 (7.74.0-1.3+deb11u16) ... +#7 20.59 Setting up gcc-10 (10.2.1-6) ... +#7 20.60 Setting up g++-10 (10.2.1-6) ... +#7 20.61 Setting up gcc (4:10.2.1-1) ... +#7 20.64 Setting up g++ (4:10.2.1-1) ... +#7 20.68 update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode +#7 20.69 Setting up build-essential (12.9) ... +#7 20.70 Processing triggers for libc-bin (2.31-13+deb11u13) ... +#7 DONE 21.0s + +#8 [3/7] WORKDIR /app +#8 DONE 0.1s + +#9 [4/7] COPY . . +#9 DONE 0.2s + +#10 [5/7] RUN sed -i 's/ip host : "IP address" = localhost/ip host : "IP address" = 0.0.0.0/' src/pyload/core/config/default.cfg +#10 DONE 0.2s + +#11 [6/7] RUN pip install --no-cache-dir --upgrade pip setuptools wheel && pip install --no-cache-dir . && pip install --no-cache-dir pyload-ng-webui +#11 1.532 Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (23.0.1) +#11 1.668 Collecting pip +#11 1.732 Downloading pip-26.1.2-py3-none-any.whl (1.8 MB) +#11 1.823 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 21.2 MB/s eta 0:00:00 +#11 1.837 Requirement already satisfied: setuptools in /usr/local/lib/python3.10/site-packages (65.5.1) +#11 2.017 Collecting setuptools +#11 2.025 Downloading setuptools-83.0.0-py3-none-any.whl (1.0 MB) +#11 2.068 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 25.8 MB/s eta 0:00:00 +#11 2.079 Requirement already satisfied: wheel in /usr/local/lib/python3.10/site-packages (0.45.1) +#11 2.117 Collecting wheel +#11 2.125 Downloading wheel-0.47.0-py3-none-any.whl (32 kB) +#11 2.155 Collecting packaging>=24.0 +#11 2.163 Downloading packaging-26.2-py3-none-any.whl (100 kB) +#11 2.168 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.2/100.2 kB 47.5 MB/s eta 0:00:00 +#11 2.289 Installing collected packages: setuptools, pip, packaging, wheel +#11 2.289 Attempting uninstall: setuptools +#11 2.290 Found existing installation: setuptools 65.5.1 +#11 2.322 Uninstalling setuptools-65.5.1: +#11 2.419 Successfully uninstalled setuptools-65.5.1 +#11 2.823 Attempting uninstall: pip +#11 2.824 Found existing installation: pip 23.0.1 +#11 2.961 Uninstalling pip-23.0.1: +#11 3.067 Successfully uninstalled pip-23.0.1 +#11 3.659 Attempting uninstall: wheel +#11 3.659 Found existing installation: wheel 0.45.1 +#11 3.666 Uninstalling wheel-0.45.1: +#11 3.684 Successfully uninstalled wheel-0.45.1 +#11 3.711 Successfully installed packaging-26.2 pip-26.1.2 setuptools-83.0.0 wheel-0.47.0 +#11 3.711 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv +#11 4.256 Processing ./. +#11 4.257 Installing build dependencies: started +#11 4.945 Installing build dependencies: finished with status 'done' +#11 4.945 Getting requirements to build wheel: started +#11 5.158 Getting requirements to build wheel: finished with status 'error' +#11 5.162 error: subprocess-exited-with-error +#11 5.162 +#11 5.162 × Getting requirements to build wheel did not run successfully. +#11 5.162 │ exit code: 1 +#11 5.162 ╰─> [17 lines of output] +#11 5.162 Traceback (most recent call last): +#11 5.162 File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in +#11 5.162 main() +#11 5.162 File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main +#11 5.162 json_out["return_val"] = hook(**hook_input["kwargs"]) +#11 5.162 File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel +#11 5.162 return hook(config_settings) +#11 5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel +#11 5.162 return self._get_build_requires(config_settings, requirements=[]) +#11 5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires +#11 5.162 self.run_setup() +#11 5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 520, in run_setup +#11 5.162 super().run_setup(setup_script=setup_script) +#11 5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup +#11 5.162 exec(code, locals()) +#11 5.162 File "", line 14, in +#11 5.162 ModuleNotFoundError: No module named 'pkg_resources' +#11 5.162 [end of output] +#11 5.162 +#11 5.162 note: This error originates from a subprocess, and is likely not a problem with pip. +#11 5.162 ERROR: Failed to build 'file:///app' when getting requirements to build wheel +#11 ERROR: process "/bin/sh -c pip install --no-cache-dir --upgrade pip setuptools wheel && pip install --no-cache-dir . && pip install --no-cache-dir pyload-ng-webui" did not complete successfully: exit code: 1 +------ + > [6/7] RUN pip install --no-cache-dir --upgrade pip setuptools wheel && pip install --no-cache-dir . && pip install --no-cache-dir pyload-ng-webui: +5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 520, in run_setup +5.162 super().run_setup(setup_script=setup_script) +5.162 File "/tmp/pip-build-env-kuxjv01b/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 317, in run_setup +5.162 exec(code, locals()) +5.162 File "", line 14, in +5.162 ModuleNotFoundError: No module named 'pkg_resources' +5.162 [end of output] +5.162 +5.162 note: This error originates from a subprocess, and is likely not a problem with pip. +5.162 ERROR: Failed to build 'file:///app' when getting requirements to build wheel +------ +Dockerfile:25 + +-------------------- + + 24 | # companion web-UI package from PyPI (pyload.webui namespace package) + + 25 | >>> RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \ + + 26 | >>> pip install --no-cache-dir . && \ + + 27 | >>> pip install --no-cache-dir pyload-ng-webui + + 28 | + +-------------------- + +failed to solve: process "/bin/sh -c pip install --no-cache-dir --upgrade pip setuptools wheel && pip install --no-cache-dir . && pip install --no-cache-dir pyload-ng-webui" did not complete successfully: exit code: 1 + + + +View build details: docker-desktop://dashboard/build/default/default/o10ww58hjnzbj749ypyzsxwnr + diff --git a/pyproject.toml b/pyproject.toml index 44f643c..bda4d0c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,6 +12,7 @@ dependencies = [ "httpx>=0.28.1", "pydantic>=2.13.4", "python-dotenv>=1.2.2", + "pyyaml>=6.0.2", "typer>=0.26.8", ] @@ -28,6 +29,7 @@ dev = [ "pytest>=9.1.1", "pytest-cov>=7.0.0", "ruff>=0.15.20", + "types-pyyaml>=6.0.12", ] [tool.ruff] @@ -36,6 +38,9 @@ line-length = 100 [tool.ruff.lint] select = ["E", "F", "W", "I", "UP", "B", "SIM", "RUF"] +[tool.ruff.lint.per-file-ignores] +"__init__.py" = ["F401"] + [tool.mypy] strict = true files = ["src", "tests"] diff --git a/src/code_audit/cli.py b/src/code_audit/cli.py index 06b1bd0..f0a1d28 100644 --- a/src/code_audit/cli.py +++ b/src/code_audit/cli.py @@ -11,9 +11,19 @@ from code_audit.agent import MODEL, complete_trace from code_audit.config import Config, ConfigError +from code_audit.dockerize import ( + BUILD_AND_START_TIMEOUT_SECONDS, + REPAIR_ATTEMPTS_DEFAULT, + DockerizeStatus, + TraceInputError, + build_trace_result_from_commit_sha, + dockerize_advisory, + load_trace_result, + missing_prerequisite, +) from code_audit.github_client import GitHubClient, GitHubClientError from code_audit.instrumentation import MetricsCollector -from code_audit.models import TraceResult +from code_audit.models import Advisory, TraceResult from code_audit.tracing import trace_advisory app = typer.Typer(no_args_is_help=True) @@ -107,6 +117,154 @@ def trace( typer.echo(trace_metrics.model_dump_json(indent=2)) +@app.command() +def dockerize( + ghsa_id: Annotated[str, typer.Argument(help="Advisory ID, e.g. GHSA-jfh8-c2jp-5v3q")], + trace: Annotated[ + bool, + typer.Option( + "--trace", + help="Run the full trace agent automatically to resolve commits, no prompt.", + ), + ] = False, + trace_result_path: Annotated[ + Path | None, + typer.Option( + "--trace-result", + help=( + "Load a previously saved trace result JSON file (from `trace`'s output), no prompt." + ), + ), + ] = None, + fixing_commit: Annotated[ + str | None, + typer.Option( + "--fixing-commit", + help="Use this commit SHA directly as the fixing commit, no prompt.", + ), + ] = None, + force: Annotated[ + bool, + typer.Option( + "--force", + help=( + "Generate and verify even if the repository already has a Dockerfile or " + "compose file. For exercising this pipeline end to end only; the existing " + "artifact is overwritten in the temporary checkout for this run, never in " + "the real repository. Still skips a repository the classifier identifies " + "as a library." + ), + ), + ] = False, + build_timeout: Annotated[ + float, + typer.Option( + "--build-timeout", + help=( + "Seconds allowed for the build-and-start phase. The default suits a typical " + "small application; a large monorepo (a compiled backend plus a large " + "frontend bundle, for example) can legitimately need much longer." + ), + ), + ] = BUILD_AND_START_TIMEOUT_SECONDS, + repair_attempts: Annotated[ + int, + typer.Option( + "--repair-attempts", + help=( + "Extra generate-and-verify tries after the first, feeding the previous " + "artifact and its classified failure back to the agent instead of reporting " + "failure after one attempt. Each attempt is a full generate-plus-verify " + "cycle, so cost and time scale with it." + ), + ), + ] = REPAIR_ATTEMPTS_DEFAULT, + debug: Annotated[ + bool, + typer.Option( + "--debug", help="Write a per-turn JSON Lines transcript of generation under debug/." + ), + ] = False, +) -> None: + """Generate and verify a Dockerfile/compose for the advisory's repository, if missing. + + This is a separate, explicitly opt-in pipeline: it clones and executes code + from the audited repository to build and run it in a sandboxed container, + which `trace` never does. It requires `git` and Docker on the machine + running this command. See README.md for what the sandboxing does and does + not protect against. + + dockerize is meant to run once an advisory's commits are already known, not + to re-investigate commit provenance itself: a deterministic-only pass can + miss cases the full trace agent resolves. With none of --trace, + --trace-result, or --fixing-commit given, it asks interactively whether to + trace now or to provide a commit directly. + """ + problem = missing_prerequisite() + if problem is not None: + typer.echo(f"Error: {problem}", err=True) + raise typer.Exit(1) + if build_timeout <= 0: + typer.echo("Error: --build-timeout must be a positive number of seconds.", err=True) + raise typer.Exit(1) + if repair_attempts < 0: + typer.echo("Error: --repair-attempts must not be negative.", err=True) + raise typer.Exit(1) + if sum([trace, trace_result_path is not None, fixing_commit is not None]) > 1: + typer.echo( + "Error: --trace, --trace-result, and --fixing-commit are mutually exclusive.", + err=True, + ) + raise typer.Exit(1) + + on_turn: Callable[[dict[str, Any]], None] | None = None + debug_file: TextIO | None = None + debug_path: Path | None = None + if debug: + debug_path, debug_file, on_turn = _debug_recorder(ghsa_id, "dockerize") + + try: + config = Config.from_env() + anthropic_client = anthropic.Anthropic(api_key=config.anthropic_api_key) + with GitHubClient(token=config.github_token) as client: + fetched_advisory = client.fetch_advisory(ghsa_id) + trace_result = _resolve_trace_result( + fetched_advisory, + client, + anthropic_client, + trace, + trace_result_path, + fixing_commit, + ) + outcome = dockerize_advisory( + fetched_advisory, + client, + anthropic_client, + force=force, + build_timeout_seconds=build_timeout, + trace_result=trace_result, + on_progress=_print_progress, + on_turn=on_turn, + repair_attempts=repair_attempts, + ) + except (ConfigError, GitHubClientError, anthropic.AnthropicError, TraceInputError) as exc: + typer.echo(f"Error: {exc}", err=True) + raise typer.Exit(1) from exc + finally: + if debug_file is not None: + debug_file.close() + + if outcome.forced_note is not None: + typer.echo(f"Note: {outcome.forced_note}") + typer.echo(outcome.message) + if outcome.output_path is not None: + typer.echo(f"Written to {outcome.output_path}") + if debug_path is not None: + typer.echo(f"Debug transcript written to {debug_path}", err=True) + if outcome.status != DockerizeStatus.SUCCEEDED: + raise typer.Exit(1) + + @app.command("trace-many") def trace_many( ghsa_ids: Annotated[ @@ -268,6 +426,54 @@ class _BatchOutcome(NamedTuple): error: str | None +def _print_progress(message: str) -> None: + typer.echo(message, err=True) + + +def _resolve_trace_result( + advisory: Advisory, + client: GitHubClient, + anthropic_client: anthropic.Anthropic, + trace: bool, + trace_result_path: Path | None, + fixing_commit: str | None, +) -> TraceResult: + """Resolve the TraceResult dockerize's fixing-commit fallback should use. + + Exactly one of the three non-interactive options short-circuits the + prompt below; the CLI already checked they are mutually exclusive. + """ + if trace: + return _run_full_trace(advisory, client, anthropic_client) + if trace_result_path is not None: + return load_trace_result(trace_result_path) + if fixing_commit is not None: + return build_trace_result_from_commit_sha(advisory, fixing_commit, client) + + # dockerize is meant to run once an advisory's commits are already known, + # not to re-investigate commit provenance itself, so this asks rather + # than silently falling back to a deterministic-only pass that can miss + # cases the full trace agent would have resolved. + if typer.confirm("Trace this advisory now to resolve its commits?"): + return _run_full_trace(advisory, client, anthropic_client) + answer = typer.prompt( + "Enter the fixing commit SHA, or a path to a saved trace result JSON file" + ) + if Path(answer).is_file(): + return load_trace_result(Path(answer)) + return build_trace_result_from_commit_sha(advisory, answer, client) + + +def _run_full_trace( + advisory: Advisory, client: GitHubClient, anthropic_client: anthropic.Anthropic +) -> TraceResult: + typer.echo("Running trace to resolve commits...", err=True) + deterministic_result = trace_advisory(advisory, client) + result = complete_trace(advisory, deterministic_result, client, anthropic_client) + typer.echo("Trace finished.", err=True) + return result + + def _collect_ids(ghsa_ids: list[str] | None, from_file: Path | None) -> list[str]: ids = list(ghsa_ids or []) if from_file is not None: diff --git a/src/code_audit/dockerize/__init__.py b/src/code_audit/dockerize/__init__.py new file mode 100644 index 0000000..919dc56 --- /dev/null +++ b/src/code_audit/dockerize/__init__.py @@ -0,0 +1,384 @@ +import re +import shutil +import tempfile +from collections.abc import Callable +from dataclasses import dataclass +from enum import StrEnum +from pathlib import Path +from typing import Any + +import anthropic +import yaml + +from code_audit.dockerize.checkout import ( + CheckoutError, + TargetCommit, + TargetCommitError, + TraceInputError, + build_trace_result_from_commit_sha, + clone_repository_at_commit, + load_trace_result, + resolve_target_commit, +) +from code_audit.dockerize.classification import ( + ClassificationResult, + _advisory_package_name, + _classification_report, + _has_build_backend_only_pyproject, + _has_manage_py, + _has_npm_start_script, + _has_python_main_serving_block, + _has_setup_py_without_entrypoint, + classify_application, + resolve_classification, +) +from code_audit.dockerize.generation import ( + GeneratedArtifacts, + ProgressReporter, + _report, + generate_docker_artifacts, + write_generated_artifacts, +) +from code_audit.dockerize.repair import RepairContext +from code_audit.dockerize.verification import ( + BUILD_AND_START_TIMEOUT_SECONDS, + ISOLATED_NETWORK_NAME, + VerificationResult, + _minimal_subprocess_env, + _sandbox_compose, + _unique_service_name, + verify_docker_artifacts, +) +from code_audit.github_client import GitHubClient +from code_audit.models import Advisory, TraceResult + +# This is the only part of the project that clones a real repository and +# executes code from it, which is why it lives in its own package and is +# only reachable through the explicit dockerize CLI command, never through +# trace. The package is split by pipeline stage: checkout (what commit to +# clone), classification (application or library), generation (the agent +# that writes the Dockerfile), verification (build and run it, sandboxed), +# and repair (feeding a classified failure back for another attempt). This +# module holds the orchestration that ties those stages together. + +# Explicit re-exports (mypy strict mode requires this for names imported +# from submodules rather than defined here) so the rest of the project can +# keep importing from code_audit.dockerize directly, the same as when this +# was a single flat module. +__all__ = [ + "BUILD_AND_START_TIMEOUT_SECONDS", + "ISOLATED_NETWORK_NAME", + "REPAIR_ATTEMPTS_DEFAULT", + "CheckoutError", + "ClassificationResult", + "DockerizeOutcome", + "DockerizeStatus", + "ExistingArtifacts", + "GeneratedArtifacts", + "ProgressReporter", + "RepairContext", + "TargetCommit", + "TargetCommitError", + "TraceInputError", + "VerificationResult", + "_advisory_package_name", + "_classification_report", + "_compose_targets_the_app", + "_has_build_backend_only_pyproject", + "_has_manage_py", + "_has_npm_start_script", + "_has_python_main_serving_block", + "_has_setup_py_without_entrypoint", + "_minimal_subprocess_env", + "_report", + "_sandbox_compose", + "_unique_service_name", + "application_artifacts_already_exist", + "build_trace_result_from_commit_sha", + "classify_application", + "clone_repository_at_commit", + "dockerize_advisory", + "find_existing_docker_artifacts", + "generate_docker_artifacts", + "load_trace_result", + "missing_prerequisite", + "resolve_classification", + "resolve_target_commit", + "verify_docker_artifacts", + "write_generated_artifacts", +] + +# Extra generate-and-verify tries after the first, feeding the specific +# failure back to the agent instead of reporting failure after one attempt. +REPAIR_ATTEMPTS_DEFAULT = 2 + + +def missing_prerequisite() -> str | None: + """Return a description of a missing prerequisite, or None when ready to run.""" + if shutil.which("git") is None: + return "git is required for the dockerize command but was not found on PATH." + if shutil.which("docker") is None: + return "Docker is required for the dockerize command but was not found on PATH." + return None + + +# Missing-artifact detection + +_DOCKERFILE_PATTERN = re.compile(r"^Dockerfile(\..+)?$") +_COMPOSE_PATTERN = re.compile(r"^(docker-)?compose(\..+)?\.ya?ml$", re.IGNORECASE) + + +@dataclass +class ExistingArtifacts: + dockerfiles: list[Path] + compose_files: list[Path] + docker_dirs: list[Path] + devcontainer_dirs: list[Path] + + +def find_existing_docker_artifacts(root: Path) -> ExistingArtifacts: + """Search the whole checkout, not just the root, for existing container artifacts.""" + paths = [path for path in root.rglob("*") if ".git" not in path.parts] + return ExistingArtifacts( + dockerfiles=sorted(p for p in paths if p.is_file() and _DOCKERFILE_PATTERN.match(p.name)), + compose_files=sorted(p for p in paths if p.is_file() and _COMPOSE_PATTERN.match(p.name)), + docker_dirs=sorted(p for p in paths if p.is_dir() and p.name == "docker"), + devcontainer_dirs=sorted(p for p in paths if p.is_dir() and p.name == ".devcontainer"), + ) + + +def _compose_targets_the_app(compose_path: Path, repository_name: str) -> bool: + """A compose file counts only if some service builds or names the app itself. + + A compose file that defines only infrastructure, such as a database or a + cache, with no service building from the repository, is not an + application artifact even though it is a real compose file. + """ + try: + data = yaml.safe_load(compose_path.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, yaml.YAMLError): + return False + if not isinstance(data, dict): + return False + services = data.get("services") + if not isinstance(services, dict): + return False + for service in services.values(): + if not isinstance(service, dict): + continue + if "build" in service: + return True + image = service.get("image") + if isinstance(image, str) and repository_name.lower() in image.lower(): + return True + return False + + +def application_artifacts_already_exist(root: Path, repository: str) -> tuple[bool, str]: + """Decide whether a genuine application Dockerfile or compose already exists.""" + artifacts = find_existing_docker_artifacts(root) + if artifacts.dockerfiles: + path = artifacts.dockerfiles[0].relative_to(root) + return True, f"A Dockerfile already exists at {path}." + if artifacts.docker_dirs: + path = artifacts.docker_dirs[0].relative_to(root) + return True, f"A docker/ directory already exists at {path}." + if artifacts.devcontainer_dirs: + path = artifacts.devcontainer_dirs[0].relative_to(root) + return True, f"A .devcontainer/ directory already exists at {path}." + repository_name = repository.split("/")[-1] + for compose_file in artifacts.compose_files: + if _compose_targets_the_app(compose_file, repository_name): + return True, f"{compose_file.relative_to(root)} already builds the application." + return False, "No application Dockerfile or compose file was found." + + +# Orchestration + + +class DockerizeStatus(StrEnum): + TARGET_COMMIT_UNRESOLVED = "target_commit_unresolved" + NOT_A_WEB_APPLICATION = "not_a_web_application" + ALREADY_CONTAINERIZED = "already_containerized" + GENERATION_FAILED = "generation_failed" + VERIFICATION_FAILED = "verification_failed" + SUCCEEDED = "succeeded" + + +@dataclass +class DockerizeOutcome: + status: DockerizeStatus + message: str + output_path: str | None = None + forced_note: str | None = None + + +def _copy_generated_artifacts(root: Path, output_dir: Path) -> Path: + output_dir.mkdir(parents=True, exist_ok=True) + shutil.copy2(root / "Dockerfile", output_dir / "Dockerfile") + compose_path = root / "compose.yml" + if compose_path.is_file(): + shutil.copy2(compose_path, output_dir / "compose.yml") + return output_dir + + +def dockerize_advisory( + advisory: Advisory, + github_client: GitHubClient, + anthropic_client: anthropic.Anthropic, + force: bool = False, + build_timeout_seconds: float = BUILD_AND_START_TIMEOUT_SECONDS, + trace_result: TraceResult | None = None, + on_progress: ProgressReporter | None = None, + on_turn: Callable[[dict[str, Any]], None] | None = None, + repair_attempts: int = REPAIR_ATTEMPTS_DEFAULT, +) -> DockerizeOutcome: + """Generate and verify a Dockerfile/compose for the advisory's repository, if needed. + + Every early exit (unresolved commit, not a web application, artifacts + already present, generation or verification failure) is reported with a + reason rather than silently skipped, since a missing explanation would be + indistinguishable from a bug when this runs unattended. + + force skips only the already-present check, so this pipeline can be + exercised against a repository that already has a Dockerfile, purely for + testing it. It never skips the application-vs-library classification: a + repository the classifier identifies as a library is still skipped + regardless of force. build_timeout_seconds overrides the default + build-and-start timeout, for a repository that legitimately needs longer + to build than a typical small application. trace_result, when given, is + used for the fixing-commit fallback in resolve_target_commit instead of + a fresh deterministic-only trace. on_progress, when given, receives one + short line as each step starts or finishes, since a run can take minutes + with otherwise no other output. on_turn, when given, receives one debug + record per generation turn, plus one more if the Tier 2 classification + tie-breaker runs. repair_attempts bounds how many extra generate-and- + verify cycles run after a verification failure, each fed the previous + artifact plus the classified failure and log tail, instead of reporting + failure after a single attempt. + """ + _report(on_progress, "Resolving target commit...") + try: + target = resolve_target_commit(advisory, github_client, trace_result) + except TargetCommitError as exc: + return DockerizeOutcome(DockerizeStatus.TARGET_COMMIT_UNRESOLVED, str(exc)) + _report(on_progress, f"Resolved target commit: {target.description}.") + + with tempfile.TemporaryDirectory(prefix="code-audit-dockerize-") as tmp_dir: + checkout_root = Path(tmp_dir) / "checkout" + _report(on_progress, f"Cloning {target.repository} at {target.fetch_ref}...") + try: + clone_repository_at_commit(target.repository, target.fetch_ref, checkout_root) + except CheckoutError as exc: + return DockerizeOutcome(DockerizeStatus.TARGET_COMMIT_UNRESOLVED, str(exc)) + _report(on_progress, "Clone finished.") + + package_name = _advisory_package_name(advisory) + _report(on_progress, "Classifying the repository as an application or a library...") + classification = resolve_classification( + checkout_root, package_name, anthropic_client, on_progress, on_turn + ) + _report( + on_progress, + "Classified as an application." + if classification.is_application + else "Classified as a library.", + ) + if not classification.is_application: + return DockerizeOutcome( + DockerizeStatus.NOT_A_WEB_APPLICATION, _classification_report(classification) + ) + + _report(on_progress, "Checking for existing Docker artifacts...") + already_present, reason = application_artifacts_already_exist( + checkout_root, target.repository + ) + forced_note = None + if already_present: + # reason ends up in the returned outcome either way (as the + # message below, or folded into forced_note), so it is not also + # sent through on_progress here, which would print it twice. + if not force: + return DockerizeOutcome(DockerizeStatus.ALREADY_CONTAINERIZED, reason) + forced_note = ( + f"{reason} Continuing anyway because --force was used: the existing artifact " + "is being overwritten in this run's temporary checkout only, never in the real " + "repository, since nothing here is pushed upstream." + ) + else: + _report(on_progress, reason) + + _report(on_progress, "Generating a Dockerfile...") + artifacts = generate_docker_artifacts(checkout_root, anthropic_client, on_progress, on_turn) + if artifacts is None: + return DockerizeOutcome( + DockerizeStatus.GENERATION_FAILED, + "The generation agent did not produce a usable Dockerfile.", + forced_note=forced_note, + ) + _report(on_progress, "Generation finished.") + write_generated_artifacts(checkout_root, artifacts) + + verification = verify_docker_artifacts( + checkout_root, artifacts, build_timeout_seconds, on_progress + ) + repairs_used = 0 + while not verification.succeeded and repairs_used < repair_attempts: + repairs_used += 1 + _report( + on_progress, + f"Repair attempt {repairs_used}/{repair_attempts}: patching based on " + f"{verification.failure_classification}...", + ) + repaired = generate_docker_artifacts( + checkout_root, + anthropic_client, + on_progress, + on_turn, + repair_context=RepairContext(artifacts, verification), + ) + if repaired is None: + # The last real verification failure is more concrete + # evidence than a repair attempt that produced nothing, so + # it is what gets reported below rather than this stall. + _report(on_progress, "Repair attempt did not produce a new artifact.") + break + artifacts = repaired + write_generated_artifacts(checkout_root, artifacts) + verification = verify_docker_artifacts( + checkout_root, artifacts, build_timeout_seconds, on_progress + ) + + if not verification.succeeded: + # The terminal-facing log_tail is short by design, but the full + # build output is what actually shows a real failure's cause (a + # truncated tail once hid the actual pnpm install error behind a + # generic "ELIFECYCLE" summary), so it is always written out in + # full, next to the artifact that failed, for inspection. + output_dir = _copy_generated_artifacts(checkout_root, Path("output") / advisory.ghsa_id) + attempts_made = repairs_used + 1 + plural = "s" if attempts_made != 1 else "" + message = ( + f"The generated artifact did not verify after {attempts_made} attempt{plural} " + f"(last failure: {verification.failure_classification}):\n{verification.log_tail}" + ) + if verification.full_log: + (output_dir / "build.log").write_text(verification.full_log, encoding="utf-8") + message += f"\n\nFull build log written to {output_dir / 'build.log'}." + return DockerizeOutcome( + DockerizeStatus.VERIFICATION_FAILED, + message, + str(output_dir), + forced_note=forced_note, + ) + + output_dir = _copy_generated_artifacts(checkout_root, Path("output") / advisory.ghsa_id) + suffix = " and compose.yml" if artifacts.compose_yaml else "" + repair_note = f" after {repairs_used} repair attempt(s)" if repairs_used else "" + return DockerizeOutcome( + DockerizeStatus.SUCCEEDED, + f"Generated a Dockerfile{suffix} that builds and runs the application " + f"({target.description}){repair_note}.", + str(output_dir), + forced_note=forced_note, + ) diff --git a/src/code_audit/dockerize/checkout.py b/src/code_audit/dockerize/checkout.py new file mode 100644 index 0000000..5575eeb --- /dev/null +++ b/src/code_audit/dockerize/checkout.py @@ -0,0 +1,228 @@ +import re +import subprocess +from dataclasses import dataclass +from pathlib import Path + +from code_audit.github_client import GitHubClient, GitHubClientError +from code_audit.models import Advisory, Commit, Tag, TraceResult +from code_audit.references import advisory_repository +from code_audit.tracing import trace_advisory + +GIT_TIMEOUT_SECONDS = 60.0 + + +class CheckoutError(Exception): + """Raised when the repository cannot be cloned at the resolved commit.""" + + +def clone_repository_at_commit(repository: str, ref: str, destination: Path) -> None: + """Shallow clone a GitHub repository at exactly one commit or tag. + + GitHub allows fetching a specific reachable commit or tag directly, so + only that one commit's tree needs to be fetched rather than the whole + repository history. + """ + destination.mkdir(parents=True, exist_ok=True) + url = f"https://github.com/{repository}.git" + _run_git(["init", "-q", "."], destination) + _run_git(["remote", "add", "origin", url], destination) + _run_git(["fetch", "--depth", "1", "origin", ref], destination) + _run_git(["checkout", "-q", "FETCH_HEAD"], destination) + + +def _run_git(args: list[str], cwd: Path) -> None: + try: + result = subprocess.run( + ["git", *args], cwd=cwd, capture_output=True, text=True, timeout=GIT_TIMEOUT_SECONDS + ) + except subprocess.TimeoutExpired as exc: + raise CheckoutError(f"git {' '.join(args)} timed out.") from exc + except FileNotFoundError as exc: + raise CheckoutError("git is not installed or not on PATH.") from exc + if result.returncode != 0: + raise CheckoutError(f"git {' '.join(args)} failed: {result.stderr.strip()}") + + +# Target commit resolution +# +# A GHSA advisory records affected version ranges, not commits, so the last +# affected version's git tag is the best proxy for "the vulnerable state of +# the application" (the reproducible target this feature is meant to build). +# When no matching tag can be resolved, the parent of the fixing commit is +# the fallback. Guessing a commit is never acceptable: an unresolved case +# raises with an explanation instead of picking something. + + +class TargetCommitError(Exception): + """Raised when no commit can be resolved to dockerize against.""" + + +@dataclass +class TargetCommit: + repository: str + sha: str + fetch_ref: str + description: str + + +_INCLUSIVE_UPPER_BOUND = re.compile(r"<=\s*([^\s,]+)") +_EXACT_VERSION = re.compile(r"^=\s*([^\s,]+)$") + + +def resolve_target_commit( + advisory: Advisory, client: GitHubClient, trace_result: TraceResult | None = None +) -> TargetCommit: + """Resolve the commit to dockerize against. + + Prefers the git tag for the advisory's last affected version; that + preference does not depend on trace_result at all. Falls back to the + parent of the fixing commit when no such tag can be resolved, sourced + from trace_result when one is given (a full trace agent's result, a + loaded file, or a bare commit SHA looked up by the caller) instead of + running a deterministic-only trace internally. + """ + repository = advisory_repository(advisory.source_code_location) + if repository is None: + raise TargetCommitError("The advisory does not reference a resolvable GitHub repository.") + + last_affected_version = _last_affected_version(advisory) + if last_affected_version is not None: + try: + tags = client.list_tags(repository) + except GitHubClientError as exc: + raise TargetCommitError(f"Could not list tags for {repository!r}: {exc}") from exc + tag = _find_matching_tag(tags, last_affected_version) + if tag is not None: + return TargetCommit( + repository, tag.sha, tag.name, f"tag {tag.name!r} (last affected version)" + ) + tag_lookup_reason = f"no tag matches the last affected version {last_affected_version!r}" + else: + # No tag lookup was attempted at all here, as opposed to one running + # and finding nothing, since the version range gave no literal + # version to search for. + tag_lookup_reason = ( + "the advisory's version range does not name a specific last affected version" + ) + + parent = _fixing_commit_parent(advisory, repository, client, trace_result) + if parent is not None: + parent_sha, fixing_sha = parent + return TargetCommit( + repository, parent_sha, parent_sha, f"parent of fixing commit {fixing_sha!r}" + ) + + raise TargetCommitError( + f"Could not resolve a target commit: {tag_lookup_reason}, and no fixing commit " + "with a known parent could be traced from the advisory." + ) + + +def _last_affected_version(advisory: Advisory) -> str | None: + if not advisory.vulnerabilities: + return None + version_range = advisory.vulnerabilities[0].vulnerable_version_range + if version_range is None: + return None + # An inclusive upper bound ('<= 2.14.1') or an exact match ('= 1.2.3') + # names the last affected version directly. An exclusive bound + # ('< 2.15.0') does not, since the version immediately before it cannot + # be derived without assuming a versioning scheme, so it is left + # unresolved rather than guessed. + match = _INCLUSIVE_UPPER_BOUND.search(version_range) + if match: + return match.group(1) + match = _EXACT_VERSION.fullmatch(version_range.strip()) + return match.group(1) if match else None + + +def _normalize_version(text: str) -> str: + text = text.strip().lower() + if len(text) > 1 and text[0] == "v" and text[1].isdigit(): + text = text[1:] + return text + + +def _find_matching_tag(tags: list[Tag], version: str) -> Tag | None: + target = _normalize_version(version) + for tag in tags: + normalized = _normalize_version(tag.name) + # Covers plain tags ('2.14.1', 'v2.14.1'), package-prefixed tags + # ('log4j-core-2.14.1', 'left-pad@2.14.1'), and path-like release + # tags ('rel/2.14.1'), without assuming one specific convention. + if normalized == target or normalized.endswith((f"@{target}", f"-{target}", f"/{target}")): + return tag + return None + + +def _fixing_commit_parent( + advisory: Advisory, + repository: str, + client: GitHubClient, + trace_result: TraceResult | None, +) -> tuple[str, str] | None: + result = trace_result if trace_result is not None else trace_advisory(advisory, client) + fixing_commit = result.fixing_commit + if fixing_commit is None: + # The advisory may reference only the fixing pull request, not the + # commit directly (this is common: GHSA-jfh8-c2jp-5v3q does exactly + # this). The merged pull request's merge commit is the fixing commit + # in that case, one more deterministic API call away. + fixing_commit = _fetch_merge_commit(result, repository, client) + if fixing_commit is None or not fixing_commit.parents: + return None + return fixing_commit.parents[0].sha, fixing_commit.sha + + +def _fetch_merge_commit( + result: TraceResult, repository: str, client: GitHubClient +) -> Commit | None: + pull_request = result.fixing_pull_request + if pull_request is None or not pull_request.merged or pull_request.merge_commit_sha is None: + return None + try: + return client.fetch_commit(repository, pull_request.merge_commit_sha) + except GitHubClientError: + return None + + +# Trace result input +# +# dockerize is meant to run once an advisory's commits are already known, not +# to re-investigate commit provenance itself. These build the TraceResult +# that resolve_target_commit's fallback path consults, from whichever source +# the caller already has: a bare SHA, or a trace result file saved earlier. + + +class TraceInputError(Exception): + """Raised when a fixing commit or trace result cannot be loaded from user input.""" + + +def build_trace_result_from_commit_sha( + advisory: Advisory, sha: str, client: GitHubClient +) -> TraceResult: + """Build a TraceResult from just a fixing commit SHA, given directly by the caller. + + The commit is refetched through the GitHub API for its parents, for + consistency with a TraceResult loaded from a file. + """ + repository = advisory_repository(advisory.source_code_location) + if repository is None: + raise TraceInputError("The advisory does not reference a resolvable GitHub repository.") + try: + commit = client.fetch_commit(repository, sha) + except GitHubClientError as exc: + raise TraceInputError(f"Could not fetch commit {sha!r} from {repository!r}: {exc}") from exc + return TraceResult(ghsa_id=advisory.ghsa_id, fixing_commit=commit) + + +def load_trace_result(path: Path) -> TraceResult: + """Load a trace result from a file, in the same JSON shape `trace` prints to stdout.""" + try: + data = path.read_text(encoding="utf-8") + except OSError as exc: + raise TraceInputError(f"Could not read trace result file {str(path)!r}: {exc}") from exc + try: + return TraceResult.model_validate_json(data) + except ValueError as exc: + raise TraceInputError(f"{str(path)!r} is not a valid trace result: {exc}") from exc diff --git a/src/code_audit/dockerize/classification.py b/src/code_audit/dockerize/classification.py new file mode 100644 index 0000000..5456070 --- /dev/null +++ b/src/code_audit/dockerize/classification.py @@ -0,0 +1,332 @@ +import json +import re +import tomllib +from collections.abc import Callable +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import anthropic +from anthropic.types import Message +from pydantic import BaseModel, ConfigDict, Field + +from code_audit.dockerize.generation import GENERATION_MODEL, ProgressReporter, _list_files, _report +from code_audit.models import Advisory + +# Most GHSA advisories target libraries, not applications, so the interesting +# classification problem is not "web versus not web": it is "application +# versus library". Each signal below is a small, named, individually +# testable check rather than one large function, since this heuristic will +# likely need tuning as more repositories are seen. +# +# Two tiers: Tier 1 (classify_application) is these deterministic file +# checks. Tier 2 is a small LLM call for the cases Tier 1 leaves ambiguous, +# used instead of hand-adding a new deterministic rule every time another +# packaging convention turns up (PEP 621 project.scripts, setup.cfg +# entry_points, and likely more to come); resolve_classification is the +# entry point that runs both. + + +def _advisory_package_name(advisory: Advisory) -> str | None: + if not advisory.vulnerabilities: + return None + package = advisory.vulnerabilities[0].package + return package.name if package is not None else None + + +_GO_LISTEN_PATTERN = re.compile(r"\.Listen\(|ListenAndServe\(") +_JS_LISTEN_PATTERN = re.compile(r"\.listen\(") +_PYTHON_SERVE_CALL_PATTERN = re.compile(r"\.run\(|\.serve\(") +_DEPLOYMENT_MANIFEST_NAMES = ("app.yaml", "fly.toml", "render.yaml", "vercel.json") + + +def _read_text(path: Path) -> str: + try: + return path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + return "" + + +def _load_json_object(path: Path) -> dict[str, Any] | None: + if not path.is_file(): + return None + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, json.JSONDecodeError): + return None + return data if isinstance(data, dict) else None + + +def _load_toml_object(path: Path) -> dict[str, Any] | None: + if not path.is_file(): + return None + try: + return tomllib.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, tomllib.TOMLDecodeError): + return None + + +def _has_manage_py(root: Path, package_name: str | None) -> bool: + return (root / "manage.py").is_file() + + +def _has_go_listen_entrypoint(root: Path, package_name: str | None) -> bool: + return any(_GO_LISTEN_PATTERN.search(_read_text(path)) for path in root.rglob("main.go")) + + +def _has_node_listen_entrypoint(root: Path, package_name: str | None) -> bool: + candidates = [path for name in ("server.js", "index.js") for path in root.rglob(name)] + return any(_JS_LISTEN_PATTERN.search(_read_text(path)) for path in candidates) + + +def _has_npm_start_script(root: Path, package_name: str | None) -> bool: + package_json = _load_json_object(root / "package.json") + scripts = package_json.get("scripts") if package_json else None + return isinstance(scripts, dict) and "start" in scripts + + +def _has_python_main_serving_block(root: Path, package_name: str | None) -> bool: + for path in root.rglob("*.py"): + text = _read_text(path) + if "__main__" in text and _PYTHON_SERVE_CALL_PATTERN.search(text): + return True + return False + + +def _has_procfile(root: Path, package_name: str | None) -> bool: + return (root / "Procfile").is_file() + + +def _has_deployment_manifest(root: Path, package_name: str | None) -> bool: + return any((root / name).is_file() for name in _DEPLOYMENT_MANIFEST_NAMES) + + +def _has_build_backend_only_pyproject(root: Path, package_name: str | None) -> bool: + pyproject = _load_toml_object(root / "pyproject.toml") + if pyproject is None or "build-system" not in pyproject: + return False + has_scripts = bool(pyproject.get("project", {}).get("scripts")) + return not has_scripts and not _has_manage_py(root, package_name) + + +def _has_setup_py_without_entrypoint(root: Path, package_name: str | None) -> bool: + if not (root / "setup.py").is_file(): + return False + return not _has_manage_py(root, package_name) and not _has_procfile(root, package_name) + + +def _has_npm_package_without_start_script(root: Path, package_name: str | None) -> bool: + if not (root / "package.json").is_file(): + return False + return not _has_npm_start_script(root, package_name) + + +def _project_name(root: Path) -> str | None: + pyproject = _load_toml_object(root / "pyproject.toml") + if pyproject is not None: + name = pyproject.get("project", {}).get("name") + if isinstance(name, str): + return name + package_json = _load_json_object(root / "package.json") + if package_json is not None: + name = package_json.get("name") + if isinstance(name, str): + return name + return None + + +def _normalize_package_name(name: str) -> str: + return re.sub(r"[-_.]+", "-", name.strip().lower()) + + +def _repository_matches_advisory_package_name(root: Path, package_name: str | None) -> bool: + if package_name is None: + return False + project_name = _project_name(root) + if project_name is None: + return False + return _normalize_package_name(project_name) == _normalize_package_name(package_name) + + +_APPLICATION_CHECKS: list[tuple[str, Callable[[Path, str | None], bool]]] = [ + ("manage.py", _has_manage_py), + ("go_listen_entrypoint", _has_go_listen_entrypoint), + ("node_listen_entrypoint", _has_node_listen_entrypoint), + ("npm_start_script", _has_npm_start_script), + ("python_main_serving_block", _has_python_main_serving_block), + ("procfile", _has_procfile), + ("deployment_manifest", _has_deployment_manifest), +] + +_LIBRARY_CHECKS: list[tuple[str, Callable[[Path, str | None], bool]]] = [ + ("build_backend_only_pyproject", _has_build_backend_only_pyproject), + ("setup_py_without_entrypoint", _has_setup_py_without_entrypoint), + ("npm_package_without_start_script", _has_npm_package_without_start_script), + ("repository_matches_package_name", _repository_matches_advisory_package_name), +] + + +@dataclass +class ClassificationResult: + is_application: bool + application_signals: list[str] + library_signals: list[str] + tier2_reason: str | None = None + + +def classify_application(root: Path, package_name: str | None) -> ClassificationResult: + """Tier 1: score application-ness against library-ness using file checks alone. + + is_application here is only a simple majority (more application signals + than library signals); resolve_classification is what decides whether + that margin is clear enough to trust outright or whether Tier 2 should + settle it instead. + """ + application_signals = [name for name, check in _APPLICATION_CHECKS if check(root, package_name)] + library_signals = [name for name, check in _LIBRARY_CHECKS if check(root, package_name)] + is_application = len(application_signals) > len(library_signals) + return ClassificationResult(is_application, application_signals, library_signals) + + +# A margin below this is not "clearly" on one side, so Tier 2 settles it +# instead of trusting Tier 1's simple majority. +CLEAR_SIGNAL_MARGIN = 2 + +TIER2_MODEL = GENERATION_MODEL +TIER2_MAX_TOKENS = 2000 + +_TIER2_MANIFEST_FILENAMES = ("pyproject.toml", "setup.cfg", "setup.py", "package.json", "go.mod") +_TIER2_README_FILENAMES = ("README.md", "README.rst", "README.txt", "README") +_TIER2_MANIFEST_FILE_CHARS = 4000 +_TIER2_README_CHARS = 2000 + +TIER2_SYSTEM_PROMPT = """\ +You are deciding whether a repository is a runnable application (something \ +meant to be started as a service, for example a web server) or a library or \ +package (something meant to be imported or depended on by other code, not \ +run directly on its own). + +Deterministic file-based signals already looked at this repository and found \ +the evidence ambiguous, which is why you are being asked. Use the file tree, \ +any manifest file contents, and the README excerpt given to make the call. + +Give your best judgment even if you remain uncertain; do not refuse to \ +answer. Explain your reasoning in exactly one clear sentence.""" + + +class TieBreakerJudgment(BaseModel): + """The Tier 2 LLM's application-or-library judgment. + + Also the source of the JSON schema that constrains the model's response. + """ + + model_config = ConfigDict(extra="forbid") + + is_application: bool = Field( + description="True if the repository is a runnable application, false if it is a library." + ) + reason: str = Field(description="Exactly one sentence explaining the judgment.") + + +def _classification_digest(root: Path) -> str: + """Build a compact, root-level summary of the repository for Tier 2. + + Kept small since Tier 2 only needs to settle cases Tier 1's signals + already found ambiguous, not to read the whole repository the way + generation does. + """ + parts = [f"Files:\n{_list_files(root)}"] + for name in _TIER2_MANIFEST_FILENAMES: + path = root / name + if path.is_file(): + content = _read_text(path)[:_TIER2_MANIFEST_FILE_CHARS] + parts.append(f"{name}:\n{content}") + for name in _TIER2_README_FILENAMES: + path = root / name + if path.is_file(): + content = _read_text(path)[:_TIER2_README_CHARS] + parts.append(f"{name} (excerpt):\n{content}") + break + return "\n\n".join(parts) + + +def _build_tier2_debug_record(response: Message, final_response_text: str | None) -> dict[str, Any]: + thinking = next( + (block.thinking for block in response.content if block.type == "thinking"), None + ) + return { + "step": "classification_tiebreaker", + "thinking": thinking, + "stop_reason": response.stop_reason, + "final_response_text": final_response_text, + } + + +def _run_tier2_classification( + root: Path, + anthropic_client: anthropic.Anthropic, + on_turn: Callable[[dict[str, Any]], None] | None, +) -> tuple[bool, str] | None: + """Run the Tier 2 tie-breaker call, or return None if it stalls. + + None (falling back to Tier 1's own verdict) covers a refusal or any + other stop reason that leaves no usable text, the same way a stalled + generation falls back rather than crashing. + """ + response = anthropic_client.messages.create( + model=TIER2_MODEL, + max_tokens=TIER2_MAX_TOKENS, + system=TIER2_SYSTEM_PROMPT, + output_config={ + "format": {"type": "json_schema", "schema": TieBreakerJudgment.model_json_schema()} + }, + messages=[{"role": "user", "content": _classification_digest(root)}], + ) + final_text = next((block.text for block in response.content if block.type == "text"), None) + if on_turn is not None: + on_turn(_build_tier2_debug_record(response, final_text)) + if final_text is None: + return None + judgment = TieBreakerJudgment.model_validate_json(final_text) + return judgment.is_application, judgment.reason + + +def resolve_classification( + root: Path, + package_name: str | None, + anthropic_client: anthropic.Anthropic, + on_progress: ProgressReporter | None = None, + on_turn: Callable[[dict[str, Any]], None] | None = None, +) -> ClassificationResult: + """Classify a repository as an application or a library, in two tiers. + + Tier 1 is trusted outright once its signals clearly favor one side + (a margin of CLEAR_SIGNAL_MARGIN or more); otherwise a Tier 2 LLM call + settles it from a compact repository digest, and its verdict and + one-sentence reason are what is returned, not Tier 1's simple majority. + """ + tier1 = classify_application(root, package_name) + margin = len(tier1.application_signals) - len(tier1.library_signals) + if abs(margin) >= CLEAR_SIGNAL_MARGIN: + return tier1 + + _report(on_progress, "Tier 1 signals are ambiguous, asking the classification tie-breaker...") + tier2 = _run_tier2_classification(root, anthropic_client, on_turn) + if tier2 is None: + return tier1 + is_application, reason = tier2 + return ClassificationResult( + is_application, tier1.application_signals, tier1.library_signals, reason + ) + + +def _classification_report(classification: ClassificationResult) -> str: + app_signals = ", ".join(classification.application_signals) or "none" + lib_signals = ", ".join(classification.library_signals) or "none" + message = ( + "Skipping: the repository does not look clearly like a web application. " + f"Application signals found: {app_signals}. Library signals found: {lib_signals}." + ) + if classification.tier2_reason is not None: + message += f" Tie-breaker judgment: {classification.tier2_reason}" + return message diff --git a/src/code_audit/dockerize/generation.py b/src/code_audit/dockerize/generation.py new file mode 100644 index 0000000..44a2992 --- /dev/null +++ b/src/code_audit/dockerize/generation.py @@ -0,0 +1,301 @@ +import json +import re +from collections.abc import Callable +from pathlib import Path +from typing import Any, cast + +import anthropic +from anthropic.types import Message, MessageParam, ToolParam, ToolResultBlockParam, ToolUseBlock +from pydantic import BaseModel, ConfigDict, Field + +from code_audit.dockerize.repair import RepairContext, _build_repair_task + +# A single short line per step, so a run that can take minutes is not silent +# the whole time. The caller decides where these go (the CLI sends them to +# stderr); this module only decides when to report, not how. Lives here +# (rather than a dedicated module) because generation is already a shared +# dependency of classification and verification. +ProgressReporter = Callable[[str], None] + + +def _report(on_progress: ProgressReporter | None, message: str) -> None: + if on_progress is not None: + on_progress(message) + + +GENERATION_MODEL = "claude-opus-4-8" +GENERATION_MAX_TOKENS = 16000 +GENERATION_MAX_TURNS = 15 + +# The agent only ever sees the cloned checkout through these two tools: it +# cannot execute anything, only read files, since generation should be +# informed by what the repository actually contains rather than assumptions +# about the framework's usual conventions. + +MAX_GENERATION_FILE_CHARS = 20_000 +MAX_LISTED_FILES = 500 + +DOCKERIZE_TOOLS: list[ToolParam] = [ + { + "name": "list_files", + "description": ( + "List every file path in the cloned repository checkout, relative to its root." + ), + "input_schema": {"type": "object", "properties": {}, "required": []}, + }, + { + "name": "read_file", + "description": ( + "Read a text file from the cloned repository checkout by its path relative to the root." + ), + "input_schema": { + "type": "object", + "properties": {"path": {"type": "string"}}, + "required": ["path"], + }, + }, +] + +GENERATION_SYSTEM_PROMPT = """\ +You are generating a Dockerfile (and, if needed, a compose.yml) that builds \ +and runs a cloned application repository as a working web service. + +Use the provided tools to read the repository's files before writing \ +anything: find the language, framework, dependency manifest, entrypoint, and \ +any configuration the app needs to start (a required environment variable, a \ +database, a cache). Base the Dockerfile on what the files actually show, not \ +assumptions from the framework's usual conventions. + +Before writing the Dockerfile, state in a sentence or two what you found: the \ +runtime and its version, the package manager, the framework (if any), the \ +entrypoint file or command, and the port the app listens on. This is meant to \ +keep your reasoning grounded in the files you actually read, not a checklist \ +to pad out. + +Keep the setup as simple as the application allows. Produce a compose.yml \ +only when the application genuinely needs more than one container to run \ +(for example, it fails to start without a real database connection); a \ +single-container app needs only a Dockerfile. When a database or cache is \ +required, add it as a service in compose.yml with sensible defaults so the \ +app can connect without extra configuration from outside. + +The image must actually start the application server, listening on a TCP \ +port reachable from outside the container. Do not produce a Dockerfile whose \ +command runs tests, a shell, or anything other than starting the application. + +Report the exact port the application listens on inside the container. This \ +is the only thing verification will use to confirm the app is reachable, so \ +it must be correct.""" + +_GENERATION_TASK = ( + "Generate a Dockerfile (and compose.yml if needed) for the repository checked out at " + "the tool root. Start by calling list_files to see what is there, then read the " + "dependency manifest and entrypoint files before writing anything." +) + + +class DockerizeToolError(Exception): + """Raised when a generation tool call cannot be satisfied.""" + + +def _resolve_within_root(root: Path, relative_path: str) -> Path | None: + candidate = (root / relative_path).resolve() + if not candidate.is_relative_to(root.resolve()): + return None + return candidate + + +def _list_files(root: Path) -> str: + paths = sorted( + str(path.relative_to(root)) + for path in root.rglob("*") + if path.is_file() and ".git" not in path.parts + ) + truncated = paths[:MAX_LISTED_FILES] + return json.dumps({"files": truncated, "truncated": len(paths) > MAX_LISTED_FILES}) + + +def _read_file(root: Path, path: str) -> str: + resolved = _resolve_within_root(root, path) + if resolved is None or not resolved.is_file(): + raise DockerizeToolError(f"No such file: {path!r}") + try: + text = resolved.read_text(encoding="utf-8") + except UnicodeDecodeError as exc: + raise DockerizeToolError(f"{path!r} is not valid UTF-8 text.") from exc + if len(text) > MAX_GENERATION_FILE_CHARS: + text = text[:MAX_GENERATION_FILE_CHARS] + "\n[truncated]" + return json.dumps({"path": path, "content": text}) + + +_DOCKERIZE_HANDLERS: dict[str, Callable[[Path, dict[str, Any]], str]] = { + "list_files": lambda root, args: _list_files(root), + "read_file": lambda root, args: _read_file(root, args["path"]), +} + + +def run_dockerize_tool(root: Path, block: ToolUseBlock) -> ToolResultBlockParam: + """Execute one generation tool call and wrap the outcome as a tool result. + + Tool names and inputs are model generated and not guaranteed to match the + schemas, so unknown tools and malformed arguments are reported back to + the agent instead of crashing generation. + """ + handler = _DOCKERIZE_HANDLERS.get(block.name) + if handler is None: + return _tool_error(block.id, f"Unknown tool: {block.name!r}") + arguments = cast(dict[str, Any], block.input) + try: + content = handler(root, arguments) + except DockerizeToolError as exc: + return _tool_error(block.id, str(exc)) + except (KeyError, TypeError) as exc: + return _tool_error(block.id, f"Invalid input for tool {block.name!r}: {exc!r}") + return {"type": "tool_result", "tool_use_id": block.id, "content": content} + + +def _tool_error(tool_use_id: str, message: str) -> ToolResultBlockParam: + return {"type": "tool_result", "tool_use_id": tool_use_id, "content": message, "is_error": True} + + +class GeneratedArtifacts(BaseModel): + """The Dockerfile (and optional compose.yml) produced by the generation agent. + + Also the source of the JSON schema that constrains the agent's final + response, so parsing cannot drift from what the model is asked to produce. + """ + + model_config = ConfigDict(extra="forbid") + + dockerfile: str = Field(description="Full contents of the Dockerfile.") + compose_yaml: str | None = Field( + description="Full contents of compose.yml, or null if a single Dockerfile is enough." + ) + port: int = Field(description="The TCP port the application listens on inside the container.") + explanation: str = Field(description="A short explanation of the chosen setup.") + + +_FROM_INSTRUCTION_PATTERN = re.compile(r"^[ \t]*FROM\s+\S+", re.IGNORECASE | re.MULTILINE) + + +def _looks_like_a_dockerfile(text: str) -> bool: + """A minimal sanity check: non-empty and containing a FROM instruction. + + An empty or FROM-less dockerfile field passes GeneratedArtifacts + validation, since there is no constraint on the field's content, so this + is what actually catches it, with a specific reason, before Docker + itself would with a much less useful error. + """ + return _FROM_INSTRUCTION_PATTERN.search(text) is not None + + +_GENERATION_DEBUG_PREVIEW_CHARS = 300 + + +def _generation_debug_preview(content: str) -> str: + if len(content) <= _GENERATION_DEBUG_PREVIEW_CHARS: + return content + return content[:_GENERATION_DEBUG_PREVIEW_CHARS] + "...[truncated]" + + +def _build_generation_turn_record( + turn: int, + response: Message, + tool_results: list[ToolResultBlockParam], + final_response_text: str | None = None, +) -> dict[str, Any]: + thinking = next( + (block.thinking for block in response.content if block.type == "thinking"), None + ) + tool_use = [ + {"name": block.name, "input": block.input} + for block in response.content + if block.type == "tool_use" + ] + results = [ + { + "tool_use_id": result["tool_use_id"], + "content_preview": _generation_debug_preview(cast(str, result["content"])), + } + for result in tool_results + ] + return { + "turn": turn, + "thinking": thinking, + "tool_use": tool_use, + "tool_results": results, + "stop_reason": response.stop_reason, + "final_response_text": final_response_text, + } + + +def generate_docker_artifacts( + root: Path, + anthropic_client: anthropic.Anthropic, + on_progress: ProgressReporter | None = None, + on_turn: Callable[[dict[str, Any]], None] | None = None, + repair_context: RepairContext | None = None, +) -> GeneratedArtifacts | None: + """Run the generation agent's tool loop and return its artifacts, or None if it stalls. + + None is also returned when the terminal turn parses but the dockerfile + field is not usable (empty, or missing a FROM instruction): the same + outcome as a stall, since the caller reports GENERATION_FAILED either way. + + repair_context, when given, replaces the initial task with the previous + artifact and its classified verification failure, so a repair attempt + asks the agent to fix that specific failure rather than starting over. + """ + task = _build_repair_task(repair_context) if repair_context is not None else _GENERATION_TASK + messages: list[MessageParam] = [{"role": "user", "content": task}] + for turn in range(1, GENERATION_MAX_TURNS + 1): + _report(on_progress, f"Generation turn {turn}/{GENERATION_MAX_TURNS}...") + response = anthropic_client.messages.create( + model=GENERATION_MODEL, + max_tokens=GENERATION_MAX_TOKENS, + system=GENERATION_SYSTEM_PROMPT, + # display defaults to "omitted" on this model generation, which + # returns thinking blocks with empty text even though the model + # is reasoning (see agent.py's MODEL, fixed for the same reason). + # Without a separate thinking budget, reasoning consumed the + # visible max_tokens budget directly, which is what let a + # complex repository hit max_tokens with no tool call attempted. + thinking={"type": "adaptive", "display": "summarized"}, + tools=DOCKERIZE_TOOLS, + output_config={ + "format": {"type": "json_schema", "schema": GeneratedArtifacts.model_json_schema()} + }, + messages=messages, + ) + if response.stop_reason != "tool_use": + final_text = None + if response.stop_reason == "end_turn": + final_text = next( + (block.text for block in response.content if block.type == "text"), None + ) + if on_turn is not None: + on_turn(_build_generation_turn_record(turn, response, [], final_text)) + if final_text is None: + return None + artifacts = GeneratedArtifacts.model_validate_json(final_text) + if not _looks_like_a_dockerfile(artifacts.dockerfile): + return None + return artifacts + assistant_content = [ + block + for block in response.content + if not (block.type == "text" and not block.text.strip()) + ] + messages.append({"role": "assistant", "content": assistant_content}) + tool_use_blocks = [block for block in response.content if block.type == "tool_use"] + tool_results = [run_dockerize_tool(root, block) for block in tool_use_blocks] + if on_turn is not None: + on_turn(_build_generation_turn_record(turn, response, tool_results)) + messages.append({"role": "user", "content": tool_results}) + return None + + +def write_generated_artifacts(root: Path, artifacts: GeneratedArtifacts) -> None: + (root / "Dockerfile").write_text(artifacts.dockerfile, encoding="utf-8") + if artifacts.compose_yaml is not None: + (root / "compose.yml").write_text(artifacts.compose_yaml, encoding="utf-8") diff --git a/src/code_audit/dockerize/repair.py b/src/code_audit/dockerize/repair.py new file mode 100644 index 0000000..42149db --- /dev/null +++ b/src/code_audit/dockerize/repair.py @@ -0,0 +1,79 @@ +from dataclasses import dataclass +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + # Only needed for type hints below. A real import here would cycle with + # generation.py (which imports RepairContext and _build_repair_task from + # this module) and verification.py (which imports the failure labels + # below), so these two types stay as string forward references instead, + # the same trick the original single-file module used for the same + # reason (VerificationResult was defined later in that file). + from code_audit.dockerize.generation import GeneratedArtifacts + from code_audit.dockerize.verification import VerificationResult + +# Lightweight, deterministic labels for why verification failed, handed to +# the repair loop's generation call alongside the log tail so the agent's +# context states what kind of failure this is, rather than leaving it to +# infer that from a raw log alone. +FAILURE_INVALID_ARTIFACT = "the generated compose file could not be used" +FAILURE_IMAGE_PULL = "the build image failed to pull" +FAILURE_DEPENDENCY_INSTALL = "dependency resolution or install failed during the build" +FAILURE_CONTAINER_NEVER_STARTED = "the build succeeded but the container never started" +FAILURE_HEALTH_CHECK_TIMEOUT = "the container started but the health check timed out" + +_IMAGE_PULL_FAILURE_MARKERS = ( + "pull access denied", + "manifest unknown", + "manifest for", + "no matching manifest", + "failed to authorize", + "unauthorized:", + "toomanyrequests", + "no such host", + "failed to resolve source metadata", +) + + +def _classify_build_failure(log: str) -> str: + """Distinguish an image pull failure from everything else that can fail mid-build. + + Defaults to a dependency failure when nothing marks it as a pull + failure, since that is the more common real-world case (an npm/pip/go + install failing partway through a RUN instruction). + """ + lower = log.lower() + if any(marker in lower for marker in _IMAGE_PULL_FAILURE_MARKERS): + return FAILURE_IMAGE_PULL + return FAILURE_DEPENDENCY_INSTALL + + +@dataclass +class RepairContext: + """The previous attempt's artifact and its verification failure. + + Passed to generate_docker_artifacts on a repair attempt so the agent is + asked to fix a specific, classified failure rather than starting over + from nothing. + """ + + previous_artifacts: "GeneratedArtifacts" + verification: "VerificationResult" + + +def _build_repair_task(repair_context: RepairContext) -> str: + previous = repair_context.previous_artifacts + verification = repair_context.verification + compose_section = ( + f"\n\nPrevious compose.yml:\n{previous.compose_yaml}" if previous.compose_yaml else "" + ) + return ( + "The Dockerfile (and compose.yml, if any) generated for this repository failed " + "verification.\n\n" + f"Classified failure: {verification.failure_classification}\n\n" + f"Previous Dockerfile:\n{previous.dockerfile}" + f"{compose_section}\n\n" + f"Build/run log (tail):\n{verification.log_tail}\n\n" + "Fix the specific failure described above. Use list_files and read_file if you " + "need to look at anything else in the repository to reconsider your approach, " + "rather than starting over from nothing." + ) diff --git a/src/code_audit/dockerize/verification.py b/src/code_audit/dockerize/verification.py new file mode 100644 index 0000000..3a52c91 --- /dev/null +++ b/src/code_audit/dockerize/verification.py @@ -0,0 +1,324 @@ +import json +import os +import subprocess +import uuid +from dataclasses import dataclass +from pathlib import Path +from typing import Any, cast + +import yaml + +from code_audit.dockerize.generation import GeneratedArtifacts, ProgressReporter, _report +from code_audit.dockerize.repair import ( + FAILURE_CONTAINER_NEVER_STARTED, + FAILURE_HEALTH_CHECK_TIMEOUT, + FAILURE_INVALID_ARTIFACT, + _classify_build_failure, +) + +BUILD_AND_START_TIMEOUT_SECONDS = 300.0 +HEALTH_CHECK_TIMEOUT_SECONDS = 120.0 +COMPOSE_DOWN_TIMEOUT_SECONDS = 60.0 +LOG_TAIL_LINES = 500 + +# Verification always runs through a synthesized or sanitized compose file, +# even for a Dockerfile-only artifact, so build, start, sandboxing, and +# cleanup share one code path. +# +# The running services join one internal Docker network so nothing they run +# can reach the internet. That isolation is real (confirmed against a real +# container: an internal network gets no host port-publish rule at all, and +# the host cannot even reach a container's bridge IP directly on it), but it +# also means the host cannot poll the app's port itself. A small prober +# service joins the same internal network and runs the actual health check +# from inside it, over the container-to-container DNS name compose sets up, +# rather than using compose's own `--wait`/healthcheck machinery (a generated +# image has no guarantee of having curl or wget installed to run a +# healthcheck command inside the container). + +ISOLATED_NETWORK_NAME = "code_audit_verify_internal" +PROBER_IMAGE = "python:3.12-slim" + +_HEALTH_CHECK_SCRIPT = """ +import sys +import time +import urllib.error +import urllib.request + +target, timeout = sys.argv[1], float(sys.argv[2]) +deadline = time.monotonic() + timeout +while time.monotonic() < deadline: + try: + urllib.request.urlopen(target, timeout=5) + sys.exit(0) + except urllib.error.HTTPError: + # Any HTTP response, even an error status, proves the server started. + sys.exit(0) + except (urllib.error.URLError, TimeoutError, ConnectionError): + time.sleep(2) +sys.exit(1) +""" + +# The subprocess running docker/docker compose only gets these, not the full +# host environment: an allowlist of what the CLI itself needs to find the +# right daemon, not a denylist of the two secrets this project happens to +# use. A generated compose.yml is not trusted input, so any other secret on +# the host (cloud credentials, unrelated API keys) must not be reachable +# through it either. +_SUBPROCESS_ENV_ALLOWLIST = ( + "PATH", + "HOME", + "DOCKER_HOST", + "DOCKER_CONFIG", + "DOCKER_CONTEXT", + "DOCKER_CERT_PATH", + "DOCKER_TLS_VERIFY", +) + + +def _minimal_subprocess_env() -> dict[str, str]: + return {key: os.environ[key] for key in _SUBPROCESS_ENV_ALLOWLIST if key in os.environ} + + +class VerificationError(Exception): + """Raised when the generated artifact cannot even be prepared for verification.""" + + +@dataclass +class VerificationResult: + succeeded: bool + log_tail: str + full_log: str = "" + failure_classification: str = "" + + +_SANDBOX_LIMITS: dict[str, object] = { + "cap_drop": ["ALL"], + "mem_limit": "512m", + "cpus": 1.0, + "pids_limit": 256, +} + + +def _build_verification_compose(artifacts: GeneratedArtifacts) -> dict[str, Any]: + if artifacts.compose_yaml is None: + return {"services": {"app": {"build": "."}}} + try: + data = yaml.safe_load(artifacts.compose_yaml) + except yaml.YAMLError as exc: + raise VerificationError(f"Generated compose.yml is not valid YAML: {exc}") from exc + if not isinstance(data, dict) or not isinstance(data.get("services"), dict): + raise VerificationError("Generated compose.yml has no services.") + return cast(dict[str, Any], data) + + +def _find_app_service_name(services: dict[str, Any]) -> str: + for name, service in services.items(): + if isinstance(service, dict) and "build" in service: + return name + return next(iter(services)) + + +def _is_named_volume(entry: object) -> bool: + if isinstance(entry, dict): + return entry.get("type") == "volume" + if isinstance(entry, str): + source = entry.split(":", 1)[0] + return source != "" and not source.startswith((".", "/")) + return False + + +def _unique_service_name(services: dict[str, Any], base: str) -> str: + name = base + suffix = 0 + while name in services: + suffix += 1 + name = f"{base}{suffix}" + return name + + +def _sandbox_compose(data: dict[str, Any]) -> str: + """Strip dangerous settings, cap resources, and isolate the network for every service. + + Runs regardless of whether the compose came from the generation agent or + was synthesized here, since a generated artifact is not trusted input: + the repository being containerized is itself the subject of a security + advisory. Every service, including the newly added prober, joins one + internal network and gets the same resource limits; none of them keep a + host port mapping, since the internal network already makes one + impossible to use. + + Returns the name of the added prober service. + """ + services = cast(dict[str, Any], data["services"]) + prober_name = _unique_service_name(services, "code_audit_prober") + services[prober_name] = {"image": PROBER_IMAGE, "command": ["sleep", "infinity"]} + + for service in services.values(): + if not isinstance(service, dict): + continue + service.pop("network_mode", None) + service.pop("privileged", None) + service.pop("cap_add", None) + service.pop("ports", None) + service["volumes"] = [v for v in service.get("volumes", []) if _is_named_volume(v)] + service["networks"] = [ISOLATED_NETWORK_NAME] + service.update(_SANDBOX_LIMITS) + + data["networks"] = {ISOLATED_NETWORK_NAME: {"internal": True}} + return prober_name + + +def _write_temp_compose(data: dict[str, Any], directory: Path) -> Path: + path = directory / "_verification.compose.yml" + path.write_text(yaml.safe_dump(data), encoding="utf-8") + return path + + +def _decode(value: Any) -> str: + if value is None: + return "" + return value.decode() if isinstance(value, bytes) else cast(str, value) + + +def _tail(text: str) -> str: + return "\n".join(text.splitlines()[-LOG_TAIL_LINES:]) + + +def _run_subprocess( + args: list[str], cwd: Path, env: dict[str, str], timeout: float +) -> subprocess.CompletedProcess[str]: + try: + return subprocess.run( + args, cwd=cwd, env=env, capture_output=True, text=True, timeout=timeout + ) + except subprocess.TimeoutExpired as exc: + stderr = f"{_decode(exc.stderr)}\ntimed out after {timeout}s" + return subprocess.CompletedProcess(args, 124, _decode(exc.stdout), stderr) + + +def _wait_for_http_response_via_prober( + compose_command: list[str], + prober_service: str, + target_url: str, + timeout: float, + cwd: Path, + env: dict[str, str], +) -> bool: + result = _run_subprocess( + [ + *compose_command, + "exec", + "-T", + prober_service, + "python3", + "-c", + _HEALTH_CHECK_SCRIPT, + target_url, + str(timeout), + ], + cwd, + env, + timeout + 30.0, + ) + return result.returncode == 0 + + +def _app_service_state( + compose_command: list[str], app_service: str, cwd: Path, env: dict[str, str] +) -> str: + """Return the app service's container state (for example "running" or "exited"). + + Returns "unknown" if it cannot be determined, so callers do not overclaim + a crash they cannot actually confirm. + """ + result = _run_subprocess( + [*compose_command, "ps", "--format", "json", app_service], cwd, env, 15.0 + ) + if result.returncode != 0 or not result.stdout.strip(): + return "unknown" + try: + first_line = result.stdout.strip().splitlines()[0] + return str(json.loads(first_line).get("State", "unknown")) + except (json.JSONDecodeError, IndexError): + return "unknown" + + +def verify_docker_artifacts( + root: Path, + artifacts: GeneratedArtifacts, + build_timeout_seconds: float = BUILD_AND_START_TIMEOUT_SECONDS, + on_progress: ProgressReporter | None = None, +) -> VerificationResult: + """Build and run the generated artifact in a sandboxed, network-isolated compose project. + + Success means the application answered an HTTP request after starting. + Containers, images, and networks created for the attempt are always + removed afterward, whether verification succeeded or failed. + + build_timeout_seconds bounds the build-and-start phase; the default suits + a typical small application, but a large monorepo (a Go backend compiled + alongside a large frontend bundle, for example) can legitimately need + much longer, so callers may override it. + """ + try: + compose_data = _build_verification_compose(artifacts) + app_service = _find_app_service_name(compose_data["services"]) + prober_service = _sandbox_compose(compose_data) + except VerificationError as exc: + return VerificationResult(False, str(exc), failure_classification=FAILURE_INVALID_ARTIFACT) + + compose_path = _write_temp_compose(compose_data, root) + project = f"code-audit-verify-{uuid.uuid4().hex[:8]}" + env = _minimal_subprocess_env() + compose_command = ["docker", "compose", "-f", str(compose_path), "-p", project] + + try: + _report( + on_progress, + f"Building and starting containers (timeout {int(build_timeout_seconds)}s)...", + ) + up_result = _run_subprocess( + [*compose_command, "up", "--build", "-d"], root, env, build_timeout_seconds + ) + if up_result.returncode != 0: + full_log = up_result.stdout + up_result.stderr + return VerificationResult( + False, _tail(full_log), full_log, _classify_build_failure(full_log) + ) + _report(on_progress, "Build and start finished.") + + target_url = f"http://{app_service}:{artifacts.port}/" + _report( + on_progress, + f"Waiting for {target_url} to respond " + f"(timeout {int(HEALTH_CHECK_TIMEOUT_SECONDS)}s)...", + ) + if _wait_for_http_response_via_prober( + compose_command, prober_service, target_url, HEALTH_CHECK_TIMEOUT_SECONDS, root, env + ): + _report(on_progress, "Health check succeeded.") + return VerificationResult(True, "") + + logs = _run_subprocess([*compose_command, "logs"], root, env, 30.0) + state = _app_service_state(compose_command, app_service, root, env) + classification = ( + FAILURE_HEALTH_CHECK_TIMEOUT + if state in ("running", "unknown") + else FAILURE_CONTAINER_NEVER_STARTED + ) + return VerificationResult( + False, + f"No HTTP response on {target_url} within " + f"{int(HEALTH_CHECK_TIMEOUT_SECONDS)}s.\n{_tail(logs.stdout)}", + logs.stdout, + classification, + ) + finally: + _report(on_progress, "Cleaning up containers and images...") + _run_subprocess( + [*compose_command, "down", "--volumes", "--rmi", "local", "-t", "5"], + root, + env, + COMPOSE_DOWN_TIMEOUT_SECONDS, + ) diff --git a/src/code_audit/github_client.py b/src/code_audit/github_client.py index 9bb29a6..3e0f085 100644 --- a/src/code_audit/github_client.py +++ b/src/code_audit/github_client.py @@ -6,7 +6,16 @@ import httpx -from code_audit.models import Advisory, Commit, CommitFile, Comparison, FileContent, PullRequest +from code_audit.models import ( + Advisory, + Commit, + CommitFile, + CommitParent, + Comparison, + FileContent, + PullRequest, + Tag, +) API_BASE_URL = "https://api.github.com" @@ -39,6 +48,10 @@ class ContentNotFoundError(GitHubClientError): """Raised when the requested repository content does not exist.""" +class RepositoryNotFoundError(GitHubClientError): + """Raised when the requested repository does not exist.""" + + class GitHubClient: """Synchronous client for the GitHub REST API.""" @@ -224,6 +237,22 @@ def _rename_source( return parents[0]["sha"], previous_filename return None + def list_tags(self, repository: str, max_pages: int = 3) -> list[Tag]: + """List up to max_pages*100 tags of a repository, in the API's own order.""" + not_found_error = RepositoryNotFoundError(f"Repository {repository!r} was not found.") + params = {"per_page": "100"} + response = self._get(f"/repos/{repository}/tags", not_found_error, params=params) + tags = [_parse_tag(item) for item in response.json()] + pages_fetched = 1 + while pages_fetched < max_pages: + next_link = response.links.get("next") + if next_link is None: + break + response = self._get(next_link["url"], not_found_error) + tags.extend(_parse_tag(item) for item in response.json()) + pages_fetched += 1 + return tags + def close(self) -> None: self._client.close() @@ -310,4 +339,9 @@ def _retry_delay(response: httpx.Response, backoff: float) -> float: def _parse_commit(data: dict[str, Any]) -> Commit: files = [CommitFile.model_validate(item) for item in data.get("files", [])] - return Commit(sha=data["sha"], message=data["commit"]["message"], files=files) + parents = [CommitParent.model_validate(item) for item in data.get("parents", [])] + return Commit(sha=data["sha"], message=data["commit"]["message"], files=files, parents=parents) + + +def _parse_tag(data: dict[str, Any]) -> Tag: + return Tag(name=data["name"], sha=data["commit"]["sha"]) diff --git a/src/code_audit/models.py b/src/code_audit/models.py index ebce27f..3e3903b 100644 --- a/src/code_audit/models.py +++ b/src/code_audit/models.py @@ -1,6 +1,19 @@ from pydantic import BaseModel, Field +class VulnerabilityPackage(BaseModel): + ecosystem: str | None = None + name: str | None = None + + +class Vulnerability(BaseModel): + """One affected-package entry from a GitHub Security Advisory.""" + + package: VulnerabilityPackage | None = None + vulnerable_version_range: str | None = None + first_patched_version: str | None = None + + class Advisory(BaseModel): """A GitHub security advisory, reduced to the fields needed for tracing.""" @@ -10,6 +23,7 @@ class Advisory(BaseModel): severity: str | None = None source_code_location: str | None = None references: list[str] = Field(default_factory=list) + vulnerabilities: list[Vulnerability] = Field(default_factory=list) class CommitFile(BaseModel): @@ -21,6 +35,10 @@ class CommitFile(BaseModel): previous_filename: str | None = None +class CommitParent(BaseModel): + sha: str + + class Commit(BaseModel): """A commit from the GitHub REST API, reduced to the fields needed for tracing. @@ -31,6 +49,7 @@ class Commit(BaseModel): sha: str message: str files: list[CommitFile] = Field(default_factory=list) + parents: list[CommitParent] = Field(default_factory=list) class PullRequest(BaseModel): @@ -60,6 +79,13 @@ class FileContent(BaseModel): content: str +class Tag(BaseModel): + """A git tag from the GitHub REST API.""" + + name: str + sha: str + + class TraceResult(BaseModel): """The traced commits and pull requests for a single advisory.""" diff --git a/tests/test_cli.py b/tests/test_cli.py index f494a4a..6f526aa 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -1,5 +1,6 @@ import json from pathlib import Path +from typing import Any import pytest from anthropic.types import Usage @@ -8,6 +9,7 @@ import code_audit.cli from code_audit.cli import app from code_audit.config import Config, ConfigError +from code_audit.dockerize import DockerizeOutcome, DockerizeStatus from code_audit.github_client import GitHubClient, GitHubClientError from code_audit.models import Advisory, Commit, PullRequest, TraceResult @@ -16,6 +18,26 @@ ADVISORY = Advisory(ghsa_id="GHSA-jfh8-c2jp-5v3q", summary="Remote code injection in Log4j") +def _prepare_dockerize(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + Config, + "from_env", + classmethod(lambda cls: Config(github_token="t", anthropic_api_key="k")), + ) + monkeypatch.setattr(GitHubClient, "fetch_advisory", lambda self, ghsa_id: ADVISORY) + monkeypatch.setattr(code_audit.cli, "missing_prerequisite", lambda: None) + + +def _capturing_dockerize_advisory(received: dict[str, Any]) -> Any: + def fake( + advisory: Advisory, github_client: GitHubClient, anthropic_client: object, **kwargs: Any + ) -> DockerizeOutcome: + received.update(kwargs) + return DockerizeOutcome(DockerizeStatus.SUCCEEDED, "Generated a Dockerfile.", "output/x") + + return fake + + def test_trace_runs_the_full_pipeline(monkeypatch: pytest.MonkeyPatch) -> None: deterministic = TraceResult(ghsa_id=ADVISORY.ghsa_id) completed = deterministic.model_copy( @@ -412,3 +434,120 @@ def fake_complete_trace( lines = transcript.read_text(encoding="utf-8").splitlines() assert len(lines) == 1 assert json.loads(lines[0])["turn"] == 1 + + +def test_dockerize_trace_result_option_loads_a_saved_trace_json( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + saved = TraceResult(ghsa_id=ADVISORY.ghsa_id, fixing_commit=Commit(sha="a" * 40, message="Fix")) + trace_path = tmp_path / "trace.json" + trace_path.write_text(saved.model_dump_json(), encoding="utf-8") + _prepare_dockerize(monkeypatch) + received: dict[str, Any] = {} + monkeypatch.setattr( + code_audit.cli, "dockerize_advisory", _capturing_dockerize_advisory(received) + ) + + result = runner.invoke(app, ["dockerize", ADVISORY.ghsa_id, "--trace-result", str(trace_path)]) + + assert result.exit_code == 0, result.output + assert received["trace_result"] == saved + + +def test_dockerize_fixing_commit_option_builds_a_trace_result( + monkeypatch: pytest.MonkeyPatch, +) -> None: + built = TraceResult(ghsa_id=ADVISORY.ghsa_id, fixing_commit=Commit(sha="b" * 40, message="Fix")) + _prepare_dockerize(monkeypatch) + monkeypatch.setattr( + code_audit.cli, + "build_trace_result_from_commit_sha", + lambda advisory, sha, client: built, + ) + received: dict[str, Any] = {} + monkeypatch.setattr( + code_audit.cli, "dockerize_advisory", _capturing_dockerize_advisory(received) + ) + + result = runner.invoke(app, ["dockerize", ADVISORY.ghsa_id, "--fixing-commit", "b" * 40]) + + assert result.exit_code == 0, result.output + assert received["trace_result"] == built + + +def test_dockerize_trace_option_runs_the_full_trace_agent(monkeypatch: pytest.MonkeyPatch) -> None: + deterministic = TraceResult(ghsa_id=ADVISORY.ghsa_id) + completed = TraceResult( + ghsa_id=ADVISORY.ghsa_id, fixing_commit=Commit(sha="c" * 40, message="Fix") + ) + _prepare_dockerize(monkeypatch) + monkeypatch.setattr(code_audit.cli, "trace_advisory", lambda advisory, client: deterministic) + monkeypatch.setattr( + code_audit.cli, + "complete_trace", + lambda advisory, deterministic_result, client, anthropic_client: completed, + ) + received: dict[str, Any] = {} + monkeypatch.setattr( + code_audit.cli, "dockerize_advisory", _capturing_dockerize_advisory(received) + ) + + result = runner.invoke(app, ["dockerize", ADVISORY.ghsa_id, "--trace"]) + + assert result.exit_code == 0, result.output + assert received["trace_result"] == completed + + +def test_dockerize_rejects_multiple_trace_source_options(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(code_audit.cli, "missing_prerequisite", lambda: None) + + result = runner.invoke( + app, ["dockerize", ADVISORY.ghsa_id, "--trace", "--fixing-commit", "a" * 40] + ) + + assert result.exit_code == 1 + assert "mutually exclusive" in result.stderr + + +def test_dockerize_interactive_prompt_accepts_trace(monkeypatch: pytest.MonkeyPatch) -> None: + deterministic = TraceResult(ghsa_id=ADVISORY.ghsa_id) + completed = TraceResult( + ghsa_id=ADVISORY.ghsa_id, fixing_commit=Commit(sha="d" * 40, message="Fix") + ) + _prepare_dockerize(monkeypatch) + monkeypatch.setattr(code_audit.cli, "trace_advisory", lambda advisory, client: deterministic) + monkeypatch.setattr( + code_audit.cli, + "complete_trace", + lambda advisory, deterministic_result, client, anthropic_client: completed, + ) + received: dict[str, Any] = {} + monkeypatch.setattr( + code_audit.cli, "dockerize_advisory", _capturing_dockerize_advisory(received) + ) + + result = runner.invoke(app, ["dockerize", ADVISORY.ghsa_id], input="y\n") + + assert result.exit_code == 0, result.output + assert received["trace_result"] == completed + + +def test_dockerize_interactive_prompt_declines_trace_and_accepts_a_sha( + monkeypatch: pytest.MonkeyPatch, +) -> None: + built = TraceResult(ghsa_id=ADVISORY.ghsa_id, fixing_commit=Commit(sha="e" * 40, message="Fix")) + _prepare_dockerize(monkeypatch) + monkeypatch.setattr( + code_audit.cli, + "build_trace_result_from_commit_sha", + lambda advisory, sha, client: built, + ) + received: dict[str, Any] = {} + monkeypatch.setattr( + code_audit.cli, "dockerize_advisory", _capturing_dockerize_advisory(received) + ) + + result = runner.invoke(app, ["dockerize", ADVISORY.ghsa_id], input=f"n\n{'e' * 40}\n") + + assert result.exit_code == 0, result.output + assert received["trace_result"] == built diff --git a/tests/test_dockerize.py b/tests/test_dockerize.py new file mode 100644 index 0000000..ad891db --- /dev/null +++ b/tests/test_dockerize.py @@ -0,0 +1,1097 @@ +# Building and running containers with Docker, and the sandboxed verification +# step, are not practical to unit test: they need a real Docker daemon. Those +# are exercised manually, the same way the project's own Dockerfile is not +# covered by an automated test. The generation agent's tool loop is testable +# with a scripted fake Anthropic client, the same pattern test_agent.py uses. + +import json +from pathlib import Path +from types import SimpleNamespace +from typing import Any, cast + +import anthropic +import httpx +import pytest + +import code_audit.dockerize +from code_audit.dockerize import ( + ISOLATED_NETWORK_NAME, + ClassificationResult, + DockerizeStatus, + GeneratedArtifacts, + TargetCommit, + TargetCommitError, + VerificationResult, + _has_build_backend_only_pyproject, + _has_manage_py, + _has_npm_start_script, + _has_python_main_serving_block, + _has_setup_py_without_entrypoint, + _minimal_subprocess_env, + _sandbox_compose, + _unique_service_name, + application_artifacts_already_exist, + classify_application, + dockerize_advisory, + generate_docker_artifacts, + resolve_classification, + resolve_target_commit, +) +from code_audit.github_client import GitHubClient +from code_audit.models import Advisory, Vulnerability, VulnerabilityPackage + +REPOSITORY = "apache/logging-log4j2" +FIXING_SHA = "c77b3cb39312b83b053d23a2158b6e528f9a6ab9" +PARENT_SHA = "44569090f1cf1e92c711fb96dfd18cd7dccc72ea" + + +def make_client(responses: dict[str, httpx.Response]) -> GitHubClient: + def handler(request: httpx.Request) -> httpx.Response: + assert request.url.path in responses, f"unexpected request: {request.url.path}" + return responses[request.url.path] + + return GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) + + +def make_advisory( + vulnerable_version_range: str | None = None, references: list[str] | None = None +) -> Advisory: + vulnerabilities = [] + if vulnerable_version_range is not None: + vulnerabilities.append( + Vulnerability( + package=VulnerabilityPackage(ecosystem="maven", name="log4j-core"), + vulnerable_version_range=vulnerable_version_range, + first_patched_version="2.15.0", + ) + ) + return Advisory( + ghsa_id="GHSA-jfh8-c2jp-5v3q", + summary="Remote code injection in Log4j", + source_code_location=f"https://github.com/{REPOSITORY}", + references=references or [], + vulnerabilities=vulnerabilities, + ) + + +def tags_response(tags: list[tuple[str, str]]) -> httpx.Response: + return httpx.Response(200, json=[{"name": name, "commit": {"sha": sha}} for name, sha in tags]) + + +def fixing_commit_response(sha: str, parent_sha: str | None) -> httpx.Response: + parents = [{"sha": parent_sha}] if parent_sha else [] + return httpx.Response(200, json={"sha": sha, "commit": {"message": "Fix"}, "parents": parents}) + + +def pull_request_response( + number: int, merge_commit_sha: str, merged: bool = True +) -> httpx.Response: + return httpx.Response( + 200, + json={ + "number": number, + "title": "Fix", + "state": "closed", + "merged": merged, + "merge_commit_sha": merge_commit_sha, + }, + ) + + +# Target commit resolution + + +def test_resolve_target_commit_prefers_tag_for_last_affected_version() -> None: + advisory = make_advisory(vulnerable_version_range="<= 2.14.1") + responses = { + f"/repos/{REPOSITORY}/tags": tags_response( + [("rel/2.13.0", "a" * 40), ("rel/2.14.1", "b" * 40), ("rel/2.15.0", "c" * 40)] + ), + } + + with make_client(responses) as client: + target = resolve_target_commit(advisory, client) + + assert target.sha == "b" * 40 + assert target.fetch_ref == "rel/2.14.1" + assert "last affected version" in target.description + + +def test_resolve_target_commit_falls_back_when_no_tag_matches() -> None: + advisory = make_advisory( + vulnerable_version_range="<= 2.14.1", + references=[f"https://github.com/{REPOSITORY}/commit/{FIXING_SHA}"], + ) + responses = { + f"/repos/{REPOSITORY}/tags": tags_response([("rel/2.15.0", "c" * 40)]), + f"/repos/{REPOSITORY}/commits/{FIXING_SHA}": fixing_commit_response(FIXING_SHA, PARENT_SHA), + } + + with make_client(responses) as client: + target = resolve_target_commit(advisory, client) + + assert target.sha == PARENT_SHA + assert target.fetch_ref == PARENT_SHA + assert "parent of fixing commit" in target.description + + +def test_resolve_target_commit_skips_tag_lookup_for_exclusive_only_range() -> None: + # '< 2.15.0' has no literal last-affected version, so no tags request + # should ever be made; make_client fails the test if one is attempted. + advisory = make_advisory( + vulnerable_version_range="< 2.15.0", + references=[f"https://github.com/{REPOSITORY}/commit/{FIXING_SHA}"], + ) + responses = { + f"/repos/{REPOSITORY}/commits/{FIXING_SHA}": fixing_commit_response(FIXING_SHA, PARENT_SHA), + } + + with make_client(responses) as client: + target = resolve_target_commit(advisory, client) + + assert target.sha == PARENT_SHA + + +def test_resolve_target_commit_falls_back_via_merged_pull_requests_merge_commit() -> None: + # The advisory references only the fixing pull request, not the commit + # directly (this is how GHSA-jfh8-c2jp-5v3q itself references its fix), + # so trace_advisory's deterministic result has fixing_commit as None and + # only fixing_pull_request set. + advisory = make_advisory(references=[f"https://github.com/{REPOSITORY}/pull/608"]) + responses = { + f"/repos/{REPOSITORY}/pulls/608": pull_request_response(608, FIXING_SHA), + f"/repos/{REPOSITORY}/commits/{FIXING_SHA}": fixing_commit_response(FIXING_SHA, PARENT_SHA), + } + + with make_client(responses) as client: + target = resolve_target_commit(advisory, client) + + assert target.sha == PARENT_SHA + assert target.fetch_ref == PARENT_SHA + assert "parent of fixing commit" in target.description + + +def test_resolve_target_commit_ignores_an_unmerged_pull_request() -> None: + advisory = make_advisory(references=[f"https://github.com/{REPOSITORY}/pull/608"]) + responses = { + f"/repos/{REPOSITORY}/pulls/608": pull_request_response(608, FIXING_SHA, merged=False), + } + + with make_client(responses) as client, pytest.raises(TargetCommitError): + resolve_target_commit(advisory, client) + + +def test_resolve_target_commit_raises_when_nothing_resolves() -> None: + advisory = make_advisory() + + with make_client({}) as client, pytest.raises(TargetCommitError) as excinfo: + resolve_target_commit(advisory, client) + + # No version range at all means no tag lookup was ever attempted, so the + # message must not claim one ran and failed. + assert "does not name a specific last affected version" in str(excinfo.value) + assert "no tag matches" not in str(excinfo.value) + + +def test_resolve_target_commit_error_message_reports_a_failed_tag_lookup() -> None: + advisory = make_advisory(vulnerable_version_range="<= 2.14.1") + responses = { + f"/repos/{REPOSITORY}/tags": tags_response([("rel/2.15.0", "c" * 40)]), + } + + with make_client(responses) as client, pytest.raises(TargetCommitError) as excinfo: + resolve_target_commit(advisory, client) + + # A tag lookup did run here and found nothing, which is a different + # situation from never attempting one, so the message must say so. + assert "no tag matches the last affected version '2.14.1'" in str(excinfo.value) + + +def test_resolve_target_commit_raises_when_repository_is_unresolvable() -> None: + advisory = Advisory(ghsa_id="GHSA-x", summary="s") + + with make_client({}) as client, pytest.raises(TargetCommitError): + resolve_target_commit(advisory, client) + + +def test_resolve_target_commit_raises_when_fixing_commit_has_no_parent() -> None: + advisory = make_advisory(references=[f"https://github.com/{REPOSITORY}/commit/{FIXING_SHA}"]) + responses = { + f"/repos/{REPOSITORY}/commits/{FIXING_SHA}": fixing_commit_response(FIXING_SHA, None), + } + + with make_client(responses) as client, pytest.raises(TargetCommitError): + resolve_target_commit(advisory, client) + + +# Classification signals + + +def test_has_manage_py(tmp_path: Path) -> None: + assert _has_manage_py(tmp_path, None) is False + (tmp_path / "manage.py").write_text("", encoding="utf-8") + assert _has_manage_py(tmp_path, None) is True + + +def test_has_npm_start_script(tmp_path: Path) -> None: + (tmp_path / "package.json").write_text('{"name": "app"}', encoding="utf-8") + assert _has_npm_start_script(tmp_path, None) is False + (tmp_path / "package.json").write_text( + '{"name": "app", "scripts": {"start": "node index.js"}}', encoding="utf-8" + ) + assert _has_npm_start_script(tmp_path, None) is True + + +def test_has_python_main_serving_block(tmp_path: Path) -> None: + (tmp_path / "app.py").write_text("def helper():\n pass\n", encoding="utf-8") + assert _has_python_main_serving_block(tmp_path, None) is False + (tmp_path / "app.py").write_text( + 'if __name__ == "__main__":\n app.run(host="0.0.0.0")\n', encoding="utf-8" + ) + assert _has_python_main_serving_block(tmp_path, None) is True + + +def test_has_build_backend_only_pyproject(tmp_path: Path) -> None: + (tmp_path / "pyproject.toml").write_text( + '[build-system]\nrequires = ["setuptools"]\n\n[project]\nname = "mylib"\n', + encoding="utf-8", + ) + assert _has_build_backend_only_pyproject(tmp_path, None) is True + + (tmp_path / "pyproject.toml").write_text( + '[build-system]\nrequires = ["setuptools"]\n\n' + '[project]\nname = "mylib"\nscripts = { mylib = "mylib.cli:main" }\n', + encoding="utf-8", + ) + assert _has_build_backend_only_pyproject(tmp_path, None) is False + + +def test_has_setup_py_without_entrypoint(tmp_path: Path) -> None: + assert _has_setup_py_without_entrypoint(tmp_path, None) is False + (tmp_path / "setup.py").write_text("", encoding="utf-8") + assert _has_setup_py_without_entrypoint(tmp_path, None) is True + (tmp_path / "manage.py").write_text("", encoding="utf-8") + assert _has_setup_py_without_entrypoint(tmp_path, None) is False + + +def test_classify_application_detects_a_clear_application(tmp_path: Path) -> None: + (tmp_path / "manage.py").write_text("", encoding="utf-8") + (tmp_path / "Procfile").write_text("web: gunicorn app:app\n", encoding="utf-8") + + result = classify_application(tmp_path, None) + + assert result.is_application is True + assert "manage.py" in result.application_signals + + +def test_classify_application_detects_a_clear_library(tmp_path: Path) -> None: + (tmp_path / "pyproject.toml").write_text( + '[build-system]\nrequires = ["setuptools"]\n\n[project]\nname = "mylib"\n', + encoding="utf-8", + ) + + result = classify_application(tmp_path, None) + + assert result.is_application is False + assert "build_backend_only_pyproject" in result.library_signals + + +def test_classify_application_treats_a_tie_as_not_application(tmp_path: Path) -> None: + # A Procfile (application signal) alongside a package.json with no start + # script (library signal): the score ties, so this must not be guessed + # as an application. + (tmp_path / "Procfile").write_text("web: node server.js\n", encoding="utf-8") + (tmp_path / "package.json").write_text('{"name": "app"}', encoding="utf-8") + + result = classify_application(tmp_path, None) + + assert result.is_application is False + + +def _tier2_response(is_application: bool, reason: str) -> Any: + payload = {"is_application": is_application, "reason": reason} + return SimpleNamespace( + stop_reason="end_turn", content=[SimpleNamespace(type="text", text=json.dumps(payload))] + ) + + +def test_resolve_classification_trusts_a_clear_tier1_margin_without_calling_tier2( + tmp_path: Path, +) -> None: + # manage.py plus a Procfile is 2 application signals against 0 library + # signals, a clear enough margin that Tier 1's own verdict should be + # trusted directly, with no Tier 2 call at all. + (tmp_path / "manage.py").write_text("", encoding="utf-8") + (tmp_path / "Procfile").write_text("web: gunicorn app:app\n", encoding="utf-8") + client, fake = _scripted_anthropic_client([]) + + result = resolve_classification(tmp_path, None, client) + + assert result.is_application is True + assert result.tier2_reason is None + assert fake.requests == [] + + +def test_resolve_classification_calls_tier2_when_tier1_is_ambiguous(tmp_path: Path) -> None: + # The same Procfile-plus-no-start-script tie as + # test_classify_application_treats_a_tie_as_not_application: Tier 1 has + # no clear margin, so this must defer to Tier 2 instead of guessing, and + # Tier 2's verdict is what is actually returned, not Tier 1's own tie. + (tmp_path / "Procfile").write_text("web: node server.js\n", encoding="utf-8") + (tmp_path / "package.json").write_text('{"name": "app"}', encoding="utf-8") + response = _tier2_response(True, "It starts a server via a Procfile, so it is an application.") + client, fake = _scripted_anthropic_client([response]) + records: list[dict[str, Any]] = [] + + result = resolve_classification(tmp_path, None, client, on_turn=records.append) + + assert len(fake.requests) == 1 + assert result.is_application is True + assert result.tier2_reason == "It starts a server via a Procfile, so it is an application." + assert len(records) == 1 + assert records[0]["step"] == "classification_tiebreaker" + assert records[0]["final_response_text"] == response.content[0].text + + +# Missing-artifact detection + + +def test_dockerfile_anywhere_in_the_tree_counts_as_present(tmp_path: Path) -> None: + nested = tmp_path / "backend" + nested.mkdir() + (nested / "Dockerfile").write_text("FROM python:3.12-slim\n", encoding="utf-8") + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is True + assert "Dockerfile" in reason + + +def test_infra_only_compose_does_not_count_as_present(tmp_path: Path) -> None: + (tmp_path / "docker-compose.yml").write_text( + "services:\n db:\n image: postgres:16\n", encoding="utf-8" + ) + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is False + assert "No application" in reason + + +def test_compose_with_a_build_service_counts_as_present(tmp_path: Path) -> None: + (tmp_path / "compose.yml").write_text( + "services:\n app:\n build: .\n db:\n image: postgres:16\n", encoding="utf-8" + ) + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is True + assert "compose.yml" in reason + + +def test_compose_with_image_matching_repository_name_counts_as_present(tmp_path: Path) -> None: + (tmp_path / "docker-compose.yaml").write_text( + "services:\n web:\n image: myrepo:latest\n", encoding="utf-8" + ) + + present, _reason = application_artifacts_already_exist(tmp_path, "owner/myrepo") + + assert present is True + + +def test_docker_directory_counts_as_present(tmp_path: Path) -> None: + (tmp_path / "docker").mkdir() + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is True + assert "docker/" in reason + + +def test_devcontainer_directory_counts_as_present(tmp_path: Path) -> None: + devcontainer = tmp_path / ".devcontainer" + devcontainer.mkdir() + (devcontainer / "devcontainer.json").write_text("{}", encoding="utf-8") + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is True + assert ".devcontainer" in reason + + +def test_no_artifacts_reports_missing(tmp_path: Path) -> None: + (tmp_path / "app.py").write_text("", encoding="utf-8") + + present, reason = application_artifacts_already_exist(tmp_path, "owner/repo") + + assert present is False + assert "No application" in reason + + +# Sandboxing +# +# These test the pure dict-transformation logic directly, without a real +# Docker daemon; the actual isolation (an internal network really blocks +# egress, a prober can still reach the app over it) was verified manually +# against a real container, since that is Docker's own behavior, not +# something this function can be unit tested into being true. + + +def test_sandbox_compose_strips_privileged_and_cap_add() -> None: + data = {"services": {"app": {"build": ".", "privileged": True, "cap_add": ["NET_ADMIN"]}}} + + _sandbox_compose(data) + + assert "privileged" not in data["services"]["app"] + assert "cap_add" not in data["services"]["app"] + assert data["services"]["app"]["cap_drop"] == ["ALL"] + + +def test_sandbox_compose_strips_network_mode() -> None: + data = {"services": {"app": {"build": ".", "network_mode": "host"}}} + + _sandbox_compose(data) + + assert "network_mode" not in data["services"]["app"] + + +def test_sandbox_compose_strips_host_bind_mounts_but_keeps_named_volumes() -> None: + data = { + "services": { + "app": { + "build": ".", + "volumes": ["./data:/data", "/etc/passwd:/etc/passwd", "named-vol:/var/lib/data"], + } + } + } + + _sandbox_compose(data) + + assert data["services"]["app"]["volumes"] == ["named-vol:/var/lib/data"] + + +def test_sandbox_compose_removes_ports_since_the_isolated_network_cannot_publish_them() -> None: + data = {"services": {"app": {"build": ".", "ports": ["8000"]}}} + + _sandbox_compose(data) + + assert "ports" not in data["services"]["app"] + + +def test_sandbox_compose_puts_every_service_including_the_prober_on_the_isolated_network() -> None: + data: dict[str, Any] = {"services": {"app": {"build": "."}, "db": {"image": "postgres:16"}}} + + prober_name = _sandbox_compose(data) + + for service in data["services"].values(): + assert service["networks"] == [ISOLATED_NETWORK_NAME] + assert data["networks"] == {ISOLATED_NETWORK_NAME: {"internal": True}} + assert prober_name in data["services"] + + +def test_sandbox_compose_applies_resource_limits_to_every_service() -> None: + data: dict[str, Any] = {"services": {"app": {"build": "."}, "db": {"image": "postgres:16"}}} + + prober_name = _sandbox_compose(data) + + for service in data["services"].values(): + assert service["mem_limit"] == "512m" + assert service["cpus"] == 1.0 + assert service["pids_limit"] == 256 + assert data["services"][prober_name]["cap_drop"] == ["ALL"] + + +def test_unique_service_name_avoids_colliding_with_an_existing_service() -> None: + services: dict[str, Any] = {"code_audit_prober": {}} + + name = _unique_service_name(services, "code_audit_prober") + + assert name == "code_audit_prober1" + assert name not in services + + +def test_minimal_subprocess_env_only_includes_the_allowlist( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("PATH", "/usr/bin") + monkeypatch.setenv("HOME", "/home/tester") + monkeypatch.setenv("GITHUB_TOKEN", "leaked-github-token") + monkeypatch.setenv("ANTHROPIC_API_KEY", "leaked-anthropic-key") + monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "leaked-aws-secret") + + env = _minimal_subprocess_env() + + assert env["PATH"] == "/usr/bin" + assert env["HOME"] == "/home/tester" + assert "GITHUB_TOKEN" not in env + assert "ANTHROPIC_API_KEY" not in env + assert "AWS_SECRET_ACCESS_KEY" not in env + + +# Orchestration + + +def _fake_anthropic_client() -> anthropic.Anthropic: + return cast(anthropic.Anthropic, object()) + + +class _ScriptedAnthropicClient: + """Plays back scripted responses and records the requests it receives. + + The same pattern test_agent.py's FakeAnthropicClient uses. + """ + + def __init__(self, responses: list[Any]) -> None: + self.requests: list[dict[str, Any]] = [] + self._responses = iter(responses) + self.messages = SimpleNamespace(create=self._create) + + def _create(self, **kwargs: Any) -> Any: + self.requests.append(kwargs) + return next(self._responses) + + +def _scripted_anthropic_client(responses: list[Any]) -> tuple[anthropic.Anthropic, Any]: + fake = _ScriptedAnthropicClient(responses) + return cast(anthropic.Anthropic, fake), fake + + +def _generation_response(dockerfile: str, compose_yaml: str | None = None, port: int = 8000) -> Any: + payload = { + "dockerfile": dockerfile, + "compose_yaml": compose_yaml, + "port": port, + "explanation": "test", + } + return SimpleNamespace( + stop_reason="end_turn", content=[SimpleNamespace(type="text", text=json.dumps(payload))] + ) + + +def test_dockerize_advisory_reports_existing_artifact_message_only_once( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(True, ["manage.py"], []) + ), + ) + reason = "A Dockerfile already exists at Dockerfile." + monkeypatch.setattr( + code_audit.dockerize, + "application_artifacts_already_exist", + lambda root, repository: (True, reason), + ) + progress_messages: list[str] = [] + + advisory = Advisory(ghsa_id="GHSA-dup-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory( + advisory, client, _fake_anthropic_client(), on_progress=progress_messages.append + ) + + assert outcome.status == DockerizeStatus.ALREADY_CONTAINERIZED + assert outcome.message == reason + # The reason must appear exactly once across everything the CLI would + # print: only in the returned outcome message, never also echoed as a + # separate on_progress line to stderr. + assert reason not in progress_messages + + +def test_generate_docker_artifacts_rejects_an_empty_dockerfile(tmp_path: Path) -> None: + client, _fake = _scripted_anthropic_client([_generation_response(dockerfile="")]) + + artifacts = generate_docker_artifacts(tmp_path, client) + + assert artifacts is None + + +def test_generate_docker_artifacts_rejects_a_dockerfile_without_from(tmp_path: Path) -> None: + response = _generation_response(dockerfile='RUN echo hello\nCMD ["true"]\n') + client, _fake = _scripted_anthropic_client([response]) + + artifacts = generate_docker_artifacts(tmp_path, client) + + assert artifacts is None + + +def test_generate_docker_artifacts_accepts_a_dockerfile_with_from(tmp_path: Path) -> None: + response = _generation_response(dockerfile='FROM python:3.12-slim\nCMD ["true"]\n') + client, _fake = _scripted_anthropic_client([response]) + + artifacts = generate_docker_artifacts(tmp_path, client) + + assert artifacts is not None + assert artifacts.dockerfile.startswith("FROM") + + +def test_generate_docker_artifacts_requests_a_separate_thinking_budget(tmp_path: Path) -> None: + # A generation turn previously hit max_tokens with no tool call attempted, + # because reasoning shared the same visible token budget as the final + # structured output. Enabling thinking gives reasoning its own budget, + # the same fix agent.py already applies to the main tracing agent. + response = _generation_response(dockerfile="FROM python:3.12-slim\n") + client, fake = _scripted_anthropic_client([response]) + + generate_docker_artifacts(tmp_path, client) + + assert fake.requests[0]["thinking"] == {"type": "adaptive", "display": "summarized"} + assert fake.requests[0]["max_tokens"] == 16000 + + +def test_generate_docker_artifacts_debug_callback_receives_one_record_per_turn( + tmp_path: Path, +) -> None: + (tmp_path / "app.py").write_text("print('hi')\n", encoding="utf-8") + thinking_block = SimpleNamespace(type="thinking", thinking="checking the entrypoint") + tool_block = SimpleNamespace(type="tool_use", id="toolu_1", name="list_files", input={}) + tool_turn = SimpleNamespace(stop_reason="tool_use", content=[thinking_block, tool_block]) + final = _generation_response(dockerfile='FROM python:3.12-slim\nCMD ["true"]\n') + client, _fake = _scripted_anthropic_client([tool_turn, final]) + records: list[dict[str, Any]] = [] + + artifacts = generate_docker_artifacts(tmp_path, client, on_turn=records.append) + + assert artifacts is not None + assert [record["turn"] for record in records] == [1, 2] + first = records[0] + assert first["thinking"] == "checking the entrypoint" + assert first["tool_use"] == [{"name": "list_files", "input": {}}] + assert first["tool_results"][0]["tool_use_id"] == "toolu_1" + assert first["stop_reason"] == "tool_use" + assert first["final_response_text"] is None + second = records[1] + assert second["thinking"] is None + assert second["tool_use"] == [] + assert second["tool_results"] == [] + assert second["stop_reason"] == "end_turn" + assert second["final_response_text"] == final.content[0].text + + +def test_generate_docker_artifacts_debug_record_captures_raw_text_when_rejected( + tmp_path: Path, +) -> None: + final = _generation_response(dockerfile="") + client, _fake = _scripted_anthropic_client([final]) + records: list[dict[str, Any]] = [] + + artifacts = generate_docker_artifacts(tmp_path, client, on_turn=records.append) + + assert artifacts is None + assert len(records) == 1 + assert records[0]["final_response_text"] == final.content[0].text + + +def test_dockerize_advisory_force_bypasses_already_containerized_check( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(True, ["manage.py"], []) + ), + ) + monkeypatch.setattr( + code_audit.dockerize, + "application_artifacts_already_exist", + lambda root, repository: (True, "A Dockerfile already exists at Dockerfile."), + ) + generation_calls: list[Path] = [] + artifacts = GeneratedArtifacts( + dockerfile="FROM python:3.12-slim", compose_yaml=None, port=8000, explanation="test" + ) + + def fake_generate( + root: Path, + client: anthropic.Anthropic, + on_progress: object = None, + on_turn: object = None, + ) -> GeneratedArtifacts: + generation_calls.append(root) + return artifacts + + monkeypatch.setattr(code_audit.dockerize, "generate_docker_artifacts", fake_generate) + monkeypatch.setattr( + code_audit.dockerize, + "verify_docker_artifacts", + lambda root, artifacts, build_timeout_seconds, on_progress=None: VerificationResult( + True, "" + ), + ) + + advisory = Advisory(ghsa_id="GHSA-force-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client(), force=True) + + assert outcome.status == DockerizeStatus.SUCCEEDED + # Generation actually ran, proving force bypassed the already-present exit. + assert len(generation_calls) == 1 + assert outcome.forced_note is not None + assert "already exists" in outcome.forced_note + assert "--force" in outcome.forced_note + + +def _dockerize_advisory_ready_to_verify( + monkeypatch: pytest.MonkeyPatch, verification: VerificationResult +) -> Advisory: + """Wire dockerize_advisory up to reach verification with a scripted result. + + Every earlier step is mocked to succeed, isolating the behavior of the + verification-failure branch itself (writing build.log, setting + output_path), without needing a real clone, agent, or Docker daemon. + """ + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(True, ["manage.py"], []) + ), + ) + monkeypatch.setattr( + code_audit.dockerize, + "application_artifacts_already_exist", + lambda root, repository: (False, "No application Dockerfile or compose file was found."), + ) + artifacts = GeneratedArtifacts( + dockerfile="FROM python:3.12-slim", compose_yaml=None, port=8000, explanation="test" + ) + monkeypatch.setattr( + code_audit.dockerize, + "generate_docker_artifacts", + lambda root, client, on_progress=None, on_turn=None, repair_context=None: artifacts, + ) + monkeypatch.setattr( + code_audit.dockerize, + "verify_docker_artifacts", + lambda root, artifacts, build_timeout_seconds, on_progress=None: verification, + ) + return Advisory(ghsa_id="GHSA-buildlog-test", summary="s") + + +def test_dockerize_advisory_writes_full_build_log_on_verification_failure( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + # A truncated tail once hid the actual pnpm install error behind a + # generic "ELIFECYCLE" summary; the full, untruncated output must still + # be recoverable from disk. + full_log = "npm ERR! actual pnpm install failure detail\n" * 200 + full_log += "ELIFECYCLE Command failed with exit code 1" + verification = VerificationResult(False, "ELIFECYCLE Command failed with exit code 1", full_log) + advisory = _dockerize_advisory_ready_to_verify(monkeypatch, verification) + + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client(), repair_attempts=0) + + assert outcome.status == DockerizeStatus.VERIFICATION_FAILED + assert outcome.output_path is not None + build_log_path = Path(outcome.output_path) / "build.log" + assert build_log_path.is_file() + assert build_log_path.read_text(encoding="utf-8") == full_log + assert str(build_log_path) in outcome.message + + +def test_dockerize_advisory_does_not_write_an_empty_build_log( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + # No subprocess ever ran here (an invalid generated compose.yml), so + # there is no full build output to write out. + verification = VerificationResult(False, "Generated compose.yml has no services.") + advisory = _dockerize_advisory_ready_to_verify(monkeypatch, verification) + + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client(), repair_attempts=0) + + assert outcome.status == DockerizeStatus.VERIFICATION_FAILED + assert outcome.output_path is not None + assert not (Path(outcome.output_path) / "build.log").exists() + + +def _mock_pipeline_up_to_generation(monkeypatch: pytest.MonkeyPatch) -> None: + """Mock every dockerize_advisory step before generation to succeed. + + Shared setup for the repair-loop tests below, which each control + generation and verification themselves to script a multi-attempt cycle. + """ + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(True, ["manage.py"], []) + ), + ) + monkeypatch.setattr( + code_audit.dockerize, + "application_artifacts_already_exist", + lambda root, repository: (False, "No application Dockerfile or compose file was found."), + ) + + +def test_dockerize_advisory_repairs_and_succeeds_on_second_attempt( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + _mock_pipeline_up_to_generation(monkeypatch) + + broken_artifacts = GeneratedArtifacts( + dockerfile="FROM python:3.12-slim", compose_yaml=None, port=8000, explanation="first try" + ) + repaired_artifacts = GeneratedArtifacts( + dockerfile="FROM python:3.12-slim\nRUN pip install flask", + compose_yaml=None, + port=8000, + explanation="repaired", + ) + generate_calls: list[Any] = [] + + def fake_generate( + root: Path, + client: anthropic.Anthropic, + on_progress: object = None, + on_turn: object = None, + repair_context: Any = None, + ) -> GeneratedArtifacts: + generate_calls.append(repair_context) + return repaired_artifacts if repair_context is not None else broken_artifacts + + monkeypatch.setattr(code_audit.dockerize, "generate_docker_artifacts", fake_generate) + + verify_calls: list[GeneratedArtifacts] = [] + + def fake_verify( + root: Path, + artifacts: GeneratedArtifacts, + build_timeout_seconds: float, + on_progress: object = None, + ) -> VerificationResult: + verify_calls.append(artifacts) + if artifacts is broken_artifacts: + return VerificationResult( + False, + "npm ERR! something failed", + failure_classification="dependency resolution or install failed during the build", + ) + return VerificationResult(True, "") + + monkeypatch.setattr(code_audit.dockerize, "verify_docker_artifacts", fake_verify) + + advisory = Advisory(ghsa_id="GHSA-repair-success-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client()) + + assert outcome.status == DockerizeStatus.SUCCEEDED + assert verify_calls == [broken_artifacts, repaired_artifacts] + # The second generation call is the repair attempt, seeded with the + # first attempt's artifact and its classified verification failure. + assert generate_calls[0] is None + assert generate_calls[1] is not None + assert generate_calls[1].previous_artifacts is broken_artifacts + assert ( + generate_calls[1].verification.failure_classification + == "dependency resolution or install failed during the build" + ) + assert "1 repair attempt" in outcome.message + + +def test_dockerize_advisory_reports_failure_after_exhausting_repair_attempts( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + _mock_pipeline_up_to_generation(monkeypatch) + + attempt_count = 0 + + def fake_generate( + root: Path, + client: anthropic.Anthropic, + on_progress: object = None, + on_turn: object = None, + repair_context: Any = None, + ) -> GeneratedArtifacts: + nonlocal attempt_count + attempt_count += 1 + return GeneratedArtifacts( + dockerfile=f"FROM python:3.12-slim\n# attempt {attempt_count}", + compose_yaml=None, + port=8000, + explanation="test", + ) + + monkeypatch.setattr(code_audit.dockerize, "generate_docker_artifacts", fake_generate) + + verify_count = 0 + + def fake_verify( + root: Path, + artifacts: GeneratedArtifacts, + build_timeout_seconds: float, + on_progress: object = None, + ) -> VerificationResult: + nonlocal verify_count + verify_count += 1 + return VerificationResult( + False, + f"health check timed out on attempt {verify_count}", + failure_classification="the container started but the health check timed out", + ) + + monkeypatch.setattr(code_audit.dockerize, "verify_docker_artifacts", fake_verify) + + advisory = Advisory(ghsa_id="GHSA-repair-exhausted-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client(), repair_attempts=2) + + assert outcome.status == DockerizeStatus.VERIFICATION_FAILED + # One initial attempt plus both repair attempts were made and none succeeded. + assert attempt_count == 3 + assert verify_count == 3 + assert "3 attempts" in outcome.message + # The final attempt's failure is what gets reported, not an earlier one. + assert "health check timed out on attempt 3" in outcome.message + assert "the container started but the health check timed out" in outcome.message + + +def test_dockerize_advisory_passes_build_timeout_seconds_through_to_verification( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(True, ["manage.py"], []) + ), + ) + monkeypatch.setattr( + code_audit.dockerize, + "application_artifacts_already_exist", + lambda root, repository: (False, "No application Dockerfile or compose file was found."), + ) + artifacts = GeneratedArtifacts( + dockerfile="FROM python:3.12-slim", compose_yaml=None, port=8000, explanation="test" + ) + monkeypatch.setattr( + code_audit.dockerize, + "generate_docker_artifacts", + lambda root, client, on_progress=None, on_turn=None: artifacts, + ) + received_timeouts: list[float] = [] + + def fake_verify( + root: Path, + artifacts: GeneratedArtifacts, + build_timeout_seconds: float, + on_progress: object = None, + ) -> VerificationResult: + received_timeouts.append(build_timeout_seconds) + return VerificationResult(True, "") + + monkeypatch.setattr(code_audit.dockerize, "verify_docker_artifacts", fake_verify) + + advisory = Advisory(ghsa_id="GHSA-timeout-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory( + advisory, client, _fake_anthropic_client(), build_timeout_seconds=900.0 + ) + + assert outcome.status == DockerizeStatus.SUCCEEDED + assert received_timeouts == [900.0] + + +def test_dockerize_advisory_force_does_not_bypass_library_classification( + monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + monkeypatch.chdir(tmp_path) + target = TargetCommit(REPOSITORY, "a" * 40, "a" * 40, "tag 'v1.0.0' (last affected version)") + monkeypatch.setattr( + code_audit.dockerize, + "resolve_target_commit", + lambda advisory, client, trace_result=None: target, + ) + monkeypatch.setattr( + code_audit.dockerize, + "clone_repository_at_commit", + lambda repository, ref, destination: destination.mkdir(parents=True, exist_ok=True), + ) + monkeypatch.setattr( + code_audit.dockerize, + "resolve_classification", + lambda root, package_name, anthropic_client, on_progress=None, on_turn=None: ( + ClassificationResult(False, [], ["build_backend_only_pyproject"]) + ), + ) + generation_calls: list[Path] = [] + + def fake_generate(root: Path, client: anthropic.Anthropic) -> GeneratedArtifacts: + generation_calls.append(root) + raise AssertionError("generation must not run for a library, force or not") + + monkeypatch.setattr(code_audit.dockerize, "generate_docker_artifacts", fake_generate) + + advisory = Advisory(ghsa_id="GHSA-force-library-test", summary="s") + with make_client({}) as client: + outcome = dockerize_advisory(advisory, client, _fake_anthropic_client(), force=True) + + assert outcome.status == DockerizeStatus.NOT_A_WEB_APPLICATION + assert generation_calls == [] diff --git a/tests/test_github_client.py b/tests/test_github_client.py index ab69561..5592615 100644 --- a/tests/test_github_client.py +++ b/tests/test_github_client.py @@ -12,6 +12,7 @@ GitHubClient, GitHubClientError, PullRequestNotFoundError, + RepositoryNotFoundError, ) ADVISORY_PAYLOAD = { @@ -121,6 +122,28 @@ def handler(request: httpx.Request) -> httpx.Response: client.fetch_commit("apache/logging-log4j2", "0000000") +def test_fetch_commit_parses_parents() -> None: + payload = {**COMMIT_PAYLOAD, "parents": [{"sha": "d" * 40}, {"sha": "e" * 40}]} + + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(200, json=payload) + + with GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) as client: + commit = client.fetch_commit("apache/logging-log4j2", COMMIT_SHA) + + assert [parent.sha for parent in commit.parents] == ["d" * 40, "e" * 40] + + +def test_fetch_commit_without_parents_defaults_to_empty_list() -> None: + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(200, json=COMMIT_PAYLOAD) + + with GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) as client: + commit = client.fetch_commit("apache/logging-log4j2", COMMIT_SHA) + + assert commit.parents == [] + + def test_fetch_pull_request_happy_path() -> None: def handler(request: httpx.Request) -> httpx.Response: assert request.url.path == "/repos/apache/logging-log4j2/pulls/608" @@ -464,6 +487,61 @@ def test_list_commits_rename_follow_respects_max_pages() -> None: assert [commit.sha for commit in commits] == [NEW_SHA, RENAME_SHA] +def test_list_tags_happy_path() -> None: + payload = [ + {"name": "rel/2.15.0", "commit": {"sha": "a" * 40}}, + {"name": "rel/2.14.1", "commit": {"sha": "b" * 40}}, + ] + + def handler(request: httpx.Request) -> httpx.Response: + assert request.url.path == "/repos/apache/logging-log4j2/tags" + assert request.url.params["per_page"] == "100" + return httpx.Response(200, json=payload) + + with GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) as client: + tags = client.list_tags("apache/logging-log4j2") + + assert [(tag.name, tag.sha) for tag in tags] == [ + ("rel/2.15.0", "a" * 40), + ("rel/2.14.1", "b" * 40), + ] + + +def test_list_tags_follows_link_headers_up_to_max_pages() -> None: + page_one = [{"name": "v1.0.0", "commit": {"sha": "a" * 40}}] + page_two = [{"name": "v2.0.0", "commit": {"sha": "b" * 40}}] + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.params.get("page") is None: + return httpx.Response( + 200, + json=page_one, + headers={ + "Link": ( + "; rel="next"' + ) + }, + ) + return httpx.Response(200, json=page_two) + + with GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) as client: + tags = client.list_tags("apache/logging-log4j2", max_pages=2) + + assert [tag.name for tag in tags] == ["v1.0.0", "v2.0.0"] + + +def test_list_tags_repository_not_found() -> None: + def handler(request: httpx.Request) -> httpx.Response: + return httpx.Response(404, json={"message": "Not Found"}) + + with ( + GitHubClient(token="test-token", transport=httpx.MockTransport(handler)) as client, + pytest.raises(RepositoryNotFoundError), + ): + client.list_tags("apache/does-not-exist") + + def test_get_retries_secondary_rate_limit_then_succeeds(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr("code_audit.github_client.time.sleep", lambda _seconds: None) calls: list[httpx.Request] = [] diff --git a/uv.lock b/uv.lock index c726e03..9e11c86 100644 --- a/uv.lock +++ b/uv.lock @@ -115,6 +115,7 @@ dependencies = [ { name = "httpx" }, { name = "pydantic" }, { name = "python-dotenv" }, + { name = "pyyaml" }, { name = "typer" }, ] @@ -124,6 +125,7 @@ dev = [ { name = "pytest" }, { name = "pytest-cov" }, { name = "ruff" }, + { name = "types-pyyaml" }, ] [package.metadata] @@ -132,6 +134,7 @@ requires-dist = [ { name = "httpx", specifier = ">=0.28.1" }, { name = "pydantic", specifier = ">=2.13.4" }, { name = "python-dotenv", specifier = ">=1.2.2" }, + { name = "pyyaml", specifier = ">=6.0.2" }, { name = "typer", specifier = ">=0.26.8" }, ] @@ -141,6 +144,7 @@ dev = [ { name = "pytest", specifier = ">=9.1.1" }, { name = "pytest-cov", specifier = ">=7.0.0" }, { name = "ruff", specifier = ">=0.15.20" }, + { name = "types-pyyaml", specifier = ">=6.0.12" }, ] [[package]] @@ -663,6 +667,52 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" }, ] +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, +] + [[package]] name = "rich" version = "15.0.0" @@ -734,6 +784,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/80/87/b9fd69c92c6102a066e1b86a35243f53e70bd4c709f2a26d9f4fee4f4dc0/typer-0.26.8-py3-none-any.whl", hash = "sha256:3512ca79ac5c11113414b36e80281b872884477722440691c89d1112e321a49c", size = 122564, upload-time = "2026-06-26T09:22:44.72Z" }, ] +[[package]] +name = "types-pyyaml" +version = "6.0.12.20260518" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b8/83/4a1afc3fbfcf5b8d46fc390cd95ed6b0dc9010a265f4e9f46314efffa37a/types_pyyaml-6.0.12.20260518.tar.gz", hash = "sha256:d917f83fb38462550338c1297faedd860b3ec83912b96b1e3d73255f7473e466", size = 17850, upload-time = "2026-05-18T06:01:58.675Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/06/a2/c01db32be2ae7d6a1689972f3c492b149ee4e164b12fdfd9f64b50888215/types_pyyaml-6.0.12.20260518-py3-none-any.whl", hash = "sha256:d2150f75a231c9fe9c7463bd29487d93e60bac90400287351384bc2284eba7cd", size = 20312, upload-time = "2026-05-18T06:01:57.368Z" }, +] + [[package]] name = "typing-extensions" version = "4.16.0"