From c4550a131e1ffb4080c31c1e92268723630a284c Mon Sep 17 00:00:00 2001 From: MicBur <164754856+MicBur@users.noreply.github.com> Date: Tue, 4 Aug 2026 09:06:47 +0200 Subject: [PATCH] =?UTF-8?q?release:=20version=200.13.0=20=E2=80=94=20remot?= =?UTF-8?q?e=20preprocessing=20on=20by=20default,=20worker=20sandboxing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Headline: the client no longer runs -E for eligible TUs (#42). It runs the cheap -MM scan, ships raw source + a project-header bundle, and the worker preprocesses. -42% client CPU on Linux, 3.19x wall-clock on a real 101-TU Windows build. On by default after byte-identity was verified on both platforms (240/240 vs native on Linux; Windows PE/COFF cross-compile identical bar path-string normalisation, with the machine code byte-identical). Time-macro, C++20-module and MSVC TUs fall back automatically; SUCO_REMOTE_PREPROCESS=0 restores the classic path and CI pins both. Also in: opt-in worker compile sandboxing with per-job isolation (#43), a VS Code extension, and the fixes that made a Windows worker functional at all (#100), closed a Windows-specific path-escape hole in the bundle guard (#88), corrected the V3 cross-compile compiler resolution (#66) and the PPA version read (#49). Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 15 +++++++++++++++ CMakeLists.txt | 2 +- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 16a3a89..57b2bd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,21 @@ All notable changes to the SUCO distributed compilation system will be documented in this file. +## [0.13.0] - 2026-08-04 + +### Added +- **Remote preprocessing — the client no longer runs `-E` (#42).** Instead of expanding a translation unit locally and shipping 1–3 MB of preprocessed text, the client now runs the compiler's cheap `-MM` dependency scan, packs the project headers into a content-addressed bundle, and sends the **raw** source plus that bundle; the *worker* preprocesses. Measured: **−42 % client CPU** on a header-heavy Linux corpus and **3.19× wall-clock** on a real 101-TU Windows build (20.78 s → 6.52 s). The win scales with header weight — heavy templates and umbrella headers benefit most, trivial TUs barely at all. + + **This is on by default.** Byte-identity was the gate and was verified on both platforms before flipping it: 240/240 objects byte-identical to a native compile across `-O0/-O2/-O3` on Linux; on a real Windows→Linux PE/COFF cross-compile, 13/15 identical with the two exceptions fully explained (`.text` machine code byte-identical, sizes equal, identical to the normal grid object, 100 % deterministic across runs — the delta is path-string normalisation, the same normalisation the grid already applied). A 101-TU Windows project builds, links and runs. Translation units that cannot be preprocessed remotely fall back to the old path automatically: anything using `__DATE__`/`__TIME__`/`__TIMESTAMP__` (in the source *or* in a project header — the worker's clock would differ), C++20 modules, and MSVC. Both paths are pinned in CI, so the classic path stays exercised. Escape hatch: `SUCO_REMOTE_PREPROCESS=0`. +- **The worker can sandbox the compiler itself (#43),** opt-in via `SUCO_SANDBOX=1`. A compile is untrusted execution, so it now runs with a read-only filesystem, no network, and **only its own job directory writable** — the object and the compiler's own intermediates (`TMPDIR`) live inside that directory, so two concurrent jobs on a worker cannot see or clobber each other's scratch. Prefers `bwrap` when installed, falls back to `unshare(1)`; both were verified to produce byte-identical objects, because namespaces do not change compiler output. Off by default, and a no-op when off. +- **A VS Code extension** (`extension/vscode/`) showing grid status in the status bar (workers online, cache-hit rate, active jobs), a one-click toggle that injects `CMAKE_CXX_COMPILER_LAUNCHER=suco-cl++` into CMake Tools, and a shortcut to the dashboard. No runtime dependencies. The coordinator host defaults to localhost and otherwise follows `SUCO_COORDINATOR_HOST`, so an already-configured machine needs no setup. A Visual Studio 2022 counterpart is scaffolded (`extension/visualstudio/`) but has **not** yet been loaded in a live VS instance. + +### Fixed +- **A Windows worker could not run any dispatched job at all.** Three POSIX assumptions killed it before it ever compiled: the toolchain was unpacked with `tar -I zstd`, which is GNU tar syntax that the **bsdtar** shipped as `tar.exe` on Windows rejects outright; the received toolchain archive was written to a hardcoded `/tmp/…` that does not exist there; and the header-cache directory fell back to `/tmp/.cache/…` whenever `HOME` was unset — the normal case on Windows. All three are now platform-correct (bsdtar auto-detects zstd, so plain `-xf` is right; `std::filesystem::temp_directory_path()`; `LOCALAPPDATA`/`USERPROFILE`). Found by actually running a Windows worker and reporting that the log never showed a job. +- **A path-escape hole in the header-bundle unpack guard, specific to Windows.** The guard rejected absolute paths and `..`, which is not enough: `"C:foo.h"` is drive-*relative*, so `is_absolute()` is false, yet `dest / "C:foo.h"` resolves to `"C:foo.h"` — `std::filesystem` discards the destination when the right-hand side carries a different root name — writing outside the job directory. UNC paths and root-directory paths escape the same way. Bundle entries are now validated on the raw string, so a Linux worker and a Windows worker accept or reject the same bundle identically. +- **Remote-preprocessed cross-compiles used the wrong compiler.** The new dispatch path took the *local* compiler name rather than the resolved target-qualified one, so a MinGW target would have been built with the host `g++`. It now mirrors the existing dispatch path's compiler resolution. +- **`make_ppa_source.sh` read the wrong version** — it matched the first `VERSION x.y.z` in `CMakeLists.txt`, which is `cmake_minimum_required(VERSION 3.15)`, so PPA source packages were versioned `3.15` instead of the project version. + ## [0.12.0] - 2026-07-26 ### Added diff --git a/CMakeLists.txt b/CMakeLists.txt index 6be1881..7c8e7e5 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,5 +1,5 @@ cmake_minimum_required(VERSION 3.15) -project(suco VERSION 0.12.0 LANGUAGES C CXX) +project(suco VERSION 0.13.0 LANGUAGES C CXX) set(CMAKE_CXX_STANDARD 20) set(CMAKE_CXX_STANDARD_REQUIRED ON)