Skip to content

fix: what a real acceptance run found, and one thing it could not fix - #5

Merged
ttncode merged 6 commits into
mainfrom
fix/e2e-acceptance-findings
Sep 6, 2026
Merged

ttncode merged 6 commits into
mainfrom
fix/e2e-acceptance-findings

Conversation

@ttncode

@ttncode ttncode commented Sep 5, 2026

Copy link
Copy Markdown
Owner

What this changes

Five lanes ran the walkthrough end to end on four fresh private repositories — feature branch, pull request, green checks, merge, release, docker pull, scaffold add — plus one local lane for the two service driver cells no repository reaches. No admin bypass, no force-push, no disabled check. This is what they found.

A security defect. mise exec walks upward for config and trusts what it finds, persistently, and trusting means loading — which runs any _.source or {{ exec(...) }} in that config's [env]. Generating a project inside a tree you did not write executed someone else's mise.toml and permanently trusted it. Proven with a marker file created during a real scaffold new run. scaffold add had it too.

The guard for this already existed and was applied to one call site of four. The test for it also already existed, and its skip guard fired whenever the environment pre-trusts configs — which is exactly what CI=true does. Its only two outcomes were "skipped on a runner" and "failed everywhere else", so the property was never verified anywhere and the leak shipped.

laravel-inertia could never pass its own CI. Wayfinder's generated types and the Vite manifest are gitignored, and ci-unit ran check and test but never build. A clean checkout failed twice over. Every local run passed, because the pre-push checklist runs build and leaves both artifacts on disk. Its starter kit also brought a dependabot.yml into apps/app/.github/ that GitHub ignores and zizmor does not — a red security / zizmor on the first pull request of every such project.

The toolbox did not satisfy its own prettier config. .editorconfig sets quote_type = single for YAML and the shipped files used double quotes, so lefthook's pre-commit rewrote five committed files on a project's first commit — pnpm-lock.yaml among them. Separately, nestjs's .prettierignore omitted dist, which made checklist fail on its own build output from the second run onward: a pre-push hook that works exactly once.

Three smaller ones: lefthook.yml was collapsed to a single line of flow YAML by the fragment merge; every nextjs image reported unhealthy from boot because its HEALTHCHECK probed a route create-next-app does not generate; and the walkthrough was wrong in six places, each found by following it literally.

How it was verified

  • mise run lint — clean.
  • mise run test-runner — 184 ok, 1 not ok. The failure is ECONNRESET from registry.npmjs.org under four parallel bats lanes, not a regression; tests/compose.bats alone in the same environment is 13/13.
  • Each fix has a negative control. Re-introducing the double-quoted key brings [warn] pnpm-workspace.yaml back; deleting dist from the ignore file brings back Code style issues found in 7 files; the trust test is not ok without the ceiling and ok with it, including under CI=true.
  • laravel-inertia verified on a fresh git clone of a generated project, which is what CI actually checks out — 39/39 phpunit, phpstan clean, where the same clone without the fix gives TS2307 and 12 ViteManifestNotFound failures. It costs ~4s on a 52s checklist.
  • The prettier fix took three generate-and-run-the-hook rounds. The first two each looked fixed and were not: yq keeps the style of its first merge input, so the redis command had to become a block sequence in the base fragment, not the prod one. Only the third round left git status --porcelain empty.

What this does not fix

The released stack has never run. Measured after the run against the real images it had just published, following install.sh: neither application is told how to reach its database. Prisma says Environment variable not found: DATABASE_URL; Laravel does not even try, because config/database.php defaults to env('DB_CONNECTION', 'sqlite') and reads the DB_DATABASE=app it did get as a sqlite filename.

Three more, each independently fatal to serving traffic: nest binds 3000 while its healthcheck probes 3001 and compose publishes 8080; laravel's CMD is php-fpm on 9000 with no web server in the stack; and /var/www/{storage,bootstrap/cache} are root-owned while php-fpm runs as www-data, so the framework cannot boot.

That is ADR-0014's seam 2, missing. It needs a design — which environment contract each adapter family declares, where the composed value is assembled, and what serves HTTP in a laravel image — not a patch bolted onto this branch.

Related

ttncode/.github#1 fixes ci / changes failing on every push to main, which meant no config root's ci-unit had ever run on main in any generated project. Proven against a live repository: ci / changes ✓ in 5s and all four roots green, where the previous run on the same branch was failure. It reaches projects only when the v1 tag moves.

Checklist

  • mise run lint passes
  • mise run test-runner passes — 184/185, the one failure an ECONNRESET that passes on re-run
  • New behaviour has a test that fails without the change
  • Docs that describe changed behaviour were updated in the same commit
  • No unrelated changes

https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv

`mise exec` walks upward for config and trusts what it finds — persistently,
in ~/.local/state/mise — and trusting means loading, which runs any
`_.source` or `{{ exec(...) }}` in that config's [env]. So generating a
project inside a tree you did not write executed someone else's mise.toml and
permanently trusted it, with none of the asking mise exists to do. Measured:
an [env] entry in the parent created its marker file during a real
`scaffold new` run.

MISE_CEILING_PATHS stops the walk, and the toolbox already knew that — the
guard was written for one call site and never applied to its older sibling
two functions away, leaving resolve_minimum_release_age's frozen install,
sync_workspace_lockfile and every adapter generator unguarded. Export it once
per command instead of threading it through one function's third parameter:
this run makes mise subprocesses from four places, and a guard has to cover
all of them or it covers nothing. `scaffold add` had the same hole.

The test for this has existed since the guard did, and never ran. Its skip
guard fired whenever the environment pre-trusted configs, which is exactly
what CI=true does — so its only two outcomes were "skipped on a runner" and
"failed everywhere else", and the leak shipped. The suite owns its own
MISE_STATE_DIR now, so the precondition holds in both environments and the
skip is gone. That also stops every run writing an entry for a deleted
tmpdir into the developer's real trust store, which had grown past 7600.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
Two gitignored artifacts are generated by nothing ci-unit runs.
resources/js/{actions,routes} come from the vite build's wayfinder plugin
and are what `npm run types:check` compiles against; public/build/manifest.json
is the vite manifest, without which every test rendering an inertia page
returns 500. ci-unit ran check and test but never build, so a clean checkout
failed twice over — TS2307 on `@/routes`, then 12 ViteManifestNotFound
failures — while every local run passed, because the pre-push checklist runs
build and leaves both on disk.

`npm run build` moves into install, which is the order the starter kit's own
CI uses: `composer setup` ends with it, and only then does `composer ci:check`
run types:check and the tests. Verified on a fresh `git clone` of a generated
project, which is what CI actually checks out: 39/39 phpunit, phpstan clean.
It costs about 4 seconds on a 52-second checklist, because build then runs
the vite build a second time.

The kit also brings its own .github/ along into apps/app/. GitHub ignores
both files at that path, so they configure nothing — but zizmor audits every
workflow and dependabot file in the tree, and the kit's dependabot.yml sets
cooldown.default-days: 5, which failed the security gate with exit 13 on the
first pull request of every laravel-inertia project. Inert config whose only
effect was a red check.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
.editorconfig sets `quote_type = single` for yaml, prettier honours it, and
the toolbox shipped double quotes everywhere. A generated project therefore
failed its own format check before anyone touched it — apps/web/pnpm-workspace.yaml
in a standalone shape, where sync_standalone_build_policy copies the root
file into a directory a config root does check. Everywhere else it was
invisible, because no config root covers the repository root, and the only
thing that noticed was lefthook's pre-commit prettier, which rewrote five
committed files on the first commit anyone made.

pnpm-lock.yaml was among them, growing 2-4 KB, and CHANGELOG.md would have
been on every release — Release Please writes `*` bullets, prettier wants `-`.
Neither is written by hand, so neither is prettier's to format: a root
.prettierignore now covers both.

The redis command had to become a block sequence in the *base* fragment, not
the prod one: yq's merge keeps the style of its first input, and as a flow
sequence the assembled command ran past prettier's print width. Three
generate-and-run-the-hook rounds to find that; the first two each looked
fixed and were not.

Separately, nestjs's .prettierignore listed only pnpm-lock.yaml while
//apps/api:build writes dist/ and //apps/api:format is `prettier --check .`.
Within one checklist run format precedes build, so the first run was clean
and every run after it failed on its own output — which is a pre-push hook
that works exactly once. CI never saw it: a fresh checkout has no dist.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
yq propagates the style of the fragment it merges to the whole document, and
an adapter contributing no hook ships a comment plus `{}`. One flow mapping
collapsed all 26 lines of lefthook.yml onto a single line and replaced every
comment in it. The hooks still ran; the file a human has to read and review
did not survive. -P keeps it block style.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
Both Dockerfiles probed /api/health. create-next-app generates no such route,
so every image built from either reported unhealthy from first boot until
somebody noticed and wrote one. / is what the generated app serves; an app
that adds a real health endpoint should point this at it.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
Six things, each found by following the page literally on a fresh private
repository:

- config_roots is shown as a top-level key; the generated mise.toml nests it
  under [monorepo]. Reading the top level returns nothing, which gives a loop
  that runs no command and exits 0 — five separate runs reported a green step
  they never executed, from exactly that.
- step 6 names apps/api/.env and apps/web/.next as the local state to move
  aside. For laravel-inertia the state that matters is elsewhere, and
  following the page literally gives a green local run and a red CI, which is
  the failure the step exists to catch.
- `gh pr checks --watch` immediately after `gh pr create` exits 1 with "no
  checks reported": GitHub has not registered them yet and --watch does not
  wait. It reads as broken CI on every first pull request.
- the check count is not seven; it depends on how many config roots the
  commit touched.
- the release pull request's runs do not stay at `Action required`. They end
  as failure, explained by "This run likely failed because of a workflow file
  issue", which is not true and not actionable.
- step 11's promises about pnpm-workspace.yaml and lefthook.yml are both
  conditional, and a diff without them is not a finding.

Also corrects the mongodb driver's comment: measurement says authSource=admin
is the parameter that connection depends on, not directConnection, whose
stated reason could not be demonstrated.

Claude-Session: https://claude.ai/code/session_01J4HB8qJjdZwaAv42k6HMpv
@ttncode
ttncode merged commit 193b0bd into main Sep 6, 2026
14 checks passed
@ttncode
ttncode deleted the fix/e2e-acceptance-findings branch September 6, 2026 00:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants