Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fold-at-admission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'cargo-hauler': patch
---

A lane head now folds compatible queued requests when it wins its admission permit instead of when the lane first takes it. Under a saturated permit pool the head could wait half an hour or more for a permit while a sibling `cargo test -p other --lib -- filter` or `cargo check -p other` submitted a second later sat behind it, then waited out a permit of its own; those requests now ride the head's composite run. The fold rules themselves are unchanged.
5 changes: 5 additions & 0 deletions .changeset/fold-lockfile-assertions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'cargo-hauler': patch
---

Batch folding now accepts `--locked`, `--frozen`, and `--offline` when every participant passes the same ones. These flags only assert things about the lockfile and network access for the whole invocation, but until now any of them kept a request out of every compile batch and test composite. In the last day a quarter of brokered `build`/`check`/`clippy`/`test` leaders carried `--locked`. A one-sided flag still refuses the fold, because the composite runs with the leader's flags.
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ Within a lane, the daemon can reduce work in three ways:
check.
3. **Batch folding.** The daemon combines compatible queued compile or test
requests into one invocation.
The lane head folds when it wins its admission permit, so requests that
join the lane while it waits for one still ride it.

A leading `env NAME=value … cargo …` folds into the request environment, so
the daemon schedules, estimates, attaches, and phase-tracks the cargo behind
Expand Down Expand Up @@ -270,7 +272,9 @@ Requests that name the same packages share no compile and still need the same
filters. Unfiltered runs fold only with unfiltered runs. `--skip`,
`--ignored`, `--include-ignored`, `--list`, `--format`, `--logfile`, or any
other harness flag keeps a run out of composites. `cargo nextest run`
requests fold only on an identical filterset. On success, every participant
requests fold only on an identical filterset. Of the cargo flags the daemon
does not model, only `--locked`, `--frozen`, and `--offline` fold, and only
when every participant passes the same ones. On success, every participant
shares the exit. When the composite fails, a participant inherits that
failure only if it named every package and every filter the composite ran.
Otherwise the failing tests may belong to another participant's package or
Expand Down
2 changes: 1 addition & 1 deletion artifact/agent-bundle.compile-evidence.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion artifact/agent-bundle.manifest.json

Large diffs are not rendered by default.

31 changes: 23 additions & 8 deletions artifact/bin/cargo-hauler-flight.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -22043,12 +22043,26 @@ __webpack_require__.d(__webpack_exports__, {
'test'
]);
/** Upper bound on packages merged into one composite invocation. */ const maxBatchPackages = 16;
/**
* Unmodeled cargo flags that assert something about the whole invocation's
* lockfile or network access and never select or shape a package's units.
* The composite carries the leader's, so participants must make the same
* assertions (`sameInvocationAssertions`).
*/ const invocationAssertions = new Set([
'--frozen',
'--locked',
'--offline'
]);
const sameStringSet = (left, right)=>left.every((value)=>right.includes(value)) && right.every((value)=>left.includes(value));
const invocationAssertionsOf = (intent)=>intent.opaqueArguments.filter((argument)=>invocationAssertions.has(argument));
/** Unmodeled flags are foldable when each is an invocation assertion or in `extra`. */ const onlyFoldableOpaque = (intent, extra = new Set())=>intent.opaqueArguments.every((argument)=>invocationAssertions.has(argument) || extra.has(argument));
const sameInvocationAssertions = (leader, candidate)=>sameStringSet(invocationAssertionsOf(leader), invocationAssertionsOf(candidate));
/**
* Whether an intent has the explicit-package shape composable into a batch.
* A `--` trailer (`cargo clippy … -- -D warnings`) is allowed: the composite
* keeps the leader's trailer once, so `batchCompatible` admits only
* followers whose trailer is byte-equal (#86).
*/ const batchLeaderEligible = (intent)=>batchableSubcommands.has(intent.subcommand) && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && intent.opaqueArguments.length === 0;
*/ const batchLeaderEligible = (intent)=>batchableSubcommands.has(intent.subcommand) && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && onlyFoldableOpaque(intent);
/**
* Whether `candidate` can be folded into a composite invocation led by
* `leader`: same batchable subcommand, identical compile surface, target
Expand All @@ -22058,7 +22072,7 @@ __webpack_require__.d(__webpack_exports__, {
* `-- -D warnings` another participant's warnings fail the composite; the
* demux still proves a follower whose own units compiled cleanly, and the
* rest requeue to run alone.
*/ const batchCompatible = (leader, candidate)=>leader.subcommand === candidate.subcommand && batchLeaderEligible(leader) && batchLeaderEligible(candidate) && sameCompileSurface(leader, candidate) && stringArraysEqual(leader.targets, candidate.targets) && stringArraysEqual(leader.passthrough, candidate.passthrough);
*/ const batchCompatible = (leader, candidate)=>leader.subcommand === candidate.subcommand && batchLeaderEligible(leader) && batchLeaderEligible(candidate) && sameCompileSurface(leader, candidate) && sameInvocationAssertions(leader, candidate) && stringArraysEqual(leader.targets, candidate.targets) && stringArraysEqual(leader.passthrough, candidate.passthrough);
/** Packages on `candidate` that the leader invocation does not already name. */ const extraPackagesFor = (leader, candidate)=>candidate.packages.filter((name)=>!leader.packages.includes(name));
/**
* Where the leader's argv stops taking cargo flags: the earlier of the demux
Expand Down Expand Up @@ -22093,7 +22107,9 @@ __webpack_require__.d(__webpack_exports__, {
...argv.slice(insertAt)
];
};
/** The composite always re-adds `--no-fail-fast`, so it is a benign opaque. */ const onlyNoFailFast = (opaque)=>opaque.every((argument)=>argument === '--no-fail-fast');
/** The composite always re-adds `--no-fail-fast`, so it is a benign opaque. */ const testRunOpaque = new Set([
'--no-fail-fast'
]);
/**
* Walks a `cargo test` trailer one libtest argument at a time — a bare name
* filter, or a foldable harness flag — or returns null at the first argument
Expand Down Expand Up @@ -22243,14 +22259,14 @@ const integrationTestTargetPrefix = 'test:';
* target narrowing expressible as `--test` / `--lib` flags, a trailer of
* bare name filters and foldable harness flags only (`classifyTestTrailer`),
* and no unmodeled cargo flags.
*/ const testBatchEligible = (intent)=>intent.subcommand === 'test' && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && onlyNoFailFast(intent.opaqueArguments) && foldableTestTargets(intent.targets) && classifyTestTrailer(intent.passthrough) !== null;
*/ const testBatchEligible = (intent)=>intent.subcommand === 'test' && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && onlyFoldableOpaque(intent, testRunOpaque) && foldableTestTargets(intent.targets) && classifyTestTrailer(intent.passthrough) !== null;
/**
* Whether a `cargo nextest run` intent can fold: explicit packages keep the
* composite's build scope tight (an -E-only participant would need a
* workspace-wide build to be a superset), and the whole selection must be
* expressible as one filterset — positional filters and trailing arguments
* intersect with `-E` in nextest, so their presence disqualifies folding.
*/ const nextestBatchEligible = (intent)=>intent.subcommand === 'nextest' && intent.nextestCommand === 'run' && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && onlyNoFailFast(intent.opaqueArguments) && intent.targets.length === 0 && intent.testFilters.length === 0 && intent.passthrough.length === 0;
*/ const nextestBatchEligible = (intent)=>intent.subcommand === 'nextest' && intent.nextestCommand === 'run' && !intent.workspace && intent.packages.length > 0 && intent.excludes.length === 0 && onlyFoldableOpaque(intent, testRunOpaque) && intent.targets.length === 0 && intent.testFilters.length === 0 && intent.passthrough.length === 0;
/** How (if at all) this intent can lead or join a composite invocation. */ const batchKindFor = (intent)=>{
if (batchLeaderEligible(intent)) {
return 'compile';
Expand All @@ -22263,7 +22279,6 @@ const integrationTestTargetPrefix = 'test:';
}
return null;
};
const sameStringSet = (left, right)=>left.every((value)=>right.includes(value)) && right.every((value)=>left.includes(value));
/**
* Whether two `cargo test` selections can share one composite (#87). The
* `--test` / `--lib` target set and the harness flags must match exactly:
Expand Down Expand Up @@ -22314,9 +22329,9 @@ const sameStringSet = (left, right)=>left.every((value)=>right.includes(value))
case 'compile':
return batchCompatible(leader, candidate);
case 'test':
return testBatchEligible(candidate) && sameCompileSurface(leader, candidate) && testSelectionsFold(leader, candidate);
return testBatchEligible(candidate) && sameCompileSurface(leader, candidate) && sameInvocationAssertions(leader, candidate) && testSelectionsFold(leader, candidate);
case 'nextest':
return nextestBatchEligible(candidate) && sameCompileSurface(leader, candidate) && stringArraysEqual(leader.filterExpressions, candidate.filterExpressions);
return nextestBatchEligible(candidate) && sameCompileSurface(leader, candidate) && sameInvocationAssertions(leader, candidate) && stringArraysEqual(leader.filterExpressions, candidate.filterExpressions);
default:
{
const exhaustive = kind;
Expand Down
Loading
Loading