refactor(runner): abortOnFailure is input-only + scenarios cleanup + input-schema flatten - #21
Open
DaveHanns wants to merge 1 commit into
Conversation
DaveHanns
force-pushed
the
runner-ignore-yaml-abort-flatten-input-schema
branch
from
July 2, 2026 23:00
6aa65ac to
bbb14e3
Compare
… + thematic reorder
Three coordinated cleanups to make the runner input the single source of
truth for the abortOnFailure knob, and simplify the input form the Apify
Console renders.
1. main.ts stops reading meta.abortOnFailure. The resolution collapses to
a single line:
const abortOnFailure = input.abortOnFailure ?? false;
Any `abortOnFailure:` key in a scenario's YAML frontmatter is now
ignored (not prohibited — backward compat — but not consulted). The
source-labeling variable and override-detection log block go away
because there is only one source now.
2. parseScenario stops copying abortOnFailure from YAML into ScenarioMeta.
The field is dropped from the type. Silently discarding rather than
erroring keeps existing scenarios that still have the YAML line
working (they just have no effect).
3. input_schema.json: all sectionCaption groupings removed (the Console
form used to render "Authentication" and "Behavior" headers; those
groupings didn't earn their visual weight). Fields reordered
thematically:
- agent, model (who runs)
- scenario, systemPrompt (what to run)
- maxBudgetUsd, maxRetries, maxTurns, abortOnFailure (runtime knobs)
- envVariables, preAuthenticate (auth)
- initPreset, initBashScript, mcpConfigJson (tool setup)
abortOnFailure now has an explicit `default: false` so the Console
form pre-fills.
Companion PR on the eval side: apify/agentic-actor-dev-eval#11 removes
`abortOnFailure: false` from every scenario's YAML frontmatter and
build-inputs.sh sets it explicitly on the runner input.
Also fixes two doc misses the exhaustive sweep found:
- Root .actor/input_schema.json prefill (was still teaching `abortOnFailure: false`
in the scenario YAML example, contradicting the new input-only semantic).
- CLAUDE.md scenario-format YAML example (same fix).
Scenarios cleanup (input-only migration follow-through):
- Strip `abortOnFailure:` YAML line from 22 scenarios under scenarios/*.md (parser silently
accepts + discards it now, so the YAML key was inert but misleading).
- Delete scenarios/us5-abort-on-failure.md — this scenario's name, description, and test
structure were entirely predicated on abortOnFailure=true cascading. With the flag now
input-only + defaulting to false, the scenario has no meaningful behavior distinct from
any other. Its cross-references (hardcoded TestDef in actors/runner/test/e2e/run-e2e.ts,
historical log bullet in docs/04-implementation-log.md, static row in docs/reference.html)
were updated in the same commit.
- Edit scenarios/us5-multi-step.md description to drop " and abortOnFailure" — the two
tests remain independent + meaningful; only the descriptive text needed a trim.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DaveHanns
force-pushed
the
runner-ignore-yaml-abort-flatten-input-schema
branch
from
July 2, 2026 23:45
bbb14e3 to
1cde7de
Compare
DaveHanns
changed the title
DaveHanns
added a commit
that referenced
this pull request
Jul 3, 2026
With initPreset now echoing "all" for all-stack runs (companion PRs apify/agentic-actor-dev-eval#11 + #21), the value IS the stack identifier — surface it as the first column of the Eval Results view so all vs none rows are visually distinguishable without opening the raw JSON. Label is "Stack" (not "Preset") because that's the concept users think in — the underlying schema field name stays initPreset to avoid a schema-wide rename. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the runner input the single source of truth for
abortOnFailure. Rolls the scenarios cleanup in here (originally planned as a separate PR) so everything can be tested from one branch.Changes
Runner code
actors/runner/src/main.ts: collapses abortOnFailure resolution toinput.abortOnFailure ?? false. Drops themeta.abortOnFailurefallback, the source-labeling variable, and the override-detection log block.shared/src/scenario-parser.ts: stops copyingabortOnFailurefrom YAML into ScenarioMeta. YAML key is now silently discarded (backward compat, no error).shared/src/types.ts: dropsabortOnFailure: booleanfrom ScenarioMeta.expect(meta.abortOnFailure)...assertions).Input schema
actors/runner/.actor/input_schema.json: adds top-levelabortOnFailurefield withdefault: false,nullable: true, no sectionCaption grouping. Field ordering thematic (agent, model, scenario, systemPrompt, maxBudgetUsd, maxRetries, maxTurns, abortOnFailure, envVariables, preAuthenticate, initPreset, initBashScript, mcpConfigJson)..actor/input_schema.json: prefill string cleaned (was still teachingabortOnFailure: falsein scenario YAML example).Scenarios cleanup (this amend)
abortOnFailure:YAML line from 22 scenarios underscenarios/*.md. Now the parser silently discards, keeping the key was inert but misleading.scenarios/us5-abort-on-failure.md— solely built to demonstrate the flag; its name, description, and test structure no longer make sense once the flag is input-only. Removed the matching hardcoded TestDef block inactors/runner/test/e2e/run-e2e.ts(which would have thrown ENOENT at module-load); pruned the stale row indocs/reference.htmland log bullet indocs/04-implementation-log.md.scenarios/us5-multi-step.mddescription to drop the trailing "and abortOnFailure" phrase (the two tests remain independent + meaningful).Docs
actors/runner/README.md: YAML-frontmatter section describes input-only semantic.docs/reference.html: abortOnFailure YAML row removed from frontmatter table.CLAUDE.md: scenario-format YAML example no longer showsabortOnFailure; added an out-of-fence note pointing to the top-level input.Companion PR
apify/agentic-actor-dev-eval#11 — eval harness: strips the YAML key from 14 eval scenarios;
build-inputs.shemitsabortOnFailurevia--argjsonmirroring thepreAuthenticatepattern (per-stack helperabort_for_stack()returning "false" for every current stack; extensible if a stack needs to opt in).Test plan
grep -l "abortOnFailure" scenarios/*.md→ empty (all 22 clean; us5-abort-on-failure.md gone)npm test --workspace sharedpasses (244/244)npm run build --workspacesclean