Skip to content

feat(agent): ask for the command after a reply without one (plan #11) - #22

Merged
canyang25 merged 6 commits into
ruoshi/data-full-run-1from
ruoshi/feat-act-nudge
Oct 9, 2026
Merged

canyang25 merged 6 commits into
ruoshi/data-full-run-1from
ruoshi/feat-act-nudge

Conversation

@rzhang569

Copy link
Copy Markdown
Collaborator

Stacked on #21 (retargets to main once #21 merges).

When a reply ends without a command and wasn't cut off ("Let's start by exploring the environment."), the agent now asks for the command itself and samples the next reply at temperature 0.7. Cut-off replies and the give-up rule are unchanged.

On the 10-task slice, 3 trials:

  • give-ups on short replies: 0 (3 of 7 in the run before); the next reply has a command 82% of the time (65% before)
  • passes 14/30 vs 12/30 baseline and 16/30 in the 16k run: within noise, so this is merged for the failure it removes, not for the pass count

Also records the 16k run (plan #12): halves cut-off give-ups, doesn't fix thinking loops; to be rerun on top of this. Details in docs/experiments.md (2026-10-09).

rzhang569 and others added 6 commits October 8, 2026 21:40
… none

A reply that ends without a command and wasn't cut off now gets a nudge
that asks for the command itself, and the next request runs at 0.7. At
0.2 the model repeated the same one-line preamble until the agent gave
up. Cut-off replies and the give-up rule are unchanged.
ruoshi/feat-act-nudge at 31e2499 on the 10-task dev slice, LLM_MAX_TOKENS=8192,
concurrency 2. Exported with scripts/export_trials.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@canyang25
canyang25 merged commit 7b9315a into ruoshi/data-full-run-1 Oct 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants