Skip to content

feat: make orchestrator/sub-agent sampling temperature configurable - #8

Open
Captain-Tripps wants to merge 1 commit into
kyuz0:mainfrom
Captain-Tripps:feat/configurable-orchestrator-temperature
Open

Captain-Tripps wants to merge 1 commit into
kyuz0:mainfrom
Captain-Tripps:feat/configurable-orchestrator-temperature

Conversation

@Captain-Tripps

Copy link
Copy Markdown

Summary

  • _get_default_options() hardcoded temperature=0.0 (fully greedy decoding) with no way to change it short of editing engine code directly.
  • Greedy decoding is a reasonable default for tool-call schema correctness, but it has no randomness to escape a locally-greedy dead end — in practice this can show up as a model announcing an action in prose and then stopping instead of actually emitting the follow-up tool call, since there's nothing to perturb it off whatever single trajectory looked best token-by-token. This is more likely to bite smaller, quantized, or heavily fine-tuned local models than large hosted ones.
  • Adds settings.temperature to config.yaml (default 0.0, matching prior behavior exactly — no behavior change for existing configs that don't set it), documented in config.py's _DEFAULTS and config_template.yaml. Lets users trade a little determinism for better tool-call follow-through on a per-project basis without touching engine code.

Test plan

  • Verified _get_default_options() reads config.cfg["settings"]["temperature"] with a default of 0.0.
  • Verified existing configs without settings.temperature set behave identically to before (temperature stays 0.0).

_get_default_options() hardcoded temperature=0.0 (fully greedy decoding) with
no way to change it short of editing engine code directly. Greedy decoding is
a reasonable default for tool-call schema correctness, but it has no
randomness to escape a locally-greedy dead end - in practice this can show up
as a model announcing an action in prose and then stopping instead of
actually emitting the follow-up tool call, since there's nothing to perturb
it off whatever single trajectory looked best token-by-token. This is more
likely to bite smaller, quantized, or heavily fine-tuned local models than
large hosted ones.

Adds settings.temperature to config.yaml (default 0.0, matching prior
behavior exactly - no behavior change for existing configs that don't set
it), documented in config.py's _DEFAULTS and config_template.yaml. Lets users
trade a little determinism for better tool-call follow-through on a
per-project basis without touching engine code.
@Captain-Tripps

Copy link
Copy Markdown
Author

Love your videos and work here. I'm enjoying this local agent builder! Hoping this feature might help others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant