[docs] - size Ollama num_ctx for the agentic real-repo coding loop, not just RAG - #805
Conversation
… just RAG The existing num_ctx guidance (OLLAMA_SETUP.md, setup-guide.md) derives its 10,000-12,288 recommendation solely from the /query RAG path's budget (max_context_tokens + local_llm.max_tokens). agentic/real_repo_loop.py's real-repo-run/-plan pathway drives the same Ollama instance with a substantially larger, differently-shaped prompt (declared plan, read_paths file contents, verification feedback, GitHub context) -- summing the loop's own documented per-component caps totals roughly 9,750-10,000 input tokens for a single iteration alone, which can already approach the window sized for the smaller RAG formula. Add a subsection covering this math, a recommended larger OLLAMA_CONTEXT_LENGTH, and an Apple-Silicon-specific unified-memory tradeoff note (repo currently has zero guidance on either). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FW9QmnLBGyPu8QvJQ897hf
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3212e3cf79
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Neither proposer client (`agentic/harness_optimizer/model_adapter.py`'s | ||
| `LocalProposerClient`, used by default, or | ||
| `agentic/deepagent_github/chat_client.py`'s `ChatModelProposerClient`, used | ||
| with `--provider`) sends `num_ctx` in its own request — exactly like the RAG | ||
| path, this is 100% an out-of-band, operator-set Ollama setting, and neither |
There was a problem hiding this comment.
When --provider is used, agentic/cli.py constructs ChatModelProposerClient, which calls a LangChain cloud chat model via build_chat_model, not the local Ollama endpoint. Including that client in the statement that the setting is an out-of-band Ollama num_ctx knob misleads Grok/Claude users into tuning an Ollama context window that cannot affect those calls, while leaving the real provider-side context limit unaddressed.
Useful? React with 👍 / 👎.
1 participant
Proposed changes
CyClaw's existing Ollama
num_ctxguidance (docs/! How-To-Guides/OLLAMA_SETUP.md,setup-guide.md) derives its 10,000–12,288 recommendation solely from the/queryRAG path's budget (retrieval.max_context_tokens+models.local_llm.max_tokens).agentic/real_repo_loop.py'sreal-repo-run/real-repo-run-planpathway (alsoharness's/api/agent/run) drives the same Ollama instance with a substantially larger, differently-shaped prompt — a declared plan,read_pathsfile contents, verification feedback, quoted GitHub context — and the existing docs say nothing about it.Summing that pathway's own documented per-component caps (
_MAX_PLAN_CHARS=6,000,_MAX_TOTAL_READ_CHARS=12,000,_MAX_FEEDBACK_TOTAL_CHARS>=4,000, GitHub context cap inagentic/cli.py=8,000, fixed system prompt ~900, harness instruction cap=8,192) totals roughly 39,000–40,000 characters of input alone for a single iteration — about 9,750–10,000 tokens at this repo's own ~4-chars/token convention. That can by itself approach or exceed the 10,000–12,288 window the docs currently recommend, which was sized only for the smaller RAG formula, before any output-token budget is even reserved.Adds:
OLLAMA_SETUP.md's existing "Ollama Context Size (Advanced)" section covering this arithmetic, a recommended largerOLLAMA_CONTEXT_LENGTH, confirmation that neither proposer client (LocalProposerClient/ChatModelProposerClient) sendsnum_ctxitself (same as the RAG path — it's 100% operator-set), and an Apple-Silicon-specific unified-memory tradeoff note (the repo currently has zero mentions of "unified memory" or a RAM figure anywhere).setup-guide.md, matching its existing "Full detail... OLLAMA_SETUP.md" pointer style.Invariant / Governance Impact: None. Pure documentation; no code, config, or graph topology touched.
Types of changes
Scope note: Docs only.
Benefits / why
This pathway is measurably more likely to hit the documented "0% processing" stall than
/queryever is, with zero repo guidance pointing that out — an operator (e.g. running the shippedqwen3.6:27bon Apple Silicon) who correctly sizednum_ctxper the existing RAG-path recommendation could still stall on a real-repo-run that uses a plan file plus a fewread_paths. This closes that gap before it causes a confusing "CyClaw hangs" report.Risks to monitor
None expected — additive documentation only. The specific numbers cited (39–40k chars / ~10k tokens worst case) are my own arithmetic over the loop's own stated constants, clearly labeled as such in the text rather than presented as a repo-declared guarantee; if any of those per-component caps changes in the future, this section's math would need a refresh (flagging for future doc-sync awareness, though today's
doc_sync.pyhas no automated check over free-text arithmetic like this).Checklist
doc_sync.py(0 drift) +invariant-guard(33/0) both clean; not a Python change so no pytest run required.Generated by Claude Code