Bone stock, 5328 tokens to say "hi"

[ohcedar]

Does this seem right? I thought the whole point was to be super lean, this seems pretty big for an agent called "little-coder". Or am I doing something wrong (totally possible!)?

[bdutta]

Okay 5328 tokens do look a bit excessive, but I'm coming from spending about 2 weeks with pi and having looked a bit at the completions API invocations (to troubleshoot some provider/model combinations that weren't working as I expected them). Even with pi without any extentions a simple 'What is the meaning of life' (rather silly thing to ask pi because this is not a chat-harness), I was hitting around 6K tokens as per groq. Now I could never figure out what exactly groq was seeing (I didn't have the patience to setup a MITM proxy and do full packet capture there), but I did use the pi-llm-debugging extension to dump the API call as seen by pi, and there I could see that it had conflated my prompt to about 1800 tokens. It is because what is there is not just my prompt or system prompt but more. A fair comparison might be to see, what is the token count comparison with other coding harnesses that are not as lean as pi, say opencode.

[omni-front]

yeah, 5328 tokens does sound a bit high for a lightweight agent like little-coder. the key thing to check is whether you have any extra extensions or skill markdown files loaded that aren't necessary for your simple tasks. each one can add more tokens to your session tree. also, make sure the system prompt isn't bloated, as it typically expects around 1000 tokens. tweaking these might help trim down the token count.

[ohcedar]

I appreciate those ideas for slimming it down, but given mine is stock and a fresh install, this is the “out of the box “ experience with no custom prompts, skills, nothing, which does seem a bit token heavy.

[bdutta]

If I am not mistaken, little-coder out-of-the-box adds skills that bare-bones pi doesn't. little-coder goes beyond pi by building the small-model aware scaffolding, which AFAIU is codified as one or more skills.

Here's what README.md says:

pi is the minimal substrate — agent loop, multi-provider API, TUI, session tree, compaction, extension model. Four built-in tools (read / write / edit / bash) and a ~1000-token system prompt.

little-coder is pi + 20 extensions + 30 skill markdown files + a Python benchmark harness.