[term-challenge SDK] Add reset_stats() method to LLM class for per-task cost tracking #34
Description
Project
cortex
Description
LLM statistics (total_tokens, total_cost, request_count) accumulate across the lifetime of the LLM instance.
No way to reset stats between tasks for accurate per-task tracking.
Use Case
- Track cost per individual task in benchmarks
- Monitor token usage per task for optimization
- Generate accurate per-task reports
Proposed Solution
Add reset_stats() method:
def reset_stats(self):
self.total_tokens = 0
self.total_cost = 0.0
self.request_count = 0
self.stats = {}
Usage:
llm.reset_stats() # Call between tasks
Alternatives Considered
- Create new LLM instance per task - expensive, loses connection pooling
- Track stats externally - duplicates SDK functionality
- Auto-reset on /start - may break agents relying on cumulative stats
Impact
Most users
Additional Context
No response