Feature Summary
Redesign the annotation UI's formatting for human annotators. Three areas:
- Parse notification timestamps from content for correct chronological ordering
- Replace regex-based observation parsing with eval-based typed object access, format user actions as function calls, filter faulty pairs, expand notification coverage, and sort messages by timestamp
- Fix UI timestamp timezone (local to UTC) and progress counter bug
Motivation and Use Case
Human annotators currently see:
- Raw LLM internals (Thought sections, ReAct format) in user action messages
- Fragile regex-parsed dataclass reprs that break on nested quotes and new fields
- Hex IDs in notifications instead of human-readable names
- Missing notification types (shopping, cab, apartment, notes not handled)
- Proposals appearing before notifications due to missing notification timestamps
- Timestamps displayed in local timezone instead of UTC (showing 01:00 instead of 09:00)
- Progress counter showing 1/19 after annotating 1 of 20 samples
This degrades annotation quality and annotator experience.
Proposed Solution
Notification timestamps: Add _parse_notification_timestamp() to trace parser. Parse last non-None [YYYY-MM-DD HH:MM:SS] from notification content and assign as message timestamp.
Formatter redesign: Format user actions as tool_name(key='value', ...) stripping Thought sections. Filter faulty user_action + error observation pairs. Expand _SAFE_EVAL_NAMESPACE with all @user_tool return types. Replace regex observation parsing with eval() + attribute access. Expand notification formatter with template-prefix matching for all notification types. Sort messages by timestamp in to_api_response().
UI fixes: Fix formatTimestamp() to use getUTCHours/Minutes/Seconds. Fix get_progress() to return fixed total count.
Alternatives Considered
- Keeping regex-based parsing: rejected because it breaks on nested quotes, missing fields, and requires per-type regex maintenance
- Fixing ordering at display time only: rejected because the root cause is missing timestamps on notification messages
- Using world log timestamps for notifications: rejected because those represent pre-step injection time, not actual event time
Component
Other
Priority
High - Would significantly improve my workflow
Pre-submission Checklist
Feature Summary
Redesign the annotation UI's formatting for human annotators. Three areas:
Motivation and Use Case
Human annotators currently see:
This degrades annotation quality and annotator experience.
Proposed Solution
Notification timestamps: Add
_parse_notification_timestamp()to trace parser. Parse last non-None[YYYY-MM-DD HH:MM:SS]from notification content and assign as message timestamp.Formatter redesign: Format user actions as
tool_name(key='value', ...)stripping Thought sections. Filter faulty user_action + error observation pairs. Expand_SAFE_EVAL_NAMESPACEwith all@user_toolreturn types. Replace regex observation parsing witheval()+ attribute access. Expand notification formatter with template-prefix matching for all notification types. Sort messages by timestamp into_api_response().UI fixes: Fix
formatTimestamp()to usegetUTCHours/Minutes/Seconds. Fixget_progress()to return fixed total count.Alternatives Considered
Component
Other
Priority
High - Would significantly improve my workflow
Pre-submission Checklist