Skip to content

Chat deadline timeouts with partial responses (ChatDeadlineExpiredPartial) #82

Description

@krishagel

Summary

Agents are hitting ChatDeadlineExpiredPartial errors — the chat deadline expires with a partial response (last_state=aborted). Two occurrences in the last 4 hours, both on the same session. The agent processes a large number of internal events (72-163 agent events) before the deadline fires, suggesting the agent is running out of time while still processing.

Definition of Done

  • Root cause identified: determine whether the deadline is too short, the agent is doing too much work per turn, or there is a performance bottleneck
  • Event counts and timing analyzed to find the bottleneck
  • Proposed fix documented (e.g., increase chat deadline, optimize agent event processing, or add progress checkpoints)
  • (manual) Fix applied and verified in staging

Affected failures

ID Occurred At (UTC) User Source Error
255 2026-07-29 23:11:51 hagelk@psd401.net harness ChatDeadlineExpiredPartial — chat deadline expired with partial response (last_state=aborted)
251 2026-07-29 22:34:00 hagelk@psd401.net harness ChatDeadlineExpiredPartial — chat deadline expired with partial response (last_state=aborted)

Affected users

Error details

ID 255:

error_class: ChatDeadlineExpiredPartial
error_message: chat deadline expired with partial response (last_state=aborted)
context: {"phase": "deadline", "last_state": "aborted", "event_counts": {"res": 1, "event:chat": 4, "event:tick": 5, "event:agent": 72, "event:health": 3}}

ID 251:

error_class: ChatDeadlineExpiredPartial
error_message: chat deadline expired with partial response (last_state=aborted)
context: {"phase": "deadline", "last_state": "aborted", "event_counts": {"res": 1, "event:chat": 6, "event:tick": 10, "event:agent": 163, "event:health": 5}}

Acceptance tests / E2E flows

N/A — this is a performance/configuration issue.

Affected areas

  • Chat deadline configuration
  • Agent event processing loop
  • Harness timeout settings

Out of scope

  • Changing the agent architecture
  • Removing health checks

Risk & rollback

  • Risk: medium — deadline changes affect all agent turns
  • Rollback: revert deadline configuration to previous value

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions