Skip to content

Feature: Graceful shutdown to avoid dropping active sessions on restart #4

Description

@slacki-ai

Problem

When claudex is restarted (e.g. to deploy a new version), any in-flight handleMessage calls are abruptly killed. This means:

  • The user gets no response to their message
  • The session appears to silently fail, requiring a follow-up "rescue" message to resume
  • There's no indication to the user that an interruption occurred

This has been observed multiple times in practice and is a poor experience.

Proposed Solution

1. Graceful shutdown on SIGTERM

Instead of dying immediately on SIGTERM, claudex should:

  • Stop accepting new messages (remove or pause the Slack event listener)
  • Wait for all active handleMessage calls (tracked in activeThreads) to complete
  • Then exit cleanly

The activeThreads set in events.ts already tracks in-flight sessions — we can use this directly:

process.on('SIGTERM', async () => {
  console.log('SIGTERM received, waiting for active sessions to finish...');
  app.stop(); // stop accepting new Slack events
  // Poll until activeThreads is empty (with a max wait timeout)
  const deadline = Date.now() + 5 * 60 * 1000; // 5 min max
  while (activeThreads.size > 0 && Date.now() < deadline) {
    await new Promise(r => setTimeout(r, 1000));
  }
  process.exit(0);
});

2. Notify active threads on imminent restart

If a graceful drain isn't feasible (e.g. forced restart), post a warning to any active Slack threads before shutting down:

for (const threadKey of activeThreads) {
  const [channelId, threadTs] = threadKey.split(':');
  await client.chat.postMessage({
    channel: channelId,
    thread_ts: threadTs,
    text: '⚠️ claudex is restarting — your session will resume automatically on your next message.',
  });
}

3. Resume notification

When claudex resumes a previously interrupted session, proactively tell the user it was interrupted and is now resuming, rather than silently picking up mid-task.

Implementation Notes

  • activeThreads in events.ts is the right hook for tracking in-flight sessions
  • manage.sh uses kill to stop the process — it should send SIGTERM (not SIGKILL) and wait for the process to exit cleanly before proceeding with restart
  • A maximum drain timeout (e.g. 5 minutes) should be enforced to avoid blocking deploys indefinitely

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions