Skip to content

Add human-in-the-loop Inbox + alerts#30

Merged
mlund01 merged 3 commits into
mainfrom
hitl-commander-store
Apr 28, 2026
Merged

Add human-in-the-loop Inbox + alerts#30
mlund01 merged 3 commits into
mainfrom
hitl-commander-store

Conversation

@mlund01
Copy link
Copy Markdown
Owner

@mlund01 mlund01 commented Apr 28, 2026

Summary

Commander half of the HITL feature: operators see and answer questions agents have asked via builtins.human.ask through a new Inbox surface, plus toast + chime alerts when new questions arrive away from the Inbox.

What's wired

API proxy (internal/api/humaninputs.go)

  • GET /instances/:id/human-inputs — list (proxied to squadron, the source of truth)
  • POST /instances/:id/human-inputs/:callId — submit a resolution; commander tags responder identity from the session
  • GET /instances/:id/human-inputs/stream — SSE stream of HumanInputRequested / HumanInputResolved events

Inbox page

  • List mode + Focus mode (oldest-first carousel that auto-advances after submit, for ripping through a backlog).
  • Single-select → quick-reply buttons.
  • Multi-select → toggle buttons with ✓/○ glyph + a Send (N) button that JSON-encodes the selected set.
  • Free-text "Other…" fallback always available.

Inline card (HumanInputCard.tsx) reuses the same widget on mission detail. Resolved responses render through formatResolvedResponse so multi-select shows A, C instead of ["A","C"].

Alerts hook (use-human-input-alerts.ts)

  • Subscribes to the SSE stream; fires toast + chime on every new question.
  • Chime is the snappy first-second of sonar-ping.mp3, capped via Web Audio gain envelope.
  • HTMLAudio fallback for backgrounded tabs where AudioContext can suspend in ways resume() alone doesn't recover. Both paths primed on first user gesture (Safari/Chrome autoplay policy).
  • Skips toast + chime when on the Inbox page (operator is presumably handling things).

Tests

  • HumanInputCard.test.ts (vitest, 9 cases): formatResolvedResponse — verbatim for single/free-text; expand JSON array to comma list for multi-select; graceful fallback on malformed JSON, non-array, non-string entries, empty arrays, missing response.

Dependencies

This PR depends on mlund01/squadron-wire#11 — the wire types for HumanInputRecord, HumanInputRequestedData, HumanInputResolvedData, and the MultiSelect field.

go.mod keeps the existing replace github.com/mlund01/squadron-wire => ../squadron-wire directive (with its TEMP: revert before merging comment) until wire #11 merges and a new tag is published. Drop the replace + bump the require version before merging this PR.

Test plan

  • npm run dev from commander/web runs cleanly; npm run test passes.
  • go build ./... from commander/ succeeds (after wire dep is published).
  • Run ask_human_test mission end-to-end: questions appear in the Inbox; toast + chime fire when the tab is not focused on the Inbox; multi-select Q4 shows toggle buttons + Send (N); resolutions submitted in commander clear the row in real time and update the corresponding Discord message.
  • Cmd+Tab away from the browser, trigger a new question, verify the chime fires (HTMLAudio fallback path).

🤖 Generated with Claude Code

mlund01 added 3 commits April 27, 2026 20:33
The commander half of the HITL feature: an Inbox surface where
operators see and answer questions agents have asked via the
builtins.human.ask tool, plus toast + chime alerts for new questions
that arrive while the operator isn't looking at the Inbox.

API proxy (internal/api/humaninputs.go)
  - GET  /instances/:id/human-inputs           — list (proxies to squadron)
  - POST /instances/:id/human-inputs/:callId   — submit a resolution
  - GET  /instances/:id/human-inputs/stream    — SSE stream of
    HumanInputRequested / HumanInputResolved events for live updates.
  Squadron is the source of truth; commander is a passthrough that
  decorates with the responder's session identity (auth-on) or empty
  string (local dev with no auth).

Hub (internal/hub/connection.go)
  Per-connection fan-out of mission events that match the SSE stream's
  filter, routed to subscriber channels.

Web UI

  Inbox page (InboxPage.tsx)
    - List mode: scannable rows with short-summary, mission/task,
      time-since.
    - Focus mode: oldest-first carousel that auto-advances after a
      submit so an operator can rip through a backlog.
    - Single-select questions render as quick-reply buttons; multi-
      select questions render as toggle buttons with a checkmark
      glyph and a separate Send (N) button that JSON-encodes the
      selected set on submit.
    - Free-text fallback ("Other…") always available.

  HumanInputCard component
    - Inline card used on mission detail and as the carousel cell.
    - Multi-select: toggle behavior + dedicated Send button.
    - Resolved view runs response through formatResolvedResponse so
      multi-select answers display as "A, C" rather than raw JSON.

  use-human-input-alerts hook
    - Subscribes to the SSE stream for the active instance.
    - Fires a toast and a chime on every new question (regardless of
      tab focus — the chime is the away-from-app signal).
    - Chime: preloaded sonar-ping.mp3, capped at 1s with a fade-out
      gain envelope. Web Audio is the primary path; an HTMLAudio
      fallback handles backgrounded tabs where AudioContext can be
      suspended in ways that resume() alone doesn't reliably recover.
    - Both audio paths are primed on the user's first page-interaction
      gesture (Safari/Chrome autoplay policy).
    - Skips toast + chime when the operator is already on the Inbox
      page since they're presumably handling things there.

  Sidebar + layout
    - Inbox entry in AppSidebar with an unanswered-count badge.
    - Layout mounts the alerts hook so it runs everywhere.

Tests
  - HumanInputCard.test.ts: formatResolvedResponse — verbatim for
    single-select / free-text; expand JSON array to comma list for
    multi-select; fall back to raw on malformed JSON, non-array, or
    non-string array entries; render empty array as empty string.

Dependencies
  squadron-wire is pinned at v0.0.40 with a TEMP local replace until
  mlund01/squadron-wire#11 (HITL wire types)
  merges and a new tag is published. Drop the replace + bump the
  require version before merging this PR.
squadron-wire #11 (HITL types) is merged. Drop the local replace
directive and bump the require to a pseudo-version pinned at the
merge commit. Update to the next semver tag once cut.
@mlund01 mlund01 merged commit c6e3100 into main Apr 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant