Skip to content

feat: merge task-broker into superplane#4873

Open
superleomori wants to merge 12 commits into
mainfrom
agent/issue-4872-task-broker-merge
Open

feat: merge task-broker into superplane#4873
superleomori wants to merge 12 commits into
mainfrom
agent/issue-4872-task-broker-merge

Conversation

@superleomori
Copy link
Copy Markdown

Summary

Closes #4872. Absorbs the standalone task-broker service directly into SuperPlane so fleet-manager instances can be registered, managed, and used without a separate deployment.

  • DB migrations: New runner_fleets (installation-scoped fleet registry) and runner_tasks (per-execution task linkage) tables
  • pkg/runners/ package: RunnerFleet/RunnerTask GORM models, Store interface + Postgres implementation, FleetClient that talks to fleet-manager (replaces BrokerClient)
  • Runner component updated: resolves fleet from DB via spec.fleet_id; falls back to TASK_BROKER_* env vars for zero-downtime backwards compatibility. Webhook URL now points to the new POST /webhooks/runner/complete/:runnerTaskID endpoint
  • New webhook endpoint (POST /api/v1/webhooks/runner/complete/:runnerTaskID): looks up RunnerTaskCanvasNodeExecution → dispatches to the runner action handler
  • Fleet management API (installation-admin only):
    • POST /admin/api/v1/runner/fleets — register a fleet
    • GET /admin/api/v1/runner/fleets — list fleets
    • DELETE /admin/api/v1/runner/fleets/{id} — remove a fleet
  • Live log streaming (runner_live_log_stream.go): resolves fleet URL/token via DB before env-var fallback

Test plan

  • go test ./pkg/components/runner/... -short — all runner unit tests pass (mock store, no DB needed)
  • go test ./pkg/runners/... -short — compiles cleanly (no test files yet; store tested via integration)
  • Verify runner_fleets and runner_tasks tables are created after running migrations
  • Register a fleet via POST /admin/api/v1/runner/fleets and confirm it is returned by GET
  • Create a canvas node with fleet_id set; confirm execution dispatches to the correct fleet-manager
  • Confirm legacy TASK_BROKER_* env-var path still works when no fleet_id is specified

🤖 Generated with Claude Code

@superplanehq-integration
Copy link
Copy Markdown

👋 Commands for maintainers:

  • /sp start - Start an ephemeral machine (takes ~30s)
  • /sp stop - Stop a running machine (auto-executed on pr close)

@superleomori superleomori force-pushed the agent/issue-4872-task-broker-merge branch 2 times, most recently from ec9bd7a to 62b8f7e Compare May 18, 2026 22:27
@shiroyasha shiroyasha closed this May 19, 2026
@shiroyasha shiroyasha reopened this May 19, 2026
@shiroyasha shiroyasha closed this May 19, 2026
@shiroyasha shiroyasha reopened this May 19, 2026
@shiroyasha
Copy link
Copy Markdown
Collaborator

/sem-approve

shiroyasha
shiroyasha previously approved these changes May 19, 2026
shiroyasha and others added 8 commits May 19, 2026 16:22
Absorbs the standalone task-broker service directly into SuperPlane so
fleet-manager instances can be configured and managed without a separate
deployment.

- Add `runner_fleets` and `runner_tasks` DB tables (migration)
- New `pkg/runners/` package: RunnerFleet/RunnerTask models, postgres
  store, FleetClient (replaces BrokerClient)
- Runner component now resolves fleet from DB (by spec `fleet_id`) or
  falls back to TASK_BROKER_* env vars for backwards compatibility
- Webhook URL now points to `POST /webhooks/runner/complete/:runnerTaskID`
  which looks up the RunnerTask and dispatches to the action handler
- Fleet management REST API: POST/GET/DELETE `/admin/api/v1/runner/fleets`
  (installation-admin only)
- Live log stream resolves fleet via DB before falling back to env vars

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
- Regenerate docs/components/Core.mdx with updated runner documentation
  (broker → fleet terminology)
- Update db/structure.sql to include runner_fleets and runner_tasks
  tables from the 20260518120000 migration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
…ates

Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Sarcevic <igor@superplane.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
SuperPlane holds the job queue; fleet-manager pulls via sync and reports
completion with task_log metadata. Bridge fleets enqueue without pushing
to fleet_url. Live logs tail CloudWatch directly. Includes migration,
FM API routes, and EC2-oriented store fixes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
@shiroyasha shiroyasha force-pushed the agent/issue-4872-task-broker-merge branch from f52408d to 5138f51 Compare May 19, 2026 14:22
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Signed-off-by: Igor Šarčević <igor@operately.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runners: Merge Task Broker into SuperPlane

2 participants