🦀 Rusty Orchestrator

A high-performance CI/CD pipeline runner written in Rust.

Executes task pipelines defined in YAML, runs independent tasks in parallel, skips unchanged work via content-addressable caching, and understands GitHub Actions workflow syntax natively.

---

Installation

Option 1 — cargo install (recommended)

Requires Rust 1.70+. Installs the rustyochestrator binary directly from crates.io:

cargo install rustyochestrator

Upgrade to the latest version at any time:

cargo install rustyochestrator --force

Option 2 — pre-built binary

Download the binary for your platform from the latest GitHub release, then move it onto your PATH:

# macOS / Linux
tar xzf rustyochestrator-*.tar.gz
sudo mv rustyochestrator /usr/local/bin/

# verify
rustyochestrator --version

Option 3 — one-liner installer

curl -fsSL https://github.com/KodeSage/rustyochestrator/releases/latest/download/install.sh | sh

Option 4 — build from source

git clone https://github.com/KodeSage/rustyochestrator
cd rustyochestrator
cargo build --release
./target/release/rustyochestrator --version

---

Quick start

No folder setup needed

rustyochestrator init creates the pipeline file in your current directory — no manual folder creation required:

rustyochestrator init
rustyochestrator run pipeline.yaml

Dependencies are not auto-installed

rustyochestrator only runs the commands you define. If your project needs packages installed, declare it as a task:

tasks:
  - id: install
    command: "npm install" # or pip install, cargo fetch, etc.

  - id: build
    command: "npm run build"
    depends_on: [install]

---

Local usage

Everything below works offline with no dashboard, no account, and no network access required.

Run a pipeline

rustyochestrator run pipeline.yaml

Run a second time — every unchanged task is skipped instantly from cache:

rustyochestrator run pipeline.yaml
# [CACHE HIT] Skipping task: build
# [CACHE HIT] Skipping task: test
# [CACHE HIT] Skipping task: deploy

All CLI commands

# Execute a pipeline
rustyochestrator run pipeline.yaml
rustyochestrator run pipeline.yaml --concurrency 4   # limit worker count
rustyochestrator run .github/workflows/ci.yml        # GitHub Actions format

# Validate without running
rustyochestrator validate pipeline.yaml

# Show execution order by stage
rustyochestrator list pipeline.yaml

# Print the dependency graph
rustyochestrator graph pipeline.yaml

# Inspect the local cache
rustyochestrator cache show

# Clear the local cache (forces full re-run next time)
rustyochestrator cache clean

# Scaffold a new pipeline.yaml
rustyochestrator init
rustyochestrator init my-pipeline.yaml   # custom filename

# Run all workflows in a directory simultaneously
rustyochestrator run-all .github/workflows
rustyochestrator run-all ./my-pipelines --concurrency 4

# Debug logging
RUST_LOG=debug rustyochestrator run pipeline.yaml

Developing from source

Clone the repo and use cargo run in place of the installed binary:

git clone https://github.com/yourname/rusty
cd rusty

cargo run -- run examples/pipeline.yaml
cargo run -- run examples/pipeline.yaml --concurrency 2
cargo run -- validate examples/pipeline.yaml
cargo run -- list examples/pipeline.yaml
cargo run -- graph examples/pipeline.yaml
cargo run -- cache show
cargo run -- cache clean
cargo run -- init
cargo run -- run-all .github/workflows

# Build and run the release binary directly
cargo build --release
./target/release/rustyochestrator run examples/pipeline.yaml

---

Manage the connection

rustyochestrator status       # show connected dashboard and user
rustyochestrator disconnect   # remove connection

---

Language support

rustyochestrator is completely language-agnostic. The command field runs anything your shell can execute.

Node.js / npm

tasks:
  - id: install
    command: "npm install"

  - id: lint
    command: "npm run lint"
    depends_on: [install]

  - id: test
    command: "npm test"
    depends_on: [install]

  - id: build
    command: "npm run build"
    depends_on: [lint, test]

Python

tasks:
  - id: install
    command: "pip install -r requirements.txt"

  - id: lint
    command: "flake8 src/"
    depends_on: [install]

  - id: test
    command: "pytest tests/"
    depends_on: [install]

  - id: docker-build
    command: "docker build -t myapp:latest ."
    depends_on: [lint, test]

Terraform / infrastructure

tasks:
  - id: tf-init
    command: "terraform init"

  - id: tf-plan
    command: "terraform plan -out=plan.tfplan"
    depends_on: [tf-init]

  - id: tf-apply
    command: "terraform apply plan.tfplan"
    depends_on: [tf-plan]

Mixed stack

tasks:
  - id: backend-test
    command: "cargo test"

  - id: frontend-test
    command: "npm test"

  - id: build-image
    command: "docker build -t myapp ."
    depends_on: [backend-test, frontend-test]

  - id: push-image
    command: "docker push myapp:latest"
    depends_on: [build-image]

Any command that runs in sh -c works — shell scripts, Python scripts, Makefiles, Docker, cloud CLIs, or anything else.

---

Features

• Parallel execution — worker pool backed by Tokio; concurrency defaults to the number of logical CPUs
• DAG scheduling — dependencies are resolved at runtime; tasks run as soon as their deps finish
• Content-addressable cache — each task is hashed by its command + dependency IDs + env; unchanged tasks are skipped instantly
• GitHub Actions compatibility — parse and run .github/workflows/*.yml files directly
• Parallel workflow execution — run-all runs every workflow file in a directory simultaneously, with each workflow's output prefixed by its name
• Live TUI dashboard — colour-coded per-task progress view with spinners, elapsed time, and a summary bar; auto-detects TTY and falls back to plain log output in CI
• Environment variables & secrets — declare env: at pipeline or task level; reference shell secrets with ${{ secrets.NAME }}; missing secrets abort before execution starts
• Retry logic — failed tasks are retried up to 2 times before being marked failed
• Failure propagation — when a task fails its entire transitive dependent subtree is cancelled immediately
• Real-time output — stdout and stderr from every task are streamed line-by-line as they run
• Cycle detection — circular dependencies are caught before execution starts
• Live dashboard — optional dashhy integration streams pipeline events to a hosted monitoring UI

---

Pipeline format

Native YAML

tasks:
  - id: build
    command: "cargo build"

  - id: lint
    command: "cargo clippy -- -D warnings"
    depends_on: [build]

  - id: test
    command: "cargo test"
    depends_on: [build]

  - id: package
    command: "tar czf dist.tar.gz target/release/myapp"
    depends_on: [lint, test]

Task fields:

Field	Required	Description
id	yes	Unique identifier for the task
command	yes	Shell command to run (executed via sh -c)
depends_on	no	List of task IDs that must succeed first
env	no	Map of environment variables for this task

Multi-line commands work with YAML block scalars:

tasks:
  - id: report
    command: |
      echo "=== build info ==="
      rustc --version
      du -sh target/release/myapp
    depends_on: [build]

Environment variables & secrets

Declare environment variables at the pipeline level (applied to every task) or at the task level (overrides pipeline-level values for that task only).

env:
  NODE_ENV: production
  API_URL: https://api.example.com

tasks:
  - id: build
    command: "npm run build"

  - id: deploy
    command: "npm run deploy"
    env:
      API_URL: https://staging.example.com  # overrides pipeline-level value
      API_KEY: "${{ secrets.DEPLOY_KEY }}"  # resolved from shell env at runtime

Secret references use the ${{ secrets.NAME }} syntax. At runtime, rustyochestrator reads NAME from the current shell environment and passes it to the task process — the value is never written to disk.

Pre-flight validation: all secrets are resolved before any task starts. If a referenced secret is missing, the run aborts immediately with a clear error:

Error: secret 'DEPLOY_KEY' referenced by env key 'API_KEY' in task 'deploy' is not set in the environment

Debug logging prints env keys when RUST_LOG=debug is set. Values for keys containing SECRET, TOKEN, KEY, or PASSWORD (case-insensitive) are redacted as ***.

RUST_LOG=debug rustyochestrator run pipeline.yaml
# DEBUG task=deploy key=API_URL value=https://staging.example.com
# DEBUG task=deploy key=API_KEY value=***

Cache invalidation: changing any env value (including secrets) invalidates the task's cache hash, forcing a re-run.

In GitHub Actions workflow files, env: blocks at the workflow, job, and step levels are all parsed and merged. Plain values and ${{ secrets.NAME }} references are forwarded; other ${{ }} expressions (matrix variables, context references) are silently dropped since they require a real Actions runner.

GitHub Actions format

Rusty can run GitHub Actions workflow files directly — useful for local testing before pushing.

# .github/workflows/ci.yml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Compile
        run: cargo build --release

  test:
    runs-on: ubuntu-latest
    needs: [build]
    steps:
      - name: Unit tests
        run: cargo test --all

rustyochestrator run .github/workflows/ci.yml

Mapping rules:

• Each run: step becomes one task
• Steps within a job are sequential
• needs: wires the first step of a job to the last step of each required job
• uses: steps (actions) are silently skipped

---

CLI reference

run — execute a pipeline

rustyochestrator run <pipeline.yaml> [--concurrency <N>] [--no-tui]

rustyochestrator run pipeline.yaml
rustyochestrator run pipeline.yaml --concurrency 4
rustyochestrator run .github/workflows/ci.yml      # GitHub Actions format
rustyochestrator run pipeline.yaml --no-tui        # force plain log output
RUST_LOG=debug rustyochestrator run pipeline.yaml  # verbose logging

When stdout is a TTY the TUI dashboard is shown automatically:

rustyochestrator — pipeline.yaml   elapsed 00:00:12

  ✓ toolchain                        0.8s   [cached]
  ✓ fmt                              1.2s
  ⠸ clippy                           12s    [running]
  ⠸ build-debug                      9s     [running]
  ◌ test                                    [waiting]
  ◌ build-release                           [waiting]
  ◌ smoke-test                              [waiting]

  ████████░░░░░░░░░░░░░░░░  2/7  2 done  2 running  3 pending  0 failed

Use --no-tui to force plain scrolling output (e.g. when piping to a file or running in CI without a pseudo-TTY).

In non-TTY environments (CI, | tee, > file) the dashboard is suppressed automatically and the plain log format is used instead.

---

run-all — run all workflows in a directory simultaneously

Discovers every .yml and .yaml file in the given directory, loads them all, then runs them concurrently — just like GitHub Actions fires multiple workflow files in parallel. Each workflow's output is prefixed with its filename so interleaved logs are always identifiable.

rustyochestrator run-all <dir> [--concurrency <N>]

rustyochestrator run-all .github/workflows          # default directory
rustyochestrator run-all ./pipelines                # any folder
rustyochestrator run-all examples --concurrency 2   # limit workers per workflow

Example output with two workflows running simultaneously:

INFO  running workflows simultaneously count=2 dir=.github/workflows
INFO  loaded workflow=ci tasks=7
INFO  loaded workflow=release tasks=7
[ci] [INFO] Starting task: lint__cargo_fmt___check
[release] [INFO] Starting task: build__Install_cross_...
[ci]   [lint__cargo_fmt___check] ...
[release]   [build__Install_cross_...|err] Compiling libc v0.2.183
[ci] [INFO] Completed task: lint__cargo_fmt___check
[release] [INFO] Completed task: build__Install_cross_...

• All pipelines are validated before any execution starts — parse errors surface immediately
• Each workflow runs its own independent DAG scheduler with its own cache
• Exit code is non-zero if any workflow fails; the failing workflow name is reported
• Works with both native pipeline format and GitHub Actions format files in the same directory

---

validate — check without running

Parses the file, checks for missing dependencies and cycles, prints each task and its deps. Exits non-zero on any error.

rustyochestrator validate pipeline.yaml

  7 tasks
  [ok] build
  [ok] test  (needs: build)
  [ok] deploy  (needs: test)

pipeline 'pipeline.yaml' is valid.

---

list — show execution order

Groups tasks into parallel stages so you can see exactly what runs when.

rustyochestrator list pipeline.yaml

Execution order for 'pipeline.yaml':

  Stage 0 — 2 task(s) run in parallel:
    1. toolchain
    2. fmt

  Stage 1 — 2 task(s) run in parallel:
    3. clippy  (after: fmt)
    4. build-debug  (after: fmt)

  Stage 2 — 1 task(s) run in parallel:
    5. test  (after: build-debug, clippy)

---

graph — ASCII dependency graph

rustyochestrator graph pipeline.yaml

Dependency graph for 'pipeline.yaml':

  Stage 0  (no deps):
    toolchain
    fmt

  Stage 1:
    clippy  ◄── [fmt]
    build-debug  ◄── [fmt]

  Stage 2:
    test  ◄── [build-debug, clippy]

---

cache show — inspect the cache

rustyochestrator cache show

  task                           status     hash
  ------------------------------------------------------------------------
  build                          ok         b3d10802f5217f42
  clippy                         ok         9eeaf9f8bc055df3
  fmt                            ok         810e75f0d3dee10e
  test                           ok         257080d6e3e17348

  4 cached task(s).

---

cache clean — clear the cache

Forces every task to re-run on the next rustyochestrator run.

rustyochestrator cache clean
# Cache cleared.

---

init — scaffold a new pipeline

Creates a starter pipeline.yaml (or a custom filename) in the current directory.

rustyochestrator init                    # creates pipeline.yaml
rustyochestrator init my-pipeline.yaml   # custom filename

---

connect — link to dashhy dashboard

rustyochestrator connect --token <jwt> --url <dashboard-url>

Saves the connection to ~/.rustyochestrator/connect.json. All subsequent run commands will report live to the dashboard.

---

disconnect — remove dashboard connection

rustyochestrator disconnect

---

status — show connection status

rustyochestrator status
# Connected
#   Dashboard : https://your-dashhy.vercel.app
#   User      : @your-github-username

---

Caching

Cache entries are stored in .rustyochestrator/cache.json.

A task is a cache hit when:

1. Its SHA-256 hash (of command + dependency IDs + env key/value pairs) matches the stored entry
2. The previous run recorded success: true

{
  "entries": {
    "build": {
      "hash": "a3f1c2...",
      "success": true
    }
  }
}

To force a full re-run, delete the cache:

rustyochestrator cache clean
# or
rm -rf .rustyochestrator

---

License

MIT