-
Notifications
You must be signed in to change notification settings - Fork 317
feat(linux): systemd switchyard service #866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| .PHONY: install-linux install-linux-dry-run uninstall-linux | ||
|
|
||
| ## Install the Switchyard background server as a systemd user service. | ||
| install-linux: | ||
| @scripts/linux/install.sh | ||
|
|
||
| ## Print what install-linux would do, without changing anything. | ||
| install-linux-dry-run: | ||
| @scripts/linux/install.sh --dry-run | ||
|
|
||
| ## Remove the systemd user service, the sy Codex profile, and the codex alias. | ||
| uninstall-linux: | ||
| @scripts/linux/uninstall.sh | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Paths, markers, and helpers shared by install.sh and uninstall.sh. | ||
| # The markers must match between the two, which is why they live here. | ||
|
|
||
| SY_HOME="${SY_HOME:-$HOME/.switchyard}" | ||
| SY_PORT="${SY_PORT:-4123}" | ||
| SERVICE_NAME="switchyard.service" | ||
| SYSTEMD_USER_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user" | ||
| CODEX_DIR="${CODEX_HOME:-$HOME/.codex}" | ||
|
Comment on lines
+7
to
+11
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: sed -n '1,48p;150,182p' scripts/linux/install.sh
sed -n '1,45p' scripts/linux/common.shRepository: NVIDIA-NeMo/Switchyard Length of output: 3791 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- references ---'
rg -n --no-heading 'SY_HOME|SY_PORT|XDG_CONFIG_HOME|CODEX_HOME|SYSTEMD_USER_DIR|CODEX_DIR|write_always|ExecStart|StandardOutput|StandardError' scripts/linux
printf '%s\n' '--- install relevant ranges ---'
nl -ba scripts/linux/install.sh | sed -n '1,240p'
printf '%s\n' '--- common ---'
nl -ba scripts/linux/common.sh | sed -n '1,80p'
printf '%s\n' '--- diff against merge base ---'
git diff --stat a601a9a3f9db149a1ad430fa43b1463a170c8a82 e518721574d8a11785440a8e0d9f46f55736ff72 -- scripts/linux/common.sh scripts/linux/install.sh
git diff --unified=20 a601a9a3f9db149a1ad430fa43b1463a170c8a82 e518721574d8a11785440a8e0d9f46f55736ff72 -- scripts/linux/common.sh scripts/linux/install.shRepository: NVIDIA-NeMo/Switchyard Length of output: 20076 🏁 Script executed: #!/bin/bash
set -euo pipefail
if command -v man >/dev/null 2>&1; then
man systemd.syntax 2>/dev/null | col -b | sed -n '1,180p'
else
printf '%s\n' 'man is unavailable'
fi
printf '%s\n' '--- systemd tools ---'
command -v systemd-analyze || true
systemd-analyze --version 2>/dev/null | sed -n '1,5p' || trueRepository: NVIDIA-NeMo/Switchyard Length of output: 201 Validate
This is a malformed-configuration and service-startup issue. The available code does not establish a privilege escalation or other security-boundary bypass. Suggested fixdiff --git a/scripts/linux/common.sh b/scripts/linux/common.sh
@@
CODEX_PROFILE_CONFIG="$CODEX_DIR/sy.config.toml"
+
+validate_unit_value() {
+ local name="$1" value="$2"
+ if [[ "$value" =~ [[:space:][:cntrl:]] || "$value" == *\\ ]]; then
+ printf 'error: %s contains whitespace, control characters, or a trailing backslash\n' "$name" >&2
+ return 1
+ fi
+}diff --git a/scripts/linux/install.sh b/scripts/linux/install.sh
@@
source "$SCRIPT_DIR/common.sh"
+validate_unit_value SY_HOME "$SY_HOME" || exit 1
+if [[ ! "$SY_PORT" =~ ^[0-9]+$ ]]; then
+ printf 'error: SY_PORT must be numeric\n' >&2
+ exit 1
+fi
+
# Runs a command, or prints it when dry running.
run() {🧰 Tools🪛 Shellcheck (0.11.0)[warning] 9-9: SERVICE_NAME appears unused. Verify use (or export if used externally). (SC2034) [warning] 10-10: SYSTEMD_USER_DIR appears unused. Verify use (or export if used externally). (SC2034) 🤖 Prompt for AI Agents |
||
| # Codex reads `--profile sy` from its own file next to config.toml. A | ||
| # [profiles.sy] table in config.toml is rejected outright as legacy config. | ||
|
Comment on lines
+12
to
+13
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With Codex CLI 0.133.0, The comment also says Codex rejects Suggested wording: "Since Codex 0.134.0, |
||
| CODEX_PROFILE_CONFIG="$CODEX_DIR/sy.config.toml" | ||
| ALIAS_START="# >>> switchyard codex alias >>>" | ||
| ALIAS_END="# <<< switchyard codex alias <<<" | ||
|
|
||
| say() { printf '%s\n' "$*"; } | ||
| step() { printf '\n==> %s\n' "$*"; } | ||
|
|
||
| # Deletes the marked block, inclusive, leaving the rest of the file alone. | ||
| strip_block() { | ||
| local path="$1" start="$2" end="$3" label="${4:-the switchyard block}" | ||
| if [[ ! -f "$path" ]] || ! grep -qF "$start" "$path"; then | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If a hand edit removes the end marker, uninstall deletes every line from the alias to the end of the rc file and still prints "removed the codex alias". This line checks only for the start marker, so the In my test, I deleted the Please also check for if ! grep -qF "$end" "$path"; then
say " $path has no end marker for $label; not editing it" >&2
return 1
fiI tested this together with the caller change from the |
||
| return 1 | ||
| fi | ||
| if (( DRY_RUN )); then | ||
| say " would remove $label from $path" | ||
| return 0 | ||
| fi | ||
| local temp | ||
| temp="$(mktemp)" | ||
| awk -v start="$start" -v end="$end" ' | ||
| index($0, start) { skipping = 1 } | ||
| !skipping { print } | ||
| index($0, end) { skipping = 0 } | ||
| ' "$path" > "$temp" | ||
| cat "$temp" > "$path" | ||
| rm -f "$temp" | ||
| say " removed $label from $path" | ||
| return 0 | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,218 @@ | ||
| #!/usr/bin/env bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
| # | ||
| # Installs the Switchyard background server as a systemd --user service, sets | ||
| # up a `sy` Codex profile, and aliases `codex` to use it. | ||
| # | ||
| # Every step is idempotent and never overwrites a file you have edited. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This comment says the script never overwrites a file you have edited, but a second run replaces the unit file and In my test, I added an Please say what happens to each file. Please also point users to |
||
| # Run with --dry-run to print what would happen. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)" | ||
|
|
||
| DRY_RUN=0 | ||
| [[ "${1:-}" == "--dry-run" ]] && DRY_RUN=1 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Unknown arguments silently run the real action. Both scripts only test for the exact string
📍 Affects 2 files
🤖 Prompt for AI Agents |
||
|
|
||
| # shellcheck source=scripts/linux/common.sh | ||
| source "$SCRIPT_DIR/common.sh" | ||
|
|
||
| # Runs a command, or prints it when dry running. | ||
| run() { | ||
| if (( DRY_RUN )); then | ||
| say " would run: $*" | ||
| else | ||
| "$@" | ||
| fi | ||
| } | ||
|
|
||
| # Writes stdin to a file, leaving an existing file untouched. | ||
| write_once() { | ||
| local path="$1" | ||
| if [[ -f "$path" ]]; then | ||
| say " keeping existing $path" | ||
| cat >/dev/null | ||
| return | ||
| fi | ||
| if (( DRY_RUN )); then | ||
| say " would create $path" | ||
| cat >/dev/null | ||
| else | ||
| mkdir -p "$(dirname "$path")" | ||
| cat > "$path" | ||
| say " created $path" | ||
| fi | ||
| } | ||
|
|
||
| # Writes stdin to a file, keeping any existing version as a timestamped backup. | ||
| write_with_backup() { | ||
| local path="$1" | ||
| if (( DRY_RUN )); then | ||
| [[ -f "$path" ]] && say " would back up $path" | ||
| say " would write $path" | ||
| cat >/dev/null | ||
| return | ||
| fi | ||
| mkdir -p "$(dirname "$path")" | ||
| local incoming | ||
| incoming="$(mktemp)" | ||
| cat > "$incoming" | ||
| if [[ -f "$path" ]] && cmp -s "$incoming" "$path"; then | ||
| say " $path is already up to date" | ||
| rm -f "$incoming" | ||
| return | ||
| fi | ||
| if [[ -f "$path" ]]; then | ||
| local backup | ||
| backup="$path.switchyard-backup.$(date +%Y%m%d%H%M%S)" | ||
| cp "$path" "$backup" | ||
| say " backed up $path to $backup" | ||
| fi | ||
| cat "$incoming" > "$path" | ||
| rm -f "$incoming" | ||
| say " wrote $path" | ||
| } | ||
|
|
||
| # Writes stdin to a file, replacing it. Used only for files this script owns. | ||
| write_always() { | ||
| local path="$1" | ||
| if (( DRY_RUN )); then | ||
| say " would write $path" | ||
| cat >/dev/null | ||
| else | ||
| mkdir -p "$(dirname "$path")" | ||
| cat > "$path" | ||
| say " wrote $path" | ||
| fi | ||
| } | ||
|
|
||
| if [[ "$(uname -s)" != "Linux" ]]; then | ||
| say "This installer is for Linux only." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| step "Building release binary" | ||
| run cargo build --release --manifest-path "$REPO_ROOT/Cargo.toml" -p switchyard-server | ||
|
|
||
| step "Installing binary into $SY_HOME/bin" | ||
| run mkdir -p "$SY_HOME/bin" "$SY_HOME/logs" | ||
| run install -m 755 "$REPO_ROOT/target/release/switchyard-server" "$SY_HOME/bin/switchyard-server" | ||
|
|
||
| step "Writing server config" | ||
| # The composite router from examples/run_codex.sh: Terra classifies each user | ||
| # turn and sets the tier, Stage drives the tool loop underneath it. | ||
| write_once "$SY_HOME/composite.toml" <<'EOF' | ||
| schema_version = 1 | ||
|
|
||
| [llm_clients.chatgpt_backend] | ||
| format = "openai_responses" | ||
| base_url = "https://chatgpt.com/backend-api/codex" | ||
| forward_auth = true | ||
|
|
||
| [targets.capable] | ||
| id = "gpt-5.6-sol" | ||
| llm_client = "chatgpt_backend" | ||
|
|
||
| [targets.efficient] | ||
| id = "gpt-5.6-luna" | ||
| llm_client = "chatgpt_backend" | ||
|
|
||
| # chatgpt.com/backend-api/codex is Codex CLI's own private endpoint, not the | ||
| # public OpenAI Responses API. It 400s unless store=false and stream=true are | ||
| # set explicitly, and it rejects max_output_tokens outright, so the | ||
| # classifier's own token cap has to be dropped before the request goes out. | ||
| [targets.terra] | ||
| id = "gpt-5.6-terra" | ||
| llm_client = "chatgpt_backend" | ||
| extra_body = { store = false, stream = true } | ||
| omit_body_fields = ["max_output_tokens"] | ||
|
|
||
| [routes.switchyard] | ||
| id = "switchyard" | ||
| type = "composite" | ||
|
|
||
| [routes.switchyard.classifier] | ||
| target = "terra" | ||
| base_threshold = 0.5 | ||
| classify_trigger = "user_turn" | ||
|
|
||
| [routes.switchyard.stage] | ||
| capable_target = "capable" | ||
| efficient_target = "efficient" | ||
| confidence_threshold = 0.5 | ||
| EOF | ||
|
|
||
| step "Validating the server config" | ||
| if (( DRY_RUN )); then | ||
| say " would run: $SY_HOME/bin/switchyard-server --config $SY_HOME/composite.toml --dry-run" | ||
| else | ||
| "$SY_HOME/bin/switchyard-server" --config "$SY_HOME/composite.toml" --dry-run | ||
| fi | ||
|
|
||
| step "Writing the systemd user service" | ||
| write_always "$SYSTEMD_USER_DIR/$SERVICE_NAME" <<EOF | ||
| [Unit] | ||
| Description=Switchyard LLM router server | ||
|
|
||
| [Service] | ||
| Type=simple | ||
| ExecStart=$SY_HOME/bin/switchyard-server --config $SY_HOME/composite.toml --host 127.0.0.1 --port $SY_PORT --routing-log-file $SY_HOME/routing.jsonl | ||
| Restart=on-failure | ||
| Environment=RUST_LOG=info | ||
| StandardOutput=append:$SY_HOME/logs/server.log | ||
| StandardError=append:$SY_HOME/logs/server.err.log | ||
|
Comment on lines
+164
to
+165
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With these two lines, In my test, each request added about 1.1 KB to If you remove these two lines, journald keeps the logs and rotates them. Users then read them with |
||
|
|
||
| [Install] | ||
| WantedBy=default.target | ||
| EOF | ||
|
|
||
| step "Loading the systemd user service" | ||
| if (( DRY_RUN )); then | ||
| say " would run: systemctl --user daemon-reload" | ||
| say " would run: systemctl --user enable --now $SERVICE_NAME" | ||
| else | ||
| systemctl --user daemon-reload | ||
| systemctl --user enable --now "$SERVICE_NAME" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the service is already running, a second I re-ran the installer with Please run |
||
| say " enabled and started $SERVICE_NAME" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the server exits right after it starts, this line still prints "enabled and started", and the script goes on to write the Codex profile and the alias. With In my test, a second After starting the unit, please wait a couple of seconds and then check |
||
| fi | ||
|
|
||
| step "Adding the sy Codex profile" | ||
| # `codex --profile sy` reads ~/.codex/sy.config.toml. A [profiles.sy] table in | ||
| # config.toml is legacy config that Codex refuses to start with, so the | ||
| # profile lives in its own file instead. | ||
| # | ||
| # This profile only changes which router answers. Approval and sandbox | ||
| # settings are deliberately left out, so the profile cannot loosen how Codex | ||
| # asks before it acts. Set those yourself if you want them. | ||
| write_with_backup "$CODEX_PROFILE_CONFIG" <<EOF | ||
| model = "switchyard" | ||
| model_provider = "sy" | ||
|
|
||
| [model_providers.sy] | ||
| name = "Switchyard" | ||
| base_url = "http://127.0.0.1:$SY_PORT/v1" | ||
| wire_api = "responses" | ||
| requires_openai_auth = true | ||
| EOF | ||
|
|
||
| step "Aliasing codex" | ||
| for rc in "$HOME/.zshrc" "$HOME/.bashrc"; do | ||
| if [[ -f "$rc" ]] && grep -qF "$ALIAS_START" "$rc"; then | ||
| say " alias already in $rc" | ||
| elif (( DRY_RUN )); then | ||
| say " would add the codex alias to $rc" | ||
| else | ||
| printf '\n%s\nalias codex="codex --profile sy"\n%s\n' "$ALIAS_START" "$ALIAS_END" >> "$rc" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With this alias,
The simplest fix is to drop the alias and print Also, the loop at line 201 creates |
||
| say " added the codex alias to $rc" | ||
| fi | ||
| done | ||
|
|
||
| step "Done" | ||
| say "Server: http://127.0.0.1:$SY_PORT (logs in $SY_HOME/logs)" | ||
| say "Manage it with: systemctl --user {status,restart,stop} $SERVICE_NAME" | ||
| say "Open a new shell, or run: alias codex=\"codex --profile sy\"" | ||
| say "" | ||
| say "If you want the server running even when you are logged out, run:" | ||
| say " loginctl enable-linger \$USER" | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,57 @@ | ||||||||||||||||
| #!/usr/bin/env bash | ||||||||||||||||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||||||||||||||||
| # SPDX-License-Identifier: Apache-2.0 | ||||||||||||||||
| # | ||||||||||||||||
| # Removes what install.sh added: the systemd user service, the `sy` Codex | ||||||||||||||||
| # profile, and the codex alias. Your config, routing log, and binaries stay | ||||||||||||||||
| # put; the paths are printed so you can delete them yourself. | ||||||||||||||||
| # | ||||||||||||||||
| # Run with --dry-run to print what would happen. | ||||||||||||||||
|
|
||||||||||||||||
| set -euo pipefail | ||||||||||||||||
|
|
||||||||||||||||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||||||||||||||||
|
|
||||||||||||||||
| DRY_RUN=0 | ||||||||||||||||
| [[ "${1:-}" == "--dry-run" ]] && DRY_RUN=1 | ||||||||||||||||
|
|
||||||||||||||||
| # shellcheck source=scripts/linux/common.sh | ||||||||||||||||
| source "$SCRIPT_DIR/common.sh" | ||||||||||||||||
|
|
||||||||||||||||
| # Deletes a file this script owns. | ||||||||||||||||
| remove_file() { | ||||||||||||||||
| local path="$1" | ||||||||||||||||
| if [[ ! -e "$path" ]]; then | ||||||||||||||||
| say " nothing to remove at $path" | ||||||||||||||||
| elif (( DRY_RUN )); then | ||||||||||||||||
| say " would delete $path" | ||||||||||||||||
| else | ||||||||||||||||
| rm -f "$path" | ||||||||||||||||
| say " deleted $path" | ||||||||||||||||
| fi | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| step "Stopping the systemd user service" | ||||||||||||||||
| if (( DRY_RUN )); then | ||||||||||||||||
| say " would run: systemctl --user disable --now $SERVICE_NAME" | ||||||||||||||||
| say " would delete $SYSTEMD_USER_DIR/$SERVICE_NAME" | ||||||||||||||||
| else | ||||||||||||||||
| systemctl --user disable --now "$SERVICE_NAME" 2>/dev/null || true | ||||||||||||||||
| rm -f "$SYSTEMD_USER_DIR/$SERVICE_NAME" | ||||||||||||||||
| systemctl --user daemon-reload | ||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If After that, Please move this block after the profile and alias steps. I tried that order with a failing |
||||||||||||||||
| say " stopped and removed $SERVICE_NAME" | ||||||||||||||||
| fi | ||||||||||||||||
|
|
||||||||||||||||
| step "Removing the sy Codex profile" | ||||||||||||||||
| remove_file "$CODEX_PROFILE_CONFIG" | ||||||||||||||||
|
|
||||||||||||||||
| step "Removing the codex alias" | ||||||||||||||||
| for rc in "$HOME/.zshrc" "$HOME/.bashrc"; do | ||||||||||||||||
| strip_block "$rc" "$ALIAS_START" "$ALIAS_END" "the codex alias" || | ||||||||||||||||
| say " no alias in $rc" | ||||||||||||||||
|
Comment on lines
+50
to
+51
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If
I reproduced this with a GNU-style Please check for the marker in the caller instead. Then
Suggested change
With this change, the same test stops at |
||||||||||||||||
| done | ||||||||||||||||
|
|
||||||||||||||||
| step "Done" | ||||||||||||||||
| say "Left in place, delete them if you want:" | ||||||||||||||||
| say " $SY_HOME (binary, config, routing log, logs)" | ||||||||||||||||
| say " $CODEX_PROFILE_CONFIG.switchyard-backup.* (backups taken at install time)" | ||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A plain
makein the repo root builds and starts the service and edits the user's shell rc files.install-linuxis the first target in this file, somakeruns it by default.make -nat the repo root printsscripts/linux/install.sh. The repo had noMakefilebefore, so people may typemakehere out of habit.Please make a harmless target the default. For example, add a
helptarget that lists the three targets, and either put it first or select it with.DEFAULT_GOAL := help.