diff --git a/.github/workflows/docs-build.yml b/.github/workflows/docusaurus-build.yml similarity index 86% rename from .github/workflows/docs-build.yml rename to .github/workflows/docusaurus-build.yml index 9d4f47d92..f13495d8c 100644 --- a/.github/workflows/docs-build.yml +++ b/.github/workflows/docusaurus-build.yml @@ -1,5 +1,5 @@ --- -name: Docs Build +name: Docusaurus Build on: pull_request: @@ -18,4 +18,4 @@ jobs: - name: Fetch justfiles run: just fetch - name: Test build website - run: just docs::build + run: just docusaurus-build diff --git a/.github/workflows/docs-deploy.yml b/.github/workflows/docusaurus-deploy.yml similarity index 95% rename from .github/workflows/docs-deploy.yml rename to .github/workflows/docusaurus-deploy.yml index b141f66d6..beccbdc10 100644 --- a/.github/workflows/docs-deploy.yml +++ b/.github/workflows/docusaurus-deploy.yml @@ -1,5 +1,5 @@ --- -name: Deploy to GitHub Pages +name: Docusaurus Deploy on: push: @@ -23,7 +23,7 @@ jobs: - name: Fetch justfiles run: just fetch - name: Build website - run: just docs::build + run: just docusaurus-build # Popular action to deploy to GitHub Pages: # Docs: https://github.com/peaceiris/actions-gh-pages#%EF%B8%8F-docusaurus diff --git a/.github/workflows/docs-lint.yml b/.github/workflows/docusaurus-lint.yml similarity index 78% rename from .github/workflows/docs-lint.yml rename to .github/workflows/docusaurus-lint.yml index 1223d3363..8e735e06d 100644 --- a/.github/workflows/docs-lint.yml +++ b/.github/workflows/docusaurus-lint.yml @@ -1,5 +1,5 @@ --- -name: Docs Lint +name: Docusaurus Lint on: push: @@ -17,5 +17,5 @@ jobs: uses: extractions/setup-just@v4 - name: Fetch justfiles run: just fetch - - name: Lint docs - run: just docs::fmt-check + - name: Lint the site + run: just docusaurus-fmt-check diff --git a/.github/workflows/markdown-lint.yml b/.github/workflows/markdown-lint.yml new file mode 100644 index 000000000..06bb7b366 --- /dev/null +++ b/.github/workflows/markdown-lint.yml @@ -0,0 +1,22 @@ +--- +name: Markdown Lint + +on: + push: + branches: ["main"] + pull_request: + branches: ["main"] + +jobs: + lint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - name: Install uv + uses: astral-sh/setup-uv@v5 + - name: Install just + uses: extractions/setup-just@v4 + - name: Fetch justfiles + run: just fetch + - name: Lint markdown + run: just md-fmt-check diff --git a/AI_POLICY.md b/AI_POLICY.md index ac2ff37d7..3a47def70 100644 --- a/AI_POLICY.md +++ b/AI_POLICY.md @@ -2,64 +2,60 @@ The osapi-io project has strict rules for AI usage: -- **All AI usage in any form must be disclosed.** You must state - the tool you used (e.g. Claude Code, Cursor, Amp) along with - the extent that the work was AI-assisted. +- **All AI usage in any form must be disclosed.** You must state the tool you + used (e.g. Claude Code, Cursor, Amp) along with the extent that the work was + AI-assisted. -- **The human-in-the-loop must fully understand all code.** If you - can't explain what your changes do and how they interact with the - greater system without the aid of AI tools, do not contribute - to this project. +- **The human-in-the-loop must fully understand all code.** If you can't explain + what your changes do and how they interact with the greater system without the + aid of AI tools, do not contribute to this project. - **Issues and discussions can use AI assistance but must have a full - human-in-the-loop.** This means that any content generated with AI - must have been reviewed _and edited_ by a human before submission. - AI is very good at being overly verbose and including noise that - distracts from the main point. Humans must do their research and - trim this down. - -- **No AI-generated media is allowed (art, images, videos, audio, etc.).** - Text and code are the only acceptable AI-generated content, per the - other rules in this policy. - -- **Bad AI drivers will be denounced** People who produce bad contributions - that are clearly AI (slop) will be added to our public denouncement list. - This list will block all future contributions. Additionally, the list - is public and may be used by other projects to be aware of bad actors. - We love to help junior developers learn and grow, but - if you're interested in that then don't use AI, and we'll help you. - I'm sorry that bad AI drivers have ruined this for you. - -These rules apply only to outside contributions to osapi-io. Maintainers -are exempt from these rules and may use AI tools at their discretion; -they've proven themselves trustworthy to apply good judgment. + human-in-the-loop.** This means that any content generated with AI must have + been reviewed _and edited_ by a human before submission. AI is very good at + being overly verbose and including noise that distracts from the main point. + Humans must do their research and trim this down. + +- **No AI-generated media is allowed (art, images, videos, audio, etc.).** Text + and code are the only acceptable AI-generated content, per the other rules in + this policy. + +- **Bad AI drivers will be denounced** People who produce bad contributions that + are clearly AI (slop) will be added to our public denouncement list. This list + will block all future contributions. Additionally, the list is public and may + be used by other projects to be aware of bad actors. We love to help junior + developers learn and grow, but if you're interested in that then don't use AI, + and we'll help you. I'm sorry that bad AI drivers have ruined this for you. + +These rules apply only to outside contributions to osapi-io. Maintainers are +exempt from these rules and may use AI tools at their discretion; they've proven +themselves trustworthy to apply good judgment. ## There are Humans Here Please remember that osapi-io is maintained by humans. -Every discussion, issue, and pull request is read and reviewed by -humans (and sometimes machines, too). It is a boundary point at which -people interact with each other and the work done. It is rude and -disrespectful to approach this boundary with low-effort, unqualified -work, since it puts the burden of validation on the maintainer. +Every discussion, issue, and pull request is read and reviewed by humans (and +sometimes machines, too). It is a boundary point at which people interact with +each other and the work done. It is rude and disrespectful to approach this +boundary with low-effort, unqualified work, since it puts the burden of +validation on the maintainer. -In a perfect world, AI would produce high-quality, accurate work -every time. But today, that reality depends on the driver of the AI. -And today, most drivers of AI are just not good enough. So, until either -the people get better, the AI gets better, or both, we have to have -strict rules to protect maintainers. +In a perfect world, AI would produce high-quality, accurate work every time. But +today, that reality depends on the driver of the AI. And today, most drivers of +AI are just not good enough. So, until either the people get better, the AI gets +better, or both, we have to have strict rules to protect maintainers. ## AI is Welcome Here -osapi-io is written with plenty of AI assistance, and many maintainers -embrace AI tools as a productive tool in their workflow. As a project, -we welcome AI as a tool! +osapi-io is written with plenty of AI assistance, and many maintainers embrace +AI tools as a productive tool in their workflow. As a project, we welcome AI as +a tool! -**Our reason for the strict AI policy is not due to an anti-AI stance**, -but instead due to the number of highly unqualified people using AI. -It's the people, not the tools, that are the problem. +**Our reason for the strict AI policy is not due to an anti-AI stance**, but +instead due to the number of highly unqualified people using AI. It's the +people, not the tools, that are the problem. -I include this section to be transparent about the project's usage about -AI for people who may disagree with it, and to address the misconception -that this policy is anti-AI in nature. +I include this section to be transparent about the project's usage about AI for +people who may disagree with it, and to address the misconception that this +policy is anti-AI in nature. diff --git a/CLAUDE.md b/CLAUDE.md index aeea9d6b7..46882adbe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,24 +1,36 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +This file provides guidance to Claude Code (claude.ai/code) when working with +code in this repository. ## Project Overview -OSAPI is a Linux system management REST API and CLI written in Go 1.25. It uses NATS JetStream for distributed async job processing with a KV-first, stream-notification architecture. +OSAPI is a Linux system management REST API and CLI written in Go 1.25. It uses +NATS JetStream for distributed async job processing with a KV-first, +stream-notification architecture. ## Development Reference For setup, building, testing, and contributing, see the Docusaurus docs: -- @docs/docs/sidebar/development/development.md - Prerequisites, setup, code style, testing, commit conventions -- @docs/docs/sidebar/development/contributing.md - PR workflow and contribution guidelines -- @docs/docs/sidebar/development/testing.md - How to run tests and list just recipes -- @docs/docs/sidebar/development/ui-development.md - UI prerequisites, setup, code style, components -- @docs/docs/sidebar/architecture/principles.md - Guiding principles (simplicity, minimalism, design philosophy) -- @docs/docs/sidebar/architecture/api-guidelines.md - API design guidelines (REST conventions, endpoint structure) -- @docs/docs/sidebar/architecture/ui.md - UI architecture, embedding, component layers -- @docs/docs/sidebar/usage/configuration.md - Configuration reference (osapi.yaml, env overrides) -- @docs/docs/sidebar/architecture/architecture.md - Architecture overview (links to system and job architecture) +- @docs/docs/sidebar/development/development.md - Prerequisites, setup, code + style, testing, commit conventions +- @docs/docs/sidebar/development/contributing.md - PR workflow and contribution + guidelines +- @docs/docs/sidebar/development/testing.md - How to run tests and list just + recipes +- @docs/docs/sidebar/development/ui-development.md - UI prerequisites, setup, + code style, components +- @docs/docs/sidebar/architecture/principles.md - Guiding principles + (simplicity, minimalism, design philosophy) +- @docs/docs/sidebar/architecture/api-guidelines.md - API design guidelines + (REST conventions, endpoint structure) +- @docs/docs/sidebar/architecture/ui.md - UI architecture, embedding, component + layers +- @docs/docs/sidebar/usage/configuration.md - Configuration reference + (osapi.yaml, env overrides) +- @docs/docs/sidebar/architecture/architecture.md - Architecture overview (links + to system and job architecture) Quick reference for common commands: @@ -40,96 +52,128 @@ just react-fmt # Format UI with Prettier ## Architecture (Quick Reference) -- **`cmd/`** - Cobra CLI commands (`client`, `node agent`, `controller.api`, `nats server`) -- **`internal/controller/api/`** - Echo REST API. Node-targeted handlers nest under `node/{domain}/`. Controller-only handlers are top-level (`job/`, `health/`, etc.). Each domain has its own `gen/` with OpenAPI spec. Combined spec: `api/gen/api.yaml` -- **`internal/job/`** - Job domain types, subject routing. `client/` for high-level ops -- **`internal/agent/`** - Node agent: consumer/handler/processor pipeline for job execution -- **`internal/telemetry/tracing/`** - OpenTelemetry tracer initialization, slog trace handler, context propagation\ -- **`internal/telemetry/metrics/`** - Per-component Prometheus metrics server with isolated registries\ -- **`internal/provider/`** - Operation implementations organized by category then domain. Browse the directory to see current providers -- **`internal/telemetry/process/`** - Agent self-metrics (CPU%, RSS, goroutines) and process condition evaluation for heartbeat -- **`internal/controller/notify/`** - Pluggable condition notification system: watches registry KV for condition transitions, dispatches via `Notifier` interface (`log` backend) -- **`internal/config/`** - Viper-based config from `osapi.yaml`. Struct fields use `validate` tags (same validator as API handlers). Defaults are set via `viper.SetDefault()` in `cmd/root.go` -- **`pkg/sdk/`** - Go SDK for programmatic REST API access (`client/` client library). See @docs/docs/sidebar/sdk/guidelines.md for SDK development rules -- Shared `nats-client` and `nats-server` are sibling repos linked via `replace` in `go.mod` +- **`cmd/`** - Cobra CLI commands (`client`, `node agent`, `controller.api`, + `nats server`) +- **`internal/controller/api/`** - Echo REST API. Node-targeted handlers nest + under `node/{domain}/`. Controller-only handlers are top-level (`job/`, + `health/`, etc.). Each domain has its own `gen/` with OpenAPI spec. Combined + spec: `api/gen/api.yaml` +- **`internal/job/`** - Job domain types, subject routing. `client/` for + high-level ops +- **`internal/agent/`** - Node agent: consumer/handler/processor pipeline for + job execution +- **`internal/telemetry/tracing/`** - OpenTelemetry tracer initialization, slog + trace handler, context propagation\\ +- **`internal/telemetry/metrics/`** - Per-component Prometheus metrics server + with isolated registries\\ +- **`internal/provider/`** - Operation implementations organized by category + then domain. Browse the directory to see current providers +- **`internal/telemetry/process/`** - Agent self-metrics (CPU%, RSS, goroutines) + and process condition evaluation for heartbeat +- **`internal/controller/notify/`** - Pluggable condition notification system: + watches registry KV for condition transitions, dispatches via `Notifier` + interface (`log` backend) +- **`internal/config/`** - Viper-based config from `osapi.yaml`. Struct fields + use `validate` tags (same validator as API handlers). Defaults are set via + `viper.SetDefault()` in `cmd/root.go` +- **`pkg/sdk/`** - Go SDK for programmatic REST API access (`client/` client + library). See @docs/docs/sidebar/sdk/guidelines.md for SDK development rules +- Shared `nats-client` and `nats-server` are sibling repos linked via `replace` + in `go.mod` ## UI Architecture (Quick Reference) -- **`ui/`** - React 19 + TypeScript + Vite + Tailwind CSS v4. Embedded into the Go binary at build time. -- **`ui/src/sdk/gen/`** - Generated TypeScript SDK (orval) from the combined OpenAPI spec. DO NOT EDIT. -- **`ui/src/sdk/fetch.ts`** - Hand-written fetch mutator: auth token + base URL wiring for orval. -- **`ui/src/components/ui/`** - Reusable UI primitives. Every visual pattern is a component. -- **`ui/src/components/layout/`** - Page structure (Navbar, PageLayout, ContentArea, NetworkMapBackground). -- **`ui/src/components/domain/`** - Domain-specific components (blocks, cards, pickers). +- **`ui/`** - React 19 + TypeScript + Vite + Tailwind CSS v4. Embedded into the + Go binary at build time. +- **`ui/src/sdk/gen/`** - Generated TypeScript SDK (orval) from the combined + OpenAPI spec. DO NOT EDIT. +- **`ui/src/sdk/fetch.ts`** - Hand-written fetch mutator: auth token + base URL + wiring for orval. +- **`ui/src/components/ui/`** - Reusable UI primitives. Every visual pattern is + a component. +- **`ui/src/components/layout/`** - Page structure (Navbar, PageLayout, + ContentArea, NetworkMapBackground). +- **`ui/src/components/domain/`** - Domain-specific components (blocks, cards, + pickers). - **`ui/src/hooks/`** - Data fetching, state, keyboard navigation hooks. - **`ui/src/lib/`** - `cn.ts`, `auth.tsx`, `permissions.ts`, `features.ts`. - **`ui/embed.go`** - `//go:embed dist/*` directive exposing `ui.Assets`. -- **`internal/controller/api/ui/`** - SPA serving handler (static files + `index.html` fallback for client-side routing). +- **`internal/controller/api/ui/`** - SPA serving handler (static files + + `index.html` fallback for client-side routing). - Config: `controller.ui.enabled` in `osapi.yaml` (default `true`). -- `//go:embed dist/*` requires `ui/dist/` to have files at compile time. Always use `just build` / `just test` / `just ready` — these run `just react-build` first. Running `go build` / `go test` directly without a prior UI build will fail. +- `//go:embed dist/*` requires `ui/dist/` to have files at compile time. Always + use `just build` / `just test` / `just ready` — these run `just react-build` + first. Running `go build` / `go test` directly without a prior UI build will + fail. ## UI Conventions (MANDATORY) -- One component per file. Use `cva` for variants, `cn()` for conditional classes. +- One component per file. Use `cva` for variants, `cn()` for conditional + classes. - Icons from lucide-react only. No inline styles — Tailwind only. -- Always use the `Text` component for styled text. Always use `Dropdown` — never ``. - Tailwind scale only (`text-xs`, `text-sm`). Never arbitrary pixel values. -- File naming: components `kebab-case.tsx`, hooks `use-kebab-case.ts`, utilities `kebab-case.ts`. -- Colors defined in `ui/src/index.css` via Tailwind v4 `@theme`. Never use raw hex values. -- **Block** = single operation (command, file deploy, cron create, etc.); **Stack** = saved composition of one or more blocks with targets. Do NOT use "runlist" — that term has been replaced. -- Regenerate the TypeScript SDK via `just generate` at the repository root (runs `redocly join`, `go generate`, copies the combined spec to `ui/src/sdk/gen/api.yaml`, and runs `orval`). +- File naming: components `kebab-case.tsx`, hooks `use-kebab-case.ts`, utilities + `kebab-case.ts`. +- Colors defined in `ui/src/index.css` via Tailwind v4 `@theme`. Never use raw + hex values. +- **Block** = single operation (command, file deploy, cron create, etc.); + **Stack** = saved composition of one or more blocks with targets. Do NOT use + "runlist" — that term has been replaced. +- Regenerate the TypeScript SDK via `just generate` at the repository root (runs + `redocly join`, `go generate`, copies the combined spec to + `ui/src/sdk/gen/api.yaml`, and runs `orval`). ## Adding a New API Domain -When adding a new domain, follow existing domains as reference. -Node-targeted operations live under -`internal/controller/api/node/{domain}/`. Controller-only operations -live under `internal/controller/api/{domain}/`. Read existing -domains before creating new ones — the codebase IS the reference. +When adding a new domain, follow existing domains as reference. Node-targeted +operations live under `internal/controller/api/node/{domain}/`. Controller-only +operations live under `internal/controller/api/{domain}/`. Read existing domains +before creating new ones — the codebase IS the reference. ### Cross-Layer Consistency (MANDATORY) -Every domain MUST be consistent across all layers: provider, agent -processor, API handler, SDK service, CLI commands, docs, and tests. -When adding a new domain, look at a recently completed domain (like -`ntp` or `sysctl`) and replicate the same set of artifacts across -every layer. If something exists for `sysctl` but not for your new -domain, it's missing. +Every domain MUST be consistent across all layers: provider, agent processor, +API handler, SDK service, CLI commands, docs, and tests. When adding a new +domain, look at a recently completed domain (like `ntp` or `sysctl`) and +replicate the same set of artifacts across every layer. If something exists for +`sysctl` but not for your new domain, it's missing. -The principle: **pick any existing domain and `find`/`grep` for it -across the codebase. Your new domain should appear in all the same -places.** This includes code, tests, examples, SDK docs, CLI docs, -feature docs, docusaurus config, and permissions tables. +The principle: **pick any existing domain and `find`/`grep` for it across the +codebase. Your new domain should appear in all the same places.** This includes +code, tests, examples, SDK docs, CLI docs, feature docs, docusaurus config, and +permissions tables. ### Step 0: Provider Implementation -Providers are the operations layer — they execute the actual work on -agent hosts. Every operation under `/node/{hostname}/...` is backed -by a provider. The request flows: +Providers are the operations layer — they execute the actual work on agent +hosts. Every operation under `/node/{hostname}/...` is backed by a provider. The +request flows: ``` CLI → SDK → REST API → Job Client → NATS → Agent → Provider ``` -The provider runs on the agent, not the controller. It receives -parameters from the job payload and returns a result. +The provider runs on the agent, not the controller. It receives parameters from +the job payload and returns a result. #### Provider Types -Three provider patterns exist. Check existing providers for -examples of each: +Three provider patterns exist. Check existing providers for examples of each: -**Direct providers** interact with the system directly via -commands or system calls. No file management. +**Direct providers** interact with the system directly via commands or system +calls. No file management. -**Meta providers** delegate file writes to `file.Deployer` for -SHA tracking, idempotency, and template rendering. +**Meta providers** delegate file writes to `file.Deployer` for SHA tracking, +idempotency, and template rendering. -**Direct-write providers** manage their own config files via -`avfs.VFS` without `file.Deployer`. They use the `osapi-` filename -prefix to identify managed files. +**Direct-write providers** manage their own config files via `avfs.VFS` without +`file.Deployer`. They use the `osapi-` filename prefix to identify managed +files. Meta providers depend on `file.Deployer` (the narrow interface): + ```go type Deployer interface { Deploy(ctx context.Context, req DeployRequest) (*DeployResult, error) @@ -137,16 +181,17 @@ type Deployer interface { } ``` -Meta providers store domain-specific metadata in the -`FileState.Metadata` map (e.g., schedule, interval, user for cron). -The file provider persists this in the file-state KV alongside SHA, -path, and mode — one KV bucket for all providers. +Meta providers store domain-specific metadata in the `FileState.Metadata` map +(e.g., schedule, interval, user for cron). The file provider persists this in +the file-state KV alongside SHA, path, and mode — one KV bucket for all +providers. Reference: look at existing meta providers in the codebase. #### File Structure Platform-specific providers: + ``` internal/provider/{category}/{domain}/ types.go — Provider interface + domain types @@ -160,6 +205,7 @@ internal/provider/{category}/{domain}/ ``` SDK-based providers (no platform variants): + ``` internal/provider/{category}/{domain}/ types.go — Provider interface + domain types @@ -169,9 +215,9 @@ internal/provider/{category}/{domain}/ generate.go — //go:generate mockgen directive ``` -For top-level providers: `internal/provider/{domain}/`. -For categorized providers: `internal/provider/{category}/{domain}/`. -Look at existing providers to see both patterns. +For top-level providers: `internal/provider/{domain}/`. For categorized +providers: `internal/provider/{category}/{domain}/`. Look at existing providers +to see both patterns. #### Provider Interface @@ -186,32 +232,32 @@ type Provider interface { } ``` -Every method takes `context.Context` as the first parameter. -Result types include `Changed bool` for mutations and `Error string` -for per-operation error reporting. +Every method takes `context.Context` as the first parameter. Result types +include `Changed bool` for mutations and `Error string` for per-operation error +reporting. #### Idempotency (MANDATORY) -All provider mutations follow Ansible-style desired-state semantics. -Operations MUST be idempotent: +All provider mutations follow Ansible-style desired-state semantics. Operations +MUST be idempotent: -| Operation | Resource exists | Resource absent | -| ---------- | --------------------- | ---------------------- | -| **Create** | `Changed: false`, nil | Creates it | -| **Update** | Updates it | Error (not found) | -| **Delete** | Removes it | `Changed: false`, nil | +| Operation | Resource exists | Resource absent | +| ---------- | --------------------- | --------------------- | +| **Create** | `Changed: false`, nil | Creates it | +| **Update** | Updates it | Error (not found) | +| **Delete** | Removes it | `Changed: false`, nil | - **Create** when the resource already exists returns success with `Changed: false` — the desired state (present) is already met. - **Delete** when the resource doesn't exist returns success with `Changed: false` — the desired state (absent) is already met. -- **Update** when the resource doesn't exist returns an error — - there is nothing to update. -- `ErrUnsupported` (wrong OS family) maps to `StatusSkipped` at - the agent layer, which is distinct from `Changed: false`. +- **Update** when the resource doesn't exist returns an error — there is nothing + to update. +- `ErrUnsupported` (wrong OS family) maps to `StatusSkipped` at the agent layer, + which is distinct from `Changed: false`. -Every concrete provider struct MUST embed `provider.FactsAware` and -include a compile-time check: +Every concrete provider struct MUST embed `provider.FactsAware` and include a +compile-time check: ```go // Compile-time check: Debian must satisfy FactsSetter. @@ -224,21 +270,21 @@ type Debian struct { } ``` -The provider must also be passed to `provider.WireProviderFacts()` -in `internal/agent/agent.go` so facts are injected at startup. +The provider must also be passed to `provider.WireProviderFacts()` in +`internal/agent/agent.go` so facts are injected at startup. #### Platform-Specific Implementations -OSAPI follows Ansible's OS family naming. Implementations are -selected at runtime via `platform.Detect()`: +OSAPI follows Ansible's OS family naming. Implementations are selected at +runtime via `platform.Detect()`: - `debian.go` — Debian family (Ubuntu, Debian, Raspbian) - `darwin.go` — macOS (for development) - `linux.go` — generic Linux fallback -Unsupported platforms return `provider.ErrUnsupported`. The agent -marks the job as `skipped` (not `failed`) so the caller knows the -operation isn't available on that host rather than broken. +Unsupported platforms return `provider.ErrUnsupported`. The agent marks the job +as `skipped` (not `failed`) so the caller knows the operation isn't available on +that host rather than broken. ```go // darwin.go @@ -251,36 +297,36 @@ func (d *Darwin) List( #### Provider Naming Conventions -There are three provider implementation patterns. The naming -convention determines the struct name, constructor, and file layout. +There are three provider implementation patterns. The naming convention +determines the struct name, constructor, and file layout. **1. Platform-specific providers** (most common) -One struct per OS family, each in its own file. Constructor names -follow `New{Platform}Provider()`. Methods that are large or testable -go in separate files named `{platform}_{operation}.go`. +One struct per OS family, each in its own file. Constructor names follow +`New{Platform}Provider()`. Methods that are large or testable go in separate +files named `{platform}_{operation}.go`. -| Struct | Constructor | File(s) | -| -------- | ------------------------ | -------------------------------- | -| `Debian` | `NewDebianProvider(...)` | `debian.go`, `debian_get_*.go` | -| `Darwin` | `NewDarwinProvider(...)` | `darwin.go`, `darwin_get_*.go` | -| `Linux` | `NewLinuxProvider()` | `linux.go`, `linux_get_*.go` | +| Struct | Constructor | File(s) | +| -------- | ------------------------ | ------------------------------ | +| `Debian` | `NewDebianProvider(...)` | `debian.go`, `debian_get_*.go` | +| `Darwin` | `NewDarwinProvider(...)` | `darwin.go`, `darwin_get_*.go` | +| `Linux` | `NewLinuxProvider()` | `linux.go`, `linux_get_*.go` | Most providers under `node/` and `network/` follow this pattern. **2. Container-aware platform providers** -When a provider's behavior differs inside a Docker container (e.g., -hostname is read-only, DNS uses `/etc/resolv.conf` instead of -`resolvectl`), add a `DebianDocker` variant alongside the regular -`Debian` struct. The agent selects it via `platform.IsContainer()`. +When a provider's behavior differs inside a Docker container (e.g., hostname is +read-only, DNS uses `/etc/resolv.conf` instead of `resolvectl`), add a +`DebianDocker` variant alongside the regular `Debian` struct. The agent selects +it via `platform.IsContainer()`. -| Struct | Constructor | File(s) | -| --------------- | ------------------------------- | ------------------------------------ | -| `DebianDocker` | `NewDebianDockerProvider(...)` | `debian_docker.go`, `debian_docker_*.go` | +| Struct | Constructor | File(s) | +| -------------- | ------------------------------ | ---------------------------------------- | +| `DebianDocker` | `NewDebianDockerProvider(...)` | `debian_docker.go`, `debian_docker_*.go` | -`DebianDocker` either embeds `Debian` (delegating reads, overriding -writes) or stands alone. It satisfies the same `Provider` interface. +`DebianDocker` either embeds `Debian` (delegating reads, overriding writes) or +stands alone. It satisfies the same `Provider` interface. ```go // agent_setup.go wiring @@ -292,21 +338,20 @@ case "debian": } ``` -Examples: `node/host` (embeds `Debian`, blocks `UpdateHostname`), -`network/dns` (standalone, reads `/etc/resolv.conf` directly). +Examples: `node/host` (embeds `Debian`, blocks `UpdateHostname`), `network/dns` +(standalone, reads `/etc/resolv.conf` directly). **3. SDK-based providers** (no platform variants) -Providers that talk to an external API (not the OS) use a single -`Client` struct with `New()` / `NewWithClient()` constructors. -No `debian.go` / `darwin.go` / `linux.go` files — the provider -works the same on all platforms. Availability is checked at startup -(e.g., Docker daemon ping). +Providers that talk to an external API (not the OS) use a single `Client` struct +with `New()` / `NewWithClient()` constructors. No `debian.go` / `darwin.go` / +`linux.go` files — the provider works the same on all platforms. Availability is +checked at startup (e.g., Docker daemon ping). -| Struct | Constructor | File(s) | -| -------- | ------------------- | -------------------- | -| `Client` | `New()` | `docker.go` | -| | `NewWithClient(c)` | (same file, testing) | +| Struct | Constructor | File(s) | +| -------- | ------------------ | -------------------- | +| `Client` | `New()` | `docker.go` | +| | `NewWithClient(c)` | (same file, testing) | ```go // agent_setup.go wiring — no platform switch @@ -322,9 +367,9 @@ Examples: `container/docker`. #### FactsAware -Embed `provider.FactsAware` in the provider struct to access agent -facts (OS family, architecture, hostname, network interfaces) at -runtime. The agent wires facts via `provider.WireProviderFacts()`. +Embed `provider.FactsAware` in the provider struct to access agent facts (OS +family, architecture, hostname, network interfaces) at runtime. The agent wires +facts via `provider.WireProviderFacts()`. ```go type Debian struct { @@ -334,19 +379,19 @@ type Debian struct { } ``` -Facts are available in template rendering via `{{ .Facts.os_family }}` -when using the file provider's template support. +Facts are available in template rendering via `{{ .Facts.os_family }}` when +using the file provider's template support. #### Agent Wiring Two files connect a provider to the agent: -1. **`internal/agent/processor_{domain}.go`** — create helper - functions that dispatch sub-operations to the provider. If the - domain gets its own category (like `schedule`, `docker`), create - a `NewXxxProcessor` factory. If the domain belongs under an - existing category (like `node`), add a `case` to that category's - processor and delegate to helpers in a new file: +1. **`internal/agent/processor_{domain}.go`** — create helper functions that + dispatch sub-operations to the provider. If the domain gets its own category + (like `schedule`, `docker`), create a `NewXxxProcessor` factory. If the + domain belongs under an existing category (like `node`), add a `case` to that + category's processor and delegate to helpers in a new file: + ```go // processor_{domain}.go func process{Domain}Operation( @@ -358,10 +403,11 @@ Two files connect a provider to the agent: } ``` -2. **`cmd/agent_setup.go`** — create the provider and register it - with the `ProviderRegistry`. For new categories, use a separate - `Register` call. For existing categories (e.g., `node`), pass - the provider to the existing processor factory: +2. **`cmd/agent_setup.go`** — create the provider and register it with the + `ProviderRegistry`. For new categories, use a separate `Register` call. For + existing categories (e.g., `node`), pass the provider to the existing + processor factory: + ```go // New category example (like schedule, docker): registry.Register("mydomain", @@ -374,34 +420,30 @@ Two files connect a provider to the agent: // Read cmd/agent_setup.go to see the current parameter list. ``` -That's it. No changes to `agent/types.go`, `agent/agent.go`, or -the `JobClient` interface. The registry handles dispatch and -FactsAware wiring automatically. +That's it. No changes to `agent/types.go`, `agent/agent.go`, or the `JobClient` +interface. The registry handles dispatch and FactsAware wiring automatically. #### Provider Testing -- **Filesystem:** Use `avfs` — `memfs.New()` for in-memory, - `failfs.New()` for targeted error injection. Never use `afero`. -- **Mocks:** Use gomock for all interfaces (`FileDeployer`, - `KeyValue`, `ObjectStore`). Generated mocks live in - `{package}/mocks/`. -- **Platform stubs:** Test that Darwin and Linux stubs return - `ErrUnsupported` for every method. -- **export_test.go:** Use for testing unexported variable swaps - (e.g., `marshalJSON`). Public tests import via the bridge. -- **Table-driven:** One suite method per provider method, all - scenarios as rows. +- **Filesystem:** Use `avfs` — `memfs.New()` for in-memory, `failfs.New()` for + targeted error injection. Never use `afero`. +- **Mocks:** Use gomock for all interfaces (`FileDeployer`, `KeyValue`, + `ObjectStore`). Generated mocks live in `{package}/mocks/`. +- **Platform stubs:** Test that Darwin and Linux stubs return `ErrUnsupported` + for every method. +- **export_test.go:** Use for testing unexported variable swaps (e.g., + `marshalJSON`). Public tests import via the bridge. +- **Table-driven:** One suite method per provider method, all scenarios as rows. ### Step 1: OpenAPI Spec + Code Generation -For node-targeted domains, create -`internal/controller/api/node/{domain}/gen/` with three hand-written -files. For controller-only domains, create +For node-targeted domains, create `internal/controller/api/node/{domain}/gen/` +with three hand-written files. For controller-only domains, create `internal/controller/api/{domain}/gen/` instead: - `api.yaml` — OpenAPI spec with paths, schemas, and `BearerAuth` security -- `cfg.yaml` — oapi-codegen config (`strict-server: true`, import-mapping - for `common/gen`) +- `cfg.yaml` — oapi-codegen config (`strict-server: true`, import-mapping for + `common/gen`) - `generate.go` — `//go:generate` directive #### HTTP Verb Conventions @@ -413,20 +455,19 @@ Mutable domains MUST use separate verbs for create and update: - `GET` — read/list resources - `DELETE` — remove a resource -Do NOT combine create and update into a single "set" or "upsert" -endpoint. The cron domain is the reference: `POST` creates, -`PUT /{name}` updates. This separation gives clear 404 semantics -(update fails if not found, create fails if already exists) and -matches REST conventions. +Do NOT combine create and update into a single "set" or "upsert" endpoint. The +cron domain is the reference: `POST` creates, `PUT /{name}` updates. This +separation gives clear 404 semantics (update fails if not found, create fails if +already exists) and matches REST conventions. #### Validation in OpenAPI Specs -The OpenAPI spec is the **source of truth** for input validation. All user -input must be validated, and the spec must declare how: +The OpenAPI spec is the **source of truth** for input validation. All user input +must be validated, and the spec must declare how: -- **Request body properties**: Add `x-oapi-codegen-extra-tags` with - `validate:` tags. These generate Go struct tags that - `validation.Struct()` enforces at runtime. +- **Request body properties**: Add `x-oapi-codegen-extra-tags` with `validate:` + tags. These generate Go struct tags that `validation.Struct()` enforces at + runtime. ```yaml properties: address: @@ -434,10 +475,10 @@ input must be validated, and the spec must declare how: x-oapi-codegen-extra-tags: validate: required,ip ``` -- **Path parameters (UUID)**: Use `format: uuid` on the schema. This - causes oapi-codegen to generate `openapi_types.UUID` type, and the - router validates the format before the handler runs. No manual - validation needed in the handler. +- **Path parameters (UUID)**: Use `format: uuid` on the schema. This causes + oapi-codegen to generate `openapi_types.UUID` type, and the router validates + the format before the handler runs. No manual validation needed in the + handler. ```yaml parameters: - name: id @@ -447,11 +488,10 @@ input must be validated, and the spec must declare how: type: string format: uuid ``` -- **Query parameters**: Place `x-oapi-codegen-extra-tags` at the - **parameter level** (sibling of `name`/`in`/`schema`), NOT inside - `schema:`. At parameter level, oapi-codegen generates `validate:` - tags on the `*Params` struct fields. Use `enum` for constrained - string values (generates `oneof` validation). +- **Query parameters**: Place `x-oapi-codegen-extra-tags` at the **parameter + level** (sibling of `name`/`in`/`schema`), NOT inside `schema:`. At parameter + level, oapi-codegen generates `validate:` tags on the `*Params` struct fields. + Use `enum` for constrained string values (generates `oneof` validation). ```yaml parameters: - name: limit @@ -471,32 +511,32 @@ input must be validated, and the spec must declare how: return gen.GetFoo400JSONResponse{Error: &errMsg}, nil } ``` - **NOTE:** `x-oapi-codegen-extra-tags` on **path parameters** does - NOT generate tags on `RequestObject` structs in **strict-server - mode** (upstream limitation — see oapi-codegen issue). Keep the - `x-oapi-codegen-extra-tags` in the spec for documentation and add - a YAML comment noting validation is handled manually. Path params - that need validation beyond `format: uuid` (e.g., `valid_target`) - use a shared helper like `node.validateHostname()` which calls + **NOTE:** `x-oapi-codegen-extra-tags` on **path parameters** does NOT generate + tags on `RequestObject` structs in **strict-server mode** (upstream limitation + — see oapi-codegen issue). Keep the `x-oapi-codegen-extra-tags` in the spec + for documentation and add a YAML comment noting validation is handled + manually. Path params that need validation beyond `format: uuid` (e.g., + `valid_target`) use a shared helper like `node.validateHostname()` which calls `validation.Var()`. **IMPORTANT — every endpoint with user input MUST have:** -1. `x-oapi-codegen-extra-tags` with `validate:` tags on all request - body properties and query params in the OpenAPI spec + +1. `x-oapi-codegen-extra-tags` with `validate:` tags on all request body + properties and query params in the OpenAPI spec 2. `validation.Struct(request.Params)` in the handler for query params, `validation.Struct(request.Body)` for request bodies -3. A `400` response defined in the OpenAPI spec for endpoints that - accept user input +3. A `400` response defined in the OpenAPI spec for endpoints that accept user + input 4. HTTP wiring tests (`TestXxxHTTP` / `TestXxxRBACHTTP` methods in the - `*_public_test.go` suite) that send raw HTTP through the full Echo - middleware stack and verify: + `*_public_test.go` suite) that send raw HTTP through the full Echo middleware + stack and verify: - Validation errors return correct status codes and error messages - RBAC: 401 (no token), 403 (wrong permissions), 200 (valid token) -**Defense-in-depth validation**: When validation calls cannot -currently fail (e.g., all fields use `omitempty`), keep the call -but add a comment explaining why. This guards against future field -additions breaking validation silently: +**Defense-in-depth validation**: When validation calls cannot currently fail +(e.g., all fields use `omitempty`), keep the call but add a comment explaining +why. This guards against future field additions breaking validation silently: + ```go // Defense in depth: current fields use omitempty so validation // always passes, but guards against future field additions. @@ -507,36 +547,33 @@ if errMsg, ok := validation.Struct(request.Body); !ok { ### Step 2: Handler Implementation -For node-targeted domains, create -`internal/controller/api/node/{domain}/`. For controller-only -domains, create `internal/controller/api/{domain}/`: +For node-targeted domains, create `internal/controller/api/node/{domain}/`. For +controller-only domains, create `internal/controller/api/{domain}/`: - `types.go` — domain struct, dependency interfaces (e.g., `Checker`) - `{domain}.go` — `New()` factory, compile-time interface check: `var _ gen.StrictServerInterface = (*Domain)(nil)` -- One file per endpoint (e.g., `{operation}_get.go`). Every handler - that accepts user input MUST call `validation.Struct()` and return - a 400 on failure. -- Tests: `{operation}_get_public_test.go` (testify/suite, table-driven). - Must cover validation failures (400), success, and error paths. - Each public test suite also includes HTTP wiring methods: - - `TestXxxHTTP` — sends raw HTTP through the full Echo middleware - stack to verify validation (valid input, invalid input → 400). - - `TestXxxRBACHTTP` — verifies auth middleware: no token (401), - wrong permissions (403), valid token (200). Uses `api.New()` + - `{domain}.Handler()` + `server.RegisterHandlers()` to wire - through `ScopeMiddleware`. - Follow existing handler test files in the codebase. +- One file per endpoint (e.g., `{operation}_get.go`). Every handler that accepts + user input MUST call `validation.Struct()` and return a 400 on failure. +- Tests: `{operation}_get_public_test.go` (testify/suite, table-driven). Must + cover validation failures (400), success, and error paths. Each public test + suite also includes HTTP wiring methods: + - `TestXxxHTTP` — sends raw HTTP through the full Echo middleware stack to + verify validation (valid input, invalid input → 400). + - `TestXxxRBACHTTP` — verifies auth middleware: no token (401), wrong + permissions (403), valid token (200). Uses `api.New()` + + `{domain}.Handler()` + `server.RegisterHandlers()` to wire through + `ScopeMiddleware`. Follow existing handler test files in the codebase. #### Broadcast Support (MANDATORY for node-targeted operations) -Every operation under `/node/{hostname}/...` MUST support broadcast -targeting (`_all`, `_any`, hostname, label selectors). The handler -checks `job.IsBroadcastTarget(hostname)` and routes to a broadcast -function. Both single-target and broadcast paths return the same -collection response shape. +Every operation under `/node/{hostname}/...` MUST support broadcast targeting +(`_all`, `_any`, hostname, label selectors). The handler checks +`job.IsBroadcastTarget(hostname)` and routes to a broadcast function. Both +single-target and broadcast paths return the same collection response shape. **Response pattern** — all node-targeted operations return: + ```json { "job_id": "...", @@ -547,11 +584,12 @@ collection response shape. } ``` -Every result item MUST have `hostname` and `error` fields. -Single-target returns 1 result; broadcast returns N results. -Failed/skipped agents appear as entries with `error` set. +Every result item MUST have `hostname` and `error` fields. Single-target returns +1 result; broadcast returns N results. Failed/skipped agents appear as entries +with `error` set. **Handler pattern:** + ```go func (s *Handler) PostOperation(ctx, request) { validate(request) @@ -563,10 +601,11 @@ func (s *Handler) PostOperation(ctx, request) { } ``` -**Job client** — the `JobClient` interface has 4 generic methods: -`Query`, `QueryBroadcast`, `Modify`, `ModifyBroadcast`. Handlers -call these with a category string and operation constant. No new -methods are needed when adding operations. Example: +**Job client** — the `JobClient` interface has 4 generic methods: `Query`, +`QueryBroadcast`, `Modify`, `ModifyBroadcast`. Handlers call these with a +category string and operation constant. No new methods are needed when adding +operations. Example: + ```go jobID, resp, err := s.JobClient.Modify( ctx, hostname, "node", job.OperationSysctlCreate, data) @@ -576,9 +615,9 @@ Read existing handlers in the codebase for reference. ### Step 3: Handler Registration -Each domain package exports a `Handler()` function that creates the -handler, wraps it with auth middleware, and returns route -registration closures. No changes to the `Server` struct are needed. +Each domain package exports a `Handler()` function that creates the handler, +wraps it with auth middleware, and returns route registration closures. No +changes to the `Server` struct are needed. Create `handler.go` in your domain package: @@ -614,13 +653,12 @@ func Handler( } ``` -Add a `handler_public_test.go` that tests route registration and -middleware execution. Follow existing domain handler tests. +Add a `handler_public_test.go` that tests route registration and middleware +execution. Follow existing domain handler tests. ### Step 4: Startup Wiring -- `cmd/controller_setup.go` — add one line to - `registerControllerHandlers`: +- `cmd/controller_setup.go` — add one line to `registerControllerHandlers`: ```go handlers = append(handlers, {domain}API.Handler(log, jc, signingKey, customRoles)...) @@ -632,14 +670,14 @@ middleware execution. Follow existing domain handler tests. The SDK client library lives in `pkg/sdk/client/`. Its generated HTTP client uses the same combined OpenAPI spec as the server (`internal/controller/api/gen/api.yaml`). Follow the rules in -@docs/docs/sidebar/sdk/guidelines.md — especially: never expose `gen` -types in public method signatures, add JSON tags to all result types, -and wrap errors with context. +@docs/docs/sidebar/sdk/guidelines.md — especially: never expose `gen` types in +public method signatures, add JSON tags to all result types, and wrap errors +with context. **When modifying existing API specs:** -1. Make changes to the domain's `gen/api.yaml` (under `api/node/{domain}/` - for node-targeted domains or `api/{domain}/` for controller-only domains) +1. Make changes to the domain's `gen/api.yaml` (under `api/node/{domain}/` for + node-targeted domains or `api/{domain}/` for controller-only domains) 2. Run `just generate` to regenerate server code (this also regenerates the combined spec via `redocly join`) 3. Run `go generate ./pkg/sdk/client/gen/...` to regenerate the SDK client @@ -653,49 +691,48 @@ and wrap errors with context. - `{service}.go` — `{Service}Service` struct + methods - `{service}_types.go` — SDK result types + gen→SDK conversions - `{service}_public_test.go` — service method tests - - `{service}_types_public_test.go` — conversion function tests - Each service gets its own files — do NOT add methods or types to - an existing service's files. -2. Add a field to the `Client` struct in `osapi.go` and wire it - in `New()` -3. Run `go generate ./pkg/sdk/client/gen/...` to pick up the new - domain's spec from the combined `api.yaml` -4. Add an SDK example in `examples/sdk/client/{service}.go` — one - file per SDK service (e.g., `hostname.go`, `disk.go`, `ntp.go`). - The example file name matches the Client field name in lowercase. -5. Add an SDK doc page under the appropriate category subdirectory - in `docs/docs/sidebar/sdk/client/`. SDK docs are grouped by - concern (e.g., `node-info/`, `system-config/`, `operations/`). - Place the new page in the matching group — look at the existing - directory structure to find the right one. Use the Client field - name as the page title (e.g., `# Power`), NOT the Go struct - name. Update `client.md` to add the service to its category - table. -6. Add the new service to the SDK navbar dropdown in - `docs/docusaurus.config.ts` under the matching category header. - The dropdown is grouped the same way as the sidebar. + - `{service}_types_public_test.go` — conversion function tests Each service + gets its own files — do NOT add methods or types to an existing service's + files. +2. Add a field to the `Client` struct in `osapi.go` and wire it in `New()` +3. Run `go generate ./pkg/sdk/client/gen/...` to pick up the new domain's spec + from the combined `api.yaml` +4. Add an SDK example in `examples/sdk/client/{service}.go` — one file per SDK + service (e.g., `hostname.go`, `disk.go`, `ntp.go`). The example file name + matches the Client field name in lowercase. +5. Add an SDK doc page under the appropriate category subdirectory in + `docs/docs/sidebar/sdk/client/`. SDK docs are grouped by concern (e.g., + `node-info/`, `system-config/`, `operations/`). Place the new page in the + matching group — look at the existing directory structure to find the right + one. Use the Client field name as the page title (e.g., `# Power`), NOT the + Go struct name. Update `client.md` to add the service to its category table. +6. Add the new service to the SDK navbar dropdown in `docs/docusaurus.config.ts` + under the matching category header. The dropdown is grouped the same way as + the sidebar. #### SDK method naming (MANDATORY) -Method names MUST be clean verbs — NEVER repeat the service name. -The service struct already provides the namespace. Stuttering like +Method names MUST be clean verbs — NEVER repeat the service name. The service +struct already provides the namespace. Stuttering like `SysctlService.SysctlGet()` is wrong — use `SysctlService.Get()`. Standard verbs: -| Verb | HTTP | Description | -| ---------- | ---- | ------------------------------------ | -| `List` | GET | List collection | -| `Get` | GET | Get single resource / read state | -| `Create` | POST | Create new resource | -| `Update` | PUT | Update existing resource | -| `Delete` | DEL | Remove resource | +| Verb | HTTP | Description | +| -------- | ---- | -------------------------------- | +| `List` | GET | List collection | +| `Get` | GET | Get single resource / read state | +| `Create` | POST | Create new resource | +| `Update` | PUT | Update existing resource | +| `Delete` | DEL | Remove resource | Rare exceptions for action operations (no persistent resource): + - `Ping.Do()` — one-shot action - `Command.Exec()`, `Command.Shell()` — execute commands Examples: + ```go // GOOD — clean verbs, no stuttering client.Sysctl.Get(ctx, host, key) @@ -711,61 +748,59 @@ client.NTP.NtpCreate(ctx, host, opts) #### SDK example conventions -SDK examples live in `examples/sdk/client/`, one file per SDK -service. Follow the same principles as the orchestrator examples: +SDK examples live in `examples/sdk/client/`, one file per SDK service. Follow +the same principles as the orchestrator examples: -- **One service per file**: demonstrate the service's SDK operations. - Don't mix in other services. -- **Self-contained**: for read-only operations, just call and print. - For mutating operations, cleanup at the start so the example is - repeatable. -- **Print results**: decode and print at least one result so the - example isn't silent. +- **One service per file**: demonstrate the service's SDK operations. Don't mix + in other services. +- **Self-contained**: for read-only operations, just call and print. For + mutating operations, cleanup at the start so the example is repeatable. +- **Print results**: decode and print at least one result so the example isn't + silent. - **Keep it short**: under ~100 lines of code (excluding license). -- **Handle errors inline**: use `log.Fatalf` for unexpected errors. - For operations that may fail on some platforms, check the error - and print a message instead of crashing. +- **Handle errors inline**: use `log.Fatalf` for unexpected errors. For + operations that may fail on some platforms, check the error and print a + message instead of crashing. ### Step 6: CLI Commands - `cmd/client_node_{domain}.go` — parent command registered under `clientNodeCmd` (for node-targeted domains) -- `cmd/client_node_{domain}_{operation}.go` — one subcommand per - endpoint (e.g., `client_node_sysctl_get.go`) +- `cmd/client_node_{domain}_{operation}.go` — one subcommand per endpoint (e.g., + `client_node_sysctl_get.go`) - All commands support `--json` for raw output -- Use `printKV` for inline key-value output and `printStyledTable` for - multi-row tabular data (both in `cmd/ui.go`) -- Use flags (e.g., `--job-id`, `--audit-id`) instead of positional args - for resource IDs -- Handle **all** API response codes in the `switch resp.StatusCode()` - block: 200, 400 (`handleUnknownError`), 401/403 (`handleAuthError`), - 404 (`handleUnknownError`), 500 (`handleUnknownError`). Match the - responses declared in the OpenAPI spec. +- Use `printKV` for inline key-value output and `printStyledTable` for multi-row + tabular data (both in `cmd/ui.go`) +- Use flags (e.g., `--job-id`, `--audit-id`) instead of positional args for + resource IDs +- Handle **all** API response codes in the `switch resp.StatusCode()` block: + 200, 400 (`handleUnknownError`), 401/403 (`handleAuthError`), 404 + (`handleUnknownError`), 500 (`handleUnknownError`). Match the responses + declared in the OpenAPI spec. ### Step 7: Documentation -- `docs/docs/sidebar/features/{domain}-management.md` — feature page. - Follow existing feature pages for the template. -- `docs/docs/sidebar/usage/cli/client/node/{domain}/{domain}.md` — - CLI landing page with `` -- `docs/docs/sidebar/usage/cli/client/node/{domain}/{verb}.md` — one - page per CLI subcommand (e.g., `get.md`, `create.md`, `update.md`) +- `docs/docs/sidebar/features/{domain}-management.md` — feature page. Follow + existing feature pages for the template. +- `docs/docs/sidebar/usage/cli/client/node/{domain}/{domain}.md` — CLI landing + page with `` +- `docs/docs/sidebar/usage/cli/client/node/{domain}/{verb}.md` — one page per + CLI subcommand (e.g., `get.md`, `create.md`, `update.md`) - Update `docs/docusaurus.config.ts`: - Add the new feature to the "Features" navbar dropdown - Add the new SDK service to the "SDK" → "Client Library" dropdown -- Update `docs/docs/sidebar/features/features.md` — add the new - domain to the features landing page table -- Update `docs/docs/sidebar/usage/configuration.md` — add any new - permissions to the roles table and permissions comments in the - YAML reference -- Update `docs/docs/sidebar/features/authentication.md` — add new - permissions to the roles/permissions tables -- Update `docs/docs/sidebar/architecture/architecture.md` — add link - to the new feature page in the features list -- Update `docs/docs/sidebar/architecture/api-guidelines.md` — add - new endpoints to the path pattern table -- Update `docs/docs/sidebar/architecture/system-architecture.md` — add - endpoints to the health/endpoint tables if applicable +- Update `docs/docs/sidebar/features/features.md` — add the new domain to the + features landing page table +- Update `docs/docs/sidebar/usage/configuration.md` — add any new permissions to + the roles table and permissions comments in the YAML reference +- Update `docs/docs/sidebar/features/authentication.md` — add new permissions to + the roles/permissions tables +- Update `docs/docs/sidebar/architecture/architecture.md` — add link to the new + feature page in the features list +- Update `docs/docs/sidebar/architecture/api-guidelines.md` — add new endpoints + to the path pattern table +- Update `docs/docs/sidebar/architecture/system-architecture.md` — add endpoints + to the health/endpoint tables if applicable ### Step 8: Verify @@ -781,6 +816,7 @@ just go-vet # lint passes ### Function Signatures ALL function signatures MUST use multi-line format: + ```go func FunctionName( param1 type1, @@ -792,52 +828,51 @@ func FunctionName( ### Testing Three test layers: -- **Unit tests** (`*_test.go`, `*_public_test.go`) — fast, mocked - dependencies, run with `just go-unit`. Includes `TestXxxHTTP` / - `TestXxxRBACHTTP` methods that send raw HTTP through real Echo - middleware with mocked backends. -- **Integration tests** (`test/integration/`) — build and start a real - `osapi` binary, exercise CLI commands end-to-end. Guarded by - `//go:build integration` tag, run with `just go-unit-int`. New API - domains should include a `{domain}_test.go` smoke suite. Write tests - (mutations) must be guarded by `skipWrite(s.T())` so CI can run - read-only tests by default (`OSAPI_INTEGRATION_WRITES=1` enables - writes). + +- **Unit tests** (`*_test.go`, `*_public_test.go`) — fast, mocked dependencies, + run with `just go-unit`. Includes `TestXxxHTTP` / `TestXxxRBACHTTP` methods + that send raw HTTP through real Echo middleware with mocked backends. +- **Integration tests** (`test/integration/`) — build and start a real `osapi` + binary, exercise CLI commands end-to-end. Guarded by `//go:build integration` + tag, run with `just go-unit-int`. New API domains should include a + `{domain}_test.go` smoke suite. Write tests (mutations) must be guarded by + `skipWrite(s.T())` so CI can run read-only tests by default + (`OSAPI_INTEGRATION_WRITES=1` enables writes). Conventions: + - ALL tests MUST use `testify/suite` with table-driven patterns -- Public tests: `*_public_test.go` in test package (e.g., - `package job_test`) for exported functions. This is the default — - all new tests should be public tests. +- Public tests: `*_public_test.go` in test package (e.g., `package job_test`) + for exported functions. This is the default — all new tests should be public + tests. - Suite naming: `*_public_test.go` → `{Name}PublicTestSuite` - Table-driven structure with `validateFunc` callbacks -- One suite method per function under test — all scenarios (success, - errors, edge cases) as rows in one table -- Avoid generic file names like `helpers.go` or `utils.go` — name - files after what they contain -- **`types.go` is for types only**: `types.go` files MUST contain - only type definitions (structs, interfaces, constants, type - aliases). Never put functions or methods in `types.go` — put - them in a file named after what they do (e.g., `nats.go` for - NATS config methods, `options.go` for option functions). -- **Test file naming**: every test file MUST have a corresponding - production file with a matching name. `foo_public_test.go` tests - `foo.go`. Never create test files with names that don't match a - production file (e.g., don't create `check_error_public_test.go` - if the code lives in `response.go` — name it - `response_public_test.go`). If a production file is too large and - you want to split tests by concern, split the production file - first (e.g., `agent.go` → `agent.go` + `agent_drain.go` + - `agent_timeline.go`), then create matching test files. +- One suite method per function under test — all scenarios (success, errors, + edge cases) as rows in one table +- Avoid generic file names like `helpers.go` or `utils.go` — name files after + what they contain +- **`types.go` is for types only**: `types.go` files MUST contain only type + definitions (structs, interfaces, constants, type aliases). Never put + functions or methods in `types.go` — put them in a file named after what they + do (e.g., `nats.go` for NATS config methods, `options.go` for option + functions). +- **Test file naming**: every test file MUST have a corresponding production + file with a matching name. `foo_public_test.go` tests `foo.go`. Never create + test files with names that don't match a production file (e.g., don't create + `check_error_public_test.go` if the code lives in `response.go` — name it + `response_public_test.go`). If a production file is too large and you want to + split tests by concern, split the production file first (e.g., `agent.go` → + `agent.go` + `agent_drain.go` + `agent_timeline.go`), then create matching + test files. #### Mocking -- **Always use gomock** (`go:generate mockgen`) for interface mocks. - Generated mocks live in `{package}/mocks/` directories alongside - their source interfaces. Never hand-roll mock structs. -- **export_test.go pattern** for testing unexported internals: create - an `export_test.go` file in the production package that exposes - unexported variables or functions to the `_test` package: +- **Always use gomock** (`go:generate mockgen`) for interface mocks. Generated + mocks live in `{package}/mocks/` directories alongside their source + interfaces. Never hand-roll mock structs. +- **export_test.go pattern** for testing unexported internals: create an + `export_test.go` file in the production package that exposes unexported + variables or functions to the `_test` package: ```go // export_test.go — package file package file @@ -848,19 +883,19 @@ Conventions: func ResetMarshalJSON() { marshalJSON = json.Marshal } ``` Public tests then call `file.SetMarshalJSON(...)` and - `defer file.ResetMarshalJSON()`. This avoids internal tests, - import cycles, and hand-rolled stubs. -- **TearDownSubTest** — use `suite.TearDownSubTest()` to reset - swapped variables between table-driven sub-tests, not `defer` - inside the loop. + `defer file.ResetMarshalJSON()`. This avoids internal tests, import cycles, + and hand-rolled stubs. +- **TearDownSubTest** — use `suite.TearDownSubTest()` to reset swapped variables + between table-driven sub-tests, not `defer` inside the loop. - **Filesystem testing** — use `avfs` (`memfs.New()` for in-memory, - `failfs.New()` for targeted error injection). Never use - `afero`. The only exception for hand-rolled types is stdlib - interfaces like `fs.FS` or `net.Conn` where gomock is impractical. + `failfs.New()` for targeted error injection). Never use `afero`. The only + exception for hand-rolled types is stdlib interfaces like `fs.FS` or + `net.Conn` where gomock is impractical. ### Go Patterns -- Non-blocking lifecycle: `Start()` returns immediately, `Stop(ctx)` shuts down with deadline +- Non-blocking lifecycle: `Start()` returns immediately, `Stop(ctx)` shuts down + with deadline - Error wrapping: `fmt.Errorf("context: %w", err)` - Early returns over nested if-else - Unused parameters: rename to `_` @@ -870,31 +905,34 @@ Conventions: All logging uses Go's `log/slog` structured logger. Follow these rules: -- **Subsystem labels**: Every component that holds a logger MUST wrap it - with `logger.With(slog.String("subsystem", "..."))` at construction - time. This auto-tags every log line from that component. Examples: - `"agent"`, `"agent.seed"`, `"api.schedule"`, `"provider.file"`, - `"job.client"`, `"metrics"`, `"controller.heartbeat"`. +- **Subsystem labels**: Every component that holds a logger MUST wrap it with + `logger.With(slog.String("subsystem", "..."))` at construction time. This + auto-tags every log line from that component. Examples: `"agent"`, + `"agent.seed"`, `"api.schedule"`, `"provider.file"`, `"job.client"`, + `"metrics"`, `"controller.heartbeat"`. - **Always use typed attributes**: Use `slog.String("key", val)`, - `slog.Int("key", val)`, `slog.Bool("key", val)`, `slog.Any("key", val)`. - Never use positional pairs like `"key", val` — they compile but - bypass type safety and are inconsistent with the codebase. -- **Standard field names**: `error` for errors, `hostname` for hosts, - `path` for file paths, `job_id` for job IDs, `name` for entry names, - `addr` for addresses. -- **Error fields**: Use `slog.String("error", err.Error())` for string - context or `slog.Any("error", err)` to preserve the error type. -- **Log levels**: `Debug` for operation dispatch and idempotency skips, - `Info` for lifecycle events and state changes, `Warn` for degraded - but functional states, `Error` for failures that need attention. + `slog.Int("key", val)`, `slog.Bool("key", val)`, `slog.Any("key", val)`. Never + use positional pairs like `"key", val` — they compile but bypass type safety + and are inconsistent with the codebase. +- **Standard field names**: `error` for errors, `hostname` for hosts, `path` for + file paths, `job_id` for job IDs, `name` for entry names, `addr` for + addresses. +- **Error fields**: Use `slog.String("error", err.Error())` for string context + or `slog.Any("error", err)` to preserve the error type. +- **Log levels**: `Debug` for operation dispatch and idempotency skips, `Info` + for lifecycle events and state changes, `Warn` for degraded but functional + states, `Error` for failures that need attention. ### Linting -golangci-lint with: errcheck, errname, goimports, govet, prealloc, predeclared, revive, staticcheck. Generated files (`*.gen.go`, `*.pb.go`) are excluded from formatting. +golangci-lint with: errcheck, errname, goimports, govet, prealloc, predeclared, +revive, staticcheck. Generated files (`*.gen.go`, `*.pb.go`) are excluded from +formatting. ### Branching -See @docs/docs/sidebar/development/development.md#branching for full conventions. +See @docs/docs/sidebar/development/development.md#branching for full +conventions. When committing changes via `/commit`, create a feature branch first if currently on `main`. Branch names use the pattern `type/short-description` @@ -902,12 +940,14 @@ currently on `main`. Branch names use the pattern `type/short-description` ### Commit Messages -See @docs/docs/sidebar/development/development.md#commit-messages for full conventions. +See @docs/docs/sidebar/development/development.md#commit-messages for full +conventions. Follow [Conventional Commits](https://www.conventionalcommits.org/) with the 50/72 rule. Format: `type(scope): description`. When committing via Claude Code, end with: + - `🤖 Generated with [Claude Code](https://claude.ai/code)` - `Co-Authored-By: Claude ` diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 1012d1564..c0fe38a0b 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -6,8 +6,8 @@ We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, -nationality, personal appearance, race, religion, or sexual identity -and orientation. +nationality, personal appearance, race, religion, or sexual identity and +orientation. We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. @@ -17,23 +17,23 @@ diverse, inclusive, and healthy community. Examples of behavior that contributes to a positive environment for our community include: -* Demonstrating empathy and kindness toward other people -* Being respectful of differing opinions, viewpoints, and experiences -* Giving and gracefully accepting constructive feedback -* Accepting responsibility and apologizing to those affected by our mistakes, +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience -* Focusing on what is best not just for us as individuals, but for the - overall community +- Focusing on what is best not just for us as individuals, but for the overall + community Examples of unacceptable behavior include: -* The use of sexualized language or imagery, and sexual attention or - advances of any kind -* Trolling, insulting or derogatory comments, and personal or political attacks -* Public or private harassment -* Publishing others' private information, such as a physical or email - address, without their explicit permission -* Other conduct which could reasonably be considered inappropriate in a +- The use of sexualized language or imagery, and sexual attention or advances of + any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or email address, + without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting ## Enforcement Responsibilities @@ -59,8 +59,7 @@ representative at an online or offline event. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported to the community leaders responsible for enforcement at -john@dewey.ws. +reported to the community leaders responsible for enforcement at john@dewey.ws. All complaints will be reviewed and investigated promptly and fairly. All community leaders are obligated to respect the privacy and security of the @@ -82,15 +81,15 @@ behavior was inappropriate. A public apology may be requested. ### 2. Warning -**Community Impact**: A violation through a single incident or series -of actions. +**Community Impact**: A violation through a single incident or series of +actions. **Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels -like social media. Violating these terms may lead to a temporary or -permanent ban. +like social media. Violating these terms may lead to a temporary or permanent +ban. ### 3. Temporary Ban @@ -106,11 +105,11 @@ Violating these terms may lead to a permanent ban. ### 4. Permanent Ban **Community Impact**: Demonstrating a pattern of violation of community -standards, including sustained inappropriate behavior, harassment of an +standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. -**Consequence**: A permanent ban from any sort of public interaction within -the community. +**Consequence**: A permanent ban from any sort of public interaction within the +community. ## Attribution @@ -118,11 +117,11 @@ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.0, available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. -Community Impact Guidelines were inspired by [Mozilla's code of conduct -enforcement ladder](https://github.com/mozilla/diversity). - -[homepage]: https://www.contributor-covenant.org +Community Impact Guidelines were inspired by +[Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity). For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations. + +[homepage]: https://www.contributor-covenant.org diff --git a/README.md b/README.md index 2d133cb68..78d1eb1ab 100644 --- a/README.md +++ b/README.md @@ -28,29 +28,27 @@ them to be used as appliances. ## 📖 Documentation -[Getting Started][] | [API][] | [Usage][] | [SDK][] - -[Getting Started]: https://osapi-io.github.io/osapi/ -[API]: https://osapi-io.github.io/osapi/category/api -[Usage]: https://osapi-io.github.io/osapi/sidebar/usage -[SDK]: https://osapi-io.github.io/osapi/sidebar/sdk +[Getting Started] | [API] | [Usage] | [SDK] ## 🔗 Sister Projects -| Project | Description | -| --- | --- | -| [gohai][] | A Go-based system fact collector inspired by Chef Ohai | -| [nats-client][] | A Go package for connecting to and interacting with a NATS server | -| [nats-server][] | A Go package for running an embedded NATS server | -| [osapi-orchestrator][] | Declarative infrastructure orchestration DSL built on the OSAPI SDK | +| Project | Description | +| -------------------- | ------------------------------------------------------------------- | +| [gohai] | A Go-based system fact collector inspired by Chef Ohai | +| [nats-client] | A Go package for connecting to and interacting with a NATS server | +| [nats-server] | A Go package for running an embedded NATS server | +| [osapi-orchestrator] | Declarative infrastructure orchestration DSL built on the OSAPI SDK | + +## 📄 License + +The [MIT] License. +[api]: https://osapi-io.github.io/osapi/category/api +[getting started]: https://osapi-io.github.io/osapi/ [gohai]: https://github.com/osapi-io/gohai +[mit]: LICENSE [nats-client]: https://github.com/osapi-io/nats-client [nats-server]: https://github.com/osapi-io/nats-server [osapi-orchestrator]: https://github.com/osapi-io/osapi-orchestrator - -## 📄 License - -The [MIT][] License. - -[MIT]: LICENSE +[sdk]: https://osapi-io.github.io/osapi/sidebar/sdk +[usage]: https://osapi-io.github.io/osapi/sidebar/usage diff --git a/docs/docs/sidebar/development/development.md b/docs/docs/sidebar/development/development.md index a289da270..a306d5e10 100644 --- a/docs/docs/sidebar/development/development.md +++ b/docs/docs/sidebar/development/development.md @@ -91,12 +91,15 @@ server (`just react-dev`), code style, and component conventions. OSAPI uses [Docusaurus][] to host a documentation server. Content is written in Markdown and located in the `docs/docs` directory. All Markdown documents should -have an 80 character line wrap limit (enforced by Prettier). +have an 80 character line wrap limit. The site is formatted by Prettier through +the `docusaurus` module; markdown outside it is formatted by mdformat through +the `md` module. ```bash -just docs::start # Start local docs server (requires bun) -just docs::build # Build docs for production -just docs::fmt-check # Check docs formatting +just docusaurus-start # Start local docs server (requires bun) +just docusaurus-build # Build docs for production +just docusaurus-fmt-check # Check site formatting +just md-fmt-check # Check markdown outside the site ``` ## Testing diff --git a/justfile b/justfile index 199872176..457799da8 100644 --- a/justfile +++ b/justfile @@ -16,9 +16,9 @@ go_coverage_target := "99.9" import? '.just/remote/go.just' -mod? docs '.just/remote/docs.mod.just' +import? '.just/remote/docusaurus.just' +import? '.just/remote/md.just' mod? just '.just/remote/just.mod.just' -mod? docker '.just/remote/docker.mod.just' import? '.just/remote/react.just' @@ -28,12 +28,10 @@ import? '.just/remote/react.just' fetch: mkdir -p .just/remote curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/go/go.just -o .just/remote/go.just - curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/docs.mod.just -o .just/remote/docs.mod.just - curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/docs.just -o .just/remote/docs.just + curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/docusaurus/docusaurus.just -o .just/remote/docusaurus.just + curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/md/md.just -o .just/remote/md.just curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/just.mod.just -o .just/remote/just.mod.just curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/just.just -o .just/remote/just.just - curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/docker.mod.just -o .just/remote/docker.mod.just - curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/docker.just -o .just/remote/docker.just curl -sSfL https://raw.githubusercontent.com/osapi-io/osapi-justfiles/refs/heads/main/react/react.just -o .just/remote/react.just # --- Top-level orchestration --- @@ -42,7 +40,7 @@ fetch: deps: just go-deps just go-mod - just docs::deps + just docusaurus-deps just react-deps # Build production binary (includes embedded UI) @@ -60,7 +58,7 @@ test: linux-tune generate: redocly join --prefix-tags-with-info-prop title -o internal/controller/api/gen/api.yaml internal/controller/api/*/gen/api.yaml internal/controller/api/node/*/gen/api.yaml just go-generate - just docs::generate + just docusaurus-generate cp internal/controller/api/gen/api.yaml ui/src/sdk/gen/api.yaml just react-generate @@ -68,7 +66,8 @@ generate: ready: just generate just just::fmt - just docs::fmt + just docusaurus-fmt + just md-fmt just go-fmt just go-vet just react-fmt diff --git a/ui/AI_POLICY.md b/ui/AI_POLICY.md index e3f38b1e0..c42fb3169 100644 --- a/ui/AI_POLICY.md +++ b/ui/AI_POLICY.md @@ -2,64 +2,60 @@ The OSAPI project has strict rules for AI usage: -- **All AI usage in any form must be disclosed.** You must state - the tool you used (e.g. Claude Code, Cursor, Amp) along with - the extent that the work was AI-assisted. +- **All AI usage in any form must be disclosed.** You must state the tool you + used (e.g. Claude Code, Cursor, Amp) along with the extent that the work was + AI-assisted. -- **The human-in-the-loop must fully understand all code.** If you - can't explain what your changes do and how they interact with the - greater system without the aid of AI tools, do not contribute - to this project. +- **The human-in-the-loop must fully understand all code.** If you can't explain + what your changes do and how they interact with the greater system without the + aid of AI tools, do not contribute to this project. - **Issues and discussions can use AI assistance but must have a full - human-in-the-loop.** This means that any content generated with AI - must have been reviewed _and edited_ by a human before submission. - AI is very good at being overly verbose and including noise that - distracts from the main point. Humans must do their research and - trim this down. - -- **No AI-generated media is allowed (art, images, videos, audio, etc.).** - Text and code are the only acceptable AI-generated content, per the - other rules in this policy. - -- **Bad AI drivers will be denounced** People who produce bad contributions - that are clearly AI (slop) will be added to our public denouncement list. - This list will block all future contributions. Additionally, the list - is public and may be used by other projects to be aware of bad actors. - We love to help junior developers learn and grow, but - if you're interested in that then don't use AI, and we'll help you. - I'm sorry that bad AI drivers have ruined this for you. - -These rules apply only to outside contributions to OSAPI. Maintainers -are exempt from these rules and may use AI tools at their discretion; -they've proven themselves trustworthy to apply good judgment. + human-in-the-loop.** This means that any content generated with AI must have + been reviewed _and edited_ by a human before submission. AI is very good at + being overly verbose and including noise that distracts from the main point. + Humans must do their research and trim this down. + +- **No AI-generated media is allowed (art, images, videos, audio, etc.).** Text + and code are the only acceptable AI-generated content, per the other rules in + this policy. + +- **Bad AI drivers will be denounced** People who produce bad contributions that + are clearly AI (slop) will be added to our public denouncement list. This list + will block all future contributions. Additionally, the list is public and may + be used by other projects to be aware of bad actors. We love to help junior + developers learn and grow, but if you're interested in that then don't use AI, + and we'll help you. I'm sorry that bad AI drivers have ruined this for you. + +These rules apply only to outside contributions to OSAPI. Maintainers are exempt +from these rules and may use AI tools at their discretion; they've proven +themselves trustworthy to apply good judgment. ## There are Humans Here Please remember that OSAPI is maintained by humans. -Every discussion, issue, and pull request is read and reviewed by -humans (and sometimes machines, too). It is a boundary point at which -people interact with each other and the work done. It is rude and -disrespectful to approach this boundary with low-effort, unqualified -work, since it puts the burden of validation on the maintainer. +Every discussion, issue, and pull request is read and reviewed by humans (and +sometimes machines, too). It is a boundary point at which people interact with +each other and the work done. It is rude and disrespectful to approach this +boundary with low-effort, unqualified work, since it puts the burden of +validation on the maintainer. -In a perfect world, AI would produce high-quality, accurate work -every time. But today, that reality depends on the driver of the AI. -And today, most drivers of AI are just not good enough. So, until either -the people get better, the AI gets better, or both, we have to have -strict rules to protect maintainers. +In a perfect world, AI would produce high-quality, accurate work every time. But +today, that reality depends on the driver of the AI. And today, most drivers of +AI are just not good enough. So, until either the people get better, the AI gets +better, or both, we have to have strict rules to protect maintainers. ## AI is Welcome Here -OSAPI is written with plenty of AI assistance, and many maintainers -embrace AI tools as a productive tool in their workflow. As a project, -we welcome AI as a tool! +OSAPI is written with plenty of AI assistance, and many maintainers embrace AI +tools as a productive tool in their workflow. As a project, we welcome AI as a +tool! -**Our reason for the strict AI policy is not due to an anti-AI stance**, -but instead due to the number of highly unqualified people using AI. -It's the people, not the tools, that are the problem. +**Our reason for the strict AI policy is not due to an anti-AI stance**, but +instead due to the number of highly unqualified people using AI. It's the +people, not the tools, that are the problem. -I include this section to be transparent about the project's usage about -AI for people who may disagree with it, and to address the misconception -that this policy is anti-AI in nature. +I include this section to be transparent about the project's usage about AI for +people who may disagree with it, and to address the misconception that this +policy is anti-AI in nature. diff --git a/ui/README.md b/ui/README.md index 227f3632f..4776aa569 100644 --- a/ui/README.md +++ b/ui/README.md @@ -9,8 +9,8 @@ # OSAPI UI -A React management dashboard for [OSAPI][] with a meshtastic-inspired -design language. +A React management dashboard for [OSAPI] with a meshtastic-inspired design +language. ## Screenshots @@ -21,25 +21,24 @@ design language. ## ✨ Features -| Feature | Description | -| --- | --- | -| Dashboard | Fleet health with controller/NATS components, streams, KV stores, object store, and agent cards | -| Configure | Block-based operations builder with per-block target selection and result rendering | -| Auth & RBAC | JWT sign-in with role-based permission gating (Admin, Operator, Viewer) | -| Agent Management | Drain/undrain agents from the dashboard with RBAC-gated controls | -| @fact. References | Auto-complete fact references in DNS and network fields from live API | -| Generated SDK | Typed fetch functions from OSAPI's OpenAPI spec via [orval](https://orval.dev/) | - -[OSAPI]: https://github.com/osapi-io/osapi +| Feature | Description | +| ----------------- | ----------------------------------------------------------------------------------------------- | +| Dashboard | Fleet health with controller/NATS components, streams, KV stores, object store, and agent cards | +| Configure | Block-based operations builder with per-block target selection and result rendering | +| Auth & RBAC | JWT sign-in with role-based permission gating (Admin, Operator, Viewer) | +| Agent Management | Drain/undrain agents from the dashboard with RBAC-gated controls | +| @fact. References | Auto-complete fact references in DNS and network fields from live API | +| Generated SDK | Typed fetch functions from OSAPI's OpenAPI spec via [orval](https://orval.dev/) | ## 🤝 Contributing -See the [Development](docs/development.md) guide for prerequisites, setup, -and conventions. See the [Contributing](docs/contributing.md) guide before +See the [Development](docs/development.md) guide for prerequisites, setup, and +conventions. See the [Contributing](docs/contributing.md) guide before submitting a PR. ## 📄 License -The [MIT][] License. +The [MIT] License. -[MIT]: LICENSE +[mit]: LICENSE +[osapi]: https://github.com/osapi-io/osapi diff --git a/ui/docs/architecture.md b/ui/docs/architecture.md index 297e44711..cb1f6cbf5 100644 --- a/ui/docs/architecture.md +++ b/ui/docs/architecture.md @@ -1,7 +1,7 @@ # Architecture -OSAPI UI is a React single-page application that provides a management -dashboard and operations builder for [OSAPI](https://github.com/osapi-io/osapi). +OSAPI UI is a React single-page application that provides a management dashboard +and operations builder for [OSAPI](https://github.com/osapi-io/osapi). ## Tech Stack @@ -62,34 +62,34 @@ User opens app ### JWT Token OSAPI issues JWTs via `osapi token generate`. The token contains: + - `roles` claim — array of role strings (`admin`, `write`, `read`) - Standard JWT claims (sub, exp, iat) -The UI decodes the token client-side (no verification — that's the -server's job) to extract roles. The token is sent as a Bearer header -on every API request via the fetch mutator. +The UI decodes the token client-side (no verification — that's the server's job) +to extract roles. The token is sent as a Bearer header on every API request via +the fetch mutator. ### RBAC Model Three built-in roles with hierarchical permissions: -| Role | JWT Value | Permissions | -| --- | --- | --- | -| Admin | `admin` | All 17 permissions including `audit:read` | -| Operator | `write` | Read + write + execute (no audit) | -| Viewer | `read` | Read-only access | +| Role | JWT Value | Permissions | +| -------- | --------- | ----------------------------------------- | +| Admin | `admin` | All 17 permissions including `audit:read` | +| Operator | `write` | Read + write + execute (no audit) | +| Viewer | `read` | Read-only access | -Permissions use `resource:verb` format matching osapi's Go model: -`agent:read`, `file:write`, `command:execute`, `docker:execute`, etc. +Permissions use `resource:verb` format matching osapi's Go model: `agent:read`, +`file:write`, `command:execute`, `docker:execute`, etc. ### Permission Gating -- **Configure blocks** — each block type maps to a required permission - in `BLOCK_PERMISSIONS`. Unauthorized blocks show greyed out with a - lock icon. +- **Configure blocks** — each block type maps to a required permission in + `BLOCK_PERMISSIONS`. Unauthorized blocks show greyed out with a lock icon. - **Agent drain/undrain** — requires `agent:write`. -- **Role override dropdown** — lets users preview what other roles see - without changing the token. +- **Role override dropdown** — lets users preview what other roles see without + changing the token. ## SDK Generation @@ -109,16 +109,17 @@ osapi repo osapi-ui repo └────────────────────┘ └──────────────────────┘ ``` -1. In the osapi repo, `just generate` runs oapi-codegen for each domain - then `redocly join` to produce the combined spec at +1. In the osapi repo, `just generate` runs oapi-codegen for each domain then + `redocly join` to produce the combined spec at `internal/controller/api/gen/api.yaml`. 2. In osapi-ui, `just generate` copies that combined spec into - `src/sdk/gen/api.yaml` and runs orval to produce typed fetch - functions and schema types. + `src/sdk/gen/api.yaml` and runs orval to produce typed fetch functions and + schema types. ### Fetch Mutator `src/sdk/fetch.ts` is the only hand-written file in `src/sdk/`. It: + - Reads the base URL from `OSAPI_API_URL` env var - Gets the auth token from the React auth context module - Sends it as a `Bearer` header @@ -139,42 +140,41 @@ Pages (Dashboard, Configure, Roles, SignIn) Framework-level reusable components. These define the visual language: -| Component | Purpose | -| --- | --- | -| Text | Styled text with variant/size props — the default for all text | -| Card, CardHeader, CardTitle, CardContent | Container with variant borders/shadows | -| Button | Primary/secondary/ghost/destructive with sizes | -| Badge | Status indicators (ready/pending/running/error/applied/muted) | -| Input | Form input with label, autofill suppression | -| Dropdown | Custom popover dropdown (replaces native select) | -| FactInput | Input with @fact. reference autocomplete | -| FormField | Label + Input wrapper for consistent form layout | -| PageHeader | Page title + subtitle + optional actions slot | -| SectionLabel | Uppercase section header with optional icon | -| StatCard | Label + big value + detail text in a card | -| DataTable | Typed table with header/rows inside a card | -| HealthDot | Colored status dot (ok/error/muted) | -| ErrorBanner | Error message with icon (sm/md sizes) | -| MetricValue | Formatted metric display (label + value) | -| CodeBlock | Styled code/pre block with border | -| IdBadge | Monospace ID pill for identifiers | -| IconButton | Icon-only button with ghost/danger/accent variants | -| CollapsibleSection | Togglable section with chevron, icon, right content | -| Modal | Dialog overlay with close button | -| EmptyState | Centered message with dashed border and optional icon | -| SearchBox | Inline search input with close button | -| ScrollButton | Directional scroll arrow (left/right) | -| InfoBox | Subtle container for hints and info text | -| KeyValue | Inline key:value display pair | -| LabelTag | Accent-colored key:value label pill | -| ConditionAlert | Warning condition with triangle icon | -| Popover, PopoverItem, PopoverPanel | Floating popover menu system | +| Component | Purpose | +| ---------------------------------------- | -------------------------------------------------------------- | +| Text | Styled text with variant/size props — the default for all text | +| Card, CardHeader, CardTitle, CardContent | Container with variant borders/shadows | +| Button | Primary/secondary/ghost/destructive with sizes | +| Badge | Status indicators (ready/pending/running/error/applied/muted) | +| Input | Form input with label, autofill suppression | +| Dropdown | Custom popover dropdown (replaces native select) | +| FactInput | Input with @fact. reference autocomplete | +| FormField | Label + Input wrapper for consistent form layout | +| PageHeader | Page title + subtitle + optional actions slot | +| SectionLabel | Uppercase section header with optional icon | +| StatCard | Label + big value + detail text in a card | +| DataTable | Typed table with header/rows inside a card | +| HealthDot | Colored status dot (ok/error/muted) | +| ErrorBanner | Error message with icon (sm/md sizes) | +| MetricValue | Formatted metric display (label + value) | +| CodeBlock | Styled code/pre block with border | +| IdBadge | Monospace ID pill for identifiers | +| IconButton | Icon-only button with ghost/danger/accent variants | +| CollapsibleSection | Togglable section with chevron, icon, right content | +| Modal | Dialog overlay with close button | +| EmptyState | Centered message with dashed border and optional icon | +| SearchBox | Inline search input with close button | +| ScrollButton | Directional scroll arrow (left/right) | +| InfoBox | Subtle container for hints and info text | +| KeyValue | Inline key:value display pair | +| LabelTag | Accent-colored key:value label pill | +| ConditionAlert | Warning condition with triangle icon | +| Popover, PopoverItem, PopoverPanel | Floating popover menu system | ### Text Component -The `Text` component is the standard way to render styled text. Never -write inline Tailwind text classes — always use `Text` with the -appropriate variant: +The `Text` component is the standard way to render styled text. Never write +inline Tailwind text classes — always use `Text` with the appropriate variant: ```tsx Secondary text @@ -196,31 +196,30 @@ Business logic components specific to OSAPI: - **Block system** — BlockCard, BlockStack, ApplyButton, ResultCard, SaveStackDialog, StackBar - **Block forms** — one per operation type (CommandBlock, CronBlock, - DockerBlock, DockerExecBlock, FileBlock, FileUploadBlock, - FileDeleteBlock, CronDeleteBlock, ContainerActionBlock, - DnsUpdateBlock, SingleInputBlock) + DockerBlock, DockerExecBlock, FileBlock, FileUploadBlock, FileDeleteBlock, + CronDeleteBlock, ContainerActionBlock, DnsUpdateBlock, SingleInputBlock) - **Pickers** — TargetPicker (agents/labels), ObjectPicker (files), ContainerPicker, CronPicker - **Dashboard** — AgentCard, ComponentRow, JobDetail, HostGroupHeader ### Hooks (`src/hooks/`) -| Hook | Purpose | -| --- | --- | -| useHealth | Poll `/health/status` every 10s | -| useAgents | Poll `/agent` every 10s with refresh callback | -| useStack | Block state management (add, remove, apply, reset) | -| useStacks | Saved stack management (behind feature flag) | -| useTargets | Build target options from agent list | -| useObjects | Fetch file objects for pickers | -| useFacts | Fetch @fact. keys from `/facts/keys` | +| Hook | Purpose | +| ---------- | -------------------------------------------------- | +| useHealth | Poll `/health/status` every 10s | +| useAgents | Poll `/agent` every 10s with refresh callback | +| useStack | Block state management (add, remove, apply, reset) | +| useStacks | Saved stack management (behind feature flag) | +| useTargets | Build target options from agent list | +| useObjects | Fetch file objects for pickers | +| useFacts | Fetch @fact. keys from `/facts/keys` | ## Feature Flags Feature flags gate unreleased functionality via env vars: -| Flag | Default | Description | -| --- | --- | --- | +| Flag | Default | Description | +| ---------------------- | ------- | -------------------------------- | | `OSAPI_FEATURE_STACKS` | `false` | Saved stacks UI (pending Go API) | ## Pages @@ -228,6 +227,7 @@ Feature flags gate unreleased functionality via env vars: ### Dashboard (`/`) Fleet health overview: + - Summary stat cards (NATS, Jobs, Agents, Consumers) - Controller and NATS Server component health with hostname/cpu/memory - JetStream streams table with message counts and consumer counts @@ -238,6 +238,7 @@ Fleet health overview: ### Configure (`/configure`) Block-based operations builder: + - Sidebar with block categories (Cron, File, Docker, Command, DNS, Network) - Blocks gated by RBAC permissions - Per-block target picker (\_all, \_any, hostname, labels) @@ -247,6 +248,7 @@ Block-based operations builder: ### Roles (`/roles`) RBAC reference: + - Current session info with role badge - Role definitions table - Full permission matrix (permission × role) @@ -255,6 +257,7 @@ RBAC reference: ### SignIn JWT token authentication: + - Token paste field with validation - Role extraction from JWT claims - CLI hint for `osapi token generate` diff --git a/ui/docs/contributing.md b/ui/docs/contributing.md index 97d7cba86..f470d760a 100644 --- a/ui/docs/contributing.md +++ b/ui/docs/contributing.md @@ -29,16 +29,15 @@ document before submitting a PR. When adding a new operation to the Configure page: -1. Add the `BlockType` entry to `src/hooks/use-stack.ts` in - `ALL_BLOCK_TYPES` and the appropriate category. +1. Add the `BlockType` entry to `src/hooks/use-stack.ts` in `ALL_BLOCK_TYPES` + and the appropriate category. 2. Add the required permission to `BLOCK_PERMISSIONS` in `src/lib/permissions.ts`. -3. Create a block form component in `src/components/domain/` if the - block needs input fields. Use `SingleInputBlock` for simple - single-field blocks. +3. Create a block form component in `src/components/domain/` if the block needs + input fields. Use `SingleInputBlock` for simple single-field blocks. 4. Add the apply handler case in `src/pages/configure.tsx`. -5. Add result rendering in `src/components/domain/result-card.tsx` if - the response shape isn't handled by existing patterns. +5. Add result rendering in `src/components/domain/result-card.tsx` if the + response shape isn't handled by existing patterns. 6. Add the icon mapping in `blockIcons` in `configure.tsx`. ### Adding new UI components @@ -73,6 +72,6 @@ answer questions. > I'm stuck, where can I get help? -If you have questions, feel free to open a [Discussion][] on GitHub. +If you have questions, feel free to open a [Discussion] on GitHub. -[Discussion]: https://github.com/osapi-io/osapi-ui/discussions +[discussion]: https://github.com/osapi-io/osapi-ui/discussions diff --git a/ui/docs/development.md b/ui/docs/development.md index aade1f4a4..8ce0e4678 100644 --- a/ui/docs/development.md +++ b/ui/docs/development.md @@ -1,25 +1,24 @@ # Development -This guide covers the tools, setup, and conventions needed to work on -osapi-ui. +This guide covers the tools, setup, and conventions needed to work on osapi-ui. ## Prerequisites -Install tools using [mise][]: +Install tools using [mise]: ```bash mise install ``` -- **[Node.js][]** — Required runtime. -- **[Bun][]** — Package manager and script runner. -- **[just][]** — Task runner used for building, testing, formatting, and other +- **[Node.js]** — Required runtime. +- **[Bun]** — Package manager and script runner. +- **[just]** — Task runner used for building, testing, formatting, and other development workflows. Install with `brew install just`. ### Claude Code -If you use [Claude Code][] for development, install these plugins from the -default marketplace: +If you use [Claude Code] for development, install these plugins from the default +marketplace: ``` /plugin install commit-commands@claude-plugins-official @@ -59,11 +58,11 @@ OSAPI_BEARER_TOKEN= All env vars use the `OSAPI_` prefix (configured in `vite.config.ts`). -| Variable | Default | Description | -| ---------------------- | ---------------------- | ------------------------ | -| `OSAPI_API_URL` | `http://localhost:8080` | OSAPI API base URL | -| `OSAPI_BEARER_TOKEN` | (empty) | JWT token for auto-login | -| `OSAPI_FEATURE_STACKS` | `false` | Enable saved stacks UI | +| Variable | Default | Description | +| ---------------------- | ----------------------- | ------------------------ | +| `OSAPI_API_URL` | `http://localhost:8080` | OSAPI API base URL | +| `OSAPI_BEARER_TOKEN` | (empty) | JWT token for auto-login | +| `OSAPI_FEATURE_STACKS` | `false` | Enable saved stacks UI | Generate a bearer token with the OSAPI CLI: @@ -71,13 +70,13 @@ Generate a bearer token with the OSAPI CLI: osapi token generate ``` -If `OSAPI_BEARER_TOKEN` is set, the app auto-authenticates and skips the -sign-in page. If not set, users paste their token on the sign-in page. +If `OSAPI_BEARER_TOKEN` is set, the app auto-authenticates and skips the sign-in +page. If not set, users paste their token on the sign-in page. ## Code style -TypeScript and CSS should be formatted by [Prettier][] and linted using -[ESLint][]. This style is enforced by CI. +TypeScript and CSS should be formatted by [Prettier] and linted using [ESLint]. +This style is enforced by CI. ```bash just react-fmt # Auto-fix formatting @@ -91,8 +90,8 @@ just react-lint # Run ESLint - Use the `cn()` helper for conditional Tailwind classes. - Icons from lucide-react only. - No inline styles — Tailwind classes only. -- Use shared UI primitives (see [Architecture](architecture.md) docs) instead - of repeating Tailwind patterns inline. +- Use shared UI primitives (see [Architecture](architecture.md) docs) instead of + repeating Tailwind patterns inline. - **Always use `Text` for styled text** — never write `text-xs text-text-muted` inline. Use `` instead. - Always use the custom `Dropdown` component. Never use native `