Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@
# DATA_OTEL_EXPORTER="console"
# DATA_OTEL_ENDPOINT=null
# DATA_OTEL_PROTOCOL="grpc"
# DATA_AUTH_ENABLED=false
# DATA_AUTH_REQUIRED=false
# DATA_AUTH_SERVER_URL=null
# DATA_DATABASE_NAME="madsci_data"
# DATA_COLLECTION_NAME="datapoints"
# DOCUMENT_DB_URL="mongodb://localhost:27017"
Expand Down Expand Up @@ -89,6 +92,9 @@
# EVENT_OTEL_EXPORTER="console"
# EVENT_OTEL_ENDPOINT=null
# EVENT_OTEL_PROTOCOL="grpc"
# EVENT_AUTH_ENABLED=false
# EVENT_AUTH_REQUIRED=false
# EVENT_AUTH_SERVER_URL=null
# DOCUMENT_DB_URL="mongodb://localhost:27017"
# EVENT_DATABASE_NAME="madsci_events"
# EVENT_COLLECTION_NAME="events"
Expand Down Expand Up @@ -131,6 +137,9 @@
# WORKCELL_OTEL_EXPORTER="console"
# WORKCELL_OTEL_ENDPOINT=null
# WORKCELL_OTEL_PROTOCOL="grpc"
# WORKCELL_AUTH_ENABLED=false
# WORKCELL_AUTH_REQUIRED=false
# WORKCELL_AUTH_SERVER_URL=null
# WORKCELL_NODES=null
# WORKCELLS_DIRECTORY=".madsci/workcells"
# CACHE_HOST="localhost"
Expand Down Expand Up @@ -172,6 +181,9 @@
# EXPERIMENT_OTEL_EXPORTER="console"
# EXPERIMENT_OTEL_ENDPOINT=null
# EXPERIMENT_OTEL_PROTOCOL="grpc"
# EXPERIMENT_AUTH_ENABLED=false
# EXPERIMENT_AUTH_REQUIRED=false
# EXPERIMENT_AUTH_SERVER_URL=null
# DOCUMENT_DB_URL="mongodb://localhost:27017"
# EXPERIMENT_DATABASE_NAME="madsci_experiments"
# EXPERIMENT_COLLECTION_NAME="experiments"
Expand Down Expand Up @@ -201,6 +213,9 @@
# RESOURCE_OTEL_EXPORTER="console"
# RESOURCE_OTEL_ENDPOINT=null
# RESOURCE_OTEL_PROTOCOL="grpc"
# RESOURCE_AUTH_ENABLED=false
# RESOURCE_AUTH_REQUIRED=false
# RESOURCE_AUTH_SERVER_URL=null
# RESOURCE_DEFAULT_TEMPLATES=null
# RESOURCE_DB_URL="postgresql://madsci:madsci@localhost:5434/resources"

Expand Down Expand Up @@ -229,6 +244,9 @@
# LAB_OTEL_EXPORTER="console"
# LAB_OTEL_ENDPOINT=null
# LAB_OTEL_PROTOCOL="grpc"
# LAB_AUTH_ENABLED=false
# LAB_AUTH_REQUIRED=false
# LAB_AUTH_SERVER_URL=null
# LAB_DASHBOARD_FILES_PATH="~/MADSci/ui/dist"

### LocationManagerSettings
Expand Down Expand Up @@ -256,6 +274,9 @@
# LOCATION_OTEL_EXPORTER="console"
# LOCATION_OTEL_ENDPOINT=null
# LOCATION_OTEL_PROTOCOL="grpc"
# LOCATION_AUTH_ENABLED=false
# LOCATION_AUTH_REQUIRED=false
# LOCATION_AUTH_SERVER_URL=null
# LOCATION_RECONCILIATION_INTERVAL_SECONDS=30.0
# LOCATION_RECONCILIATION_ENABLED=true
# LOCATION_LAB_CONFIG_FILE="locations.yaml"
Expand Down Expand Up @@ -536,3 +557,45 @@
# RESOURCES_MIGRATION_BACKUP_ONLY=false
# RESOURCES_MIGRATION_RESTORE_FROM=null
# RESOURCES_MIGRATION_GENERATE_MIGRATION=null

### AuthManagerSettings

# AUTH_SERVER_URL="http://localhost:8007/"
# AUTH_MANAGER_ID=null
# AUTH_MANAGER_TYPE="auth_manager"
# AUTH_RATE_LIMIT_ENABLED=true
# AUTH_RATE_LIMIT_REQUESTS=300
# AUTH_RATE_LIMIT_WINDOW=60
# AUTH_RATE_LIMIT_SHORT_REQUESTS=50
# AUTH_RATE_LIMIT_SHORT_WINDOW=1
# AUTH_RATE_LIMIT_CLEANUP_INTERVAL=300
# AUTH_RATE_LIMIT_EXEMPT_IPS=null
# AUTH_UVICORN_WORKERS=null
# AUTH_UVICORN_LIMIT_CONCURRENCY=null
# AUTH_UVICORN_LIMIT_MAX_REQUESTS=null
# AUTH_ENABLE_REGISTRY_RESOLUTION=true
# AUTH_MANAGER_NAME=null
# AUTH_MANAGER_DESCRIPTION=null
# AUTH_LAB_URL=null
# AUTH_REGISTRY_LOCK_TIMEOUT=60.0
# AUTH_OTEL_ENABLED=false
# AUTH_OTEL_SERVICE_NAME=null
# AUTH_OTEL_EXPORTER="console"
# AUTH_OTEL_ENDPOINT=null
# AUTH_OTEL_PROTOCOL="grpc"
# AUTH_AUTH_ENABLED=true
# AUTH_AUTH_REQUIRED=true
# AUTH_AUTH_SERVER_URL=null
# DATABASE_URL="postgresql://madsci:madsci@localhost/madsci_auth"
# AUTH_LAB_ID=null
# AUTH_ACCESS_TOKEN_TTL=900
# AUTH_REFRESH_TOKEN_TTL=2592000
# AUTH_SIGNING_KEY_TTL=7776000
# AUTH_ARGON2_TIME_COST=3
# AUTH_ARGON2_MEMORY_COST=65536
# AUTH_ARGON2_PARALLELISM=4
# AUTH_DENY_LIST_PERSIST_GRACE=300
# AUTH_TOKEN_CLOCK_SKEW_SECONDS=30
# AUTH_TRUST_FORWARDED_FOR=false
# AUTH_LOCAL_AUDIT_LOG_PATH=null
# AUTH_LOCAL_AUDIT_LOG_MAX_BYTES=104857600
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ The system follows a microservices architecture with the following main componen
- **madsci_data_manager**: Data capture, storage, and querying (Port 8004)
- **madsci_workcell_manager**: Workflow coordination and scheduling (Port 8005)
- **madsci_location_manager**: Laboratory location management, resource attachments, and node-specific references (Port 8006)
- **madsci_auth_manager**: Authentication and authorization service — users, projects, service accounts, node identities, JWT issuance, JWKS, RBAC (Port 8007). Default-disabled across all managers (`auth_enabled=False`); opt in per the rollout in `docs/guides/auth_operator.md`. Use `AuthClient` (`from madsci.client.auth_client import AuthClient`) for programmatic access; install ambient propagation via `auth_client_context()` so other service clients pick up bearer tokens automatically. Apply `@requires(permission=...)` from `madsci.common.auth_decorators` to enforce per-endpoint authorization.

### Frontend
- **ui/**: Vue 3 + Vuetify dashboard for lab management and monitoring
Expand Down Expand Up @@ -429,6 +430,7 @@ When TUI screens, CLI commands, or application code needs to communicate with MA
| `DataClient` | Data Manager | 8004 |
| `WorkcellClient` | Workcell Manager | 8005 |
| `LocationClient` | Location Manager | 8006 |
| `AuthClient` | Auth Manager | 8007 |
| `RestNodeClient` | Direct node communication | varies |

Client classes handle Pydantic model deserialization (avoiding field alias bugs like `_id` vs `experiment_id`), retry strategies, connection pooling, consistent error handling, and rate limiting.
Expand Down
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,19 +115,31 @@ docker compose up # Starts all services with example configuration

MADSci uses environment variables for configuration with hierarchical precedence. Key patterns:

- **Service URLs**: Each manager defaults to `localhost` with specific ports (Event: 8001, Experiment: 8002, Resource: 8003, Data: 8004, Workcell: 8005, Location: 8006, etc.)
- **Service URLs**: Each manager defaults to `localhost` with specific ports (Event: 8001, Experiment: 8002, Resource: 8003, Data: 8004, Workcell: 8005, Location: 8006, Auth: 8007)
- **Database connections**: FerretDB (document database)/PostgreSQL on localhost by default
- **File storage**: Defaults to `~/.madsci/` subdirectories
- **Environment prefixes**: Each service has a unique prefix (e.g., `WORKCELL_`, `EVENT_`, `LOCATION_`)
- **OpenTelemetry**: Configurable per-manager with `*_OTEL_ENABLED`, `*_OTEL_ENDPOINT`, etc.

See [Configuration.md](docs/Configuration.md) for comprehensive options, [example_lab/](./examples/example_lab/) for working configurations, and [OBSERVABILITY.md](./docs/guides/observability.md) for OpenTelemetry setup.

## Auth Manager (v0.8 — opt-in)

MADSci ships an OAuth 2.0 + OIDC-style **Auth Manager** (port 8007) for users, projects, service-accounts, node identities, and JWT-based service-to-service trust. It is **default-disabled** so existing deployments keep working unchanged; deployments opt in by:

1. Bootstrapping the Auth Manager (`madsci auth bootstrap`).
2. Registering each consuming manager and node, distributing client secrets.
3. Setting `auth_enabled=True, auth_required=False` on each manager (migration mode).
4. Flipping `auth_required=True` once traffic is clean.

See [`docs/guides/auth.md`](docs/guides/auth.md) for the architecture and token model and [`docs/guides/auth_operator.md`](docs/guides/auth_operator.md) for the bootstrap and rollout runbook.

## Roadmap

We're working on bringing the following additional components to MADSci:

- **Auth Manager**: For handling authentication and user and group management for an autonomous lab.
- **Globus / ORCID OIDC federation** — cross-lab user identity (follow-on to the Auth Manager).
- **mTLS for nodes** — TLS-based node trust, layered onto the existing `NodeIdentity` model (slots into the reserved `mtls_cert_fingerprint` field).

## Getting Started

Expand Down
34 changes: 33 additions & 1 deletion docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Removed
### Security

#### Auth Manager hardening (review-driven follow-ups)
- **Auth Manager admin endpoints now require authentication.** Every admin route on the Auth Manager itself (`POST /users`, `/projects`, `/roles`, `/roles/grant`, `/service-accounts`, `/node-identities`, `POST /credentials/{id}/rotate`, `POST /keys/rotate`, `DELETE /keys/{kid}`, every `GET` listing endpoint) carries an explicit `@requires(permission=...)` check, and the Auth Manager mounts `AuthMiddleware` on itself with an explicit unauthenticated allowlist (`/token`, `/.well-known/jwks.json`, `/health`, `/health/keys`, `/settings`, `/deny-list`, `/introspect`). New permission strings: `auth.user.{read,write}`, `auth.project.{read,write}`, `auth.role.{read,write,grant}`, `auth.principal.write`, `auth.credentials.rotate`, `auth.key.{read,rotate,retire}`, `auth.token.{introspect,revoke}`.
- **`POST /introspect` follows RFC 7662** — unauthenticated callers (or callers without `auth.token.introspect`) receive `{"active": false}` rather than the full claims dump.
- **`POST /revoke` requires authentication** — self-revocation (matching `sub`) is allowed; revoking another principal's token requires `auth.token.revoke`.
- **JWT verification pins `RS256`.** Both `TokenService.verify_token` and `AuthClient.verify_jwt` reject any other JWS algorithm before the JOSE library touches the token (and pass the same allowlist to the library's own `algorithms=` argument), closing the alg-confusion attack class (e.g., HS256-with-public-key forgery).
- **JWT verification applies a configurable clock-skew leeway** (`AuthManagerSettings.token_clock_skew_seconds`, default 30s). Applies to both server and client verification.
- **Refresh-token consumption is atomic.** `consume_refresh_token` now uses an atomic claim/marker pattern so two parallel consumers of the same refresh token cannot both succeed; the loser fires the family-revoke reuse-detection path. The `rotated_to` column links parent → child for forensic walks. New Alembic migration `0002_refresh_token_partial_unique_index` adds a partial unique index on `refresh_tokens(token_hash) WHERE revoked_at IS NULL`.
- **Auth Manager refuses to start without a bound `lab_id`.** The `"lab-unbound"` placeholder audience is gone; two unbound deployments can no longer mutually trust each other's tokens.
- **`madsci auth bootstrap` no longer accepts `--password` on argv** (leaks via `ps`). Source the password from `MADSCI_AUTH_BOOTSTRAP_PASSWORD` env var or the interactive prompt.
- **`X-Forwarded-For` is no longer trusted by default.** New `AuthManagerSettings.trust_forwarded_for` (default `False`) gates whether `_client_ip` reads the header. Operators behind a real proxy must opt in.
- **`examples/example_lab/compose.yaml` Auth Manager DB URL fix.** The `auth_manager` service uses `network_mode: host` (like every other manager), but the `AUTH_DATABASE_URL` env var was set to the container DNS name `madsci_postgres_auth:5432` — which doesn't resolve under host networking, so Auth Manager crashed on startup. Changed to `localhost:${AUTH_POSTGRES_PORT:-5435}` (the host-mapped port, matching the pattern other managers use for postgres). Caught during the auth-manager-security-hardening smoke test.
- **Auth Manager defaults to enforcing auth on its own admin surface.** Mitigates the security review's HIGH finding that `AuthManagerSettings` inheriting `auth_enabled=False` from `ManagerSettings` left a fresh deployment fully unauthenticated. `AuthManagerSettings` now overrides both `auth_enabled` and `auth_required` to `True`; `AuthManager` installs a self-verifying `AuthMiddleware` (using its own in-process `TokenService` rather than a remote `AuthClient`); and `AuthManager.run_server()` refuses to bind unless both flags are `True`. Tests that need to bypass enforcement must opt out explicitly (`settings.auth_enabled = False`).
- **`/revoke` honors self-vs-other on the refresh-token branch.** Previously, the self-vs-other permission check applied only when revoking access tokens; refresh-token revocation ran unconditionally for any authenticated principal who knew the bearer string. Now both branches probe the target principal's `sub` and require `auth.token.revoke` for cross-principal revocation.
- **`AuthClient` validates `iss` and `aud` claims** when the new `expected_issuer` and `expected_audience` constructor args are supplied. `manager_base._setup_auth_middleware` now passes the manager's `auth_server_url` as `expected_issuer` and the manager's `lab_id` as `expected_audience` — defense-in-depth against any future cross-lab token-confusion vector.

### Changed

- **JOSE library swapped from `Authlib` to `joserfc`.** Authlib 1.7+ deprecates `authlib.jose` in favor of `joserfc` (the same author's successor library). `madsci.auth_manager` and `madsci.client.auth_client` both call `joserfc.jwt.encode`/`decode`, build keys via `joserfc.jwk.RSAKey`/`KeySet`, and validate claims via `JWTClaimsRegistry`. The `Authlib>=1.3.0` dependency is replaced with `joserfc>=1.0.0` in `madsci.auth_manager` and `madsci.client`. No behavioral change for callers of `AuthClient`; the JWT format and verification semantics are identical. Eliminates the runtime `AuthlibDeprecationWarning`.
- **`madsci new workcell` subcommand**: The workcell template generated an orphaned YAML format that didn't correspond to any Pydantic model. Workcell configuration is handled by `WorkcellManagerSettings` via `settings.yaml`.
- **`LabClient.get_definition()`**: The Lab Manager no longer serves a `/definition` endpoint. Use `get_lab_context()` or `get_lab_health()` instead.
- **`WORKCELL` template category**: Removed from `TemplateCategory` enum.
Expand All @@ -18,6 +37,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **`LabManagerDefinition`**: Now emits `MadsciDeprecationWarning` on instantiation (v0.7.0 removal). Use `LabManagerSettings` for configuration.

### Added

#### Auth Manager (Foundation)
- **`madsci_auth_manager` package (port 8007)**: New OAuth 2.0 + OIDC-style identity service. Issues RS256 JWT access tokens (15-min default) signed by a rotating keypair; opaque refresh tokens stored server-side with reuse-detection. Exposes `/token` (password, refresh_token, client_credentials), `/introspect`, `/revoke`, `/.well-known/jwks.json`, `/deny-list` (with `ETag` conditional fetch), users, projects, roles, service-accounts, node-identities, and key-rotation endpoints. Single-tenant (per Decision 12 — `aud = lab_id`). PostgreSQL-backed via `SQLAlchemyHandler`; in-memory `SQLiteHandler` for tests. Default-disabled at all consuming managers; opt-in via `auth_enabled=True`.
- **`AuthClient`** (`madsci.client.auth_client`): `login()`, `refresh()`, `client_credentials_login()`, `verify_jwt()` (JWKS-cached + force-refresh on signature failure), `introspect()`, `revoke()`, deny-list polling, plus admin surface (users, projects, roles, service-accounts, nodes, credentials, keys).
- **`AuthMiddleware`** on `AbstractManagerBase`: When `auth_enabled=True`, validates `Authorization: Bearer <jwt>` against cached JWKS, populates `request.state.principal`, and enters an `ownership_context()` for the request lifetime. `auth_required=False` migration mode passes unauth'd requests through with a structured warning.
- **`auth_client_context()`**: Ambient `AuthClient` propagation. When set, `create_httpx_client()` automatically injects `Authorization: Bearer <token>` and force-refreshes-and-retries on 401.
- **`@requires(permission=..., project_from=...)`**: Decorator for `Routable` endpoints. 401 unauthenticated, 403 missing permission, 403 if `project_from` resolves a project the principal is not a member of.
- **`madsci auth` CLI**: `bootstrap`, `user create|deactivate|password|grant|list`, `project create|list|members`, `manager register|list`, `node register|list`, `credentials rotate`, `keys rotate|list|retire`.
- **Migration guides**: [`docs/guides/auth.md`](guides/auth.md) (architecture, token model, RBAC) and [`docs/guides/auth_operator.md`](guides/auth_operator.md) (bootstrap, secret distribution with required `0600` mode and `.gitignore` treatment, key rotation, HTTPS termination + `X-Forwarded-For`, audit-log retention, local audit-log fallback bound + rotation alerting, the `auth_enabled → auth_required` rollout).
- **`MadsciBaseSettings`/`ManagerSettings` new fields**: `auth_enabled`, `auth_required`, `auth_server_url`. Defaults preserve current behavior (auth disabled).
- **`OwnershipInfo.from_jwt_claims(claims)`**: Canonical mapping from validated `JWTClaims` to `OwnershipInfo` (claims-sourced fields override body-supplied values when auth is enabled, per Decision 10).
- **Port 8007 reserved** for the Auth Manager across CLAUDE.md, doctor checks, and `madsci start --mode=local`.

- **`target_model` field on `TemplateManifest`**: Templates that generate YAML/JSON config can now declare which Pydantic model the output should validate against. Automated tests verify template output matches the declared model.
- **Settings file validation in `madsci validate`**: The validate command now supports `settings.yaml` and `*.settings.yaml` files, validating against per-manager settings classes (Lab, Event, Experiment, Resource, Data, Workcell, Location).
- **Pydantic-first data modeling guidance**: Added to CLAUDE.md, AGENTS.md, and agent skills to codify the rule that every YAML/JSON config format must have a corresponding Pydantic model.
Expand Down
Loading
Loading