Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ go test ./...
go vet ./...
go build ./examples/hello-scheduled-task
go build ./examples/hello-runtime-host
go build ./examples/hello-network-access
gofmt -l .
```

Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ The SDK ships protobuf contracts for every capability the host understands:
- `request_router.v1`
- `scan_source.v1`
- `watch_sync_provider.v1`
- `network_access_provider.v1`
- `audiobook_backend.v1`
- `ebook_backend.v1`

Expand All @@ -50,7 +51,7 @@ A typical plugin:
3. Supports the `manifest` subcommand via `pkg/pluginsdk/runtime` so the host can introspect manifests without launching the plugin.
4. Is installed either from a catalog or by uploading a trusted binary to a Silo server.

For a minimal self-describing plugin, see [`examples/hello-scheduled-task`](examples/hello-scheduled-task). For a plugin that calls back into the host via `RuntimeHost` (publishing events, listing libraries), see [`examples/hello-runtime-host`](examples/hello-runtime-host).
For a minimal self-describing plugin, see [`examples/hello-scheduled-task`](examples/hello-scheduled-task). For a plugin that calls back into the host via `RuntimeHost` (publishing events, listing libraries), see [`examples/hello-runtime-host`](examples/hello-runtime-host). For a stub overlay-network provider, see [`examples/hello-network-access`](examples/hello-network-access).

## Operator-facing presentation

Expand Down Expand Up @@ -204,6 +205,16 @@ and the order of returned watchlist states is the remote list order. Event
`list_position` is presence-aware: an explicit zero means the first position,
while omission means no requested ordering.

## Network access providers

`network_access_provider.v1` lets a resident plugin give the deployment an
overlay-network identity (Tailscale, NetBird) and reverse-proxy overlay
traffic to the host's local listeners. The host starts these plugins at boot,
restarts them on crash, stores their per-instance state encrypted, and
aggregates status across the API server and proxy nodes. See
[docs/network-access-provider.md](docs/network-access-provider.md) for the
proxy contract, `GetHostInfo` fields, instance state, and enrollment rules.

## Scan sources

The `scan_source.v1` capability is for Autoscan providers. The host owns the
Expand Down
180 changes: 180 additions & 0 deletions docs/network-access-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Network access providers (`network_access_provider.v1`)

A network access provider gives a Silo deployment an overlay-network identity
(Tailscale via tsnet first; NetBird later) so clients reach Silo without port
forwarding or a public reverse proxy. The plugin owns the overlay client and a
reverse proxy in front of the host's local listeners. The host owns
supervision, per-instance state storage, status aggregation, access-path
routing, and the admin API.

Added in SDK v0.16.0. Server support arrives in stages; check the host's
`/api/v2/network-access/capabilities` before assuming a feature exists.

## Manifest

Declare the capability with the typed descriptor. The host lists providers
from the descriptor without launching the plugin.

```json
{
"type": "network_access_provider.v1",
"id": "tailscale",
"display_name": "Tailscale",
"network_access_provider": {
"provider": "tailscale",
"display_name": "Tailscale"
}
}
```

`provider` is a stable lowercase, path-safe slug; it appears in admin API
paths such as `/api/v2/admin/network-access/{provider}/status`. Reserved
slugs today: `tailscale`, `netbird`.

## Resident lifecycle

Plugins declaring this capability are resident. The host:

- starts them when the API listener binds (and on every proxy node),
- restarts them on exit or failed health with exponential backoff,
- stops them last during shutdown.

Every host (the API server and each proxy node) runs its own instance of the
same installation. Each instance has its own overlay identity and its own
instance-state scope. Do not assume a single process.

## gRPC service

```proto
service NetworkAccessProvider {
rpc Connect(NetworkAccessConnectRequest) returns (NetworkAccessStatus);
rpc Disconnect(NetworkAccessDisconnectRequest) returns (NetworkAccessStatus);
rpc GetStatus(NetworkAccessGetStatusRequest) returns (NetworkAccessStatus);
}
```

Register it with `runtime.CapabilityServers{NetworkAccessProvider: ...}`.

`NetworkAccessStatus.state` is one of `disconnected`,
`awaiting_authorization`, `connecting`, `connected`, `error`. The vocabulary
is open; the host tolerates values it does not know. `Connect` may return
`awaiting_authorization` or `connecting` and finish enrollment in the
background; push each later transition with
`RuntimeHost.ReportNetworkAccessStatus` so the host does not poll. The host
still calls `GetStatus` on demand.

`desired_connected` is plugin-owned intent. Persist it in instance state (see
below), report it in every status, and reconnect on start when it is true.
The host never writes it.

## What the host tells you: `GetHostInfo`

`RuntimeHost.GetHostInfo` carries everything the proxy needs:

| Field | Meaning |
|---|---|
| `host_role` | `api` or `proxy`. |
| `host_name` | Node name on proxies, server name on the API host. |
| `node_id` | `stream_nodes.id` on proxies, `0` on the API host. |
| `ingress_token` | Per-process-start secret; see below. |
| `listeners` | Local listeners to expose: `name`, loopback `address`, `default_port`. |

`listeners` always contains `api`. On the API host it also carries `jellyfin`
and `abs` when those listeners are enabled. Proxies report only `api`.
`default_port` is the port the plugin should expose that listener on (443 for
`api` under HTTPS, 8096 for `jellyfin`, 13378 for `abs`); zero means use the
provider default. Expose every listener the host reports and return one
`NetworkAccessListener` per exposed listener in the status; `origin` is the
`api` listener's origin.

The typed client is `runtimehost.Client.GetHostInfo`, which returns
`runtimehost.HostInfo` with `Listener(name)` for lookup.

## Proxy contract

For every request the plugin forwards to a host listener:

- Preserve the incoming `Host` header.
- Overwrite `X-Forwarded-Proto` with `https`.
- Set `X-Forwarded-For` to the overlay peer address.
- Set `X-Silo-Ingress-Token` to the value from `GetHostInfo`, replacing any
client-supplied value; the host answers 403 when the header carries more
than one value.

The host validates the token, strips the header, and records the request's
access path so stream URLs point tailnet clients at overlay origins. The
plugin's loopback source is in the host's default trusted-proxy list, so the
forwarded headers are honoured.

The ingress token rotates on every host process start and is compared in
constant time. Keep it in memory only; never log or persist it. After a host
or plugin restart, call `GetHostInfo` again before proxying; a stale token is
rejected with `403`.

## Instance state

Per-instance state (tsnet node keys, profile state, `desired_connected`)
lives in the host, encrypted, scoped to installation plus host. The plugin
never sees the scope and never writes files.

```proto
rpc ReadInstanceState(ReadInstanceStateRequest) returns (ReadInstanceStateResponse);
rpc WriteInstanceState(WriteInstanceStateRequest) returns (WriteInstanceStateResponse);
```

Limits: key up to 256 bytes, value up to 256 KiB, up to 256 keys per scope.
The SDK client checks the key and value limits before the round trip.

`runtimehost.InstanceStateStore` exposes the RPCs in the two-method shape of
tailscale's `ipn.StateStore`:

```go
type StateStore interface {
ReadState(key string) ([]byte, error) // runtimehost.ErrStateNotExist when absent
WriteState(key string, value []byte) error
}
```

The SDK does not import tailscale, and `ipn.StateStore` takes `ipn.StateKey`
rather than `string`, so wrap it in the plugin:

```go
type tsStore struct{ inner *runtimehost.InstanceStateStore }

func (s tsStore) ReadState(k ipn.StateKey) ([]byte, error) {
b, err := s.inner.ReadState(string(k))
if errors.Is(err, runtimehost.ErrStateNotExist) {
return nil, ipn.ErrStateNotExist
}
return b, err
}

func (s tsStore) WriteState(k ipn.StateKey, v []byte) error {
return s.inner.WriteState(string(k), v)
}

srv := &tsnet.Server{Store: tsStore{inner: host.InstanceStateStore()}}
```

Each `ReadState`/`WriteState` call uses a 30 s deadline by default;
`WithTimeout` changes it.

## Enrollment

Support both an auth key in plugin config (for hands-off enrollment across
many nodes) and an interactive auth URL as the fallback. While enrollment is
pending, report `state: awaiting_authorization` with `auth_url` set. The
`auth_url` is admin-only: never log it, and clear it once enrollment
completes.

## Logging

Log state transitions and errors. Never log `auth_url`, `ingress_token`,
auth keys, or instance state values.

## Example

[`examples/hello-network-access`](../examples/hello-network-access) is a stub
provider with no overlay network. It shows the manifest, registration,
instance-state persistence of `desired_connected`, and the status push, and
serves as a test fixture for the host.
5 changes: 4 additions & 1 deletion docs/runtime-host.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ plugin, the plugin calls back into the host.
| `PublishEvent(name, payload)` | Publish onto the host's event bus. The host stamps `plugin.<plugin_id>.` in front of `name` server-side. |
| `PublishEventTo(target_plugin_id, name, payload)` | Publish an event addressed to one installed plugin by stable `plugin_id`. |
| `PublishEventToInstallation(target_installation_id, name, payload)` | Publish an event addressed to one specific plugin installation. |
| `GetHostInfo()` | Return public-safe host URLs for callbacks and plugin-served links. |
| `GetHostInfo()` | Return public-safe host URLs for callbacks and plugin-served links, plus host role, listeners, and the ingress token for network access providers. |
| `ListLibraries(user_id)` | Return libraries (optionally scoped to a user). |
| `CheckMediaPresence(provider, media_type, ids)` | Batched lookup: which external IDs already exist in the host catalog. v1 supports provider="tmdb" only. |
| `ListInstalledPlugins()` | Return installed plugins and their advertised capabilities. |
Expand All @@ -21,6 +21,9 @@ plugin, the plugin calls back into the host.
| `ResolveCatalogImageURLs(paths, variant)` | Resolve stored catalog image paths into host-generated browser URL targets. |
| `MintScopedStream(request)` | Mint a short-lived stream grant for plugin-owned public access workflows. |
| `CallPluginHTTP(request)` | Invoke another installed plugin's HTTP route through the host control plane. |
| `ReadInstanceState(key)` | Read one key of the calling instance's encrypted, host-scoped state. |
| `WriteInstanceState(key, value)` | Write one key of that state. Key ≤ 256 bytes, value ≤ 256 KiB, ≤ 256 keys per scope. |
| `ReportNetworkAccessStatus(status)` | Push a `network_access_provider.v1` status change so the host does not poll. |

## Using it from a plugin

Expand Down
1 change: 1 addition & 0 deletions examples/hello-network-access/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
hello-network-access
27 changes: 27 additions & 0 deletions examples/hello-network-access/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Hello Network Access

A stub `network_access_provider.v1` plugin with no overlay network. It shows
the contract shape and doubles as a host-side test fixture:

- `Connect` marks the instance connected, builds a fake origin per listener
reported by `RuntimeHost.GetHostInfo`, persists `desired_connected` in
host-provided instance state, and pushes the status with
`ReportNetworkAccessStatus`.
- `Disconnect` reverses it.
- `GetStatus` restores `desired_connected` from instance state on first call
so intent survives a plugin restart.

There is no tsnet dependency. A real provider is described in
[docs/network-access-provider.md](../../docs/network-access-provider.md).

## Build

```sh
go build -o hello-network-access ./examples/hello-network-access
```

## Inspect the manifest

```sh
./hello-network-access manifest
```
Loading
Loading