Skip to content

docs: add the Docker deployment details the server guide is dropping #38

Description

@Quick104

The manual doesn't cover several things people running Silo with Docker Compose need: using Valkey instead of Redis, the full set of PostgreSQL tuning variables, where prepared downloads are stored, how the Meilisearch index rebuilds, the mount for SQLite per-user data, and what large database migrations need. These gaps hit hardest on servers with an external database, separate nodes, or a large library. Silo-Server/silo-server#1635 removes the server repository's Docker deployment guide, so the manual becomes the only place this material lives.

Each item links the section of the old guide it came from. Parts of that guide were stale, so check each claim against current silo-server code before publishing.

What to change

src/content/docs/docs/running-a-server/docker.md

  • Ports (old guide): add PROXY_PORT (default 8083) and TRANSCODE_PORT (default 8082). They set the host ports of the commented proxy and transcode node examples in the Compose file.
  • Data directories (old guide): transcode is mounted at /tmp/silo-transcode inside the container.
  • Optional Meilisearch (old guide):
    • After the restart, Silo builds the search index in the background and retries failures every minute.
    • Search status, in the advanced part of Admin > Settings > Library & Metadata > Search, shows which engine is answering searches while the index builds: Meilisearch, Meilisearch (keyword only), or Postgres full-text.
    • The panel's Rebuild index button opens the scheduled task that rebuilds the index by hand.
    • Settings that change the index format, including turning on meaning-based search, start an automatic rebuild after restart. A compatible older index keeps serving keyword results until its replacement is ready.
  • Add a "Valkey in place of Redis" section (old guide):
    • Start with the support warning: "Silo is currently tested only against Redis. Valkey support is provided as-is, with no support offered for Valkey-specific problems."

    • Silo connects to Valkey with a redis:// URL. There is no Valkey-specific setting.

    • For an existing Valkey server, follow External PostgreSQL and Redis and set the silo service's REDIS_URL to that server.

    • For a new installation, save this override as valkey-override.yml. The base Compose file's redis-cli ping health check works with the official Valkey image.

      services:
        redis:
          image: valkey/valkey:alpine

      Then check the merged files and start the stack:

      docker compose -f docker-compose.yml -f valkey-override.yml config --quiet
      docker compose -f docker-compose.yml -f valkey-override.yml up -d
    • For an existing installation, check the Redis version and follow Valkey's migration guide before switching images. The bundled Compose file reuses the same /data mount, and Valkey can't read data files written by Redis 7.4 or later.

src/content/docs/docs/running-a-server/configuration.md

  • MEDIA_CONTAINER_ROOT (old guide): on an existing installation, keep the container path that library records already store. If you change it after adding libraries, they point at folders that no longer exist.

  • PostgreSQL tuning (old guide): replace "The other POSTGRES_TUNE_* values in .env.example…" with the table:

    Variable Default Description
    POSTGRES_TUNE_PROFILE oltp Tuning profile; only oltp is currently supported.
    POSTGRES_TUNE_MEMORY auto Server or container RAM, such as 8GB or 32GB; explicit values are used as-is.
    POSTGRES_TUNE_MEMORY_BUDGET_PERCENT 75 Percentage of auto-detected RAM used for PostgreSQL recommendations.
    POSTGRES_TUNE_CPUS auto CPU count used for worker recommendations.
    POSTGRES_TUNE_STORAGE ssd One of hdd, ssd, san, or nvme.
    POSTGRES_TUNE_DB_SIZE auto Automatic classification, or less_ram, mid_ram, or greater_ram.
    POSTGRES_TUNE_CONNECTIONS 100 PostgreSQL max_connections; raised when the Silo application pool is larger.
    POSTGRES_SHM_SIZE 8gb Docker /dev/shm size for bundled PostgreSQL.
  • With POSTGRES_TUNE_MEMORY=auto, Silo reads memory from the first source it trusts: a finite Docker cgroup memory limit, then the bundled read-only /host/proc/meminfo mount, then /proc/meminfo. By default it leaves 25% of that memory for Silo, Redis, plugins, transcodes, and the operating system.

  • If Silo runs in a container with no memory limit and no /host/proc/meminfo mount, and /proc/meminfo reports more than 128 GB, Silo skips tuning and logs postgres auto-tuning disabled. Set POSTGRES_TUNE_MEMORY, or mount /proc/meminfo:/host/proc/meminfo:ro on the silo service.

  • The docker compose restart postgres silo step is easiest right after the first start, before you add libraries.

  • The bundled database user already has the permissions tuning needs. Silo tunes with the same DATABASE_URL identity it runs with and has no separate tuning credential. Granting that user ALTER SYSTEM on an external database lets anyone holding the application credential change server-wide PostgreSQL settings.

  • To tune a trusted external database anyway, set POSTGRES_TUNE_MEMORY and POSTGRES_TUNE_CPUS to the database host's values and grant the DATABASE_URL user ALTER SYSTEM. Automatic detection measures the Silo container, not the database machine.

  • Server modes (old guide): in api mode the main server can still transcode locally when node routing falls back to it.

src/content/docs/docs/running-a-server/transcode-nodes.md (or a Downloads settings page, if one is added)

Prepared downloads (old guide):

  • Transcode nodes can prepare downloads. The node keeps the file on its own disk and serves it through Silo's authenticated artifact API, so nodes need no shared download mount.
  • Admin > Settings > Downloads > Prepared file directory (download.artifact_dir) sets where prepared downloads are written, on transcode nodes and on the main server when it prepares them itself. Restart each process after changing it.
  • The path must be persistent and mounted on every process that can prepare downloads.
  • When the setting is blank, a transcode node writes to download-artifacts inside its transcode directory. The main server writes to silo-download-artifacts beside its transcode directory, which is /tmp/silo-download-artifacts in the default stack.
  • The default Compose file doesn't mount /tmp/silo-download-artifacts, so downloads the main server prepared are lost whenever its container is recreated, for example by an update. Tell operators to mount that path or set Prepared file directory to a mounted one. This comes from current server code; the old guide doesn't mention it.
  • Downloads with a server-wide or per-user bandwidth limit (Server bandwidth, Per-user bandwidth) are always served by the main server, so the limits stay exact whatever the topology.

src/content/docs/docs/running-a-server/playback.md

  • NVIDIA_GPU_COUNT in .env (default 1) sets how many GPUs docker-compose.nvidia.yml reserves for Silo (old guide).
  • Optional, since the page covers Linux: on Windows, separate COMPOSE_FILE entries with ; instead of :.

src/content/docs/docs/running-a-server/backup-restore.md

  • In "Per-user data on SQLite", replace "Mount it from the host" with this override (old guide). Add it before switching to SQLite, and run docker compose config --quiet before recreating the container.

    services:
      silo:
        volumes:
          - ${SILO_DATA_ROOT:-/opt/silo}/userdb:/var/lib/silo/userdb

src/content/docs/docs/running-a-server/updates.md

Large migrations (old guide):

  • Some releases rewrite large tables in place. The bigint id widening, for example, rewrites users, media_files, and media_folders and holds an ACCESS EXCLUSIVE lock on every table that references them, so Silo can't read or write most of its data until the migration commits.
  • A table rewrite needs free disk for a second copy of the table plus its rebuilt indexes. Check free space on the database disk before such an update.
  • Restarting Silo during a migration can leave a PostgreSQL backend holding the migration lock, in addition to abandoning the run.
  • Needs a maintainer decision: whether to document a reversible rollback. The old guide says to stop the stack and run docker compose run --rm silo --migrate-down-to <version> before starting the previous image, and warns that some migrations discard data on the way down. The manual currently says to restore the backup and ask for help instead.

src/content/docs/docs/get-started/installation-options.md

Don't carry over

  • The old guide's Meilisearch steps name Admin > Settings > Search. The setting is under Admin > Settings > Library & Metadata > Search, as the manual already says.
  • The old guide says /metrics "is unauthenticated on node listeners, matching the API listener". The main server's public port returns 404 for /metrics. Its metrics are served only on the separate SILO_METRICS_LISTEN listener, as Logs and monitoring already says.
  • The old guide checks the dump with a host-side pg_restore --list, which needs PostgreSQL client tools on the host. Keep the manual's check, which runs inside the postgres container.

AI disclosure: drafted with Claude Code (claude-opus-5-5[1m]) from an audit of the server repository's docs against the manual and current server code.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions