docs: add S3 provider setup and bucket permissions to the storage guide #37

The storage guide explains Silo's storage settings but not how to prepare a provider for them. It doesn't list the bucket permissions Silo needs and has no steps for Cloudflare R2 or Garage. It names Cloudflare signed token without the Cloudflare rule that mode depends on, and it leaves out three storage behaviors admins run into. Admins who put artwork in S3 have to work all of this out alone. Silo-Server/silo-server#1635 removes the server repository's copy of this material, so the manual becomes its only home.

The old docs are linked for reference: S3 storage setup and Artwork storage. They use stale UI labels, so check each claim against current silo-server code before publishing. The code links below are pinned to commit 20866241.

What to change

src/content/docs/docs/running-a-server/s3-storage.md

• Choose artwork storage:
  • Add Keep provider artwork, found in Admin > Settings > Library & Metadata under Artwork (setting metadata.cache_images).
  • Keep provider artwork is on by default. It copies provider posters and backdrops into artwork storage, on local disk or in S3.
  • When Keep provider artwork is off, clients load artwork straight from the providers.
  • After "Silo records the location the first time it stores a file", add: when a private bucket is configured, Silo records its Endpoint, Bucket, and Folder inside the bucket at startup, even if the bucket is empty. Changing them afterwards takes a storage transition.
• Set up S3:
  • Silo works with any S3-compatible service, including AWS S3, Ceph RGW, MinIO, Garage, and Cloudflare R2.
  • The access key needs s3:PutObject, s3:GetObject, s3:DeleteObject, and s3:ListBucket on the bucket.
  • s3:ListBucket covers the HeadBucket call behind Check Connection and /api/v1/ready, and the object listing that storage transitions, diagnostics cleanup, and catalog seed imports use. The old doc left it out.
  • When Region is blank, Silo signs requests for us-east-1.
  • Set Region when the provider checks the signing region, as Garage does.
• Link modes (after "Signed links let clients fetch a file…"):
  • Anyone with the link: the bucket, or a domain in front of it, must serve files without a signature.
  • Anyone with the link: enter that domain in Address clients download from, for example https://cdn.example.com. Anyone who has a file's URL can download it.
  • Cloudflare signed token uses four fields: Address clients download from (the R2 custom domain), Token Secret (must match the Cloudflare rule), Token query parameter (default verify), and Link lifetime in seconds (default 10800, three hours).
  • For revisioned artwork, Silo reuses a token for a quarter of Link lifetime, up to a day, so clients and Cloudflare keep their cached images.
  • Each token URL stays valid for at least three quarters of Link lifetime.
  • A longer Link lifetime means fewer URL changes, and a leaked URL that keeps working longer.
• Change storage later:
  • Moving from S3 to Local disk clears every S3 setting, public and private.
  • After that move, profile avatars, diagnostic bundles, and catalog files are stored on local disk too. The chosen policy decides which existing files are copied.
• New "Cloudflare R2" section:
  • For both options, set Endpoint to https://<account_id>.r2.cloudflarestorage.com.
  • For both options, enter the Access Key and Secret Key of an R2 API token with read and write access to the bucket.
  • For both options, turn on Put the bucket name in the URL path.
  • Public custom domain: connect a custom domain under R2 > your bucket > Settings > Custom Domains. Set How asset links are authorized to Anyone with the link and Address clients download from to https://your-custom-domain.com.
  • Signed token: this option needs a Cloudflare Pro plan or higher, for the is_timed_hmac_valid_v0() rule function. Steps:

    1. Generate a secret with openssl rand -hex 32.

    2. In the Cloudflare dashboard, open your zone > Security > WAF > Custom rules and create a rule named Silo CDN Token Auth. Check this path against current Cloudflare docs.

    3. Switch to Edit expression and enter:

       (http.host eq "your-cdn-domain.com" and not is_timed_hmac_valid_v0("YOUR_SECRET", http.request.uri, 10800, http.request.timestamp.sec, 8))
       

       Replace your-cdn-domain.com with the R2 custom domain and YOUR_SECRET with the secret. 10800 must equal Silo's Link lifetime. 8 is the separator length: the length of Token query parameter plus 2 (?verify= is 8 characters).

    4. Set the action to Block and deploy the rule.

    5. In Silo, set How asset links are authorized to Cloudflare signed token, Address clients download from to https://your-cdn-domain.com, Token Secret to the same secret, Token query parameter to verify, and Link lifetime to 10800.

    6. Restart Silo. Image URLs in the app should look like https://your-cdn-domain.com/<object key>?verify=<unix time>-<signature>.

    7. If images don't load, change the rule's action from Block to Log while you debug.

• New "Garage" section:
  • Garage supports the S3 calls Silo uses, but has no bucket policies or object ACLs (compatibility matrix).
  • Unsigned public reads work only through Garage's website access.
  • Signed links (recommended):
    1. garage bucket create <bucket-name>
    2. garage key create <key-name>
    3. garage bucket allow --key <key-name-or-ID> --read --write <bucket-name>
    4. In Silo, enter the endpoint, bucket, and keys, and turn on Put the bucket name in the URL path.
    5. Set Region to the [s3_api].s3_region value in garage.toml. Garage has no default for it, and its quick-start uses garage. A mismatch breaks signed links.
    6. Keep Signed links (recommended).
  • Public domain:

    1. Name the bucket after the exact public domain, for example garage bucket create assets.example.com. Garage can also match <bucket>.<root_domain>.

    2. Run garage bucket website --allow assets.example.com.

    3. Add a website listener to garage.toml; Garage doesn't start one by default. root_domain is required (checked on Garage v2.4.1) even if you only use exact bucket-name matching:

       [s3_web]
       bind_addr = "[::]:3902"
       root_domain = ".web.example.com"

    4. Route assets.example.com to port 3902 through a reverse proxy that preserves the Host header.

    5. In Silo, set Endpoint to Garage's S3 API (for example https://garage.example.com:3900), Bucket to assets.example.com, and Region as in the signed-links steps. Turn on Put the bucket name in the URL path.

    6. Set How asset links are authorized to Anyone with the link and Address clients download from to https://assets.example.com.

    7. Keep the private bucket on Signed links (recommended) against the normal S3 API port.

  • Neither Garage listener serves TLS. For an https:// endpoint, put a reverse proxy in front of Garage and point Endpoint at the proxy.
  • Test the public route with a known object path. A bare GET https://assets.example.com/ returns 404 unless the bucket has an index document; that is expected.

Don't carry over

• Old UI labels. Use the current ones:
  • "Admin > Settings > Storage" and "Settings → Infrastructure → Artwork storage" are now Admin > Settings > Storage & Database, Storage group, Backend.
  • "URL Auth Method" is now How asset links are authorized.
  • "S3 Presigned URLs", "Public (no auth)", and "Cloudflare Token Auth" are now Signed links (recommended), Anyone with the link, and Cloudflare signed token.
  • "Read Endpoint" is now Address clients download from.
  • "Path Style" is now Put the bucket name in the URL path.
  • "Token Param" and "Token TTL" are now Token query parameter and Link lifetime.
• The claim that private buckets hold "metadata, user DB". Private storage holds profile avatars, diagnostic bundles, and catalog seed artifacts, as the manual already says.
• The old permission list of PutObject, GetObject, and DeleteObject alone. It is missing s3:ListBucket.
• The MakeObjectPublic note. It describes server internals, not admin setup.

Code references

• Storage settings labels: web/src/pages/admin-settings/InfrastructureSettings.tsx#L517-L597
• Blank Region becomes us-east-1: internal/s3client/client.go#L104-L108
• HeadBucket in readiness: internal/api/handlers/health.go#L182
• HeadBucket in Check Connection: internal/api/handlers/admin_settings_checks.go#L464
• Object listing: storage transitions, diagnostics cleanup, catalog seed import
• Token reuse window: internal/s3client/client.go#L365-L375
• Token URL format: internal/s3client/client.go#L433-L448
• Keep provider artwork: web/src/pages/admin-settings/LibraryMetadataSettings.tsx#L150-L161, default at internal/config/admin_settings.go#L132
• Private bucket recorded at startup: internal/blobstore/open.go#L105-L109 and #L154-L160
• Private storage contents: internal/blobstore/open.go#L43-L47
• S3 to Local disk clears the S3 settings: internal/storagetransition/service.go#L559-L570, key lists at #L83-L93

AI disclosure: drafted with Claude Code (claude-opus-5-5[1m]) from an audit of the server repository's docs against the manual and current server code.