Add optional automatic TLS Docker Compose example #215
+306
−2
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # This opt-in example uses Wings' built-in ACME client to issue and renew a | ||
| # Let's Encrypt certificate. The default Compose example remains unchanged. | ||
| # It requires exclusive access to port 80 on the published host address. When | ||
| # Panel or another reverse proxy shares that address, terminate TLS there and | ||
| # proxy to Wings instead, or bind Wings to a separate public address. | ||
| # | ||
| # Before starting: | ||
| # 1. Set WINGS_HOSTNAME to the node FQDN without a scheme or port. | ||
| # 2. Set WINGS_API_PORT and WINGS_SFTP_PORT to the same ports configured in | ||
| # /etc/pelican/config.yml. Port 80 is reserved for the ACME HTTP-01 challenge. | ||
| # 3. Create the Panel node with HTTPS enabled and matching daemon ports. | ||
| # 4. Point the FQDN directly at this host. When using Cloudflare, keep the | ||
| # record DNS only because the configured SFTP port cannot use the proxy. | ||
| # 5. Allow inbound TCP traffic on port 80 and both configured service ports. | ||
| # Every published A and AAAA address must reach this host on those ports. | ||
| # | ||
| # Example .env: | ||
| # WINGS_HOSTNAME=node.example.com | ||
| # WINGS_API_PORT=8080 | ||
| # WINGS_SFTP_PORT=2022 | ||
| # | ||
| # Validate the configured ports and start the stack through the included | ||
| # wrapper. It rejects invalid, reserved, and conflicting service ports before | ||
| # Docker Compose attempts to bind them: | ||
| # sh ./docker-compose.auto-tls.sh up -d | ||
| # | ||
| # The existing /var/lib/pelican volume persists the ACME account and certificate | ||
| # cache at /var/lib/pelican/.tls-cache across container restarts. | ||
|
|
||
| services: | ||
| wings: | ||
| image: ghcr.io/pelican/wings:latest | ||
| restart: always | ||
| command: | ||
| - /usr/bin/wings | ||
| - --config | ||
| - /etc/pelican/config.yml | ||
| - --auto-tls | ||
| - --tls-hostname | ||
| - "${WINGS_HOSTNAME:?Set WINGS_HOSTNAME to the node FQDN}" | ||
| networks: | ||
| - wings0 | ||
| ports: | ||
| - "80:80" | ||
| - "${WINGS_API_PORT:?Set WINGS_API_PORT to the API port in config.yml}:${WINGS_API_PORT:?Set WINGS_API_PORT to the API port in config.yml}" | ||
| - "${WINGS_SFTP_PORT:?Set WINGS_SFTP_PORT to the SFTP port in config.yml}:${WINGS_SFTP_PORT:?Set WINGS_SFTP_PORT to the SFTP port in config.yml}" | ||
|
Comment on lines
+43
to
+46
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- docker-compose.auto-tls.example.yml ---'
cat -n docker-compose.auto-tls.example.yml | sed -n '1,90p'
printf '%s\n' '--- port-related references ---'
rg -n -S 'WINGS_(API|SFTP)_PORT|auto-tls|tls-hostname|80:80|SFTP_PORT|API_PORT' \
--glob '!node_modules' --glob '!dist' --glob '!build' . | head -250
printf '%s\n' '--- compose tooling availability ---'
command -v docker || true
docker compose version 2>/dev/null || true
command -v podman || trueRepository: pelican/wings Length of output: 5301 🌐 Web query:
💡 Result: Docker Compose supports variable interpolation with a Bash-like syntax. For mandatory variables, you can use the following patterns [1][2][3]: - ${VARIABLE:?error}: Exits with an error message if the variable is unset or empty [1][2][3]. - ${VARIABLE?error}: Exits with an error message if the variable is unset (but allows empty values) [1][2][3]. Regarding port mapping and host port ranges, duplicate ports or overlaps often cause errors because they conflict with network binding rules on the host [4][5]. Docker requires unique port mappings to avoid "address already in use" errors [6][4][7]. If you encounter duplicate port errors: 1. Ensure your service configuration does not inadvertently merge duplicate ports (e.g., when using extends or multiple Compose files) [6][7]. 2. Use docker compose config to inspect the fully resolved configuration and identify where duplicates are occurring [8]. 3. For dynamic port ranges, assigning a range (e.g., 8000-8010:80) allows the engine to select an available port within that range, which helps avoid conflicts when running multiple replicas [4][5]. Avoid hardcoding specific host-side ports if you need to scale services on the same host [4]. If you face specific "must be unique" errors, check that your variables for port mappings do not resolve to the same value across different entries in the ports array [8][7]. Citations:
🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- auto-TLS and listener implementation ---'
ast-grep outline cmd/root.go --lang go
rg -n -C 8 'auto.?tls|tls.?hostname|Listen|listen|API|SFTP|http|:80|tls-cache' cmd internal *.go 2>/dev/null | head -300
printf '%s\n' '--- port configuration and validation ---'
rg -n -S 'type .*Port|API.*Port|SFTP.*Port|api_port|sftp_port|port.*uint|port.*int|Validate.*Port|ListenAndServe|net.Listen' \
--glob '*.go' . | head -300Repository: pelican/wings Length of output: 16803 🏁 Script executed: #!/bin/bash
set -e
printf '%s\n' '--- API and SFTP configuration definitions ---'
cat -n config/config.go | sed -n '55,110p'
ast-grep outline sftp/server.go --lang go
cat -n sftp/server.go | sed -n '1,125p'
printf '%s\n' '--- configuration examples and port validation ---'
rg -n -C 6 'api:|sftp:|port:|Port:|Listen:|validate|Validate' config . --glob '*.yml' --glob '*.yaml' --glob '*.go' | head -350
printf '%s\n' '--- deterministic listener-conflict probe ---'
python3 - <<'PY'
import socket
def bind_pair(port_a, port_b):
sockets = []
try:
for port in (port_a, port_b):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", port))
sockets.append(s)
return "no conflict"
except OSError as exc:
return f"{type(exc).__name__}: {exc}"
finally:
for s in sockets:
s.close()
for name, ports in {
"ACME HTTP-01 plus API on 80": (80, 80),
"API and SFTP equal": (2022, 2022),
"documented distinct ports": (8080, 2022),
}.items():
print(f"{name}: {bind_pair(*ports)}")
PYRepository: pelican/wings Length of output: 27635 🏁 Script executed: #!/bin/bash
set -e
python3 - <<'PY'
import socket
def free_port():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
s.close()
return port
def bind_pair(a, b):
sockets = []
try:
for port in (a, b):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", port))
sockets.append(s)
return "bind succeeds"
except OSError as exc:
return f"bind fails: {exc}"
finally:
for s in sockets:
s.close()
distinct_a = free_port()
distinct_b = free_port()
while distinct_b == distinct_a:
distinct_b = free_port()
for label, a, b in [
("ACME HTTP-01 plus API on 80", 80, 80),
("API and SFTP equal", distinct_a, distinct_a),
("API and SFTP distinct", distinct_a, distinct_b),
]:
print(f"{label}: {bind_pair(a, b)}")
def valid_service_port(value):
try:
port = int(value, 10)
except ValueError:
return False
return 1 <= port <= 65535 and port != 80
for value in ["", "0", "1", "80", "2022", "65535", "65536", "-1", "abc"]:
print(f"service-port {value!r}: {'accepted' if valid_service_port(value) else 'rejected'}")
PYRepository: pelican/wings Length of output: 586 🌐 Web query:
💡 Result: In containerization, specifically within Docker and Docker Compose, port mappings facilitate communication between a host machine and a container [1][2]. Target ports and published ports are defined as follows: - Target: Refers to the specific port on which the service inside the container is listening [1]. - Published: Refers to the port on the host machine that is exposed to the external network [1]. When traffic hits this published port on the host, it is routed to the corresponding target port inside the container [1]. Regarding unique and duplicate port mappings: - Port mappings must be unique per host interface. Because a specific port on a host machine can only be bound by one process at a time, you cannot map multiple containers to the same published host port on the same IP address [1]. - Attempting to map multiple containers to the same published host port will result in a bind error, typically indicating that the address is already in use [1]. - If you do not specify a published port, the container runtime can automatically allocate an unassigned, unique port on the host [1]. - Port mapping should not be used when network_mode is set to host, as the container already shares the host's networking stack directly, making explicit mappings redundant and resulting in a runtime error [1]. Citations:
Reject invalid or conflicting service-port values before Compose starts. The 🤖 Prompt for AI Agents |
||
| tty: true | ||
| environment: | ||
| TZ: "UTC" | ||
| WINGS_UID: 988 | ||
| WINGS_GID: 988 | ||
| WINGS_USERNAME: pelican | ||
| volumes: | ||
| - "/var/run/docker.sock:/var/run/docker.sock" | ||
| - "/var/lib/docker/containers/:/var/lib/docker/containers/" | ||
| - "/etc/pelican/:/etc/pelican/" | ||
| - "/var/lib/pelican/:/var/lib/pelican/" | ||
| - "/var/log/pelican/:/var/log/pelican/" | ||
| - "/tmp/pelican/:/tmp/pelican/" | ||
| - "/etc/ssl/certs:/etc/ssl/certs:ro" | ||
| # You may need /srv/daemon-data when upgrading from an old daemon. | ||
| #- "/srv/daemon-data/:/srv/daemon-data/" | ||
|
|
||
| networks: | ||
| wings0: | ||
| name: wings0 | ||
| driver: bridge | ||
| ipam: | ||
| config: | ||
| - subnet: "172.21.0.0/16" | ||
| driver_opts: | ||
| com.docker.network.bridge.name: wings0 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -eu | ||
|
|
||
| script_dir=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd) | ||
| compose_file="$script_dir/docker-compose.auto-tls.example.yml" | ||
| env_file='' | ||
| env_file_count=0 | ||
| check_only=false | ||
| expect_env_file=false | ||
|
|
||
| for argument do | ||
| if [ "$expect_env_file" = true ]; then | ||
| env_file=$argument | ||
| env_file_count=$((env_file_count + 1)) | ||
| expect_env_file=false | ||
| continue | ||
| fi | ||
|
|
||
| case $argument in | ||
| --env-file) | ||
| expect_env_file=true | ||
| ;; | ||
| --env-file=*) | ||
| env_file=${argument#--env-file=} | ||
| env_file_count=$((env_file_count + 1)) | ||
| ;; | ||
| --check) | ||
| check_only=true | ||
| ;; | ||
| esac | ||
| done | ||
|
|
||
| if [ "$expect_env_file" = true ]; then | ||
| printf '%s\n' 'Automatic TLS preflight failed: --env-file requires a path.' >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ "$env_file_count" -gt 1 ]; then | ||
| printf '%s\n' 'Automatic TLS preflight failed: only one --env-file is supported.' >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ "$env_file_count" -eq 1 ] && [ -z "$env_file" ]; then | ||
| printf '%s\n' 'Automatic TLS preflight failed: --env-file requires a path.' >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Compose, rather than this shell, must expand the interpolation expressions. | ||
| # shellcheck disable=SC2016 | ||
| compose_probe='services: | ||
| preflight: | ||
| image: scratch | ||
| environment: | ||
| WINGS_API_PORT: "${WINGS_API_PORT-}" | ||
| WINGS_SFTP_PORT: "${WINGS_SFTP_PORT-}"' | ||
|
|
||
| if [ "$env_file_count" -eq 1 ]; then | ||
| if ! compose_environment=$(printf '%s\n' "$compose_probe" | docker compose \ | ||
| --project-directory "$script_dir" \ | ||
| --env-file "$env_file" \ | ||
| -f - \ | ||
| config --environment); then | ||
| printf '%s\n' 'Automatic TLS preflight failed: Docker Compose could not resolve the environment.' >&2 | ||
| exit 1 | ||
| fi | ||
| else | ||
| if ! compose_environment=$(printf '%s\n' "$compose_probe" | docker compose \ | ||
| --project-directory "$script_dir" \ | ||
| -f - \ | ||
| config --environment); then | ||
| printf '%s\n' 'Automatic TLS preflight failed: Docker Compose could not resolve the environment.' >&2 | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| read_resolved_value() { | ||
| variable_name=$1 | ||
|
|
||
| printf '%s\n' "$compose_environment" | awk -v wanted="$variable_name" ' | ||
| index($0, wanted "=") == 1 { | ||
| print substr($0, length(wanted) + 2) | ||
| exit | ||
| } | ||
| ' | ||
| } | ||
|
|
||
| api_port=$(read_resolved_value WINGS_API_PORT) | ||
| sftp_port=$(read_resolved_value WINGS_SFTP_PORT) | ||
|
|
||
| validate_port() { | ||
| variable_name=$1 | ||
| port=$2 | ||
|
|
||
| if [ -z "$port" ]; then | ||
| printf 'Automatic TLS preflight failed: %s is required.\n' "$variable_name" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| case $port in | ||
| *[!0-9]*) | ||
| printf 'Automatic TLS preflight failed: %s must be a numeric port from 1 through 65535.\n' "$variable_name" >&2 | ||
| exit 1 | ||
| ;; | ||
| esac | ||
|
|
||
| if ! awk -v port="$port" 'BEGIN { exit !(port >= 1 && port <= 65535) }'; then | ||
| printf 'Automatic TLS preflight failed: %s must be a numeric port from 1 through 65535.\n' "$variable_name" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if awk -v port="$port" 'BEGIN { exit !(port == 80) }'; then | ||
| printf 'Automatic TLS preflight failed: %s cannot use port 80; Wings reserves it for the ACME HTTP-01 challenge.\n' "$variable_name" >&2 | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| validate_port WINGS_API_PORT "$api_port" | ||
| validate_port WINGS_SFTP_PORT "$sftp_port" | ||
|
|
||
| if awk -v api="$api_port" -v sftp="$sftp_port" 'BEGIN { exit !(api == sftp) }'; then | ||
| printf '%s\n' 'Automatic TLS preflight failed: WINGS_API_PORT and WINGS_SFTP_PORT must be different.' >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| printf 'Automatic TLS port preflight passed (API %s, SFTP %s).\n' "$api_port" "$sftp_port" | ||
|
|
||
| if [ "$check_only" = true ]; then | ||
| exit 0 | ||
| fi | ||
|
|
||
| exec docker compose --project-directory "$script_dir" -f "$compose_file" "$@" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| #!/bin/sh | ||
|
|
||
| set -eu | ||
|
|
||
| repository_root=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd) | ||
| wrapper="$repository_root/docker-compose.auto-tls.sh" | ||
| empty_env=$(mktemp) | ||
| configured_env=$(mktemp) | ||
| project_directory=$(mktemp -d) | ||
| working_directory=$(mktemp -d) | ||
| trap 'rm -f "$empty_env" "$configured_env"; rm -rf "$project_directory" "$working_directory"' EXIT HUP INT TERM | ||
|
|
||
| printf '%s\n' \ | ||
| 'WINGS_API_PORT="8443"' \ | ||
| 'WINGS_SFTP_PORT=2022 # default SFTP port' \ | ||
| > "$configured_env" | ||
|
|
||
| cp "$wrapper" "$project_directory/docker-compose.auto-tls.sh" | ||
| cp "$repository_root/docker-compose.auto-tls.example.yml" \ | ||
| "$project_directory/docker-compose.auto-tls.example.yml" | ||
| # Preserve these expressions literally so Docker Compose resolves them. | ||
| # shellcheck disable=SC2016 | ||
| printf '%s\n' \ | ||
| 'DEFAULT_API_PORT=8443' \ | ||
| 'WINGS_API_PORT=${DEFAULT_API_PORT:-443}' \ | ||
| 'WINGS_SFTP_PORT=${DEFAULT_SFTP_PORT:-2022}' \ | ||
| > "$project_directory/.env" | ||
|
|
||
| expect_success() { | ||
| description=$1 | ||
| shift | ||
|
|
||
| if ! output=$("$@" 2>&1); then | ||
| printf 'FAIL: %s\n%s\n' "$description" "$output" >&2 | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| expect_failure() { | ||
| description=$1 | ||
| expected=$2 | ||
| shift 2 | ||
|
|
||
| if output=$("$@" 2>&1); then | ||
| printf 'FAIL: %s unexpectedly passed.\n' "$description" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! printf '%s\n' "$output" | grep -F "$expected" >/dev/null; then | ||
| printf 'FAIL: %s returned an unexpected error.\n%s\n' "$description" "$output" >&2 | ||
| exit 1 | ||
| fi | ||
| } | ||
|
|
||
| expect_failure 'unset API port' 'WINGS_API_PORT is required' \ | ||
| env -u WINGS_API_PORT -u WINGS_SFTP_PORT sh "$wrapper" --env-file "$empty_env" --check | ||
| expect_failure 'non-numeric API port' 'WINGS_API_PORT must be a numeric port from 1 through 65535' \ | ||
| env WINGS_API_PORT=invalid WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_failure 'port below the lower boundary' 'WINGS_API_PORT must be a numeric port from 1 through 65535' \ | ||
| env WINGS_API_PORT=0 WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_failure 'port above the upper boundary' 'WINGS_API_PORT must be a numeric port from 1 through 65535' \ | ||
| env WINGS_API_PORT=65536 WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_failure 'reserved ACME port' 'WINGS_API_PORT cannot use port 80' \ | ||
| env WINGS_API_PORT=80 WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_failure 'duplicate service ports' 'WINGS_API_PORT and WINGS_SFTP_PORT must be different' \ | ||
| env WINGS_API_PORT=2022 WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_failure 'multiple environment files' 'only one --env-file is supported' \ | ||
| env WINGS_API_PORT=8443 WINGS_SFTP_PORT=2022 sh "$wrapper" \ | ||
| --env-file "$empty_env" --env-file "$configured_env" --check | ||
|
|
||
| expect_success 'lower and upper boundaries' \ | ||
| env WINGS_API_PORT=1 WINGS_SFTP_PORT=65535 sh "$wrapper" --check | ||
| expect_success 'typical API and SFTP ports' \ | ||
| env WINGS_API_PORT=8443 WINGS_SFTP_PORT=2022 sh "$wrapper" --check | ||
| expect_success 'ports loaded from a Compose environment file' \ | ||
| env -u WINGS_API_PORT -u WINGS_SFTP_PORT sh "$wrapper" --env-file "$configured_env" --check | ||
| # Positional parameters are intentionally expanded by the child shell. | ||
| # shellcheck disable=SC2016 | ||
| expect_success 'project environment interpolation from another working directory' \ | ||
| env -u WINGS_API_PORT -u WINGS_SFTP_PORT sh -c \ | ||
| 'cd "$1" && sh "$2/docker-compose.auto-tls.sh" --check' \ | ||
| test "$working_directory" "$project_directory" | ||
|
|
||
| printf '%s\n' 'Automatic TLS Compose preflight tests passed.' |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.