When secondary (non-init) server nodes attempt to join the cluster, the RKE2 installation process executes repeatedly in a loop (observed up to 4 times) before successfully joining. This results in duplicate, identical configuration blocks being appended repeatedly to /var/lib/elemental/kubernetes/server.yaml.
This behavior was observed immediately after switching the cluster ingress from rke2-ingress-nginx to rke2-traefik.
Because rke2-traefik is significantly lighter and initializes faster than rke2-ingress-nginx, a race condition appears to be occurring. MetalLB may be announcing the kubernetes-vip before the secondary nodes are fully ready to process/record it. This transient failure causes the bootstrap logic to interpret the join as failed, fallback, and re-trigger the entire RKE2 installation routine. With rke2-ingress-nginx, the slower startup may be masking this timing window.
Steps to Reproduce
- Configure an Elemental/RKE2 cluster setup using a Virtual IP (
kubernetes-vip) managed by MetalLB.
- Set the ingress controller to
rke2-traefik (instead of rke2-ingress-nginx).
- Boot the initial master node, followed by secondary master nodes.
- Check
/var/lib/elemental/kubernetes/server.yaml and system logs on the secondary nodes during the join phase.
Expected Behavior
Secondary nodes should copy the configuration, run the embedded artifact installation exactly once, and smoothly join the cluster without duplicating file entries.
Actual Behavior
The installation sequence loops multiple times. Inspection of /var/lib/elemental/kubernetes/server.yaml reveals that the same configuration block is appended multiple times (corresponding to the number of install retries).
Relevant Logs
The block below repeats up to 4 times in the Stack Validation environment before the node settles and may be different on different environments.
Jun 04 06:15:37 uchost3rke2 bash[2077]: Copying RKE2 config file /var/lib/elemental/kubernetes/server.yaml
Jun 04 06:15:37 uchost3rke2 bash[2077]: Installing RKE2 from embedded artifacts...
Jun 04 06:15:37 uchost3rke2 bash[2083]: [INFO] staging local checksums from /opt/k8s/install/sha256sum-amd64.txt
Jun 04 06:15:37 uchost3rke2 bash[2083]: [INFO] staging tarball from /opt/k8s/install/rke2.linux-amd64.tar.gz
Jun 04 06:15:37 uchost3rke2 bash[2083]: [INFO] verifying tarball
Jun 04 06:15:37 uchost3rke2 bash[2083]: [INFO] unpacking tarball file to /opt/rke2
Jun 04 06:15:38 uchost3rke2 bash[2083]: [INFO] updating tarball contents to reflect install path
Jun 04 06:15:38 uchost3rke2 bash[2083]: [INFO] moving systemd units to /etc/systemd/system
Jun 04 06:15:38 uchost3rke2 bash[2083]: [INFO] install complete; you may want to run: export PATH=$PATH:/opt/rke2/bin
When secondary (non-init) server nodes attempt to join the cluster, the RKE2 installation process executes repeatedly in a loop (observed up to 4 times) before successfully joining. This results in duplicate, identical configuration blocks being appended repeatedly to
/var/lib/elemental/kubernetes/server.yaml.This behavior was observed immediately after switching the cluster ingress from
rke2-ingress-nginxtorke2-traefik.Because
rke2-traefikis significantly lighter and initializes faster thanrke2-ingress-nginx, a race condition appears to be occurring.MetalLBmay be announcing thekubernetes-vipbefore the secondary nodes are fully ready to process/record it. This transient failure causes the bootstrap logic to interpret the join as failed, fallback, and re-trigger the entire RKE2 installation routine. Withrke2-ingress-nginx, the slower startup may be masking this timing window.Steps to Reproduce
kubernetes-vip) managed by MetalLB.rke2-traefik(instead ofrke2-ingress-nginx)./var/lib/elemental/kubernetes/server.yamland system logs on the secondary nodes during the join phase.Expected Behavior
Secondary nodes should copy the configuration, run the embedded artifact installation exactly once, and smoothly join the cluster without duplicating file entries.
Actual Behavior
The installation sequence loops multiple times. Inspection of
/var/lib/elemental/kubernetes/server.yamlreveals that the same configuration block is appended multiple times (corresponding to the number of install retries).Relevant Logs
The block below repeats up to 4 times in the Stack Validation environment before the node settles and may be different on different environments.