Summary
The osac-prereqs chart hardcodes KC_HOSTNAME=keycloak.keycloak.svc.cluster.local and the Keycloak TLS certificate only includes internal SANs. This causes fulfillment-controller to crash-loop on a fresh install because OIDC discovery fails.
Root cause
Two issues in charts/osac-prereqs/templates/keycloak/resources.yaml:
1. KC_HOSTNAME set to internal service name
- name: KC_HOSTNAME
value: "keycloak.keycloak.svc.cluster.local"
The fulfillment-controller is configured with --auth-issuer-url=https://keycloak-keycloak.apps.<cluster>/realms/osac (external route). When KC_HOSTNAME is the internal name, OIDC discovery at the external URL returns the internal hostname as issuer — the issuer mismatch causes the controller's OIDC library to discard the response, leaving the token endpoint empty.
The fulfillment-grpc-server also validates tokens with --grpc-authn-trusted-token-issuers set to the external URL, so the issuer in tokens must match.
2. TLS certificate missing external route hostname
dnsNames:
- keycloak
- keycloak.keycloak.svc.cluster.local
Even after fixing KC_HOSTNAME, the controller gets x509: certificate is valid for keycloak, keycloak.keycloak.svc.cluster.local, not keycloak-keycloak.apps.<cluster> when trying to reach Keycloak via the external route.
Error from controller logs
Failed to fetch metadata
url: https://keycloak-keycloak.apps.osac.192-168-254-254.sslip.io/realms/osac/.well-known/openid-configuration
error: tls: failed to verify certificate: x509: certificate is valid for keycloak, keycloak.keycloak.svc.cluster.local, not keycloak-keycloak.apps.osac.192-168-254-254.sslip.io
Followed by:
Failed to send token form
endpoint: ""
error: Post "": unsupported protocol scheme ""
Fix applied
# 1. Add external route hostname to TLS cert
kubectl patch certificate keycloak-tls -n keycloak --type=json \
-p '[{"op":"add","path":"/spec/dnsNames/-","value":"keycloak-keycloak.apps.osac.192-168-254-254.sslip.io"}]'
# 2. Set KC_HOSTNAME to external route
kubectl set env deploy/keycloak-service -n keycloak \
KC_HOSTNAME=keycloak-keycloak.apps.osac.192-168-254-254.sslip.io
# 3. Restart Keycloak, then fulfillment-controller
kubectl rollout restart deploy/keycloak-service -n keycloak
kubectl rollout restart deploy/fulfillment-controller -n osac
Suggested chart fix
The route doesn't set an explicit host, so OpenShift auto-generates it. The chart should either:
- Accept the external hostname as a Helm value (e.g.
keycloak.externalHostname) and use it for both KC_HOSTNAME and the cert dnsNames, or
- Template it from the cluster's ingress domain (e.g.
keycloak-keycloak.apps.{{ .Values.clusterDomain }})
Either way, the external route hostname must appear in both the KC_HOSTNAME env var and the TLS certificate SANs.
Summary
The
osac-prereqschart hardcodesKC_HOSTNAME=keycloak.keycloak.svc.cluster.localand the Keycloak TLS certificate only includes internal SANs. This causesfulfillment-controllerto crash-loop on a fresh install because OIDC discovery fails.Root cause
Two issues in
charts/osac-prereqs/templates/keycloak/resources.yaml:1.
KC_HOSTNAMEset to internal service nameThe
fulfillment-controlleris configured with--auth-issuer-url=https://keycloak-keycloak.apps.<cluster>/realms/osac(external route). WhenKC_HOSTNAMEis the internal name, OIDC discovery at the external URL returns the internal hostname as issuer — the issuer mismatch causes the controller's OIDC library to discard the response, leaving the token endpoint empty.The
fulfillment-grpc-serveralso validates tokens with--grpc-authn-trusted-token-issuersset to the external URL, so the issuer in tokens must match.2. TLS certificate missing external route hostname
Even after fixing
KC_HOSTNAME, the controller getsx509: certificate is valid for keycloak, keycloak.keycloak.svc.cluster.local, not keycloak-keycloak.apps.<cluster>when trying to reach Keycloak via the external route.Error from controller logs
Followed by:
Fix applied
Suggested chart fix
The route doesn't set an explicit
host, so OpenShift auto-generates it. The chart should either:keycloak.externalHostname) and use it for bothKC_HOSTNAMEand the certdnsNames, orkeycloak-keycloak.apps.{{ .Values.clusterDomain }})Either way, the external route hostname must appear in both the
KC_HOSTNAMEenv var and the TLS certificate SANs.