Skip to content

Keycloak hostname and TLS cert break fulfillment-controller OIDC discovery #57

Description

@karmab

Summary

The osac-prereqs chart hardcodes KC_HOSTNAME=keycloak.keycloak.svc.cluster.local and the Keycloak TLS certificate only includes internal SANs. This causes fulfillment-controller to crash-loop on a fresh install because OIDC discovery fails.

Root cause

Two issues in charts/osac-prereqs/templates/keycloak/resources.yaml:

1. KC_HOSTNAME set to internal service name

- name: KC_HOSTNAME
  value: "keycloak.keycloak.svc.cluster.local"

The fulfillment-controller is configured with --auth-issuer-url=https://keycloak-keycloak.apps.<cluster>/realms/osac (external route). When KC_HOSTNAME is the internal name, OIDC discovery at the external URL returns the internal hostname as issuer — the issuer mismatch causes the controller's OIDC library to discard the response, leaving the token endpoint empty.

The fulfillment-grpc-server also validates tokens with --grpc-authn-trusted-token-issuers set to the external URL, so the issuer in tokens must match.

2. TLS certificate missing external route hostname

dnsNames:
- keycloak
- keycloak.keycloak.svc.cluster.local

Even after fixing KC_HOSTNAME, the controller gets x509: certificate is valid for keycloak, keycloak.keycloak.svc.cluster.local, not keycloak-keycloak.apps.<cluster> when trying to reach Keycloak via the external route.

Error from controller logs

Failed to fetch metadata
url: https://keycloak-keycloak.apps.osac.192-168-254-254.sslip.io/realms/osac/.well-known/openid-configuration
error: tls: failed to verify certificate: x509: certificate is valid for keycloak, keycloak.keycloak.svc.cluster.local, not keycloak-keycloak.apps.osac.192-168-254-254.sslip.io

Followed by:

Failed to send token form
endpoint: ""
error: Post "": unsupported protocol scheme ""

Fix applied

# 1. Add external route hostname to TLS cert
kubectl patch certificate keycloak-tls -n keycloak --type=json \
  -p '[{"op":"add","path":"/spec/dnsNames/-","value":"keycloak-keycloak.apps.osac.192-168-254-254.sslip.io"}]'

# 2. Set KC_HOSTNAME to external route
kubectl set env deploy/keycloak-service -n keycloak \
  KC_HOSTNAME=keycloak-keycloak.apps.osac.192-168-254-254.sslip.io

# 3. Restart Keycloak, then fulfillment-controller
kubectl rollout restart deploy/keycloak-service -n keycloak
kubectl rollout restart deploy/fulfillment-controller -n osac

Suggested chart fix

The route doesn't set an explicit host, so OpenShift auto-generates it. The chart should either:

  1. Accept the external hostname as a Helm value (e.g. keycloak.externalHostname) and use it for both KC_HOSTNAME and the cert dnsNames, or
  2. Template it from the cluster's ingress domain (e.g. keycloak-keycloak.apps.{{ .Values.clusterDomain }})

Either way, the external route hostname must appear in both the KC_HOSTNAME env var and the TLS certificate SANs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions