not all config parameters are parsed

[chuegel]

Hi,

ProxyInjector is a really nice idea so we played around a bit and noticed that not all config parameters are parsed into the container when launching a annotated application.

Here is my ProxyInjector deployment:

---
# Source: proxyinjector/templates/rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: 0.0.1
    chart: "proxyinjector-0.0.1"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: 0.0.1
    chart: "proxyinjector-0.0.1"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector-role
  namespace: test-proxyinjector
rules:
- apiGroups:
  - ""
  - "extensions"
  - "apps"
  resources:
  - deployments
  - daemonsets
  - statefulsets
  - services
  verbs:
  - list
  - get
  - watch
  - update
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: 0.0.1
    chart: "proxyinjector-0.0.1"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector-role-binding
  namespace: test-proxyinjector
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: proxyinjector-role
subjects:
- kind: ServiceAccount
  name: proxyinjector
  namespace: test-proxyinjector




---
# Source: proxyinjector/templates/deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: 0.0.1
    chart: "proxyinjector-0.0.1"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector
spec:
  replicas: 1
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: proxyinjector
      group: com.stakater.platform
      provider: stakater
  template:
    metadata:
      labels:
        app: proxyinjector
        group: com.stakater.platform
        provider: stakater
    spec:
      containers:
      - env:
        image: "stakater/proxyinjector:0.0.1"
        imagePullPolicy: Always
        name: proxyinjector
      serviceAccountName: proxyinjector

---
# Source: proxyinjector/templates/deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: v0.0.16
    chart: "proxyinjector-v0.0.16"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector
  namespace: idm-test
spec:
  replicas: 1
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: proxyinjector
      group: com.stakater.platform
      provider: stakater
  template:
    metadata:
      labels:
        app: proxyinjector
        group: com.stakater.platform
        provider: stakater
    spec:
      containers:
      - env:
        - name: CONFIG_FILE_PATH
          value: "/etc/ProxyInjector/config.yml"
        image: "stakater/proxyinjector:v0.0.16"
        imagePullPolicy: IfNotPresent
        name: proxyinjector
        volumeMounts:
          - mountPath: /etc/ProxyInjector
            name: config-volume
      serviceAccountName: proxyinjector
      volumes:
      - configMap:
          name: proxyinjector
        name: config-volume

---
# Source: proxyinjector/templates/rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: v0.0.16
    chart: "proxyinjector-v0.0.16"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: v0.0.16
    chart: "proxyinjector-v0.0.16"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector-role
  namespace: idm-test
rules:
  - apiGroups:
      - ""
      - "extensions"
      - "apps"
    resources:
      - deployments
      - daemonsets
      - statefulsets
      - services
      - configmaps
    verbs:
      - list
      - get
      - watch
      - update
      - create
      - patch
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  labels:
    app: proxyinjector
    group: com.stakater.platform
    provider: stakater
    version: v0.0.16
    chart: "proxyinjector-v0.0.16"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector-role-binding
  namespace: idm-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: proxyinjector-role
subjects:
  - kind: ServiceAccount
    name: proxyinjector
    namespace: idm-test
---
# Source: proxyinjector/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: proxyinjector
    version: v0.0.16
    group: com.stakater.platform
    provider: stakater
    chart: "proxyinjector-v0.0.16"
    release: "RELEASE-NAME"
    heritage: "Tiller"
  name: proxyinjector
data:
  config.yml:
    |-
      gatekeeper-image: "keycloak/keycloak-gatekeeper:latest"
      discovery-url: http://nas-10-keycloak.lan:8080/auth/realms/sso
      client-id: metropolis
      listen: 0.0.0.0:3000
      client-secret: #####0ef-4c84-##19-80ca-9c21#####4e7
      encryption-key: AgXa7xRco#####0ZDSH4X0XhL5Q#####
      preserve-host: true
      http-only-cookie: true
      enable-token-header: true
      enable-login-handler: true
      enable-authorization-header: true
      enable-refresh-tokens: true
      enable-session-cookies: true
      redirection-url: http://nas-10-metropolis.lan
      secure-cookie: false
      http-only-cookie: true
      verbose: true
      enable-default-deny: true
      add-claims:
       - name
       - email
       - given_name
      resources:
      - uri: /*
        roles:
        - metropolis
      - uri: /superman/*
        roles:
        - metropolis-admins

the created configmap:

gatekeeper-image: "keycloak/keycloak-gatekeeper:latest"
discovery-url: http://nas-10-keycloak.lan:8080/auth/realms/sso
client-id: metropolis
listen: 0.0.0.0:3000
client-secret: #####0ef-4c84-##19-80ca-9c21#####4e7
encryption-key: AgXa7xRco#####0ZDSH4X0XhL5Q#####
preserve-host: true
http-only-cookie: true
enable-token-header: true
enable-login-handler: true
enable-authorization-header: true
enable-refresh-tokens: true
enable-session-cookies: true
redirection-url: http://nas-10-metropolis.lan
secure-cookie: false
http-only-cookie: true
verbose: true
enable-default-deny: true
add-claims:
 - name
 - email
 - given_name
resources:
- uri: /*
  roles: 
  - metropolis
- uri: /superman/*
  roles:
  - metropolis-admins

and here the yaml of the deployed application:

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  annotations:
    authproxy.stakater.com/enabled: "true"
    authproxy.stakater.com/listen: 0.0.0.0:3000
    authproxy.stakater.com/redirection-url: http://nas-10-metropolis.lan
    authproxy.stakater.com/source-service-name: metropolis
    authproxy.stakater.com/upstream-url: http://127.0.0.1
    deployment.kubernetes.io/revision: "2"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{"authproxy.stakater.com/enabled":"true","authproxy.stakater.com/listen":"0.0.0.0:3000","authproxy.stakater.com/redirection-url":"http://nas-10-metropolis.lan","authproxy.stakater.com/source-service-name":"metropolis","authproxy.stakater.com/upstream-url":"http://127.0.0.1"},"name":"metropolis","namespace":"idm-test"},"spec":{"replicas":1,"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx:1.7.9","name":"nginx","ports":[{"containerPort":80}]}]}}}}
  creationTimestamp: "2019-05-20T16:48:57Z"
  generation: 2
  labels:
    app: nginx
  name: metropolis
  namespace: idm-test
  resourceVersion: "225959"
  selfLink: /apis/apps/v1beta2/namespaces/idm-test/deployments/metropolis
  uid: 295c27ce-7b1f-11e9-af34-c6fdae0627ad
spec:
  progressDeadlineSeconds: 2147483647
  replicas: 1
  revisionHistoryLimit: 2147483647
  selector:
    matchLabels:
      app: nginx
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - args:
        - --client-id=metropolis
        - --client-secret= #####0ef-4c84-##19-80ca-9c21#####4e7
        - --discovery-url=http://nas-10-keycloak.lan:8080/auth/realms/sso
        - --secure-cookie=false
        - --verbose=true
        - --resources=uri=/*
        - --resources=uri=/superman/*
        - --listen=0.0.0.0:3000
        - --redirection-url=http://nas-10-metropolis.lan
        - --upstream-url=http://127.0.0.1
        image: keycloak/keycloak-gatekeeper:latest
        imagePullPolicy: Always
        name: proxy
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - image: nginx:1.7.9
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2019-05-20T16:48:58Z"
    lastUpdateTime: "2019-05-20T16:48:58Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 2
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

as you see there are a lot of config options missing.

Edit:

when deploying the app annotated like this

authproxy.stakater.com/resources: uri=/*|roles=metropolis,metropolis-admins

it works:

...
template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - args:
        - --client-id=metropolis
        - --client-secret=#####0ef-4c84-##19-80ca-9c21#####4e7
        - --discovery-url=http://nas-10-keycloak.lan:8080/auth/realms/sso
        - --secure-cookie=false
        - --verbose=true
        - --listen=0.0.0.0:3000
        - --redirection-url=http://nas-10-metropolis.lan
        - --upstream-url=http://127.0.0.1
        - --resources=uri=/*|roles=metropolis,metropolis-admins
        image: keycloak/keycloak-gatekeeper:latest
...

[chuegel]

I wounder how ProxyInjector invokes keycloak-gatekeeper: shouldnt it be through the config.yml file like /keycloak-gatekeeper --config config.yml?

[rasheedamir]

@huegelc is this issue still valid?

[chuegel]

@rasheedamir
yes, it is

This is the secret config.yaml for proxyinjector:

discovery-url: https://sso.xxxx.com/auth/realms/nv-services
gatekeeper-image: "keycloak/keycloak-gatekeeper:7.0.0"
upstream-url: http://127.0.0.1:8080
redirection-url: https://hello.xxxxx.com
target-port: 3000
source-service-name: http-svc
client-id: hello
listen: 0.0.0.0:3000
client-secret: xxxxxxxxx
encryption-key: AgXa7xRxxxxxxxxxxxhL5Qy2xxxxx
preserve-host: true
http-only-cookie: true
enable-token-header: true
enable-login-handler: true
enable-authorization-header: true
enable-refresh-tokens: true
enable-session-cookies: true
secure-cookie: true
verbose: true
enable-default-deny: true
add-claims: name
add-claims: email
add-claims: given_name

and this are the env of the pod:

---snip---
spec:
  containers:
  - args:
    - --client-id=hello
    - --client-secret=xxxxxxxxxxxxxxxxxxxxx
    - --discovery-url=https://sso.xxxx.com/auth/realms/nv-services
    - --listen=0.0.0.0:3000
    - --secure-cookie=true
    - --verbose=true
    - --redirection-url=https://hello.xxxx.com
    - --upstream-url=http://127.0.0.1:8080
    - --resources=uri=/*|roles=someother-role|require-any-role=true
    image: keycloak/keycloak-gatekeeper:7.0.0
    imagePullPolicy: IfNotPresent
    name: proxy
----snip----

You see, there are a lot of config options missing/not passed to the proxy pod

And here is the annotation of the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-svc
  annotations:
    authproxy.stakater.com/enabled: "true"
#    authproxy.stakater.com/redirection-url: https://hello.xxxxx.com
#    authproxy.stakater.com/resources: uri=/*|roles=g-xxxx-Admin|require-any-role=true
#    authproxy.stakater.com/source-service-name: http-svc
#    authproxy.stakater.com/target-port: "3000"
#    authproxy.stakater.com/upstream-url: http://127.0.0.1:8080

It seems that resources: uri=/*|roles=g-xxxx-Admin|require-any-role=truein the annotation is mandatory as the configuration yaml doesnt pass the env variable
level=error msg="Failed to get latest version of Service: resource name may not be empty"
This is due to the fact that the syntax is probably wrong in the config.yaml

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  labels:
    app: proxyinjector
    version: v0.0.23
    group: com.stakater.platform
    provider: stakater
    chart: "proxyinjector-v0.0.23"
    release: "proxyinjector"
    heritage: "Tiller"
  name: proxyinjector
  namespace: default
stringData:
  config.yaml: |-
    discovery-url: https://sso.xxxxx.com/auth/realms/nv-services
    gatekeeper-image: "keycloak/keycloak-gatekeeper:7.0.0"
    upstream-url: http://127.0.0.1:8080
    redirection-url: https://hello.xxxxx.com
    target-port: 3000
    listen: 0.0.0.0:3000
    resources:
      - uri: /*
        roles:
        - g-xxxx-Admins
        require-any-role: true
    client-secret: xxxxxxxxxx
    encryption-key: xxxxxxxxx
    preserve-host: true
    http-only-cookie: true
    enable-token-header: true
    enable-login-handler: true
    enable-authorization-header: true
    enable-refresh-tokens: true
    enable-session-cookies: true
    secure-cookie: true
    verbose: true
    enable-default-deny: true
    add-claims: name
    add-claims: email
    add-claims: given_name