Workshop Namespace Operator

Kubernetes Operator for setting up Kubernetes Namespace and User privileged for workshop sessions. Repository also provides CRD APIs for golang in folder pkg/apis.

About

Workshop Namespace Operator is created using operator-sdk framework.

The operator will ensure that there is namespace for each WorkshopNamespace CR.

For example this Custom Resource:

apiVersion: operator.prgcont.cz/v1alpha1
kind: WorkshopNamespace
metadata:
  name: example-ns

will result in:

• Namespace: example-ns
  • ServiceAccount: workshop-user
  • RoleBinding granting workshop-user namespaced cluster-admin privileges
• Namespace: default (or namespace where operator runs in)
  • Secret: kubeconfig-example-ns
    • Secret contains kubeconfig for created ServiceAccount

Configuration

Operator is configured with ConfigMap kubernetes-server, it must contain key data.server which declares which Server will be set in all generated kubeconfigs.

Example config:

apiVersion: v1 
kind: ConfigMap
metadata:
  name: kubernetes-server
data:
  server: https://workshop.prgcont.cz:443

Deploy to k8s cluster

Publish container:

operator-sdk build prgcont/workshop-namespace-operator:v0.0.2
docker push prgcont/workshop-namespace-operator:v0.0.2

Create CRD in target cluster:

export OPERATOR_NAMESPACE=default
kubectl -n ${OPERATOR_NAMESPACE} create -f deploy/crds/workshopnamespaces_v1alpha1_operator_crd.yaml

Create Operator

kubectl -n ${OPERATOR_NAMESPACE} create -f deploy/role.yaml,deploy/role_binding.yaml,deploy/clusterrole_binding.yaml,deploy/service_account.yaml
# Update operator container image and deploy to cluster 
sed 's/{{ REPLACE_IMAGE }}/prgcont\/workshop-namespace-operator:v0.0.2/' deploy/operator.yaml | kubectl -n ${OPERATOR_NAMESPACE} create -f -
sed 's/{{ KUBERNETES_SERVER }}/https:\/\/192.168.64.21:8443/' deploy/config.yaml | kubectl -n ${OPERATOR_NAMESPACE} create -f -

Create test CR to verify if namespace is created:

kubectl -n ${OPERATOR_NAMESPACE} create -f deploy/crds/workshopnamespaces_v1alpha1_operator_cr.yaml

Verify that Namespace test-ns was created

kubectl get ns

# Objects in namespace
kubectl -n test-ns get serviceaccount,rolebinding
# NAME               SECRETS   AGE
# sa/default         1         3d
# sa/workshop-user   1         3d

# NAME                        KIND                                       SUBJECTS
# rolebindings/test-nsadmin   RoleBinding.v1.rbac.authorization.k8s.io   1 item(s)

Cleanup cluster

kubectl -n ${OPERATOR_NAMESPACE} delete workshopnamespace $(kubectl get workshopnamespace -o jsonpath='{.items[*].metadata.name}')
kubectl -n ${OPERATOR_NAMESPACE} delete -f deploy/role.yaml,deploy/role_binding.yaml,deploy/clusterrole_binding.yaml,deploy/service_account.yaml
kubectl -n ${OPERATOR_NAMESPACE} delete deployment workshop-namespace-operator
kubectl -n ${OPERATOR_NAMESPACE} delete configmap kubernetes-server
kubectl delete -f deploy/crds/workshopnamespaces_v1alpha1_operator_crd.yaml

Local testing

Prerequisites

• git
• docker version 17.03+.
• kubectl version v1.9.0+.
• ansible version v2.6.0+
• ansible-runner version v1.1.0+
• ansible-runner-http version v1.0.0+
• dep version v0.5.0+. (Optional if you aren’t installing from source)
• go version v1.10+. (Optional if you aren’t installing from source)
• Access to a Kubernetes v.1.9.0+ cluster.

See official prerequisites for more details.

Register CRD:

kubectl apply -f ./deploy/crds/workshopnamespaces_v1alpha1_operator_crd.yaml

Getting Started

Start minikube cluster

minikube start --kubernetes-version v1.12.4

Create CRD in k8s API

kubectl apply -f deploy/crds/workshopnamespaces_v1alpha1_operator_crd.yaml

Update watches.yaml role section to reflect path on your computer, e.g. /home/<USERNAME>/workshop-namespace-operator/roles/workshopnamespace.

Start Operator locally:

# Either start runner directly
ansible-runner -vv --rotate-artifacts 1 --role workshopnamespace --roles-path ~/.go/src/github.com/prgcont/workshop-namespace-operator/roles/ --hosts localhost -i test run ./

# or using operator-sdk
operator-sdk up local

Create test CR:

kubectl apply -f ./deploy/crds/workshopnamespaces_v1alpha1_operator_cr.yaml

Develop

Adding k8s go client CRD using kubebuilder (already done).

kubebuilder init --domain prgcont.cz --license apache2 --owner "The Prgcont Team"
kubebuilder create api --group operator --version v1alpha1 --kind WorkshopNamespace

Re-generate go client libraries for WorkshopNamespace CRD.

vendor/k8s.io/code-generator/generate-groups.sh all \
                                                github.com/prgcont/workshop-namespace-operator/pkg/client \
                                                github.com/prgcont/workshop-namespace-operator/pkg/apis \
                                                operator:v1alpha1

It is necessary to re-generate client libraries every CRD is updated.