Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
### PostgreSQL Operator ###
config/
# Written by the fabric8 Kubernetes client when the tests run against the Dev Service
operator/.kube/

### STS ###
.apt_generated
Expand Down
166 changes: 128 additions & 38 deletions docs/cluster-connection.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ Other Custom Resources (like `Database`, `Role`, `Schema`, `Grant`, `DefaultPriv

## Spec

| Field | Type | Description | Required | Mutable |
|----------------------|----------------------|-----------------------------------------------------------------------|----------|---------|
| `host` | `string` | The hostname of the PostgreSQL instance. | Yes | Yes |
| `port` | `integer` | The port of the PostgreSQL instance (1-65535). | Yes | Yes |
| `database` | `string` | The database to connect to (usually `postgres` for admin operations). | Yes | Yes |
| `adminSecretRef` | `ResourceRef` | Reference to the Kubernetes Secret containing the admin credentials. | No | Yes |
| `adminSecretFileRef` | `FileRef` | Reference to a file containing the admin credentials. | No | Yes |
| `parameters` | `map[string]string` | Additional connection parameters. | No | Yes |
| Field | Type | Description | Required | Mutable |
|----------------------|---------------------|-----------------------------------------------------------------------|----------|---------|
| `host` | `string` | The hostname of the PostgreSQL instance. | Yes | Yes |
| `port` | `integer` | The port of the PostgreSQL instance (1-65535). | Yes | Yes |
| `database` | `string` | The database to connect to (usually `postgres` for admin operations). | Yes | Yes |
| `adminSecretRef` | `ResourceRef` | Reference to the Kubernetes Secret containing the admin credentials. | No | Yes |
| `adminSecretFileRef` | `FileRef` | Reference to a file containing the admin credentials. | No | Yes |
| `parameters` | `map[string]string` | Additional connection parameters. | No | Yes |

> **Note:** Exactly one of `adminSecretRef` or `adminSecretFileRef` must be provided.

Expand All @@ -29,11 +29,12 @@ The referenced secret must be of type `kubernetes.io/basic-auth` and contain the

### FileRef (`adminSecretFileRef`)

| Field | Type | Description | Required |
|--------|----------|----------------------------------------------------------------|----------|
| `path` | `string` | The path to the file containing the admin credentials. | Yes |
Use this option when the credentials should be mounted as a file inside the operator Pod instead of reading a Kubernetes Secret directly.

| Field | Type | Description | Required |
|--------|----------|-----------------------------------------------------------------------------------------|----------|
| `path` | `string` | The absolute path inside the operator Pod to the file containing the admin credentials. | Yes |

Use this option when the credentials are mounted as a file instead of a Kubernetes Secret.

#### File format

Expand All @@ -51,35 +52,23 @@ The file must contain JSON with the following fields:

#### Mount the credentials file

The file must be accessible inside the operator pod at the path specified in `adminSecretFileRef.path`. Mount it using a Volume and VolumeMount on the operator Deployment:
The file must be accessible inside the operator Pod at the path in `adminSecretFileRef.path`.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgresql-operator
spec:
template:
spec:
containers:
- name: postgresql-operator
volumeMounts:
- name: db-credentials
mountPath: /mnt/secrets
readOnly: true
volumes:
- name: db-credentials
secret:
secretName: db-credentials-secret
```
The Helm chart exposes the `app.volumes` and `app.volumeMounts` values for this.
Both take the raw Kubernetes syntax, so any volume source that provides a file works.

> **Note:** The volume source can be any type that provides a file.
The value of `adminSecretFileRef.path` is the `mountPath` plus the name of the file. The volume source decides the file name:

> **Note:** The Helm chart does not support extra volumes yet.
| Volume source | The file name comes from |
|----------------------------------|---------------------------------|
| `secret` | the key of the Secret |
| `csi` (Secrets Store CSI driver) | the `objectAlias` of the object |

### Examples
See [Using a file reference](#using-a-file-reference-adminsecretfileref) in the examples for a complete setup with each volume source.

#### Using a Kubernetes Secret (`adminSecretRef`)
## Examples

### Using a Kubernetes Secret (`adminSecretRef`)

```yaml
apiVersion: v1
Expand Down Expand Up @@ -110,18 +99,119 @@ spec:
#connectTimeout: "10" # Timeout in seconds for connection attempts
```

#### Using a file reference (`adminSecretFileRef`)
### Using a file reference (`adminSecretFileRef`)

```yaml
apiVersion: postgresql.aboutbits.it/v1
kind: ClusterConnection
metadata:
name: quarkus-postgres-connection
name: my-postgres-connection
spec:
adminSecretFileRef:
path: "/mnt/secrets/db-credentials.json"
host: localhost
port: 5432
database: postgres
# Example parameters
parameters:
ApplicationName: "k8s-operator" # Helps identify this connection in Postgres logs
#sslmode: "require" # Enforce SSL encryption
#connectTimeout: "10" # Timeout in seconds for connection attempts
```

The mount that creates `/mnt/secrets/db-credentials.json` depends on the volume source.

#### From a Secret volume

Create the Secret. Its key becomes the file name:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: db-credentials-secret
stringData:
db-credentials.json: |
{
"username": "root",
"password": "password"
}
```

Then mount it through the chart values:

```yaml
app:
volumes:
- name: db-credentials
secret:
secretName: db-credentials-secret
volumeMounts:
- name: db-credentials
mountPath: /mnt/secrets
readOnly: true
```

#### From the Secrets Store CSI driver

Use this option to read the credentials from an external secret store, for example AWS Secrets Manager.

> **Note:** Install the [Secrets Store CSI driver](https://secrets-store-csi-driver.sigs.k8s.io/getting-started/installation) and the [provider](https://secrets-store-csi-driver.sigs.k8s.io/providers) for your secret store first. Neither the operator nor the chart installs them. Without the driver, the operator Pod stays in `ContainerCreating` and reports a failed mount.

The chart does not create the `SecretProviderClass`, so you have to apply it yourself. Its `objectAlias` becomes the file name:

```yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: db-credentials
spec:
provider: aws
parameters:
objects: |
- objectName: "my/db/credentials"
objectAlias: "db-credentials.json"
```

> **Note:** The `SecretProviderClass` must live in the namespace of the operator.

Then mount it through the chart values:

```yaml
app:
volumes:
- name: db-credentials
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: db-credentials
volumeMounts:
- name: db-credentials
mountPath: /mnt/secrets

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is just me but both new sections stop at the mount, no?
But the point of the PR is making adminSecretFileRef usable from the chart. Could you end each example with the matching path?

adminSecretFileRef:
      path: /mnt/secrets/db-credentials.json

With the CSI one especially it's not obvious that the filename comes from objectAlias. And a line saying the Secrets Store CSI driver has to be installed first would save someone a confused pod.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for carefully reading this 🙏🏼

I reworked the whole file, as since PR #60 it was really confusing and duplicated some stuff from the #### Mount the credentials file section partly again in ### Examples.

I moved these sections now under ## Examples and cleaned it up a bit.

readOnly: true
```

#### Without the Helm chart

If you deploy the operator directly from the OCI image, set the same `volumes` and `volumeMounts` fields on the Deployment:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgresql-operator
spec:
template:
spec:
containers:
- name: postgresql-operator
volumeMounts:
- name: db-credentials
mountPath: /mnt/secrets
readOnly: true
volumes:
- name: db-credentials
secret:
secretName: db-credentials-secret
```
22 changes: 22 additions & 0 deletions operator/src/main/helm/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
# This file overrides the default values that the quarkus-helm extension generates.
#
# Why it exists: a list field of the operator Deployment becomes a Helm value only if the key
# already exists in `src/main/kubernetes/kubernetes.yml`. An empty list `[]` does not survive
# there. The fabric8 model marks `PodSpec.imagePullSecrets`, `PodSpec.volumes` and
# `Container.volumeMounts` with `@JsonInclude(NON_EMPTY)`. A list with one null element does
# survive, but the generated default is then unusable, so this file replaces it with a real
# empty list.
#
# The unusable default takes one of two shapes, and the path of the value decides which:
# - A plain path, such as `spec.template.spec.volumes`, produces `- {}`, a list that holds
# one empty object. A user who copies that default and appends an entry gets invalid YAML.
# - A container-filtered path, such as
# `spec.template.spec.containers.(name == postgresql-operator).volumeMounts`, produces
# `{}`, an object. That shape also contradicts the `type: array` of `values.schema.json`.
#
# See https://github.com/quarkiverse/quarkus-helm/issues/453
app:
imagePullSecrets: []
volumes: []
volumeMounts: []
9 changes: 9 additions & 0 deletions operator/src/main/kubernetes/kubernetes.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,19 @@
---
# See https://quarkus.io/guides/deploying-to-kubernetes#using-existing-resources
apiVersion: apps/v1
kind: Deployment
metadata:
# The name must match `quarkus.kubernetes.name`, otherwise Dekorate adds a second Deployment.
# `HelmTest` has a test that makes sure this never drifts apart.
name: postgresql-operator
spec:
template:
spec:
affinity: {}
# The `[~]` placeholders are required, see operator/src/main/helm/values.yaml for the reason.
imagePullSecrets: [~]
volumes: [~]
containers:
# The name must match `quarkus.kubernetes.name`, otherwise Dekorate adds a second container.
- name: postgresql-operator
volumeMounts: [~]
32 changes: 28 additions & 4 deletions operator/src/main/resources/application.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,9 @@ quarkus:
- (kind == Deployment).spec.template.spec.containers.(name == ${quarkus.kubernetes.name}).imagePullPolicy
image-pull-secrets:
property: imagePullSecrets
value:
- null
paths:
- (kind == Deployment).spec.template.spec.imagePullSecrets
expression: "{{- if eq (toYaml .Values.app.imagePullSecrets | trim) \"- {}\" }} null{{- else }}{{- toYaml .Values.app.imagePullSecrets | nindent 8 }}{{- end }}"
description: Kubernetes image pull secrets to use if the OCI image is hosted on a private registry
expression: "{{- toYaml (.Values.app.imagePullSecrets | default list) | nindent 8 }}"
resource-requests-cpu:
property: resources.requests.cpu
value: ${quarkus.kubernetes.resources.requests.cpu}
Expand All @@ -113,6 +110,16 @@ quarkus:
paths:
- (kind == Deployment).spec.template.spec.affinity
description: Kubernetes affinity configuration for Pod scheduling
volumes:
property: volumes
paths:
- (kind == Deployment).spec.template.spec.volumes
expression: "{{- toYaml (.Values.app.volumes | default list) | nindent 8 }}"
volume-mounts:
property: volumeMounts
paths:
- (kind == Deployment).spec.template.spec.containers.(name == ${quarkus.kubernetes.name}).volumeMounts
expression: "{{- toYaml (.Values.app.volumeMounts | default list) | nindent 12 }}"
console-color:
property: envs.QUARKUS_CONSOLE_COLOR
value-as-bool: ${quarkus.console.color}
Expand All @@ -127,9 +134,26 @@ quarkus:
description: Specify the format of the produced JSON. Supported values are "DEFAULT", "ECS", and "GCP".
values-schema:
properties:
# The type must be set explicitly for every non-scalar value, because the generated
# schema otherwise falls back to `string`.
#
# A value that `src/main/helm/values.yaml` provides also loses the `description` of its
# `quarkus.helm.values` entry, so the description belongs here instead.
"affinity":
name: app.affinity
type: object
"imagePullSecrets":
name: app.imagePullSecrets
type: array
description: Kubernetes image pull secrets to use if the OCI image is hosted on a private registry
"volumes":
name: app.volumes
type: array
description: Additional volumes for the operator Pod, for example a Secret volume or a Secrets Store CSI volume
"volumeMounts":
name: app.volumeMounts
type: array
description: Additional volume mounts for the operator container
expressions:
release-name-labels:
expression: "{{ .Release.Name }}"
Expand Down
Loading