DevOps Challenge: Helm, Kustomize, TLS, and AWS Secrets Integration

.github/workflows/crewmeister-ci.yml

@@ -0,0 +1,102 @@
+name: crewmeister-ci
+
+on:
+  push:
+    branches:
+      - main
+      - feature/*
+
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'temurin'
+
+      - name: Build with Maven
+        run: mvn clean install -DskipTests
+
+  code-quality:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up JDK 17
+        uses: actions/setup-java@v3
+        with:
+          java-version: '17'
+          distribution: 'temurin'
+
+      - name: Run Code Quality Checks
+        run: mvn verify
+
+  docker:
+    runs-on: ubuntu-latest
+    needs: build  
+    if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/heads/feature/')
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+
+      - name: Log in to Docker Hub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+
+      - name: Extract Git SHA
+        run: echo "IMAGE_TAG=$(git rev-parse --short HEAD)" >> $GITHUB_ENV
+
+      - name: Build and Push Docker Image
+        run: |
+          docker buildx build --platform linux/amd64 \
+            -t ${{ secrets.DOCKER_USERNAME }}/devops-coding-challenge:${{ env.IMAGE_TAG }} --push .
+
+  updatek8s:
+    runs-on: ubuntu-latest
+    needs: docker  
+    outputs:
+      aws_access_key_id: ${{ steps.auth.outputs.aws_access_key_id }}
+      aws_secret_access_key: ${{ steps.auth.outputs.aws_secret_access_key }}
+      aws_session_token: ${{ steps.auth.outputs.aws_session_token }}
+    steps:
+      - name: Checkout Code
+        uses: actions/checkout@v3
+
+      - name: Configure AWS Credentials (OIDC)
+        id: auth
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
+          aws-region: ${{ secrets.AWS_REGION }}
+          audience: sts.amazonaws.com
+
+      - name: Verify AWS Identity
+        run: aws sts get-caller-identity
+
+      - name: Update kubeconfig for EKS
+        run: aws eks update-kubeconfig --name ${{ secrets.EKS_CLUSTER_NAME }} --region ${{ secrets.AWS_REGION }}
+
+      - name: Deploy Helm Chart
+        run: |
+          helm upgrade --install my-app ./k8s/helm \
+            --namespace dev \
+            -f ./k8s/values/dev-values.yaml \
+            --set image.tag=${{ env.sha_tag }} \
+            --set aws.region=${{ secrets.AWS_REGION }} \
+

.gitignore

@@ -31,3 +31,21 @@ build/
 
 ### VS Code ###
 .vscode/
+# Terraform State Files
+*.tfstate
+*.tfstate.backup
+errored.tfstate
+
+# Ignore Terraform lock file
+.terraform.lock.hcl
+
+# Ignore Terraform Caches and Modules
+.terraform/
+.terraform/*
+
+# Ignore any sensitive variable files
+*.auto.tfvars
+*.tfvars.json
+
+# Ignore generated logs or debug files
+crash.log

Dockerfile

@@ -0,0 +1,37 @@
+# Build Stage
+FROM eclipse-temurin:17-jdk-jammy AS builder
+
+WORKDIR /app
+
+# Copy Maven wrapper and pom.xml separately for better caching
+COPY mvnw pom.xml ./
+COPY .mvn .mvn
+
+# Download dependencies (cache-friendly)
+RUN ./mvnw dependency:go-offline
+
+# Copy source code
+COPY src ./src
+
+# Package the application
+RUN ./mvnw clean package -DskipTests
+
+# Runtime Stage (minimal image)
+FROM eclipse-temurin:17-jre-jammy
+
+# Create a non-root user to run the app securely
+RUN groupadd -r appgroup && useradd -r -g appgroup appuser
+
+WORKDIR /app
+
+# Copy jar from builder
+COPY --from=builder /app/target/*.jar app.jar
+
+# Change ownership for security purposes
+RUN chown appuser:appgroup app.jar
+
+USER appuser
+
+EXPOSE 8080
+
+ENTRYPOINT ["java", "-jar", "app.jar"]

README.md

@@ -1,63 +1,179 @@
-# Crewmeister Challenge
+# 🚀 Spring Boot + MySQL on Kubernetes
 
-## Background
+A production-grade Kubernetes deployment for a Spring Boot application backed by MySQL, built with security, scalability, and automation in mind.
 
-At Crewmeister, our development team is continuously growing. We aim to hire the best educated, motivated, and enthusiastic people in the field who have fun building up Crewmeister in our vision to empower small businesses to thrive in a digital world. For this quest, we are continuously getting new applicants from all over the world. To filter which candidates could be a good fit, we provide our candidates with a coding challenge that we manually review and evaluate.
+---
+
+## 📆 Stack Overview
+
+| Layer                  | Tech/Tool |
+|------------------------|-----------|
+| Application            | Spring Boot (Java 17) |
+| Database               | MySQL (StatefulSet + PVC) |
+| Secrets Management     | AWS Secrets Manager + External Secrets Operator |
+| Kubernetes Templating  | Helm |
+| Env-specific Overrides | Kustomize |
+| CI/CD Pipeline         | GitHub Actions |
+| Observability          | Prometheus-ready integration hooks |
+
+---
+
+## ✨ Key Features
+
+| Feature                         | Description |
+|----------------------------------|-------------|
+| ✅ Dockerized Spring Boot        | With Flyway DB migrations |
+| ✅ MySQL via StatefulSet         | With persistent volumes |
+| ✅ AWS Secrets Manager           | Centralized secret storage |
+| ✅ External Secrets Operator     | Automatic sync of AWS secrets to K8s |
+| ✅ Helm Charts                   | Clean, reusable templating |
+| ✅ Kustomize Overlays            | Per-environment configurations (dev, staging, prod) |
+| ✅ TLS via cert-manager + Let's Encrypt | Secure HTTPS via Ingress |
+| ✅ Ingress (NGINX)               | With routing, HTTPS, and rewrite support |
+| ✅ Horizontal Pod Autoscaler     | Auto-scales app based on CPU |
+| ✅ CI/CD Pipeline                | GitHub Actions with multi-stage flow |
+| ✅ Secure Image Handling         | Versioned image tags, no `latest` used |
+| ✅ Health Probes & Limits        | Readiness/liveness probes and resource limits |
+
+---
+
+## 📂 Folder Structure
+
+```bash
+k8s/
+├── base/                          # Common base manifests for Kustomize
+│   ├── deployment.yaml
+│   ├── mysql-service.yaml
+│   ├── mysql-statefulset.yaml
+│   ├── service.yaml
+│   └── kustomization.yaml
+
+├── overlays/                      # Environment-specific overlays (Kustomize)
+│   ├── dev/
+│   │   ├── configmap.yaml
+│   │   ├── kustomization.yaml
+│   │   ├── mysql-external-secret.yaml
+│   │   ├── namespace.yaml
+│   │   ├── secret-store.yaml
+│   │   └── cluster-issuer.yaml    # cert-manager ClusterIssuer (staging/prod)
+│   └── staging/
+│   └── prod/
+
+├── helm/                          # Helm chart for full app deployment
+│   ├── Chart.yaml
+│   ├── values.yaml                # Default config
+│   └── templates/
+│       ├── _helpers.tpl
+│       ├── configmap-mysql.yaml
+│       ├── secret-external-mysql.yaml
+│       ├── secretstore-aws.yaml
+│       ├── serviceaccount.yaml
+│       ├── mysql/
+│       │   ├── service-mysql-headless.yaml
+│       │   └── statefulset-mysql.yaml
+│       ├── springboot/
+│       │   ├── deployment-springboot.yaml
+│       │   ├── hpa-springboot.yaml
+│       │   ├── ingress-springboot.yaml
+│       │   └── service-springboot.yaml
+│       └── tests/
+│           └── test-connection.yaml
+
+├── values/                        # Helm values per environment
+│   ├── dev-values.yaml
+│   ├── staging-values.yaml
+│   └── prod-values.yaml
+
+.github/
+└── workflows/
+    └── crewmeister-ci.yaml       # CI/CD pipeline definition
+```
+
+---
+
+## 🛠️ Usage Guide
+
+### ✅ Apply Kustomize Overlay (Dev)
+
+```bash
+kubectl apply -k k8s/overlays/dev
+```
+
+### ❌ Delete Dev Resources
+
+```bash
+kubectl delete -k k8s/overlays/dev
+```
 
 ---
 
-## DevOps Engineer Task
+### 🥤 Helm Usage
 
-As a DevOps Engineer at Crewmeister, you will be in charge of several challenging tasks in your daily work. One of your core responsibilities will be to ensure that the system is always running smoothly and that the application is deployed successfully to our customers.
+```bash
+# Install
+helm install my-app ./k8s/helm -f ./k8s/values/dev-values.yaml
 
-In this challenge, you should use DevOps best practices to architect and implement the complete cycle of building, packaging, and deploying a Java application (specified later in this document). 
+# Upgrade
+helm upgrade my-app ./k8s/helm -f ./k8s/values/dev-values.yaml
 
-The following are core technologies/tools that should be present in the solution:
+# Uninstall
+helm uninstall my-app
+```
 
-- Dockerfile
-- Helm Chart
-- Terraform to interact with the Kubernetes cluster
+---
 
-You have the flexibility to utilize any cloud provider of your choice to deploy and run the application effectively. Additionally, it should be designed to operate seamlessly on local machines, allowing for a versatile setup that caters to various operational preferences and environments.
+### 🌐 Port Forward for Local Access
 
-## Plus:
+```bash
+kubectl port-forward svc/my-app-service -n dev 8080:80
+```
 
-- Create a CI Pipeline in Github to automate the application lifecycle
+Then access: [http://localhost:8080](http://localhost:8080)
 
-- Add monitoring tools to check the health of the application
+---
 
-## Important Points:
+## 🔐 AWS Secrets Setup
 
-- At Crewmeister, we value creativity and pushing for better. You are encouraged to expand the solution as you find fit. To do so, you must ensure high-quality documentation and that the base solution is correctly executed.
-- All the tools used must be publicly accessible or explicitly documented on how to authenticate.
-- All the tools must be free to use.
+Before deploying, manually create the AWS credentials secret in your cluster:
 
-## Challenge Application
+```bash
+kubectl create secret generic aws-credentials \
+  --from-literal=access-key-id=<AWS_ACCESS_KEY_ID> \
+  --from-literal=secret-access-key=<AWS_SECRET_ACCESS_KEY>
+```
 
-A Spring Boot application that provides a simple user management REST API.
+> **Note**: Never commit your AWS credentials to Git or store them in `values.yaml`.
 
-### Technologies Used
+---
 
-- Java 17
-- Spring Boot 3.3.5
-- MySQL Database
-- Flyway Migration
-- Maven
-- Spring Data JPA
-- Spring Actuator
+## ⚙️ GitHub Actions CI/CD (`.github/workflows/crewmeister-ci.yaml`)
 
-### Pre-requisites
+| Stage         | Description |
+|---------------|-------------|
+| `build`       | Compiles the app with Maven |
+| `code-quality`| Runs linting/tests |
+| `docker`      | Builds versioned image, pushes to Docker Hub |
+| `updatek8s`   | Deploys Helm chart to EKS using OIDC-authenticated GitHub Action |
 
-- JDK 17
-- MySQL
-- Maven
+---
 
-### API Endpoints
+## 🌟 Observability & Reliability
 
-#### GET /user
+- ✅ `readinessProbe` and `livenessProbe` configured
+- ✅ `resources.requests` and `limits` defined for CPU/memory
+- ✅ Horizontal Pod Autoscaler enabled for autoscaling
+- ✅ Ingress TLS via cert-manager (staging & prod)
+- ✅ Prepped for Prometheus metrics
+
+---
 
-Retrieves a user by ID
+## ✅ Best Practices Followed
 
-#### POST /user
+- No use of `:latest` tag in Docker builds
+- All secrets pulled dynamically via AWS Secret Manager
+- Values managed via Helm `values.yaml`
+- GitOps-compatible structure via Kustomize overlays
+- Externalized environment handling
+- Linted and validated YAML
+- Modular and extensible folder layout
 
-Creates a new user