Demo projects and notes while following along the course.
Reference: Module 3 - Application Vulnerability Scanning
Description: Set-up a CI pipeline and pre-commit script to scan for secrets using GitLeaks.
- Configure GitLeaks job in CI to check commits for secrets.
- Create a Git pre-commit hook script that runs GitLeaks using Docker, and performs GitLeaks scan before the code is committed.
Used technologies:
- Uses the juice-shop repository found here on branch
feature/starting-codeas a submodule. - GitHub workflow, GitLeaks, Git
Overview: The pipeline project-1.yml can be found under .github/workflows.
The pre-commit hook project-1-pre-commit.sh is available in the root folder of this repository.
Reference: Module 3 - Application Vulnerability Scanning
Description: Set-up a CI pipeline for Static Application Secret Testing (SAST) using NJSScan and Semgrep:
- Configure NJSScan in CI to run SAST scan against repository code.
- Configure Semgrep in CI to run additional SAST scan against repository code.
Used technologies:
- NJSScan
- Semgrep
- Note: Uses
semgrep scaninstead ofsemgrep cidue to changes in the dependency.
- Note: Uses
- GitHub workflow, Git
Overview: The pipeline project-2.yml can be found under .github/workflows.
Reference: Module 4 - Vulnerability Management and Remediation
Description: Create a Python script that automatically uploads security scan findings to DefectDojo, a vulnerability management tool, using CI.
- Create a Python script that connects to DefectDojo via API key
- Create a Python script to upload GitLeaks, Semgrep, and NJSScan files from CI security scanning jobs to DefectDojo
- Add a new job in CI pipeline to run the Python script to upload findings to DefectDojo as part of the pipeline execution.
Used technologies:
- DefectDojo
- Note: You might need to re-generate a new API v2 key via the DefectDojo demo webpage.
Overview: The pipeline project-3.yml can be found under .github/workflows.
Project 4 - Remediate Weak Cryptography and SQL Injection Vulnerabilities in Application based on Security Findings
Description: Review NJSScan and Semgrep security findings in DefectDojo, then remediate weak cryptography and SQL injection vulnerabilities in application code.
- Update application code to remediate weak hash function (e.g. juice-shop\Gruntfile.js - md5 checksum), based on NJSScan security finding in DefectDojo
- Update application code to remediate SQL injection vulnerability, based on Semgrep security findings in DefectDojo.
Used technologies:
- JavaScript
- DefectDojo
- Git
- GitHub
- Semgrep
- NJSScan
Reference: Module 5 - Vulnerability Scanning for Application Dependencies
Description: Set-up Software Composition Analysis (SCA) scanning in CI and automatically upload scan reports to DefectDojo.
- Create a new job in the CI pipeline for automated SCA scanning using RetireJS
- Configure the job to save the SCA scan report as an artifact
- Upload the RetireJS scan report to DefectDojo using Python automation script
Used technologies:
- DefectDojo
- RetireJS
Overview: The pipeline project-5.yml can be found under .github/workflows.
Reference: Module 6 - Build a CD Pipeline
Description: Configure CI pipeline with AWS user credentials to upload images to AWS ECR via CI/CD pipeline.
- Create access keys for user in AWS IAM
- Add AWS access keys as CI secrete environment variables for use in the pipeline
- Update CI pipeline to build and push Docker images to ECR using the AWS access keys
Used technologies:
- AWS IAM
- AWS ECR
- GitHub Workflow
Overview: The pipeline project-6.yml can be found under .github/workflows.
Reference: Module 6 - Build a CD Pipeline
Description: Deploy application to AWS EC2 instance using CI/CD pipeline.
- Configure EC2 instance with Docker and ECR credentials
- Create new deployment job in CI that automatically:
- Connects to EC2 instance using SSH
- Pulls latest Docker images from AWS ECR
- Stops previous running container, if applicable
- Runs latest Docker container
Used technologies:
- AWS IAM
- AWS EC2
- GitHub Workflow
- Docker
Overview: The pipeline project-7.yml can be found under .github/workflows. The required configuration can be found under project-7/configure.sh.
Reference: Module 6 - Build a CD Pipeline
Description: Create an EC2 instance and configure it as a GitHub CI/CD self-managed runner.
- Create new AWS EC2 instance
- Register the EC2 instance with GitHub CI/CD as a self-hosted runner
- Configure EC2 instance as a GitHub runner with Shell executor instead of Docker executor
Used Technologies:
- AWS EC2
- GitHub CI/CD
Overview: The required configuration can be found under project-8/configure.sh.
Project 9 + 10 - Scan Docker Images using Trivy with CI/CD Pipeline + Upload Trivy Image Scanning Results to DefectDojo using CI/CD Pipeline
Reference: Module 7 - Image Scanning - Build Secure Docker Images
Description: Update CI/CD pipeline to perform Docker image scanning using Trivy and upload Trivy image scan results to DefectDojo
- Create a new job in CI/CD pipeline that:
- Pulls docker image from private AWS ECR
- Runs Trivy image scan on the image
- Fails Trivy job only if high or critical level security findings are detected
- Updates Trivy job to export image security findings reported as pipeline artifacts
- Updates Python script to automatically upload Trivy security findings to DefectDojo
- Update upload reports job to execute Python upload script for Trivy scan reports
Used technologies:
- Trivy
- Python
- DefectDojo
- Git
- GitHub Workflow
- Docker
- AWS ECR
Overview: The pipeline project-9+10.yml can be found under .github/workflows.
The pipeline uses the AWS IAM access keys of the github user instead of the root user as defined in Project 11.
Container deployed to the EC2 instance app-server from github-runner:
Juice-shop available via public ip of EC2 instance on port 3000:
Reference: Module 9 - Secure Continuous Deployment & DAST
Description: Create a dedicated AWS IAM user for CI/CD with least priviliges accesss to securely upload image to AWS ECR.
- Create AWS IAM user for CI/CD and assign permission policy for ECR access only
- Configure CI/CD user with only AWS CLI access
- Create AWS CLI access keys for the
githubuser - Update CI/CD to use the
githubuser access keys instead of the admin user keys
Used technologies:
- AWS IAM
- GitHub Workflow
Overview: Dedicataed AWS IAM user for CI/CD.
Reference: Module 9 - Secure Continuous Deployment & DAST
Description: Create new EC2 role to allow more secure access from CI/CD using AWS SSM instead of SSH.
- Remove SSH firewall rule from AWS EC2 security group
- Create new IAM role for EC2 instance with
SSMManagedInstanceCorepolicy, assign the role to EC2 instance for SSM permission - Remove SSH commands from CI pipeline
- Add SSM access policy to GitHub IAM user permissions
- Update GitHub workflow deploy job to:
- Connect to private AWS ECR repository
- Run docker image pull, stop, and run commands
- Connect to EC2 instance with SSM and run docker application deployment commands
Used technologies:
- AWS IAM
- GitHub Workflow
- AWS EC2
Overview: Dedicated SSM roles (attached to both instances).
Reference: Module 9 - Secure Continuous Deployment & DAST
Description: Create ZAP job in CI/CD pipeline to run automated Dynamic Application Security Testing (DAST) scans.
- Configure CI pipeline to deploy to test and prod environments
- Create DAST job in CI:
- Run automated ZAP scans against Docker application deployed on EC2 instance
- Fail the ZAP job for security findings above the warning severity level
- Export the ZAP scan results as pipeline artifacts
Used technologies:
- Zed Attack Proxy (ZAP)
- GitHub Workflow,
- Docker
- AWS
Reference: Module 10 - IaC and GitOps for DevSecOps
Description: Automate entire process of provisioning EC2 servers, AWS networking resources, IAM roles and permissions using Terraform.
- Create new IAM user for Terraform operations
- Create IAM roles with needed policies for application server and GitLab runner instances
- Create VPC and Security Group resources for EC2 networking
- Create and configure two EC2 instances:
- Create application server with secure firewall configuration
- Configure script to automatically install Docker and application dependencies on the application server
- Create GitHub CI server with secure firewall configuration
- Configure script to automatically install Docker and register the instance as a GitHub runner
Used technologies:
- Terraform
- Docker
- Git
- AWS (VPC, EC2, IAM)
- GitHub Workflow
Overview: Dedicated iac-user:
For S3 bucket access policies need to be added:
Reference: Module 10 - IaC and GitOps for DevSecOps
Description: Create a GitHub CI/CD pipeline for Terraform Infrastructure with built-in security scanning and code validation - GitOps
- Create AWS S3 bucket and configure Terraform to use bucket for storing Terraform state
- Create new GitLab CI pipeline for Terraform infrastructure that:
- Initializes Terraform and builds a plan artifact
- Validates Terraform configuration and syntax
- Runs a TFSec security scan on Terraform code and produces scan result artifact
- Deploys the Terraform code to AWS
Used technologies:
- AWS S3
- GitHub Workflow
- Terraform
- Git
- TFSec
Note: TFsec seems to be part of trivy now, but it can be installed manually as seen in .github/workflows/project-15.yml.
Reference: Module 11 - Logging & Monitoring for Security
Description: Create a trail in CloudTrail that captures events across all AWS regions and configure it to send event logs to CloudWatch.
- Create CloudTrail Trail that:
- Forwards trail logs to new CloudWatch Log Group
- Saves CloudTrail events to S3 bucket for all AWS regions
- Create filters on CloudTrail Trail events to quickly view events like failed login attempts
- Create filter in CloudWatch Log Group to quickly view logging metrics
Used technologies:
- AWS (S3, CloudTrail, CloudWatch)
Overview: Cloud-trail:
Reference: Module 11 - Logging & Monitoring for Security
Description: Create CloudWatch Alarm that sends an alert email when an EC2 instance has failed status checks for over five minutes
- Create new CloudWatch metric alarm that:
- Checks if an EC2 instance failed status checks over a five minute period
- Configure the metric to the “In alarm” state if this condition is met
- Create a new SNS topic that sends an email when the metric is in the alarm state
Overview: Alarm overview:
After bringing down the EC2 instance using ifconfig <interface> down and waiting for some time:
Reference: Module 11 - Logging & Monitoring for Security
Description: Create a custom metric filter in CloudWatch that checks for failed AWS console logins, create an alarm to notify when failed logins exceeds a threshold.
- Create Custom metric filter in CloudWatch log group that:
- Checks for failed logins across entire log group
- Creates new metric namespace for custom metric
- Counts the number of failed login attempts
- Create CloudWatch alarm that:
- Uses the custom metric filter to count failed login attempts over a 5 minute period, and triggers the alarm if there are more than 7 failures in that period
- Creates a new SNS topic that sends an email when the metric is in the alarm state
Used technologies:
- AWS (CloudWatch, SNS)
Overview: Alarm overview:
Reference: Module 11 - Logging & Monitoring for Security
Description: Create an AWS Budgets alarm that sends an email alert if AWS costs exceed a threshold.
- Create AWS Budget that:
- Sets a desired AWS monthly cost budget
- Emails recipients when threshold is:
- at 85% of actual budget spend
- at 100% of actual budget spend
- and if forecasted amount is expected to reach 100% of actual budget spend
Used technologies:
- AWS Budgets
Overview: I've actually have created this budget right in the beginning of the AWS journey:
Reference:
- Module 12: Introduction to Kubernetes Security
- Module 13: Kubernetes Access Management with AWS EKS
Description: Provision EKS cluster with Secure Access Management for developer and admin roles using Security Best Practices
- Infrastructure as Code configuration to:
- Provision a base EKS cluster
- Add Configuration for Access Management:
- Configure AWS IAM Roles for Access on AWS Level
- Define Kubernetes Access with Role Based Access Control (RBAC) - creating K8s Roles and ClusterRoles
- Configure Mapping between IAM Roles and K8s Users
Used technologies:
- AWS EKS
- AWS IAM
- Terraform
- Kubernetes
Overview: I've adjusted the cluster size to save cost.
The terraform files can be found under project-20/ in the root folder.
Validation:
aws configure // e.g. with the admin user
aws eks update-kubeconfig --name myapp-eks --region eu-north-1
kubectl get node
kubectl get pod -ATo connect with another user, create access keys and export them, i.e. AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the current terminal session.
Verify with aws sts get-caller-identity.
kubectl get pod -A returns unauthorized.
Assuming a role: https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html
Description: Create IaC Pipeline that establishes a secure connection with AWS using trusted, third-party web identity provider.
- AWS Cloud Configuration:
- Configure Authentication with GitLab OIDC Identity Provider
- Create Web Identity on AWS cloud
- Create IAM Role with with GitLab’s Identity Provider as Trusted Entity
- Configure Authentication with GitLab OIDC Identity Provider
- GitLab Pipeline Configuration
- Add Pipeline configuration that establishes a secure connection with temporary credentials on every job execution
- Terraform Configuration in Release Pipeline:
- Configure TF commands to provision EKS cluster
Used technologies:
- GitHub workflow
- AWS EKS
- AWS STS
- AWS IAM
- Terraform
- Kubernetes
Overview: OpenID Connect from GitHub to AWS: https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services
Note: I left out the apply step because I did not want to incur any cost.
Description: Deploy cluster add-ons (Cluster Autoscaler, Metric Server, Load Balancer Controller) using EKS Blueprints
- Deploy and configure Cluster Autoscaler for dynamic up & down scaling of Nodes based on resource demand
- Deploy Metrics Server needed for Autoscaler
- Deploy AWS Load Balancer Controller that provisions AWS Load Balancers for cluster services
Used technologies:
- EKS Blueprints
- GitHub Workflow
- AWS EKS
- AWS STS
- AWS IAM,
- Terraform
- Kubernetes
Overview: Blueprints in-deployment:
Project 23 - Completely automated end-to-end CI/CD Pipeline with GitOps principles using GitHub CI and Argo CD
Description: Deploy ArgoCD in EKS Cluster using IaC Pipeline
- Install ArgoCD in EKS cluster
- Create ArgoCD Application manifest
- Configure Connection and Automatic Syncing with GitOps repository
- Execute IaC Pipeline to deploy ArgoCD in EKS cluster
Used technologies:
- Argo CD
- GitHub Workflow
- AWS EKS
- Terraform
- Kubernetes
- Helm Chart
Reference: 16.05, 16.06, 16.07
Note: Uses Creating a fine-grained personal access token for accessing the private repository odlot/devsecops-bootcamp-online-boutique-gitops.
To connect with another user, create access keys and export them, i.e. AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the current terminal session.
Verify with aws sts get-caller-identity.
eval $(aws sts assume-role --role-arn "arn:aws:iam::<ID>:role/external-admin" --role-session-name "K8Session" | jq -r '.Credentials | "export AWS_ACCESS_KEY_ID=\(.AccessKeyId) AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey) AWS_SESSION_TOKEN=\(.SessionToken)"')Verify with aws sts get-caller-identity.
Authenticate with eks: aws eks --region eu-north-1 update-kubeconfig --name myapp-eks.
List all pods deployed in the argocd namespace: kubectl get pod -n argocd.
Retrieve the ArgoCD UI password for login: kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d.
Localhost port-forwarding to be able to access the ArgoCD UI:
kubectl describe pod argocd-server-... -n argocd
kubectl port-forward -n argocd pods/argocd-server-... 8080:8080
Description: Automatically deploy K8s manifest files in Cluster via ArgoCD
- Write K8s manifest files for all microservices
- Write K8s manifest for external access using Ingress
- ArgoCD deploys microservices automatically as soon as manifest files are pushed into Git repository
Used technologies:
- Kubenertes
- Kustomize
- ArgoCD
- Git
- AWS EKS
Reference: 16.08
Note: I was running a lower maximum size in my terraform scripts to save cost which led to pods being unschedulable. After increasing the maximum size everything synced and was fine.
Description: Create Pipeline for GitOps project to update Kustomization file
- Configure GitHub CI pipeline
- to update image version of updated microservice in Kustomization file
- and push commit to Git repository
- Configure GitHub access with project deploy token
Used technologies:
- Kubenertes
- Kustomize
- ArgoCD
- Git
Reference: 16.09
Description: Create Pipeline for microservices application that triggers gitops pipeline
- Create CI pipeline for microservices that
- tests the code
- scans for security vulnerabilities
- builds and pushes to Docker repository
- Add job to CI pipeline that triggers the GitOps pipeline and passes environment variables
- Full CI/CD pipeline is triggered, where ArgoCD deploy the app as soon as code is pushed to Git repository by GitHub CI
Used technologies:
- Kubenertes
- Kustomize
- ArgoCD
- Git
Reference: 16.10
Description: Deploy OPA Gatekeeper in EKS Cluster using IaC Pipeline
- Configure Gatekeeper in Terraform using Helm Chart
- Execute IaC Pipeline to deploy OPA Gatekeeper in EKS cluster
Used technologies:
- OPA Gatekeeper
- ArgoCD
- GitLab CI
- AWS EKS
- Terraform
- Kubernetes
- Helm Chart
Note: Use AWS user to assume K8s admin role and connect to the EKS cluster.
Check the namespace for the openpolicyagent namespace: kubectl get ns.
Check what has been deployed into the openpolicyagent namespace: kubectl get all -n openpolicyagent.
List all gatekeeper crd templates available with kubectl get crd -n openpolicyagent.
Description: Configure policies to enforce best practices automatically: i) Block service with type NodePort ii) Disallow priviliged containers
- Create constraint template and constraint manifest files to disallow
NodePortservice types - Create Kustomization file for deployment
- Create ArgoCD application configuration to sync Git repository
- Adjust GitHub CI IaC pipeline to deploy ArgoCD application
- Push manifest file with service type
NodePortto test successful rejection by OPA - Create constraint template and constraint manifest files to disallow privileged containers
- Push manifest file with privileged container configuration to test succesful rejection by OPA
Used technologies:
- OPA Gatekeeper
- Kubernetes
- Kustomize
- ArgoCD
- Git
Project 29 - Store K8s secrets centrally in AWS Secrets Manager to fetch them in cluster using External Secrets Operator
Description: Deploy external secrets operator in EKS cluster via IaC pipeline
- Enable External secrets EKS add-on in Terraform's EKS blueprints module
- Execute IaC pipeline to deploy external secrets add-on in EKS cluster
Used technologies:
- External Secrets Operator
- GitHub Workflow (CI)
- AWS EKS
- Terraform
- K8s
Description: Create secrets in AWS Secrets Manager and give access to K8s Service Account to access the secrets
- Add TF configuration to create IAM role with permission to access the AWS Secrets Manager secrets
- Add TF config to create K8s Service Account that maps to the IAM role
- Deploy the TF configuration via IaC pipeline
Used technologies:
- AWS Secrets Manager
- GitHub Workflow (CI)
- AWS EKS
- Terraform
- K8s
Project 31 - Fetch secrets from AWS Secrets Manager using External Secrets Operator CRDs and map them to K8s secrets
Description: Fetch secrets into K8s cluster securely from AWS Secrets Manager
- Create ClusterSecretStore to connect K8s cluster to the AWS Secrets Manager, using K8s service account for authentication with the Secrets Manager
- Create ExternalSecret to map secret from AWS Secrets Manager to K8s native Secret component
- Deploy the above CRDs via ArgoCD
- Mount the fetched secret into a pod to validate the secret has been fetched into cluster
- Update secret value in AWS Secrets Manager to showcase the automatic syncing of the secret value into K8s cluster
Used technologies:
- AWS Secrets Manager
- External Secrets Operator
- Kubernetes
- Kustomize
- ArgoCD
- GitHub Workflow
Description: Deploy Istio Service Mesh in EKS Cluster via IaC pipeline
- Install Istio and Istio gateway in EKS cluster using Terraform configuration, enable proxy injection in app namespace
- Execute IaC Pipeline to deploy Istio in EKS cluster
Used technologies:
- Istio
- GitHub Workflow
- AWS EKS
- Terraform
- Kubernetes
Reference: 19.03
Description: Expose micro services app through Istio gateway via secure Https access
- Configure traffic routing in cluster via Istio
- Create Virtual Service for the frontend of the application to route traffic to microservice app via gateway
- Generate self-signed certificate for Istio gateway TLS and store in AWS Secrets Manager
- Configure Istio Gateway to only access secure https connection
Used technologies:
- Istio
- Istio gateway
- AWS Secrets Manager
- GitHub Workflow
- AWS EKS
- ArgoCD
- Kubernetes
Reference: 19.04, 19.05
Notes:
Create a certificate via openssl: openssl req -x509 -newkey rsa:4096 -sha256 -days 30 -nodes -keyout key.pem -out cert.pem -subj "/CN=*.twnn.com"
Description: Validate Istio mTLS traffic enabled in the app namespace
- Deploy a test pod to validate mTLS traffic between pods with envoy proxies
- Enable Strict mTLS mode to disallow any unencrypted traffic between pods
Used technologies:
- Istio
- AWS EKS
- Kubernetes
References: 19.06
Description: Secure Pod Traffic using Istio Authorization Policy components
- Create Authorization Policy for pods within the application namespace
- Enable proxy injection in ArgoCD namespace
- Validate restricted traffic rules by sending requests from different pods within the cluster
- Create Authorization Policy to restrict traffic from application to ArgoCD namespace
Used technologies:
- Istio
- AWS EKS
- ArgoCD
- Kustomization
- Kubernetes
References: 19.09
Description: Setup AWS Config to run automated compliance audit
- Setup rule to validate ssh ports aren’t open on EC2 servers
- Rule to check CloudTrail is always enabled and logging
Used technologies:
- AWS
- AWS Config
- Security Group
- CloudTrail
- AWS EC2
References: 20.05
Description: Configure auto remediation for compliance issues
- Create IAM role for SSM to modify security group and CloudTrail resources
- Configure auto-remediation with SSM script to remove ssh ingress rule from security group
- Configure auto-remediation to enable logging in CloudTrail
Used technologies:
- AWS
- AWS Config
- Security Group
- CloudTrail
- AWS EC2
References: 20.06, 20.07
Description: Setup AWS Config to run automated compliance checks for EKS cluster configuration
- Add rule to validate using supported K8s version in EKS clusters in AWS account
- Add rule to ensure all EKS control plane components have logging enabled
- Manually enable logging of EKS cluster components to make the cluster compliant
Used technologies:
- AWS
- AWS Config
- AWS EKS
- Kubernetes
References: 20.08
