🚀 Mattermost Ops Kit Infrastructure

Production-ready Mattermost deployment on AWS with OpenTofu

Enterprise-grade team collaboration platform for Mattermost deployed on AWS

Features • Quick Start • Architecture • Documentation

---

📋 Overview

This repository contains Infrastructure as Code (IaC) for deploying a complete Mattermost collaboration platform on AWS using OpenTofu with production-ready security, monitoring, and scalability.

Features

✅ Secure by Default - TLS 1.3, encryption at rest/transit, private subnets ✅ Highly Available - Multi-AZ deployment with Application Load Balancer ✅ Scalable - Easy vertical/horizontal scaling options ✅ Observable - CloudWatch dashboards, logs, and alarms ✅ Cost Optimized - ~$79-90/month with VPC endpoints and optional NAT ✅ Battle Tested - Pre-commit hooks, security scanning, automated testing

Deployment Specs

Component	Configuration
Region	Configurable (default: us-west-1)
Domain	mattermost.example.com
Compute	Docker on EC2 (t3.medium)
Database	PostgreSQL 16 on RDS (db.t4g.micro)
Storage	S3 with versioning and lifecycle policies
Security	Locked down signups, email verification, MFA enabled
Email	AWS SES SMTP integration
Monitoring	CloudWatch dashboards, logs, and alarms

🏗️ Architecture

High-level architecture of the Mattermost deployment:

graph TB
    Internet([🌐 Internet])
    ALB[Application Load Balancer<br/>Public Subnets]
    MM1[🐳 Mattermost<br/>Docker Container<br/>EC2 - Private Subnet AZ-A]
    MM2[🐳 Mattermost<br/>Docker Container<br/>EC2 - Private Subnet AZ-B<br/><i>Future Scaling</i>]
    RDS[(🗄️ PostgreSQL 16<br/>RDS<br/>Private Subnet)]
    S3[📦 S3 Bucket<br/>File Storage<br/>Versioned + Encrypted]
    NAT[🌐 NAT Gateway]

    Internet -->|HTTPS/TLS 1.3| ALB
    ALB -->|HTTP:8065| MM1
    ALB -.->|Future| MM2
    MM1 -->|PostgreSQL| RDS
    MM2 -.->|PostgreSQL| RDS
    MM1 -->|S3 API| S3
    MM2 -.->|S3 API| S3
    MM1 -.->|Outbound| NAT
    MM2 -.->|Outbound| NAT
    NAT -.->|Internet<br/>Updates/SES| Internet

    classDef public fill:#e1f5ff,stroke:#01579b
    classDef private fill:#fff3e0,stroke:#e65100
    classDef storage fill:#f3e5f5,stroke:#4a148c

    class ALB,NAT public
    class MM1,MM2,RDS private
    class S3 storage

Loading

Network Architecture

graph LR
    subgraph VPC["🏢 VPC (10.0.0.0/16)"]
        subgraph AZ1["Availability Zone A"]
            PubSub1[Public Subnet<br/>10.0.1.0/24]
            PrivSub1[Private Subnet<br/>10.0.11.0/24]
        end

        subgraph AZ2["Availability Zone B"]
            PubSub2[Public Subnet<br/>10.0.2.0/24]
            PrivSub2[Private Subnet<br/>10.0.12.0/24]
        end

        PubSub1 --> PrivSub1
        PubSub2 --> PrivSub2
    end

    IGW[Internet Gateway] --> PubSub1
    IGW --> PubSub2
    PrivSub1 --> NAT[NAT Gateway]
    PrivSub2 --> NAT
    NAT --> IGW

Loading

📁 Project Structure

mattermost-ops-kit/
│
├── 📄 README.md                    # You are here
├── 📘 BOOTSTRAP.md                 # First-time AWS setup guide
├── 📗 DEVELOPMENT.md               # Developer environment setup
├── 📙 CONTRIBUTING.md              # Contribution guidelines
├── 📒 CHANGELOG.md                 # Module/API change history
├── 🧪 examples/                    # Starter tfvars profiles
│
└── 🗂️ tofu/                        # OpenTofu infrastructure code
    ├── 🔧 main.tf                  # Provider and common resources
    ├── 🎛️ variables.tf             # Input variables
    ├── 📊 outputs.tf               # Output values
    ├── 🌐 vpc.tf                   # Network infrastructure (VPC, subnets, NAT)
    ├── 🔒 security-groups.tf       # Security rules and firewall
    ├── 👤 iam.tf                   # IAM roles and policies
    ├── 🗄️ rds.tf                   # PostgreSQL database
    ├── 📦 s3.tf                    # S3 buckets for file storage
    ├── 🖥️ ec2.tf                   # EC2 compute instances
    ├── 🚀 user-data.sh             # Docker installation script
    ├── ⚖️ alb.tf                   # Application Load Balancer
    ├── 🔐 acm.tf                   # SSL/TLS certificates
    ├── 📈 cloudwatch.tf            # Monitoring and alerting
    ├── 📧 ses.tf                   # Email notifications (AWS SES)
    ├── 📝 tofu.tfvars.example      # Canonical example configuration
    ├── 🗂️ backend.hcl.example      # Optional remote state backend template
    └── 📘 README.md                # Module API reference

Prerequisites

Before you begin, ensure you have:

1. AWS Account with administrative access
2. Domain Registration for example.com
3. AWS CLI installed and configured
4. OpenTofu >= 1.6 installed
5. Basic knowledge of AWS and OpenTofu

For Contributors: If you're developing or modifying the infrastructure code, see DEVELOPMENT.md for complete local setup instructions including all linters and security scanners.

Quick Start

1. Install Required Tools

For deployment only, you need OpenTofu and AWS CLI. For development, see DEVELOPMENT.md for full tool setup.

OpenTofu:

# macOS
brew install opentofu

# Linux - Install script
curl -fsSL https://get.opentofu.org/install-opentofu.sh | sudo bash

# Or download binary manually
# Visit: https://opentofu.org/docs/intro/install/

AWS CLI:

# macOS
brew install awscli

# Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Verify Installation:

tofu version  # Should show OpenTofu v1.6+
aws --version  # Should show AWS CLI v2.x

2. Configure AWS Credentials

aws configure
# AWS Access Key ID: [your-access-key]
# AWS Secret Access Key: [your-secret-key]
# Default region name: us-west-1
# Default output format: json

Verify your configuration:

aws sts get-caller-identity

3. Clone and Set Up

cd tofu
cp tofu.tfvars.example tofu.tfvars
# Or start from a profile:
# cp ../examples/production.tofu.tfvars.example tofu.tfvars

4. Configure Variables

Edit tofu/tofu.tfvars with your values:

# Required variables
domain_name  = "mattermost.example.com"
alert_email  = "admin@example.com"
ssh_key_name = "mattermost-key"    # Create this in AWS EC2 console first

# Optional: Set a custom database password or leave empty for auto-generation
db_password = ""

# Optional: Configure SES email
ses_email_identity = "noreply@example.com"

5. Deploy

cd tofu

# Initialize OpenTofu
tofu init
# Optional remote state:
# cp backend.hcl.example backend.hcl
# tofu init -backend-config=backend.hcl

# Review the deployment plan
tofu plan

# Deploy infrastructure
tofu apply

Type yes when prompted to confirm.

👨‍💻 Developer Quickstart

New to the project? Here's how to get started making changes to the infrastructure.

🛠️ Setup Your Development Environment

1. Install required tools:

   # macOS
   brew install opentofu awscli pre-commit tflint trivy checkov shellcheck shfmt
   
   # Linux (Ubuntu/Debian)
   # See DEVELOPMENT.md for detailed instructions

2. Configure AWS credentials:

   aws configure
   # Enter your AWS access key, secret key, and set region to us-west-1

3. Clone and setup the repository:

   git clone <repository-url>
   cd mattermost-ops-kit
   
   # Install pre-commit hooks (runs quality checks on every commit)
   pre-commit install
   
   # Initialize OpenTofu
   cd tofu
   tofu init

🚀 Making Infrastructure Changes

1. Create a feature branch:

   git checkout -b feature/your-change-name

2. Make your changes to .tf files in the tofu/ directory

3. Test locally:

   # Format code
   tofu fmt -recursive
   
   # Validate syntax
   tofu validate
   
   # Run security checks
   make check  # or: pre-commit run --all-files
   
   # Preview changes
   tofu plan

4. Commit your changes:

   git add .
   git commit -m "feat: description of your change"
   # Pre-commit hooks will run automatically

🚀 Deploying Changes to AWS

⚠️ Warning: These commands modify production infrastructure!

cd tofu

# Always review the plan first
tofu plan

# Apply changes (requires confirmation)
tofu apply

# Monitor deployment
aws logs tail /aws/mattermost/mattermost-prod --follow

🔍 Common Development Tasks

Access the EC2 instance:

aws ssm start-session --target $(cd tofu && tofu output -raw ec2_instance_id)

View Mattermost logs:

# Via CloudWatch
aws logs tail /aws/mattermost/mattermost-prod --follow

# Or directly on instance
aws ssm start-session --target $(cd tofu && tofu output -raw ec2_instance_id)
sudo docker logs mattermost -f

Verify S3 file storage:

aws s3 ls s3://$(cd tofu && tofu output -raw s3_bucket_name)/ --recursive

Restart Mattermost:

aws ssm start-session --target $(cd tofu && tofu output -raw ec2_instance_id)
sudo systemctl restart mattermost

📚 Additional Resources

• Full setup guide: See DEVELOPMENT.md for complete tool installation
• First-time deployment: See BOOTSTRAP.md for AWS account setup
• Contributing guidelines: See CONTRIBUTING.md for code standards
• Architecture details: See tofu/README.md for technical documentation

---

Detailed Bootstrapping Instructions

Step 1: AWS Account Setup

Create IAM User for OpenTofu

For security, create a dedicated IAM user instead of using root credentials:

1. Log into AWS Console

2. Go to IAM > Users > Add User

3. User name: tofu-deployer

4. Enable: "Programmatic access"

5. Attach policies:

   • AdministratorAccess (for initial setup)
   • Or use a custom policy with minimum required permissions

6. Save the Access Key ID and Secret Access Key

Create SSH Key Pair

# In AWS Console: EC2 > Key Pairs > Create Key Pair
# Name: mattermost-key
# Type: RSA
# Format: .pem

# Or via CLI:
aws ec2 create-key-pair \
  --key-name mattermost-key \
  --query 'KeyMaterial' \
  --output text > ~/.ssh/mattermost-key.pem

chmod 400 ~/.ssh/mattermost-key.pem

Step 2: Domain Registration

Register example.com with a domain registrar:

1. Choose a registrar (e.g., Route53, Namecheap, GoDaddy)
2. Register example.com
3. Note the nameservers

Option A: Use AWS Route53 (Recommended)

# Create hosted zone
aws route53 create-hosted-zone --name example.com --caller-reference $(date +%s)

# Note the nameservers from output
# Update your domain registrar to use these nameservers

Option B: Use External DNS Provider

After deployment, create DNS records manually.

Step 3: OpenTofu State Management (Recommended)

For production, store OpenTofu state in S3:

# Create S3 bucket for state
aws s3 mb s3://mattermost-ops-tofu-state --region us-west-1

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket mattermost-ops-tofu-state \
  --versioning-configuration Status=Enabled

# Enable encryption
aws s3api put-bucket-encryption \
  --bucket mattermost-ops-tofu-state \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      }
    }]
  }'

# Create DynamoDB table for state locking
aws dynamodb create-table \
  --table-name mattermost-ops-tofu-locks \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --region us-west-1

Create tofu/backend.hcl with your backend settings:

bucket         = "mattermost-ops-tofu-state"
key            = "mattermost/tofu.tfstate"
region         = "us-west-1"
encrypt        = true
dynamodb_table = "mattermost-ops-tofu-locks"

Then initialize with backend config:

tofu init -backend-config=backend.hcl

Step 4: SES Email Setup (Optional)

If you want email notifications:

# Verify email identity
aws ses verify-email-identity \
  --email-address noreply@example.com \
  --region us-west-1

# Check your inbox and click verification link

# Request production access (to move out of sandbox)
# Go to AWS Console > SES > Account Dashboard > Request production access

Step 5: Deploy Infrastructure

cd tofu

# Initialize (first time only)
tofu init -backend-config=backend.hcl

# Validate configuration
tofu validate

# See what will be created
tofu plan

# Deploy (will take ~15-20 minutes)
tofu apply

# Save outputs
tofu output > outputs.txt

Step 6: Post-Deployment Configuration

Configure DNS

If using Route53:

# Get ALB DNS name from tofu output
ALB_DNS=$(tofu output -raw alb_dns_name)
ALB_ZONE_ID=$(tofu output -raw alb_zone_id)
HOSTED_ZONE_ID="Z1234567890ABC"  # Your Route53 zone ID

# Create alias record
aws route53 change-resource-record-sets \
  --hosted-zone-id $HOSTED_ZONE_ID \
  --change-batch '{
    "Changes": [{
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "mattermost.example.com",
        "Type": "A",
        "AliasTarget": {
          "HostedZoneId": "'$ALB_ZONE_ID'",
          "DNSName": "'$ALB_DNS'",
          "EvaluateTargetHealth": true
        }
      }
    }]
  }'

If using external DNS:

Create an A or CNAME record:

• Name: mattermost
• Type: CNAME
• Value: [ALB DNS from tofu output]
• TTL: 300

Verify SSL Certificate

# Check certificate status
aws acm describe-certificate \
  --certificate-arn $(tofu output -raw acm_certificate_arn) \
  --region us-west-1

# Add DNS validation record if needed

For DNS validation, add the CNAME record shown in the certificate details to your DNS provider.

Subscribe to Alerts

Check the email you provided for SNS subscription confirmation and click the link.

Step 7: Access Mattermost

Once DNS propagates (5-30 minutes):

1. Visit: https://mattermost.example.com
2. Create first admin account
3. Configure your workspace

Managing the Infrastructure

View Current Resources

tofu show
tofu state list

Update Configuration

# Modify tofu.tfvars or .tf files
vim tofu/tofu.tfvars

# Preview changes
tofu plan

# Apply changes
tofu apply

View Logs

# CloudWatch logs
aws logs tail /aws/mattermost/mattermost-prod --follow --region us-west-1

# Access EC2 via Systems Manager
INSTANCE_ID=$(tofu output -raw ec2_instance_id)
aws ssm start-session --target $INSTANCE_ID --region us-west-1

# Inside the instance
sudo docker logs mattermost -f
sudo journalctl -u mattermost -f

Update Mattermost Version

# SSH to instance
aws ssm start-session --target $(tofu output -raw ec2_instance_id)

# Update docker-compose
cd /opt/mattermost
sudo vim .env  # Change MATTERMOST_VERSION=9.5.0

# Restart
sudo systemctl restart mattermost

Backup and Restore

# Create RDS snapshot
aws rds create-db-snapshot \
  --db-instance-identifier mattermost-prod-db \
  --db-snapshot-identifier mattermost-backup-$(date +%Y%m%d) \
  --region us-west-1

# List S3 versions (automatic due to versioning)
aws s3api list-object-versions \
  --bucket $(tofu output -raw s3_bucket_name)

📈 Scaling

As your user base grows, scale the infrastructure:

🔝 Vertical Scaling (Easiest)

Increase instance sizes in tofu.tfvars:

ec2_instance_type = "t3.large"    # 2 vCPU → 4 vCPU
db_instance_class = "db.t4g.small" # 1 GB → 2 GB RAM

Then apply: tofu apply

Suitable for: 50-200 concurrent users

➡️ Horizontal Scaling (Advanced)

For larger scale (500+ users), consider:

1. 🔄 Auto Scaling Group - Multiple EC2 instances with auto-scaling
2. 🗄️ Aurora PostgreSQL - Clustered database with read replicas
3. ⚡ ElastiCache Redis - Session caching and performance
4. 🌐 CloudFront - CDN for static assets
5. 📊 Read Replicas - Offload read traffic from primary DB

Suitable for: 200+ concurrent users

🔒 Security Best Practices

✅ Already Implemented

• ✅ Encryption at rest - RDS and S3 encrypted
• ✅ Encryption in transit - TLS 1.3 for all HTTPS traffic
• ✅ Private subnets - EC2 and RDS not directly accessible
• ✅ Security groups - Least-privilege firewall rules
• ✅ MFA support - Enabled for Mattermost users
• ✅ Locked down signups - Admin-only user creation
• ✅ Email verification - Required for all accounts
• ✅ S3 versioning - File backup and recovery
• ✅ VPC Flow Logs - Network traffic monitoring
• ✅ CloudWatch monitoring - Logs and alarms
• ✅ Systems Manager - Secure instance access (no SSH)

📋 Recommended Additional Measures

• Enable MFA for AWS root account
• Use least-privilege IAM policies for human users
• Enable AWS Config for compliance monitoring
• Enable GuardDuty for threat detection
• Set up AWS WAF on ALB for DDoS protection
• Regular security updates via Systems Manager Patch Manager
• Enable CloudTrail for audit logging
• Review security groups quarterly
• Implement AWS Backup for automated backups
• Set up AWS Security Hub for centralized security view

💰 Cost Estimation

Estimated monthly costs in us-west-1 (Northern California):

Default Configuration (NAT Gateway Disabled)

Resource	Configuration	Monthly Cost
🖥️ EC2 Instance	t3.medium	~$35
🗄️ RDS Database	db.t4g.micro, PostgreSQL 16	~$15
⚖️ Application Load Balancer	Standard	~$18
📦 S3 Storage	10 GB + requests	~$0.25
🔌 VPC Endpoints	S3 (free) + SES (~$7)	~$7
📊 CloudWatch	Logs + metrics	~$2-5
🌍 Data Transfer	Outbound (via endpoints)	~$2-10
	Estimated Total	💵 ~$79-90/month

💡 Cost Savings: By using VPC endpoints and disabling the NAT Gateway by default, this deployment saves ~$32/month compared to traditional NAT-based architectures.

With NAT Gateway Enabled (For Maintenance)

When you need to perform system updates or Docker pulls, temporarily enable the NAT Gateway:

Additional Resource	Monthly Cost
🌐 NAT Gateway (temporary, ~2 hours/month)	~$0.20
Total with occasional NAT	💵 ~$79-91/month

See docs/MAINTENANCE.md for how to enable/disable NAT Gateway for maintenance windows.

💡 Cost Optimization Tips

• 🎟️ Use Reserved Instances - Save 40-60% on EC2 and RDS
• ⏰ Schedule shutdowns - Stop non-production during off-hours
• 🗂️ S3 lifecycle policies - Archive old files to Glacier
• 🔍 Monitor regularly - Use AWS Cost Explorer and billing alerts
• 🚀 Right-size resources - Adjust instance types based on actual usage
• 🔌 VPC endpoints - Already implemented for S3 and SES (saves data transfer costs)
• 🚪 Keep NAT disabled - Only enable for maintenance windows (default configuration)

Troubleshooting

OpenTofu Issues

# Refresh state
tofu refresh

# Fix state drift
tofu apply -refresh-only

# View detailed logs
export TF_LOG=DEBUG
tofu apply

Connection Issues

# Check ALB target health
aws elbv2 describe-target-health \
  --target-group-arn [arn-from-output] \
  --region us-west-1

# Check EC2 instance status
aws ec2 describe-instance-status \
  --instance-ids [instance-id] \
  --region us-west-1

# Test from inside VPC
aws ssm start-session --target [instance-id]
curl http://localhost:8065/api/v4/system/ping

Docker Issues

# SSH to instance
aws ssm start-session --target [instance-id]

# Check Docker status
sudo systemctl status mattermost
sudo docker ps
sudo docker logs mattermost

# Restart services
sudo systemctl restart mattermost

Cleanup

To destroy all resources:

cd tofu

# Disable deletion protection first
vim rds.tf  # Set deletion_protection = false

tofu apply  # Apply the change

# Destroy everything
tofu destroy

Warning: This will permanently delete all data. Ensure you have backups!

📚 Documentation

Document	Description
📘 BOOTSTRAP.md	Complete first-time AWS setup and deployment guide
📗 DEVELOPMENT.md	Developer environment setup and tool installation
📙 CONTRIBUTING.md	Code standards, PR process, and quality checks
📘 tofu/README.md	Module API reference (inputs, outputs, behavior toggles)
📒 CHANGELOG.md	History of API and behavior changes
🧪 examples/README.md	Starter configuration profiles

🔗 External Resources

• Mattermost Documentation: docs.mattermost.com
• OpenTofu Documentation: opentofu.org/docs
• AWS Documentation: docs.aws.amazon.com
• OpenTofu AWS Provider: registry.opentofu.org/providers/hashicorp/aws

---

🤝 Contributing

This is an open-source infrastructure repository. We welcome contributions.

Quick Start for Contributors

# Install development tools and pre-commit hooks
make setup

# Run all quality checks
make check

# See all available commands
make help

Code Quality Standards

This project maintains high quality standards with:

• ✅ Pre-commit hooks - Automatic checks before every commit
• ✅ OpenTofu formatting - Consistent code style (tofu fmt)
• ✅ Security scanning - Trivy and Checkov for vulnerabilities
• ✅ Shell linting - ShellCheck for bash scripts
• ✅ Documentation linting - Markdown and YAML validation
• ✅ CI/CD pipeline - All checks run on every PR

See CONTRIBUTING.md for detailed guidelines.

---

📄 License

MIT License. See LICENSE.

---

Questions or Issues? Open an issue

Made with ❤️ by Mattermost Ops Kit Infrastructure Team