Memphis is a next-generation alternative to traditional message brokers.

Docs - Twitter - YouTube

A simple, robust, and durable cloud-native message broker wrapped with
an entire ecosystem that enables cost-effective, fast, and reliable development of modern queue-based use cases.

Memphis enables the building of modern queue-based applications that require
large volumes of streamed and enriched data, modern protocols, zero ops, rapid development,
extreme cost reduction, and a significantly lower amount of dev time for data-oriented developers and data engineers.

Memphis Kubernetes Deployment

If you prefer using Terraform, head here

Helm is a k8s package manager that allows users to deploy apps in a single, configurable command.
More information about Helm can be found here.

Memphis is cloud-native and cloud-agnostic to any Kubernetes on any cloud.

Requirements

Minimum Requirements (Without high availability)

Resource	Quantity
Minimum Kubernetes version	1.20 and above
K8S Nodes	1
CPU	2 CPU
Memory	4GB RAM
Storage	12GB PVC

---

Recommended Requirements (With high availability)

Resource	Minimum Quantity
Minimum Kubernetes version	1.20 and above
K8S Nodes	3
CPU	4 CPU
Memory	8GB RAM
Storage	12GB PVC Per node

Installation

Production

Production-grade Memphis with three memphis brokers configured in cluster-mode

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --set global.cluster.enabled="true" --create-namespace --namespace memphis --wait

Dev

Standard installation of Memphis with a single broker

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && 
helm install memphis memphis/memphis --create-namespace --namespace memphis --wait

* Optional * Helm deployment options

Option	Description	Default Value	Example
rootPwd	Root password for the dashboard	"memphis"	"memphis"
user_pass_based_auth	Authentication method selector. true = User + pass false = User + connection token	"true"	"true"
connectionToken	Token for connecting an app to the Memphis Message Queue. Auto generated	""	"memphis"
dashboard.port	Dashboard's (GUI) port	9000	9000
global.cluster.enabled	Cluster mode for HA and Performance	"false"	"false"
exporter.enabled	Prometheus exporter	"false"	"false"
analytics	Collection of anonymous metadata	"true"	"true"
websocket.tls.secret.name	Optional Memphis GUI using websockets for live rendering. K8S secret name for the certs	""	"memphis-ws-tls-secret"
websocket.tls.cert	Optional Memphis GUI using websockets for live rendering. .pem file to use	""	"memphis_local.pem"
websocket.tls.key	Optional Memphis GUI using websockets for live rendering. key file	""	"memphis-key_local.pem"
memphis.tls.verify	Optional For encrypted client-memphis communication. Verification for the CA autority. SSL.	""	"true"
memphis.tls.secret.name	Optional For encrypted client-memphis communication. K8S secret name that holds the certs. SSL.	""	"memphis-client-tls-secret"
memphis.tls.cert	Optional For encrypted client-memphis communication. .pem file to use. SSL.	""	"memphis_client.pem"
memphis.tls.key	Optional For encrypted client-memphis communication. Private key file to use. SSL.	""	"memphis-key_client.pem"
memphis.tls.ca	Optional For encrypted client-memphis communication. CA file to use. SSL.	""	"rootCA.pem"

Here is how to run an installation command with additional options -

helm install memphis --set cluster.replicas=3,rootPwd="rootpassword" memphis/memphis --create-namespace --namespace memphis

Deployed pods

• memphis. Memphis broker.
• memphis-rest-gateway. Memphis REST Gateway.
• memphis-metadata. Metadata store.

For more information on each component, please head to the architecture section.

Deploy Memphis with TLS (encrypted communication via SSL)

0. Optional: Create self-signed certificates

a) Generate a self-signed certificate using mkcert

$ mkcert -client \
-cert-file memphis_client.pem \
-key-file memphis-key_client.pem  \
"127.0.0.1" "localhost" "*.memphis.dev" ::1 \
email@localhost valera@Valeras-MBP-2.lan

b) Find the rootCA

$ mkcert -CAROOT

c) Create self-signed certificates for client

$ mkcert -client -cert-file client.pem -key-file key-client.pem  localhost ::1 

1. Create namespace + secret for the TLS certs

a) Create a dedicated namespace for memphis

kubectl create namespace memphis

b) Create a k8s secret with the required certs

{% code overflow="wrap" lineNumbers="true" %}

kubectl create secret generic memphis-client-tls-secret \
--from-file=memphis_client.pem \
--from-file=memphis-key_client.pem \
--from-file=rootCA.pem -n memphis

tls:
  secret:
    name: memphis-client-tls-secret
  ca: "rootCA.pem"
  cert: "memphis_client.pem"
  key: "memphis-key_client.pem"

2. Deploy Memphis with the generated certificate

helm install memphis memphis \
--create-namespace --namespace memphis --wait \
--set \
cluster.enabled="true",\
memphis.tls.verify="true",\
memphis.tls.cert="memphis_client.pem",\
memphis.tls.key="memphis-key_client.pem",\
memphis.tls.secret.name="memphis-client-tls-secret",\
memphis.tls.ca="rootCA.pem"

Upgrade existing deployment

For adding TLS support

1. Create a k8s secret with the provided TLS certs

kubectl create secret generic memphis-client-tls-secret \
--from-file=memphis_client.pem \
--from-file=memphis-key_client.pem \
--from-file=rootCA.pem -n memphis

2. Upgrade Memphis to use the TLS certs

helm upgrade memphis memphis -n memphis --reuse-values \
--set \
memphis.tls.verify="true",\
memphis.tls.cert="memphis_client.pem",\
memphis.tls.key="memphis-key_client.pem",\
memphis.tls.secret.name="tls-client-secret",\
memphis.tls.ca="rootCA.pem"

Deployment diagram