Ask or search…
K
Links
Comment on page

1 - Installation

Deploy Memphis over Kubernetes
Helm is a K8s package manager that allows users to deploy apps in a single, configurable command. More information about Helm can be found here.
Memphis is cloud-native and cloud-agnostic to any Kubernetes on any cloud.

Requirements

Minimum Requirements (Without high availability)
Resource
Quantity
Minimum Kubernetes version
1.20 and above
K8S Nodes
1
CPU
2 CPU
Memory
4GB RAM
Storage
12GB PVC
Recommended Requirements (With high availability)
Resource
Minimum Quantity
Minimum Kubernetes version
1.20 and above
K8S Nodes
3
CPU
4 CPU
Memory
8GB RAM
Storage
12GB PVC Per node

Installation

Production
Production-ready Memphis deployment with initial three memphis brokers configured in cluster mode for high availability and higher throughput.
Stable release
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --set global.cluster.enabled="true" --create-namespace --namespace memphis --wait
Latest release
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install --set global.cluster.enabled="true" memphis memphis/memphis --create-namespace --namespace memphis --wait
Development
Minimal deployment of Memphis with a single broker
Stable release
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --create-namespace --namespace memphis --wait
Latest release
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --create-namespace --namespace memphis --wait
Here is how to run an installation command with additional options -
helm install memphis --set cluster.replicas=3,memphis.creds.rootPwd=rootpassword" memphis/memphis --create-namespace --namespace memphis

Deployed pods

  • memphis. Memphis broker.
  • memphis-rest-gateway. Memphis REST Gateway.
  • memphis-metadata. Metadata store.
  • memphis-metadata-coordinator. Metadata coordinator
For more information on each component, please head to the architecture section.

Deploy Memphis with TLS

0. Optional: Create self-signed certificates

a) Generate a self-signed certificate using mkcert
$ mkcert -client \
-cert-file memphis_client.pem \
-key-file memphis-key_client.pem \
"127.0.0.1" "localhost" "*.memphis.dev" ::1 \
email@localhost [email protected]
b) Find the rootCA
$ mkcert -CAROOT
c) Create self-signed certificates for client
$ mkcert -client -cert-file client.pem -key-file key-client.pem localhost ::1

1. Create namespace + secret for the TLS certs

a) Create a dedicated namespace for memphis
kubectl create namespace memphis
b) Create a k8s secret with the required certs
1
kubectl create secret generic memphis-client-tls-secret \
2
--from-file=memphis_client.pem \
3
--from-file=memphis-key_client.pem \
4
--from-file=rootCA.pem -n memphis
memphis-client-tls-secret
1
tls:
2
secret:
3
name: memphis-client-tls-secret
4
ca: "rootCA.pem"
5
cert: "memphis_client.pem"
6
key: "memphis-key_client.pem"

2. Deploy Memphis with the generated certificate

1
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update
2
helm install memphis memphis/memphis \
3
--create-namespace --namespace memphis --wait \
4
--set \
5
global.cluster.enabled="true",\
6
memphis.tls.verify="true",\
7
memphis.tls.cert="memphis_client.pem",\
8
memphis.tls.key="memphis-key_client.pem",\
9
memphis.tls.secret.name="memphis-client-tls-secret",\
10
memphis.tls.ca="rootCA.pem"

Upgrade existing deployment

For adding TLS support

  1. 1.
    Create a k8s secret with the provided TLS certs
kubectl create secret generic memphis-client-tls-secret \
--from-file=memphis_client.pem \
--from-file=memphis-key_client.pem \
--from-file=rootCA.pem -n memphis
  1. 2.
    Upgrade Memphis to use the TLS certs
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update
helm upgrade memphis memphis/memphis -n memphis --reuse-values \
--set \
memphis.tls.verify="true",\
memphis.tls.cert="memphis_client.pem",\
memphis.tls.key="memphis-key_client.pem",\
memphis.tls.secret.name="tls-client-secret",\
memphis.tls.ca="rootCA.pem"

Deploy Memphis with an external PostgreSQL instance

Step 1: Create postgresql_values.yaml according to the following example:

metadata:
enabled: false
external:
enabled: true
dbTlsMutual: true
dbName: memphis
dbHost: <URL>
dbPort: 5432
dbUser: postgres
dbPass: "12345678"

Step 2: Deploy Memphis cluster with external PostgreSQL:

helm install memphis memphis/memphis -f postgresql_values.yaml \
--create-namespace --namespace memphis --wait \
--set \
global.cluster.enabled="true"

Deployment diagram

Appendix A: Dedicated options per K8S-distribution

Red Hat OpenShift
To deploy the Memphis cluster on top of Red Hat Openshift it's necessary to configure default security context parameters as follows:
helm repo add memphis https://k8s.memphis.dev/charts/ --force-update &&
helm install memphis memphis/memphis --set \
global.cluster.enabled="true",\
metadata.postgresql.containerSecurityContext.enabled="false",\
metadata.postgresql.podSecurityContext.enabled="false",\
metadata.pgpool.containerSecurityContext.enabled="false",\
metadata.pgpool.podSecurityContext.enabled="false" \
--create-namespace --namespace memphis --wait

Appendix B: Helm deployment options

Option
Description
Default Value
Example
global.cluster.enabled
Cluster mode for HA and Performance
"false"
"false"
exporter.enabled
Prometheus exporter
"false"
"false"
exporter.serviceExposed.enbaled
Expose metrics port with memphis service
"false"
"true"
cluster.enabled
Enables Memphis cluster deployment. For fully HA configuration use global.cluster.enabled
"false"
"true"
cluster.replicas
Memphis broker replicas
"3"
"5"
memphis.image
Memphis image name
"memphisos/memphis:x.x.x-stable"
"memphisos/memphis:latest"
memphis.ui.port
Dashboard's (GUI) port
9000
9000
memphis.hosts.uiHostName
Which URL should be seen as the "UI hostname"
""
"https://memphis.example.com"
memphis.hosts.restgwHostName
Which URL should be seen as the "REST Gateway hostname"
""
"https://restgw.memphis.example.com"
memphis.hosts.brokerHostName
Which URL should be seen as the "broker hostname"
""
"memphis.example.com"
memphis.configFile.logsRetentionInDays
Amount of days to retain system logs
3
3
memphis.configFile.gcProducerConsumerRetentionInHours
Amount of hours to retain producer/consumer in system
3
3
memphis.configFile.tieredStorageUploadIntervalSeconds
nterval in seconds between uploads to tiered storage
8
8
memphis.configFile.dlsRetentionHours
Amount of hours to retain messages in DLS
3
3
memphis.configFile.userPassBasedAuth
Authentication method selector. true = User + pass false = User + connection token
"true"
"true"
memphis.creds.rootPwd
Root password for the dashboard. Randomly generated.
""
"superpass"
memphis.creds.connectionToken
Token for connecting an app to the Memphis Message Queue. Auto generated.Randomly generated.
""
"connectionToken
memphis.creds.jwtSecret
For internal traffic. Randomly generated.
""
"<JWT_TOKEN>"
memphis.creds.refreshJwtSecret
For internal traffic. Randomly generated.
""
"<JWT_TOKEN>"
memphis.creds.encryptionSecretKey
Encryption secret key for internal encryption. Randomly generated.
""
""
memphis.creds.secretConfig.name
Name of the secret with memphis creds
"memphis-creds"
"external-creds"
memphis.creds.secretConfig.existingSecret
*Optional* For use of the existing secret with memphis creds
"false"
"true"
memphis.creds.secretConfig.rootPwd_key
*Optional* Name of the key in secret
"ROOT_PASSWORD"
"ROOT_PASSWORD"
memphis.creds.secretConfig.connectionToken_key
*Optional* Name of the key in secret
"CONNECTION_TOKEN"
"CONNECTION_TOKEN"
memphis.creds.secretConfig.jwtSecret_key
*Optional* Name of the key in secret
"JWT_SECRET"
"JWT_SECRET"
memphis.creds.secretConfig.refreshJwtSecret_key
*Optional* Name of the key in secret
"REFRESH_JWT_SECRET"
"REFRESH_JWT_SECRET"
memphis.creds.secretConfig.encryptionSecretKey_key
*Optional* Name of the key in secret
"ENCRYPTION_SECRET_KEY"
"ENCRYPTION_SECRET_KEY"
memphis.creds.secretConfig.refreshJwtSecretRestGW_key
*Optional* Name of the key in secret
"REFRESH_JWT_SECRET_REST_GW"
"REFRESH_JWT_SECRET_REST_GW"
memphis.creds.secretConfig.jwtSecretRestGW_key
*Optional* Name of the key in secret
"JWT_SECRET_REST_GW"
"JWT_SECRET_REST_GW"
memphis.extraEnvironmentVars.enabled
*Optional* List of additional environment variables for memphis.
""
vars: - name: KEY - valye: value
memphis.tls.verify
*Optional* For encrypted client-memphis communication. Verification for the CA autority. SSL.
""
"true"
memphis.tls.secret.name
*Optional* For encrypted client-memphis communication. K8S secret name that holds the certs. SSL.
""
"memphis-client-tls-secret"
memphis.tls.cert
*Optional* For encrypted client-memphis communication. .pem file to use. SSL.
""
"memphis_client.pem"
memphis.tls.key
*Optional* For encrypted client-memphis communication. Private key file to use. SSL.
""
"memphis-key_client.pem"
memphis.tls.ca
*Optional* For encrypted client-memphis communication. CA file to use. SSL.
""
"rootCA.pem"
websocket.tls.secret.name
*Optional* Memphis GUI using websockets for live rendering. K8S secret name for the certs
""
"memphis-ws-tls-secret"
websocket.tls.cert
*Optional* Memphis GUI using websockets for live rendering. .pem file to use
""
"memphis_local.pem"
websocket.tls.key
*Optional* Memphis GUI using websockets for live rendering. key file
""
"memphis-key_local.pem"
metadata.postgresql.username
*Optional* Username for postgres db
"postgres"
"postgres"
metadata.pgpool.tls.enabled
*Optional* Enabling TLS-based communication with PG
"false"
"false"
metadata.pgpool.tls.certificatesSecret
*Optional* PG TLS cert secret to be used
""
"tls-secret"
metadata.pgpool.tls.certFilename
*Optional* PG TLS cert file to be used
""
"tls.crt"
metadata.pgpool.tls.certKeyFilename
*Optional* PG TLS key to be used
""
"tls.key"
metadata.pgpool.tls.certCAFilename
*Optional* PG TLS cert CA to be used
""
"ca.crt"
metadata.external.enabled
*Optional* For using external PG instead of deploying dedicated one for Memphis
"false"
"true"
metadata.external.dbTlsMutual
*Optional* External PG TLS-basec communication
"true"
"true"
metadata.external.dbName
*Optional* External PG db name
""
"memphis"
metadata.external.dbHost
*Optional* External PG db hostname
""
"metadata.example.url"
metadata.external.dbPort
*Optional* External PG db port
""
5432
metadata.external.dbUser
*Optional* External PG db user
""
"postgres"
metadata.external.dbPass
*Optional* External PG db password
""
"12345678"
metadata.external.secret.enabled
*Optional* Enable an option to use secret for password store
"false"
"true"
metadata.external.secret.name
*Optional* Secret name
""
"metadata-secret"
metadata.external.secret.dbPass_key
*Optional* Name of the key in the secret
""
"dbPass"
restGateway.enabled
*Optional* Memphis Rest Gateway can be disabled if not in use
"true"
"false"
restGateway.jwtSecret
*Optional* Manual Jwt Token configurtion
""
""
restGateway.refreshJwtSecret
*Optional* Manual Refresh Jwt Token configurtion
""
""
Search terms: SSL
Last modified 15d ago