LogoLogo
CloudDiscordGitHub
  • 👉Getting Started
    • Introduction
    • Quick start
    • Learn by example
    • Case studies
    • How to contribute?
  • ⭐Memphis Broker
    • Architecture
    • Key concepts
      • Message broker
      • Station
      • Producer API
      • Consumer API
      • Consumer Group
      • Storage and Redundancy
      • Security/Authentication
      • Scaling
      • Ordering
      • Dead-letter Station (DLS)
      • Delayed messages
      • Data exchange
      • Idempotency (Duplicate processing)
      • Failover Scenarios
      • Troubleshooting process
      • Connectors
    • Best practices
      • Producer optimization
      • Compression
    • Memphis configuration
    • Comparisons
      • NATS Jetstream vs Memphis
      • RabbitMQ vs Memphis
      • AWS SQS vs Memphis
      • Apache Kafka vs Memphis
      • Apache Pulsar vs Memphis
      • ZeroMQ vs Memphis
      • Apache NiFi vs Memphis
    • Privacy Policy
  • ⭐Memphis Schemaverse
    • Overview
    • Getting started
      • Management
      • Produce/Consume
        • Protobuf
        • JSON Schema
        • GraphQL
        • Avro
    • Comparison
    • KB
  • 📦Open-Source Installation
    • Kubernetes
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
      • Terraform
        • Deploy on AWS
        • Deploy on GCP
        • Deploy on DigitalOcean
      • Guides
        • Deploy/Upgrade Memphis utilizing predefined secrets
        • Monitoring/Alerts Recommendations
        • Production Best Practices
        • NGINX Ingress Controller and Cloud-Agnostic Memphis Deployments
        • Migrate Memphis storage between storageClass's
        • Expanding Memphis Disk Storage
        • Scale-out Memphis cluster
        • TLS - Deploy Memphis with TLS Connection to Metadata Frontend
        • TLS - Memphis TLS websocket configuration
        • TLS - Securing Memphis Client with TLS
        • Installing Memphis with an External Metadata Database
    • Docker
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
    • Open-source Support
  • Client Libraries
    • REST (Webhook)
    • Node.js / TypeScript / NestJS
    • Go
    • Python
    • Kotlin (Community)
    • .NET
    • Java
    • Rust (Community)
    • NATS
    • Scala
  • 🔌Integrations Center
    • Index
    • Processing
      • Zapier
    • Change data Capture (CDC)
      • Debezium
    • Monitoring
      • Datadog
      • Grafana
    • Notifications
      • Slack
    • Storage tiering
      • S3-Compatible Object Storage
    • Source code
      • GitHub
    • Other platforms
      • Argo
  • 🗒️Release notes
    • KB
    • Releases
      • v1.4.3 - latest/stable
      • v1.4.2
      • v1.4.1
      • v1.4.0
      • v1.3.1
      • v1.3.0
      • v1.2.0
      • v1.1.1
      • v1.1.0
      • v1.0.3
      • v1.0.2
      • v1.0.1
      • V1.0.0 - GA
      • v0.4.5 - beta
      • v0.4.4 - beta
      • v0.4.3 - beta
      • v0.4.2 - beta
      • v0.4.1 - beta
      • v0.4.0 - beta
      • v0.3.6 - beta
      • v0.3.5 - beta
      • v0.3.0 - beta
      • v0.2.2 - beta
      • v0.2.1 - beta
      • v0.2.0 - beta
      • v0.1.0 - beta
Powered by GitBook
LogoLogo

Legal

  • Terms of Service
  • Privacy Policy

All rights reserved to Memphis.dev 2023

On this page
  • Requirements
  • Installation
  • Deployed pods
  • Deployment diagram
  • Appendix A: Install Memphis using predefined parameters
  • Execute Helm install with the created values file:
  • Creating users
  • Appendix B: Dedicated options per specific K8S distributions
  • Appendix C: Helm deployment options

Was this helpful?

  1. Open-Source Installation
  2. Kubernetes

1 - Installation

Deploy Memphis over Kubernetes

Last updated 11 months ago

Was this helpful?

Helm is a K8s package manager that allows users to deploy apps in a single, configurable command. More information about Helm can be found .

Memphis is cloud-native and cloud-agnostic to any Kubernetes on any cloud.

Requirements

Minimum Requirements (Without high availability)

Resource
Quantity

Minimum Kubernetes version

1.20 and above

K8S Nodes

1

CPU

2 CPU

Memory

4GB RAM

Storage

12GB PVC

Recommended Requirements (With high availability)

Resource
Minimum Quantity

Minimum Kubernetes version

1.20 and above

K8S Nodes

3

CPU

4 CPU

Memory

8GB RAM

Storage

12GB PVC Per node

Installation

Production

Production-ready Memphis deployment with initial three memphis brokers configured in cluster mode for high availability and higher throughput.

Stable release

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --set global.cluster.enabled="true" --create-namespace --namespace memphis --wait --version=1.4.4

Latest release

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install --set global.cluster.enabled="true" memphis memphis/memphis --create-namespace --namespace memphis --wait
Development

Minimal deployment of Memphis with a single broker

Stable release

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --create-namespace --namespace memphis --wait --version=1.4.4

Latest release

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install memphis memphis/memphis --create-namespace --namespace memphis --wait

Deployed pods

  • memphis. Memphis broker.

  • memphis-rest-gateway. Memphis REST Gateway.

  • memphis-metadata. Metadata store.

  • memphis-metadata-coordinator. Metadata coordinator

Deployment diagram

Appendix A: Install Memphis using predefined parameters

Currently, you can use this for creating users during deployment.

Execute Helm install with the created values file:

helm install my-memphis memphis -f config.yaml --create-namespace --namespace memphis --wait

Creating users

(Based on Memphis password policy: at least 8 characters long, contains both uppercase and lowercase, and at least one number and one special character(!?-@#$%):

config.yaml
auth:
#By default, Memphis sets this option to "false," enabling first user creation during the initial login.
  enabled: true
  users:
    mgmt:
    - user: admin
      password: Admin123456!
    - user: test_mgmt
      password: Test123456!
    - user: test
      password: Test123456@
    client:
    - user: test_app
      password: Test123456!@
    - user: test_app2
      password: Test123456@!

Appendix B: Dedicated options per specific K8S distributions

To deploy the Memphis cluster on top of Red Hat Openshift it's necessary to configure default security context parameters as follows:

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && 
helm install memphis memphis/memphis --set \
global.cluster.enabled="true",\
metadata.postgresql.containerSecurityContext.enabled="false",\
metadata.postgresql.podSecurityContext.enabled="false",\
metadata.pgpool.containerSecurityContext.enabled="false",\
metadata.pgpool.podSecurityContext.enabled="false" \
--create-namespace --namespace memphis --wait

Appendix C: Helm deployment options

Option
Description
Default Value
Example

global.cluster.enabled

Cluster mode for HA and Performance

"false"

"false"

exporter.enabled

Prometheus exporter

"false"

"false"

exporter.serviceExposed.enbaled

Expose metrics port with memphis service

"false"

"true"

cluster.enabled

Enables Memphis cluster deployment. For fully HA configuration use global.cluster.enabled

"false"

"true"

cluster.replicas

Memphis broker replicas

"3"

"5"

memphis.image

Memphis image name

"memphisos/memphis:x.x.x-stable"

"memphisos/memphis:latest"

memphis.ui.port

Dashboard's (GUI) port

9000

9000

memphis.hosts.uiHostName

Which URL should be seen as the "UI hostname"

""

"https://memphis.example.com"

memphis.hosts.restgwHostName

Which URL should be seen as the "REST Gateway hostname"

""

"https://restgw.memphis.example.com"

memphis.hosts.brokerHostName

Which URL should be seen as the "broker hostname"

""

"memphis.example.com"

memphis.configFile.logsRetentionInDays

Amount of days to retain system logs

3

3

memphis.configFile.gcProducerConsumerRetentionInHours

Amount of hours to retain producer/consumer in system

3

3

memphis.configFile.tieredStorageUploadIntervalSeconds

nterval in seconds between uploads to tiered storage

8

8

memphis.configFile.dlsRetentionHours

Amount of hours to retain messages in DLS

3

3

memphis.configFile.userPassBasedAuth

Authentication method selector. true = User + pass false = User + connection token

"true"

"true"

memphis.creds.rootPwd

Root password for the dashboard. Randomly generated.

""

"superpass"

memphis.creds.connectionToken

Token for connecting an app to the Memphis Message Queue. Auto generated.Randomly generated.

""

"connectionToken

memphis.creds.jwtSecret

For internal traffic. Randomly generated.

""

"<JWT_TOKEN>"

memphis.creds.refreshJwtSecret

For internal traffic. Randomly generated.

""

"<JWT_TOKEN>"

memphis.creds.encryptionSecretKey

Encryption secret key for internal encryption. Randomly generated.

""

""

memphis.creds.secretConfig.name

Name of the secret with memphis creds

"memphis-creds"

"external-creds"

memphis.creds.secretConfig.existingSecret

*Optional* For use of the existing secret with memphis creds

"false"

"true"

memphis.creds.secretConfig.rootPwd_key

*Optional* Name of the key in secret

"ROOT_PASSWORD"

"ROOT_PASSWORD"

memphis.creds.secretConfig.connectionToken_key

*Optional* Name of the key in secret

"CONNECTION_TOKEN"

"CONNECTION_TOKEN"

memphis.creds.secretConfig.jwtSecret_key

*Optional* Name of the key in secret

"JWT_SECRET"

"JWT_SECRET"

memphis.creds.secretConfig.refreshJwtSecret_key

*Optional* Name of the key in secret

"REFRESH_JWT_SECRET"

"REFRESH_JWT_SECRET"

memphis.creds.secretConfig.encryptionSecretKey_key

*Optional* Name of the key in secret

"ENCRYPTION_SECRET_KEY"

"ENCRYPTION_SECRET_KEY"

memphis.creds.secretConfig.refreshJwtSecretRestGW_key

*Optional* Name of the key in secret

"REFRESH_JWT_SECRET_REST_GW"

"REFRESH_JWT_SECRET_REST_GW"

memphis.creds.secretConfig.jwtSecretRestGW_key

*Optional* Name of the key in secret

"JWT_SECRET_REST_GW"

"JWT_SECRET_REST_GW"

memphis.extraEnvironmentVars.enabled

*Optional* List of additional environment variables for memphis.

""

vars: - name: KEY - valye: value

memphis.tls.verify

*Optional* For encrypted client-memphis communication. Verification for the CA autority. SSL.

""

"true"

memphis.tls.secret.name

*Optional* For encrypted client-memphis communication. K8S secret name that holds the certs. SSL.

""

"memphis-client-tls-secret"

memphis.tls.cert

*Optional* For encrypted client-memphis communication. .pem file to use. SSL.

""

"memphis_client.pem"

memphis.tls.key

*Optional* For encrypted client-memphis communication. Private key file to use. SSL.

""

"memphis-key_client.pem"

memphis.tls.ca

*Optional* For encrypted client-memphis communication. CA file to use. SSL.

""

"rootCA.pem"

websocket.enabled

Memphis GUI using websockets for live rendering.

"true"

"false"

websocket.port

Memphis GUI using websockets for live rendering. The port can be configured

"7770"

""

websocket.host

Websocket host can be handled on separate LB/DNS.

"localhost"

"ws.example.com"

websocket.noTLS

Websocket can be configured with tls, default is noTLS.

"true"

"false"

websocket.tls.secret.name

*Optional* Memphis GUI using websockets for live rendering. K8S secret name for the certs

""

"memphis-ws-tls-secret"

websocket.tls.cert

*Optional* Memphis GUI using websockets for live rendering. .pem file to use

""

"memphis_local.pem"

websocket.tls.key

*Optional* Memphis GUI using websockets for live rendering. key file

""

"memphis-key_local.pem"

metadata.postgresql.username

*Optional* Username for postgres db

"postgres"

"postgres"

metadata.postgresql.existingSecret

*Optional* An ability to provide predefined secret for metadata PostgreSQL credentials

""

"metadata-creds.yaml"

metadata.pgpool.existingSecret

*Optional* An ability to provide predefined secret for metadata PG credentials

""

"metadata-creds.yaml"

metadata.pgpool.tls.enabled

*Optional* Enabling TLS-based communication with PG

"false"

"false"

metadata.pgpool.tls.certificatesSecret

*Optional* PG TLS cert secret to be used

""

"tls-secret"

metadata.pgpool.tls.certFilename

*Optional* PG TLS cert file to be used

""

"tls.crt"

metadata.pgpool.tls.certKeyFilename

*Optional* PG TLS key to be used

""

"tls.key"

metadata.pgpool.tls.certCAFilename

*Optional* PG TLS cert CA to be used

""

"ca.crt"

metadata.external.enabled

*Optional* For using external PG instead of deploying dedicated one for Memphis

"false"

"true"

metadata.external.dbTlsMutual

*Optional* External PG TLS-basec communication

"true"

"true"

metadata.external.dbName

*Optional* External PG db name

""

"memphis"

metadata.external.dbHost

*Optional* External PG db hostname

""

"metadata.example.url"

metadata.external.dbPort

*Optional* External PG db port

""

5432

metadata.external.dbUser

*Optional* External PG db user

""

"postgres"

metadata.external.dbPass

*Optional* External PG db password

""

"12345678"

metadata.external.secret.enabled

*Optional* Enable an option to use secret for password store

"false"

"true"

metadata.external.secret.name

*Optional* Secret name

""

"metadata-secret"

metadata.external.secret.dbPass_key

*Optional* Name of the key in the secret

""

"dbPass"

restGateway.enabled

*Optional* Memphis Rest Gateway can be disabled if not in use

"true"

"false"

restGateway.jwtSecret

*Optional* Manual Jwt Token configurtion

""

""

restGateway.refreshJwtSecret

*Optional* Manual Refresh Jwt Token configurtion

""

""

auth.enabled

*Optional* Enable using predefined parameters

"false"

"true"

auth.enabled.mgmt

*Optional* Management users that will be created at first deployment

auth.enabled.client

*Optional* Client users that will be created at first deployment

Search terms: SSL

Additional helm options can be found .

For more information on each component, please head to the .

Refer to the example file for guidance:

📦
here
example/initial_config_values.yaml
here
architecture section