Comment on page
Production Best Practices
This section describes the best practices to run Memphis in Production environment and to maximize
To deploy Memphis in a Kubernetes environment, allocate one dedicated worker node for each Memphis broker. This practice serves several vital purposes:
- Resource isolation: Memphis assumes that all available system resources are exclusively assigned to Memphis brokers, allowing them to handle substantial data loads and traffic efficiently.
- Fault tolerance: Distributing Memphis brokers across separate worker nodes minimizes the impact of a faulty node, ensuring minimal disruption to the entire cluster.
- High Availability (HA): Where feasible, consider employing at least two Availability Zones (AZs) and distributing worker nodes across them to enhance high availability.
- For production environments, a minimum of 4 CPU cores is recommended.
- Memphis supports both x86_64 and Arm64 processors.
- A minimum of 2 GB of memory per core is required.
- Filesystem: Utilize the XFS or ext4 filesystem for your storage needs.
- Dedicated Persistent Volumes: Assign dedicated Persistent Volumes to each Memphis broker for efficient data storage.
- Disk Type: It is recommended to use NVMe disks for storage, while it is advisable to avoid using HDDs.
- Disk Size: Allocate a minimum of 50GB disk space per broker. The exact size may vary based on the expected workload, calculated as a function of the "average message size multiplied by the number of messages."
- Ensure data-at-rest encryption for enhanced security. Additionally, expose the Memphis URLs using HTTPS to protect data in transit.
- Minimum 10Gbps (10Gige NICs) for optimal performance.
- Deploy a Memphis cluster with a minimum of three replicas. Memphis recommends an odd number of replicas (3, 5, 7, etc.).
- By default, when
global.cluster.enabled="true"value is provided in the Memphis Helm chart, it deploys:
- Three Memphis brokers. Users can specify the number of Memphis brokers in the
- Two metadata pods with a metadata coordinator.
- Two RestGW component instances.
- Use PersistentVolumes backed by high-performance NVMe disks, which are strongly recommended.
- The Memphis Helm chart uses the default
storageClassconfigured in the Kubernetes cluster to create PVCs (Persistent Volume Claims) for each broker. Make sure it is configured.
- The default PVC size is 30GB but can be configured using Helm values, for example,
- Memphis requires instances with a minimum of 10Gige NICs for optimal performance.
- Use a LoadBalancer in front of Memphis brokers for UI and WebSocket (WS) ports to enhance the user experience. Load Balancers provide fault tolerance and improve reliability.
- Memphis brokers and clients co-located within the same Kubernetes cluster utilize a headless ClusterIP service for their communication. This service is automatically created as part of the Memphis deployment and is associated with the Memphis StatefulSet.
- Any client deployed within the same Kubernetes cluster should employ the internal network with the default internal record:
- External clients, not having access to internal service records, require configuration with externally exposed records.
- Memphis supports network encryption through TLS. For details, refer to the [TLS page].
- Disk encryption, particularly in cloud distributions, can be configured in the
storageClassconfiguration. For example:....provisioner: ebs.csi.aws.comparameters:csi.storage.k8s.io/fstype: xfsencrypted: 'true'....
- 1.Use Memphis in cluster mode to enable parallel usage across multiple brokers.
- 2.Spread the workloads across as many partitions as possible (if possible) to spread leaders across different brokers.
- 3.Use NVMe disks for storage.
- 4.Stretch the retention to days/hours.
- 5.Make sure to utilize as many cores as possible within the app itself. For example, in node.js - use threads.
- 6.Use affinity rules and separate the client K8s worker from the memphis workers.
Last modified 14d ago