Links

Production Best Practices

This section describes the best practices to run Memphis in Production environment and to maximize

Rule(s)-of-thumb

  1. 1.
    Use Memphis in cluster mode to enable parallel usage across multiple brokers
  2. 2.
    Spread the workloads across as many stations as possible (if possible) to spread leaders across different brokers
  3. 3.
    Use memory as the primary storage type
  4. 4.
    Use a low amount of replicas or none
  5. 5.
    Stretch the retention to days/hours
  6. 6.
    Make sure to utilize as many cores as possible within the app itself. For example, in node.js - use threads and cluster
  7. 7.
    Use affinity rules and separate the client K8s worker from the memphis workers
  8. 8.
    The most recommended K8s hardware would be arm-based CPUs with at least 2 cores (for example, AWS Graviton) and at least 8GB of memory. Again, not mandatory, but definitely improves performance

Memphis Metadata TLS connection configuration

Step 1: Generate certificates

openssl genrsa -des3 -passout pass:password -out ./client_tls.key
openssl rsa -in ./client_tls.key -passin pass:password -out ./client_tls.key
openssl req -new -key ./client_tls.key -out ./client_tls.csr -subj "/C=AU/ST=NSW/L=DY/O=MyOrg/OU=Dev/CN=postgres"
openssl x509 -req -in ./client_tls.csr -CA ./ca.crt -CAkey ./ca.key -out ./client_tls.crt -CAcreateserial

Step 2: Create a secret with the certificates

kubectl create secret generic tls-secret --from-file=client_tls.crt --from-file=client_tls.key --from-file=ca.crt -n memphis

Step 3: Create postgresql_tls.yaml values file

metadata:
pgpool:
tls:
enabled: true
autoGenerated: false
preferServerCiphers: true
certificatesSecret: "tls-secret"
certFilename: "client_tls.crt"
certKeyFilename: "client_tls.key"
certCAFilename: "ca.crt"

Step 4: Deploy Memphis with the new values file

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && helm install -f postgresql_values_tls.yaml memphis memphis/memphis --create-namespace --namespace memphis --wait
Search terms: "what are the best practices for running memphis in production?"