Getting Started
Deployment
Known Issues
Kubernetes
Deploy Memphis over Kubernetes using Helm
Helm is a k8s package manager that allows users to deploy apps in a single, configurable command.
Memphis is cloud-native and agnostic to any Kubernetes on any cloud. This means it can be deployed over production environments with Helm.
Step 1: Adding Memphis Helm Repo
The following command will add the Memphis repo to the user's repo list:
1
helm repo add memphis https://k8s.memphis.dev/charts/
Copied!
Running "help repo list" will show the user if Memphis has been added correctly.
1
$# helm repo list
2
NAME URL
3
descheduler https://kubernetes-sigs.github.io/descheduler/
4
loki https://grafana.github.io/loki/charts
5
ingress-nginx https://kubernetes.github.io/ingress-nginx
6
nginx-stable https://helm.nginx.com/stable
7
memphis https://k8s.memphis.dev/charts/
Copied!
To make sure the local version of Helm is fully updated, please run the following:
1
$# helm repo update
2
Hang tight while we grab the latest from your chart repositories...
3
...Successfully got an update from the "descheduler" chart repository
4
...Successfully got an update from the "memphis" chart repository
5
...Successfully got an update from the "nginx-stable" chart repository
6
...Successfully got an update from the "loki" chart repository
7
...Successfully got an update from the "grafana" chart repository
8
...Successfully got an update from the "yugabytedb" chart repository
9
Update Complete. ⎈Happy Helming!⎈
Copied!
Step 2: Install
Helm Install Options
Option
Description
Default Value
rootPwd
Root password for the dashboard
"memphis"connectionToken
Token for connecting an app to the Memphis Message Queue. Auto generated
""dashboard.port
Dashboard's (GUI) port
80
cluster.enabled
Cluster mode for HA and Performance
"false"
analytics
Collection of anonymous metadata
"true"
1
helm install <name of the deployment> memphis/memphis --create-namespace --namespace <name os the ns>
Copied!
An example of a command with real values:
1
helm install my-memphis memphis/memphis --create-namespace --namespace memphis
Copied!
An example with configured options:
1
helm install my-memphis --set cluster.replicas=1,rootPwd="rootpassword" memphis/memphis --create-namespace --namespace memphis
Copied!
1
NAME: my-memphis
2
LAST DEPLOYED: Thu May 5 17:05:40 2022
3
NAMESPACE: memphis2
4
STATUS: deployed
5
REVISION: 1
6
NOTES:
7
__ __ _ _
8
| \/ | ___ _ __ ___ _ __ | |__ (_)___
9
| |\/| |/ _ \ '_ ` _ \| '_ \| '_ \| / __|
10
| | | | __/ | | | | | |_) | | | | \__ \
11
|_| |_|\___|_| |_| |_| .__/|_| |_|_|___/
12
|_|
13
​
14
Melvis thank you for installing memphis!
15
A dev first event-processing platform.
16
​
17
---------------------------------------------------------------------------------------------------------------------------------------------
18
Memphis UI can be accessed on the following DNS name from within your cluster: memphis-ui.memphis2.svc.cluster.local
19
To access Memphis from localhost, run the below commands:
20
1. kubectl port-forward service/memphis-ui 9000:80 --namespace memphis2 > /dev/null &
21
2. kubectl port-forward service/memphis-cluster 7766:7766 6666:6666 5555:5555 --namespace memphis2 > /dev/null &
22
​
23
Dashboard: http://localhost:9000
24
Memphis broker: localhost:5555 (Management Port) / 7766 (Data Port) / 6666 (TCP Port)
25
​
26
---------------------------------------------------------------------------------------------------------------------------------------------
27
Read more about networking options here: https://docs.memphis.dev/deployment/kubernetes
28
​
29
Website: https://memphis.dev
30
Documentations: https://docs.memphis.dev
31
​
32
Deployment Information
33
-------------------------
34
## Secrets ##
35
UI/CLI root username - root
36
UI/CLI root Password - kubectl get secret memphis-creds -n memphis2 -o jsonpath="{.data.ROOT_PASSWORD}" | base64 --decode
37
​
38
## Components ##
39
3 Brokers - MQ
40
UI - GUI Dashboard
41
MongoDB - Internal Database for management
Copied!
1
$# kubectl get pods -w
2
NAME READY STATUS RESTARTS AGE
3
k8s-busybox-5cdcbc967c-h9ns4 1/1 Running 0 7s
4
memphis-broker-0 0/3 ContainerCreating 0 7s
5
memphis-broker-1 0/3 ContainerCreating 0 7s
6
memphis-broker-2 0/3 ContainerCreating 0 7s
7
memphis-ui-845445595b-vm85k 0/1 CrashLoopBackOff 1 7s
8
mongodb-replica-0 0/1 ContainerCreating 0 7s
9
memphis-broker-1 2/3 Running 0 14s
10
memphis-broker-0 2/3 Running 0 15s
11
memphis-broker-2 2/3 Running 0 18s
12
mongodb-replica-0 1/1 Running 0 18s
13
mongodb-replica-1 0/1 Pending 0 0s
14
k8s-busybox-5cdcbc967c-h9ns4 0/1 Completed 0 20s
15
k8s-busybox-5cdcbc967c-h9ns4 1/1 Running 1 22s
16
memphis-ui-845445595b-vm85k 0/1 Error 2 23s
17
k8s-busybox-5cdcbc967c-h9ns4 0/1 Completed 1 23s
18
mongodb-replica-1 0/1 Pending 0 6s
19
mongodb-replica-1 0/1 ContainerCreating 0 6s
20
memphis-broker-0 2/3 Running 0 26s
21
memphis-broker-0 3/3 Running 0 26s
22
memphis-broker-1 2/3 Running 0 26s
23
memphis-broker-1 3/3 Running 0 26s
24
memphis-broker-2 2/3 Running 0 26s
25
memphis-broker-2 3/3 Running 0 26s
26
mongodb-replica-1 1/1 Running 0 15s
27
k8s-busybox-5cdcbc967c-h9ns4 0/1 CrashLoopBackOff 1 36s
28
memphis-ui-845445595b-vm85k 0/1 CrashLoopBackOff 2 37s
29
k8s-busybox-5cdcbc967c-h9ns4 1/1 Running 2 38s
30
k8s-busybox-5cdcbc967c-h9ns4 0/1 Completed 2 41s
Copied!
It will take a few minutes untill all pods are shown as running. We are working on improving the rolling process.
If the output did not save, or a regeneration is needed, please use the following command:
1
helm get notes my-memphis -n memphis
Copied!
Deployed Pods
- memphis-broker-0: Three (default) MQ workers (Jetstream- and Memphis-made replicas). Responsible for data ingestion and process, just like "Kafka's" brokers
- memphis-ui-xxx: UI. Responsible for delivering a graphical user interface for managing the cluster.
- mongodb-replica-0/1: MongoDB, for Memphis internal usage.
Step 3: Access via UI / CLI / SDK
UI
CLI
SDK
Expose the UI in a localhost environment using "port-forward":
1
$# kubectl port-forward service/memphis-ui 9000:80 --namespace memphis & >/dev/null
Copied!
1
http://localhost:9000
Copied!
​
​
Expose the UI in a production environment:
- Nodeport1$# kubectl patch svc memphis-ui --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'Copied!1$# kubectl describe svc memphis-ui | grep -i Endpoints2Endpoints: 192.150.201.138:80Copied!NodePort will expose the UI via one of the k8s workers' IPs.
​
- Load Balancer
1
$# kubectl patch svc memphis-ui --type='json' -p '[{"op":"replace","path":"/spec/type","value":"LoadBalancer"}]'
Copied!
1
$# kubectl describe svc memphis-ui | grep -i "LoadBalancer Ingress"
2
LoadBalancer Ingress: a2d0fd26a0d7941a29d444ac4d03acd3-1181102898.eu-central-1.elb.amazonaws.com
Copied!
​
- Ingress - Please use the following file:
1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
name: memphis-ui-ingress
5
namespace: memphis
6
annotations:
7
acme.cert-manager.io/http01-edit-in-place: "true"
8
kubernetes.io/ingress.class: nginx
9
cert-manager.io/cluster-issuer: letsencrypt-staging
10
spec:
11
tls:
12
- hosts:
13
- "demo.memphis.dev"
14
secretName: demo-tls
15
rules:
16
- host: "demo.memphis.dev"
17
http:
18
paths:
19
- path: /
20
pathType: Prefix
21
backend:
22
service:
23
name: memphis-ui
24
port:
25
number: 80
Copied!
For the entire CLI reference and how to install it, please head to the following page:
​
The CLI client connects to Memphis via the UI.
After installing the CLI client and exposing the UI through one of the methods listed under the UI tab, please run the following:
1
$# mem connect
2
​
3
Use command: mem connect --user <user> --password <password> --server <server>
4
​
5
Example: mem connect -u root -p memphis -s <host>:<management_port>
6
Connection configuration to Memphis
7
Usage: index connect <command> [options]
8
​
9
Connection to Memphis
10
​
11
Options:
12
-u, --user <user> User
13
-p, --password <password> Password
14
-s, --server <server> Memphis
15
-h, --help display help for command
Copied!
​
1
$# mem connect -u root -p memphis -s http://localhost:9000
2
Connected successfully to Memphis control plane.
Copied!
For more detailed information, please head to the SDKs section below.
​
Memphis Node.JS SDK can be used to demonstrate the required parameters.
1
await memphis.connect({
2
host: "<memphis-host>",
3
managementPort: <management-port>, // defaults to 5555
4
tcpPort: <tcp-port>, // defaults to 6666
5
dataPort: <data-port>, // defaults to 7766
6
username: "<username>", // (application type user)
7
connectionToken: "<broker-token>", // you will get it on application type user creation
8
reconnect: true, // defaults to false
9
maxReconnect: 10, // defaults to 10
10
reconnectIntervalMs: 1500, // defaults to 1500
11
timeoutMs: 1500 // defaults to 1500
12
});
Copied!
- host: Usually "memphis-cluster" or the UI URL.
- managementPort: Usually 5555. For management purposes.
- tcpPort: For enabling Memphis UI and CLI to display the consumers and producers.
- username: Usually "root". Head to the users' section via the UI or CLI to add more.
- connectionToken: Each app that produces and/or consumer data with Memphis uses token authentication. To display the initial one, please run the following:
1
kubectl get secret memphis-creds -n <namespace> -o jsonpath="{.data.CONNECTION_TOKEN}" | base64 --decode
Copied!
​
​
​
​
​
​
​
Last modified 10d ago