LogoLogo
CloudDiscordGitHub
  • 👉Getting Started
    • Introduction
    • Quick start
    • Learn by example
    • Case studies
    • How to contribute?
  • ⭐Memphis Broker
    • Architecture
    • Key concepts
      • Message broker
      • Station
      • Producer API
      • Consumer API
      • Consumer Group
      • Storage and Redundancy
      • Security/Authentication
      • Scaling
      • Ordering
      • Dead-letter Station (DLS)
      • Delayed messages
      • Data exchange
      • Idempotency (Duplicate processing)
      • Failover Scenarios
      • Troubleshooting process
      • Connectors
    • Best practices
      • Producer optimization
      • Compression
    • Memphis configuration
    • Comparisons
      • NATS Jetstream vs Memphis
      • RabbitMQ vs Memphis
      • AWS SQS vs Memphis
      • Apache Kafka vs Memphis
      • Apache Pulsar vs Memphis
      • ZeroMQ vs Memphis
      • Apache NiFi vs Memphis
    • Privacy Policy
  • ⭐Memphis Schemaverse
    • Overview
    • Getting started
      • Management
      • Produce/Consume
        • Protobuf
        • JSON Schema
        • GraphQL
        • Avro
    • Comparison
    • KB
  • 📦Open-Source Installation
    • Kubernetes
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
      • Terraform
        • Deploy on AWS
        • Deploy on GCP
        • Deploy on DigitalOcean
      • Guides
        • Deploy/Upgrade Memphis utilizing predefined secrets
        • Monitoring/Alerts Recommendations
        • Production Best Practices
        • NGINX Ingress Controller and Cloud-Agnostic Memphis Deployments
        • Migrate Memphis storage between storageClass's
        • Expanding Memphis Disk Storage
        • Scale-out Memphis cluster
        • TLS - Deploy Memphis with TLS Connection to Metadata Frontend
        • TLS - Memphis TLS websocket configuration
        • TLS - Securing Memphis Client with TLS
        • Installing Memphis with an External Metadata Database
    • Docker
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
    • Open-source Support
  • Client Libraries
    • REST (Webhook)
    • Node.js / TypeScript / NestJS
    • Go
    • Python
    • Kotlin (Community)
    • .NET
    • Java
    • Rust (Community)
    • NATS
    • Scala
  • 🔌Integrations Center
    • Index
    • Processing
      • Zapier
    • Change data Capture (CDC)
      • Debezium
    • Monitoring
      • Datadog
      • Grafana
    • Notifications
      • Slack
    • Storage tiering
      • S3-Compatible Object Storage
    • Source code
      • GitHub
    • Other platforms
      • Argo
  • 🗒️Release notes
    • KB
    • Releases
      • v1.4.3 - latest/stable
      • v1.4.2
      • v1.4.1
      • v1.4.0
      • v1.3.1
      • v1.3.0
      • v1.2.0
      • v1.1.1
      • v1.1.0
      • v1.0.3
      • v1.0.2
      • v1.0.1
      • V1.0.0 - GA
      • v0.4.5 - beta
      • v0.4.4 - beta
      • v0.4.3 - beta
      • v0.4.2 - beta
      • v0.4.1 - beta
      • v0.4.0 - beta
      • v0.3.6 - beta
      • v0.3.5 - beta
      • v0.3.0 - beta
      • v0.2.2 - beta
      • v0.2.1 - beta
      • v0.2.0 - beta
      • v0.1.0 - beta
Powered by GitBook
LogoLogo

Legal

  • Terms of Service
  • Privacy Policy

All rights reserved to Memphis.dev 2023

On this page

Was this helpful?

  1. Memphis Broker
  2. Key concepts

Data exchange

What is an exchange? What are routing keys? How are exchanges and stations associated with each other? When should I use them and how? This page explains it all

Last updated 1 year ago

Was this helpful?

Exchange is a well-familiar concept in RabbitMQ and refers to a mechanism where a message is sent to a kind of router instead of directly to some queue. This router then directs the message to a specific queue, depending on a defined policy or, more precisely, based on a routing key.

Exchange is an excellent tool for scenarios where each message needs to be streamed to a different destination based on certain conditions or payload.

Memphis provides the functionality to route specific messages to one or several stations, depending on predefined headers.

How it works

Step 1: Assign a dedicated header to each message to indicate the specific station it should be directed to:

  • This can be accomplished by implementing tagging at the producer level or through a tagging function, which adds tags and headers to each message according to a highly customizable policy.

Step 2: Set up a station-type sink:

  1. Create a station that will be designated as the "exchange" station

  2. Add a Memphis station sink

  1. Two routing strategies can be selected: Header-based or Station-name-based. The Header-based strategy involves dynamic routing for each message, determined by a specific header attached to the message. In this method, the receiving system (sink) interprets the header's value to identify the destination station and routes the message accordingly. The Station Name strategy involves routing all new messages directly to a predetermined station.

If the destination station does not exist, it will be automatically created

  1. Example: We implemented a station-type sink that utilizes a header-based routing strategy, keyed by "station."

const { memphis } = require("memphis-dev");

(async function () {
    let memphisConnection

    try {
        memphisConnection = await memphis.connect({
            host: 'aws-eu-central-1.cloud.memphis.dev',
            username: '*****',
            password: '******',
            accountId: 111222333
        });

        const producer = await memphisConnection.producer({
            stationName: 'exchange',
            producerName: 'demo-producer-1'
        });
        const headers = memphis.headers()
        headers.add("station", "destination1")

        await producer.produce({
            message: { "test": "payload" }
            asyncProduce: true,
            headers: headers
        });

        memphisConnection.close();
    } catch (ex) {
        console.log(ex);
        if (memphisConnection) memphisConnection.close();
    }
})();

The following code will produce a message to the station "exchange" which will reroute it to a station called "destination1"

Another enhancement involves employing a custom Memphis Function, which tags each message at the broker level using a unique logic tailored for every message.

⭐