LogoLogo
CloudDiscordGitHub
  • 👉Getting Started
    • Introduction
    • Quick start
    • Learn by example
    • Case studies
    • How to contribute?
  • ⭐Memphis Broker
    • Architecture
    • Key concepts
      • Message broker
      • Station
      • Producer API
      • Consumer API
      • Consumer Group
      • Storage and Redundancy
      • Security/Authentication
      • Scaling
      • Ordering
      • Dead-letter Station (DLS)
      • Delayed messages
      • Data exchange
      • Idempotency (Duplicate processing)
      • Failover Scenarios
      • Troubleshooting process
      • Connectors
    • Best practices
      • Producer optimization
      • Compression
    • Memphis configuration
    • Comparisons
      • NATS Jetstream vs Memphis
      • RabbitMQ vs Memphis
      • AWS SQS vs Memphis
      • Apache Kafka vs Memphis
      • Apache Pulsar vs Memphis
      • ZeroMQ vs Memphis
      • Apache NiFi vs Memphis
    • Privacy Policy
  • ⭐Memphis Schemaverse
    • Overview
    • Getting started
      • Management
      • Produce/Consume
        • Protobuf
        • JSON Schema
        • GraphQL
        • Avro
    • Comparison
    • KB
  • 📦Open-Source Installation
    • Kubernetes
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
      • Terraform
        • Deploy on AWS
        • Deploy on GCP
        • Deploy on DigitalOcean
      • Guides
        • Deploy/Upgrade Memphis utilizing predefined secrets
        • Monitoring/Alerts Recommendations
        • Production Best Practices
        • NGINX Ingress Controller and Cloud-Agnostic Memphis Deployments
        • Migrate Memphis storage between storageClass's
        • Expanding Memphis Disk Storage
        • Scale-out Memphis cluster
        • TLS - Deploy Memphis with TLS Connection to Metadata Frontend
        • TLS - Memphis TLS websocket configuration
        • TLS - Securing Memphis Client with TLS
        • Installing Memphis with an External Metadata Database
    • Docker
      • 1 - Installation
      • 2 - Access
      • 3 - Upgrade
    • Open-source Support
  • Client Libraries
    • REST (Webhook)
    • Node.js / TypeScript / NestJS
    • Go
    • Python
    • Kotlin (Community)
    • .NET
    • Java
    • Rust (Community)
    • NATS
    • Scala
  • 🔌Integrations Center
    • Index
    • Processing
      • Zapier
    • Change data Capture (CDC)
      • Debezium
    • Monitoring
      • Datadog
      • Grafana
    • Notifications
      • Slack
    • Storage tiering
      • S3-Compatible Object Storage
    • Source code
      • GitHub
    • Other platforms
      • Argo
  • 🗒️Release notes
    • KB
    • Releases
      • v1.4.3 - latest/stable
      • v1.4.2
      • v1.4.1
      • v1.4.0
      • v1.3.1
      • v1.3.0
      • v1.2.0
      • v1.1.1
      • v1.1.0
      • v1.0.3
      • v1.0.2
      • v1.0.1
      • V1.0.0 - GA
      • v0.4.5 - beta
      • v0.4.4 - beta
      • v0.4.3 - beta
      • v0.4.2 - beta
      • v0.4.1 - beta
      • v0.4.0 - beta
      • v0.3.6 - beta
      • v0.3.5 - beta
      • v0.3.0 - beta
      • v0.2.2 - beta
      • v0.2.1 - beta
      • v0.2.0 - beta
      • v0.1.0 - beta
Powered by GitBook
LogoLogo

Legal

  • Terms of Service
  • Privacy Policy

All rights reserved to Memphis.dev 2023

On this page
  • What is a producer?
  • Broker's Data Format
  • Performance Optimizations
  • 1. Non-blocking producer
  • 2. Partitions
  • Produce messages to multiple stations simultaneously
  • Supported Protocols

Was this helpful?

  1. Memphis Broker
  2. Key concepts

Producer API

This section describes Memphis producer API

Last updated 1 year ago

Was this helpful?

What is a producer?

A producer represents the originating application or service responsible for sending data or messages to the broker or, more specifically, to a station. When a client establishes a connection with Memphis, it consists of various components:

  1. Connection - This involves an open socket linking the client to Memphis.

  2. Producer - To write data or messages into a Memphis station, you need to declare a producer entity.

  3. (And/Or) Consumer - To retrieve data or messages from a Memphis station, a consumer entity declaration is required.

Broker's Data Format

Memphis employs binary data encoding for reading, storing, and writing data to enhance performance, ensure format alignment, and optimize memory allocation. When a producer generates a message for a Memphis station, it undergoes conversion into a binary format.

An example from the node.js SDK using Buffer.from -

await producer.produce({
  message: Buffer.from("Message: Hello world"),
});

Nonexistent stations will be automatically generated by the SDK upon the initial connection of a producer or consumer.

Full API guide and code examples can be found here Client Libraries

Performance Optimizations

1. Non-blocking producer

Opting for a non-blocking producer type ensures that the client won't halt the subsequent system call or message production while waiting for an acknowledgment from the broker regarding the message transmission. This choice empowers the client's CPU to harness its resources for increased parallelism, effectively eliminating periods of idle waiting.

await producer.produce({
    message: 'test'
    asyncProduce: true // true for non-blocking, false for blocking
});

2. Partitions

Memphis incorporates a partitioning system, which is fundamental to enhancing concurrent operations within its stations. This approach involves dividing the active station into multiple partitions or segments distributed across one or more Memphis brokers. This partitioning strategy optimizes the processing and management of data, ensuring efficient and scalable handling of information flow.

Produce messages to multiple stations simultaneously

When specifying a producer, you have the option to send a message simultaneously to multiple stations.

stationName: ['<station-name>', '<station-name>']

Supported Protocols

A full guide can be found

The parameter is called asyncProduce and can be found in any supported client library or through the following example -

Read to learn more and how to use it.

An example from the :

here
here
Node.js SDK
TCP-based SDKs
HTTP
⭐
here
Memphis partitions
Page cover image