Producer API

This section describes Memphis producer API

What is a producer?

A producer represents the originating application or service responsible for sending data or messages to the broker or, more specifically, to a station. When a client establishes a connection with Memphis, it consists of various components:

1. Connection - This involves an open socket linking the client to Memphis.

2. Producer - To write data or messages into a Memphis station, you need to declare a producer entity.

3. (And/Or) Consumer - To retrieve data or messages from a Memphis station, a consumer entity declaration is required.

Broker's Data Format

Memphis employs binary data encoding for reading, storing, and writing data to enhance performance, ensure format alignment, and optimize memory allocation. When a producer generates a message for a Memphis station, it undergoes conversion into a binary format.

An example from the node.js SDK using Buffer.from -

await producer.produce({
  message: Buffer.from("Message: Hello world"),
});

Nonexistent stations will be automatically generated by the SDK upon the initial connection of a producer or consumer.

Full API guide and code examples can be found here Client Libraries

Performance Optimizations

A full guide can be found here

1. Non-blocking producer

Opting for a non-blocking producer type ensures that the client won't halt the subsequent system call or message production while waiting for an acknowledgment from the broker regarding the message transmission. This choice empowers the client's CPU to harness its resources for increased parallelism, effectively eliminating periods of idle waiting.

The parameter is called asyncProduce and can be found in any supported client library here or through the following example -

await producer.produce({
    message: 'test'
    asyncProduce: true // true for non-blocking, false for blocking
});

2. Partitions

Memphis incorporates a partitioning system, which is fundamental to enhancing concurrent operations within its stations. This approach involves dividing the active station into multiple partitions or segments distributed across one or more Memphis brokers. This partitioning strategy optimizes the processing and management of data, ensuring efficient and scalable handling of information flow.

Memphis partitions

Read here to learn more and how to use it.

Produce messages to multiple stations simultaneously

When specifying a producer, you have the option to send a message simultaneously to multiple stations.

An example from the Node.js SDK:

stationName: ['<station-name>', '<station-name>']

Supported Protocols

• TCP-based SDKs

• HTTP