Introducing Memphis Cloud Message Broker
Written by Kay Ewbank   
Thursday, 29 June 2023

Memphis.dev has introduced a message broker designed to support the development of real-time applications. Memphis Cloud is based on the Memphis open source project with added security and features for stateless stream processing at scale.

memphislogo

Memphis has three main components; the Memphis broker which acts as the data storage layer. That is the component that stores and controls the ingested messages and their entire lifecycle management. The second element is a metadata store that is responsible for storing the platform metadata only, such as general information, monitoring, GUI state, and pointers to dead-letter messages. The metadata store uses Postgres. The third component is a REST gateway that is responsible for exposing Memphis management and data ingestion through REST requests.

Memphis.dev is developed from this. It is described as an intelligent, frictionless message broker that accelerates development of real-time applications. It can be used to build modern queue-based applications that require large volumes of streamed and enriched data. The resilient, distributed architecture is cloud-native and can run on any Kubernetes, on any cloud without zookeeper, bookeeper, or JVM.

The additions to Memphis Cloud include augmentation of Kafka clusters, built-in schema management, enforcement, and transformation; multi-tenancy for traffic isolation; use-based billing and true multi-cloud capabilities. The augmentation of Kafka clusters comes from the fact that while Kafka users can set up inter-cluster data flows with Kafka's MirrorMaker, Memphis cloud users can create more Memphis clusters and form a supercluster that replicates data in an async manner between the clusters. Memphis Cloud can also run alongside Kafka, ensuring messages are delivered in order and without any loss.

Memphis Cloud also includes a module called Schemaverse that is responsible for built-in schema management, enforcement, and transformation. Schemaverse provides a schema store and schema management layer on top of Memphis broker. It provides runtime-level rendering of existing producers and consumers, and the ability to modify schemas without rebooting producers. Schemaverse has a number of SDKs: including Go, Python, and Node.js (Nest / Typescript). Supported protocols include Protobuf, Avro, JSON and GraphQL.

Memphis says Memphis Cloud is a true multi-cloud product with the ability to create primary instances on GCP, and a replica on AWS, and it is also multi-tenancy for users of SaaS platforms who want to isolate traffic between their customers. The serverless nature of the platform is pay-as-you-use, with the ability to scale to trillions of events without any concern to the infrastructure. The developers say all of this means that Memphis Cloud is poised to disrupt the dominance of the combination of Kafka and Flink. 

Yaniv Ben Hemo, co-founder and CEO, Memphis, said: 

"The world is asynchronous and built out of events. Message brokers are the engine behind their flow in the modern software architecture, and the modern message broker should be much more intelligent with much less friction. Memphis.dev takes five minutes on average for a user to get to production and start building queue-based applications and distributed streaming pipelines."

Memphis Cloud is available now.

memphislogo

More Information

Memphis Dev Website

Memphis On GitHub

Related Articles

Kafka KRaft Now In Production

Apache Flink ML 2.0 Released

Apache Flink 1.9 Adds New Query Engine

Kafka Adds KRaft-Based Authorizer  

Kafka 3.1 Adds OIDC Support

Kafka 3.0 Released With KRaft  

Kafka Replaces Zookeeper With Quorum

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Researchers Use AI To Decode Dog Language
21/06/2024

Scientists from the University of Michigan have used AI to decode what dogs mean by different types of bark. Wav2Vec2 succeeded at four classification tasks - dog recognition, breed identification, ge [ ... ]



Ladybird - An Independent Web Browser
03/07/2024

Ladybird sets out to be an independent Web Browser, free of Google (or any) advertising. It has taken the step of becoming a non-profit project on receiving a $1 million donation from GitHub founder,  [ ... ]


More News

kotlin book

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 29 June 2023 )