Kafka 3.0 Released With KRaft
Written by Kay Ewbank   
Friday, 24 September 2021

Apache has released Kafka 3.0 with new features, breaking API changes, and improvements to KRaft—Apache Kafka’s built-in consensus mechanism that will replace Apache ZooKeeper.

Apache Kafka is a distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications.


It was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

One major improvement to Apache 3.0 is the more mature version of KRaft, the internal Raft implementation that allows you to run Kafka brokers without Apache ZooKeeper. KRaft is not yet recommended for production, but the team says they have made many improvements to the KRaft metadata and APIs. A new major feature in 3.0 is the ability for KRaft Controllers and KRaft Brokers to generate, replicate, and load snapshots for the metadata topic partition that is used by the Kafka Cluster to store and replicate metadata information about the cluster like Broker configuration, topic partition assignment, and leadership,

Kafka Connect is another area to have been improved. Connect is a Kafka component that acts as a data hub for data integration between databases, key-value stores, search indexes, and file systems. The new version handles task restarts more smoothly. Until now, if you wanted to restart the whole connector, you had to make individual calls to restart the Connector instance and the Task instances. In 3.0, KIP-745 gives the ability to the users to restart either all or only the failed of a connector’s Connector and Task instances with a single call.

Kafka Streams has improvements in timestamp-based synchronization. The way Streams tasks fetch records has been enhanced, and this is supported by a new method in the Kafka Consumer API to deal locally with the consumer lag of a specific partition.

Another change of note is to MirrorMaker.This is a stand-alone tool for copying data between two Apache Kafka clusters, and it now has more flexible configuration options.

Kafka 3 is available now. 



More Information

Kafka Website

Related Articles

Kafka Replaces Zookeeper With Quorum

Apache Kafka 2.7 Updates Broker

Kafka 2.5 Adds New Metrics And Improves Security

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams



To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


Quantum Computing Prize Awarded

John Preskill, Professor of Theoretical Physics at the California Institute of Technology, is the eighth recipient of the John Stewart Bell Prize for Research on Fundamental Issues in Quantu [ ... ]

Rust Twice As Productive As C++

Google director of engineering, Lars Bergstrom, gave a talk at the recent Rust Nation UK conference and claimed that Rust was twice as productive as C++. Given how good Google is at C++, this is quite [ ... ]

More News

raspberry pi books



or email your comment to: comments@i-programmer.info