|Kafka Adds KRaft-Based Authorizer|
|Written by Kay Ewbank|
|Thursday, 19 May 2022|
Apache Kafka, the distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications, has been updated with improvements including a KRaft-based authorizer and a proposal for marking KRaft mode as production ready in Apache Kafka 3.3.
Kafka was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.
KRaft is Kafka Raft, the consensus protocol that has been added to remove Apache Kafka's dependency on ZooKeeper for metadata management. Until now, users making use of KRaft mode have still had to have an Apache ZooKeeper instance if they want to use AclAuthorizer. The new built-in Authorizer for KRaft mode doesn't depend on ZooKeeper, so avoids this.
This release also sees moves to using KRaft as production-ready. KRaft mode is popular as it addresses many urgent scalability and performance issues in Kafka, as well as eliminating the need to run an Apache ZooKeeper cluster alongside every Kafka cluster. KRaft entered preview status in Kafka 3.0, and this release opens the discussion on it becoming production-ready for new clusters in the upcoming Kafka 3.3 release. If this is agreed, ZooKeeper mode will be deprecated in Kafka 3.4 and removed entirely in Kafka 4.0.
Another improvement means the Kafka controller can send a hint to the partition leader to recover the partition in situations where the transaction state has been left inconsistent following an unclean election. Another change means that Kafka Streams can distribute its standby replicas over distinct “racks”.
Kafka 3.2 is available now.
or email your comment to: email@example.com
|Last Updated ( Thursday, 19 May 2022 )|