Kafka Adds KRaft-Based Authorizer
Written by Kay Ewbank   
Thursday, 19 May 2022

Apache Kafka, the distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications, has been updated with improvements including a KRaft-based authorizer and a proposal for marking KRaft mode as production ready in Apache Kafka 3.3.

Kafka was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

kafka

KRaft is Kafka Raft, the consensus protocol that has been added to remove Apache Kafka's dependency on ZooKeeper for metadata management. Until now, users making use of KRaft mode have still had to have an Apache ZooKeeper instance if they want to use AclAuthorizer. The new built-in Authorizer for KRaft mode doesn't depend on ZooKeeper, so avoids this.

This release also sees moves to using KRaft as production-ready. KRaft mode is popular as it addresses many urgent scalability and performance issues in Kafka, as well as eliminating the need to run an Apache ZooKeeper cluster alongside every Kafka cluster. KRaft entered preview status in Kafka 3.0, and this release opens the discussion on it becoming production-ready for new clusters in the upcoming Kafka 3.3 release. If this is agreed, ZooKeeper mode will be deprecated in Kafka 3.4 and removed entirely in Kafka 4.0.

Another improvement means the Kafka controller can send a hint to the partition leader to recover the partition in situations where the transaction state has been left inconsistent following an unclean election. Another change means that Kafka Streams can distribute its standby replicas over distinct “racks”.

Kafka 3.2 is available now. 

kafka

More Information

Kafka Website

Related Articles

Kafka 3.1 Adds OIDC Support

Kafka 3.0 Released With KRaft  

Kafka Replaces Zookeeper With Quorum

Apache Kafka 2.7 Updates Broker

Kafka 2.5 Adds New Metrics And Improves Security

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


What Is the Shift-Left Approach in DevOps?
03/06/2022

The short answer is that it is a methodology that brings software testing into earlier stages of the software development lifecycle. We look at the importance of this new approach and how to impl [ ... ]



Videos From Inaugural Computer History Conference
12/06/2022

The First International Research Conference on the History of Computing, dubbed Computing's Woodstock, gathered together a global elite of computer pioneers. It took place in June 1976 and now the Com [ ... ]


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 19 May 2022 )