Kafka 3.1 Adds OIDC Support
Written by Kay Ewbank   
Monday, 31 January 2022

Apache Kafka, the distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications, has been updated. Improvements include extending SASL/OAUTHBEARER to add support for Open ID Connect (OIDC).



Kafka was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

The new support for Open ID Connect (OIDC) aims to extend earlier improvements. The OAuth Authentication via SASL/OAUTHBEARER addition introduced a framework that allowed for integration with OAuth-compliant providers. With this framework in place, Kafka clients can pass a JWT access token to a broker when initializing the connection as a means of authentication, so Kafka can use these standards for authorization and authentication. The improvement provides a concrete implementation of the interfaces to allow Kafka to connect to an Open ID identity provider for authentication and token retrieval.

Elsewhere, this release adds support for infinite endpoints for range queries in Kafka Streams KV stores. This extends the semantics of the existing range and reverseRange interfaces in the ReadOnlyKeyValueStore to support unbounded ranges. Specifically, the interfaces now support the use of null values as a way to represent unbounded ranges.

The new version also introduces two new metrics that are exposed by both the ZooKeeper and KRaft controller. ActiveBrokerCount and FencedBrokerCount expose the number of active brokers in the cluster known by the controller and the number of fenced brokers known by the controller.

RocksDB support has been improved in two areas. Firstly, there's no longer any need to set WAL-related configs for RocksDB instances within Streams as Streams disables the write-ahead log (WAL) provided by RocksDB since it replicates the data in changelog topics. Secondly, the RocksDBStoreTest and RocksDBKeyValueStoreTest classes have been consolidated.

Support has also been added for Java 17.

Kafka 3.1 is available now. 


More Information

Kafka Website

Related Articles

Kafka 3.0 Released With KRaft  

Kafka Replaces Zookeeper With Quorum

Apache Kafka 2.7 Updates Broker

Kafka 2.5 Adds New Metrics And Improves Security

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams



To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


JetBrains Launches IDE Services

JetBrains has launched a new product suite for enterprises. JetBrains IDE Services is designed for use by large organizations with the aim of boosting developer productivity at scale.

Eclipse JKube 1.16 Goes GA

Eclipse JKube makes deploying your Java application to a Kubernetes cluster a breeze. Let's find out what's new.

More News

raspberry pi books



or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 01 February 2022 )