Kafka 2.5 Adds New Metrics And Improves Security
Written by Kay Ewbank   
Thursday, 23 April 2020

Apache Kafka, the big data and data streams platform, has been updated in a new version that adds new metrics and improves security.


Apache Kafka is a distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications. It was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

Work has continued in this released to the removal of direct access to Apache ZooKeeper from Kafka's admin tools. This will improve security, decouple the server-side metadata format from the client, and is a necessary first step towards storing Kafka metadata in Kafka.

The work carried out on this means that dynamic configs will no longer require ZooKeeper access. The new release also ships with a more up-to-date version of ZooKeeper that adds ZooKeeper TLS support. This is used when deploying a secure Kafka cluster, remove direct ZooKeeper access from the Kafka Administrative tools.

Another noteworthy change simplifies the API for applications that read from and write to Kafka transactionally. Until now, you had to use separate producer instances for each input partition, but that's no longer the case, so it's easier to build EOS applications that consume large numbers of partitions. The developers say this is foundational for a similar improvement in Kafka Streams in the next release.

Reliability has been improved for the idempotent/transactional producer, overcoming a problem with producer state retention on the broker. Previously, when the log was truncated to enforce retention or truncated from a call to delete records, the broker dropped producer state, which led to errors. With this improvement, the broker instead retains producer state until expiration.


More Information

Kafka Website

Related Articles

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams

Kafka Webview Released

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


Apache Drill 1.19 Milestone Release Adds Cassandra Connector

Apache Drill has been updated in what the developers are calling its biggest release ever. Version 1.19 adds new connectors for Apache Cassandra, Elasticsearch, and Splunk, along with Avro support for [ ... ]

Microsoft Open Sources Java Garbage Collection Analyzer

Microsoft has made a collection of libraries for analyzing HotSpot Java garbage collection (GC) log files available in an open source form. GCToolkit parses log files into discrete events and provides [ ... ]

More News





or email your comment to: comments@i-programmer.info