Kafka 2.5 Adds New Metrics And Improves Security
Written by Kay Ewbank   
Thursday, 23 April 2020

Apache Kafka, the big data and data streams platform, has been updated in a new version that adds new metrics and improves security.


kakfalogo

Apache Kafka is a distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications. It was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

Work has continued in this released to the removal of direct access to Apache ZooKeeper from Kafka's admin tools. This will improve security, decouple the server-side metadata format from the client, and is a necessary first step towards storing Kafka metadata in Kafka.

The work carried out on this means that dynamic configs will no longer require ZooKeeper access. The new release also ships with a more up-to-date version of ZooKeeper that adds ZooKeeper TLS support. This is used when deploying a secure Kafka cluster, remove direct ZooKeeper access from the Kafka Administrative tools.

Another noteworthy change simplifies the API for applications that read from and write to Kafka transactionally. Until now, you had to use separate producer instances for each input partition, but that's no longer the case, so it's easier to build EOS applications that consume large numbers of partitions. The developers say this is foundational for a similar improvement in Kafka Streams in the next release.

Reliability has been improved for the idempotent/transactional producer, overcoming a problem with producer state retention on the broker. Previously, when the log was truncated to enforce retention or truncated from a call to delete records, the broker dropped producer state, which led to errors. With this improvement, the broker instead retains producer state until expiration.


kakfalogo

More Information

Kafka Website

Related Articles

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams

Kafka Webview Released

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Learn Cryptography Without The Math
09/07/2024

Are you sick of the math associated with cryptography?
You don't have to be any more. Applied Cryptography from the University of Tartu shows cryptography without the math! At last, a hands-o [ ... ]



MySQL 9 Adds Support For JavaScript Stored Procedures
11/07/2024

Oracle has announced that support for JavaScript functions and stored procedures has been added to the MySQL database server. The enhancements were added to MySQL 9, an 'innovation release', whic [ ... ]


More News

kotlin book

 

Comments




or email your comment to: comments@i-programmer.info