Kafka 2.5 Adds New Metrics And Improves Security
Written by Kay Ewbank   
Thursday, 23 April 2020

Apache Kafka, the big data and data streams platform, has been updated in a new version that adds new metrics and improves security.


kakfalogo

Apache Kafka is a distributed streaming platform that can be used for building real-time streaming data pipelines between systems or applications. It was originally developed at LinkedIn, from where it was taken on as an Apache project. It is a fast, scalable, durable, and fault-tolerant publish-subscribe messaging system that can be used in place of traditional message brokers.

Work has continued in this released to the removal of direct access to Apache ZooKeeper from Kafka's admin tools. This will improve security, decouple the server-side metadata format from the client, and is a necessary first step towards storing Kafka metadata in Kafka.

The work carried out on this means that dynamic configs will no longer require ZooKeeper access. The new release also ships with a more up-to-date version of ZooKeeper that adds ZooKeeper TLS support. This is used when deploying a secure Kafka cluster, remove direct ZooKeeper access from the Kafka Administrative tools.

Another noteworthy change simplifies the API for applications that read from and write to Kafka transactionally. Until now, you had to use separate producer instances for each input partition, but that's no longer the case, so it's easier to build EOS applications that consume large numbers of partitions. The developers say this is foundational for a similar improvement in Kafka Streams in the next release.

Reliability has been improved for the idempotent/transactional producer, overcoming a problem with producer state retention on the broker. Previously, when the log was truncated to enforce retention or truncated from a call to delete records, the broker dropped producer state, which led to errors. With this improvement, the broker instead retains producer state until expiration.


kakfalogo

More Information

Kafka Website

Related Articles

Kafka 2 Adds Support For ACLs

Kafka Graphs Framework Extends Kafka Streams

Kafka Webview Released

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.

Banner


Hasura GraphQL Adds REST Support
02/03/2021

The open source Hasura GraphQL Engine has been updated with improvements including support for using REST and a GraphQL API gateway.



Amazon Amplify Adds Flutter And Vue Support
01/03/2021

Amazon Amplify, the open source library for interacting with cloud services that use JavaScript applications, now has support for Flutter and Vue 3.


More News

square

 



 

Comments




or email your comment to: comments@i-programmer.info