Kafka 1 Becomes More Tolerant
Kafka 1 Becomes More Tolerant
Written by Kay Ewbank   
Tuesday, 07 November 2017

 Apache Kafka is now available in version 1.0 as it is now stable enough for business use. The new release of the distributed streaming engine is more tolerant of disk failures, supports Java 9, and has better metrics.


kakfalogo

The Apache Kafka distributed data streaming engine has finally hit version 1.0, a milestone that indicates the platform is finally ready for mainstream business. Kafka is a message broker that was originally developed at LinkedIn. It was then donated to Apache and made open source. Its benefits are that it is lightweight, scalable, and fast. Unlike traditional message brokers that just handle the interchange of messages and data streams between languages, Kafka provides low-latency, high-throughput, fault-tolerant publish and subscribe pipelines and is able to process streams of events. It can publish and subscribe to streams of data like a messaging system, process streams of data efficiently and in real time and store streams of data safely in a distributed, replicated cluster. 

The improvements to the new version include performance improvements with exactly-once semantics, significantly faster TLS and CRC32C implementations with Java 9 support, significantly faster controlled shutdown, and better JBOD support.

The achievement of exactly-once semantics means that if a problem occurs somewhere in the distributed system, perhaps a broker crashing or a network problem, and the producer retries sending the message, that message will be delivered exactly once to the end consumer. This is the ideal outcome, but is difficult to achieve because it requires the messaging system itself and the application producing and consuming the messages to cooperate.

Another improvement in the new version is to the Streams API. Kafka Streams is a client library for building applications and microservices, where the input and output data are stored in Kafka clusters. The builder API for Streams has been improved to hide internal methods; and a new API has been added to expose the state of active tasks at runtime. There's also a new cogroup API that can be used to deal with partitioned aggregates with fewer StateStores and fewer moving parts in your code.

Another improvement is the ability of Kafka to tolerate disk failures better. Until now, JBOD (Just a Bunch Of Disks) storage configurations have not been recommended, so requiring RAID instead. With this release, Kafka will handle disk failures better so that a single disk failure in a JBOD broker will not bring the entire broker down. Instead, the broker will continue serving any log files that remain on functioning disks.
 

 kakfalogo

 

More Information

Kafka Website

Related Articles

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

GoKa Stream Processing For Kafka

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

Banner


Cloud Foundry Training and Certification
02/11/2017

A free introductory course on the edX platform, which has just re-opened, can be the initial step on the route to Cloud Foundry certification. And even if you don't want to get certified it will help  [ ... ]



Facebook Open Sources RacerD For Android Race Detection
23/10/2017

It is a hardly discussed problem, but Android is a tough platform to use with concurrent programming. As a result most apps don't use it to its full potential. Facebook has now gifted you a tool that  [ ... ]


More News

 

 
 

 

blog comments powered by Disqus

 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2017 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.