There's a new release of Apache Kudu which adds location awareness and support for Docker scripts. Kudu is a columnar storage engine that fills the gap between the Hadoop Distributed File System (HDFS) and the HBase NoSQL database. Kudu provides C++, Java and Python client APIs.

Apache Kudu was originally a Cloudera project that is now part of the Apache Hadoop ecosystem. Apache says it can be used to enable fast analytics on fast data. Kudu's advantage is that is combines support for fast inserts and updates with efficient columnar scans, meaning you can use it for multiple real-time analytic workloads across a single storage layer. Cloudera's Grant Henke says that for that reason, Kudu fits well into a data pipeline as the place to store real-time data that needs to be queryable immediately. Additionally, Kudu supports updating and deleting rows in real-time allowing support for late arriving data and data correction.

The main improvement to Kudu 1.9 is support for location awareness. Kudu’s ordinary re-replication methods are designed to make sure a cluster is available if a single node fails, but clusters can be vulnerable to correlated failures of multiple nodes. For example, all of the physical hosts on the same rack in a datacenter may become unavailable simultaneously if the top-of-rack switch fails. The new location or rack awareness feature is designed to avoid this. When configured, Kudu will make a best effort to avoid placing a majority of replicas for a given tablet at the same location.

The next improvement is the addition of Docker scripts for building and running Kudu on various operating systems.

Testing using Java has also been improved; developers integrating with Kudu can now write Java tests that start a Kudu mini cluster without having to first locally build and install Kudu. This has been made possible by the Kudu team providing platform-specific binaries available to Gradle or Maven for download and install at test time.

More Information

Kudu Website

Related Articles

Apache Kudu Improves Web Interface  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info