Apache Kudu 1.9 Adds Location Awareness
Written by Kay Ewbank   
Tuesday, 26 March 2019

There's a new release of Apache Kudu which adds location awareness and support for Docker scripts. Kudu is a columnar storage engine that fills the gap between the Hadoop Distributed File System (HDFS) and the HBase NoSQL database. Kudu provides C++, Java and Python client APIs.

Apache Kudu was originally a Cloudera project that is now part of the Apache Hadoop ecosystem. Apache says it can be used to enable fast analytics on fast data. Kudu's advantage is that is combines support for fast inserts and updates with efficient columnar scans, meaning you can use it for multiple real-time analytic workloads across a single storage layer. Cloudera's Grant Henke says that for that reason, Kudu fits well into a data pipeline as the place to store real-time data that needs to be queryable immediately. Additionally, Kudu supports updating and deleting rows in real-time allowing support for late arriving data and data correction.

 

kudu

The main improvement to Kudu 1.9 is support for location awareness. Kudu’s ordinary re-replication methods are designed to make sure a cluster is available if a single node fails, but clusters can be vulnerable to correlated failures of multiple nodes. For example, all of the physical hosts on the same rack in a datacenter may become unavailable simultaneously if the top-of-rack switch fails. The new location or rack awareness feature is designed to avoid this. When configured, Kudu will make a best effort to avoid placing a majority of replicas for a given tablet at the same location.

The next improvement is the addition of Docker scripts for building and running Kudu on various operating systems.

Testing using Java has also been improved; developers integrating with Kudu can now write Java tests that start a Kudu mini cluster without having to first locally build and install Kudu. This has been made possible by the Kudu team providing platform-specific binaries available to Gradle or Maven for download and install at test time.

 

kudu

More Information

Kudu Website

Related Articles

Apache Kudu Improves Web Interface  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Microsoft Announces Fabric Data Analytics
01/06/2023

Microsoft has announced Fabric, a new data analytics product that combines elements of Microsoft Power BI, Azure Synapse, and Azure Data Factory into one unified software as a service (SaaS) platform. [ ... ]



GPT For Regex - Pros And Cons
24/05/2023

RegExGPT is a online playground that lets you enter a source  and a target string to let GPT generate the regular expression for a match. But what are the pros and cons of the GPT approach t [ ... ]


More News

picobook

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 26 March 2019 )