Apache Kylin Adds RDBMS Support
Written by Kay Ewbank   
Wednesday, 06 September 2017

 Apache Kylin 2.1 has been released with new features including support for RDBMS and the ability to edit metadata model's JSON directly on the web.

Kylin is an open source analytics solution. It was originally developed at eBay before becoming an Apache project. It has a SQL interface, and can be used to carry out OLAP multidimensional analysis on Hadoop supporting extremely large datasets.



The Kylin OLAP Engine is made up of a metadata engine, a query engine, a job engine and a storage engine. It also includes a REST Server to service client requests. The query engine is based on Apache Calcite.

Apache Kylin v2.0.0 introduced a new cubing engine based on Apache Spark that can be selected to replace the original MapReduce engine. By default, Kylin uses MapReduce Cube Engine built on the Hadoop MapReduce framework to aggregate source data. Initial tests showed that the Spark engine could cut the build time to 50% in most cases. Release 2 also added support for the snowflake data model and for TPC-H queries. 

Apache Kylin lets you query massive data sets in three steps:

  1. Identify a Star Schema on Hadoop.
  2. Build a Cube from the identified tables.
  3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.

Kylin currently offers integration capability with BI Tools inlcuding Tableau, PowerBI and Excel. Integration with Microstrategy is coming soon.

The new release is a major release after 2.0, with more than 100 bug fixes and enhancements.

The addition of support for RDBMS data sources means that data in formats such as Oracle, SQL Server and MySQL can be used within the analyses. Tools such as Apache Sqoop can also be used to export the data from RDBMS to HDFS to make it easier for Kylin to get the data and then build that into cubes.

The second improvement adds the ability to edit metadata JSON. In the previous version, when a model's metadata was broken, the only way to fix the metadata was to use bin/metastore.sh. This new release lets the administrator edit the model's JSON directly in the web.

Other improvements include better handling of subqueries, and the routing of unsupported queries back to source.



More Information

Kylin Home Page 

Related Articles

Apache Spark 2.0 Released 

Apache Kafka Adds New Streams API

Apache Spark With Structured Streaming


To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.



Let Oracle's Coding Assistant Do The Grunt Work

Oracle is getting into the coding assistants game. The aptly named "Oracle Code Assist" is going to be optimized for Java, SQL, and Oracle Cloud-based applications.

ScyllaDB 6 Adds Node Distribution Feature

ScyllaDB 6.0 has been released with two major features that change the way it works: a dynamic way to distribute data across nodes that significantly improves scalability; and support for strongly con [ ... ]

More News


kotlin book



or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 06 September 2017 )