Apache Kylin Adds RDBMS Support
Written by Kay Ewbank   
Wednesday, 06 September 2017

 Apache Kylin 2.1 has been released with new features including support for RDBMS and the ability to edit metadata model's JSON directly on the web.

Kylin is an open source analytics solution. It was originally developed at eBay before becoming an Apache project. It has a SQL interface, and can be used to carry out OLAP multidimensional analysis on Hadoop supporting extremely large datasets.


kylin

 

The Kylin OLAP Engine is made up of a metadata engine, a query engine, a job engine and a storage engine. It also includes a REST Server to service client requests. The query engine is based on Apache Calcite.

Apache Kylin v2.0.0 introduced a new cubing engine based on Apache Spark that can be selected to replace the original MapReduce engine. By default, Kylin uses MapReduce Cube Engine built on the Hadoop MapReduce framework to aggregate source data. Initial tests showed that the Spark engine could cut the build time to 50% in most cases. Release 2 also added support for the snowflake data model and for TPC-H queries. 

Apache Kylin lets you query massive data sets in three steps:

  1. Identify a Star Schema on Hadoop.
  2. Build a Cube from the identified tables.
  3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.

Kylin currently offers integration capability with BI Tools inlcuding Tableau, PowerBI and Excel. Integration with Microstrategy is coming soon.

The new release is a major release after 2.0, with more than 100 bug fixes and enhancements.

The addition of support for RDBMS data sources means that data in formats such as Oracle, SQL Server and MySQL can be used within the analyses. Tools such as Apache Sqoop can also be used to export the data from RDBMS to HDFS to make it easier for Kylin to get the data and then build that into cubes.

The second improvement adds the ability to edit metadata JSON. In the previous version, when a model's metadata was broken, the only way to fix the metadata was to use bin/metastore.sh. This new release lets the administrator edit the model's JSON directly in the web.

Other improvements include better handling of subqueries, and the routing of unsupported queries back to source.

 

kylin

More Information

Kylin Home Page 

Related Articles

Apache Spark 2.0 Released 

Apache Kafka Adds New Streams API

Apache Spark With Structured Streaming

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


Apache Shiro 2.0 Released
21/03/2024

Apache Shiro 2.0 has been released. The Java security framework now requires at least Java 11, and has added support for Jakarta EE 10.



Call For Code 2024 Focuses On Generative AI
01/03/2024

This year's Call for Code challenge has been launched with a theme of the use of generative AI technology for solutions that aim to improve equitable access to resources and opportunities for historic [ ... ]


More News

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 06 September 2017 )