Apache Drill Adds YARN Support
Apache Drill Adds YARN Support
Written by Kay Ewbank   
Tuesday, 17 April 2018

The new release of Apache Drill has added the ability to run Drill as a YARN application, along with support for HTTP Kerberos authentication using SPNEGO, and SQL syntax highlighting of queries.

Apache Drill is a distributed SQL query engine that works with most non-relational datastores, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. Drill can be used by analysts, data scientists and developers to explore and analyze non-relational data without having to make the data more regimented for analysis. Drill processes the data in-situ without requiring users to define schemas or transform data.

drill2

 

Drill is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation and schema discovery during query execution. Drill was designed to provide good query performance with the flexibility offered by the JSON document model.

The main change to the new version is the ability to run Drill as a YARN application if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. When used like this, YARN handles the work of resource assignment to applications in the cluster so you don't have to deal with your own resource allocation. YARN automatically deploys the Drill software onto each Drill node and manages the Drill cluster. When used like this, Drill can become a long-running application, and you can monitor the Drill-on-YARN cluster using the Application Master web UI.

The next improvement is the addition of SPNEGO support.  SPENGO is Simple and Protected GSSAPI Negotiation Mechanism, and it is an alternative to Kerebos. You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.

SQL Syntax support has also been improved. The query syntax now appears highlighted in the Drill Web Console, and auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. If you're working in an appropriate browser such as Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. There's also an auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.

 

drill

 

More Information

Apache Drill

Drill On YARN

Related Articles

MapR Releases Docker Container For Local Development

Apache Drill Reaches 0.6

Perform Data Queries Faster With Drill

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

Banner


Google Launches Data Video Series
09/07/2018

The Google Analytics team has launched a video series called Measure Matters that looks at best practices on using Google Analytics with data. While some of the topics sound rather too 'marketeery', o [ ... ]



DevTube - A Curated Collection of Videos
26/06/2018

There's a new resource for developers. Dev.Tube is where you will find YouTube videos from conferences collected by developers for developers. The website already includes 10,000 videos - of whic [ ... ]


More News

justjsquare

 



 

Comments




or email your comment to: comments@i-programmer.info

 
 

   
Banner
Banner
RSS feed of news items only
I Programmer News
Copyright © 2018 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.