Apache Drill Adds YARN Support
Written by Kay Ewbank   
Tuesday, 17 April 2018

The new release of Apache Drill has added the ability to run Drill as a YARN application, along with support for HTTP Kerberos authentication using SPNEGO, and SQL syntax highlighting of queries.

Apache Drill is a distributed SQL query engine that works with most non-relational datastores, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. Drill can be used by analysts, data scientists and developers to explore and analyze non-relational data without having to make the data more regimented for analysis. Drill processes the data in-situ without requiring users to define schemas or transform data.

drill2

 

Drill is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation and schema discovery during query execution. Drill was designed to provide good query performance with the flexibility offered by the JSON document model.

The main change to the new version is the ability to run Drill as a YARN application if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. When used like this, YARN handles the work of resource assignment to applications in the cluster so you don't have to deal with your own resource allocation. YARN automatically deploys the Drill software onto each Drill node and manages the Drill cluster. When used like this, Drill can become a long-running application, and you can monitor the Drill-on-YARN cluster using the Application Master web UI.

The next improvement is the addition of SPNEGO support.  SPENGO is Simple and Protected GSSAPI Negotiation Mechanism, and it is an alternative to Kerebos. You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.

SQL Syntax support has also been improved. The query syntax now appears highlighted in the Drill Web Console, and auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. If you're working in an appropriate browser such as Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. There's also an auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.

 

drill

 

More Information

Apache Drill

Drill On YARN

Related Articles

MapR Releases Docker Container For Local Development

Apache Drill Reaches 0.6

Perform Data Queries Faster With Drill

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Couchbase's Coding Assistant Goes GA
11/03/2024

Capella iQ, the AI coding assistant for developers that makes interacting with Couchbase using natural language possible, has gone from private beta to being generally available.



TornadoInsight Brings The Power Of TornadoVM Inside Intellij
22/02/2024

TornadoInsight is plugin for Jetbrain's popular Intellij IDE for Java developers, that makes working with TornadoVM a much smoother experience.


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info