Apache Drill Adds YARN Support
Written by Kay Ewbank   
Tuesday, 17 April 2018

The new release of Apache Drill has added the ability to run Drill as a YARN application, along with support for HTTP Kerberos authentication using SPNEGO, and SQL syntax highlighting of queries.

Apache Drill is a distributed SQL query engine that works with most non-relational datastores, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. Drill can be used by analysts, data scientists and developers to explore and analyze non-relational data without having to make the data more regimented for analysis. Drill processes the data in-situ without requiring users to define schemas or transform data.



Drill is the only columnar execution engine that supports complex and schema-free data, and the only execution engine that performs data-driven query compilation and schema discovery during query execution. Drill was designed to provide good query performance with the flexibility offered by the JSON document model.

The main change to the new version is the ability to run Drill as a YARN application if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. When used like this, YARN handles the work of resource assignment to applications in the cluster so you don't have to deal with your own resource allocation. YARN automatically deploys the Drill software onto each Drill node and manages the Drill cluster. When used like this, Drill can become a long-running application, and you can monitor the Drill-on-YARN cluster using the Application Master web UI.

The next improvement is the addition of SPNEGO support.  SPENGO is Simple and Protected GSSAPI Negotiation Mechanism, and it is an alternative to Kerebos. You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.

SQL Syntax support has also been improved. The query syntax now appears highlighted in the Drill Web Console, and auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. If you're working in an appropriate browser such as Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. There's also an auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.




More Information

Apache Drill

Drill On YARN

Related Articles

MapR Releases Docker Container For Local Development

Apache Drill Reaches 0.6

Perform Data Queries Faster With Drill

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


Carla Gomes Receives 2021 AAAI Allen Newell Award

Carla Gomes of  Cornell University has been named as the 2021 recipient of the ACM-AAAI Allen Newell Award for her work in Computational Sustainability and for foundational contribution [ ... ]

Apple Improves Developer Support

Apple has announced a range of new tools and technologies at its annual Worldwide Developer Conference (WWDC), including making Xcode Cloud open to all members of the Apple Developer Program.

More News





or email your comment to: comments@i-programmer.info