Spark Announcements
Written by Kay Ewbank   
Tuesday, 23 February 2016

Details of the next version of Apache Spark, a free community edition of the Databricks cloud-based big data platform and Dashboards, a reporting front end, have all been announced.

The announcements were made at the recent Spark Summit East by Matei Zaharia, Databricks CTO and Spark creator.

Apache Spark is an open source data processing engine, and Zaharia said in the conference keynote that the next major version, Spark 2.0, is coming in April or May this year. The new version will see speed improvements of five to ten times; a new structured streaming real-time engine on SQL/dataframes; and unification of datasets and dataframes.

The new streaming engine is built on the Spark SQL engine, and also supports interactive and batch queries that aggregate data in a stream, then serve using JDBC. 

spark2

 

Alongside the announcement of Spark 2, Databricks has also announced the beta release of Databricks Community Edition, a free version of the cloud-based big data platform. Databricks was the founder of Apache Spark, and the largest contributor to Spark development. This service will provide users with access to a micro-cluster as well as a cluster manager and notebook environment. The idea is that developers can use the environment to learn Spark without the need to set up and run their own cluster environment.The initial beta rollout is invitation only. Wider access is planned over the next few months, with general availability planned for late Q2 2016.

 

databricks

The third announcement is Databricks Dashboards, a visual reporting application for Apache Spark clusters that can be used to provide reports and interative queries. Dashboards is actually an alternative view of a Databrick notebook aimed at end users who want to see different views of their data. Once a dashboard has been built, it can be shared with other users via its URL. The dashboard can be created with drop-down menus that can be used to choose or input parameters to alter the data being retrieved. The users needn't have any Spark knowledge, not any access to critical code. The dashboards can be updated automatically as underlying data changes.  sparklogo

More Information

Spark Site

Spark Summit East

Databricks

Related Articles

Apache Releases Spark 1.6

Spark 1.4 Released

MOOC On Apache Spark 

Learning Spark (book review) 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter,subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin

 

Banner

 


Databricks Delta Adds Faster Parquet Import
05/03/2019

There's an updated version of Databricks Delta that improves the speed that Parquet data can be imported and has stronger merge features. The analytics engine has also been made available on Amazon AW [ ... ]



GitHub Bounty Program Increases Rewards
20/02/2019

GitHub's Security Bug Bounty Program is now five years old and has been updated again with better rewards and a wider remit. Now a Microsoft-owned company, GitHub has also added Legal Safe Harbor [ ... ]


More News

 

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 23 February 2016 )