Hortonworks Plans To Take Hadoop Cloud Native
Written by Kay Ewbank   
Wednesday, 19 September 2018

Hortonworks has announced an initiative with IBM and Red Hat in the drive to make Hadoop cloud-native and able to run well in hybrid environments.

While the current versions of Hadoop can be used in cloud environments, Hadoop itself largely ignores the fact it's being used in the cloud. Despite this, it is increasingly deployed in the cloud, hence the new announcement. Hortonworks has also released a roadmap of when Hadoop will be ready to run in hybrid environments.

Hortonworks says that it has been working towards the goal of hybrid running, which according to them requires:

  • Cloud-native Hadoop for public cloud – delivered with Hortonworks Data Platform (HDP) and Hortonworks DataFlow (HDF) on IaaS.
  • Data flow and management to and from the edge – delivered with HDF, and specifically with MiNiFi.
  • Consistent security and data governance across all tiers – delivered with DPS.
  • A consistent architecture in the cloud and on-premises. This is the last mile.

DPS (Data Plane Services) is a catalog of services that run as plug-ins for tasks such as lifecycle management, replication management, access control, and data flow management. 

The initiative with IBM and Red Hat is aimed at creating the consistent architecture no matter where Hadoop is running. This requires storage to be decoupled from the computing environment, and the use of containerized computing resources to ensure software isolation. Services should be shared across all tiers to help with governance and security, and tools provided for managing services and workloads to allow spin-up/down programmatically. The final element of this is to be able to designate workloads that are specific to particular uses such as as EDW or data science rather than sharing everything in a multi-tenant Hadoop cluster.

As the initial phase of the initiative, Hortonworks, Red Hat and IBM will work together to optimize Hortonworks Data Platform, Hortonworks DataFlow, Hortonworks DataPlane and IBM Cloud Private for Data for use on Red Hat OpenShift, an enterprise container and Kubernetes application platform. This should make it possible to develop and deploy containerized big data workloads. IBM and Hortonworks will continue working to integrate services offered through Hortonworks DataPlane with IBM Cloud Private for Data.

hortonworks

 

More Information

Hortonworks

Related Articles

Hadoop 3 Adds HDFS Erasure Coding

Hadoop 2.9 Adds Resource Estimator

Hadoop Adds In-Memory Caching

Hadoop SQL Query Engine Launched

Hadoop 2 Introduces YARN

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

Banner


AI System Shows Food Image Based On Recipe
19/01/2019

Researchers from Tel Aviv University have developed a deep learning-based system that can automatically generate pictures of a finished meal from a simple text-based recipe.



Crowd Sourcing And GPUs Bring 3Km Weather Forecasts
12/01/2019

Weather forecasts are always wrong, but with more data they could be right. Not only more accurate, but also more specific. How to get the data and how to process it? The Weather Company might have an [ ... ]


More News

Python

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 19 September 2018 )