Hortonworks Plans To Take Hadoop Cloud Native
Written by Kay Ewbank   
Wednesday, 19 September 2018

Hortonworks has announced an initiative with IBM and Red Hat in the drive to make Hadoop cloud-native and able to run well in hybrid environments.

While the current versions of Hadoop can be used in cloud environments, Hadoop itself largely ignores the fact it's being used in the cloud. Despite this, it is increasingly deployed in the cloud, hence the new announcement. Hortonworks has also released a roadmap of when Hadoop will be ready to run in hybrid environments.

Hortonworks says that it has been working towards the goal of hybrid running, which according to them requires:

  • Cloud-native Hadoop for public cloud – delivered with Hortonworks Data Platform (HDP) and Hortonworks DataFlow (HDF) on IaaS.
  • Data flow and management to and from the edge – delivered with HDF, and specifically with MiNiFi.
  • Consistent security and data governance across all tiers – delivered with DPS.
  • A consistent architecture in the cloud and on-premises. This is the last mile.

DPS (Data Plane Services) is a catalog of services that run as plug-ins for tasks such as lifecycle management, replication management, access control, and data flow management. 

The initiative with IBM and Red Hat is aimed at creating the consistent architecture no matter where Hadoop is running. This requires storage to be decoupled from the computing environment, and the use of containerized computing resources to ensure software isolation. Services should be shared across all tiers to help with governance and security, and tools provided for managing services and workloads to allow spin-up/down programmatically. The final element of this is to be able to designate workloads that are specific to particular uses such as as EDW or data science rather than sharing everything in a multi-tenant Hadoop cluster.

As the initial phase of the initiative, Hortonworks, Red Hat and IBM will work together to optimize Hortonworks Data Platform, Hortonworks DataFlow, Hortonworks DataPlane and IBM Cloud Private for Data for use on Red Hat OpenShift, an enterprise container and Kubernetes application platform. This should make it possible to develop and deploy containerized big data workloads. IBM and Hortonworks will continue working to integrate services offered through Hortonworks DataPlane with IBM Cloud Private for Data.



More Information


Related Articles

Hadoop 3 Adds HDFS Erasure Coding

Hadoop 2.9 Adds Resource Estimator

Hadoop Adds In-Memory Caching

Hadoop SQL Query Engine Launched

Hadoop 2 Introduces YARN


To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.


Nvidia's AI Supercomputer For Medical Research And Drug Discovery

Last month Nvidia unveiled plans to build a supercomputer intended for AI research in health care. This prompts us to look at AI's potential role in health care and how it is already being used.

 [ ... ]

What Languages Do Devs Want to Learn?

Python, of course. What comes as a surprise is the size of the margin between Python and the rest of the pack. The other unexpected finding is that JavaScript doesn't even make it into the top 10. So, [ ... ]

More News





or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 19 September 2018 )