|Google Open Sources C/C++ MapReduce Framework|
|Written by Kay Ewbank|
|Friday, 06 March 2015|
MapReduce Framework for C (MR4C) will let you run native code in Hadoop, allowing you to use image processing libraries developed in C and C++ on data held in Hadoop.
The framework was originally developed at Skybox Imaging, a satellite imagery company that Google acquired in June 2014, for the purpose of large scale satellite image processing and geospatial data science. There are a number of proprietary systems that execute native code in MapReduce frameworks, but MR4C is designed to be more flexible and as it is open source, can be freely used and further developed.
You can also string workflows of multiple algorithms together using an automatically generated configuration. There are callbacks in place for logging and progress reporting, and the reports can be viewed using the Hadoop JobTracker interface. Your workflow can be built and tested on a local machine using exactly the same interface employed on the target cluster.
The blog post says that the goal of this project is to abstract the important details of the MapReduce framework and allow users to focus on developing valuable algorithms.
There’s more information on the MR4C github page: https://github.com/google/mr4c
To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin, or sign up for our weekly newsletter.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Friday, 06 March 2015 )|