IBM Open Sources CodeFlare
Written by Kay Ewbank   
Friday, 14 January 2022

IBM has announced improvements to CodeFlare, its serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare has also now been made open source.

CodeFlare is a framework that simplifies the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.

codeflare

CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines.

The improved version of CodeFlare has been refined so that it can take on foundation models, and is now available as open-source software. The CodeFlare team says this effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud.

The main improvement to the actual framework is faster, automated foundation model training. Foundation models underpin much of machine analysis, but it takes a long time, running into weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks. The CodeFlare team's goal is to make the generation of downstream models ‘one-click’ easy for data scientists.

Foundation model pipelines consist of a sequence of multiple, often heterogeneous steps that can be difficult to create. CodeFlare provides a Python-based interface for the foundation model pipelines, making it possible to fully automate the tasks of preprocessing, validating, and adapting foundation models.

The developers provide an example of sentiment analysis in which  CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Following this CodeFlare tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis.

The developers say that:

"with just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes.

Our goal is to make generating downstream models as easy for data scientists as a single mouse click."

CodeFlare is available on GitHub now.

codeflare 

More Information

CodeFlare On GitHub

IBM Ray

Related Articles

IBM Introduces Hybrid Cloud AI/ML Framework

IBM Releases CodeNet Dataset For AI Coding 

IBM's Elyra AI Toolkit

New MIT–IBM Watson AI Lab

Google Comics Factory Makes ML Easy

Nearly A Third Of Devs Using AI And ML

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Google Announces Framework For Data Science Predictions
13/01/2022

Google has released Prediction Framework, which the developers describe as a time saver for data science prediction projects. The framework provides a way to put together a reusable project that inclu [ ... ]



Brain In A Dish Learns To Play Pong
12/01/2022

... and it learns faster than an artificial neural network. It is arguable that none of the approaches we take to general AI will ever lead to general AI. The reason is that we understand them too wel [ ... ]


More News

square

 



 

Comments




or email your comment to: comments@i-programmer.info