IBM Open Sources CodeFlare
Written by Kay Ewbank   
Friday, 14 January 2022

IBM has announced improvements to CodeFlare, its serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare has also now been made open source.

CodeFlare is a framework that simplifies the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.


CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines.

The improved version of CodeFlare has been refined so that it can take on foundation models, and is now available as open-source software. The CodeFlare team says this effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud.

The main improvement to the actual framework is faster, automated foundation model training. Foundation models underpin much of machine analysis, but it takes a long time, running into weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks. The CodeFlare team's goal is to make the generation of downstream models ‘one-click’ easy for data scientists.

Foundation model pipelines consist of a sequence of multiple, often heterogeneous steps that can be difficult to create. CodeFlare provides a Python-based interface for the foundation model pipelines, making it possible to fully automate the tasks of preprocessing, validating, and adapting foundation models.

The developers provide an example of sentiment analysis in which  CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Following this CodeFlare tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis.

The developers say that:

"with just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes.

Our goal is to make generating downstream models as easy for data scientists as a single mouse click."

CodeFlare is available on GitHub now.


More Information

CodeFlare On GitHub


Related Articles

IBM Introduces Hybrid Cloud AI/ML Framework

IBM Releases CodeNet Dataset For AI Coding 

IBM's Elyra AI Toolkit

New MIT–IBM Watson AI Lab

Google Comics Factory Makes ML Easy

Nearly A Third Of Devs Using AI And ML


To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


Couchbase Adds Azure Support To Capella

Couchbase has announced that Capella, the fully managed service version of its distributed NoSQL database that includes mobile and IoT application services, now supports Microsoft Azure. 

SourceBuddy Brings Eval To Java

SourceBuddy is a Java library that compiles and loads dynamically generated Java source code. This has the advantage of providing Java with an eval facility such as those found in interpreted lan [ ... ]

More News





or email your comment to: