IBM Open Sources CodeFlare
Written by Kay Ewbank   
Friday, 14 January 2022

IBM has announced improvements to CodeFlare, its serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare has also now been made open source.

CodeFlare is a framework that simplifies the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.


CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines.

The improved version of CodeFlare has been refined so that it can take on foundation models, and is now available as open-source software. The CodeFlare team says this effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud.

The main improvement to the actual framework is faster, automated foundation model training. Foundation models underpin much of machine analysis, but it takes a long time, running into weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks. The CodeFlare team's goal is to make the generation of downstream models ‘one-click’ easy for data scientists.

Foundation model pipelines consist of a sequence of multiple, often heterogeneous steps that can be difficult to create. CodeFlare provides a Python-based interface for the foundation model pipelines, making it possible to fully automate the tasks of preprocessing, validating, and adapting foundation models.

The developers provide an example of sentiment analysis in which  CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Following this CodeFlare tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis.

The developers say that:

"with just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes.

Our goal is to make generating downstream models as easy for data scientists as a single mouse click."

CodeFlare is available on GitHub now.


More Information

CodeFlare On GitHub


Related Articles

IBM Introduces Hybrid Cloud AI/ML Framework

IBM Releases CodeNet Dataset For AI Coding 

IBM's Elyra AI Toolkit

New MIT–IBM Watson AI Lab

Google Comics Factory Makes ML Easy

Nearly A Third Of Devs Using AI And ML


To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.


Look Once to Hear - A Spy's Dream Come True

Deep learning has triumphed again. You can don a pair of headphones, look at a person talking and from then on the system will track the person so you can hear them as they move away or become swamped [ ... ]

Mirascope-Python's Alternative To Langchain

Mirascope is a Python library that lets you access a range of Large Language Models, but in a more straightforward and Pythonic way.

More News

kotlin book



or email your comment to: