IBM Open Sources CodeFlare
Written by Kay Ewbank   
Friday, 14 January 2022

IBM has announced improvements to CodeFlare, its serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare has also now been made open source.

CodeFlare is a framework that simplifies the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.

codeflare

CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines.

The improved version of CodeFlare has been refined so that it can take on foundation models, and is now available as open-source software. The CodeFlare team says this effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud.

The main improvement to the actual framework is faster, automated foundation model training. Foundation models underpin much of machine analysis, but it takes a long time, running into weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks. The CodeFlare team's goal is to make the generation of downstream models ‘one-click’ easy for data scientists.

Foundation model pipelines consist of a sequence of multiple, often heterogeneous steps that can be difficult to create. CodeFlare provides a Python-based interface for the foundation model pipelines, making it possible to fully automate the tasks of preprocessing, validating, and adapting foundation models.

The developers provide an example of sentiment analysis in which  CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Following this CodeFlare tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis.

The developers say that:

"with just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes.

Our goal is to make generating downstream models as easy for data scientists as a single mouse click."

CodeFlare is available on GitHub now.

codeflare 

More Information

CodeFlare On GitHub

IBM Ray

Related Articles

IBM Introduces Hybrid Cloud AI/ML Framework

IBM Releases CodeNet Dataset For AI Coding 

IBM's Elyra AI Toolkit

New MIT–IBM Watson AI Lab

Google Comics Factory Makes ML Easy

Nearly A Third Of Devs Using AI And ML

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Flox Releases Flox Hub
13/03/2024

Flox has announced that its Command Line Interface (CLI) and FloxHub are now generally available. The CLI is open source and FloxHub is free for anyone to use.



Apache Shiro 2.0 Released
21/03/2024

Apache Shiro 2.0 has been released. The Java security framework now requires at least Java 11, and has added support for Jakarta EE 10.


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info