Amazon Offers Debugger For Machine Learning Models
Written by Alex Denham   
Thursday, 20 January 2022

Amazon has developed a method that automatically discovers machine learning model errors on particular types of input and provides a way to correct them. Defuse is a tool that the developers say can be used to train more robust models.

Defuse is based on a technique that trains a generative model on a classifier’s training dataset and then uses the latent space to generate new samples that are no longer correctly predicted by the classifier.

awslogo

Given a trained image classification model (a classifier), Defuse generates realistic-looking new images that are variations on test-set inputs that the classifier mishandles. Defuse then sorts misclassified images into high-level “model bugs” — groups of similar images that consistently cause errors. These can be used by the people who are creating the machine learning model to identify scenarios under which their models would fail and train more robust models.

The way Defuse works is that the data is augmented using a variational autoencoder (VAE) on the classifier’s training data. A VAE is a model trained to output the same data that it takes as input, but in-between, it produces a vector representation that captures salient properties of the input. That vector representation can be used to identify data that occupies latent space and shares similarities by proximity .

Once the VAE is trained, the tool uses its latent space to generate new image data.

The researchers carried out experiments on three public benchmark datasets, assessing accuracy on both the misclassification region test data and the original test set after performing correction, and found that the correction step in Defuse is highly effective at correcting the errors discovered during identification and distillation.

The research and resulting tool was presented at the NeurIPS 2021 Workshop on Explainable AI Approaches for Debugging and Diagnosis (XAI4Debugging), and the team says that to spur further research on this problem, they have publicly released the code for Defuse on GitHub.

 awslogo

More Information

Defuse On GitHub

Related Articles

Amazon's DeepRacer - Careful What You Wish For

Amazon's Giant Push Into Machine Learning

Amazon Redshift ML Now Generally Available

Better Face Detection in Amazon Rekognition

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


C#11 Will Support Raw String Literals
26/04/2022

Details of what Microsoft is adding to C# 11 have been released, with changes concentrating on better performance along with new language features including raw string literals and checked user-define [ ... ]



Scratch At 15 - Worth Knowing About
18/05/2022

This week is Scratch Week, a global, virtual celebration of the block programming language for kids from MIT that this year celebrates its 15th anniversary.


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 20 January 2022 )