Microsoft Adds Error Analysis To Open Source AI Toolkits
Written by Kay Ewbank   
Thursday, 04 March 2021

Microsoft has added a new toolkit to its collection of 'responsible AI'. The new addition is for Error Analysis and uses machine learning to partition model errors. As well as being available as open source kits,the AI tools are also integrated within Azure Machine Learning.

The tools in the original collection, which were introduced at Microsoft Build in 2020. - InterpretML, Fairlearn, and SmartNoise - let machine learning data scientists understand model predictions, assess fairness, and protect sensitive data.




InterpretML can be used to improve the accuracy of ML models, and to perform what-if analysis to see what happens to the model predictions when you change feature values. Fairlearn can be used to make sure the outcomes of your models are fairer for everyone. It lets you assess model fairness and mitigate unfairness while optimizing model performance.

The final tool in the original collection, SmartNoise, can be used to create differential privacy best practices. It can be used to inject statistical noise in data, to help prevent disclosure of private information, without significant accuracy loss, and to manage exposure risk by tracking the information budget used by individual queries and limiting further queries as appropriate. SmartNoise has now been updated with the ability to protect entire datasets using a new synthetic data capability. A synthetic dataset represents a man-made sample derived from the original dataset while retaining as many statistical characteristics as possible. The original dataset is combined with the synthetic one to create a "differentially private synthetic dataset" that can be analyzed many times without increasing the privacy risk.

The main new addition is an Error Analysis toolkit. Error Analysis uses machine learning to partition model errors. The errors are organized by meaningful dimensions so data scientists get a clearer understanding of the patterns in the errors. The idea is to more easily identify subgroups with higher inaccuracy and visually diagnose the root causes behind these errors.

The plan is that Error Analysis, alongside the other Responsible AI toolkits will be combined into a larger model assessment dashboard available in both OSS and Azure Machine Learning by mid 2021.



More Information

Azure Machine Learning

Error Analysis


Differential Privacy Toolkit


Related Articles 

Machine Learning For .NET 

Infer.NET Machine Learning Framework Now Open Source   

Google Provides Free Machine Learning For All 

Azure Machine Learning Service Goes Live 

Machine Learning Goes Azure - Azure ML Announced  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.



The Year of AI Breakthroughs 2022

Louis Bouchard has compiled a great list of research papers covering AI breakthroughs that were published during last year. His introductions and links to the papers and even to code make this a great [ ... ]

The End Of Google?

There are moments where technology changes just enough to disrupt a situation that you thought would survive forever. Is the success of foundational models like ChatGPT one of them? Google seems to th [ ... ]

More News





or email your comment to:


Last Updated ( Thursday, 04 March 2021 )