Responsible machine learning (responsible ML)
Develop, use, and govern AI solutions responsibly with Azure AI
Build trusted solutions responsibly with Azure AI
Take advantage of state-of-the-art technology to build responsible AI applications, from development to deployment. Put responsible AI principles into practise in machine learning and cognitive services, and build trust with your customers. Learn how to apply responsible AI in the financial and healthcare industries.
Responsible development of AI solutions for fairness, reliability, and explainability to deliver trusted outcomes
Responsible usage of AI solutions by applying guidance to optimise performance while minimising harm when deployed
Responsible governance of AI solutions for transparency and accountability to achieve positive outcomes
Develop responsibly for fairness and explainability
Assess your machine learning model using the responsible AI dashboard. Evaluate with reproducible and automated workflows to assess model fairness, explainability, error analysis, causal analysis, model performance, and exploratory data analysis.
Use AI solutions responsibly in deployment
Make real-life interventions and policies with causal analysis in the responsible AI dashboard. Generate a responsible AI scorecard for trained machine learning models in your Azure Machine Learning workspace at deployment time.
Govern for transparency and accountability
Export the responsible AI scorecard for your machine learning models to a PDF to contextualize responsible AI metrics. Share it with both technical and non-technical audiences to involve stakeholders and streamline compliance review.
Develop AI Responsibly webinar
Businesses often don't have a clear understanding of machine learning. Bring AI into the business mainstream with responsible machine learning.
IDC report: Empowering Your Organisation with Responsible AI
Learn how to approach responsible AI from an end-to-end development lifecycle perspective.
Driving Business Value with Responsible AI webinar
Watch IDC and Microsoft experts explain how to build responsible AI solutions to cultivate trust in machine learning.
Comprehensive security and compliance, built in
-
Microsoft invests more than USD 1 billion annually on cybersecurity research and development.
-
We employ more than 3,500 security experts dedicated to data security and privacy.
-
Azure has more certifications than any other cloud provider. View the comprehensive list.
Get started with an Azure free account
Start free. Get $200 credit to use within 30 days. While you have your credit, get free amounts of many of our most popular services, plus free amounts of 40+ other services that are always free.
After your credit, move to pay as you go to keep building with the same free services. Pay only if you use more than your free monthly amounts.
After 12 months, you'll keep getting 40+ always-free services—and still pay only for what you use beyond your free monthly amounts.
Customers using Azure responsible AI
Northumbria Healthcare NHS Foundation Trust
Mike Reed, Consultant Trauma and Orthopedic Surgeon, Northumbria Healthcare NHS Foundation Trust"The responsible AI dashboard will give us a better indication about what would happen to patients requiring hip and knee surgery, which will allow us to have a more meaningful conversation with them as we work to provide the best care possible."
EY
Alex Mohelsky, Partner and Advisory Data, Analytic, and AI Leader, EY Canada"We see Azure Machine Learning and our partnership with Microsoft as critical to driving increased adoption and acceptance of AI from the regulators."
SAS
Daniel Engberg, Head of Data Analytics and Artificial Intelligence, Scandinavian Airlines"With model interpretability in Azure Machine Learning, we have a high degree of confidence that our machine learning model is generating meaningful and fair results."
Resources
Tools
See responsible ML in action
Get better visibility into models to explain their behavior
Get better visibility into models to explain their behavior
How to build trusted models?
Susan, a data scientist, works for an online streaming platform. Her company is launching a new show next month and she has been asked to build a model to identify customers who would be interested in watching it and thus should receive a promotion email inviting them.
Her first model shows an accuracy of 90% on historical data. It seems too good to be true, so she wants to understand the model behavior to ascertain its trustworthiness.
Susan is a data scientist tasked with creating a model to identify customers for a new show. Her model shows an accuracy of 90% on historical data. She would like to understand the model behaviour to ascertain its trustworthiness.
First, get the most recent model explanations
Azure ML effortlessly surfaces model explanations via the InterpretML SDK. Either from the ‘Models’ tab in her Azure ML workspace or from her notebook, Susan can fetch model explanations from the most recent experiment run and visualise them.
Examine the strongest predictor variable/feature
On the global importance view, she sees LOCATION_NEWYORK as the strongest predictor for the promotion. This is unexpected - she knows that viewing habits can be materially different for New York compared to other locations.
Validate the predictor and assess model behavior
On the local importance view, she can inspect the results with more granularity. She finds the model has drawn the wrong conclusions about locations other than New York and realises that the model may not perform well across their wider customer base.
Improve the training data for accuracy and rerun the experiment
Susan’s hypothesis is that the bias may be coming from her training data. On reviewing, she finds that records for New York are introducing a skew in her training set. She works with her team to remove this skew and re-run the experiment. The new model is published in Azure ML, where its experiment results, including model explanations, are available for review.
Model interpretability encourages a glass box approach to your models
With model interpretability, machine learning models are no longer a black box. It helped Susan understand her model behavior at training time and identify a fault. This enabled her to avoid releasing a model which would have performed poorly in production.
With interpretability supported at inferencing time as well, it helps make model predictions transparent to customers - a key principle of responsible AI.
Model Interpretability in Azure ML InterpretML Github repo for this demo
With model interpretability, machine learning models are no longer a black box. It helped Susan understand her model behavior at training time and identify a fault. This enabled her to avoid releasing a model which would have performed poorly in production.
With interpretability supported at inferencing time as well, it helps make model predictions transparent to customers - a key principle of responsible AI.
Model Interpretability in Azure ML InterpretML Github repo for this demo
Make critical decisions with models that are without hidden biases
Make critical decisions with models that are without hidden biases
How to fairly identify the right job applicants?
Joe, a data scientist, works for an IT company. Their recruiting team is swamped with job applications for developer roles and has asked him to build a model to recommend which applicants should be interviewed. His first model shows accuracy at over 85%. Joe is worried though - his company has traditionally recruited college graduates with degrees and he want to ensure his model does not discriminate against other candidates with diplomas, certifications or relevant experience.
He turns to Azure ML, to see if there is any bias in his new model.
Joe, a data scientist, works for an IT company. Their recruiting team is swamped with job applications for developer roles and has asked him to build an unbiased model to recommend which applicants should be interviewed.
Choose a potential bias factor in the model
Azure ML provides capabilities to detect bias and ensure fairness within models via the Fairlearn SDK. From within his Jupyter notebook, Joe initiates the Fairlearn toolkit and selects HAS_DEGREE as a sensitive feature, to determine if having a degree has any impact on the final recommendation from the model.
Select the model metric for analysis
The Fairlearn toolkit offers many metrics to analyse against potential bias, like accuracy, precision, recall etc. Joe opts accuracy as the metric to determine if there is any impact on the accuracy of the model while adjusting it for fairness. This will help Joe maintain a good balance between the accuracy of the model and its fairness.
Identify if there is inherent bias
The Fairlearn toolkit offers visualisations that show the relevant details of the model. Joe observes that the accuracy is relatively good for both, degree holders and non-degree holders, however, the model is far more likely to recommend interviewing a degree holder. Joe’s concerns are validated - the model is showing bias.
Mitigate bias in the model
Within Azure ML, the Fairlearn toolkit shows more versions of the model with varied levels of bias mitigation. Joe can see the tradeoff between the accuracy and the disparity across predictions. This provides Joe the options to reduce bias and increase fairness within his model. He selects one of the improved models and publishes it for release.
Model fairness ensures that there is no bias within the model’s recommendations
With Fairlearn in Azure ML, Joe was able to quickly identify and mitigate an inherent bias in his model before it went to production. This bias could have risked the company losing valuable candidates, not to mention their reputation. With the inbuilt mitigation options, he was able to address the bias transparently.
Fairlearn enables AI systems to be inclusive and treat all people with fairness, a key principle of responsible AI.
Model fairness in Azure ML Fairlearn Toolkit Github repo for this demo
With Fairlearn in Azure ML, Joe was able to quickly identify and mitigate an inherent bias in his model before it went to production. This bias could have risked the company losing valuable candidates, not to mention their reputation. With the inbuilt mitigation options, he was able to address the bias transparently.
Fairlearn enables AI systems to be inclusive and treat all people with fairness, a key principle of responsible AI.
Model fairness in Azure ML Fairlearn Toolkit Github repo for this demo
Using differential privacy techniques to protect sensitive datasets
Using differential privacy techniques to protect sensitive datasets
How to ensure data privacy while building models?
Anna, a data scientist, has been working on a model to predict patient readmission rates but feels restricted by the redacted or masked data she can access due to the patient data privacy regulations.
With her initial models lacking the predictive rigor needed for deployment, she works with her internal compliance team to expose a more complete dataset, while still maintaining the privacy, by using the WhiteNoise differential privacy toolkit within Azure ML.
Anna, a data scientist, is trying to predict patient readmission rates, but doesn't have access to patient data due to privacy regulations. Using SmartNoise in Azure ML, Anna gets a more useful dataset, while still maintaining privacy.
Leverage SmartNoise to protect patient privacy while querying data
Anna queries the dataset using SQL in her Jupyter notebook within Azure ML, but now, via the SmartNoise toolkit. SmartNoise ensures that the result sets are carefully injected with statistical noise and the exposure risk is managed by tracking the information exposed by individual queries and limiting them as appropriate.
Leverage this differentially private dataset for model training
Anna incorporates the dataset into her existing training script. It does not require her to change anything in her code except pointing to the new dataset. She can seamlessly execute the training job on the same Azure ML compute target. Throughout the process, her compliance team can validate that her queries remain within budget and thus prove patient privacy is protected.
Validate the model for efficacy
Using the results widget, Anna can see her new model is now showing better accuracy due to the added features in her training data. She has not violated any privacy rules and her compliance team has full visibility into her querying behavior. This new model is now ready to move to production, for better predictions. She publishes it to Azure ML for release.
SmartNoise in Azure ML, protects data privacy while building models
With SmartNoise in Azure ML, Anna was able to confidently build her machine learning model without compromising privacy of sensitive data. This enabled her to use machine learning effectively and materially improve outcomes, while allowing the compliance team to rest assured they could prove sensitive data was not exposed.
SmartNoise uses differential privacy techniques to enable building machine learning while respecting privacy, a key principle of responsible AI.
SmartNoise in Azure ML SmartNoise Toolkit Github repo for this demo
With SmartNoise in Azure ML, Anna was able to confidently build her machine learning model without compromising privacy of sensitive data. This enabled her to use machine learning effectively and materially improve outcomes, while allowing the compliance team to rest assured they could prove sensitive data was not exposed.
SmartNoise uses differential privacy techniques to enable building machine learning while respecting privacy, a key principle of responsible AI.
SmartNoise in Azure ML SmartNoise Toolkit Github repo for this demo
Build your machine learning skills with Azure
Learn more about machine learning on Azure and participate in hands-on tutorials with a 30-day learning journey. By the end, you'll be prepared to take the Azure Data Scientist Associate Certification.