A curated, but probably biased and incomplete, list of awesome machine learning interpretability resources.
If you want to contribute to this list (and please do!) read over the contribution guidelines, send a pull request, or contact me @jpatrickhall.
An incomplete, imperfect blueprint for a more human-centered, lower-risk machine learning. The resources in this repository can be used to do many of these things today. The resources in this repository should not be considered legal compliance advice.
Image credit: H2O.ai Machine Learning Interpretability team, https://github.com/h2oai/mli-resources.
- Comprehensive Software Examples and Tutorials
- Explainability- or Fairness-Enhancing Software Packages
- Machine learning environment management tools
- Free Books
- Government and Regulatory Documents
- Other Interpretability and Fairness Resources and Lists
- Review and General Papers
- Classes
- Interpretable ("Whitebox") or Fair Modeling Packages
- AI Incident Tracker
- COMPAS Analysis Using Aequitas
- Explaining Measures of Fairness with SHAP
- Getting a Window into your Black Box Model
- From GLM to GBM Part 1
- From GLM to GBM Part 2
- IML
- Interpretable Machine Learning with Python
- Interpreting Machine Learning Models with the iml Package
- Interpretable Machine Learning using Counterfactuals
- Machine Learning Explainability by Kaggle Learn
- Model Interpretability with DALEX
- Model Interpretation series by Dipanjan (DJ) Sarkar:
- Partial Dependence Plots in R
- Saliency Maps for Deep Learning
- Visualizing ML Models with LIME
- Visualizing and debugging deep convolutional networks
- What does a CNN see?
- acd
- aequitas
- AI Fairness 360
- AI Explainability 360
- ALEPython
- Aletheia
- allennlp
- algofairness
- Alibi
- anchor
- BlackBoxAuditing
- casme
- captum
- causalml
- checklist
- contextual-AI
- ContrastiveExplanation (Foil Trees)
- dalex
- debiaswe
- DeepExplain
- deeplift
- deepvis
- DiCE
- DoWhy
- eli5
- fairml
- fairlearn
- fairness-comparison
- fairness_measures_code
- foolbox
- Grad-CAM (GitHub topic)
- iNNvestigate neural nets
- Integrated-Gradients
- interpret
- interpret_with_rules
- Keras-vis
- keract
- L2X
- lime
- LiFT
- lit
- lofo-importance
- lrp_toolbox
- MindsDB
- MLextend
- ml-fairness-gym
- ml_privacy_meter
- OptBinning
- parity-fairness
- PDPbox
- pyBreakDown
- PyCEbox
- pymc3
- pytorch-innvestigate
- rationale
- responsibly
- revise-tool
- robustness
- RISE
- sage
- SALib
- scikit-fairness
- shap
- Skater
- tensorfow/cleverhans
- tensorflow/lucid
- tensorflow/fairness-indicators
- tensorflow/model-analysis
- tensorflow/model-card-toolkit
- tensorflow/model-remediation
- tensorflow/privacy
- tensorflow/tcav
- tensorfuzz
- TensorWatch
- TextFooler
- tf-explain
- Themis
- themis-ml
- treeinterpreter
- woe
- xai
- xdeep
- yellowbrick
- aif360
- ALEPlot
- DrWhyAI
- DALEX
- DALEXtra
- EloML
- ExplainPrediction
- fastshap
- fairness
- fairmodels
- featureImportance
- flashlight
- forestmodel
- fscaret
- iBreakDown
- ICEbox
- iml
- ingredients
- intepret
- lightgbmExplainer
- lime
- live
- mcr
- modelDown
- modelOriented
- modelStudio
- pdp
- shapFlex
- shapleyR
- shapper
- smbinning
- vip
- xgboostExplainer
- An Introduction to Machine Learning Interpretability
- Explanatory Model Analysis
- Fairness and Machine Learning
- Interpretable Machine Learning
- Responsible Machine Learning (requires email for now)
- 12 CFR Part 1002 - Equal Credit Opportunity Act (Regulation B)
- A Regulatory Framework for AI: Recommendations for PIPEDA Reform
- AI Principles: Recommendations on the Ethical Use of Artificial Intelligence by the Department of Defense
- THE AIM INITIATIVE
- Algorithmic Accountability Act of 2019
- ALGORITHM CHARTER FOR AOTEAROA NEW ZEALAND
- Artificial Intelligence (AI) in the Securities Industry
- Article 22 EU GDPR
- Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment - Shaping Europe’s digital future - European Commission
- Audit of Governance and Protection of Department of Defense Artificial Intelligence Data and Technology
- A Primer on Artificial Intelligence in Securities Markets
- Biometric Information Privacy Act
- Booker Wyden Health Care Letters
- California Consumer Privacy Act (CCPA)
- California Privacy Rights Act (CPRA)
- Consultation on the OPC’s Proposals for ensuring appropriate regulation of artificial intelligence
- Civil liability regime for artificial intelligence
- Data Ethics Framework
- DEVELOPING FINANCIAL SECTOR RESILIENCE IN A DIGITAL WORLD: SELECTED THEMES IN TECHNOLOGY AND RELATED RISKS
- Directive on Automated Decision Making
- Executive Order on Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government
- Facial Recognition and Biometric Technology Moratorium Act of 2020
- Four Principles of Explainable Artificial Intelligence
- General principles for the use of Artificial Intelligence in the financial sector
- Gouvernance des algorithmes d’intelligence artificielle dans le secteur financier (French)
- Innovation spotlight: Providing adverse action notices when using AI/ML models
- Office of Management and Budget Guidance for Regulation of Artificial Intelligence Applications (Finalized Nov. 2020)
- On Artificial Intelligence - A European approach to excellence and trust
- Opinion of the German Data Ethics Commission
- Principles of Artificial Intelligence Ethics for the Intelligence Community
- Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform Guidelines on Employee Selection Procedures
- RE: Use of External Consumer Data and Information Sources in Underwriting for Life Insurance
- Singapore Personal Data Protection Commission (PDPC) Model Artificial Intelligence Governance Framework
- SUPERVISORY GUIDANCE ON MODEL RISK MANAGEMENT
- U.K. Information Commissioner's Office (ICO) AI Audting Framework (overview series)
- U.S FDA Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)
- U.S. House of Representatives Resolution on AI Strategy
- Using Artificial Intelligence and Algorithms
- 8 Principles of Responsible ML
- ACM FAT* 2019 Youtube Playlist
- Adversarial ML Threat Matrix
- AI Ethics Guidelines Global Inventory
- AI Incident Database
- AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
- Algorithms and prejudice
- Awesome interpretable machine learning ;)
- Awesome machine learning operations
- Awful AI
- algoaware
- BIML Interactive Machine Learning Risk Framework
- Beyond Explainability: A Practical Guide to Managing Risk in Machine Learning Models
- criticalML
- Data Feminism
- Debugging Machine Learning Models (ICLR workshop proceedings)
- Decision Points in AI Governance
- Deep Insights into Explainability and Interpretability of Machine Learning Algorithms and Applications to Risk Management
- Distill
- Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) Scholarship
- From Principles to Practice: An interdisciplinary framework to operationalise AI ethics
- How will the GDPR impact machine learning?
- Machine Learning Ethics References
- Machine Learning Interpretability Resources
- Machine Learning: Considerations for fairly and transparently expanding access to credit
- MIT AI Ethics Reading Group
- On the Responsibility of Technologists: A Prologue and Primer
- private-ai-resources
- Problems with Shapley-value-based explanations as feature importance measures
- Real-World Model Debugging Strategies
- ResponsibleAI
- Sample AI Incident Response Checklist
- Ten Questions on AI Risk
- Testing and Debugging in Machine Learning
- Troubleshooting Deep Neural Networks
- Warning Signs: The Future of Privacy and Security in an Age of Machine Learning
- XAI Resources
- You Created A Machine Learning Application Now Make Sure It's Secure
- 50 Years of Test (Un)fairness: Lessons for Machine Learning
- A Comparative Study of Fairness-Enhancing Interventions in Machine Learning
- A Survey Of Methods For Explaining Black Box Models
- A Marauder’s Map of Security and Privacy in Machine Learning
- Challenges for Transparency
- Closing the AI Accountability Gap
- Explaining by Removing: A Unified Framework for Model Explanation
- Explaining Explanations: An Overview of Interpretability of Machine Learning
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
- Interpretable Machine Learning: Definitions, Methods, and Applications
- Limitations of Interpretable Machine Learning
- Machine Learning Explainability in Finance
- On the Art and Science of Machine Learning Explanations
- Please Stop Explaining Black Box Models for High-Stakes Decisions
- Software Engineering for Machine Learning: A Case Study
- The Mythos of Model Interpretability
- Towards A Rigorous Science of Interpretable Machine Learning
- Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
- The Security of Machine Learning
- Techniques for Interpretable Machine Learning
- Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
- An Introduction to Data Ethics
- Certified Ethical Emerging Technologist
- Fairness in Machine Learning
- Fast.ai Data Ethics course
- Human-Center Machine Learning
- Introduction to Responsible Machine Learning
- Trustworthy Deep Learning
- Bayesian Case Model
- Bayesian Ors-Of-Ands
- Bayesian Rule List (BRL)
- Explainable Boosting Machine (EBM)/GA2M
- fair-classification
- Falling Rule List (FRL)
- H2O-3
- Optimal Sparse Decision Trees
- Monotonic XGBoost
- Multilayer Logical Perceptron (MLLP)
- pyGAM
- pySS3
- Risk-SLIM
- Scikit-learn
- sklearn-expertsys
- skope-rules
- Super-sparse Linear Integer models (SLIMs)
- tensorflow/lattice
- This Looks Like That
- arules
- Causal SVM
- elasticnet
- Explainable Boosting Machine (EBM)/GA2M
- gam
- glm2
- glmnet
- H2O-3
- Monotonic XGBoost
- quantreg
- rpart
- RuleFit
- Scalable Bayesian Rule Lists (SBRL)
- Mar 1988 - A blot on the profession
- Jan 2010 - Are Face-Detection Cameras Racist?
- Jul 2015 - Google says sorry for racist auto-tag in photo app
- Mar 2016 - Here Are the Microsoft Twitter Bot’s Craziest Racist Rants
- Jun 2016 - Google faulted for racial bias in image search results for black teenagers
- Oct 2016 - 'Rogue' Algorithm Blamed for Historic Crash of the British Pound
- Oct 2016 - Crime-prediction tool PredPol amplifies racially biased policing, study shows
- May 2017 - Houston Schools Must Face Teacher Evaluation Lawsuit
- Jun 2017 - When a Computer Program Keeps You in Jail
- Jul 2017 - ‘Balls have zero to me to me’: What happened when Facebook’s AI chatbots Bob & Alice created their own language
- Jul 2017 - YouTube: Boston Dynamics' Atlas Falls Over After Demo at the Congress of Future Scientists and Technologists
- Nov 2017 - Hackers Say They've Broken Face ID a Week After iPhone X Release
- Nov 2017 - India’s Friendly Robot Mitra Not Only Greets VIPs On The Stage, But Also Parties Like A Rockstar (Mitra trips over Ivanka Trump/PM Modi introduction)
- Jan 2018 - YouTube: CES 2018: Robot refuses to co-operate with LG chief - BBC News
- Feb 2018 - Study finds gender and skin-type bias in commercial artificial-intelligence systems
- Mar 2018 - Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam
- Mar 2018 - AI-Assisted Fake Porn Is Here and We're All F***ed
- Jul 2018 - Amazon’s Face Recognition Falsely Matched 28 Members of Congress With Mugshots
- Jul 2018 - IBM’s Watson supercomputer recommended ‘unsafe and incorrect’ cancer treatments, internal documents show
- Oct 2018 - Amazon scraps 'sexist AI' recruiting tool that showed bias against women
- Facial recognition system in China mistakes bus ad for jaywalker
- Dec 2018 - AI start-up that scanned babysitters halts launch following Post Report
- Jan 2019 - Cambridge Analytica’s parent pleads guilty to breaking UK data law
- May 2019 - Investor Sues After an AI’s Automated Trades Cost Him $20 Million
- May 2019 - Millions of people uploaded photos to the Ever app. Then the company used them to develop facial recognition tools.
- Jun 2019 - Google and the University of Chicago Are Sued Over Data Sharing
- Aug 2019 - LGBTQ+ creators file lawsuit against YouTube for discrimination
- Sep 2019 - The viral selfie app ImageNet Roulette seemed fun – until it called me a racist slur
- Sep 2019 - Scammer Successfully Deepfaked CEO's Voice To Fool Underling Into Transferring $243,000
- Oct 2019 - Oh dear... AI models used to flag hate speech online are, er, racist against black people
- Oct 2019 - Dissecting racial bias in an algorithm used to manage the health of populations
- Nov 2019 - NY regulator investigating Apple Card for possible gender bias
- Nov 2019 - Chinese-style facial recognition technology is trialled in Australian schools to register pupils - sparking major privacy concerns
- Dec 2019 - Tenants sounded the alarm on facial recognition in their buildings. Lawmakers are listening
- Dec 2019 - Researchers bypass airport and payment facial recognition systems using masks
- Jan 2020 - Atlantic Plaza Towers tenants won a halt to facial recognition in their building: Now they’re calling on a moratorium on all residential use
- Jan 2020 - Trivago misled consumers about hotel room rates
- Feb 2020 - An Indian politician is using deepfake technology to win new voters
- Feb 2020 - Suckers List: How Allstate’s Secret Auto Insurance Algorithm Squeezes Big Spenders
- Feb 2020 - Tesla Autopilot gets tricked into accelerating from 35 to 85 mph with modified speed limit sign
- Mar 2020 - Netherlands: Court Prohibits Government’s Use of AI Software to Detect Welfare Fraud
- Apr 2020 - Google apologizes after its Vision AI produced racist results
- Apr 2020 - Google’s medical AI was super accurate in a lab. Real life was a different story.
- May 2020 - Researchers find major demographic differences in speech recognition accuracy
- May 2020 - Access Denied: Faulty Automated Background Checks Freeze Out Renters
- May 2020 - A.C.L.U. Accuses Clearview AI of Privacy ‘Nightmare Scenario’
- May 2020 - Walmart Employees Are Out to Show Its Anti-Theft AI Doesn't Work
- May 2020 - Robodebt removed humans from Human Services, and the Government is facing the consequences
- May 2020 - The Most Devastating Software Mistake Of All Time. Why Is the Imperial Model Under Criticism?
- Jun 2020 - Government’s Use of Algorithm Serves Up False Fraud Charges
- Jun 2020 - Microsoft's robot editor confuses mixed-race Little Mix singers
- Jun 2020 - Tweet: "This algorithm probably made this mistake ..." (President Obama de-blurred into white male)
- Jun 2020 - Detroit Police Chief: Facial Recognition Software Misidentifies 96% of the Time
- Jun 2020 - Wrongfully Accused by an Algorithm
- Jun 2020 - An Algorithm that "Predicts" Criminality Based on a Face Sparks a Furor
- Jun 2020 - PwC facial recognition tool criticised for home working privacy invasion
- Jun 2020 - Santa Cruz becomes the first U.S. city to ban predictive policing
- Jun 2020 - YouTube Sued for Race Discrimination, Profiting from Hate Speech
- Jul 2020 - ISIS 'still evading detection on Facebook', report says
- Jul 2020 - Meet the Secret Algorithm That's Keeping Students Out of College
- Jul 2020 - Rite Aid deployed facial recognition systems in hundreds of U.S. stores
- Jul 2020 - Tweet: "Oh, dear ..." (GPT-3 anti-semitism)
- Jul 2020 - Google Ad Portal Equated “Black Girls” with Porn
- Jul 2020 - Facial biometrics training dataset leads to BIPA lawsuits against Amazon, Alphabet and Microsoft
- Jul 2020 - POLICE SURVEILLED GEORGE FLOYD PROTESTS WITH HELP FROM TWITTER-AFFILIATED STARTUP DATAMINR
- Aug 2020 - Police use of facial recognition unlawfully breached privacy rights, says Court of Appeal ruling
- Aug 2020 - There is nothing 'fair' about judging A-levels by algorithm
- Aug 2020 - When algorithms define kids by postcode: UK exam results chaos reveal too much reliance on data analytics
- Aug 2020 - Macy’s hit with privacy lawsuit over alleged use of controversial facial recognition software
- Aug 2020 - Google’s Advertising Platform Is Blocking Articles About Racism
- Aug 2020 - Home Office drops 'racist' algorithm from visa decisions
- Aug 2020 - De Blasio Will Reassess NYPD's Use Of Facial Recognition Tech After Protester Arrest
- Aug 2020 - Facebook algorithm recommending Holocaust denial and fascist content, report finds
- Aug 2020 - Report: AI Company Leaks Over 2.5M Medical Records
- Aug 2020 - Watchdog investigates Barclays for spying on staff
- Aug 2020 - PopID’s face-based payments pose privacy and security risks
- Aug 2020 - Tinder charges older people more
- Aug 2020 - Uber and Lyft pricing algorithms charge more in non-white areas
- Sep 2020 - Pasco’s sheriff uses data to guess who will commit crime. Then deputies ‘hunt down’ and harass them
- Sep 2020 - The Met Police didn’t check if facial recognition tech was racist before trialling it
- Sep 2020 - These students figured out their tests were graded by AI — and the easy way to cheat
- Sep 2020 - Google says Street View maps algorithm error blurred out Hong Kong protest graffiti aimed at Xi Jinping
- Sep 2020 - AI attempts to ease fear of robots, blurts out it can’t ‘avoid destroying humankind’
- Sep 2020 - Ola is facing a drivers’ legal challenge over data access rights and algorithmic management
- Sep 2020 - Instagram apologizes for removing images of Black British model
- Sep 2020 - Tesla owner in Canada charged with ‘sleeping’ while driving over 90 mph
- Sep 2020 - Female historians and male nurses do not exist, Google Translate tells its European users
- Sep 2020 - Twitter is looking into why its photo preview appears to favor white faces over Black faces
- Sep 2020 - Facebook Live’s New Music Terms of Service Unfairly Impact Artists
- Sep 2020 - CoreLogic’s screening algorithm may have discriminated against renters: lawsuit
- Sep 2020 - Gradient Photo Editing App Criticized Over 'Racist' AI Face Feature
- Sep 2020 - ExamSoft’s remote bar exam sparks privacy and facial recognition concerns
- Sep 2020 - "Trustworthiness" Study Is Basically Phrenology, Annoying Scientists, Historians, Just About Everyone
- Sep 2020 - IBM faces another age-discrimination lawsuit in Austin
- Sep 2020 - Your favorite A.I. language tool is toxic
- Sep 2020 - Catching Amazon in a lie
- Sep 2020 - Tweet: "A faculty member has been asking how to stop Zoom from removing his head ..." (Zoom erasing darker-skinned professor's head)
- Oct 2020 - Jewish Baby Stroller Image Algorithm
- Oct 2020 - Instagram blames GDPR for failure to tackle rampant self-harm and eating-disorder images
- Oct 2020 - UK passport photo checker shows bias against dark-skinned women
- Oct 2020 - States Say the Online Bar Exam Was a Success. The Test-Taker Who Peed in His Seat Disagrees
- Oct 2020 - Tiny Changes Let False Claims About COVID-19, Voting Evade Facebook Fact Checks
- Oct 2020 - Leaving Cert: Why the Government deserves an F for algorithms
- Oct 2020 - Lawsuit alleges biometric privacy violations from face recognition algorithm training
- Oct 2020 - You’re being watched: The dangers of ProctorU
- Oct 2020 - Fake naked photos of thousands of women shared online
- Oct 2020 - Researchers find evidence of racial, gender, and socioeconomic bias in chest X-ray classifiers
- Oct 2020 - Uber sued by drivers over ‘automated robo-firing'
- Oct 2020 - How an Algorithm Blocked Kidney Transplants to Black Patients
- Oct 2020 - Australian researchers have shown how you could become invisible to security cameras
- Oct 2020 - EPIC files lawsuit to force release of ICE facial recognition documents
- Oct 2020 - Researchers take a stand on algorithm design for job centers: Landing a job isn't always the right goal
- Oct 2020 - Facebook under fire for boosting right-wing news sources and throttling progressive alternatives
- Oct 2020 - AI Camera Ruins Soccer Game For Fans After Mistaking Referee's Bald Head For Ball
- Nov 2020 - Researchers show that computer vision algorithms pretrained on ImageNet exhibit multiple, distressing biases
- Nov 2020 - Trivago loses appeal over misleading website algorithm ruling
- Nov 2020 - Research finds gender bias within state funding model
- Nov 2020 - Split-Second 'Phantom' Images Can Fool Tesla's Autopilot
- Nov 2020 - Boris executes U-turn over controversial house building algorithm
- Nov 2020 - Top intel official warns of bias in military algorithms
- Nov 2020 - Opinion: Artificial 'Intelligence': Unemployment system denied legitimate COVID-19 claims
- Nov 2020 - LAPD ban facial recognition following alleged unauthorised use
- Nov 2020 - Instagram removed 80 PER CENT less graphic content about suicide during the first three months of lockdown after 'most of its moderators were sent home due to Covid rules'
- Nov 2020 - Facebook's AI Mistakenly Bans Ads for Struggling Businesses
- Nov 2020 - A Bot Made Frank Sinatra Cover Britney Spears. YouTube Removed It Over Copyright Claims.
- Dec 2020 - Concern over potential gender bias in job recruitment algorithms
- Dec 2020 - Facial Recognition Company Lied to School District About its Racist Tech
- Dec 2020 - China’s Huawei tested A.I. software that could identify Uighur Muslims and alert police, report says
- Dec 2020 - We’ve Known Brand Safety Tech Was Bad—Here’s How Bad
- Dec 2020 - Hey Alexa, what's my PIN?
- Dec 2020 - Waze sent commuters toward California wildfires, drivers say
- Dec 2020 - The Death and Life of an Admissions Algorithm
- Dec 2020 - Algorithms searching for child abuse could be banned under new EU privacy rules