Detection and Evaluation of Machine Learning Bias
Machine learning models are built using training data, which is collected from human experience and is prone to bias. Humans demonstrate a cognitive bias in their thinking and behavior, which is ultimately reflected in the collected data. From Amazon’s hiring system, which was built using ten years...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/14/6271 |
| _version_ | 1797527703092461568 |
|---|---|
| author | Salem Alelyani |
| author_facet | Salem Alelyani |
| author_sort | Salem Alelyani |
| collection | DOAJ |
| description | Machine learning models are built using training data, which is collected from human experience and is prone to bias. Humans demonstrate a cognitive bias in their thinking and behavior, which is ultimately reflected in the collected data. From Amazon’s hiring system, which was built using ten years of human hiring experience, to a judicial system that was trained using human judging practices, these systems all include some element of bias. The best machine learning models are said to mimic humans’ cognitive ability, and thus such models are also inclined towards bias. However, detecting and evaluating bias is a very important step for better explainable models. In this work, we aim to explain bias in learning models in relation to humans’ cognitive bias and propose a wrapper technique to detect and evaluate bias in machine learning models using an openly accessible dataset from UCI Machine Learning Repository. In the deployed dataset, the potentially biased attributes (PBAs) are gender and race. This study introduces the concept of alternation functions to swap the values of PBAs, and evaluates the impact on prediction using KL divergence. Results demonstrate females and Asians to be associated with low wages, placing some open research questions for the research community to ponder over. |
| first_indexed | 2024-03-10T09:46:33Z |
| format | Article |
| id | doaj.art-008452809b8841d18c2a7ebe9ce62127 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T09:46:33Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-008452809b8841d18c2a7ebe9ce621272023-11-22T03:06:56ZengMDPI AGApplied Sciences2076-34172021-07-011114627110.3390/app11146271Detection and Evaluation of Machine Learning BiasSalem Alelyani0Center for Artificial Intelligence (CAI), King Khalid University, Abha 61421, Saudi ArabiaMachine learning models are built using training data, which is collected from human experience and is prone to bias. Humans demonstrate a cognitive bias in their thinking and behavior, which is ultimately reflected in the collected data. From Amazon’s hiring system, which was built using ten years of human hiring experience, to a judicial system that was trained using human judging practices, these systems all include some element of bias. The best machine learning models are said to mimic humans’ cognitive ability, and thus such models are also inclined towards bias. However, detecting and evaluating bias is a very important step for better explainable models. In this work, we aim to explain bias in learning models in relation to humans’ cognitive bias and propose a wrapper technique to detect and evaluate bias in machine learning models using an openly accessible dataset from UCI Machine Learning Repository. In the deployed dataset, the potentially biased attributes (PBAs) are gender and race. This study introduces the concept of alternation functions to swap the values of PBAs, and evaluates the impact on prediction using KL divergence. Results demonstrate females and Asians to be associated with low wages, placing some open research questions for the research community to ponder over.https://www.mdpi.com/2076-3417/11/14/6271machine learning biasbias detectionbias evaluationKL divergenceexplainable modelscognitive bias |
| spellingShingle | Salem Alelyani Detection and Evaluation of Machine Learning Bias Applied Sciences machine learning bias bias detection bias evaluation KL divergence explainable models cognitive bias |
| title | Detection and Evaluation of Machine Learning Bias |
| title_full | Detection and Evaluation of Machine Learning Bias |
| title_fullStr | Detection and Evaluation of Machine Learning Bias |
| title_full_unstemmed | Detection and Evaluation of Machine Learning Bias |
| title_short | Detection and Evaluation of Machine Learning Bias |
| title_sort | detection and evaluation of machine learning bias |
| topic | machine learning bias bias detection bias evaluation KL divergence explainable models cognitive bias |
| url | https://www.mdpi.com/2076-3417/11/14/6271 |
| work_keys_str_mv | AT salemalelyani detectionandevaluationofmachinelearningbias |