Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications
Bias in software systems is a serious threat to human rights: when software makes decisions that allocate resources or opportunities, may disparately impact people based on personal traits (e.g., gender, ethnic group, etc.), systematically (dis)advantaging certain social groups. The cause is very of...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10058507/ |
| _version_ | 1797861349001265152 |
|---|---|
| author | Mariachiara Mecati Marco Torchiano Antonio Vetro Juan Carlos de Martin |
| author_facet | Mariachiara Mecati Marco Torchiano Antonio Vetro Juan Carlos de Martin |
| author_sort | Mariachiara Mecati |
| collection | DOAJ |
| description | Bias in software systems is a serious threat to human rights: when software makes decisions that allocate resources or opportunities, may disparately impact people based on personal traits (e.g., gender, ethnic group, etc.), systematically (dis)advantaging certain social groups. The cause is very often the imbalance of training data, that is, unequal distribution of data between the classes of an attribute. Previous studies showed that lower levels of balance in protected attributes are related to higher levels of unfairness in the output. In this paper we contribute to the current status of knowledge on balance measures as risk indicators of systematic discriminations by studying imbalance on two further aspects: the intersectionality among the classes of protected attributes, and the combination of the target variable with protected attributes. We conduct an empirical study to verify whether: i) it is possible to infer the balance of intersectional attributes from the balance of the primary attributes, ii) measures of balance on intersectional attributes are helpful to detect unfairness in the classification outcome, iii) the computation of balance on the combination of a target variable with protected attributes improves the detection of unfairness. Overall the results reveal positive answers, but not for every combination of balance measure and fairness criterion. For this reason, we recommend selecting the fairness and balance measures that are most suitable to the application context when applying our risk approach to real cases. |
| first_indexed | 2024-04-09T22:00:47Z |
| format | Article |
| id | doaj.art-5618c19cab344636b91236b3bb03141e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T22:00:47Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5618c19cab344636b91236b3bb03141e2023-03-23T23:00:16ZengIEEEIEEE Access2169-35362023-01-0111269962701110.1109/ACCESS.2023.325237010058507Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair ClassificationsMariachiara Mecati0https://orcid.org/0000-0002-0041-1809Marco Torchiano1https://orcid.org/0000-0001-5328-368XAntonio Vetro2https://orcid.org/0000-0003-2027-3308Juan Carlos de Martin3Department of Control and Computer Engineering, Politecnico di Torino, Turin, ItalyDepartment of Control and Computer Engineering, Politecnico di Torino, Turin, ItalyDepartment of Control and Computer Engineering, Politecnico di Torino, Turin, ItalyDepartment of Control and Computer Engineering, Politecnico di Torino, Turin, ItalyBias in software systems is a serious threat to human rights: when software makes decisions that allocate resources or opportunities, may disparately impact people based on personal traits (e.g., gender, ethnic group, etc.), systematically (dis)advantaging certain social groups. The cause is very often the imbalance of training data, that is, unequal distribution of data between the classes of an attribute. Previous studies showed that lower levels of balance in protected attributes are related to higher levels of unfairness in the output. In this paper we contribute to the current status of knowledge on balance measures as risk indicators of systematic discriminations by studying imbalance on two further aspects: the intersectionality among the classes of protected attributes, and the combination of the target variable with protected attributes. We conduct an empirical study to verify whether: i) it is possible to infer the balance of intersectional attributes from the balance of the primary attributes, ii) measures of balance on intersectional attributes are helpful to detect unfairness in the classification outcome, iii) the computation of balance on the combination of a target variable with protected attributes improves the detection of unfairness. Overall the results reveal positive answers, but not for every combination of balance measure and fairness criterion. For this reason, we recommend selecting the fairness and balance measures that are most suitable to the application context when applying our risk approach to real cases.https://ieeexplore.ieee.org/document/10058507/Data biasdata imbalanceintersectionalityalgorithmic fairnessautomated decision-makingdata ethics |
| spellingShingle | Mariachiara Mecati Marco Torchiano Antonio Vetro Juan Carlos de Martin Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications IEEE Access Data bias data imbalance intersectionality algorithmic fairness automated decision-making data ethics |
| title | Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications |
| title_full | Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications |
| title_fullStr | Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications |
| title_full_unstemmed | Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications |
| title_short | Measuring Imbalance on Intersectional Protected Attributes and on Target Variable to Forecast Unfair Classifications |
| title_sort | measuring imbalance on intersectional protected attributes and on target variable to forecast unfair classifications |
| topic | Data bias data imbalance intersectionality algorithmic fairness automated decision-making data ethics |
| url | https://ieeexplore.ieee.org/document/10058507/ |
| work_keys_str_mv | AT mariachiaramecati measuringimbalanceonintersectionalprotectedattributesandontargetvariabletoforecastunfairclassifications AT marcotorchiano measuringimbalanceonintersectionalprotectedattributesandontargetvariabletoforecastunfairclassifications AT antoniovetro measuringimbalanceonintersectionalprotectedattributesandontargetvariabletoforecastunfairclassifications AT juancarlosdemartin measuringimbalanceonintersectionalprotectedattributesandontargetvariabletoforecastunfairclassifications |