Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes
Facial recognition has been shown to have different accuracy for different demographic groups. When setting a threshold to achieve a specific False Match Rate (FMR) on a mixed demographic impostor distribution, some demographic groups can experience a significantly worse FMR. To mitigate this, some...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9733349/ |
| _version_ | 1811341886939463680 |
|---|---|
| author | Esteban Villalobos Domingo Mery Kevin Bowyer |
| author_facet | Esteban Villalobos Domingo Mery Kevin Bowyer |
| author_sort | Esteban Villalobos |
| collection | DOAJ |
| description | Facial recognition has been shown to have different accuracy for different demographic groups. When setting a threshold to achieve a specific False Match Rate (FMR) on a mixed demographic impostor distribution, some demographic groups can experience a significantly worse FMR. To mitigate this, some authors have proposed to use demographic-specific thresholds. However, this can be impractical in an operational scenario, as it would either require users to report their demographic group or the system to predict the demographic group of each user. Both of these options can be deemed controversial since the demographic group is a sensitive attribute. Further, this approach requires listing the possible demographic groups, which can become controversial in itself. We show that a similar mitigation effect can be achieved using non-sensitive predicted soft-biometric attributes. These attributes are based on the appearance of the users (such as hairstyle, accessories, and facial geometry) rather than how the users self-identify. Our experiments use a set of 38 binary non-sensitive attributes from the MAAD-Face dataset. We report results on the Balanced Faces in the Wild dataset, which has a balanced number of identities by race and gender. We compare clustering-based and decision-tree-based strategies for selecting thresholds. We show that the proposed strategies can reduce differential outcomes in intersectional groups twice as effectively as using gender-specific thresholds and, in some cases, are also better than using race-specific thresholds. |
| first_indexed | 2024-04-13T19:01:52Z |
| format | Article |
| id | doaj.art-e71b2e0920284ef08592e51ab68d6c41 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-13T19:01:52Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e71b2e0920284ef08592e51ab68d6c412022-12-22T02:34:05ZengIEEEIEEE Access2169-35362022-01-0110301683017910.1109/ACCESS.2022.31589679733349Fair Face Verification by Using Non-Sensitive Soft-Biometric AttributesEsteban Villalobos0https://orcid.org/0000-0002-6026-3756Domingo Mery1https://orcid.org/0000-0003-4748-3882Kevin Bowyer2https://orcid.org/0000-0002-7562-4390Department of Computer Science, School of Engineering. Pontificia Universidad Catolica de Chile, Macul, ChileDepartment of Computer Science, School of Engineering. Pontificia Universidad Catolica de Chile, Macul, ChileDepartment of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, USAFacial recognition has been shown to have different accuracy for different demographic groups. When setting a threshold to achieve a specific False Match Rate (FMR) on a mixed demographic impostor distribution, some demographic groups can experience a significantly worse FMR. To mitigate this, some authors have proposed to use demographic-specific thresholds. However, this can be impractical in an operational scenario, as it would either require users to report their demographic group or the system to predict the demographic group of each user. Both of these options can be deemed controversial since the demographic group is a sensitive attribute. Further, this approach requires listing the possible demographic groups, which can become controversial in itself. We show that a similar mitigation effect can be achieved using non-sensitive predicted soft-biometric attributes. These attributes are based on the appearance of the users (such as hairstyle, accessories, and facial geometry) rather than how the users self-identify. Our experiments use a set of 38 binary non-sensitive attributes from the MAAD-Face dataset. We report results on the Balanced Faces in the Wild dataset, which has a balanced number of identities by race and gender. We compare clustering-based and decision-tree-based strategies for selecting thresholds. We show that the proposed strategies can reduce differential outcomes in intersectional groups twice as effectively as using gender-specific thresholds and, in some cases, are also better than using race-specific thresholds.https://ieeexplore.ieee.org/document/9733349/Facial recognitionfairnessdifferential outcomessoft-biometric attributes |
| spellingShingle | Esteban Villalobos Domingo Mery Kevin Bowyer Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes IEEE Access Facial recognition fairness differential outcomes soft-biometric attributes |
| title | Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes |
| title_full | Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes |
| title_fullStr | Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes |
| title_full_unstemmed | Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes |
| title_short | Fair Face Verification by Using Non-Sensitive Soft-Biometric Attributes |
| title_sort | fair face verification by using non sensitive soft biometric attributes |
| topic | Facial recognition fairness differential outcomes soft-biometric attributes |
| url | https://ieeexplore.ieee.org/document/9733349/ |
| work_keys_str_mv | AT estebanvillalobos fairfaceverificationbyusingnonsensitivesoftbiometricattributes AT domingomery fairfaceverificationbyusingnonsensitivesoftbiometricattributes AT kevinbowyer fairfaceverificationbyusingnonsensitivesoftbiometricattributes |