Access denied: meaningful data access for quantitative algorithm audits
Independent algorithm audits hold the promise of bringing accountability to automated decision-making. However, third-party audits are often hindered by access restrictions, forcing auditors to rely on limited, low-quality data. To study how these limitations impact research integrity, we conduct au...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Association for Computing Machinery
2025
|
| _version_ | 1826317546839605248 |
|---|---|
| author | Zaccour, J Binns, R Rocher, L |
| author_facet | Zaccour, J Binns, R Rocher, L |
| author_sort | Zaccour, J |
| collection | OXFORD |
| description | Independent algorithm audits hold the promise of bringing accountability to automated decision-making. However, third-party audits are often hindered by access restrictions, forcing auditors to rely on limited, low-quality data. To study how these limitations impact research integrity, we conduct audit simulations on two realistic case studies for recidivism and healthcare coverage prediction. We examine the accuracy of estimating group parity metrics across three levels of access: (a) aggregated statistics, (b) individual-level data with model outputs, and (c) individual-level data without model outputs. Despite selecting one of the simplest tasks for algorithmic auditing, we find that data minimization and anonymization practices can strongly increase error rates on individual-level data, leading to unreliable assessments. We discuss implications for independent auditors, as well as potential avenues for HCI researchers and regulators to improve data access and enable both reliable and holistic evaluations. |
| first_indexed | 2025-03-11T16:55:38Z |
| format | Conference item |
| id | oxford-uuid:b2a9a056-462f-4e10-82b2-b3b92a6afc27 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2025-03-11T16:55:38Z |
| publishDate | 2025 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | oxford-uuid:b2a9a056-462f-4e10-82b2-b3b92a6afc272025-02-19T11:11:30ZAccess denied: meaningful data access for quantitative algorithm auditsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:b2a9a056-462f-4e10-82b2-b3b92a6afc27EnglishSymplectic ElementsAssociation for Computing Machinery2025Zaccour, JBinns, RRocher, LIndependent algorithm audits hold the promise of bringing accountability to automated decision-making. However, third-party audits are often hindered by access restrictions, forcing auditors to rely on limited, low-quality data. To study how these limitations impact research integrity, we conduct audit simulations on two realistic case studies for recidivism and healthcare coverage prediction. We examine the accuracy of estimating group parity metrics across three levels of access: (a) aggregated statistics, (b) individual-level data with model outputs, and (c) individual-level data without model outputs. Despite selecting one of the simplest tasks for algorithmic auditing, we find that data minimization and anonymization practices can strongly increase error rates on individual-level data, leading to unreliable assessments. We discuss implications for independent auditors, as well as potential avenues for HCI researchers and regulators to improve data access and enable both reliable and holistic evaluations. |
| spellingShingle | Zaccour, J Binns, R Rocher, L Access denied: meaningful data access for quantitative algorithm audits |
| title | Access denied: meaningful data access for quantitative algorithm audits |
| title_full | Access denied: meaningful data access for quantitative algorithm audits |
| title_fullStr | Access denied: meaningful data access for quantitative algorithm audits |
| title_full_unstemmed | Access denied: meaningful data access for quantitative algorithm audits |
| title_short | Access denied: meaningful data access for quantitative algorithm audits |
| title_sort | access denied meaningful data access for quantitative algorithm audits |
| work_keys_str_mv | AT zaccourj accessdeniedmeaningfuldataaccessforquantitativealgorithmaudits AT binnsr accessdeniedmeaningfuldataaccessforquantitativealgorithmaudits AT rocherl accessdeniedmeaningfuldataaccessforquantitativealgorithmaudits |