A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems
We are interested in developing a data-driven method to evaluate race-induced biases in law enforcement systems. While the recent works have addressed this question in the context of police-civilian interactions using police stop data, they have two key limitations. First, bias can only be properly...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/155468 |
| _version_ | 1826198051185754112 |
|---|---|
| author | Han, Jessy Xinyi |
| author2 | Shah, Devavrat |
| author_facet | Shah, Devavrat Han, Jessy Xinyi |
| author_sort | Han, Jessy Xinyi |
| collection | MIT |
| description | We are interested in developing a data-driven method to evaluate race-induced biases in law enforcement systems. While the recent works have addressed this question in the context of police-civilian interactions using police stop data, they have two key limitations. First, bias can only be properly quantified if true criminality is accounted for in addition to race, but it is absent in prior works1. Second, law enforcement systems are multi-stage and hence it is important to isolate the true source of bias within the “causal chain of interactions” rather than simply focusing on the end outcome; this can help guide reforms. In this work, we address these challenges by presenting a multi-stage causal framework incorporating criminality. We provide a theoretical characterization and an associated datadriven method to evaluate (a) the presence of any form of racial bias, and (b) if so, the primary source of such a bias in terms of race and criminality. Our framework identifies three canonical scenarios with distinct characteristics: in settings like (1) airport security, the primary source of observed bias against a race is likely to be bias in law enforcement against innocents of that race; (2) AI-empowered policing2, the primary source of observed bias against a race is likely to be bias in law enforcement against criminals of that race; and (3) police-civilian interaction, the primary source of observed bias against a race could be bias in law enforcement against that race or bias from the general public in reporting (e.g. via 911 calls) against the other race. Through an extensive empirical study using police-civilian interaction (stop) data and 911 call data, we find an instance of such a counter-intuitive phenomenon: in New Orleans, the observed bias is against the majority race and the likely reason for it is the over-reporting (via 911 calls) of incidents involving the minority race by the general public. |
| first_indexed | 2024-09-23T10:58:26Z |
| format | Thesis |
| id | mit-1721.1/155468 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:58:26Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1554682024-07-09T03:42:50Z A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems Han, Jessy Xinyi Shah, Devavrat Massachusetts Institute of Technology. Institute for Data, Systems, and Society We are interested in developing a data-driven method to evaluate race-induced biases in law enforcement systems. While the recent works have addressed this question in the context of police-civilian interactions using police stop data, they have two key limitations. First, bias can only be properly quantified if true criminality is accounted for in addition to race, but it is absent in prior works1. Second, law enforcement systems are multi-stage and hence it is important to isolate the true source of bias within the “causal chain of interactions” rather than simply focusing on the end outcome; this can help guide reforms. In this work, we address these challenges by presenting a multi-stage causal framework incorporating criminality. We provide a theoretical characterization and an associated datadriven method to evaluate (a) the presence of any form of racial bias, and (b) if so, the primary source of such a bias in terms of race and criminality. Our framework identifies three canonical scenarios with distinct characteristics: in settings like (1) airport security, the primary source of observed bias against a race is likely to be bias in law enforcement against innocents of that race; (2) AI-empowered policing2, the primary source of observed bias against a race is likely to be bias in law enforcement against criminals of that race; and (3) police-civilian interaction, the primary source of observed bias against a race could be bias in law enforcement against that race or bias from the general public in reporting (e.g. via 911 calls) against the other race. Through an extensive empirical study using police-civilian interaction (stop) data and 911 call data, we find an instance of such a counter-intuitive phenomenon: in New Orleans, the observed bias is against the majority race and the likely reason for it is the over-reporting (via 911 calls) of incidents involving the minority race by the general public. S.M. 2024-07-08T18:53:18Z 2024-07-08T18:53:18Z 2024-05 2024-06-03T20:46:11.030Z Thesis https://hdl.handle.net/1721.1/155468 Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Han, Jessy Xinyi A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title | A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title_full | A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title_fullStr | A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title_full_unstemmed | A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title_short | A Causal Framework to Evaluate Racial Bias in Law Enforcement Systems |
| title_sort | causal framework to evaluate racial bias in law enforcement systems |
| url | https://hdl.handle.net/1721.1/155468 |
| work_keys_str_mv | AT hanjessyxinyi acausalframeworktoevaluateracialbiasinlawenforcementsystems AT hanjessyxinyi causalframeworktoevaluateracialbiasinlawenforcementsystems |