CT_March-April_2022_Mag_Web

NIJ Update

the accuracies of models improved, so did model fairness. Interestingly, although most models received fairness penalties, race alone was not identified as a significant indicator of recidivism. This result, as well as the increase in penalty size for females across years, will be evaluated in future research. Further exploration is needed to better understand how this fairness was reflected in the Challenge and what implications these fairness results have for the field. The successful completion and initial review of the results from the Challenge demonstrate the value of open data and open competition approaches for facilitating research within departments of corrections or community supervision. Further examination is needed to: –– Identify and understand gender differences in risk assessments and the support provided for these individuals while under community supervision. –– Unpack penalty scores and understand the proper balance between fair and accurate forecasts. NIJ intends to address these research questions along with practical implications of the Challenge, discussing the balance between improved precision and practical improvement, and a meta- analysis of the relevant variables and modeling techniques identified by winners in future reports and articles. With the Challenge concluded, NIJ is seeking to encourage discussion on reentry, bias, fairness, measurement and algorithm advancement.

Exhibit 2: Penalty Size. Average penalty for winning submissions compared to all submissions for females and males across years. Averages are calculated only to include submissions that receive a penalty.

Conclusion and potential next steps The winning forecasts performed substantially better than random chance and naïve demographic models. The differences in accuracy between the winning and naïve models are likely attributed to the utilization of more advanced statistical techniques (for example, regression, random forest, neural networks) and incorporation of additional data from the Georgia Department of Community Supervision beyond the demographics used in the naïve models. Fairness and accuracy scores were also compared based on the frequency and magnitude of the fairness penalty. Penalties were observed across the winning submissions, although they were considerably smaller than the average penalty size. This suggests as

rates. That means fairness and accuracy scores were reduced when submissions incorrectly forecasted Black or white individuals to recidivate at a higher rate. Exhibit 2 presents the average penalty of submissions that received a fairness penalty for their forecasts of males and females, across years. Across all entries that received a fairness penalty, the penalty increased for females as the years progressed, but the opposite was true for males. This suggests factors that contribute to racial bias in predicting recidivism do not affect males and females in the same way. The winning submissions had lower or no penalties when predicting recidivism for females across the years, but there was no clear trend for winners’ penalties when predicting recidivism among males.

→

Corrections Today March/April 2022 — 15