2 Out Of 3 Ain’t Bad…Right?
More and more studies continue to come out regarding secret algorithm risk assessments. A recent study from Indiana courts stated: Bechtel et al. (2016) have conducted a meta-analysis of 16 studies testing the predictive validity of pretrial risk assessment tools. Conclusion: “the relative strength of the ability to predict pretrial misconduct (FTA and rearrest) outcomes is modest.” 1
That same report out of Indiana regarding validating the risk assessment tool being used in several pilot counties stated: Of the 562 arrestees studied by IUPUI:
Low Risk Arrestees: Only 8.4% had either FTA/arrest (any failure) during pretrial supervision period
Moderate Risk Arrestees: About 33% had either FTA/arrest (any failure) during pretrial supervision period
High Risk Arrestees: About 52.5% had either FTA/arrest (any failure) during pretrial supervision period
For “high risk” defendants the risk assessment correctly predicted just 1 out of 2 cases. “Moderate risk” fared slight better predicting 2 out of 3 correctly. The Indiana study stated 18% of the arrestee volume was “high risk” and 46% was “moderate risk”. Essentially, Indiana University – Purdue found the Indiana risk assessment tool accurately predicts pretrial behavior only about 68% of the time.
I never like to be a glass half empty person despite what my wife might say. I guess you could say “2 out of 3 ain’t bad” courtesy of Meatloaf (a 70’s/80’s rocker, not the food). After all, if my stock broker accurately predicted stocks 2 out of 3 times, my IRA would do extremely well every year. As a coach I would have enough wins in a team sport to likely make the playoffs (and win a best of 3 competition) each year. I would pass a test (depending on the school). So, I guess two out of three ain’t bad. Sometimes.
Other times, well, not so good…. If US businesses and industries used pretrial risk assessment algorithms to make decisions there would be:
45 billion pieces of mail lost or delivered to the wrong location annually
If every Google Maps user used the app just once annually Google would incorrectly provide directions for more than 300 million trips
3 billion wrong meals would be served by restaurants each year
Over 14 billion debit card transactions, each year, charged to the wrong account
70 million failed surgeries performed annually
In the criminal justice system, risk assessments would wrongly predict pretrial release behavior for about 3 million defendants every year! That’s 3 million defendants committing more crimes and failing to appear in court after their release.
Conversely, risk assessments would also wrongly predict and hold half of all “high risk” defendants. Wait, that would mean a 100% failure rate for all high-risk defendants? If a judge agrees to hold all high-risk defendants based on the algorithm score, the judge would wrongly hold about 50% of the high-risk population who would not commit more crime and who would attend all court dates if released. However, if the judge agreed to release all high-risk defendants, 50% would commit more crime and abscond from the law. So, the question becomes, should a judge just use a flip of the coin or a risk assessment for high-risk defendants?
Several states, counties and jurisdictions are now concluding Risk Assessments are not the panacea advocates proclaim them to be. Independent research finds risk assessments are racially biased and do not accurately predict pretrial behavior:
Economist and Legal Scholar, Megan Stevenson’s findings in “ASSESSING RISK ASSESSMENT IN ACTION Megan Stevenson December 8, 2017”: “Proponents of risk assessment tools claim that they can dramatically reduce incarceration without harming public safety. Critics claim that risk assessment will exacerbate racial disparities. Despite extensive and heated rhetoric, there is virtually no evidence on how use of this “evidence-based” tool affects key outcomes such as incarceration rates, crime, or racial disparities.” Ms. Stevenson went on to state, regarding Kentucky risk assessment use: “failures-to-appear and pretrial crime increased as well.”
In a research article entitled: “Machine Bias – There’s software used across the country to predict future criminals. And it’s biased against blacks.” by Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, ProPublica May 23, 2016, the researchers found the “risk assessment algorithm was somewhat more accurate than a coin flip.”
How can an empirical, evidence-based, scientifically developed, secret algorithm that considers important factors such as: if a defendant is male, does not have a diploma or GED, does not have a phone, lives with someone other than spouse or children, does not own an automobile, is not employed, and other factors associated with the poor only be “somewhat more accurate than a flip of the coin”?
Alaska implemented bail reform and risk assessments about 3 years ago. Now the Governor is proposing legislation to reverse it because it does not work. Crime is up as are pretrial release failures. Under a new proposal, Governor Dunleavy stated: “Judges would decide bail and pretrial release instead of following a scientific matrix.” 2
A Maryland news report found since bail was reformed: The number of arrestees held without bail at Central Booking increased by 26 percent year over year from March 2017 to March 2018. 3
Yakima County WA is demanding changes after implementing bail reform and risk assessments. Prosecuting attorney Joseph Brusic said, “Yakima county has spent over a million dollars of taxpayer money on this particular program and I cannot say we are safer now then we were before the program.” 4
New Mexico news station KOAT “Target 7” recently reported: We checked with metro court. A spokeswoman says about 3,800 people were busted for felonies last year and got out of jail. They were supposed to be under supervision but 1/3 were no shows. “This of course is the nightmare the judicial system didn’t want to see because they tried to implement these new rules, which generally are in line with what’s going on around the country,” said KOAT legal expert John Day. 5 Wait, I thought 2 out of 3 ain’t bad?
Bail reform advocates can point to New Jersey. There are no reports of new criminal activity, failure to appear, or adverse pretrial release behavior. Of course, that is because New Jersey has conveniently decided they will not provide any pretrial statistics except jail population data since implemented in 2017. Why? If risk assessments can accurately predict pretrial behavior, why not share the data? New Jersey’s monthly report does state that prosecutors filed detention motions in 49% of the cases. Does that mean the risk assessment used by New Jersey is providing a “high-risk” score for almost half of all booked into jails? Looking further into the monthly report, only about half of the detention motions were granted by judges. Does that mean the risk assessment was wrong 50% of the time? Very confusing. Or, should judges just flip a coin rather than force New Jersey taxpayers to pay over $130 million in its first year of implementation?
While the pretrial release system the US has utilized for over 200 years is not perfect, and needs some improvement, it is clear that secret formula risk assessments that provide a false sense of security are not the answer. Should we also abolish our jury system because a jury wrongly convicts an innocent person? Maybe bail elimination advocates also prefer replacing the jury system with a secret computer. Is that next on the criminal justice reform horizon?
Leave judicial discretion in place and leave it to the local jurisdictions who know their local needs better than any Federal or State legislator.
- http://indianacourts.us/times/2019/01/pretrial-part2/
- https://www.adn.com/politics/2019/01/24/dunleavy-opens-his-war-on-criminals-with-push-for-sb-91-repeal/
- https://www.wbaltv.com/article/numbers-raise-questions-about-move-to-reform-bail-in-maryland/20955205
- https://kimatv.com/news/local/judges-threaten-to-abandon-pre-trial-release-program-if-changes-arent-made?fbclid=IwAR1qpUIadN1wyIRm3CNm16L0yNIklGm952OusahrfJ9ypwpCJks11KXhUhI
- https://www.koat.com/article/alarming-number-of-suspects-are-no-shows-at-court/26080933
