Legal Empiricists

Attorney Peg Lautenschlager e-mailed me an alert to an article she happened upon in the current edition of Stanford Lawyer (Spring 2010, Volume 82) that explained legal empiricism within the academy. The link is below. I believe it is worth the read. http://stanfordlawyer.law.stanford.edu/2010/05/the-empiricists/

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

Statistical Significance

In research, the effectiveness of a treatment modality is often put in terms of whether or not it is statistically significant. What does statistically significant mean? Let’s take a look.

Most research design starts with what is called the “null hypothesis”. The null hypothesis states that the independent variable (for example, treatment such as alcohol counseling), had no effect on the dependant variable (for example, the recidivism rate). Using legal terms, there is a presumption that the treatment will not help. If that presumption cannot be overcome by statistically significant data, the treatment is considered ineffective. The burden of proof is on the proponent of the treatment modality being studied. The proponent must prove, by statistically significant evidence, that the treatment modality is effective.

For a treatment modality to be considered effective, the data must show that the treatment (alcohol counseling) affected the condition being treated (recidivism rate) and the effect was statistically significant. The difference between the recidivism rate for individuals with no treatment and the recidivism rate for individuals with treatment must be negative (meaning it has declined) and the size of the decline must make it unlikely that it is due to chance alone.

In science, depending on the required rigor of the study, most effects are considered statistically significant if the likelihood that the difference is a result of chance alone is less than 5% or less than 1% (the P-level). For example, one could have a study that showed a reduction of the recidivism rate of 20%, but because of the size and variability of the sample (and other factors), the p-level may be 40%. There is a 40% chance that the measured difference is merely an artifact of pure chance. One would not have much confidence in the result of such a study.

Flip a coin 5 times and if you get 4 heads and 1 tail, there is 60 percent difference in the results (80% heads minus 20% tails). The difference is the result of chance alone, and not due to the construction of the coin. The difference is not statistically significant. If you flipped the coin 1,000 times with the same result, it is far more likely that you have evidence of a trick coin. The larger sample size adds to the chance of finding a difference statistically significant.

Something may be statistically significant, but practically insignificant. For example, imagine a study of a drug for alcoholism involving five million people. The study may show a reduction in alcohol consumption of one tenth of one percent. Because of the size of the sample, this reduction may be statistically significant. But in the practical world of individuals attempting to treat alcohol dependent individuals, this reduction is of no practical value. It is statistically significant, but practically insignificant.

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

OWI Recidivism Reduction and Drunk Driving Reduction

Research often shows relatively small reductions in recidivism rates for OWI offenders who have received alcohol treatment. However, one must remained focused on the ultimate concern, which is not reducing recidivism, but reducing dangerous driving due to driving while intoxicated. Although I could not find any definitive study on the number of times individuals that are arrested for OWI have driven drunk and not been arrested, a survey has estimated that for every operating while intoxicated arrest, there are about 55 operating while intoxicated events. Zador, Paul, Sheila Krawchuk and B. Moore. (1997) “Drinking and Driving Trips, Stops by Police, and Arrests: Analysis of the 1995 National Survey of Drinking and Driving Attitudes and Behavior,” Rockvill, MD: Weststat, Inc.

Therefore, if one starts with 100 offenders and assumes a three year recidivism rate of 15%, and that treatment reduces the recidivism rate by 10%, the number of repeat offenders falls from 15 to 13.5 during a three year period. Assuming that for every arrest, there are 55 drunk driving events, the number of drunk driving events from these 100 offenders fall from 825 to 743. The reduction in numbers of defendants re-offending are multiplied to get a much larger reduction in the number of dangerous driving events (operating while intoxicated). A small statistically significant effect may result in a larger practical effect.

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

Does Treatment Reduce OWI Recidivism?

The scientific literature has identified alcohol treatment as an effective strategy in reducing the recidivism rate for people convicted of operating while intoxicated. Wells-Parker, Elizabeth, Robert Bangert-Drowns, Robert McMillen, and Marsha Williams, (1995) “Final Results from a meta-analysis of remedial interventions with drink/drive offenders”, Addiction, 90, 907-926; DeYoung, David J., (1997) “An evaluation of the effectiveness of alcohol treatment, driver license actions and jail terms in reducing drunk driving recidivism in California” Addiction 92(8), 989-997.

Wells-Parker et al used a meta-analysis of the literature in their study. A meta-analysis applies statistics to a systematic review of the literature. A review of the literature includes searching for relevant scientific articles and then systematically critiquing the articles. Wells-Parker et al estimated a 7-9% reduction in recidivism resulting from treatment. DeYoung’s study showed a somewhat larger reduction, depending on prior convictions for OWI.

What does a 10% reduction in recidivism mean, for example? Assuming a three year recidivism rate for OWI offenders of 15%, defined as an arrest for another OWI within a three year period, fifteen out of 100 offenders will be arrested for OWI within three years. A 10% reduction in the recidivism rate means that instead of a 15% rate we have a 13.5% rate. Instead of fifteen offenders re-offending out of 100, now there will only be 13.5 offenders out of 100 who re-offend (90% of 15).

The studies indicate that treatment probably works, but is not a silver bullet. Currently, the most effective strategies to reduce recidivism for OWI offenders is license restriction coupled with alcohol treatment.

Because of the cost of alcohol treatment, another issue is identifying the type of treatment modality that is most cost-effective—a tougher issue for researchers. I will discuss that one at another time.

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

The Effectiveness of License Restrictions

Only a couple of strategies that attempt to reduce recidivism of operating while intoxicated offenders have been consistently shown, through scientific studies, to be effective in reducing recidivism rates of OWI offenders—alcohol treatment and license restriction. According to the scientific literature, license restriction, including revocation and suspension (and soon ignition interlocks), is the most effective strategy in reducing repeat OWI offenses.

That may surprise practitioners, as we see many individuals who operate a vehicle regardless of any license restriction. Research has shown that 75 percent of individuals with suspended or revoked licenses continue to drive. However, license restrictions act both as a form of incapacitation--drivers cannot legally drive-- and deterrence--drivers view the loss of a license as a negative reinforcement for intoxicated driving. The license restrictions result in offenders who drive less often and drive more carefully, resulting in fewer alcohol related crashes. Hon, Jeffrey, Finding Common Ground: Improving Highway Safety with More Effective Interventions for Alcohol Problems, The George Washington University Medical Center, 2004; Nichols, James L., H. Lawrence Ross, “The Effectiveness of Legal Sanctions in Dealing with Drinking Drivers”, Alcohol, Drugs, and Driving 6(2):33-55.

Should society be concerned with recent developments where violation of license restriction cases involving convicted OWI defendants are not thought serious enough by some prosecutors to warrant the resources required for prosecution? Will no-prosecution policies on such cases lead to a greater level of illegal driving?

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

Law and Science in Action

Here is an instance of the law and science clashing. http://www.nytimes.com/2010/06/12/health/12psych.html?hpw

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.

Practical kernels

I was going to post regarding a recent study of alcohol and drug courts. However, my review of the study raised the question of what, if any, information could be gleaned from the study that would interest judges and other practitioners. That wasn't so easy. What may interest a researcher, may have little immediate practical use for a practitioner. That entry will have to wait.

The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.