Brendan I. Koerner has written an intelligent article about Alcoholics Anonymous in Wired, entitled “Secret of AA: After 75 years We Don’t Know How It Works.” The article describes recent research on the effectiveness of AA (It doesn’t appear to work when ordered by a court), but also discusses the physiology of alcohol addiction and the neurological effects of the addiction. The article further discusses some of the possible mechanisms that may be at work in AA’s success in reducing addiction to alcohol. One point made by the article, which legal practitioners must keep in mind, is that 90 percent of alcohol dependent people who had stopped drinking will drink again. If you are interested in the seeming intractability of the problems we see related to the abuse of alcohol, this article is worth your time. http://www.wired.com/magazine/2010/06/ff_alcoholics_anonymous/
June 30, 2010
Intelligent Article on Alcoholics Anonymous
June 28, 2010
Jail has Minimal Effect on OWI Recidivism
Research studies evaluating jail as a judicial intervention to reduce the recidivism rate of operating while intoxicated defendants have overwhelmingly found it to be ineffective.[i] However, limited research evidence exists that a short (two day) jail sentence may result in a reduced recidivism rate.[ii]
Most of these studies were done in the 1990s. The most recent study I could find, found that jail combined with treatment was the most effective strategy among three studied. However, the jail sentences were combined with treatment, so it was not possible to discern the effect of the jail component alone. The study confirmed what other studies had found—that the length of jail did not seem to be related to offenders’ rates of recidivism.[iii] I found no research that considered the effect of prison sentences on the recidivism rate of OWI offenders.
Attempting to deter an offender from re-offending is but one reason for a sentence that involves incarceration. General deterrence of other potential offenders is another reason. Jail sentences may have an educational effect on the general public by making drunk driving less culturally acceptable.[iv] Through a jail sentence, society instills the message that drunk driving is wrong and is not an acceptable behavior.
Incapacitation is another reason for incarcerating drunk drivers. An offender is at much lower risk to re-offend if incarcerated. (However, it does happen that an offender on work-release re-offends.) With some offenders, the risk to the public from their repeated drinking and driving is so great that their treatment must be in a confined setting to protect the public.
Another reason for incarcerating drunk drivers derives not from the possible consequences of the punishment, but from the deontological view that drunk driving is deserving of punishment because it is morally wrong—period. Someone, by driving drunk, has put other individuals at risk of harm or death through negligent or reckless drinking and driving behavior. The argument goes, that the wrong itself requires retribution, regardless of any practical effect from the punishment. The deontological argument doesn’t consider behavioral changes to the defendant or others, and therefore it does not lend itself to measurement and hence empirical investigation. I will return to this topic at another time from a different angle.
The problem with incarceration is that it is not free. Society must pay for jail and prison space and their attending direct costs, as well as for the collateral costs to the families of the incarcerated. In any fiscally healthy society, the cost of government must be scrutinized to ensure tax dollars are used wisely. Every tax-dollar spent on incarceration is a tax-dollar not available for another societal need. Judges cannot escape that truth when sentencing.
Does the next dollar spent on incarceration increase community safety or the perception of justice by at least a dollar? Is a perception of justice alone enough to justify the cost of this punishment? Those are the questions that are being asked more frequently as our society struggles to address its increasing public debt. I believe they are the correct questions.
[i] Voas, Robert B and Deborah A. Fisher, 2001, “Court Procedures for Handling Intoxicated Drivers”, National Institute of Alcohol Abuse and Alcoholism.
[iv] James L. Nichols, and J. Laurence Ross, (1990) “The Effectiveness of Legal Sanctions in Dealing with Drinking Drivers,” Alcohol, Drugs, and Driving, 6(2) 33-55.
June 24, 2010
Why Aren't Drunk Drivers Deterred?
According to the research, judicial interventions result in only small reductions, if any, in the OWI recidivism rate. Of course, judicial interventions only involve individuals with a prior OWI. Judges haven’t had any prior involvement with the largest group of OWI offenders—OWI 1sts.
For that group, society must rely on simple deterrence principles alone to stop drunk driving. The classical deterrence principles of Cesare Beccaria, that are still argued every day in court rooms in Wisconsin, demand that the expected costs of drunk driving must exceed the expected value of drunk driving.
This theory is underpinned by the rational actor model of human behavior. The rational actor model of human behavior has been the foundation of economics and law for centuries. (It is almost certainly wrong, but that is a discussion for another day, and does not affect the analysis below.)
A rational actor makes the decision about whether or not to drive drunk by comparing the expected costs and the expected benefits of drunk driving. If the expected benefits exceed the costs, then the rational actor drives drunk. If not, then he finds a different way home or stops drinking before he is intoxicated. The decision point occurs either at the time the next drink will cause him to be intoxicated, or if he is intoxicated, when he decides to drive.
The expected costs can be calculated as follows: (The probability of being arrested for drunk driving multiplied by the probability of being convicted and punished if arrested multiplied by the expected punishment for drunk driving) plus (the probability of being arrested for injury or homicide by intoxicated use multiplied by the probability of being convicted and punished if arrested multiplied by the expected punishment for injury or homicide by intoxicated use) plus ( the probability that one will be injured or killed themselves and the costs of those results).
Research has shown that more than 90% of those arrested for OWI are ultimately convicted. This number has increased greatly over the last twenty five years with changes in the law regarding implied consent, forced blood draws, and the use of blood tests themselves. The punishment levels, including incarceration and fines, have increased dramatically through the years. The punishment for homicide by intoxicated use has increased many fold during my legal career.
What hasn’t changed much in Wisconsin, is the probability of being arrested for drunk driving (If an accident results, the chances of being arrested are quite high). Using the one published survey figure of one drunk driving arrest for every 55 drunk driving incidents, the chance of being arrested for drunk driving is less than 2%.
Based on the low arrest rate for drunk driving, a rational actor can conclude that one will not be arrested if he or she drives drunk. The rational actor will also correctly calculate that there is a low probability of getting into an accident. Therefore, the expected punishment for drunk driving is rationally quite low primarily because of the low probability of being arrested. The classical theory of deterrence fails because the expected costs of driving drunk are less than the expected benefits.
One attempt to change this calculus was to rescind the ban in Wisconsin on sobriety check-points. The hope was to raise the expectation of being arrested for drunk driving. The legislature, after balancing the interests as they should, decided not to lift the ban on such checkpoints.
However, it is clear to me to increase deterrence of operating while intoxicated, policies must be implemented to increase the probability of arrest for drunk driving. Changes in penalties, treatment, and other judicial interventions are all small potatoes compared to changes in the expectation of arrest.
June 22, 2010
Legal Empiricists
Attorney Peg Lautenschlager e-mailed me an alert to an article she happened upon in the current edition of Stanford Lawyer (Spring 2010, Volume 82) that explained legal empiricism within the academy. The link is below. I believe it is worth the read. http://stanfordlawyer.law.
June 21, 2010
Statistical Significance
In research, the effectiveness of a treatment modality is often put in terms of whether or not it is statistically significant. What does statistically significant mean? Let’s take a look.
Most research design starts with what is called the “null hypothesis”. The null hypothesis states that the independent variable (for example, treatment such as alcohol counseling), had no effect on the dependant variable (for example, the recidivism rate). Using legal terms, there is a presumption that the treatment will not help. If that presumption cannot be overcome by statistically significant data, the treatment is considered ineffective. The burden of proof is on the proponent of the treatment modality being studied. The proponent must prove, by statistically significant evidence, that the treatment modality is effective.
For a treatment modality to be considered effective, the data must show that the treatment (alcohol counseling) affected the condition being treated (recidivism rate) and the effect was statistically significant. The difference between the recidivism rate for individuals with no treatment and the recidivism rate for individuals with treatment must be negative (meaning it has declined) and the size of the decline must make it unlikely that it is due to chance alone.
In science, depending on the required rigor of the study, most effects are considered statistically significant if the likelihood that the difference is a result of chance alone is less than 5% or less than 1% (the P-level). For example, one could have a study that showed a reduction of the recidivism rate of 20%, but because of the size and variability of the sample (and other factors), the p-level may be 40%. There is a 40% chance that the measured difference is merely an artifact of pure chance. One would not have much confidence in the result of such a study.
Flip a coin 5 times and if you get 4 heads and 1 tail, there is 60 percent difference in the results (80% heads minus 20% tails). The difference is the result of chance alone, and not due to the construction of the coin. The difference is not statistically significant. If you flipped the coin 1,000 times with the same result, it is far more likely that you have evidence of a trick coin. The larger sample size adds to the chance of finding a difference statistically significant.
Something may be statistically significant, but practically insignificant. For example, imagine a study of a drug for alcoholism involving five million people. The study may show a reduction in alcohol consumption of one tenth of one percent. Because of the size of the sample, this reduction may be statistically significant. But in the practical world of individuals attempting to treat alcohol dependent individuals, this reduction is of no practical value. It is statistically significant, but practically insignificant.June 20, 2010
OWI Recidivism Reduction and Drunk Driving Reduction
Research often shows relatively small reductions in recidivism rates for OWI offenders who have received alcohol treatment. However, one must remained focused on the ultimate concern, which is not reducing recidivism, but reducing dangerous driving due to driving while intoxicated. Although I could not find any definitive study on the number of times individuals that are arrested for OWI have driven drunk and not been arrested, a survey has estimated that for every operating while intoxicated arrest, there are about 55 operating while intoxicated events. Zador, Paul, Sheila Krawchuk and B. Moore. (1997) “Drinking and Driving Trips, Stops by Police, and Arrests: Analysis of the 1995 National Survey of Drinking and Driving Attitudes and Behavior,” Rockvill, MD: Weststat, Inc.
Therefore, if one starts with 100 offenders and assumes a three year recidivism rate of 15%, and that treatment reduces the recidivism rate by 10%, the number of repeat offenders falls from 15 to 13.5 during a three year period. Assuming that for every arrest, there are 55 drunk driving events, the number of drunk driving events from these 100 offenders fall from 825 to 743. The reduction in numbers of defendants re-offending are multiplied to get a much larger reduction in the number of dangerous driving events (operating while intoxicated). A small statistically significant effect may result in a larger practical effect.June 15, 2010
Does Treatment Reduce OWI Recidivism?
The scientific literature has identified alcohol treatment as an effective strategy in reducing the recidivism rate for people convicted of operating while intoxicated. Wells-Parker, Elizabeth, Robert Bangert-Drowns, Robert McMillen, and Marsha Williams, (1995) “Final Results from a meta-analysis of remedial interventions with drink/drive offenders”, Addiction, 90, 907-926; DeYoung, David J., (1997) “An evaluation of the effectiveness of alcohol treatment, driver license actions and jail terms in reducing drunk driving recidivism in California” Addiction 92(8), 989-997.
Wells-Parker et al used a meta-analysis of the literature in their study. A meta-analysis applies statistics to a systematic review of the literature. A review of the literature includes searching for relevant scientific articles and then systematically critiquing the articles. Wells-Parker et al estimated a 7-9% reduction in recidivism resulting from treatment. DeYoung’s study showed a somewhat larger reduction, depending on prior convictions for OWI.
What does a 10% reduction in recidivism mean, for example? Assuming a three year recidivism rate for OWI offenders of 15%, defined as an arrest for another OWI within a three year period, fifteen out of 100 offenders will be arrested for OWI within three years. A 10% reduction in the recidivism rate means that instead of a 15% rate we have a 13.5% rate. Instead of fifteen offenders re-offending out of 100, now there will only be 13.5 offenders out of 100 who re-offend (90% of 15).
The studies indicate that treatment probably works, but is not a silver bullet. Currently, the most effective strategies to reduce recidivism for OWI offenders is license restriction coupled with alcohol treatment.
Because of the cost of alcohol treatment, another issue is identifying the type of treatment modality that is most cost-effective—a tougher issue for researchers. I will discuss that one at another time.
June 13, 2010
The Effectiveness of License Restrictions
Only a couple of strategies that attempt to reduce recidivism of operating while intoxicated offenders have been consistently shown, through scientific studies, to be effective in reducing recidivism rates of OWI offenders—alcohol treatment and license restriction. According to the scientific literature, license restriction, including revocation and suspension (and soon ignition interlocks), is the most effective strategy in reducing repeat OWI offenses.
That may surprise practitioners, as we see many individuals who operate a vehicle regardless of any license restriction. Research has shown that 75 percent of individuals with suspended or revoked licenses continue to drive. However, license restrictions act both as a form of incapacitation--drivers cannot legally drive-- and deterrence--drivers view the loss of a license as a negative reinforcement for intoxicated driving. The license restrictions result in offenders who drive less often and drive more carefully, resulting in fewer alcohol related crashes. Hon, Jeffrey, Finding Common Ground: Improving Highway Safety with More Effective Interventions for Alcohol Problems, The George Washington University Medical Center, 2004; Nichols, James L., H. Lawrence Ross, “The Effectiveness of Legal Sanctions in Dealing with Drinking Drivers”, Alcohol, Drugs, and Driving 6(2):33-55.
Should society be concerned with recent developments where violation of license restriction cases involving convicted OWI defendants are not thought serious enough by some prosecutors to warrant the resources required for prosecution? Will no-prosecution policies on such cases lead to a greater level of illegal driving?
June 12, 2010
Law and Science in Action
June 11, 2010
Practical kernels
June 10, 2010
Libraries and librarians
I love libraries and librarians. I was going to write “good librarians”, but after using libraries my entire life, I can’t remember a bad experience with a librarian. (I find that statement to be quite incredulous, even to myself, understanding the bell curve of human behavior and performance in any profession.) I am almost always impressed with their helpfulness and ability to navigate the labyrinths of a library and other sources of stored knowledge.
UW-Madison Law Librarian Bonnie Schucha’s WisBlawg, http://www.law.wisc.edu/blogs/wisblawg/ has interesting information regarding Wisconsin law, including recent articles by UW-Madison law professors.
State Law Librarian Jane Colwin and her staff at the Wisconsin State Law Library are impressive. Their service to patrons rivals the best of the best, in both the public and private sector. Find them at: http://wilawlibrary.gov/ .
My love of librarians, I am sure, is primed by my nostalgia for the times I spent exploring new worlds, not available to me on the farm, in a quiet carrel among the musty stacks in the Memorial Union Library and other similar libraries. My thanks to all the librarians who have helped me over the years.
“There were nights in the library when I would look at the footnotes in some heroically constructed tome, and know that the spirit of the rigorous scholar who had written it must know its regret, for each footnote is a step onto deeper meaning, which terrifies the order of progression of the scholar’s logic, until there is no point in experience, nor any word, from which one cannot set out to explore the totality of the All, if indeed there be an All and not an expanding mystery.” Norman Mailer, The Deer Park
June 9, 2010
Recidivism
Repeat drunk drivers have been the focus of legislative action on drunk driving for years, primarily by increasing penalties. Why are they the focus? Statistics from the Court system show that approximately 40% of the individuals convicted of operating while intoxicated had been convicted at least once before. That means approximately 60% had no prior contact with the justice system for an operating while intoxicated case. (The lack of contact with this 60% is one reason the Court system cannot, alone, solve any operating while intoxicated problem.)
For example, a drunk driver may be considered a recidivist after he or she gets arrested again for a drunk driving, or gets convicted of drunk driving, or gets convicted of any traffic crime, or gets convicted of any crime. Therefore, the definition of the triggering event that makes one a recidivist is important.
Secondly, the time period over which the rate is calculated is important. There are one year recidivism rates, (i.e. how many offenders re-offender over a one year period), two year recidivism rates, three year recidivism rates, on up, to lifetime recidivism rates (how many people re-offend in their lifetimes). In reality, most studies will involve three year or less rates, as researchers want to evaluate programs as soon as enough data has been accumulated to make a judgment on the efficacy of the program.
About one year ago, I calculated recidivism rates on OWI’s in
Two year – 9.34%
Three year – 15.57%
June 7, 2010
A Challenging Task
In July, Wisconsin law relating to operating a motor vehicle while intoxicated will be changing. The legislature has responded to calls to reduce the property damage, injuries, and deaths resulting from unsafe driving by individuals who are intoxicated. I will use the next several entries discussing this issue.
Researchers have estimated that legally intoxicated drivers are fifteen times more likely than non-drinking drivers to cause a fatal crash. Individuals with alcohol in their system are eight times more likely than drivers without alcohol in their system to cause a fatal crash. Levitt, Steven D., and Jack Porter, 1999, “Estimating the Effect of Alcohol on Driver Risk Using Only Fatal Accident Statistics”, National Bureau of Economic Research, Inc., NBR Working Paper, No. 6944. http://ideas.repec.org/p/nbr/nberwo/6944.html The legislative changes are an attempt to reduce the risk of citizens being injured or killed by an intoxicated driver.
The task of reducing intoxicated drivers is a difficult one. First, the problem involves changing individual habits related to the use of intoxicants and driving. As anyone knows, changing anyone’s behavior, including our own, is very difficult. Further, in Wisconsin, drinking alcohol is part of the warp and woof of our culture. Cultural change is also difficult.
But secondly, the implementation of government policies related to operating while intoxicated is also difficult. The effectiveness of government policy is often largely dependent on the effectiveness of its implementation. (See. Eggers, William D. and John O’Leary, If We Can Put a Man on the Moon… Getting Big Things Done in Government, Harvard Business Press, 2009. )
The implementers of operating while intoxicated reduction policies are not only in different agencies within different levels of the same branch of government, but are in different branches of government altogether. The detection of drunk drivers is the responsibility of police agencies that are part of state, county, city, or town government. The prosecution of non-criminal cases (most operating while intoxicated first offenses), is the responsibility of individual local prosecutors that again are part of state, county, city, or town government. The prosecution of criminal operating while intoxicated cases is the responsibility of individual local district attorney offices that are part of state and county government. Public and private attorneys represent and counsel the defendants.
The adjudication of these cases, including imposing sentences, is the responsibility of individual municipal and circuit court judges in the judicial branch. Any prison or probationary sentence will then involve correctional staff and probation agents with the Wisconsin Department of Corrections. Any jail sentence will involve the policies of the local sheriff’s department. Any required counseling involves private and public agencies and counselors. The Wisconsin Department of Transportation is involved with any license revocation, and issuing of occupational licenses. Private companies will be providing ignition interlock devices.
For this system to be effective, i.e. reduce the likelihood of a re-offense, every individual actor within each level must know what he or she should do and how to do it. Each individual actor must understand his or her role in the system, and not, as Eggers and O’Leary warn, operate within their own “silos” without coordination with the other actors.
The Wisconsin legislature has provided the judiciary with additional tools to attempt to address the problem. Two of these tools have a focus on treatment. Courts will now be allowed to sentence defendants to a probationary sentence on second and third offenses—sentences that are not available now. Further, counties will be able to opt into reduced jail sentences conditioned on successful completion of probation and treatment.
Probation and treatment will cost taxpayer money. Success is in no way a foregone conclusion. None of the actors alone will be responsible for any success or failure. The problem is too big and the system too large. Success will only be possible if participants understand the entire system and then intelligently execute their integrated part.
June 4, 2010
Why Should We Care
Why should anyone in law care about using science and research-based practices in the law? Because this approach has been proven to be an effective tool in discovering how things work and why things don’t work. The increase in our life expectancy and our economic standard of living are all direct fruits of science and research.
Here are some reasons we should care about using science and research-based practice in the law:
1. 1. The citizens of the State of Wisconsin spend billions of dollars each year in Wisconsin on such things as incarceration and probation (over $1.1 billion for the Department of Corrections alone), juvenile justice, child protective services, other court ordered counseling and programs, and have been for years. We only recently have started to look at what we are doing with a critical eye, trying to ascertain if these practices are working or not, and if they are working, why they work. In an era of considerable budget deficits and a threatening national debt, we owe our society and future generations nothing less than the best we can give them.
June 3, 2010
Peer Review
Some of the fondest memories I have of graduate school were of the times that I spent with my fellow graduate students in our cramped office, bantering back and forth as we critiqued one another’s ideas, at times someone standing to write an equation or outline a train of thought on a blackboard. We were instilled with the idea that scientific inquiry mandated a responsibility to point out faulty thinking to our peers, and to accept our peers pointing out our faulty thinking. The criticisms required thoughtful responses, not ad hominem arguments, nor merely standing on dogmatic platitudes, and certainly not sitting quietly and not participating. Ultimately, I believe this attitude best reflects what “peer review” is all about.
One of the criteria for being an “evidence-based” practice is that the research had been “peer-reviewed”. “Peer-reviewed” means that the research has been reviewed by individuals not associated with the research but with expertise in the research topic area, usually before the research is published. The purpose of peer-review of research is to catch mistakes made in the research protocol, or the conclusions drawn from the research. Peer review provides another perspective on the research and the conclusions drawn from the research.
We have appellate review in the court system. A miniscule percentage of my decisions are ever reviewed by an appellate court. (Hopefully, mostly because the litigants thought the decisions were correct, but I am sure the cost of appealing a decision discourages much appellate review.) A small percentage of appellate cases are ever reviewed by the Wisconsin Supreme Court. And even if one of my decisions is reviewed by an appellate court, the scope of the review is often limited. Many of my decisions, as a trial court judge, are reviewed under the erroneous exercise of discretion standard. As the appellate courts point out, they may not have made the decision that was made, but it was within the realm of reason.
Because so few of my decisions are ever reviewed through appellate review, I rely on counsel for the parties for peer review. I rely upon the adversary system to help me make the correct decisions. At times a lawyer, during argument, and after I interjected a hypothetical assertion will say, “Your honor, with all due respect and with no offense intended, I think you might be wrong.”
I tell them “You will never offend me by telling me I am wrong. That’s your job. I want to hear why you think I am wrong. Maybe I am wrong. Educate me.” I then listen to the other side. I enjoy when the light comes on and I can more clearly see the issue in front of me.(Of course, ultimately I issue the decision and inform the parties which side I believe was right and wrong and why.)
Ultimately, I believe the power of peer-review lies with the appreciation of the responsibility of constructive criticism, and the appreciation of being responsibly criticized. The criticism helps identify cognitive errors that are part of the human psychology and that often lead to erroneous decisions. I believe anything we can do to ensure the best decisions possible, strengthens the legal system.
June 2, 2010
Research-based Law
I stated in my last entry that I believed that “evidence-based” law practices as defined as practices that have been shown to be effective through appropriately designed, peer-reviewed scientific statistical analysis would be difficult to obtain in the law. I believed that the law should strive toward becoming “research-based” rather than “evidence-based”. (The two terms are often interchanged in the literature in different fields.)
I define “research-based practice” as a practice developed by using empirical research of a more basic nature, that are components of a larger, or a different but similar, phenomenon being studied. For example, we read research on the efficacy of alcohol treatment on reducing alcohol abuse in individuals not convicted of operating while intoxicated. We then use that research to develop a practice to apply to a population of individuals convicted of operating while intoxicated in an attempt to reduce drunk driving. Although the targeted populations and objectives are different, we infer through this research that the newly designed practice may work in this circumstance also.
“Research-based” practices allow using basic research in areas such as psychology, behavioral economics, sociology, anthropology, and physiology—research often performed in controlled laboratory settings using rats or college freshmen as study subjects; and applying that basic research to legal issues involving human behavior. (Don’t all legal issues involve human behavior?) “Research-based” allows for more creativity, and combines the knowledge of science with the expertise of the practitioner in an attempt to be the most effective practitioner possible.
Further, I reach the conclusion that we as practitioners should be “research-based” rather than the more rigorous “evidence-based” for essentially the same reason my fifth grade nun advised me that I would be better off aiming for purgatory rather than heaven—there would be at least a chance for success.
The views expressed in this blog are solely the views of the author(s) and do not represent the views of any other public official or organization.
June 1, 2010
Evidence-based vs. research-based
Promoting a practice as “evidence-based” is currently the vogue among judges and social workers. I believe that is a good thing. However, authentic evidence-based practices in the law are few and far between.
For a practice to be “evidence-based” it must meet the following conditions: 1.) The practice has been evaluated using an experimental or quasi-experimental design. 2.) The practice, through the use of statistical analysis, has been shown to be effective against the dependent variable it was designed to combat, for example recidivism. 3.) The results have been peer-reviewed by others with expertise in such research. (Cooney et al 2007, see previous post and link. They also include an endorsement by an organization, which I wouldn’t include.) Satisfying these three conditions is not easy.
First, the type of work we do as judges is not easily configured to meet experimental design. One of the principal factors in this design is minimizing change in everything but the factor being studied, called the independent variable. As anyone involved in the law knows, no two cases are alike regardless of their apparent similarities. Further, we cannot easily randomize our treatments. For example, would it be ethical (or legal) to randomly sentence one criminal defendant to probation and into a treatment program, while denying another similarly situated defendant the same treatment, and sentencing that defendant to prison because you the judge were following an experimental design to determine the efficacy of the treatment? I don’t think so.
Those scientific design problems could be somewhat overcome through statistical methods that aggregate data or through the use of historical data (quasi-experimental design). Herein lies the next problem. The sample sizes we have available for such studies are often too small for statistical methods to detect anything but huge effects. Many double blind, randomized drug studies will include sample sizes involving tens of thousands of individuals. (A double blind study is one in which neither the person receiving the treatment or giving the treatment know what the treatment is—clearly a condition that would be close to impossible to meet in legal practice.)
Legal studies would most likely not be randomized, much less double blind, and our sample sizes are most likely much smaller. The result of smaller sample size is a corresponding reduction in the ability of statistical methods to detect a statistically significant effect.
These two conditions alone make it very difficult to have many legal practices meet the criteria of being “evidence-based.” I will address the peer-review condition and why I believe us legal practitioners should be “research-based” rather than “evidenced-based” in later entries.
