The Skeptics Society & Skeptic magazine




Positively False (book cover)

NEW ON MICHAELSHERMER.COM
Nash Equilibrium, the Omerta Rule, and Doping in Cycling

The Tour de France is underway and it is already shaping up to be one of the grandest and most epic races in the event’s century-long history. This event is so hard it is not surprising that, as usual, allegations and suspicions of doping have surrounded the race even before it began. In this week’s Skepticblog, Michael Shermer explains why race organizations have such a hard time enforcing the rules, and what can be done about it.

READ the post

FOLLOW MICHAEL SHERMER ON TWITTERFacebookTRUE/SLANT

In this week’s eSkeptic, Dr. David H. Voelker reviews Wrong: Why Experts Keep Failing Us — And How to Know When Not to Trust Them, by David H. Freedman.

Dr. David H. Voelker is a Lecturer at Stanford University, where he teaches courses in communication research methods and theory. He is also a research consultant, and is co-author of a best-selling statistics study guide. His interest areas include cognitive processing, the misuse of social science research, and the philosophy of science.


How Much Does Being Right Matter?

by Dr. David H. Voelker

THE MARKET FOR BOOKS about how ordinary people make thinking mistakes being fairly saturated (Predictably Irrational, Sway, Nudge), it makes sense that someone would turn the spotlight on a group that’s supposed to mess up less than the rest: experts. Journalist David Freedman walks us through an impressive list of false and conflicting claims made by experts in a variety of fields that really drives home the dubiousness of much — if not most — of what passes for expert wisdom. The book is worth this carefully assembled and annotated collection of dueling truth claims alone.

There’s plenty of blame for our expert misinformation to go around, says Freedman. From respected scientists to financial wizards to self-appointed relationship gurus, people whom we credit with specialized knowledge conduct sloppy research, suppress disconfirming data, and leap to unwarranted conclusions. Journalists oversimplify and misrepresent study findings. Bad advice thrives in part because the public demands easy fixes that are “resonant, provocative and colorful.”

We would expect Wrong to cover famous cases of expert fraud like the South Korean human embryo cloning scandal, and it does. Most of the expert errors documented here, however, are not intentional, but originate in the cognitive biases to which everyone is prone. Like the rest of us, experts have sharper eyes for data that supports their hypotheses, claim to have started out looking for what they eventually found, and play to their employers’ metrics (research funding agencies). Nor are peer review and other forms of self-regulation much of a remedy: Thomas Kuhn showed some fifty years ago how the practices of scientific communities reinforce and perpetuate prevailing paradigms.

It would be unremarkable to learn that experts sometimes make mistakes, but if Wrong is right, the magnitude of the problem is much greater than most people suspect. The findings of two out of every three published medical studies fail to hold up. When you consider that the truth claims of less highly-educated and credentialed experts like the inventor of the latest diet or management fad are on average even less reliable, you realize we’re awash in untrustworthy advice.

Books written to hammer a single point are vulnerable to overstating their case, and Freedman’s expert targets are sometimes damned if they do and damned if they don’t. For example, he criticizes mainstream scientific research for careless procedures and small, unrepresentative samples but later defends junk science because, even though its procedural rigor and signal to noise ratio are typically even lower, it occasionally stumbles upon a nugget missed by the pros. That’s having it both ways. Similarly, he fails to take his own advice against retrospective sense-making when he says the warning signs in a famous case of scientific fraud were “glaringly obvious” when looked back upon.

Freedman knows his task leaves him open to the charge of begging the question: the same lack of certainty he says accompanies expert judgments must also apply to his own assumptions that particular expert claims are wrong. I agree with his defense that, though any individual claim may be mistaken, accumulating and pooling evidence allows us to converge on the truth. A historical progression of geographic maps provides an example: early maps of the world are wildly discrepant, but the shapes of the continents and details in their coastlines gradually converge and stabilize across cartographers over time.

So common are the serious errors catalogued in Wrong by even the most eminent researchers and institutions, though, and so influential are false claims in directing the flow of dollars and in propping up whole industries and reputations, that by the time you reach the simple guidelines at the end of the book for knowing when to suspect that an expert opinion might be wrong, it feels the equivalent of being advised to move a foot inland from the beach to protect against tsunamis.

Wrong: Why Experts Keep Failing Us — And How to Know When Not to Trust Them (book cover)

Wrong’s abundant examples of how experts fail us demonstrate how complex the world is. The contingencies of cause-and-effect relationships can be many and difficult to trace, and good advice in one situation can lead to disastrous results in another.

Unfortunately, directing experts (or the middlemen who report their work) to qualify their truth claims with all of the ways they could be wrong, and publicizing negative findings as well as positive ones, are no solution. That just adds to the pile of stuff we have to sift through, and makes it harder for our brains to fulfill what is perhaps their most important function, and is the main reason we rely on experts in the first place: information reduction; sorting through the noise for the signal. The best we can hope to do (whether expert or layman) is narrow the confidence intervals of our predictions a little, and be wrong a little less often.

On the surface, Wrong is about the untrustworthiness of expert advice, but it has much deeper implications. As William James observed a century ago: “Truth lives, in fact, for the most part on a credit system. Our thoughts and beliefs ‘pass,’ so long as nothing challenges them, just as bank-notes pass so long as nobody refuses them.”

Even though experts might eventually converge on the truth about specific clearly defined causal relationships, across the multitude of main effects and complex interactions that each of us experiences in a typical lifetime, the saying, “the truth will out,” is false. At any given time, a substantial number of our individual and shared beliefs about the causes of those effects are simply wrong. We’re guaranteed to take to our graves false conclusions about why this diet did or didn’t work for us, why that relationship went sour, or whether our grown child wouldn’t have committed spousal abuse if only we’d spared the rod.

The degree to which the truth doesn’t “out” is the latitude that experts — and anyone, for that matter — have to construct social realities with impunity. For the objectively false claims described in Wrong and others we hear every day to have the power they do to launch movements, sell products, determine government policy and distribute social rewards, being right doesn’t matter as much as being accepted.

That, I think, is the real lesson of Wrong.


Skeptical perspectives on the history of science and skepticism
DVD cover The Origins of Skepticism and the JFK Assassination
(DVD $23.95) with Dr. Richard Popkin

In this classic lecture at Caltech, one of the world’s foremost historians of skepticism gives a clear history of skeptical philosophy and presents practical applications of skeptical thinking to modern issues, beginning with the Warren report, of which he was the first prominent public intellectual to be skeptical of the government’s claim that JFK was assassinated solely by Lee Harvey Oswald. This lecture is a classic in skeptical thinking and belongs in the library of all serious skeptics.
READ more…

DVD cover Witches, Spirits and Science: Experimental Science and the Paranormal in 17th Century England
(DVD $23.95) with Dr. Richard Olson

We all know about the paranormal belief in witches among commoners in Early Modern England. But did scientists also believe in witches, witchcraft, and spirits? Historian of science Richard Olson explores the fascinating connections between experimental science and the belief in witches and spirits in the transitional age between medieval superstitions and modern scientific methodology. READ more…

DVD cover False Memory Syndrome and the Recovered
Memory Movement

(DVD $23.95) with Dr. John Hochman

Abuse claims are now considered by many to be nearly epidemic, with some demographics suggesting that nearly 1/3 of all women were abused as children. Are the growing number of stories evidence of widespread crimes against humanity? Or is this a social movement analogous to the witch trials of the Middle Ages, where memory is acquired and the accused are automatically considered guilty? READ more…

Skeptic Magazine App on iPhone

SKEPTIC App

Whether at home or on the go, the SKEPTIC App is the easiest way to read your favorite articles. Within the app, users can purchase the current issue and back issues. Download the app today and get a 30-day free trial subscription.

Download the Skeptic Magazine App for iOS, available on the App Store
Download the Skeptic Magazine App for Android, available on Google Play
Download the Skeptic Magazine App for iOS, available on the App Store
Download the Skeptic Magazine App for Android, available on Google Play
SKEPTIC • 3938 State St., Suite 101, Santa Barbara, CA, 93105-3114 • 1-805-576-9396 • Copyright © 1992–2023. All rights reserved • Privacy Policy