Skeptic: Examining Extraordinary Claims and Promoting Science Skeptic: Examining Extraordinary Claims and Promoting Science

top navigation:

Wednesday, August 22nd, 2007 | ISSN 1556-5696

eSkeptic: the email newsletter of the Skeptics Society

Share this eSkeptic with friends online. Subscribe | Donate | Watch Lectures | Shop



In this week’s eSkeptic Charles Lambdin reviews Jerome Groopman’s book: How Doctors Think (Houghton Mifflin, 2007, ISBN 0618610030). Charles Lambdin is a graduate student at Wichita State University, where he teaches psychology and research methods.


How Doctors Think They Think

a book review by Charles Lambdin

When an expert writes a book for the public, he is writing for an audience largely unqualified to assess its claims, an audience primarily composed of uncritical consumers. This creates a problem: How can those genuinely curious but untrained sift accurate explanations of science’s best thinking from anecdotally-derived hoopla? Unfortunately, Jerome Groopman’s How Doctors Think is not always clear on this problem.

Groopman is at his best discussing the deleterious impact of HMOs, the influence of big pharma, and especially the “medicalization” of aging. He turns a skeptical eye to “erectile dysfunction,” “male menopause,” and even “ADHD,” arguing that pharmaceutical companies sometimes create drugs for nonexistent illnesses, and then produce a market for their product by inventing a “disease” for the drug to treat. Are older men’s erectile “dysfunction” and plummeting testosterone levels “illnesses,” or normal parts of aging that have been medicalized? If a child is bored and restless in the fourth grade, should he be prescribed Ritalin or advanced to the sixth grade? When asking such questions, Groopman reminds me of Thomas Szasz, who defiantly believed that mental illnesses are invented, not discovered.

How Doctors Think (cover)

Much else in the book, however, fails to impress. One would hope that a book entitled How Doctors Think would be about just that, how doctors make diagnoses, a topic on which much scientific research has been done. Unfortunately, Groopman by-and-large ignores this research, instead giving us a long collection of anecdotes wherein his doctor friends tell us how they think. And this is the book’s main failing, for if you were to write a book purporting to examine how some group of people thinks, you would have to do more than simply ask some of them.

Groopman tells us that many diagnoses are wrong, and that most medical errors are the result of cognitive biases resulting from quick and dirty rule-of-thumb heuristics we all use to make decisions when pressed for time, operating under uncertainty, or when we simply do not know a better decision rule. He tells us that doctors are often unaware of the impact such heuristics — and their accompanying biases — have on their diagnostic accuracy.

This is an excellent point, but it begs a simple question: What if the same cognitive biases that wreak havoc on doctors’ thinking — unawares — also influences their thinking about their thinking?1 In How Doctors Think, doctors tell us how simply spending more time with patients, asking open-ended rather than close-ended questions, reading “body language,” including patients’ emotions in the picture, etc., improves their diagnostic ability. How much impact do these factors have on diagnostic ability? Do they actually have any impact at all? These questions place Groopman’s book on shaky ground, for his evidence consists primarily of unrepresentative and mostly irrelevant testimonials in which doctors describe various atypical diagnostic adventures they’ve had, and how they think they either emerged victorious or dropped the ball.2 The problem is, this does not tell us anything about how doctors think, only how they think they think.

The problems get worse. Groopman tells us he is troubled that new doctors seem to be trained to “think like computers,” that they rely on diagnostic decision aids and some seductive “boiler-plate scheme” called evidence-based medicine. Groopman’s position, when his various arguments are gathered and assembled, becomes untenable. He admits doctors suffer from innumerable biases that diminish the accuracy of diagnosis, reducing many diagnoses to idiosyncratic responses fueled by mood, whether the patient is liked or disliked, advertisements recently seen, etc. Thus Groopman agrees with decision scientists’ diagnosis of doctor decision making; but then he goes on to wantonly dismiss what many of the very same researchers claim is the best (and perhaps only) remedy, the way to “debias” diagnosis: evidence-based medicine and the use of decision aids. In place of statistics what does Groopman suggest doctors rely on? Clinical intuition of course, the very source of the cognitive biases he pays lip service to throughout his book.

Unfortunately, what research Groopman cites to back his claims is somewhat one-sided and sometimes off the point. Two articles he cites both argue that decision aids pertaining to treatment (rather than diagnosis) don’t take into account when patients have multiple illnesses requiring multiple medications, which may interact with each other.3,4 This is an important point, but to attempt to argue from this single issue that decision aids shouldn’t be relied on is to make a rather specious generalization.

Most doctors do not like decision aids. They rob them of much of their power and prestige. Why go through medical school and accrue a six-figure debt if you’re simply going to use a computer to make diagnoses? One study famously showed that a successful predictive instrument for acute ischemic heart disease (which reduced the false positive rate from 71% to 0) was, after its use in randomized trials, all but discarded by doctors (only 2.8% of the sample continued to use it).5 It is no secret many doctors despise evidence-based medicine. It is impersonal “cookbook medicine.” It is “dehumanizing,” treating people like statistics. Patients do not like it either. They think less of doctors’ abilities who rely on such aids.6

The problem is that it is usually in patients’ best interest to be treated like a “statistic.” Doctors cannot outperform mechanical diagnoses because their own diagnoses are inconsistent. An algorithm guarantees the same input results in the same output, and whether one likes this or not, this maximizes accuracy. If the exact same information results in variable and individual output, error will increase. However, the psychological baggage associated with the use of statistics in medicine (doctors’ pride and patients’ insistence on “certainty”) makes this a difficult issue to overcome.

The statistics vs. clinical intuition debate has ensued for decades in psychology.7 Where one sides in the debate is largely determined by what one makes of a single phrase: “Group statistics don’t apply to individuals.” This claim, widely believed, ignores many of the most basic concepts of probability and statistics, such as error. Yes, individuals possess unique qualities, but they also share many features that allow for predictive power.8 If 95% of a sample with quality X has quality Y, insisting that someone with quality X may not have Y because “statistics don’t apply to individuals” will only decrease accuracy. Insistence on certainty decreases accuracy. As Groopman himself says, the perfect is the enemy of the good.

Groopman, quite bizarrely, then stresses the uncertainty inherent in every diagnosis while maintaining that “statistics don’t apply to individuals.” Uncertainty is exactly why statistics apply to individuals: they give us the best guess available given the level of uncertainty at hand. It does not behoove anyone to ignore uncertainty and pretend it is not there. When doctors hear a decision aid has a 25% error rate and state “I can beat that,” this should raise suspicion. This is sometimes called the “broken leg argument.”8 Imagine an algorithm predicts when people will go to the movies with 80% accuracy. I think I can beat that if I happen to know a particular individual has a broken leg, but what if I start treating every diagnosis as a “broken-leg problem?” Will my accuracy increase, or decrease?

Physicians who allow themselves to think in such discretionary ways can find “exceptions” everywhere they look, and, augmenting a decision aid as they see fit, will only end up lowering its overall diagnostic accuracy. Why? Because human beings do not apply rules consistently. Mechanical procedures always lead to the same conclusion from the same input. Doctors are subject to random fluctuations in diagnosis caused by judgmentally-irrelevant factors including availability, priming, recency effects, inconsistent weighting of information, fatigue, etc., all of which reduce accuracy. What leads to a correct decision for one case may not for another, and variables that contribute to the diagnosis made may actually be uncorrelated with it.6,7,8

Groopman makes comparisons to the perspicacious (and fictitious) “deductions” of Sherlock Holmes. Such comparisons are disingenuous. Most medical patients are not unique mysteries with atypical solutions (though the anecdotes in Groopman’s book are). Accuracy under uncertainty often boils down to what types of errors one is willing to make: a few false negatives with exotic conditions, or a great many “zebra-chasing” false positives. Oddly, Groopman rebukes doctors guilty of “zebra retreat,” but bungles the example this term is derived from. He quotes: “When you hear hoofbeats, think about horses, not zebras.” The actual lesson is, “When in Wyoming, if you hear hoofbeats and think you see stripes, it’s still probably a horse.” Groopman scoffs at this, which is unfortunate because treating every patient as a uniquely rare case will only increase error over time. When one looks at the research Groopman largely ignores, the lesson is clear. Medicine needs to become more science and less art.

References & Notes
  1. Wilson, T. & Schooler, J. 1991. “Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions.” Journal of Personality and Social Psychology, Vol. 60 (2), 181–192.
  2. For a discussion on why we often cannot accurately gauge the accuracy of our own judgments see: Einhorn, H. & Hogarth, R. 1978. “Confidence in judgment: Persistence of the illusion of validity.” Psychological Review, 85, 395–416.
  3. Tinetti, M. 2004. “Potential pitfalls of disease-specific guidelines for patients with multiple conditions.” New England Journal of Medicine, 351, 2870–2874.
  4. O’Connor, P. 2005. “Adding Value to Evidence-Based Clinical Guidelines.” Journal of the American Medical Association, 294, 741–743.
  5. Corey, G. & Merenstein, J. 1987. “Applying the Acute Ischemic Heart Disease Predictive Instrument.” Journal of Family Practitioners, 25, 127–133.
  6. Arkes, H., Shaffer, V. & Medow, M. 2007. “Patients Derogate Physicians Who use a Computer-Assisted Diagnostic Aid.” Medical Decision Making, 27(2), 189–202.
  7. Grove, W., Zald, D., Lebow, B., Snitz, B. & Nelson, C. 2000. “Clinical versus Mechanical Prediction: A Meta-Analysis.” Psychological Assessment, 12(1), 19–30.
  8. Dawes, R., Faust, D. & Meehl, P. 1989. “Clinical versus Actuarial Judgment.” Science, 243, 1668–1674.

Water, Earth & Sky

This week, Skepticality tackles matters of politics on Earth, and discoveries out in the universe. First up, environmental engineer Kelly Comstock returns to answer listener questions from our bottled water episode (listen here) and address concerns about potential dangers lurking in municipal water systems (like chlorine and fluoride).

Then, Derek & Swoopy check in with their favorite lobbyist, Lori Lipman Brown from the Secular Coalition for America, regarding ongoing investigations into abuses of non-theists serving in our military, how the question of separation of church and state is being highlighted in recent political debates, and what to look for when the legislature reconvenes in the Fall.

Lastly, astrophysicist Dr. Pamela Gay shares her insights about recent media attention paid to NASA troubles, the paper by two German physicists who claim to have broken the speed of light, and the potential discovery of inorganic cosmic dust with lifelike properties.

SUBSCRIBE to Skepticality
within iTunes

DOWNLOAD Episode #59 (45MB MP3)

SUBSCRIBE to the Skeptic RSS feed


screenshot from YouTube video

Decoding the Bible Code

The Bible Code claims that hidden in the first five books of the Bible in its original Hebrew text are hidden messages in code that made predictions thousands of years ago about current events, such as the assassination of JFK and the end of the world. In this episode experts decode the Bible Code and reveal it to be a form of numerology that serves as a supreme example of pattern-seeking (and finding) behavior of which we are so skilled. WATCH the video >

3 Comments »

3 Comments

  1. Robert says:

    Dear Mr. Lambdin:

    I think you are absolutely correct, statistics of course apply to individuals (the opposite has always struck me as nonsense), but I think you have erred in using this quote as the central rubric by which to parse the diagnosis debate–and thus oversimplified something much more complex. In a sense, the statistics/individual quote is a reformulation of the universal/particular problem. And if you know much about that (as I assume you do), you know that this problem is much trickier than your resolution allows (i.e., a mere choice of universal over particular)–whether in the case of medical diagnosis or whatever.

    In my opinion, the resolution here is much more nuanced. First, to agree with your excellent point on subjective diagnoses, yes, there should be a much greater reliance on statistical guidelines within the specific areas in which diagnosis is prone toward subjective motivations (e.g., ADHD; fibromy.; depression; psychopathology generally). However, in the sphere of life-threatening diseases (e.g., oncology; infectious dis.; cardiology; etc.), reliance on mere statistics for diagnosis would be terribly foolish (even if feasible, which seems inconceivable). Sure, statistics are a good first sweep to get all the clear “postives”; but then you must move on to the more difficult, borderline cases (the testing ground of great doctors).

    In fact, I would go so far to say that in the latter cases (i.e., those of serious disease) no one–not even you–really want specialists relying on some statistical chart for symptoms and test-results. For, if so, when you or a loved one became seriously ill and no diagnosis was immediately forthcoming to the local GP, you would go week out another just-the-statistics-mam doctor. But that is precisely the person you do not seek (indeed, that is precisely the person who has already failed). No, then, you take them to the “one-of-the-top-guys-in-the-field” sort of doctor–i.e., the person who can see beyond the statistics everyone knows, who knows more than the satistics, who knows the complexity of disease and the nuance of the human body.

    So, yes, we can treat everybody who fits the statistical bill–and we should. The point is that the truly great doctors are the ones who, through extensive experience and a unique talent to synthesize data, can find the people and fix the problems that fall outside the statistical parameters (often 30% of all afflicted with the given disease). And that is the doctor you–all of us–really want to go to. Which is to say that, in the end, when there’s a serious problem, we want a doctor who can see far beyond the statistics–who, in other words, has a firm grasp of the universals to be sure, but also recognizes the essential particularity of each invidiual.

    (And, no, I’m not a doctor, though I do come from a family of physicians.)

  2. Jonathan Rau says:

    re: “Thomas Szasz, who defiantly believed that mental illnesses are invented…”

    Two corrections:

    – Dr Szasz believes that mental illnesses are invented;

    – there’s near incontrovertible evidence to believe he’s alive and well:

    from Association for the Advancement of Philosophy and Psychiatry Bulletin (AAP&P), Volume 17, Number 2 (2010):

    Fifty years later, in the tenth decade of my life, I am pleased to read Dr. Allen Frances candidly acknowledging: “Alas, I have read dozens of definitions of mental disorder (and helped to write one) and I can’t say that any have the slightest value whatever. Historically, conditions have become mental disorders by accretion and practical necessity, not because they met some independent set of operationalized definitional criteria. Indeed, the concept of mental disorder is so amorphous, protean, and heterogeneous that it inherently defies definition. This is a hole at the center of psychiatric classification.” This is as good as saying, “Mental illness, there ain’t no such thing,” and still remain loyal to one’s profession.

  3. John says:

    While I agree that statistics are a powerful, best guess tool, there is some truth in argument against the application to the individual. The application of a statistic to a single and relatively unique individual or occurrence requires a statistic generated by a small group sample that may have numerous faults (both acknowledged and overlooked) or that may, by chance, not be representative of the total group.

    The probability of something occurring for a unique event cannot be measured or even reliably estimated because a remotely accurate estimate of probability requires a large sample group and well understood conditions. For many medical conditions, we have neither.

    This isn’t an argument against the use of statistics, but rather it’s an argument against the idea that statistics are truly reliable. They are our best and most reliable option, but they are far from a guarantee of a correct diagnosis. A good doctor recognizes the downfalls of a cut and dry application of statistics, and steps outside of the box when looking at the situation. It’s time consuming, but worth the effort.

get eSkeptic
our free newsletter

Science in your inbox every Wednesday!

eSkeptic is our free email newsletter, delivered once a week. In it, you’ll receive: fascinating articles, announcements, podcasts, book reviews, and more…


Popular Articles
on skeptic.com

Here are the articles that people have been sharing over the last few days.

Carbon Comic

Carbon Comic (by Kyle Sanders)

Carbon Comic, which appears in Skeptic magazine, is created by Kyle Sanders: a pilot and founder of Little Rock, Arkansas’ Skeptics in The Pub. He is also a cartoonist who authors Carbon Dating: a skeptical comic strip about science, pseudoscience, and relationships. It can be found at carboncomic.com.

Help the
Skeptics Society
at no cost to you!

Planning on shopping at Amazon? By clicking on our Amazon affiliate link, which will open the Amazon Store in your Internet browser, the Skeptics Society will receive a small commission on your purchase. Your prices for all products remain the same, yet you’ll provide essential financial support for the work of the nonprofit Skeptics Society.

amazon.com

See our affiliate links page for Amazon.ca, Amazon.de, Amazon.co.uk, iTunes, and Barnes & Noble links.

FREE PDF Download

Reality Check: How Science Deniers Threaten Our Future (paperback cover)

Who believes them? Why? How can you tell if they’re true?

What is a conspiracy theory, why do people believe in them, and why do they tend to proliferate? Why does belief in one conspiracy correlate to belief in others? What are the triggers of belief, and how does group identity factor into it? How can one tell the difference between a true conspiracy and a false one?

FREE PDF Download

The Science Behind Why People See Ghosts

The Science Behind Why People See Ghosts

Do you know someone who has had a mind altering experience? If so, you know how compelling they can be. They are one of the foundations of widespread belief in the paranormal. But as skeptics are well aware, accepting them as reality can be dangerous…

FREE PDF Download

Top 10 Myths About Evolution

Top 10 Myths About Evolution (and how we know it really happened)

If humans came from apes, why aren’t apes evolving into humans? Find out in this pamphlet!

FREE PDF Download

Top 10 Things You Should Know About Alternative Medicine

Top 10 Things You Should Know About Alternative Medicine

Topics include: chiropractic, the placebo effect, homeopathy, acupuncture, and the questionable benefits of organic food, detoxification, and ‘natural’ remedies.

FREE PDF Download

Learn to be a Psychic in 10 Easy Lessons

Learn to do Psychic “Cold Reading” in 10
Easy Lessons

Psychic readings and fortunetelling are an ancient art — a combination of acting and psychological manipulation.

Copyright © 1992–2014 Skeptic and its contributors. For general enquiries regarding the Skeptics Society or Skeptic magazine, email skepticssociety@skeptic.com or call 1-626-794-3119. Website-related matters: webmaster@skeptic.com. Enquiries about online store orders: orders@skeptic.com. To update your subscription address: subscriptions@skeptic.com. See our Contact Information page for more details. This website uses Google Analytics, Google AdWords, and AddThis tracking software.