The Skeptics Society & Skeptic magazine


Resolving Conflicts in Findings: Vaccine Promotion is Tricky

Aug. 26, 2015 by | Comments (9)

A few months ago I wrote about the psychology of vaccine denial. In the post I discussed two publications, one of which (Nyhan, et al.) found:

Corrective information reduced misperceptions about the vaccine/autism link but nonetheless decreased intent to vaccinate among parents who had the least favorable attitudes toward vaccines. Moreover, images of children who have MMR and a narrative about a child who had measles actually increased beliefs in serious vaccine side effects.

None of the interventions increased parents’ intent to vaccinate.

Then, a couple of weeks ago, a friend sent me a link to this piece describing research which seems to contradict that finding. The authors (Horne, et al.) concluded that

…highlighting factual information about the dangers of communicable diseases can positively impact people‚Äôs attitudes to vaccination.

These two conclusions seem to contradict each other. Which should we believe?

Many times this question comes down to the quality of the research. In this case, I believe these are both fairly well-designed studies. One, however, is more precise than the other in several ways. I believe that precision highlights the complexity of the issue as well as giving us a better idea of the direction that vaccine promotion should take.

Let’s look at the differences in sampling and method between the two studies.

The Horne study sampled 315 men and women. In the Nyhan study, the final sample was 1759 parents with children under the age of 18. In most research, 315 subjects is more than sufficient and more is not always better. The danger in larger samples is to find effects that are statistically significant, but not practically significant. However, when comparing conflicting findings, it is best to bet on the side of the larger sample.

Then there’s the question of limiting the study to parents. Although Horne compared parents to non-parents and found no significant differences in attitudes or effects, noise is noise. These two groups of people vary, and the attitudes of non-parents are not particularly relevant. Limiting the study to parents would give me more confidence in the robustness of the findings and their application in real-world activism.

Still, if both are reasonably well-designed studies by competent researchers, the end results should not contradict each other. So there must be more going on. And there is.

For one thing, this is a great example of how complex social sciences are. We should never make policy decisions based on a single study and this demonstrates why. Replication, especially with variants of measures and materials, is essential to learning the best methods of persuasion.

For another, these studies differ in more than just sampling techniques. The Horne study is much simpler and, in fact, oversimplifies. Nyhan, et al. included three outcome measures, each addressing a specific attitude:

  1. The belief that vaccines cause autism.
  2. Perceived risk of side effects from vaccines.
  3. Intent to vaccinate one’s child/children.

By contrast, the Horne study involved a single measure which combined answers to five specific questions (such as “I intend to vaccinate my child.” and “Doctors would not recommend vaccines if they were unsafe.” to come up with a more vague “vaccine attitudes” scale. Even if the answers to these questions are highly correlated, how interventions affect those answers may be very different. They certainly were in the Nyhan study. And if “effective” is defined as increasing intent to vaccinate, then the Horne study does not answer the question it purports to answer. Personally, I am more interested in intent to vaccinate than I am in any other aspect of “vaccine attitudes”, so the Nyhan study’s findings are much more meaningful to me.

In general, it is best to measure outcomes of interest as specifically as possible, but of course the more outcomes a researcher studies, the larger the sample must be.

Finally, and perhaps the most important difference between these two studies, is the timing of the experimental portion. When measuring the effect of treatments or interventions on attitudes, an experiment should be spaced over time. A researcher will measure the attitude, then wait before applying a treatment and measuring the attitude again. When polled about attitudes, those attitudes are brought to mind. This affects our receptiveness to relevant information in complex ways, ways that vary based on a number of other factors such as the strengths of our attitudes and the way the questions are worded. However, allowing subjects to forget about the initial survey provides a more accurate picture of how people confronted with information in the real world may respond to it.

The Horne experiment was conducted a day after the initial screening while the Nyhan experiment occurred about two weeks after initial screening.

My conclusion? I think the issue is complex, but while Horne’s findings appear easier to understand, Nyhan’s findings are more specific, answer more interesting questions, and can be more easily viewed within the framework of well-established knowledge about human decision-making (e.g., cognitive dissonance).

That, and we need more research if we are to develop effective ways of increasing vaccination rates.

This post also appears at Barbara Drescher’s personal blog, ICBS Everywhere.
  • Horne Z, Powell D, Hummel JE, & Holyoak KJ (2015). Countering antivaccination attitudes. Proceedings of the National Academy of Sciences of the United States of America, 112(33), 10321-4 PMID: 26240325
  • Nyhan B, Reifler J, Richey S, & Freed GL (2014). Effective messages in vaccine promotion: a randomized trial. Pediatrics, 133(4) PMID: 24590751
Barbara Drescher

Barbara Drescher taught quantitative and cognitive psychology, primarily at California State University, Northridge for a decade. Barbara was a National Science Foundation Fellow and a Phi Kappa Phi Scholar. Her research has been recognized with several awards and the findings discussed in Psychology Today. More recently, Barbara developed educational materials for the James Randi Educational Foundation. Read Barbara’s full bio or her other posts on this blog.

Inline Feedbacks
View all comments

Get eSkeptic

Be in the know!

Subscribe to eSkeptic: our free email newsletter and get great podcasts, videos, reviews and articles from Skeptic magazine, announcements, and more in your inbox twice a week. It’s free. We never share your address. Unsubscribe any time.

Sign me up!


Skeptic cover art by Pat Linse

Art of the Skeptic

In celebration of Skeptic magazine’s 100th issue, we present sage graphic art advice for skeptical groups and a gallery of art reflecting more than 47 years of skeptical activism from Skeptic’s long time Art Director, Pat Linse

Detecting Baloney

Baloney Detection Kit Sandwich (Infographic) by Deanna and Skylar (High Tech High Media Arts, San Diego, CA)

The Baloney Detection Kit Sandwich (Infographic)

For a class project, a pair of 11th grade physics students created the infographic shown below, inspired by Michael Shermer’s Baloney Detection Kit: a 16-page booklet designed to hone your critical thinking skills.

FREE PDF Download

Wisdom of Harriet Hall

Top 10 Things to Know About Alternative Medicine

Harriet Hall M.D. discusses: alternative versus conventional medicine, flu fear mongering, chiropractic, vaccines and autism, placebo effect, diet, homeopathy, acupuncture, “natural remedies,” and detoxification.

FREE Video Series

Science Based Medicine vs. Alternative Medicine

Science Based Medicine vs. Alternative Medicine

Understanding the difference could save your life! In this superb 10-part video lecture series, Harriet Hall M.D., contrasts science-based medicine with so-called “complementary and alternative” methods.

FREE PDF Download

Top 10 Myths of Terrorism

Is Terrorism an Existential Threat?

This free booklet reveals 10 myths that explain why terrorism is not a threat to our way of life or our survival.

FREE PDF Download

The Top 10 Weirdest Things

The Top Ten Strangest Beliefs

Michael Shermer has compiled a list of the top 10 strangest beliefs that he has encountered in his quarter century as a professional skeptic.

FREE PDF Download

Reality Check: How Science Deniers Threaten Our Future (paperback cover)

Who believes them? Why? How can you tell if they’re true?

What is a conspiracy theory, why do people believe in them, and can you tell the difference between a true conspiracy and a false one?

FREE PDF Download

The Science Behind Why People See Ghosts

The Science Behind Why People See Ghosts

Mind altering experiences are one of the foundations of widespread belief in the paranormal. But as skeptics are well aware, accepting them as reality can be dangerous…

FREE PDF Download

Top 10 Myths About Evolution

Top 10 Myths About Evolution (and how we know it really happened)

If humans came from apes, why aren’t apes evolving into humans? Find out in this pamphlet!

FREE PDF Download

Learn to be a Psychic in 10 Easy Lessons

Learn to do Psychic “Cold Reading” in 10
Easy Lessons

Psychic readings and fortunetelling are an ancient art — a combination of acting and psychological manipulation.

FREE PDF Download

The Yeti or Abominable Snowman

5 Cryptid Cards

Download and print 5 Cryptid Cards created by Junior Skeptic Editor Daniel Loxton. Creatures include: The Yeti, Griffin, Sasquatch/Bigfoot, Loch Ness Monster, and the Cadborosaurus.

Copyright © 1992–2021. All rights reserved. | P.O. Box 338 | Altadena, CA, 91001 | 1-626-794-3119. The Skeptics Society is a non-profit, member-supported 501(c)(3) organization (ID # 95-4550781) whose mission is to promote science & reason. As an Amazon Associate, we earn from qualifying purchases. Privacy Policy.