Learning to Think Critically Like a Scientist

Written by Kelly A. Hogan, University of North Carolina at Chapel Hill

Learning Outcomes:

– To discover “bad” scienceStreaming and download Inferno (2016)

– To recognize how media reports of scientific papers can cause the spread of misinformation and fear

– To have students suggest studies that would produce convincing data

Activity Description: Students are shown a summary of an infamous scientific article related to the MMR vaccine and autism. They are asked to evaluate if this fits the guidelines of good science. The class should come to the conclusion that it does not fit the guidelines of good science by determining what data is lacking.

Time Needed: 30 minutes with discussion

Materials Needed: Copies of the worksheet below

Activity Instructions: Students are given a summary of a monumental paper linking the MMR vaccine and autism. Students discuss their reactions and then spend time determining why the study was flawed and what kind of studies would be better. Class discussions are followed by a presentation of real data from studies similar to those they may have suggested. Lastly, the instructor reveals that the paper was retracted. Nonetheless, public fear and misinformation remain despite no supporting evidence linking the MMR vaccine to autism. (There are numerous news articles on the paper’s retraction that you can reference.)

Worksheet: Thinking Critically Worksheet

Background/Follow-up for the Instructor:

The following are questions students may come up with on their own for question 3 of the worksheet. Below each question is a brief summary of what was found in various studies.

  • Did autism rates increase after the introduction of the MMR vaccine?

– No. Sweden and UK studies (see below) showed no difference in the rate of autism before and after the introduction of MMR to society

  • Does onset of autism correlate with time of vaccination?

– No. UK studies show that age of onset does not correlate with the timing of the MMR vaccine, if given at all. (Although autism symptoms frequently present around the age this shot is most commonly administered).

  • What is the normal rate of autism (control group vs. a MMR vaccinated group)?

– Denmark: study involved over 500,000 children. No difference in autism risk in those vaccinated vs. those not vaccinated.

More specific information originally published on www.cdc.gov in 2004):

The MMR-autism theory came to the forefront when, in 1998,Wakefield and colleagues reviewed reports of children with bowel disease and regressive developmental disorders, mostly autism. The researchers suggested that MMR vaccination led to intestinal abnormalities, resulting in impaired intestinal function and developmental regression within 24 hours to a few weeks of vaccination. This hypothesis was based on 12 children. In 9 of the cases, the child’s parents or pediatrician speculated that the MMR vaccine had contributed to the behavioral problems of the children in the study. There are a number of limitations in the Wakefield et al. (1998) study:

1. The study used too few cases to make any generalizations about the causes of autism; only 12 children were included in the study. Further, the cases were referred to the researchers and may not be a representative sample of cases of autism.

2. There were no healthy control children for comparison. As a result, it is difficult to determine whether the bowel changes seen in the 12 children included in the study were similar to changes in normal children, or to determine if the rate of vaccination in autistic children was higher than in the general population.

3. The study did not identify the time period during which the cases were identified.

4. In at least 4 of the 12 cases, behavioral problems appeared before the onset of symptoms of bowel disease; that is, the effect preceded the proposed cause. It is unlikely, therefore, that bowel disease or the MMR vaccine triggered the autism.

In 2004, 10 of the 13 authors of the study retracted the paper’s interpretation, stating that the data were insufficient to establish a causal link between MMR vaccine and autism (Murch et al., 2004) The paper was fully retracted from Lancet in Feb 2010.

Epidemiologic studies have shown no relationship between MMR vaccination in children and development of autism:

In 1997, the National Childhood Encephalopathy Study (NCES) was examined to see if there was any link between measles vaccine and neurological events. The researchers found no indication that measles vaccine contributes to the development of long-term neurological damage, including educational and behavioral deficits (Miller et al., 1997).

A study by Gillberg and Heijbel (1998) examined the prevalence of autism in children born inSwedenfrom 1975-1984. There was no difference in the prevalence of autism among children born before the introduction of the MMR vaccine inSwedenand those born after the vaccine was introduced.

In 1999, the British Committee on Safety of Medicines convened a “Working Party on MMR Vaccine” to conduct a systematic review of reports of autism, gastrointestinal disease, and similar disorders after receipt of MMR or measles/rubella vaccine. It was concluded that the available information did not support the posited associations between MMR and autism and other disorders.

Taylor and colleagues (1999) studied 498 children with autism in theUKand found the age at which they were diagnosed was the same regardless of whether they received the MMR vaccine before or after 18 months of age or whether they were never vaccinated. Importantly, the first signs or diagnoses of autism were not more likely to occur within time periods following MMR vaccination than during other time periods. Also, there was no sudden increase in cases of autism after the introduction of MMR vaccine in theUK. Such a jump would have been expected if MMR vaccine was causing a substantial increase in autism.

Kaye and colleagues (2001) assessed the relationship between the risk of autism among children in theUKand MMR vaccine. Among a subgroup of boys aged 2-5 years, the risk of autism increased almost 4 fold from 1988 to 1993, while MMR vaccination coverage remained constant at approximately 95% over these same years.

Researchers in theU.S.found that among children born between 1980 and 1994 and enrolled inCaliforniakindergartens, there was a 373% relative increase in autism cases, though the relative increase in MMR vaccine coverage by the age of 24 months was only 14% (Dales et al., 2001). For more on this study, see California Data on Theory of Autism and MMR Immunization.

Researchers in theUK(Frombonne & Chakrabarti, 2001) conducted a study to test the idea that a new form, or “new variant,” of Inflammatory Bowel Disease (IBD) exists. This new variant IBD has been described as a combination of developmental regression and gastrointestinal symptoms occurring shortly after MMR immunization. Information on 96 children (95 immunized with MMR) who were born between 1992 and 1995 and were diagnosed with pervasive developmental disorder were compared with data from 2 groups of autistic patients (one group of 98 born before MMR was ever used and one group of 68 who were likely to have received MMR vaccine). No evidence was found to support a new syndrome of MMR-induced IBD/autism. For instance, the researchers found that there were no differences between vaccinated and unvaccinated groups with regard to when their parents first became concerned about their child’s development. Similarly, the rate of developmental regression reported in the vaccinated and unvaccinated groups was not different; therefore, there was no suggestion that developmental regression had increased in frequency since MMR was introduced. Of the 96 children in the first group, no inflammatory bowel disorder was reported. Furthermore, there was no association found between developmental regression and gastrointestinal symptoms.

Another group of researchers in theUK(Taylor et al., 2002) also examined whether MMR vaccination is associated with bowel problems and developmental regression in children with autism, looking for evidence of a “new variant” form of IBD/autism. The study included 278 cases of children with autism and 195 with atypical autism (cases with many of the features of childhood autism but not quite meeting the required criteria for that diagnosis, or with atypical features such as onset of symptoms after the age of 3 years). The cases included in this study were born between 1979 and 1998. The proportion of children with developmental regression or bowel symptoms did not change significantly from 1979 to 1988, a period which included the introduction of MMR vaccination in theUKin 1988. No significant difference was found in rates of bowel problems or regression in children who received the MMR vaccine before their parents became concerned about their development, compared with those who received it only after such concern and those who had not received the MMR vaccine. The findings provide no support for an MMR associated “new variant” form of autism and further evidence against involvement of MMR vaccine in autism.

Madsen et al. (2002) conducted a study of all children born inDenmarkfrom January 1991 through December 1998. There were a total of 537,303 children in the study; 440,655 of the children were vaccinated with MMR and 96,648 were not. The researchers did not find a higher risk of autism in the vaccinated than in the unvaccinated group of children. Furthermore, there was no association between the age at time of vaccination, the amount of time that had passed since vaccination, or the date of vaccination and the development of any autistic disorder. Though there were many more vaccinated than unvaccinated children in the study group, the sample was large enough to contain more statistical power than other MMR and autism studies. Therefore, this study provides strong evidence against the hypothesis that MMR vaccination causes autism.

DeStefano et al. (2004) conducted a study to see if there was a difference in the age at which children with autism and without autism received their first MMR vaccination. The study’s findings showed that children with autism received their first MMR vaccination at similar ages as children without autism.