The story of “Critical Links: Learning in the Arts and Student Academic and Social Development,” an extraordinarily ambitious collection of research on arts education, begins in 1997, when a report published by the Arts Education Partnership’s Task Force on Research emphasized a need for a review of up-to-date research to help inform program design and policy. The National Endowment for the Arts and the U.S. Department of Education commissioned “Critical Links” – eventually published in 2002 – to address this need.
The Compendium is a literature review featuring 62 arts education research studies, summarized and analyzed by leading experts across the disciplines of dance, drama, “multi-arts,” music, and visual arts. To develop criteria for inclusion in the Compendium, James S. Catterall, Lois Hetland, and Ellen Winner were chosen as “researchers” by Arts Education Partnership (AEP) through a competitive selection process. They are joined by 11 other reviewers who summarize the studies, provide initial and secondary inquiry questions, and analyze each study, with recommendations for future research. While each study is presented individually in one to two pages, the original texts of the studies are not included. After each arts section (e.g., drama or dance), a reviewer delves into deeper inquiries revealed from links among the findings in the studies.
The Compendium sets out to achieve two ambitious goals:
- Identify strong arts education research that includes the academic and social effects of arts learning, beyond the arts learning experiences themselves. In brief, the Compendium sets out to explore transference, which “denotes instances where learning in one context assists learning in a different context.” The foreword by AEP explains that the purpose of this Compendium is “to make a contribution to the national debate over such issues as how to enable all students to reach high levels of academic achievement, how to improve overall school performance, and how to create the contexts and climates in schools that are most conducive to learning.”
- “Giv[e] insight into curriculum designs and practices that will enhance the quality and impact of student learning in the arts.” Through the examined research studies, the Compendium reviewers hope to pave the way for more informed educational program design.
The Links: What Was Studied, What They Found
The studies in “Critical Links” are organized meticulously by discipline. The drama, multi-arts, and music sections examine 19, 17, and 15 studies respectively, with concluding essays by James Catterall, Rob Horowitz and Jaci Webb-Dempsey, and Larry Scripp. Notably, the dance section features only 7 studies and the visual arts section contains only four, with final essays by Karen Kohn Bradley and Terry L. Baker respectively. A final essay by Catterall called “Overview” discusses the issue of transference and makes recommendations about “where to go from here.”
The bulk of drama research examines the connection between arts education and linguistic skills, with an emphasis on reading and writing comprehension. Overall, the studies found that drama – and in particular, role-playing activities – did have a positive effect on linguistic development in the focus groups as compared with the control groups. Some of the other overarching findings from the drama research include:
- Drama education has an impact on improving concentrated thought
- Formal reflection on experiences in drama elicits/fosters interpersonal relations
- Drama improves story comprehension
Example: A study by Ann Podlozny exemplifies the link between dramatic arts and verbal skills. Entitled “Strengthening Verbal Skills Through the Use of Classroom Drama: A Clear Link,” the research study focuses on whether classroom drama helps students develop verbal ability. This is a meta-analysis combining the results of 200 studies since 1950 that address story understanding (oral and written measures), reading achievement and readiness, oral language development, vocabulary, and writing. Catterall explains that “positive effects are shown in… written and oral measures of story recall, reading achievement, reading readiness, oral language development, and writing,” and hopes that “the report will encourage teachers, teaching artists, and school administrators to include drama in their classroom practice.”
The dance research included in the Compendium focuses on, and strongly suggests, the impact of dance on creative problem-solving, reading skills, creative thinking skills, self-reflection and self confidence.
Example: In a study by Dale Rose called “The Impact of Whirlwind’s Basic Reading Through Dance Program on First Grade Student’s Basic Reading Skills: Study II,” first-graders were studied for three months to determine whether their reading abilities could be improved through a dance program in which the students used their bodies to physically represent letters. These students were studied against a control group. The experimental group improved compared to the control group, especially in their ability to relate written consonants and vowels to their sounds.
The multi-arts programs in the Compendium focus on the correlation between arts experiences and academic achievement. These studies suggested connections between arts programs and improved reading skills, verbal skills, math skills, and creative thinking.
Example: James Catterall’s study from 1998 called “Involvement in the Arts and Success in Secondary School” shows a clear link between arts involvement and improved academic achievement. The data came from 25,000 students from eighth to tenth grade who were participating in the National Educational Longitudinal Study of 1988. The outcomes demonstrated that the students who were highly engaged in artistic experiences in middle school and high school performed better in academics than their non-arts-involved peers, regardless of socio-economic status. “High arts students… earned better grades and scores, were less likely to drop out of school… had a more positive self concept, and were more involved in community service.” Nevertheless, the results do not prove causation and underscore a need for further research to unpack the nature of the association between arts and academics.
The research studies on music reveal correlations between musical study and cognitive development, language development, reading, self-efficacy (the degree to which a person believes that he can attain a goal or succeed in a certain situation), math proficiency, and spatial-temporal reasoning (“the ability to visualize spatial patterns and mentally manipulate them over a time-ordered sequence of spatial transformations”). Spatial-temporal reasoning is an important skill for solving problems in math, science, and everyday life.
Example: For “Learning to Make Music Enhances Spatial Reasoning,” Lois Hetland selected fifteen studies for a meta-analysis to explore whether active instruction in music enhances preschool and elementary students’ performance on spatial tasks. The data across the fifteen studies are so consistent in determining that music-making leads to spatial reasoning skills that differences in the type of musical instruction hardly change the results. The meta-analysis suggests that “offering a wide range of music programs in preschools and elementary schools similar to the ones reviewed… will predict that nearly 70 percent of young children will ‘show spatial improvement as a result of the music program.’”
With only four studies examined, the visual arts are the least-represented discipline in the Compendium. The studies had different variables and goals, and as such, very different outcomes. As Catterall explains in his book summary, “[In the visual arts studies] we see only preliminary indications of impacts: that drawing is an effective communicator of learning in history and contributes to organization and persistence in writing — training in visualization contributes to reading skills — reasoning about visual art seems to transfer to reasoning about science — and instruction in visual art increases reading-readiness among preschoolers.”
Example: Jeffrey D. Wilhelm’s ethnographic case study, “Reading Is Seeing: Using Visual Response to Improve the Literary Reading of Reluctant Readers,” examines whether the visual arts can be used to help reluctant and learning-disabled readers to become better readers. Over the course of nine weeks, he studies two seventh-grade boys who were asked to make arts and crafts that represented characters in the stories they were reading and draw pictures of visual impressions they had. By the end of the nine weeks, the boys “took a more active role in reading, and began to interpret the text rather than just passively read it.”
Among the 84 effects that the researchers found the arts to have on student participants, a few emerged across all of the disciplines, including reading and math skills, and social interaction. Moreover, it is suggested that the findings in these studies may be of most importance to developing social and cognitive skills in impoverished children who may not have access to gaining such skills in other environments. (See James Catterall’s book summary for a concise overview of the findings and implications of “Critical Links.”)
Thankfully for the benefit of readers, the Compendium already contains a great deal of independent analysis to aid interpretation of the study results. Each study summary is followed by a thoughtful and informed commentary section, and each larger essay on the specific arts areas explores broader themes such as transference from the arts to other subject areas, implications for future research, and implications for policy. The Compendium seems to leave no stone unturned (or in this case, no inquiry forgotten). Any time I had an analysis that I felt was original, a researcher said it a few pages later with far more eloquence than I ever could.
What I can offer instead is an overview of what seemed to be the most significant takeaways from the body of research investigated here. In the spirit of “links,” I’ve tried to address common issues that arose throughout the Compendium as a whole, and to highlight some of the thought-provoking findings and questions that emerged from the studies. I’ll also examine some of the potential weaknesses in the research cited in the report.
Understanding who is being studied
Many research studies used experimental designs involving control groups of students. While the students are often at the same grade level, we don’t always know if they’re at the same emotional or behavioral levels. Do the sample groups tell us what we want to know about the research, or does it tell us about the demographic/age of the participants? One study that highlights this issue is Jennifer Ross Goodman’s “A Naturalistic Study of the Relationship Between Literacy Development and Dramatic Play in Five-Year-Old Children.” The study examines how literacy is developed within dramatic play by integrating daily dramatic play into a preschool classroom of 17 children. Reviewer Bruce Wilson points out that despite the detailed observations and findings in the study, “the biggest shortcoming of this type of research is the lack of generalizability to other settings… [T]he sample included a preponderance of females… Might the relationships look different in a predominantly male or balanced-gender context?” To further investigate the external validity of these findings, it would be helpful to compare this study with others that explore literacy development in preschool-age children, in order to ensure that demographics are not overly skewing the results.
Understanding the mechanisms of learning
In every arts section, one question appeared repeatedly: what exactly is being learned in an artistic experience that transfers to development in another area? For example, a student may have been taught a movement class, which resulted in her achieving better test scores. But what we don’t know is exactly what element of the dance class led to her developing certain academic skills. This information is essential to inform how we develop better arts education programs going forward.
One study that shows a specific relationship between an “arts” activity and reading ability is Dale Rose’s “The Impact of Whirlwind’s Basic Reading Through Dance Program on First Grade Students’ Basic Reading Skills: Study II.” Over the course of three months, a group of 174 children improved their basic reading skills by learning to put their bodies in the shapes of letters. They were compared with a control group of 198 children. While this finding indicates that first graders may improve their basic reading skills through movement, Ellen Winner observes that “this study does not allow the conclusion that dance leads to reading, but rather that putting one’s body in the shape of letters improves basic reading skills in young children. Whether or not this activity is ‘dance’ (a matter dancers could debate), we can conclude that this activity is an innovative and enactive way of helping children master sound-symbol relationships.”
In other studies, however, the correlation between artistic learning and developed skill is not as clear. For example, Kathryn Vaughn’s and Ellen Winner’s study “SAT Scores of Students Who Study the Arts: What We Can and Cannot Conclude about the Association” is a meta-analysis exploring the relationship between SAT scores and student involvement in the arts. Vaughn and Winner seem to have found significant relationships between involvement in the arts and higher math and verbal scores, but as reviewer Robert Horowitz points out, “the correlation between participation in high school arts programs and SAT scores is not sufficient in itself to claim that arts study leads to improvement in academic performance.”
The SAT study is one of a few studies that reveal a possible issue with meta-analysis research itself. Meta-analysis is a high-level process that compares the results of multiple studies addressing a set of related research hypotheses. The Vaughn/Winner study asked students taking the SAT to voluntarily fill out a questionnaire about the number of years they participated in arts classes. While meta-analysis allows for broad observations – in this case, that students who took four or more years of arts classes had the strongest SAT scores – it tends to gloss over the specific qualities of the arts programs or the learning experiences, due to the necessary heterogeneity involved in the process of combining disparate studies. What did those students do in their art classes that led to better test scores? Which activities triggered the cognitive development to improve math and verbal test-taking skills? To paraphrase reviewer Bruce Wilson’s commentary about the meta-analysis study “Mute Those Claims: No Evidence (Yet) for a Causal Link between Arts Study and Academic Achievement,” meta-analysis oftentimes takes a very limited range of factors into consideration, and excludes unanticipated outcomes.
One way to address this issue is to look at the findings of the qualitative and ethnographic research methodology. As indicated in the Martha C. Mentzer/Boni B. Boswell study “Effects of a Movement Poetry Program on Creativity of Children with Behavioral Disorders,” qualitative methods of reporting on research can be helpful in understanding the nature of the learning that is happening. Such methods include anecdotal records, observational checklists from videos of the sessions, questionnaires and interviews, and student work. In the study, the use of this methodology helped reveal the learning styles and creative thinking (defined here as “originality, fluency, and flexibility”) of two boys, aged 7 and 10. With meticulous documentation and citations from earlier studies, this study captured the rather nuanced development of behavior changes. As reviewer Karen K. Bradley comments, “… since one boy improved in social behavior and the other in motor coordination, the union of creative movement and poetry writing provided a ‘stronger fabric’ for development, especially for children of different and challenging learning styles.” She continues that “the most useful data for understanding the outcomes the boys achieved came from anecdotal records. The field needs to recognize that movement analysis may offer the clearest depiction of what cognitive or behavioral changes occur through involvement in dance.”
In fact, this last sentence could be applied to every arts discipline. Analysis of what exactly is being learned can reveal what changes happen cognitively in all of the artistic disciplines.
The research included in “Critical Links” is so diverse, from the methodology used to the discovered outcomes, that it is difficult to make a statement about the merits of one study over another. Taken as a whole, the Compendium offers strong evidence of correlation between arts study and academic and cognitive development.
Despite the comprehensiveness of the commentary sections, two questions seemed to be missing from the conversation:
1. If visual arts, music, multi-arts, dance, and drama all possibly impact a similar skill set (e.g. reading), how do we know which arts programs are most effective at promoting these skills?
Before we even go there, it should be reiterated that the dance and visual arts studies in this Compendium are vastly outnumbered by the other studies (7 and 4, respectively, compared with at least 15). This tells us that if and when we suggest which of the arts to focus on to develop specific cognitive abilities, we first need further research in the visual arts and dance arts.
Yet there is a need to not only develop arts research within specific disciplines, but among the disciplines as well as across academic learning, as suggested in the second of Paul DiMaggio’s three fallacies in identifying the effects of the arts on communities. As described in DiMaggio’s prospectus for the “Taking the Measure of Culture” symposium at Princeton University in June 2002, the second fallacy is that of homogeneity of effects. “We …often speak as if the arts …have undifferentiated effects on people and communities, whatever these effects may be. So we ask, ‘does art education improve math learning?’ or ‘do communities with lots of artistic resources have stronger economies?’ It is unlikely that there are general answers to these questions. Effects may be heterogeneous due to interactions with other factors, so that benign effects of artistic resources or experiences are visible only in the presence of other factors that facilitate the expression of those benign effects…. [D]ecisive effects of specific kinds of arts programs on a relatively small proportion of communities will be hidden unless we know where to look for them.
In other words, we need to look at multiple and potentially confounding factors when determining the impact that any given arts activity has on an unrelated skill. We could certainly use more longitudinal studies across different academic disciplines if we are to determine how the skills that are emerging through research are being used in other contexts.
2. What is research methodology telling us about the quality of outcomes?
While it may seem that arts involvement is desirable, it isn’t usually clear how rich or textured student involvement is in other subject areas as a result of their engagement with the arts.
The most compelling studies in “Critical Links” – at least for me – were those that explored the nuances of learning that took place. One particularly resonated with me both in terms of methodology and in terms of outcome: “’Stand and Unfold Yourself,’ A Monologue on the Shakespeare & Company Research Study,” by Steve Seidel. In it, a team of research staff looked closely at the Shakespeare & Company’s National Shakespeare Institute to identify what made the program so successful and which elements of it could be transferable. (Note that there is already an assumption that the program is successful.) The program includes a one-month teacher training, followed by two months in which teaching artists guide about 400 students in 10 schools through the study and performance of Shakespeare plays.
What struck me at first was the ambiguity of the inquiry question: “How do participants [in the Shakespeare & Company program] identify the value of their participation for themselves?” The second question seemed even more elusive: “What elements of the program seemed most critical to creating those benefits?” I wondered how the study could possibly measure these things.
This is where the unique methodology of the study shone through. Unlike other studies, this one had the resources to assess two of the four years of students’ and staff participation. By the standards of the Compendium, this is a very long time. In addition, researchers held meetings with participants throughout the year and organized yearly retreats to extract what was being observed and learned; developed rich inquiry questions “around authenticity, academic rigor, applied learning, active exploration, adult relationships, and assessment practices,” and included teacher and student responses to these questions as evidence of the impacts and success of the program.
Along with these elements, Seidel focuses “on learning in … the language itself, acting, working in creative communities, and learning about oneself and linking that to social and intellectual development.” The findings were numerous, but most importantly, the study revealed that the complexity of studying Shakespeare plays allowed participants to delve into their own feelings and emotions. As Catterall comments, “the complexity of issues and emotions in the plays promotes word-by-word, emotion-by-emotion, thought-by-thought investigation of meaning. This step-by-step approach invites those who study Shakespeare to go deeply into their own experience, a process that is linked to all types of learning.”
So, do we all need to study more Shakespeare? Maybe. But I came away from this study thinking that, while it may not be realistic to replicate this kind of work extensively, it may nevertheless be prudent to invest in more in-depth qualitative research like this: research that asks more ambiguous inquiry questions and systematically measures both expected and unanticipated outcomes.
Building on the field’s existing and expansive arts education research, we have an opportunity to unlock the deeper inquiries about how the arts shape human development. If we can use what already exists (such as the wealth of findings from studies like those included in the Compendium), and take a step-by-step approach at looking at the nuances of learning, perhaps we’ll begin to form stronger links within and beyond arts education to develop rich, holistic learning experiences that can help shape future generations of critical thinkers, and creative learners and leaders.
- James S. Catterall, Book Summary
- Lois Hetland and Ellen Winner, Beyond the Evidence Given: A Critical Commentary on Critical Links (in this essay, Hetland and Winner take Catterall and editor Richard Deasy to task for inflating, in their opening and concluding essays, the degree to which causal interpretations can be drawn from the studies contained within the volume)
- Summary/synthesis by Americans for the Arts