Image by Florent Lemieux

If you’ve read a study about arts participation in the United States in the past few years, it’s a fair bet that it was authored, co-authored, or influenced by Jennifer Novak-Leonard. The University of Chicago researcher has maintained a breathtaking pace of output recently, nearly all of it focused on better understanding the ways in which people engage in and feel about arts and culture, broadly defined. The first five months of 2015 alone have seen the publication of no fewer than five texts listing Novak-Leonard as the lead author, picking apart arts participation statistics in every way imaginable over the course of some 250+ not exactly beach-reading pages.

The James Irvine Foundation, which has been a critical enabler of this work for nearly a decade, came out this spring with a cluster of arts participation studies by Novak-Leonard and others. Chief among these is “The Cultural Lives of Californians,” which synthesizes lessons from a new telephone survey of 1238 Golden State residents specially designed by Novak-Leonard and her colleagues at NORC at the University of Chicago. This so-called California Survey is an attempt to model a different approach to measuring cultural participation by offering a broader take on what “counts” as arts and culture than the statistics we typically hear about. Rather than limit the inquiry to questions about specific types of participation, the survey begins with an open-ended prompt about what role arts and culture plays in the respondent’s life:

People are involved indifferent types of activities that they enjoy or that are important to them. Please briefly tell me about any creative, cultural or artistic activities that you do.

If stumped, respondents were encouraged think broadly, with suggestions like “You could include anything you do that involves making music, dancing, roleplaying or telling stories, writing or making art. Also think of activities when you make something, or build, customize or repurpose something to your liking.” The resulting range of activities recorded is highly illuminating, and includes such seemingly off-the-wall responses as sandblasting mirrors, customizing old cars, and my favorite, making bowties that incorporate people’s personality characteristics. Clearly, Americans (or at least Californians) define cultural participation very broadly indeed. Importantly, about 15% of the responses had no correspondence with later closed-ended questions in the survey that asked about a range of specific types of participation, meaning that surveys that rely solely on closed-ended questions, such as the NEA’s Survey of Public Participation in the Arts, likely miss a substantial portion of the cultural activity that goes on. Yet even with this open-ended (and, let’s face it, highly leading) approach, a little over 6% of respondents to the California Survey either only mentioned sports-related activities or couldn’t come up with a single way in which they engage culturally, which probably sets a pretty hard upper bound on the percentage of the population that is culturally active for reasons elaborated on below.

While the bulk of “The Cultural Lives of Californians” is devoted to findings that are familiar from previous participation literature, a few new perspectives are offered. In particular, the report brings valuable attention to a consistent pattern of lower participation among immigrants, even those of the same ethnic background as non-immigrants. The authors explain that immigrants work more hours and have less leisure time than the rest of the population, which could account for the difference. “Cultural Lives” also documents a significant pattern of decreased art-making (as distinct from arts attendance) as people age, which overshadows the effect of income and education on that particular form of participation.

More than anything, the report repeatedly emphasizes the wide range of ways in which Californians engage with culture, and makes an argument for funders, researchers, and arts practitioners to use a wider aperture in conceptualizing participation. In particular, art-making activities that take place inside the home or outside of traditional arts spaces receive much attention from the authors, and the survey provides data on the demographic breakdowns of participants in such activities at a greater level of detail than we’ve seen in any other study.

Informal Yet Not (Fully) Inclusive

It turns out that “The Cultural Lives of Californians” is very timely for Createquity’s research process for that reason. A big question running through our investigation into socioeconomic status and arts participation earlier this year was whether poor and less-educated adults are less likely to participate in the arts generally, or just have different patterns of participation that don’t show up as readily in surveys and market research that focus on traditional nonprofit institutions. It’s long been an article of faith in the community arts field that the latter of these two assumptions is true; in other words, that the people who are not coming to the symphony or the ballet are instead experiencing music, dance, and other art forms in their homes, places of worship, and other “informal” contexts.

Despite its explicit goal of uncovering the hidden ways in which Californians participate in culture, there is not a lot of evidence in “Cultural Lives” to suggest that poor and less-educated adults are more likely than others to engage in the informal arts. For example, 60% of respondents with less than a high school diploma spent time with friends or family making music, compared to 73% with an advanced degree; there is a similar slightly upward trend by income associated with that activity. While we don’t see the kinds of dramatic differences across education and especially income as we do with various forms of physical attendance – the ones noted above are within the margin of error for the survey – there is no category of participation that demonstrates a clear pattern in the opposite direction, of higher participation by adults in the bottom income quartile or who never attended college. Put another way, while poor and economically disadvantaged adults may be more likely to sing to themselves or dance with friends than see the opera, the same is true of people with college degrees and well-paying jobs.

Survey Says…Don’t Trust (Most) Surveys?

Discussion about “The Cultural Lives of Californians” will no doubt focus primarily on its content and findings, but the study is no less notable for what it has to teach us about survey methodology and the art of measurement.

The project of which “Cultural Lives” is a part represents an ambitious and serious bid to move the practice of measuring cultural participation forward. In addition to the survey and accompanying analysis, the Irvine Foundation has also published a technical appendix that is longer than the report itself; a review of theoretical constructs and issues in arts participation research; and an entirely separate analysis of the California-based respondents to the NEA’s Survey of Public Participation in the Arts (SPPA). This last bit is especially important because it provides for direct comparisons between the “old” and “new” ways of measuring participation – the researchers even used the exact same question wording across surveys in many cases in order to facilitate such comparisons.

As a result, and because of the tremendous level of transparency provided by the authors, we can learn a lot about the effects that survey design and administration have on cultural participation data. Fortunately for the sake of making this article interesting, but unfortunately from the perspective of researchers and practitioners, it turns out that conclusions can differ substantially depending on the choices made in constructing the survey.

In her forward to “Cultural Lives,” Irvine Foundation arts program director Josephine Ramirez declares, “the new narrative is not about decline! Californians actually have a deep interest in the arts and lead active cultural lives.” While the report can’t comment one way or another on decline since there is no longitudinal component to the data, what I take Ramirez to mean here is that when you broaden the definition of what’s included in arts and culture, all of the sudden you see a lot more people participating than you did before. That is true, but it turns out that much of that apparent increase is attributable to the survey itself.

The values observed in the respondent set to “Cultural Lives of Californians” are consistently higher than seen in comparable questions in the SPPA and other data sources, sometimes dramatically so. For example, the California Survey reports a prevalence of acting six times higher than California respondents to the SPPA; four times higher for purchasing or acquiring art; and double the rate of attending a cultural fair or festival. These are not deceptively small adjustments within the margin of error; some of the differences here approach or exceed 30 percentage points. To their credit, the authors noticed this pattern and even made a table to summarize some of the most eye-popping differences in the technical appendix:

By way of explanation, Novak-Leonard et al. cite the broader frame of the California Survey as a whole (although this doesn’t explain the difference in directly comparable questions), and credit the open-ended question about participation at the beginning of the survey and example prompts throughout for jogging respondents’ memories. Most notably, they cast doubt on the methodology of the SPPA, implying that the abrupt transition to the set of questions about arts and culture as well as the switch in recall period from the past week to the past year are confusing for participants and result in false negatives.

These explanations are quite plausible and (with the exception of the broader frame, as discussed above) very likely account for at least some of the gap between survey results. However, the authors seem to go out of their way not to consider another possibility, which is that the California Survey may suffer from increased nonresponse bias. The response rate to the survey is substantially lower* than that of the Survey of Public Participation in the Arts, which opens up a higher risk for bias in light of the targeted nature of the survey. The SPPA is an attachment to a larger, multi-modal measurement exercise called the Current Population Survey that is administered by the Census Bureau; thus, respondents to the CPS agree to participate in the survey without knowing that they’re going to be asked questions about their arts engagement habits. By contrast, the California Survey is a telephone-only survey that was upfront about being interested in people’s cultural lives, with the result that people who have richer cultural lives may have been more likely to respond. This is seen perhaps most clearly in the discussion of volunteering for an arts organization, as follows:

Forty-one percent of California adults donated money, goods or services to an arts or cultural organization or project and almost one-third (30 percent) otherwise volunteered to help an arts or cultural organization….These rates of support are substantially higher than those generally seen in other studies that ask about support for arts and culture….In addition, a report released by the United States Census Bureau in early 2014 finds that approximately 26 percent of U.S. adults aged 25 and older volunteered in any way…

In other words, the California Survey found that adults in California volunteer for arts organizations at a higher rate than adults nationally volunteer for anything!

This is not the first time that we’ve seen this phenomenon in evidence with participation data. In 2008, Irvine published the results of a survey by Novak-Leonard (then known as Jennifer Novak) and Alan Brown that purported to measure cultural engagement patterns among Californians living in the state’s inland regions. Createquity’s analysis of that report noted the following:

While most of WolfBrown’s measures cannot be compared with those in the SPPA, many that do show significantly higher levels of activity. For instance, 30% of Cultural Engagement respondents said they “regularly” attend stage plays; only 12.5% of SPPA respondents in the Pacific region claim to have done so even once in the past year. Six percent of Cultural Engagement respondents perform dances, but just 2.1% of Pacific region SPPA respondents do.

Now, with even more direct comparisons possible with the SPPA, we see such differences persisting or even expanding. Is the variance the result, as the authors suggest, of the SPPA not giving people enough to go on as they try to think of ways in which they’ve participated in the arts? Or is the culprit nonresponse bias, meaning that the California Survey’s numbers are inflated? My guess would be that it’s some of both, and that “true” participation rates are somewhere in between these estimates. Seemingly mundane details like where a question appears in a survey have indeed been shown to have surprisingly large impacts on results in some cases. However, in its 2012 review of the threat of nonresponse bias to public opinion surveys, the Pew Research Center observed, “survey participants tend to be significantly more engaged in civic activity than those who do not participate, confirming what previous research has shown…This has serious implications for a survey’s ability to accurately gauge behaviors related to volunteerism and civic activity. For example, telephone surveys may overestimate such behaviors as church attendance, contacting elected officials, or attending campaign events.”

Regardless of the explanation, the fact that two different surveys asking the exact same questions of the exact same target population could come up with such disparate results has important implications for anyone who uses survey research in their work. Ultimately, the lesson here is that survey results are much more sensitive to both design and administration choices than we would like to think. And with response rates for surveys getting worse and worse, there seems to be a strong suggestion that even with a random sample and a professional approach, a survey that signals too strongly the subject under study can bias results if part of what’s being measured is interest in that subject. That should be a heads up for every arts organization that surveys its own audience members, an extremely common practice throughout the industry. If your market research relies on people taking the time to tell you whether they’re interested in what you have to offer, odds are you’ll be hearing from the most interested people.

Fortunately, all hope is not lost for cultural participation research. As if publishing five reports in five months wasn’t enough, Novak-Leonard has been working closely with the NEA for several years now to address some of the deficiencies in the SPPA and introduce more inclusive questions. A marriage of the more thoughtful survey design elements from the California Survey with the increased resources for ensuring a representative data set that the federal government can provide would result in the best cultural participation data we’ve yet seen. Let’s just hope that those government statistics-gathering efforts can survive political pressure and budget cuts long enough for that vision can come to pass.

* NORC researchers actually calculated two versions of the weighted response rate, each of which used different assumptions for estimating what percentage of telephone numbers that didn’t result in a completed survey were eligible to be included in the first place. The higher of these response rates, 31.6%, is the one shared in the report’s technical appendix. Following a lengthy email exchange with two individuals at NORC who worked on the survey, my sense is that the lower number, 9.9%, is likely a better estimate. I am happy to share the details of the exchange with anyone who is interested. By comparison, the response rate to the Current Population Survey, of which the SPPA is a part, is 75%.

Perhaps no arts-related research study is cited as frequently in the mainstream media these days as Americans for the Arts’s gargantuan economic impact survey, Arts & Economic Prosperity III. Its key message, that the nonprofit arts sector is responsible for $166.2 billion in economic activity nationwide, has been hammered home relentlessly to policymakers, politicians, grantmakers, and arts managers around the country since the report’s initial publication in 2007. Americans for the Arts clearly sees the report itself, along with the general theme of economic relevance, as central to its overall advocacy strategy: as AFTA’s Director of State and Local Government Affairs, Jay Dick, put it while speaking at the 2007 Wyoming Arts Summit,

In the past, when we went to do funding for the arts, we said, “fund the arts, it’s good for the soul.” […] That’s true, [but] it doesn’t work anymore. You know, we have to have a business argument for it. So, “fund the arts because it’s good for the soul—and they bring to the jobs to the economy and they bring taxes back into the [government].” That’s what we have to do.

Not everybody, however, is convinced. For one thing, the dual role that AFTA assumes as impartial researcher and impassioned advocate renders the report vulnerable to criticism on the grounds of bias, criticism that the report itself goes to great lengths to counter. Moreover, even assuming the numbers are accurate, thinkers from Tyler Cowen to Greg Sandow have assailed the very concept of economic impact studies and their utility in advocacy discussions. Indeed, when last we paid a visit to the Arts Policy Library, the authors of Gifts of the Muse: Reframing the Debate About the Benefits of the Arts argued that relying too heavily on economic and other “instrumental” arguments for the arts is a trap, pointing out that that economic impact studies

…receive criticism because most of them do not consider the relative effects of spending on the arts versus other forms of consumption—that is, they fail to consider the opportunity costs of arts spending. Some economists dispute the validity of the multipliers used in economic studies because they assume that spending on the arts represents a net addition to a local economy rather than simply a substitute for other types of spending.

The tension between the two approaches led journalist John Stoehr to set up a kind of debate between the AFTA and RAND texts in a 2007 article for the Savannah Morning News, a debate that in Stoehr’s mind Gifts of the Muse ultimately won.

As always, though, much is lost in a public debate about a study when most of the participants have only read the press release. The full Arts & Economic Prosperity III report contains some 314 pages of findings, facts, and figures, including 27 multipage data tables and one of the most thorough explanations of methodology I’ve ever encountered in a research report. So let’s dive in and find out what Arts & Economic Prosperity III actually has to say about the economic impact of nonprofit arts organizations in the United States.

Summary

The first thing to understand about Arts & Economic Prosperity III is that it is comprised of many studies in one. It makes use of an innovative distributed data-gathering strategy that involved partnerships with organizations and agencies in some 156 study areas across all 50 states and the District of Columbia. The study areas included 116 cities and counties, 35 multi-county regions, and five entire states. These 156 partners were tasked with identifying and coding the universe of nonprofit arts organizations in their area, using the Urban Institute’s NTEE codes as a guide; disseminating, collecting, and reviewing organizational expenditure surveys; conducting audience-intercept surveys at a minimum of 18 representative events in the area; and paying a modest cost-sharing fee (though the study authors take care to note that no community was turned away out of inability to pay this fee). The partners collectively produced 6,080 completed organizational surveys¹ and interviewed some 94,478 audience members about their spending over the course of 2004 and 2005.

Americans for the Arts then collected this data and created four sets of numbers with it. First, it tabulated the total organizational expenditures in each community, noting the breakdown of artistic versus administrative versus capital expenses, and calculated the averages for each of six community cohorts based on population size, labeled A-F (0-49,999, 50,000-99,999, 100,000-249,999, 250,000-499,999, 500,000-999,999, and 1 million and up). Second, AFTA tabulated the audience expenditures related to arts events in each community (excluding the cost of admission), and calculated the averages in the same way. Third, researchers at the Georgia Institute of Technology ran both the organizational and audience expenditure numbers through a sophisticated econometric tool called an input/output model to estimate the cumulative local transactions that those expenditures might cause in each community. Though those results aren’t reported in the study directly, the researchers ran them through another set of models to come up with estimated resident household income, employment figures, and state and local government revenue that could be attributed to the organizational and audience expenditures. Finally, taking the averages for each of the six population groups, AFTA calculated national estimates for all six of the metrics listed above by mapping the averages on to the populations of the 12,662 largest cities in the United States. (Note: only the 116 cities and counties, the smallest unit studied, were used in the calculation of the national estimates.)

The resulting figures will look familiar to anyone who’s read a news story about arts funding lately. Nonprofit arts organizations account for $63.1 billion in organizational spending and $103.1 billion in audience spending nationally, for a total annual industry footprint of $166.2 billion. Collectively, these expenditures support an estimated 5.7 million full-time equivalent jobs, $104.2 billion in annual household income, $7.9 billion in local government revenue, $9.1 billion in state government revenue, and $12.6 billion in federal income tax revenue.

The study reports that the typical attendee forked out $27.79 per event on top of any cost of admission—what the study calls induced spending—on things like meals, refreshments, clothing, lodging, souvenirs, child care, and transportation. As one might expect, the numbers vary dramatically between local and nonlocal attendees (nonlocal defined as traveling from outside the county). Tourists spent more than twice as much on average as residents on event-related items ($40.19 vs. $19.53), the biggest increases coming from single-night lodging (more than tenfold) and transportation (nearly threefold). Tourists also spent 40-50% more on average than residents on meals/refreshments, gifts/souvenirs, and “other.”

Arts & Economic Prosperity III is, as the title implies, the third study in a series. (A fourth is planned for launch next year.) Comparisons to the previous edition, using data collected in 2000, show a growth of 24% in the five years between studies—a rate that sounds impressive at first, but was actually outpaced slightly by growth in overall US GDP during the same period. Twenty-five communities were represented in both the second and third editions of the study; this group grew more than twice as fast as the national estimates.

Some of the most interesting statistics from the study aren’t the ones that usually make it into the press release or the media alerts. For example, the 6,080 participating organizations reported an average of 125 volunteers who donated a mean of 45.3 hours each in a year. That’s a simply astounding level of volunteerism. The total of 191,499 hours is valued at $3.4 million using Independent Sector’s 2007 valuation of volunteer time. Those hours have no economic impact as defined by the study (and are not included in the national estimates of economic activity), but the study authors take care to note that they add much value to artistic communities anyway. In addition, 71% of responding organizations received in-kind support of one kind or another, valued at an average of $47,906 per organization. The largest source of such support was corporations at 61%, with the balance from individuals, local and state government, local arts organizations, and other.

Though not reported in the study text, the audience demographics (Tables 25-27) are worth a look. Women consistently outnumbered men by nearly a 2:1 margin in almost every community. Assuming the survey samples were representative, we can conclude that arts audiences are VERY well-educated (more than 83% reported having a college degree, and fully a third had one or more graduate degrees) as well as quite affluent (30% reported a household income of more than $100,000). More than 80% of audience members are 35 or older. These results tracked quite consistently between urban and rural areas and between residents and tourists, with the exception that audiences tended to be a bit richer and better-educated in big cities.

I also found Table 9 notable for its breakdown of organizational expenditures on artists. In almost all communities, artists themselves get a truly tiny slice of the money that goes to support the nonprofit arts. Their share was only 11% overall, and ranged as low as 7% in the group of the smallest cities and counties. In a few areas, like Lauderdale County, MS and the entirety of Northwest Minnesota, the total amount spent on artists in a year was not even enough to pay one person’s salary.

Analysis

Since there seem to be a number of misperceptions about Arts & Economic Prosperity III in the media and elsewhere, perhaps the most helpful step I can take at the beginning is simply to delineate what the study is and what it is not. I can tell you that Arts & Economic Prosperity III is:

• A serious study. One thing that becomes clear from reading the entire report is that the people behind Arts & Economic Prosperity III invested significant time and care into getting the numbers right. As I mentioned earlier, A&EP III has one of the most comprehensive explanations of methodology I’ve seen in a research study – a full ten pages of information representing one-third of the non-appendix portion of the report. The authors even make a valiant effort to explain the mechanics of input/output analysis, an advanced econometric technique involving matrix algebra and other graduate-level technical sophistications. Time and again, as questions popped into my mind while I was reading along, I would find them answered in the next section or by the end of the report. Wary of any perception of bias on the part of an advocacy organization tasked with making the case for government funding of the arts, researchers took numerous steps to ensure that the final estimates would not be skewed too far in favor of that conclusion. These steps included large decisions with major implications—the country’s two largest cities, New York and Los Angeles, were excluded from individual study in part because of their outlier status among American arts scenes; areas with unusually high economic activity for their population group, like Teton County, WY and Laguna Beach, CA, were excluded from other national estimates—and small details—like the fact that the audience expenditure survey checked to make sure respondents were over 18, or that organizations collected data throughout the year in order to guard against effects of seasonality.
• A legitimate estimate of total annual nonprofit arts organization and event-related audience expenditures in the United States. Even if you find yourself confused or unconvinced by the input/output model, that $166.2 billion number has nothing to do with it. The organization expenditure estimate is a direct extrapolation from the responses of 6,080 survey participants (which is quite a robust sample) based on the populations of the communities in which they operate. There’s nothing mysterious about this part of the study.Likewise, the audience expenditures—which don’t include tickets or admission prices—are extrapolations of the information from 94,478 survey respondents and everyone in their party (so, in actuality, a sample of nearly 300,000). By excluding airfare and more than one night’s worth or lodging, researchers did their best to limit their inquiry to expenditures directly linked to arts events that would mostly be staying in the local area.
• Clear evidence that the arts are a big deal in this country. The core takeaway of Arts & Economic Prosperity III – that “the arts mean business” – is amply demonstrated by the data. $166.2 billion is a lot of money, well more than one-thousand times the direct support provided by the National Endowment for the Arts in 2005. The $63.1 billion represented by the organizational expenditures alone is more than the revenue figures for spectator sports, furniture stores, coal mining, or the hunting, fishing, and logging industries combined. And as the authors point out, most industries can’t claim the same kind of “induced” spending—related payments made by consumers to third parties in connection with a core purchase—that the arts can. Even if the numbers aren’t dead on—a possibility I’ll explore in a bit—the point is clear: nonprofit arts organizations play a far more central role in the nation’s economy than commonly believed.

On the other hand, Arts & Economic Prosperity III is not:

• A perfect study. Despite the authors’ seriousness of intent, the study does contain a few errors, idiosyncrasies, and other less than ideal aspects of its construction. These range from embarrassing but ultimately unimportant mistakes like the mislabeling of the audience income demographics in Tables 25-27 (the last column in each table should read “$100,000 or More” instead of “$120,000 or More”) to potentially more significant issues like the inclusion of both Miami and Miami-Dade County among the 116 cities and counties used for the national estimates, which would lead to an over-representation of Miami’s organizations and audiences in the sample. Other issues will be examined when we take a look at the actual numbers.
• A demonstration that the arts cause economic growth. This is perhaps the single most prevalent myth about Arts & Economic Prosperity III and economic impact studies in general, a myth that is in no way dispelled by language like “the nonprofit arts industry generates $166.2 billion in economic activity every year,” that often accompanies the report. Merely counting up the activity associated with the arts in that community doesn’t show that the arts created that activity. Indeed, they easily could have just pointed it in a different direction. If there were no arts, would the audiences who spent $40 to buy dinner across the street have gone hungry instead? Would the office bookkeeper for the local museum not have found another job elsewhere? As I understand it, the arts (or anything else) can cause a net local increase ineconomic activity under traditional definitions essentially in two ways: 1) if they satisfy an unmet need such that people are motivated to spend more money on the arts than they would have spent on other things, thus inducing demand and ultimately driving a higher standard of living; and 2) if they draw money into a community from outside of it. The arts can actually make a pretty decent case for the latter on a local level, thanks to cultural tourism. But once you combine all of those local communities together to make a national estimate, all of those “nonresidents” of your county—with the exception of international travelers—suddenly become residents of the good ol’ USA, and the economic activity associated with the arts is no longer being drawn in from outside the community but merely shifted around within it. As for the first way of creating value, it’s anyone’s guess as to whether and how much that really happens. Essentially, we would need evidence that the same people, on average, are willing to spend more money over the course of a year to attend arts events than they would if there were no arts events to attend. So willing, in fact, that they would take steps in their lives to ensure that they have more money to spend on such things, which (by economic logic) would mean that they would increase their own productivity and value to society, thus making us all better off. No one, to my knowledge, has conducted a study like that, though some researchers have made strides in showing a causal relationship between arts activity and other indicators like housing prices.
• Particularly useful for policy decisions on its own. Let’s say I have some money to give out to improve the city, and I have to decide how to spend it. You come to me and you say, “you should spend that money on nonprofit arts organizations. Nonprofit arts organizations spend a lot of money.” I reply, “umm…okay, that’s interesting.” You go on: “nonprofit arts organizations employ a lot of people.” “So do our pharmaceutical and insurance industries,” I answer. “Should we subsidize them as well?” Finally, you bring out the big guns. “Nonprofit arts organizations produce revenue for your tax base.” “Well, clearly you are already doing a great job of that without my support,” I conclude. “I don’t see a reason why I should give you any.” Do you see how most of these arguments, in a vacuum, are kind of non sequiturs? If one is making an economic argument, policymakers need to know not just what the arts do now but what they can do in the future with an additional investment—their investment. And they need to know how that compares with other potential recipients for that investment. I believe that there are ways to do this, but unfortunately, this particular study is hardly…
• A study of the arts’ “return on investment.” Out of all the report’s assertions, the only one I’d describe as downright false appears right there in the introductory letter on page one—and never again in the report:
  

  Our industry also generates nearly $30 billion in revenue to local, state and federal governments each year. By comparison, the three levels of government collectively spend less than $4 billion annually to support arts and culture—a spectacular 7:1 return on investment that would even thrill Wall Street veterans.

  It’s only a seed, but it’s been enough to sprout numerous other attempts to use this logic (like in this piece from earlier in the month in Miami-Dade County). This much is true—a 7:1 return on investment would indeed thrill Wall Street veterans. It’s too bad the examples aren’t remotely comparable. The $4 billion in government investment and $30 billion in government revenue are two different beasts, apples and oranges. As the report itself tells us, nonprofit arts organizational expenditures total $63.1 billion—which means that the $4 billion coming from the government only accounts for about 6% of this total. Take away that 6%, and you’d still have 94% of those expenditures left—and, presumably, something like 94% of the tax revenues. So, that $4 billion in government investment is really only “responsible” for that last 6%—which turns out to be about $1.9 billion, or considerably less than a 7:1 return (more like 0.5:1, for those keeping count). Now, in fairness, the real story is probably more complex than this—surely the government’s impact is not strictly linear, but makes certain projects possible where none had been before, and communities may be able to leverage that support in other ways. But to realize what a junk statistic this is, think about it this way: if a state, oh, let’s say Pennsylvania, were to get rid of its arts funding entirely, all of the sudden it would be able to claim an infinite return on investment from any arts-related tax or other revenues that come in after that! I’m not sure this is really the line we want to be pushing in these battles.

Now that we understand what A&EP III is trying to do, let’s take a close look at the numbers the researchers actually came up with. I’ll divide these into five categories: the organizational expenditures, the audience expenditures, the volunteer contributions, the input-output model, and the national estimates.

Organizational expenditures

The collection of the organizational expenditure data for each community was probably the simplest aspect of the study, so I think it’s reasonable to assume that, in most areas, the numbers represent decent estimates. A few caveats do apply, however, which I’ve listed with the direction in which they are likely to have skewed the totals (if any):

• First, it worries me a little that the partner organization in each community was given the autonomy to implement the study themselves. While it was probably the key factor in making the scale of the study possible, this distributed approach opens up a number of quality control and consistency issues. For example, who from the organizations was conducting the surveys? Was it senior management? Program staff? Interns? LIKELY IMPACT: UNKNOWN
• The report mentions that the Urban Institute’s NTEE designations were used as a starting point for identifying relevant arts nonprofits in their area. Hopefully, partners would have gone the extra mile to edit those lists, but I can tell you from experience that going by the NTEE codes will tend to result in missed organizations, sometimes important ones, that are coded incorrectly or not at all. LIKELY IMPACT: SKEW LOW
• The study only counted the numbers for organizations that responded to the survey, which means that for communities that saw less than 100% response (which was most of them), there’s almost certainly an undercount. (It also means that communities that had higher response rates were over-represented in the national estimates.) Response rates ranged from 10.4% to 100%, with an average of 41.3%. LIKELY IMPACT: SKEW LOW
• Though responding organizations were asked not to include grants to other arts organizations, any payments to other arts organizations (for example, a presenter paying a nonprofit chamber ensemble, or renting performance space from another nonprofit) could result in double-counting. LIKELY IMPACT: SKEW HIGH
• When you get down to the itemized level, there are some bizarre oddities in the data that, taken together, throw a bit of a shadow on the rest of the numbers. For example, in Table 9, Allegheny County (Pittsburgh) arts organizations are shown as spending nearly eight times as much on artists as Philadelphia, despite having only a third of the total expenditures. Similarly, Jefferson County, AL (Birmingham) is shown as spending more than three times as much on artists as Baltimore, despite budgets only 40% the size. LIKELY IMPACT: UNKNOWN

Audience expenditures

A&EP III employed the audience-intercept method for collecting information about audience expenditures, which from what I can gather is similar to the method used for exit polling in national elections. The caution I mentioned above about the autonomy of the partner organizations applies even more strongly to this portion of the study, as administering survey instruments in person is something usually done by professionals. The survey asked audience members not to report expenditures on airfare, presumably because most of that spending would not impact the local community (also because it’s unlikely that most tourists had flown to the area specifically to see that one event, which is the rationale behind counting only one night of lodging). Other notes and caveats include:

• Audience members provided information about their entire party, which might have decreased the reliability of the data since the practice assumes that respondents knew what other people in their party spent. I’d think this would be more likely to result in undercounting (missed purchases) than overcounting. LIKELY IMPACT: SKEW LOW
• Any spending on concessions at the event (e.g., buying a glass of wine in the lobby at intermission) would be double-counted, since that money would become revenue for the organization and eventually show up in its expenditures. I believe the same is true for items bought at museum gift shops. LIKELY IMPACT: SKEW HIGH
• Though audience members were instructed to report lodging expenses for the night of the event only, it’s a bit questionable how attributable some of these expenditures really were to the arts event. For example, if someone bought an outfit to wear that night, does that mean they wouldn’t have bought the same outfit on some other occasion? If someone was in town overnight, does it mean that they were there specifically for that event? LIKELY IMPACT: SKEW HIGH

Volunteer contributions

As mentioned in the previous section, the volunteer hours reported by nonprofit arts organizations are extraordinarily high. According to the study, volunteers supply the labor equivalent of two full-time staff positions to the average arts organization each year. Upon closer examination of the numbers, I couldn’t find any obvious red flags—while there’s some variation, nearly all communities reported a higher level of volunteerism than I would have expected, even when considering the contributions of board members, etc. I can think of only two plausible explanations. One is that organizations must be counting unpaid internships. The other is that some, especially smaller, organizations may be counting uncompensated time put in by founders or artistic/executive directors, which is likely to be substantial in many cases. These are not the kinds of things that normally come to mind when I think of “volunteer work,” but of course that is what they are, so I guess I’m inclined to take the results at face value. The valuation of volunteer time at $3.4 million comes from Independent Sector’s Giving and Volunteering in the United States 2006, which pegs the value of an average volunteer hour at $18.04 in FY05.

Input/output model

As explained by the report, an input/output model consists of “systems of mathematical equations that combine statistical methods and economic theory” that trace “how many times a dollar is respent within the local economy before it leaks out” and quantify “the economic impact of each round of spending.” The ”economic impact” in question is no more and no less than transactions: if I pay you $20 to serve me food, I have increased economic impact by $20 according to this definition. I won’t attempt to recreate the report’s detailed and extremely technical explanation of how the input/output model works; you can read it for yourself if you like. The basic idea is that for each community, a team led by the former chair of the school of economics at Georgia Tech, Bill Schaffer, constructed a matrix of the dollar flow between 533 industries based on data from the Department of Commerce and local tax records. After adjusting to include only local transactions, this table was then simplified to a matrix of purchase patterns of 32 industries plus households. The table was then run through an iterative model that, at each stage, sought to calculate the local requirements in terms of output to make possible the numbers seen in the previous iteration of the table. After a certain number of rounds (I think 12, but it’s a little hard to tell from the description), the numbers are all added up to get the total cumulative transactions made possible by an infusion of the amount of money represented in the organization and audience expenditures.

I don’t really have any complaint with the input/output model itself—it was constructed by a trained professional, and without having access to the actual spreadsheets and models used, there’s no way for me to verify its conclusions independently. The main issue is more conceptual: namely, that the model takes for granted that all of the money coming in as a result of organizational and audience expenditures is new money, money that would not have been available to the community otherwise. But of course this isn’t true: if people weren’t working as or for artists, and going out to see arts events, they’d probably be doing something else that would involve money—quite possibly more money than there is in the arts, given the high education levels of many in the field. One way to deal with this, at least on a local level, would be to simply exclude the spending of residents, figuring that what we care about is new money brought into the community by those outside of it. This methodology would break down significantly at the level of a national estimate, however. Perhaps this is why the study authors admit,

…as in any professional field, there is disagreement about procedures, jargon, and the best way to determine results. Ask 12 artists to define art and you will get 24 answers; expect the same of economists. You may meet an economist who believes that these studies should be done differently (for example, a cost-benefit analysis of the arts).

Indeed, I would have been very interested to see the results of a cost-benefit analysis, as it would seem to me to be a more relevant measure of the value of public investment in the arts. However, the input/output model is what we have, and so it is up to us to understand properly what it means. As with the expenditure totals, the impacts on things like employment, household income, and tax revenue are associations rather than causal links. The arts may account for 5.7 million jobs nationally, but that doesn’t mean that they’ve added 5.7 million jobs to the economy that wouldn’t be there otherwise.

National estimates

At first glance, one would assume that the national estimates of organizational and audience expenditures are almost certainly skewed low. As mentioned earlier, the study leaves out specific estimates for our nation’s two largest cities, New York and Los Angeles, instead assigning them the averages for the 1-million-and-up population group. This decision was made, according to AFTA’s Senior Director of Research Services, Ben Davidson, thanks in part to cost considerations as well as a desire to avoid overinflating the national totals. By how much does this downplay the overall national estimates of economic activity? Well, the average for the Group F (population 1,000,000+) bucket of cities and counties is $408 million in total expenditures by organizations and audiences. Two estimates for organizational expenditures alone are $5.8 billion for NYC and $1.5 billion for LA; assuming a similar spread on the audience side, we’re probably looking at a gap in the ballpark of $13 billion caused by not measuring those two cities directly. Furthermore, an untold amount of activity is left out because the study tabulated spending figures and estimated audience totals only from organizations that responded to the survey. One would hope that the most significant organizations in each community were more often than not in the responding column, but even so it’s likely that a fair amount of economic activity in the 156 study regions simply was not counted.

Despite the factors mentioned above, I think that there remains a pretty compelling reason to think that the national estimates are actually overinflated after all. The reason is simple, but easily missed. It is selection bias among the 156 study regions—and specifically, among the 15 communities in the smallest population category, that of cities and counties containing fewer than 50,000 people.

Think about it this way: in order to participate in the study, a community needed to have a nonprofit organization or government agency with the following attributes: a) a programmatic focus on the arts (preferably exclusively on the arts); b) the staff capacity, expertise, and interest to manage a research project that involved identifying and surveying all of the nonprofit arts organizations in their area and conducting in-person audience surveys at a minimum of 18 events throughout the year; and c) the financial capacity to participate in AFTA’s cost-sharing fee (note: though this fee was supposedly waived for any partners that couldn’t afford to pay it, it’s unclear to what extent partners who didn’t take the time to ask were aware that this was an option—the call for participants for Arts & Economic Prosperity IV, for example, does not mention the fee waiver.)

Out of the thousands of cities and counties in the United States with populations of less than 50,000, how many of them do you think meet these criteria? Do you think that there might be some important differences between the ones that did and chose to participate in the study versus the ones that didn’t? Like, for example, a LOT more arts organizations and arts spending?

Luckily, Arts & Economic Prosperity III studied a few different kinds of regions, including entire states, making some interesting comparative analysis possible. I put together a table below with the average economic activity per capita for each of the six population subgroups for the cities and counties, as well as the average for the 35 multi-county regions and the estimates for each of the five states studied. First, looking at the different population subgroups, rather than per-capita spending going down as populations get smaller and more spread-out as one might expect, there’s a big jump in both organization and audience expenditures from group B (50,000-99,999) to group A (under 50,000). More interesting, however, is the comparison between the cities and counties and the multi-county regions, and especially the entire states. A statewide count would not suffer from the selection bias discussed here: instead, it would incorporate urban and rural areas in proportions not all that dissimilar from the rest of the country. Similarly, some of the multi-county regions studied occupy large swaths of land, and their arts organizations could pool their resources to meet the requirements for participation in the study.

The average expenditures in the smallest city and county population group are absolutely off the charts. Group A cities’ and counties’ arts organizations spent nearly two and a half times as much per capita as the regional average and more than three times as much as the statewide average. Their audiences spent more than five times as much as the statewide average.² It’s not just group A, though—all of the city and county subgroups have per capita expenditures higher than those for the regions and states. In fact, the highest average for a state in the study was lower than the lowest average for a city/county subgroup. That’s not random. That’s selection bias.

So what happens when your national estimate is based on the city and county averages in Groups A-F (and especially sky-high group A) rather than the lower regional and statewide averages? Remember, the estimate is based on the populations of the 12,662 largest cities in the US—all the ones with populations of at least 500, according to Davidson—and more than 90% of those cities would have been in Group A.

Well, we can do a little exercise to sense-check the numbers. The total nonprofit arts organization expenditures nationally should be roughly equal to total arts organization revenues. I checked the Giving USA statistics for arts and culture, which include contributions from individuals, foundations, and corporations but not government support or earned income, and got an estimate of about $12 billion for 2006. Then I checked data from the National and Local Profiles of Cultural Support study that looked at the typical breakdown of income sources for nonprofit arts organizations and pegged the contribution of private donations from individuals, foundations, and corporations at 25% as of 1998. So if $12 billion accounts for 25% of arts organizations’ budgets, their total budgets should be around $48 billion—not outrageously off, but nevertheless only about three-quarters of AFTA’s $63.1 billion estimate. And 25% was the lowest estimate I found on the web of private contributions to nonprofit arts budgets. Now, this is not conclusive–it’s certainly possible that Giving USA itself undercounts the revenues of nonprofit arts organizations and foundation contributions. And maybe the breakdown of income sources was dramatically different in 2005 for whatever reason. But given the information we have, I think it’s fair to assume that AFTA’s expenditure numbers are probably not that conservative after all.³

Implications

So is Jay Dick right that we need “a business argument” for the arts? I think he is, and I think Arts & Economic Prosperity III is helpful in that regard—to an extent.

As I read Arts & Economic Prosperity III, I found myself coming back again and again to the theme of language. A&EP III is a serious study, one worthy of our attention and use for advocacy and research purposes. However, some of its potential impact is undermined by the hyperbolic language with which it is often presented to the press and politicians. The first several pages of the report are cluttered with spurious or misleading statistics and graphs that distract from the more strongly supported of the study’s findings. For example, on page seven there’s a graph that shows “jobs supported by nonprofit arts” apparently outnumbering several categories of specific professions, including lawyers, farmers, and computer programmers. Of course, there’s a crucial difference: the arts number includes all jobs supported by the industry (according to the study’s calculations–so, not just artists and arts organization employees but people at Staples, Guitar Center, etc.), rather than jobs in specific professions. Any of those industries—software, legal, agriculture—could likely draw up a similar graph to make itself look good. Or take the consistent use of the word “generates” when talking about the economic transactions associated with arts organizations and audiences, which clearly implies a causal connection that has not been shown to exist. There is a whole section entitled “Nonprofit Arts & Culture: A Growth Industry” that conveniently downplays the fact that the arts grew at rate slower than overall GDP between this study and the last one. And then there’s that bogus 7:1 “return on investment” figure that we dissected earlier. These overreaches ultimately provide fuel to the economic impact naysayers such as Cowen and Sandow, and may only encourage those with an axe to grind against the arts to sharpen their instruments.

That’s a shame, because I think there is a real case to be made for the economic impact of the arts. As we’ve seen in several smaller studies that focused on particular geographic areas, there seems to be strong evidence for a causal relationship between the density and proximity of arts providers today and growth in local real estate prices tomorrow. That, pure and simple, is economic impact right there. Furthermore, though I’ve spent much of this article talking about substitution effects and pooh-poohing the notion that all of the spending associated with the arts represents spending that wouldn’t have occurred otherwise, it would be equally foolish to assert that none of that spending represents new value being created, either through bringing money into the area via cultural tourism or improving quality of life such that people are willing to spend more than they would have otherwise.⁴ Even if the arts turn out not to be the absolute most surefire way to spur economic development in local communities in all cases, I think we can assume that they often represent one component of a successful growth strategy. And certainly we can argue with a clear conscience that the arts support real jobs, that they play a much bigger role in the economy than commonly assumed, and that public subsidization for the arts is not the same thing as a giveaway.

For the fourth edition of Arts & Economic Prosperity, which is in its planning stages right now, I hope Americans for the Arts will take advantage of its strong local partnerships and infrastructure to help fill in some of the gaps in what we know. The building blocks for causal analysis are already there. In fact, the audience questionnaire for A&EP III did ask audience members the reason why they were in town that evening, and one of the options was “I am here specifically to attend this arts event.” Alas, this information never found it into the data tables or the report itself. Another question on the survey could ask audience members what they would have done with their time and money instead that afternoon or evening if they had not attended the event. Even though this wouldn’t be the most reliable data in the world due to its reliance on self-reports of hypothetical situations, it would nevertheless get us a step closer to an understanding of the true economic impact of arts events. Finally, at the organization level, it seems obvious that government investment in the arts must have some return, we just have to be careful about attributing outcomes to it that would have happened anyway. So why not ask organizations where they would make cuts if deprived of government funds as part of the expenditure survey? If you really wanted to go nuts with it, you could literally ask them to submit a revised budget that doesn’t include government support and see what they end up cutting. This last approach, needless to say, would be difficult to pull off, but one wouldn’t need to do it everywhere to get some idea of what the overall picture would tell you. A few sample communities might be sufficient as a pilot.

We should be grateful to Americans for the Arts for investing as much time, capital, and seriousness of purpose into this research as it has. While there are a number of paths open for improvement of this work, the foundation upon which those improvements would be laid is solid. I look forward to finding out what Arts & Economic Prosperity IV will have in store for us.

¹Since the study focused on nonprofit arts organizations, spending by for-profit creative firms and industries was excluded, as was any spending by individual artists. However, government-sponsored arts councils and presenting facilities were part of the study, and so were select programs embedded within non-arts organizations, such as university presenters.

²In fairness, the authors did exclude Laguna Beach, CA from the organizational expenditures and Teton County, WY from the audience expenditures when calculating the national estimates. However, even with this precaution in place, the averages still compute to $210.76 and $251.93 respectively–far above the next-highest number in each category.

³Part of the problem, really, is that organizational and especially audience expenditures just aren’t that strongly correlated with population. Looking at the data tables, one sees huge variances in nonresident spending totals between cities in the same category, like Miami Beach ($72.2 million) vs. Lauderdale County, MS ($502k), or Philadelphia County ($565 million) vs. Suffolk County, NY ($5.7 million). I’m guessing there are a lot more Lauderdale Counties in this country than Miami Beaches. (Indeed, according to Davidson, this is the reason why the cities with a population of less than 500 were not included in the estimates.) If it were possible to extrapolate the national estimates using local estimates of economic activity rather than population size, this approach might yield a more reliable result.

⁴While this study documents attendance at arts events by nonresidents, it does not do much to show that their travel plans were dictated by those events. However, it does point us to resources that go farther: a 2001 study by Travel Industry Association of America and Partners in Tourism found that 65% of adult travelers attended an arts and culture event while on trip 50+ miles away from home, and that 32% of these (i.e., about 20% total) stayed longer because of event. And of those that stayed longer, 57% (or about 11% of all travelers) extended their trips by one of more nights. So we can infer from this that arts and culture events were directly responsible for one or more nights of lodging expenses for approximately 11% of adult long-range travelers in 2001.

Between 2009 and 2011, arts participation “increased” from 35% of the population to nearly 75%. Clearly we should have witnessed a paradigm shift in the arts comparable to the Renaissance in these two years, but sadly that’s not what happened. Instead, the National Endowment for the Arts, faced with a mountain of disappointing news about the rates of participation in its 2008 Survey of Public Participation in the Arts (SPPA), commissioned 3 separate monographs and wrote several notes of their own to explain the data, which led to a broadened definition of arts participation that covers three-quarters of the population. Because the SPPA data has been discussed in numerous other research documents and blogs, I will keep the summary of the original report to the most highlighted bullet points and spend more time outlining the history of the analysis.

SUMMARY

The SPPA has been conducted five times by the by the NEA since 1982 in partnership with the United States Census Bureau (with the exception of the 1997 survey, which was administered by a private research firm and is not comparable to other years). The report presents detailed findings from the 2008 SPPA alongside data from prior years. To allow data to be compared with previous years, the survey questions remain relatively stable.

The survey tracks the following kinds of participation:

• Attending arts events
• Experiencing recorded or broadcasted live performances
• Exploring arts through the Internet
• Personally performing or creating art
• Taking arts-related classes

The key indicator for participation for this and all previous versions of the SPPA is attendance at “benchmark“ events:  jazz, classical music, opera, musical plays, non-musical plays, ballet, and visits to art museums or galleries. In addition to the core benchmark activities, four modules within the survey included questions about internet and other media use, arts learning, reading, and leisure activities. To increase the response rate for the 2008 SPPA, respondents were only asked about 2 of the 4 modules. New questions focused on Latin Music, attendance at outdoor festivals, and technology. Sections on trips away from home and desire to attend more events were dropped. The response rate was 82%, for a total of 18,444 adults interviewed over two weeks in May 2008 by phone.

The report itself describes the results as “disappointing.” The percentage of adults attending at least one benchmark arts activity declined from 39% in 2002 to less than 35% in 2008, the largest drop recorded in any survey interval. Just as striking is the long-term trend; participation levels never dipped below 39% since the first survey in 1982 and even rose in 1992 when participation reached 41%. Exceptions to the recent declines are musical plays and art museums, which are both flat from the 2002 SPPA, as well as literary reading, which is also up from 2002.

Attendance at Benchmark Activities

In contrast to the population more generally, most audiences at benchmark activities held at least a college degree and/or pulled in an annual income of $75,000 or greater. Latin music, outdoor performing arts festivals, and art museums drew younger audiences than other genres, while arts and craft festivals, parks and historic sites, and jazz were more successful in reaching lower- and middle-income audiences. These profiles may not be surprising, but what has people concerned is that those aged 45 to 54, historically a large component of arts audiences, showed the steepest declines in attendance, and even the most educated Americans are attending benchmark activities less often than reported in earlier surveys. One possible factor is the economy, which had been in a recession for six months at the time of the survey. This might help explain findings that low-cost, low-travel activities such as researching art over the internet and reading rose compared to the previous survey.

Arts Creation, Performance, and Learning

Since 1982, the number of adults reporting that they had taken arts lessons at any time in their lives has been declining, and substantially fewer young adults between the ages of 18 and 24 have had lessons. For example, in 1982 61% reported having had music lessons of some kind, but only 38% reported the same in 2008.

However, the study shows some forms of participation and creation on the rise, with 10% of respondents reporting participating in at least one of the art forms within the past 12 months. Singing in a choir or vocal group drew the most participants, with 11.6 million adults, or 5.2% of the population, participating. Photography and film‐making increased from 12% in 2002 to nearly 15% of all adults.  Classical music performance or creation increased to 3.1%, after falling to 1.8% in 2002 from 1992 levels  (almost all forms of creating and performing dipped between 1992 and 2002). On the other hand, dancing, weaving/sewing, and pottery/ceramics continued to see long-term declines.

Media Participation

More Americans engage with performances through broadcasts or recordings through radio, internet or other electronic media than attend live arts events. Overall, 41% of adults watched, listened, or explored the arts through some form of electronic media, and the total number of adults watching or listening through broadcast media is double the number that attend performances. Only live theater still attracts more audiences than broadcasts or recordings of its equivalent, not including television or movies. Online, 39% of all Internet‐using adults, or 62 million Americans, viewed, listened to, downloaded, or posted artworks or performance at least once a week. (Keep in mind that this study was conducted four years ago.)

State and Regional Patterns

There are regional differences in how communities choose to participate in the arts. For example, the New England and Pacific regions reported high levels of attendance at most benchmark activities. The East South Central region, which includes Kentucky down to Alabama, reported the lowest participation rates in benchmark activities, but also showed the highest participation rates in choral singing.

History of Analysis

Soon after reviewing the SPPA results, the NEA commissioned independent researchers to mine the SPPA data for details on the following topics: arts education; the personal performance and creation of artworks; and the relationship between age and arts participation.

Case against Demographic Destiny

In Age and Arts Participation: A Case Against Demographic Destiny, Mark J. Stern of the University of Pennsylvania reports that it’s not the audiences that are greying, it’s our country. It’s true that Generation X and Millenials are participating in benchmark events at lower rates in their young adulthood than previous generations. But overall, age and generational cohort accounts for less than 1% of the variance of the total number of arts events that Americans attended between 1982- 2008. Other influences, particularly educational attainment, have a much stronger role in explaining arts participation. Although WWII and Baby Boomer generations do attend a broader array of benchmark events and attend more often, Stern argues that younger generations also have an appetite for diverse arts experiences and that “the ability of established or emerging arts groups to attract participants will have less to do with the age distribution of the population than with their ability to connect to the creative aspirations of their potential audiences.”

Beyond Attendance

WolfBrown’s Beyond Attendance: A Multi-Modal Understanding of Arts Participation suggests that the current participation numbers do not accurately reflect how Americans are choosing to participate in the arts. Using a broader definition of arts participation, Jennifer Novak-Leonard and Alan Brown conclude that three out of four Americans participate in arts activities. The new definition includes a fuller variety of artistic genres, including participation via electronic media, and personal arts creation. Interestingly, approximately 23% of U.S. adults participate in the arts, but do not attend arts events or institutions in person, suggesting there is an opportunity for arts organizations to engage communities through arts creation and performance without expecting that they will ever attend a performance or exhibition at a traditional venue. Finally, although arts attendance at benchmark activities has declined, rates of arts creation have remained stable at 41% between 2002 and 2008. This stable rate is sustained by the increase of online participation.

Arts Education in America

Nick Rabkin and E.C. Hedberg from the National Opinion Research Center (NORC) conclude in Arts Education in America: What the Declines Mean for Arts Participation that the most significant predictive factor for arts attendance at benchmark activities is arts education, rather than educational attainment or income. While it is not surprising that adults enrolled in art classes or lessons are most likely to have also participated in benchmark arts activities, it is important to note that most Americans who had arts education as an adult also had had arts education as a child. It’s clear from the data that childhood arts education has been declining over time, and Rabkin and Hedberg argue that reversing this decline will be necessary if arts education is to play a significant role in stemming the erosion of adult arts participation. Perhaps most notably, the report suggests that we need to know more about what kinds of arts education experiences inspire people to continue participating in the arts as adults.

NEA Research Notes and Audience 2.0

Three research notes and an additional research report from the NEA further analyze the geographic, community, and technological contexts of arts participation.

The first research note, “Art-Goers In Their Communities,” reports that Americans who attend arts performances, visit art museums or galleries, or read literature are particularly active members of their communities. For example, while more than half of adults who attended art museums or live arts events volunteered in the past year, only a third of the general population did so.  While it’s not possible to draw a causal inference based on this observation, it suggests that arts, literary, sports, and civic organizations may benefit from the creation of innovative partnerships to reach a larger shared audience.

The second note, “State and Regional Differences in Arts Participation: A Geographic Analysis of the 2008 SPPA,” takes an in-depth look at the regional differences already explored in the original report, including a detailed analysis for 32 states. For example, Oregon consistently ranks among the highest in attendance of the performing arts, including opera, jazz and classical music concerts.  Nebraska ranks high in the number of adults pursuing creative writing.

“Come as You Are: Informal Arts Participation in Urban and Rural Communities” focuses on the differences in participation patterns between rural and urban areas. The analysis reinforces the finding that arts participation in benchmark activities is greater in urban areas, but when “informal arts” (arts activities that are self-initiated, community-based, and often occur in homes, schools, and churches) are added to the mix, urban and rural areas participate at the same rates.  In addition, the study found that arts participation does not increase with metropolitan size beyond 250,000 people.

Audience 2.0: How Technology Influences Arts Participation reported that people who engage with art through media technologies are two to three times more likely to attend a benchmark activity.  Older adults, rural, and  minority groups are more likely to use media technologies to access certain art forms  than attend a live event. For example, more than half of Latinos used electronic media to engage with Latin music, and 20% of African Americans, more than any other ethnic group, used media to explore jazz.

Other Perspectives and Reactions

In 2009, the League of American Orchestras conducted its own study with McKinsey and reversed a long-held assumption that audiences will replenish themselves. The League’s “Audience Demographic Research Review” confirmed that participation is declining within and between generations and “we cannot assume that people will attend more as they enter the 45+ age group.”

Also in 2009, the NEA publicized a press release that used the survey results to declare the success of its Big Read literacy program.  For the first time in the survey’s history, literary reading increased from 46.7% in 2002 to 50.2% in the 2008.  Then Endowment Chairman Diana Gioia remarked in the press release that “this dramatic turnaround shows that the many programs now focused on reading, including our own Big Read, are working. Cultural decline is not inevitable.”

ANALYSIS

The SPPA’s high response rate (82%) and large sample size makes it the most reliable and representative longitudinal audience survey available to us. Because longitudinal analysis has been so essential to the implications of the data, though, it is worth taking a second look at the methodology from a historical perspective.  The first three surveys were conducted as a supplement to the Census Bureau’s National Crime Victimization Survey (NCVS), and the 1997 edition was conducted by a private consulting firm as a standalone survey. It was only starting in 2002 that the Bureau conducted the SPPA a supplement to the Census Bureau’s Current Population Survey (CPS), and although both the NCVS and CPS are based on a random sample of households, the two sampling designs do differ.  However, the impact of these differences have already been measured by the Census Bureau.

Other differences should be noted when comparing the surveys. Earlier surveys were conducted in person, rather than over the phone. This could increase the rate of affirmative answers as people may be more inclined to try to please the interviewer than over the phone. Furthermore, some earlier versions were conducted over a period of a year, rather than just a few weeks, which could impact either seasonality of attendance. Lastly, the fact that spouses or partners were used to collect data in 2008 could contribute to slightly more erratic attendance reporting (if spouses inaccurately report frequency). Taken individually, these are only minor differences, but collectively, could all of these changes lead to lower  reported rates of attendance? Possibly, yet it seems likely that all of these minor differences can be taken into account in a detailed statistical analysis and that there is very little, if any, impact on the participation numbers.

Given the strong correlation between arts education and participation, it would be helpful if the questions surrounding early arts experiences could be further developed. In Arts Education in America, Rabkin and Hedberg highlight that “participants were not asked about the depth, intensity, or longevity of their study in the arts, nor were they asked about their subjective experiences — how much they enjoyed or cared about learning in and about the arts. Private weekly piano lessons for 10 years and recorder lessons in a class of 30 second-graders for a few months are equivalent in SPPA data and recorded as childhood music education, provided that those experiences are remembered and reported.” Future versions of the survey might address this deficiency by asking about respondents’ early arts experiences in more depth.

It’s surprising that actually the reverse has occurred – arts education questions have been eliminated as others were added to keep the survey at a manageable length. Eliminated questions asked about arts lessons during specific age brackets and about in-school vs. out-of-school learning. Although Sunil Iyengar points to the Fast Response Survey System (FRSS) for K-12 arts education as a tool for learning more about arts learning in the country, it doesn’t help us to cross-tabulate early education experiences with adult participation.

IMPLICATIONS

Given the “disappointing” results of the 2008 SPPA, should we be as concerned as we are about the future of the arts?  With the exception of museums and musical plays, attendance is declining, subscriptions are down, and ultimately sustainability is in question for arts organizations large and small.  This year’s Americans for the Arts Convention was titled “The New Normal,” suggesting the financial difficulties artists and art institutions currently face won’t disappear with an uptick in the economy.  As the 2008 SPPA research notes and monographs point out, current trends in arts attendance are beyond the scapegoats of demography or the economy.  It seems these downward trends in attendance stem from deep changes in how we choose to engage not only with the arts, but  nearly all kinds of information, entertainment, and culture.

The flip side of The New Normal is that other kinds of arts participation, including active participation and engagement through technology, are at least remaining stable, and photography and filmmaking are showing significant increases. Furthermore, it’s important to note that more Americans engage with performances through broadcasts or recordings than attend live arts events.

So if The New Normal has an upside, why do we continue to give so much weight to attendance numbers? Is it productive for us to focus on “benchmark” percentages? I would argue that it’s only productive if we are convinced other modalities are not equally valuable within the spectrum of arts participation. It’s not enough to value live attendance over other kinds of participation because of institutional models that yield significant earned revenue from ticket sales. If Americans are showing an interest in participating through choirs or viewing art online, then arts organizations have an opportunity to connect with their audiences through active participation and through online media. Perhaps if we could find a way to monetize participatory arts experiences, arts organizations wouldn’t be as concerned about the effect of dwindling ticket sales on the bottom line.

Regardless of how we value the various participation modalities, we still need to have audiences for America’s best theater, dance, and opera. Nick Rabkin’s and E.C. Hedberg’s monograph Arts Education in America asserts that the most significant predictive factor for arts attendance at benchmark activities is arts education, rather than educational attainment or income.  Furthermore, Jennifer Novak-Leonard and Alan Brown in Beyond Attendance find that “having had any arts lessons increases the likelihood of arts creation by 32%.” Meanwhile, Novak-Leonard and Brown report that 23% of adults participate but do not attend, indicating that there may be certain kinds of participation that do not necessarily lead to attendance. All of this suggests that the SPPA should ask more in-depth questions about the kinds, duration, and intensity of individuals’ arts education, including formal and non-formal types of learning. We could then learn more about which types of educational experiences might lead to attendance and which will not.

Overall, it seems like the real story here is not about the decline in arts participation, but the shift in how we choose to participate. If Americans are still seeking a deep a personal relationship with art, perhaps our increased demand for personal involvement and social connectivity are creating new demands for participatory arts experiences. New research from the Greater Philadelphia Cultural Alliance shows that increased active or online participation can “be a gateway” to attendance, and this shift to participatory arts may point toward a broadening of consumption of the arts rather than a decline. Online, 39% of all Internet‐using adults, or 62 million Americans, viewed, listened to, downloaded, or posted artworks or performance at least once a week, and 45% participated in some form of creation or performance activity. It’s time for the SPPA survey to reflect this shift in participation through development of a new survey protocol, rather than continue to rely heavily on questions that are now over 30 years old.

Fortunately, the NEA agrees. Recently, the agency announced that it was revising the 2012 SPPA, noting that “it seemed time to revisit the way we ask about the arts in America.“ The primary purpose of the survey will be “to create baseline rates of arts participation inclusive of both traditional and non-traditional modes of participation.” The new survey questions will improve the ability to measure participation in emerging art forms and modes, including electronic media and non-formal learning opportunities. It will also ask more specific questions in regards to arts learning, asking specifically about informal modes of learning and whether early learning took place in our out of school. In perhaps the biggest shift in language, questions relating to attendance will not be tied to venue; the SPPA will ask if respondents attended an art exhibit, and if so, where. To maintain some ability to compare this study with previous ones, the 2012 SPPA will have two sets of core questions – the old core will be the benchmark attendance questions, and the new core will include the emerging art forms and modalities. By 2017 the old core will be completely replaced by the new core questions.

The 2012 SPPA will capture arts participation patterns in the aftermath of the biggest recession in 80 years, the heyday of social media, and the mainstreaming of the mobile web. Kickstarter, which arguably has helped democratize access to arts creation, didn’t exist in 2008, nor did the curatorial social media network Pinterest. Etsy, part of the Do-It-Yourself (DIY) movement where artists and artisans sell their wares online, has been profitable since 2009 and has raised $51 million in capital. Meanwhile, we know that arts attendance is down, but the 2008 survey does not prove that other types of participation are increasing. We shall see more clearly when the 2012 edition comes out (sometime in early 2013) if those who are participating online and in active art-making are at the vanguard of a long-term shift in participation away from attendance.

Further Reading:

• Sunil Iyengar, Taking Note: Accounting for Audience Impact — What Were They Thinking (and Feeling)!? (Art Works)
• Greg Sandow, Dire data (ArtsJournal)
• Webinar: Preview of Three New SPPA Reports 
• Betty Farrell and Maria Medvedeva, “Demographic Transformation and the Future of Museums“
• Molly Sheridan, And the Survey Says: Considering the NEA’s 2008 Survey of Public Participation in the Arts (NewMusicBox)

• The California Arts Council is in danger of losing its right to solicit voluntary contributions from California citizens through their state income tax returns. Though that wasn’t proving to be a very effective way of raising money anyway – the agency banked only $165,000 from CA’s nearly 40 million residents last year.
• Arts Council England has published an evaluation of its ambitious program to give out half a million free tickets to the theatre (in actuality slightly less than 400,000 were distributed).

MUSICAL CHAIRS

• Heather Pontonio has joined the Emily Hall Tremaine Foundation as its new arts program officer.
• Welcome Ayanna Hudson, new director of the NEA’s Arts Education program.

IN THE FIELD

• Whew! The Los Angeles Times published a brutal exposé earlier this month of problems at Watts House Project, a darling of the fledgling creative placemaking movement that attracted nearly half a million dollars in grants last year from the NEA’s Our Town program and ArtPlace and others. (Update: the NEA wrote in to clarify that while Our Town has supported projects around Watts, the grant is not associated with Watts House Project specifically.) According to the article, the Project and its founder Edgar Arceneaux have alienated some of the residents the organization is supposed to be helping by failing to deliver on promises and succumbing to mission drift. I found  this little bit of gotcha journalism particularly cute:
  

  As for [Rocco] Landesman, reached by phone in Washington, D.C., he said he based his positive impressions on a slide show by Arceneaux as well as a tour of the block, “and it all looked good.” He also talked to one enthusiastic 107th Street resident, Rosa Gutierrez, whose home received a bright flower mural as part of the program. He said he was not told she was on staff at Watts House Project.

  Consequences were swift. Arceneaux didn’t last the weekend as executive director of the organization, though the announcement leaves the door open for him to remain involved in another capacity. However, another former board member disputes elements of the article, presenting a compelling case that it was unfair to WHP. Nevertheless, the problems don’t seem like they’re going away anytime soon. (Interestingly, I heard Arceneaux’s replacement, Will Sheffie, keynote the Rustbelt to Artist Belt conference in St. Louis just a week after all this went down. He got a warm welcome from the audience, but avoided addressing the controversy in any real depth.)

• The Saint Paul Chamber Orchestra is upping the ante again on its radical drive to democratize classical music. The latest move is to offer members tickets to every single one of its concerts for just $5 a month. The article is worth reading in full; essentially they’re saying they’re all but giving up on earned revenue as a serious income driver.
• Is there a future for cash mobs to support local arts organizations?
• RIP San Antonio Opera.

RESEARCH CORNER

• The NEA has a new literature review out on audience impact, conducted by WolfBrown as part of a larger project to study audiences at NEA-funded events.
• The Nonprofit Finance Fund has released its 2012 State of the Nonprofit Sector Survey, which reports that nonprofits are still feeling the economic pinch of the recession even though we’ve officially been in recovery for almost three years. (As an aside, I always have this funny cognitive dissonance whenever I read about nonprofits having a hard time because they’re “unable to meet demand.” If only arts organizations had such problems!)
• Now conservatives are making a stink about the American Community Survey (the government’s annual replacement for what used to be the long-form Census) because of the nature of its questions. They want to make it optional to fill out, which of course would make it just another poll and destroy its statistical usefulness.
• Americans for the Arts’s Randy Cohen offers a 2012 update to his popular Top 10 Reasons to Support the Arts post. AFTA also released the latest edition of the National Arts Index this month, and this time, there’s a new website–and a nifty new Local Arts Index–to go with it.
• The Center for Effective Philanthropy finds no major differences between how grantees of color and others experience relationships with their funders.
• The Ford Foundation has made its internal records from the 1950s and ’60s available for review at the Rockefeller Archive Center in upstate NY. This was a fascinating time in Ford’s history during which it was largely responsible for the growth of symphony orchestras and the regional theater movement across the country.

• Mike Boehm has more on the important role California’s soon-to-be-defunct community redevelopment agencies have had in shaping Los Angeles’s cultural development.
• Gene Takagi provides this extremely helpful dispatch from a session on new “hybrid” legal forms such as the Benefit Corporation and L3C.
• Culture360 has published a helpful two–part history and analysis of cultural policy in South Korea.

MUSICAL CHAIRS

• The Hewlett Foundation has a new President – and just like the last one, he’s a former dean of the Stanford Law School.
• Guidestar’s CEO, Bob Ottenhoff, is moving on.
• The Tennessee Arts Commission has announced Anne Pope as its new executive director.
• Welcome Sarah Lovan, new arts program officer for the McKnight Foundation.

BIG THOUGHT

• I’ve been thinking about transparency a lot lately. It’s harder than it looks, but here are two recent examples I find admirable from two organizations that have been committed to transparency from the beginning. First, the open embrace on the part of the Center for Effective Philanthropy of ways it can improve its flagship product, the Grantee Perception Reports. And second, a fairly devastating report from GiveWell on the progress of its #1 charity recommendation from 2010, VillageReach (to which I was one of many donors). The latter seems especially dicey at first, but GiveWell goes out of its way to praise VillageReach’s continued commitment to collecting and reporting data on its activities and adds, “we always prefer discouraging observations to no observations.”
• Lisa Bernholz lauds David Sasaki of the Omidyar Network for committing to blogging about every grant he makes. But Omidyar still lags behind on other transparency standards, as Glasspockets points out.
• You’re going to be reading a lot more about the term “Collective Impact” this year, I predict. Nonprofit consultants FSG, who coined the term last year in an article for the Stanford Social Innovation Review, write about their choice not to trademark the name.
• Michael Eisen, co-founder of the open access publisher PLoS (Public Library of Science), opines on why the academic publishing model hinders scientific progress. (And yes, research on the impact of the arts, lest we forget, counts as science.)
• Check out these fun videos of “culture warriors in their native habitat” discussing Harvard Business School case studies, courtesy of Fractured Atlas Deputy Director Tim Cynova.

BIG MONEY

• The Robert Rauschenberg Foundation looks to become a major new player in visual arts philanthropy.
• NYC arts institutions will receive $100 million from Brooke Astor’s estate.
• It turns out that a portion of ArtPlace’s funding, which comes from a handful of major arts foundations, is restricted geographically to the areas that those foundations serve, leading to a disproportionate focus in some regions vs. others. While this revelation won’t be a shock for those who know the foundations in question – Knight, for example, has a particularly idiosyncratic geographic reach arising from the Knight family’s historical connections to newspapers in specific markets – it’s not going to be much comfort to the applicants who faced higher odds because of it, and reveals the challenges of relying on a patchwork of arts funders to create a truly national agenda.

IN THE FIELD

• Cute advertising conceit for a symphony concert plays on desperate fundraising campaigns.
• Is the Colorado Symphony following through on its supposedly transformative business plan? Inquiring minds want to know.
• The SAG-AFTRA merger is finally complete.
• I always appreciate interviews with artists in which they are candid about their economic circumstances and how they make money (or don’t). Jen Dziura has a nice one with musician Kim Boekbinder in the Grindstone.
• Barry Hessenius has a good interview with Doug Borwick, president of the Association of Arts Administration Educators.

RESEARCH CORNER

• The NEA is out with a new study from arts education researcher James Catterall finding that at-risk youth with “arts-rich” educational experiences outperform their peers on various metrics of success. Almost simultaneously, the National Center for Education Statistics (NCES) released updated numbers from its Fast Response Survey System covering K-12 arts education, a congressionally mandated study. Sunil Iyengar offers a first read of the result; Janet Brown expresses some skepticism at the numbers.
• The Pennsylvania-based Education Policy and Research Center has a report out providing arts education policy recommendations for state leaders.
• Survey fatigue is a real and growing problem for researchers who want to get information directly from customers, audience members or stakeholders.

CLOSURES, OPENINGS, MERGERS, AND PAY CUTS

• Gentrification claims another arts space in Williamsburg.
• The Sacramento Opera and Philharmonic are in active merger talks.
• I would like to see more of this kind of story: after Pittsburgh Symphony musicians agreed to a new contract that forced them to take a 9.7% pay cut, their music director, Manfred Honeck, announced that he would take a 10% cut too. Of course, Manfred can afford to lose a bit of income, as he made $546,700 last year. But still, it’s surprisingly rare that we see even this much of a gesture from the leaders of organizations under financial duress. Meanwhile, the Wichita Symphony players recently accepted a further 20% pay reduction, on top of a voluntary wage cut of nearly 14% over the past two years. No word on any concessions made by the music director there.

MUSICAL CHAIRS

• Paul Brest, who has been president of the William and Flora Hewlett Foundation for a little over a decade, is retiring next year.
• Antony Bugg-Levine will be the new CEO of Nonprofit Finance Fund. Bugg-Levine was formerly Managing Director of the Rockefeller Foundation, and is an expert in impact investing.
• Mario Garcia Durham, the former NEA Director of Artist Communities and Presenting, will be the new President and CEO of the Association of Arts Presenters, replacing Sandra Gibson.

PHILANTHROPY TALK

• The philanthropy/nonprofit blogspace is all abuzz over the fact that GOOD, a for-profit media company focused on social causes, has bought Jumo, the social media and crowdfunding platform for nonprofits started by Facebook co-founder Chris Hughes. The fact that Jumo is itself a nonprofit that was started with the help of $3.5 million in grants from the Ford and Knight Foundations and Omidyar Network seems to be the source of the intrigue.
• Good news for cultural diplomacy enthusiasts: New York’s Robert Sterling Clark Foundation has dramatically increased the resources it is pumping into international cultural exchange programs. A list of grants made so far is available here.
• The Save America’s Treasures grant program, administered through the National Trust for Historic Preservation, is shutting down.

RESEARCH CORNER

• Emerging arts leaders: do the Americans for the Arts Emerging Leader Council and your fellow citizens a solid, and fill out this survey to share tips, tools, and resources that you find useful with your peers.
• Musicians: do the Future of Music Coalition and your fellow citizens a solid, and fill out this survey about the ways in which you make your money (or don’t). I hear there are iPads to be won!
• I think it’s pretty awesome that the American Planning Association would publish a report called “Community Character: How Arts and Culture Strategies Create, Reinforce, and Enhance Sense of Place.”
• The Aspen Institute is studying artist-endowed foundations.
• Ways in which cities are using data and research to improve citywide out-of-school-time (OST) systems.
• Freakonomics contributor Daniel Hamermesh is out with a new book called Beauty Pays: Why Attractive People Are More Successful. I’ve been interested in this topic for at least a decade, ever since I first became aware of psychology and economics studies showing the substantial life benefits reaped by attractive people, and am glad to see it starting to enter the mainstream conversation. Curiously, the data suggests that there is more downside than upside to the attractiveness game for men as compared with women. Hamermesh goes so far as to make a (fairly cogent) argument for fashioning a legally protected class out of the ugly. Fascinating stuff!
• Sage advice for young cultural researchers from Canada’s Lidia Varbanova: “Cultural policy research field is rewarding but not an easy one: it requires a good portion of diplomacy and negotiation skills as it reflect diverse stakeholders because research without policy actions stays only in the libraries without real impact on improving the creative life of cultural professionals and the communities. It also needs patience, as in many cases undertaking practical policy actions as a result of research findings requires time, lobbying and joined advocacy efforts.”

ART AND THE LAW

• The previously-blogged New Jersey plan to require nonprofits in the state to allow program-restricted donations has been thankfully dropped.
• A little-known provision in current copyright law allows artists to apply for reinstatement of the ownership rights to their work after 35 years. That deadline is coming up and record labels, publishers, and others that rely on the assignment of copyright (and milking cash cow artists) for their business model are understandably freaking out about it and trying to challenge the law. The conflict centers on whether (for example) albums made under a recording contract can be considered “work for hire” and thus belong to the record company by default. With the legal ramifications unclear, Representative John Conyers has come out on the side of the artists.
• I never thought I’d be writing these words, but there is going to be an arts policy storyline on a major network television show.

ART AND GOVERNMENT

• Sam Brownback just doesn’t know when to quit. Not satisfied with being the first governor in history to completely defund his own state’s arts commission, now he’s having his chief of staff show up at ribbon-cutting events to mouth off about how wonderful the arts are. Meanwhile, he refuses to reconsider his decision – despite the fact that the state now has a $180 million surplus.
• Jay Dick from Americans for the Arts explains the complex web of societies for elected officials and why AFTA tries to have a presence at their convenings. In my (still developing) observation, this type of advocacy seems to be what AFTA is best at: the behind-the-scenes relationship building that takes place in different corners of the country and among policy insider circles. I sometimes think AFTA doesn’t get enough credit for its work in this area, which to my mind would be very difficult for others in the arts field to replicate. It’s definitely soft power rather than the hard power represented by massive lobbying dollars or enormous mobs with pitchforks, but soft power is better than nothing.

INTERNATIONAL AFFAIRS

• Diane Ragsdale considers the shape of arts cuts and new funding models in the Netherlands and Australia. Meanwhile, severalEuropean impresarios talk about the effect of recent budget cuts on their plans and speculate about the future.
• Sad news: a British cultural council in Afghanistan has been the focus of a terrorist attack. At least twelve people died in the fighting, mostly Afghani police and security guards.

AND SO ON…

• If you’re hankering for some great live jazz streamed direct to your computer, Tara George has your fix.
• I enjoyed this neat idea from Andrew Taylor on “job function trading cards.” He’s definitely right that small organizations risk making suboptimal use of their employees’ unique talents because they are too beholden to job descriptions.
• It sounds a little ridiculous, but I actually think these “gofer” services popping up are pretty brilliant. Besides providing value to buyers and sellers, it could help the long-term unemployed stay productive and earn a little on the side.