[Createquity Reruns] Games and the Arts in the 21st Century: An Introduction

(Jackie Hasa week at Createquity continues with the first entry in Jackie’s two-part series on games and the arts, likewise dating from Jackie’s time as a Createquity Fellow in the spring of 2012. Enjoy! -IDM)

The idea of using games as a new way to engage audiences has gained immense traction in the last 5 years. The museum world in particular has seen a great deal of discussion on this topic, from Nina Simon’s dozens of posts to this year’s Museums and the Web conference; these conversations are a natural outcropping of a much larger discussion about games in our everyday lives. I’ll be writing more about games in a later post, but I hope this one serves as an introduction to why this dialogue is happening now and what is at stake for the arts.

So why is everyone suddenly talking about games? Put simply, the immense growth of the video and social gaming industry is inspiring innovators across many sectors to ask how they might hitch themselves to this rising star. In 2011, video and computer games became the U.K.’s biggest entertainment sales category at 40.2% of the market, beating out music and video. The Entertainment Software Association notes the following staggering statistics about the 2010 U.S. market:

  • Consumers spent $25.1 billion on video games, hardware, and accessories in 2010.
  • 72% of U.S. households play computer or video games.
  • 42% of all game players are women. In fact, women over the age of 18 represent a significantly greater portion of the game-playing population (37%) than boys age 17 or younger (13%).
  • In 2011, 29% of Americans over the age of 50 played video games, an increase from 9% in 1999.

Note: Sales numbers provided here include console and PC sales only; the $25.1 billion sales total for 2010 provided by the Entertainment Software Association includes a broader range of video/online games.

The implications of this data extend far beyond the screens that limit video games to the virtual world—their newfound cultural ubiquity means that huge numbers of the population can now more easily recognize tropes and imagery from video games in real-world settings. The tools of the online gaming world (getting points for accomplishments, ascending in level, unlocking achievements, and participating virtual social circles) have become powerful ways to engage an audience for any organization, whether an arts nonprofit or a private company. Foursquare is just one well-known example of this phenomenon, as participants receive points and other rewards for visiting a particular venue in real life.

Academics and game designers have generated a number of theories about the power of these kinds of games. Jane McGonigal, probably the most visible representative of this cohort, has made a name for herself arguing that games can literally make the world a better, happier place by harnessing their power in the service of solving real-world problems, from bodily injury to oil shortages. In his review of her recent book, Reality Is Broken, Ian Bogost argues that games allow for engagement with real-world problems by providing complex modes of inquiry rather than clear solutions. Finally, folks like Jesse Schell imagine a terrifying future in which nearly every activity we undertake is part of a game in which we accumulate points via specially-designed sensors. These are powerful ideas, and they are continuing to gain traction as technology allows for the omnipresence of games in our lives through smartphones, ultralight computers, tablets, wireless internet, and even Schell’s toothbrush sensors.

While games can clearly serve numerous social, educational, or marketing goals, the debate over whether they might form a legitimate arts genre rages on. Roger Ebert famously thinks not, and undoubtedly not all games should be considered art, since most are created primarily as products to be sold to a mass audience. However, more and more institutions are placing at least certain kinds of games on the art pedestal. In just the past few years, Georgia Tech has co-hosted an Art History of Games conference, the Computerspielemuseum (Museum of Computer Games) opened in Berlin, and the Grand Palais in Paris hosted the Game Story exhibition; in May, MoMA will host a program titled “The Game as an Art Form.” This proliferation is rooted in games’ fundamental resemblance to conceptual art in which the audience, or player, is integral to the work. One can cite a range of work in this vein, including Yoko Ono’s instructional Grapefruit, Allan Kaprow’s performances, a great deal of the art produced for Burning Man, and Punchdrunk’s ongoing Sleep No More production. Other pieces like Manchester street artist Filthy Luker’s playable Space Invaders installation pay more direct homage to popular games.

Filthy Luker’s Space Invaders installation. Credit: Duncan Hull.

Arts organizations stand to gain a great deal from refining their relationship to video and real-world games. As games industry growth outpaces already-recognized art forms like music and video, institutions should certainly incorporate games into a larger marketing or audience engagement strategy to stay relevant. But beyond that, arts organizations can positively affect the trajectory of games in culture through serious investment in programs like commissions, residencies, or cross-sector collaborations. When music and video became dominant entertainment forms, they were embraced and challenged by artists willing to push the boundaries of popular practice, and arts institutions can and should encourage the same evolution in games.

Leave a comment

[Createquity Reruns] Parklets: Coming Soon to a City Near You

(Last week may have been tax policy week at Createquity, but it doubled as John Carnwath week, as both of the articles he’s written for the site were featured. So this week we’re following suit with three articles by Jackie Hasa, who likewise joined the Createquity editorial team in June. This one was Jackie’s first article for the Createquity Fellowship back in March 2012 – an examination of the phenomenon of parklets in San Francisco and their implications for the arts. – IDM)

The original PARK(ing) Day parklet. Credit: Rebar

In the last year, parklets have taken San Francisco by storm. At the start of 2011, San Francisco had four of these sidewalk-adjacent, itty-bitty public spaces created by repurposing parking spots. Now there are more than 20, with dozens of others in various stages of review. Other cities, including New York, Long Beach, Los Angeles, Philadelphia, and Vancouver have taken notice of this phenomenon as a cheap, flexible way to enliven their streets with small seating areas and green spaces. Their importance for even non-design-oriented arts organizations is twofold: 1) their evolution from guerilla art action to policy success can serve as a blueprint in other arenas; and 2) parklet programs in development present opportunities for local arts organizations to shape these urban interventions to serve their communities.

While urban planners laud parklets as revolutionary uses of public space, their short evolution from the original PARK(ing) Day in 2005 to a celebrated piece of city policy in 2011 is perhaps just as astonishing. PARK(ing) Day began in San Francisco, when the design collective Rebar paid a parking meter, set up some sod, a tree, and a bench, and used the space as a park until the two hours on the meter ran out. Then, they rolled up the sod, and the space went back to its original car-oriented purpose. By 2011, it had turned into a decentralized worldwide event, in which thousands of individuals and groups created 975 informal parks for a day across six continents. In San Francisco, the Pavement to Parks program now officially supports the development of more permanent parklets with a permitting process and guidelines. Among other things, the guidelines stipulate that all parklets are sponsored and managed by a private partner, all seating must remain available for public use, and at least some of the seating must be permanent.

The first official parklet in San Francisco, at Mojo Bicycle Cafe on Divisadero. Credit: Jeremy A. Shaw

Although there has been a great deal of support for parklets, thus far, most in San Francisco don’t quite fulfill their promise. Arts organizations have a fantastic opportunity to improve upon their current form. The requirement for a local business steward means that parklets often end up serving as little more than well-designed outdoor seating areas for cafes or restaurants, albeit technically open to the public. In a review of 19 parklets, John King, the San Francisco Chronicle’s architecture critic, found that fifteen are managed by a cafe or restaurant, two are managed by other businesses, and one is adjacent to a private home. The 19th? An example to us all: the art gallery Fabric 8 is managing its own parklet, which will host art installations that rotate each year. The current installation, by Erik Otto, is available for purchase and can be moved to another location at the close of its exhibition at Fabric 8.

Fabric 8′s first parklet, by Eric Otto. Credit: Fabric 8

Arts organizations in San Francisco and elsewhere should follow Fabric 8’s example and push for parklets that support public programs rather than supplement local businesses. A California Planning and Development Report already contains the kernel of a great idea–the creation of a sponsorship program to cover the $5,000-$10,000 in parklet start-up costs for worthy organizations that can’t afford it. Other possibilities include the coordination of parklet design to enhance (or generate!) arts districts, the development of a single permit to utilize all of a neighborhood’s parklets for a mobile arts event, the formation of new arts-business partnerships for parklet stewardship, or the creation of parklets specifically for performance. Parklets represent a version of place-based community development that even small arts organizations can deploy to achieve great impact.

Leave a comment

[Createquity Reruns] The Bottom Line on Film Tax Credits

(Tax policy week at Createquity concludes with our most popular article thus far in 2014, John Carnwath’s investigation of the value of tax credits for film and television. Are they worth the hassle? Maybe, but probably not to the extent that lawmakers in New Jersey, California, and Austin think. -IDM)

Photo by vancouverfilmschool. Some rights reserved.

Photo by vancouverfilmschool. Some rights reserved.

About a year ago the New York Times ran a series of articles on corporate tax breaks, complete with a web-accessible database of state tax incentives for businesses. All in all, the Times discovered 1,874 state and local incentive programs that give out a combined $80.4 billion to corporations each year. To put those figures in perspective, the tax breaks doled out by Oklahoma and West Virginia are worth about one third of those states’ entire budgets. Manufacturing is the most highly subsidized industry, receiving about $25.5 billion in tax breaks annually, followed by agriculture and oil, gas, and mining. Fourth on the list? Surprisingly, it’s the motion picture industry, which nets about $1.5 billion in state and local tax credits per year.

What’s behind this $1.5 billion tax rebate for filmmakers? While industries such as agriculture have been subsidized for decades, state and local tax credits for film productions are relatively new. Louisiana was the first state to introduce such an incentive in 1991, and other states were slow to follow suit. Only four states offered incentives to movie producers in 2002, but once the idea caught on it spread like wildfire, and by 2010 forty-four states offered some form of incentive to filmmakers. The specifics vary from state to state, but typically the financial incentives (for which TV productions, industrial videos, commercials and sometimes even video games are eligible) consist of some combination of tax credits, cash rebates, employment rebates, sales tax and lodging exemptions, and fee-free use of shooting locations. In order to qualify, productions must generally satisfy certain conditions, such as spending some percentage of their total budget locally, shooting a certain percentage of the footage in-state, employing a certain quota of local residents, or exceeding a minimum amount of in-state spending. In addition, some states require that the action of the films take place in a local setting or even demand that the film depict their state in a positive light in order to qualify for tax credits.

To be clear, the film industry isn’t supported out of any particular concern for cinematic art, nor are politicians incentivizing the production of films because they think we’d be better off as a society if we had more movies and TV shows to watch. As is the case with many other corporate tax breaks, the main reason for offering tax credits for filmmakers is simply jobs, jobs, jobs. State officials don’t particularly care if a company is making movies, auto parts, or toothpaste—if it has the potential to create a lot of jobs for local residents, officials want those jobs in their legislative district rather than someone else’s. They are willing to dangle tax breaks as bait on the assumption that the jobs created by the firm will bring more money into the local economy than the government will lose by providing the tax break.

So how effective are the tax incentives for film and TV productions in generating jobs and/or revenue? That depends on whom you ask. Or who funds the research you’re looking at.

The ROI of film tax credits

Reports funded by the Motion Picture Association of America (MPAA), the main lobbying arm for the movie industry, consistently show a positive return on investment for state treasuries. For example, a recent study of the New York State Film Production Tax Credit commissioned by MPAA found that “for every $1.00 of credit distributed, the State and City received a combined $2.23 in taxes.” In Florida, a research firm that was hired by MPAA found that $118.7 million in tax credits yielded $140.44 million in 2011/12, for a more modest return of $1.18 to the dollar.

By contrast, a recent report on Louisiana’s Motion Picture Tax Credit by the State Auditor found that the state supported the film industry with $196.8 million in tax credits and only received $27 million in additional taxes in return (about $0.14 for every dollar spent), and the Massachusetts Department of Revenue found its return to be only marginally higher at $0.16 per dollar spent. A 2011 article by the Tax Foundation lists several other studies by state agencies and a few that were funded by MPAA exhibiting the same discrepancies.

How do these studies arrive at such vastly different numbers? To get some insight into this question, we can take a look at two conflicting reports on Massachusetts’ Film Industry Tax Incentives that were published this year: one by the Massachusetts Department of Revenue (mentioned above) and the other by HR&A Advisors for MPAA.

MA Department of Revenue


Tax credits awarded

$44 million

$37.9 million

Taxes generated

$6.9 million

Not reported

Jobs created in MA



Economic Impact

$118 million

$375.3 million

* This is not explicitly reported as the number of jobs created, but is implied by the statement, “Massachusetts motion picture production employment increased 46.1 percent from 1,630 jobs in 2006 [the year the tax incentives were introduced] to 2,380 jobs in 2011.”


As far as I can tell, the disparity, most notably in the estimated economic impact, results primarily from the following differences in the models and their underlying assumptions:

  • Opportunity costs: Since the state is required to maintain a balanced budget, the Department of Revenue assumes that any incentives that are provided to the film industry have to be paid for by saving money somewhere else in the budget. So instead of just calculating the positive effect that the tax credits have on creating jobs, their analysis factors in the number of jobs that will be lost in other areas of the state’s budget due to cuts. Of course, these cuts have negative ripple effects throughout the economy just as the newly created jobs have a positive impact. By contrast, the MPAA report only looks at the positive effects of the new jobs that are created.
  • Wages paid to non-residents: Both reports acknowledge that some of the jobs that are created by film and television productions in Massachusetts will be held by people whose primary residence is in another state; however, the manner in which the studies correct for that varies considerably. The MPAA report merely exempts “individual employee salaries over $1 million, as it is assumed the majority of these employees are non-residents, so multiplier effects associated with this spending are not realized within the Commonwealth.” The Department of Revenue similarly excludes all payments to recipients earning more than $1 million per production, but in addition it excludes 95% of all other wages paid to non-residents. The rationale is that most of the lodging, food, and incidental expenses for non-resident employees are paid for by the production company, so that only a small portion (estimated at 5%) of the non-resident workers’ paychecks gets spent in-state.
  • “New” spending vs. total spending: The Massachusetts Department of Revenue is careful to exclude the production of TV shows and commercials that would have been produced in-state even in the absence of the incentives. This wasn’t of great consequence in 2011 (the latest year included in the study), since $174.6 million of the $176 million spent on TV and movie productions was deemed to be “new” (i.e., induced by the tax credit). However, in 2010 41% of the total film production spending ($29.5 million out of $71.6 million) would have been expected to take place in Massachusetts even if no tax credit had existed. (The large difference between 2010 and 2011 was attributed to the discontinuation of some long-running TV programs that were produced locally). The MPAA study doesn’t factor pre-existing film production into the equation, preferring to attribute all of the current film production activity to the incentive.

Despite the considerable differences between the studies’ findings and their disagreement about whether the incentives produce a net benefit for the state treasury, everyone agrees that the tax credits have a positive impact on the economy. For example, the 2013 audit in Louisiana found that every dollar of film tax credits resulted in $5.40 in economic output. However, that’s not really saying much, since almost any form of government spending is likely to have a positive impact on the economy. The question is how the impact of the tax credits compares to other things the state could have done with the money.

What about those jobs?

As mentioned above, in most cases the objective of these tax incentives isn’t necessarily revenue generation but job creation. Are film tax credits efficient means of creating jobs? One way to consider that question is how many tax dollars are being spent for each job that is created. According to the Department of Revenue, Massachusetts’s taxpayers had to fork out $128,575 in tax credits for each job that was created in the film industry in 2011. Seeing as the median wage for jobs created by the film industry in Massachusetts was $70,657, that doesn’t seem like a very good deal. The state could have employed almost twice as many people at the same income level if it had simply hired those people directly, instead of subsidizing motion picture companies. The picture looks somewhat better if one uses MPAA’s numbers, but it’s still not great. Assuming (generously) that all of the jobs that were created by TV and film productions between 2006 and 2011 can be attributed to the tax incentives, 750 jobs were created in 2011 at a cost of $37.9 million. That comes out to $50,533 in state spending for each new job. So even using MPAA’s more favorable numbers, it seems that the state paid more than two thirds of the wages for the film industry’s new employees.

Film-induced tourism

One factor we haven’t considered yet is the effect that film and television may have on tourism and the public perception of the locations where they are produced. There are certainly a number of cases in which blockbuster films have led to significant increases in tourism, as The Lord of the Rings trilogy did for New Zealand. And once local residents get past the street closures, traffic delays, and general nuisance that come with a major movie shoot, I’m sure many would agree that there’s some excitement in spotting Hollywood celebrities at local restaurants. There’s also some satisfaction to be gained from recognizing familiar landmarks on the big screen when you see movies that were shot in your hometown. In that sense, local film productions may promote a sense of pride in and attachment to a location, so that tax credits could be justified on the basis that they improve the quality of life for local residents. If nothing else, they give them something to talk about.

Both the tourism and public opinion arguments are valid reasons to support tax incentives for movies. It therefore seems appropriate that MPAA includes film-induced tourism in its recent reports on state tax incentives (in Florida and Massachusetts); however, the measurement of this effect remains problematic. Having consulted twelve representatives of Florida’s tourist industry, MPAA figures (conservatively, it claims) that 5% of all tourism in the state can be considered film-induced tourism and can therefore be added to the economic impact of the film tax incentives. The MPAA-commissioned study of Massachusetts’s film tax incentives uses a different methodology that was employed to assess the value of New Zealand’s exposure in the Lord of the Rings and Stockholm’s exposure in the Millennium trilogy. This approach equates each recognizable shot of the location where the film is set with the publicity that is achieved by a 30-second paid advertisement for the destination on primetime television. The cost of purchasing enough airtime on television to reach an equivalent number of viewers is then assumed to be the value of the advertising that the film provides for the location.

However, if tourism is the objective, the tax credits shouldn’t be granted to all movies indiscriminately. Only those that showcase the location prominently and identifiably should be eligible. I wouldn’t doubt that ABC’s Nashville is having a positive effect on the psyche of that city’s residents as well as on tourism and the local economy. As the assistant commissioner of communications and creative services for the Tennessee Department of Economic and Community Development noted, “You’re not only getting the 20-plus episodes per season”: every advertisement and preview for the TV show is essentially advertising Nashville. By contrast, it’s hard to imagine how shooting Homeland in Charlotte, NC, would do much to increase public awareness of North Carolina’s attractions, given that the show is supposed to be taking place in Washington, DC. Incentive programs that require films to be set in-state and/or depict the location in a positive light make a lot of sense from this perspective (though the latter condition may seem dangerously close to censorship).

So where does that leave us?

In December, the Bureau of Economic Analysis, which among other things is responsible for determining our nation’s Gross Domestic Product, for the first time released a calculation of the economic value of the arts and cultural production in the United States. According to the BEA, arts and culture contributed $504 billion to our GDP in 2011. To put that in perspective, that’s almost twice the $289.9 billion generated by mining (which includes all oil and gas extraction). The motion picture industry alone added $83.2 billion to the US economy, which, believe it or not, is more than the total value added by automobile manufacturing. The motion picture industry has long been touting its economic significance to argue for more favorable tax treatment, and these latest numbers will only bolster its case.

Yet critics of film tax credits claim that these policies do nothing to stimulate the economy and merely pit states against each other in a race to the bottom. Even some people within the movie industry acknowledge that if the tax credits are the only thing that a certain location has going for it, business will likely move somewhere else as soon as the financial incentives are rolled back or some other state offers even bigger tax breaks. Once the states have foregone all tax revenue from movie producers in the rush to compete with other locations, the playing field will once again be even and the industry will settle where it was to begin with—the only difference being that the production companies no longer pay taxes and state budgets are even tighter than before.

I haven’t found any data to indicate whether or not the incentives are increasing due to competition between states, but from a theoretical perspective the race-to-the-bottom argument is compelling. Even if one believes the MPAA studies that show a positive return on the states’ investment, if the states keep raising their tax incentives to compete with their neighbors, the public benefits of attracting motion picture productions will eventually approach zero.

In the material I’ve reviewed for this article, there is little to suggest that the current incentives offered by state and local governments are optimal in any sense. I have yet to come across calculations that show that a certain level of tax incentives creates the greatest number of jobs per tax dollar forgone. So why do some states offer 15% tax rebates while others offer 20%? Are the expected returns really higher in some states than others, justifying the additional investment? Or are the higher tax incentives necessary in states where the conditions for filmmaking are otherwise so poor that no producers would go there if the rates were any lower? Since I haven’t been able to find a convincing rationale for the levels of the tax credits, I assume the levels are indeed set according to inter-state competition, which is consistent with the race to the bottom scenario. If that’s the case, I’d say it’s a poor justification for public spending. (This situation isn’t unique to the motion picture industry, by the way. The same doubts about inter-state competition and the race to the bottom hold true for all sorts of corporate tax breaks that try to bribe corporations into setting up shopor keeping current plants open – in particular locations.)

So how might we improve the current system of incentivizing film and TV productions? Even if the economic argument offered up by MPAA doesn’t hold—and, personally, I’m inclined to believe the governments’ internal audits that show a net loss for the states—I don’t think one must abandon the idea of incentivizing motion pictures entirely. The system might just need to be improved to target those productions that are likely to generate the greatest public benefits.

The fact is, the current tax credits already target specific types of motion picture productions. Minimum requirements for the production budget and/or the amount of money that is spent in-state are presumably designed to ensure that the tax breaks go to big-budget productions that will employ a lot of people. The large corporations behind those productions are the ones most likely to respond to incentives, moving their operations to whichever state offers the lowest costs, whereas a small film production company in Massachusetts is likely to work in Massachusetts whether or not the state offers any incentive. However, for that same reason one might argue that the large corporations are precisely the wrong place to invest public money: they operate nationally (or internationally) and will fly in people from around the world to work on the production, so relatively few of the jobs created will go to local residents. Furthermore, as soon as another state offers bigger tax rebates, they’ll pack up shop and move there. Any spike in economic activity from a big-budget production coming to town is therefore likely to be short-lived. Wouldn’t it be better to give the tax breaks to smaller firms whose production budgets are too modest to fly in talent from out of state? Those productions might not hire a whole lot of people, but at least the paychecks will be going to local residents, and if the production does well and the company grows, that growth will happen in-state.

The one argument that does work in favor of giving tax breaks to big budget productions is their ability to reach a wide audience and potentially increase tourism or improve public opinion of a certain location. In order to fully endorse this approach, I’d need to see more credible research on the economic value of the exposure that shooting locations receive, preferably coupled with a greater capacity to predict which films are going to have a significant impact in that regard. As tempting as it is to reduce the conversation about film tax credits to a simple thumbs-up or thumbs-down, it would be smarter to consider how to determine which productions to incentivize with tax credits and where such tax expenditures would be wasted. That’s speaking from the perspective of job creation, of course. If the objective were to improve the quality of cinematic art, an entirely different set of selection criteria would be necessary.

Leave a comment

[Createquity Reruns] The Deduction for Charitable Contributions: The Sacred Cow of the Tax Code?

(Tax policy week continues at Createquity with this doozy of an analysis from editorial-board-member-to-be John Carnwath from April 2013. Believe it or not, thanks to some stellar performance in Google search rankings, an article about tax policy is now Createquity’s most-read blog post of all time! And for good reason, as John’s article expands beyond the usual rhetoric and surfaces some creative solutions in the debate around the charitable tax deduction that just might satisfy everyone. -IDM)


photo by Martha Soukup

In his most recent budget proposal, President Obama is seeking to impose a cap on itemized deductions in the personal income tax return – which includes the deduction for charitable contributions. This provision, part of the administration’s strategy to raise revenue to pay for government spending, has been a part of every White House budget proposal since 2009, and every year arts advocacy organizations join the rest of the nonprofit sector in opposing the changes. So far, the cap has been successfully warded off, but there’s growing concern that if Republicans and Democrats ever agree on sweeping tax reforms, the charitable deduction will be on the chopping block. The fear that limiting the tax deduction will lead to reduced donations to charitable organizations is particularly great this year due to the tax increases that were passed at the end of 2012, prompting the Charitable Giving Coalition to step up its resistance with a new website: protectgiving.org.

While it’s become a popular strategy on Capitol Hill to complain about the lack of progress while refusing to budge from one’s own policy positions, a case can be made that the nonprofit sector’s lobbying on behalf of the charitable deduction has neither improved the financial stability of the sector nor created greater legislative security. At best, it has limited the declines in individual giving in recent years. So rather than simply digging our heels as we head into the next round of budget debates, let’s take a moment to explore a broader range of policy options and see which might make the most sense for the arts.

Before we get to that, though, here’s a refresher on the mechanics of the charitable tax deduction for anyone who needs it.


What is the charitable deduction and how does it work?

The tax deduction for charitable donations was established in 1917, just four years after the federal income tax was introduced. While there have been some changes over the years, in its basic form this provision allows taxpayers to deduct donations to nonprofits and charities from their taxable income. So if a taxpayer earns $50,000 and gives $2,000 to charity, she only has to pay taxes on $48,000. The rationale behind this provision was initially that the taxpayer who gives away $2,000 doesn’t have that money available to spend on herself, so it shouldn’t be counted as part of her income. Nowadays, the deduction is more commonly thought of as an incentive dangled before taxpayers to coax them into donating more money to charity. By allowing taxpayers to deduct charitable donations from their taxable income, the government essentially agrees to pay for a portion of the donation.

Think about it this way: If you earn $1,000 and you’re taxed at a rate of 30%, you have to pay $300 to the IRS and you end up with $700 in your pocket. But if you donate $100 to charity, your taxable income is reduced to $900. Your tax bill then comes to $270 ($900 x 30%). In return for giving $100 dollars to charity the government reduces your taxes by $30, so in the grand scheme of things that $100 check that you write to your favorite opera company really only sets you back $70.


Who benefits from the charitable deduction?

While this all sounds great in principle, there’s a big catch: not all taxpayers benefit from the charitable deduction. Initially the income tax only applied to a rather small number of wealthy Americans, but during World War II it was expanded to affect roughly 75% of the population. Instead of having all of these tax filers list their deductions individually—$42 for prescription medicine here, a $100 donation to a museum there—the IRS introduced the “standard deduction” in 1944. The standard deduction lets all filers lower their taxable income by a fixed amount. For the 2012 tax year that amount is $5,950 for single taxpayers and $11,900 for couples. That means that you only have to keep track of your deductions and itemize them on your income tax return if they exceed $5,950 (or $11,900 if you’re married). That saves a lot of taxpayers (not to mention the IRS) a huge headache, but it also means that the 70% of filers who take the standard deduction don’t get to write off their charitable donations. (One might argue that the non-itemizers benefit from the charitable deduction in a roundabout way, since a typical deduction for charitable donations was factored in when the standard deduction was calculated back in 1944, but the fact remains that the current deduction for charitable contributions and any changes to it are only relevant to about 1/3 of American tax filers.)

For those who do itemize deductions, the amount of the government’s subsidy towards charitable donations depends on the filer’s marginal income tax rate. If you’re in the 35% bracket and you donate $100 to a good cause, the government gives you $35, but if you’re in the 10% bracket you only get $10 back from Uncle Sam. Economists say that the “price of giving” is lower for the individual in the 35% bracket than for the one in the 10% bracket (e.g. note 1 here). Giving $100 to charity “costs” the former (presumably richer) person $65 and the latter $90. While this seems sort of unfair, it’s the result of having a progressive income tax system in which those who earn a lot pay a larger percentage of their incomes into the public purse.

This means wealthy taxpayers not only have more money in their bank accounts to give away, but when they donate to charity the government covers a larger portion of their donations. It is therefore no surprise that the rich are responsible for a large share of charitable giving. Although only 3% of tax filers have annual incomes over $200,000, those households contribute 36% of the money that individuals give to charity every year—a total of $73 billion in 2008. However, the federal government foots the bill for about a third of those donations through the deduction for charitable contributions (assuming that most of the individuals with incomes over $200,000 are in tax brackets with marginal rates over 30%).

One might say, “well it’s all for a good cause, so it doesn’t really matter if the government is paying for a portion of the donations,” but it turns out that taxpayers with high incomes choose to give their money to different causes than those who are less well-off, and the charitable deduction allows them to divert large amounts of government funds to their favorite organizations. The wealthy support educational institutions and the arts to a much greater extent than poor people, who tend to focus their giving on basic needs and religious organizations. The extent to which the arts depend on donors with high incomes for their contributions is quite striking. In 2005, 94% of the funds that arts organizations received through individual contributions came from households with annual incomes over $200,000.

Of course, the donors are not the only ones who benefit from the tax deduction. All of the people who receive services from nonprofits and charities may be considered indirect beneficiaries of this provision in the tax code. However, to determine whether the charitable deduction is the best way for the government to support the work of nonprofits we must take a closer look at the incentives that are created and how people respond to them.


Do donors respond to tax incentives?

The deduction for charitable contributions affects taxpayers in two different ways. On the one hand, we have the “price effect.” As noted above, higher marginal tax rates reduce the price of giving, creating a bigger incentive to contribute to charities. However, high marginal tax rates also mean that people have less money left in their pockets after paying their taxes. In general, if people’s incomes are reduced, one would expect them to become less generous donors. After paying for rent, food, and utilities, they have less money left over for nonessentials like vacations and charitable donations. This is called the “income effect.” Note that the income and price effects work in opposite directions. Higher marginal tax rates incentivize donations through the price effect, but they simultaneously create a disincentive through the income effect.

Several economists have examined donors’ responsiveness to tax incentives over the past few decades, but the results remain inconclusive. Most studies find that donors respond to tax incentives, but the historical record shows that the level of charitable contributions remains relatively constant over time when measured as a proportion of GDP regardless of the available tax incentives. Some studies suggest that higher-earning taxpayers are more responsive to the incentive than those who are less well-off and that there are differences between types of charities (religious, social, educational, etc.) that receive donations. Many policy analyses (CRS, CBO, TPC) therefore calculate the upper and lower limits of a range into which the effects of proposed policy changes are expected to fall rather than a specific estimate.


Considering policy options: goodbye deduction?

To establish the worst-case scenario as a baseline, one might ask what would happen if the charitable deduction were eliminated completely. Independent Sector, an advocacy organization for nonprofits and charities, recently put out a list of FAQs according to which “with no deduction for charitable gifts, itemized charitable giving would drop by between 25 percent and 36 percent total.” This assertion is rather misleading. The study from which Independent Sector gets these numbers states that a taxpayer in the 30% income tax bracket might reduce his contributions by 25-36% if the deduction were eliminated. Since the incentive to donate depends on the filer’s marginal tax rate and 98% of households face rates under 30%, the reduction in the total amount of individual contributions is likely to be much smaller than Independent Sector suggests.

The truth is, we have no idea what would happen if the tax deduction were eliminated. Not only have studies of the price and income effects been inconclusive, but they are all based on observations of how donors have reacted to incremental changes in tax rates and deductibility in the past. These estimates may be useful in predicting the effect of small changes within the range of what’s been observed in the past, but there’s no reason to be believe that the response would be the same once the government’s incentive approaches zero. In fact, economic theory would predict that it’s not the same.

For example, if the deduction were eliminated completely, one might expect some donors to dig deeper into their pockets to keep their favorite charities afloat. However, some wealthy Republicans might cease all charitable donations to protest the fact that they’re having to pay more taxes, secretly hoping to blame the financial hardships of the charitable sector on the Democrats in the next elections. These types of reactions are difficult to predict. One thing is certain: if the indirect subsidy that the government provides through the charitable deduction were eliminated in order to reduce the deficit, individual donors would have to dig deeper into their pockets to sustain nonprofits at their current level of activity. And if the entire nonprofit sector were in severe financial distress, one can easily imagine that some donors would reallocate their gifts towards hospitals and basic social services, compounding the impact on the arts.


Capping the deduction

The good news is that no one has proposed eliminating the deduction altogether. Obama’s 28% cap on deductions, on the other hand, remains a very real possibility.

Obama suggests that the government could increase its revenue by capping deductions at 28% of the donor’s AGI. As mentioned above, the size of the tax incentive is generally determined by the marginal tax rate that taxpayers incur, but Obama’s proposal sets 28% as the maximum anyone can claim. For the vast majority of households, this would be of no consequence. If you’re in the 10%, 15%, or 28% tax brackets, you still get your deduction as normal. But the 2% of filers who itemize their deductions and face marginal tax rates over 28% would no longer be able to reduce the tax on their donations to zero. People in the 30% bracket, for example, would still have to pay a 2% tax on their charitable gifts. They owe 30% according to their tax bracket and they only get 28% back on the donated amount (due to the cap), so the IRS gets to keep the 2% difference.

How might this cap affect contributions to charitable causes? The short answer is that it will most likely result in a minor, but noticeable reduction in contributions. Here’s what people are saying:

  • The Center on Philanthropy at Indiana University estimates that the cap will lead to an $820 million (0.4%) reduction in charitable giving in the first year of implementation, increasing to $1.31 billion (0.7%) in the second year.
  • In 2010 the Congressional Research Service put the decline in charitable giving in the 0.16 – 1.28% range.
  • In a back-of-an-envelope calculation for the Washington Post, Harvard economist Martin Feldstein estimates that the 28% cap could reduce charitable giving from individuals by $7 billion, which amounts to a 3% decline (relative to the $230 billion in charitable contributions from individuals reported in Giving USA 2009).
  • Len Burman of the Tax Policy Center and the Center on Budget and Policy Priorities came up with similar figures in 2009.

Taking all of this together, it seems we’re talking about a 0.5% to 3% decline in gifts from individuals.

The impact on arts nonprofits is likely to be a little bit higher than that, since the cap will primarily affect the wealthy taxpayers who contribute most to the arts. The 2010 study by the Congressional Research Service includes an analysis of how the 28% cap would affect different segments of the nonprofit sector. It estimates the reduction in individual giving to the arts to be around 2.4% (compared to 0.16-1.28% overall).

The figures above were calculated based on the tax rates that applied between 2003 and 2012, but as we know, the tax rate for the highest income bracket was increased from 35% to 39.6% at the beginning of this year. How does that change things? If charitable contributions remain fully deductable, we would expect the higher marginal tax rates to increase donations due to the price effect. However, if Obama’s proposal to cap total deductions goes through, the reverse is to be expected—the higher tax rates actually exacerbate the decline in charitable giving caused by the cap. That’s because the higher tax rates reduce the taxpayers’ disposable income, bringing the income effect into play, while the cap on deductions holds the price of giving constant.

The Congressional Research Service estimates that the combined effect of the 28% cap on deductions and the higher marginal rates that Obama sought to impose on taxpayers earning more than $200,000 would reduce giving by 0.28% to 2.27%. That’s almost double the decline that they estimated for the cap on deductions alone (see above). The Center on Philanthropy arrives at similar figures when including Obama’s proposed tax hikes. Those projections still fall within the 0.5% to 3% range mentioned above. If we take the worst-case scenarios for the 28% cap and the largest estimates for the impact of the of the higher tax rates, we might be looking at a 5 or 6% decline in charitable giving.

So it looks like we don’t need to fear that individual contributions will drop by a quarter if the 28% cap were introduced, with or without increases in the top marginal tax rates. Nonetheless, a 5-6% decline is nothing to take lightly, and for organizations that are already reeling from the recent recession even a modest reduction in individual contributions could be the final straw. Moreover, the estimates apply to total charitable donations nationwide, but individual organizations could be unlucky and find that several of their major benefactors scale back their contributions more drastically than the national average, leaving gaping holes in their budgets.


Other options: expanding to non-itemizers and adding “floors”

Faced with this uncertainty, the response from arts advocacy organizations has been to dig in their heels and demand that the deduction for charitable contributions remain intact. However, as Michael Rushton notes, there’s little reason to believe that there’s anything magical about our current tax code; in fact, the charitable deduction has been criticized in the past for several reasons (notably for being inefficient, regressive, and having an unclear theoretical justification). So instead of clinging to the status quo as our only hope for survival, we might ask: what changes to the current system would lead to the best outcomes for arts organizations? How might we incentivize charitable donations while supporting the government’s goal of reducing the federal deficit?

In 2011 the Congressional Budget Office came up with 11 different policy scenarios and estimated their likely impact on tax revenue and charitable giving. These included:

  • allowing all taxpayers to write off charitable gifts on their tax returns, rather than just those who itemize deductions
  • creating a minimum donation (either a fixed dollar amount or a percentage of the donor’s AGI) which would have to be exceeded to qualify for the deduction
  • converting the deduction into a tax credit (which would give all taxpayers the same 15 or 25% tax break on charitable contributions instead of linking it to the donor’s marginal tax rate)

This study found that by extending the deduction to all filers and simultaneously establishing $500 ($1,000 for couples) as the minimum donation required to qualify for the deduction the government would be able to increase revenues by $2.5 billion annually, while boosting contributions to charitable causes by $800 million. Or even better, by replacing the deduction with a 25% tax credit for all taxpayers, the government would save almost the same amount, while driving up donations by 1.5%.

Since the government’s objective right now is to reduce the deficit, presumably without harming the nonprofit sector unnecessarily, Eugene Steuerle of the Tax Policy Center has advocated for expanding the tax deduction to all filers, with a minimum contribution of 1.7% of the donor’s AGI required to qualify. This would net the government between $10.4 billion and $11 billion per year without reducing charitable donations by a dime. The argument for establishing a minimum contribution to qualify (often referred to as a “floor”) is that people are likely to give a small amount of money to charity regardless of whether they receive a tax break or not. It’s therefore not necessary for the government to forgo any revenue for that portion of their contributions. Further, at a certain point the administrative costs of tracking small donations—acknowledging their receipt, submitting documentation to the IRS, checking for fraud—is not worthwhile. For those who object that a $1,000 donation is a far bigger sacrifice for a couple that only earns $20,000 a year than for a millionaire, a floor that is linked to the taxpayer’s AGI might pose an attractive alternative. With a 2% floor, someone earning $20,000 could claim the deduction by making a $400 donation, while someone earning $500,000 would have to donate $10,000 to qualify.


Beyond the bottom line

Reforming the charitable tax deduction might offer other benefits as well. For example, it could provide an opportunity to change the composition of our donor lists. By giving those in lower income categories greater incentives to support our work and allowing them to leverage some of the indirect subsidy that the government provides through its tax breaks, arts organizations might be able to diversify the ranks of their donors, so as to be less dependent on a small wealthy elite. Based on the CBO’s estimates, by replacing the tax deduction with a 25% credit that is subject to a low floor (say 1% of AGI), it should be possible to maintain charitable donations at their current levels or even increase them slightly while saving the government several billions of dollars annually and allowing donors from lower income categories to acquire a bigger stake in nonprofit arts organizations. A more diverse pool of donors, both in terms of their economic status and their tastes, would reduce the financial risk of artistic experimentation and could allow companies to diversify their programming in ways that their current (predominantly wealthy) donors might not support.

All in all, reforming the deduction on charitable contributions isn’t necessarily a bad thing for the arts. There are ways of changing the tax code that could actually increase revenues and diversify the sources of income for arts organizations, even while helping to reduce the federal deficit. Since any change creates uncertainty and will likely produce losers as well as winners, I can understand arts administrators and advocates who would rather stick with an imperfect status quo than commit their careers and their organizations to an uncertain future. However, I believe that participating in the discussion and shaping the outcomes to fit our sector’s interests will ultimately prove more productive than trying to block change from the start.

Leave a comment

[Createquity Reruns] Federal arts funding: a trace ingredient in the sausage factory of government spending

(Following a brief hiatus, our summer rerun programming returns this week with an homage to that wonkiest of topics – tax policy! In this post from June 2011, Createquity Fellow Aaron Andersen breaks down how the arts fit into the federal budget and puts them in context with tax breaks offered to other special interests, including private industry. “Federal arts funding” later had the honor of receiving Createquity’s first citation in a Wikipedia article. -IDM)

As has been previously reported, public funding for the arts is one of the many foci of our national debate over fiscal policy. While funding cuts for the National Endowment for the Arts and the National Endowment for the Humanities (and potential but unrealized cuts at the Smithsonian) all made national headlines, the Corporation for Public Broadcasting, which unwillingly and inaccurately functions as a proxy for NPR in the public imagination, was the hottest of the hot potatoes. The House of Representatives voted to defund the CPB entirely, but in the end, appropriations were essentially unchanged from the year before. This may seem like a dead issue for the moment, but there is an extremely good chance these battles will resurface in the fiscal 2012 budget process.

Federal arts funding is a small share of the budget
So, how much money are we talking about? The CPB is getting $455 million (of which about $90m goes to radio stations). The Smithsonian gets the biggest federal arts allocation, at $761m. If you add all arts-related federal programs together, funding for the current fiscal year totals just over $2.5 billion. Honestly, that number looks pretty large to me. I can’t imagine what a billion of anything really looks like. But the total federal budget for this fiscal year (which runs through September) is $3.82 trillion. So the federal arts funding we’ve identified is 0.066% of the total federal budget. And when we’re only shouting about CPB, we’re talking about 0.012%. That is twelve one-thousandths of one percent.

So, why? Why was this a central topic in budget debates this spring? Was it really all about this James O’Keefe scandal? Are we going to rehash the entire set of arguments again this summer and fall as we debate the fiscal 2012 budget?

Maybe the real reason we keep putting Elmo’s head on the chopping block is because we don’t really understand the numbers, after all. According to a CNN poll, most Americans do not think the CPB gets twelve one-thousandths of one percent of the budget. Actually, only 27% of those surveyed believe the CPB gets less than 1%1 of the total budget. 40% believe the CPB gets 1-5%. Everyone else believed the appropriation to be greater than 5%, and an astonishing 7% of those surveyed believe the Corporation for Public Broadcasting gets more than 50% of the budget (which would have to be close to $2 trillion). If that were true, it would put the pledge premium tote bag and mug industry completely out of business, and This American Life would be hosted by Robin Leach. The survey also asked whether funding for different funding categories should be increased, decreased, kept the same, or eliminated. 16% of respondents wanted CPB funding to be totally eliminated.

We care about other spending, too, right?
It’s interesting to examine the other spending categories in the poll. The survey asked about Medicare, Medicaid, Social Security and defense. Those are pretty big portions of the budget, so it makes perfect sense they’d be in the list. The rest of the categories are decidedly different: foreign aid (also highly overestimated by respondents), benefits for retired government workers, food and nutrition assistance for poor people, housing for the poor, and federal education funding. And that’s it.

But, what’s missing here? Quite a lot, actually. What about subsidies to the oil & gas industry, which the Obama administration claims add up to $4 billion (about 9 times CPB funding)? What about direct subsidies for farmers, which were about $5 billion last year? Tax exemptions for ethanol production aren’t mentioned, either. Nor are the $8.5 billion in subsidies given to the airlines since 9/11 simply to help them survive. These subsidies went to for-profit industries, which are theoretically subject to the rigor of the free market and exist for the profit of their shareholders. And yet, more discussion is generated by $2.5 billion in subsidies to arts organizations, both governmental and non-profit, that explicitly exist for the public benefit and do not have shareholders.

Why didn’t CNN ask about mortgage interest tax deductions of $88.5 billion in 2008, 200 times this year’s CPB funding? What about first time home buyer and hybrid vehicle tax credits? In 2008, contributions to employee retirement and pension funds, and tax deferrals on the earnings in those funds, lost the federal government $117.7 billion in tax revenue. There are many, many more examples. Decisions to fund and subsidize these sectors of the U.S. economy are just as important as decisions about arts funding. And the amounts are significantly higher than the $2.5 billion of federal arts funding in question.

How is it so easy for Congress to ignore all of this during budget battles, and instead focus on whether Juan Williams should have been fired or not? One reason is that subsidies can easily be swept under the rug, when the rug is the tax code.

Tax breaks: spending with less scrutiny
Tax deductions and credits, also called tax expenditures, are a form of government spending, as we can see clearly from the now-ended hybrid vehicle tax credits. The federal government wanted to provide incentives for the purchase of fuel-efficient vehicles. It would have cost the federal government about the same to send a check directly to hybrid buyers, perhaps processed at the dealerships, as it cost to reduce tax bills by the same amount. (Transaction and processing costs might be different, but the bulk of the cost would have been the same.)

The difference between tax expenditures and direct spending is that the former are not part of large budget bills, the kind that can shut down the government if not passed. Tax expenditures can certainly be treated as political footballs. But they are far less likely to be at the center of a showdown. Not only that, if Congress adds a tax expenditure in some legislation, it is a spending increase that can be framed to look like a tax cut, because it reduces tax revenues. That makes it more politically palatable for both parties, even if it has nothing to do with taxed income, and even if it distorts markets.

Consider a very large tax expenditure: $88.5 billion (2008 figure) worth of mortgage interest tax deductions (almost 200 years of Corporation for Public Broadcasting funding). Interest you pay on your mortgage gets deducted from your taxable income. Thus, if you’re comfortably into the 25% tax bracket, this tax expenditure is worth a quarter of what you paid in mortgage interest during the year. This creates an explicit incentive for people to buy their own homes by borrowing. If creating a home ownership and borrowing incentive sounds a little off to you, you might be recalling the financial collapse that precipitated the Great Recession. Like the repackaging of loans into mortgage-backed securities that contributed to the housing price bubble of the previous decade, this deduction effectively makes borrowing cheaper. If borrowing is cheaper for everybody, then everybody has more to spend, and if everybody would like to buy a somewhat nicer or bigger home if they could, then all the home prices are simply going to increase. This deduction, therefore, distorts the market and leads to increased prices. Who benefits if all prices in the market are inflated to take advantage of this deduction? It helps the realtors, who get paid a percentage of the sale price. And it helps the home building industry.

What is the point? Do I want a repeal of this tax deduction? Personally, no, I’ve already got a mortgage, and of course I want to keep my deduction. It is simply important to understand that this is spending, too. The mortgage interest deduction is really a government spending program that encourages people to buy instead of rent, and has the unintended effect of inflating home prices. And it’s a pretty large one! Why don’t we publicly debate this spending program, which is 200 times greater than the CPB budget, and is of debatable long-term utility? We don’t have to talk about it, because it’s not in the budget. It’s in the tax code, so it looks like a way to reduce taxes, rather than a way to subsidize the home sale industry.

Reframing the conversation
If subsidized arts workers are labeled as something like freeloaders in public discourse, then farmers, homeowners, hybrid vehicle buyers, the airlines, and the oil & gas industry are freeloaders too. Ayn Rand is very popular again among conservatives, so where is the conservative outcry against oil & gas subsidies? Instead, we are offered a redefinition of the “free market capitalist system” as something that requires government subsidy. Oxymorons rule the day when the free market must be subsidized, and arts created explicitly in the public interest, without a profit to distribute, must stand alone.

The issue of arts funding is quite likely to be revived when the fiscal 2012 budget is to be presented this fall. Conservative legislators have been able to score political points with this issue for years. But we have also seen President Obama bring greater scrutiny to bear on oil industry tax breaks, and he was making political progress in April. If uncertainty regarding oil supplies in the Middle East fades by this fall, we can perhaps expect that some part of the government spending conversation will deal with oil and gas tax expenditures.

Arts advocates, however, should not sit on our hands and wait for the President to shift the focus to federal subsidies of other industries in the budget and tax code. We are supposed to be very good at telling stories, so we ought to thoughtfully study our budget and tax code and engage with our citizenry on those issues that are most relevant and significant. It’s not just a matter of self-interest, though that is obviously part of the equation. America’s budget deficit and public debt is ours. And when we only discuss federal budgets when we launch a campaign to save our NEA grants and Sesame Street, we are lending legitimacy to those who would focus on the nickels and dimes while ignoring the big budgetary issues. We have the capacity for wider scope.

1. If you look at the wording of the poll question, you can see it is potentially a bit misleading. It asks, “Just give me your best guess — you can pick any number from one percent to a hundred percent, or if you think it was less than one percent, you can say that too.” The question first asks people to choose between 1-100%, so it anchors the idea of whole number percentages in the listener’s mind, then offers a less than 1% option as an alternative, after they’ve already framed the question in terms of whole number percentages.

Leave a comment

We’re Fixing Up the Place

Now that you’ve all helped us reach our Indiegogo campaign goal (thanks again!), it’s time to start putting that money to good work. We just returned from our staff retreat (more updates coming soon for those on the “Early Warning” list), and now we’re beginning the search for someone to help us redesign createquity.com.

It’s been four years since our last redesign by the excellent Evan Stein/VANPOP, and much has changed in the world of Createquity, and digital strategy. We’re in need of a new logo, new features for the site, and a refreshed design aesthetic. More details for technical requirements and the proposal process in the RFP, but if you know of a designer/developer with experience adapting WordPress themes, please help us reach out to them this week. We’re looking for a response to the RFP by this Friday, July 18.

We want this redesign to better support our expanded goals for Createquity, as well as your needs, dear reader. Help us help you, and pass on any tips for excellent digital professionals.

Leave a comment

Thank you!

All I can say is WOW. In the last 40+ hours of our campaign, more than $4,000 came in from 33 donors to put us over the top and then some. In the end, we blew past our goal, raising $11,430 (counting an employer match that was not included in the Indiegogo totals). Thank you to everyone who made this possible!

We’ll be following up with all of our donors individually, but I want to briefly thank a few folks in particular. First, I want to acknowledge our contributors at the Founding Sponsor level, MailChimp and Pamela York Klainer, without both of whom it would have been much harder for this campaign to meet its goal. You’ll be seeing their sponsor credits on our About page when we relaunch. Thomas Cott, Diane Ragsdale, Nina Simon, and Andrew Taylor all enhanced our campaign in special ways by either recording videos of support or donating items for us to offer as perks, and provided invaluable “social proof” for this enterprise. Our entire network of people who have been directly involved with Createquity came through huge, both contributing and spreading the word: 100% of our editorial team donated, along with many of our former Writing Fellows and guest bloggers, and we also received amazing support from coworkers, bosses, and current and former interns. And finally, I want to give a special thank you to my fellow editorial team member Jackie Hasa, who was incredibly helpful and effective in managing the campaign from behind the scenes.

There’s no time to sit back and savor the moment: all this while as the campaign has been going on, we’ve been hard at work planning an editorial retreat that is set to take place this weekend. Stay tuned for more announcements as we start to put these resources into action. We promise it will be money well spent.

Leave a comment

[Createquity Reruns] Solving the Underpants Gnomes Problem: Towards an Evidence-Based Arts Policy

(Arts Research Week at Createquity concludes with this speech/post originally delivered at the University of Chicago’s Cultural Policy Center on November 14, 2012 and published on the blog in February 2013. This diagnosis of how our arts research infrastructure is failing us, a vision for how we could fix it, and why it all matters – a lot – is emblematic of the more advocacy-driven approach we intend to take upon our relaunch in the fall. I’m glad to say that there has been progress on some of these recommendations even in just the past year and a half, in particular the formation of the Cultural Research Network to connect researchers with each other and start the process of field-building. Another reason this talk is significant is that it led to my first connection with current Createquity editorial team member John Carnwath! -IDM)

The actual lecture portion of this talk occupies the first 52 minutes of the video, and the first 27 of those minutes are a recap/synthesis of material that will be familiar to regular readers of this blog (specifically, Creative Placemaking Has an Outcomes Problem and In Defense of Logic Models). Since I didn’t write out the speech in advance, I don’t have a transcript for it. However, below is a reconstruction of the new material from my notes, so you can get a taste for it if you don’t have time to watch the whole thing right now. (You’ll notice I make a number of generalizations in the speech about the ways in which arts practitioners interact with research. These are based on observation and personal experience, and are best understood as my working hypotheses.)


[starting at 26:55]

Why is this integration between data and strategy important? Because research is only valuable insofar as it influences decisions. This is why logic models are awesome – they are a visual depiction of strategy. And there is no such thing as strategy without cause and effect. Think about that for a second. Our lives can be understood as a set of circumstances and decisions. We make decisions to try to improve our circumstances, and sometimes the circumstances of those around us. Every decision you make is based on a prediction, whether explicitly articulated or not, about the results of that decision. Every decision, therefore, carries with it some degree of uncertainty. This uncertainty can be expressed another way: as an assumption about the way the world works and the context in which your decision is being made. These assumptions are distinguished from known facts.

If you can reduce the uncertainty associated with your assumptions, the chances that you will make the right decision will increase. So, how do you reduce that uncertainty? Through research, of course! Studying what has happened in the past can inform what is likely to happen in the future. Studying what has happened in other contexts can inform what is likely to happen in your context. And studying what is happening now can tell you whether your assumptions seem spot on or off by a mile. Alas, research and practice in our field are frequently disconnected in problematic ways. Six issues are preventing us from reaching our potential.

Issue #1: Capacity

Supply and demand apply as much to research as it does to artists. There are far more studies out there than a normal arts professional can possibly fully process. I wish I could tell you how many research reports are published in the arts each year, but nobody knows! To establish a lower bound, I went back over last year’s [2011] “around the horn” posts, which report new research studies that I hear about. I counted at least 41 relevant arts-research-related publications – a tiny fraction, I’m sure, of total output. To make matters worse, research reports are long, and arts professionals are busy. For the Createquity Writing Fellowship program, participants are required to analyze a work of arts research for the Createquity Arts Policy Library. I collect data on how long it takes to do this, and consistently, it requires 30-80 hours to research, analyze and write just one piece! Multiply this by the number of new studies each year, and you can start to see the magnitude of the problem.

Issue #2: Dissemination

Which research reports is an arts practitioner likely to even know about? Certainly not all of them, because there is almost no meaningful connection between the academic research infrastructure and the professional arts ecosystem. Lots of research relevant to the arts is published in academic journals each year, but unless the faculty member was commissioned to do their work by a foundation, we never hear about it. Academic papers are typically behind a pay firewall, and most arts organizations don’t have journal subscriptions. To give an example, after I wrote about Richard Florida’s Rise of the Creative Class, Florida pointed me to a study in two parts by two Dutch researchers. It’s one of the best resources I’ve come across for creative class theory, but I’ve never heard anyone even mention either study other than him and me.

Issue #3: Interpretation

Research reports inevitably reflect the researcher’s voice and agenda. This is especially true of executive summaries and press releases, which is often all anyone “reads” of research “reports.” Probably the most common agenda, of course, is to convey that the researcher knows what he/she is talking about. Another common agenda is to ensure repeat business from, or at least a continuing relationship with, the client who commissioned the study. The reality, however, is that research varies widely in quality. There’s no certification process; anyone can call themselves a researcher. But even highly respected professionals can make mistakes, pursue questionable methods, or overlook obvious holes in their logic. And, in my experience, the reality of any given research effort is usually nuanced – some aspects of it are much more valuable than others. Unfortunately, many arts professionals lack expertise to properly evaluate research reports, not having had even basic statistics training.

Issue #4: Objectivity

Research is about uncovering the truth, but sometimes people don’t want to know the truth. Advocacy goals often precede research. How many times have you heard somebody say a version of the following: “We need research to back this up”? That statement suggests a kind of research study that we see all too often: one that is conducted to affirm decisions that have already been made. By contrast, when we create a logic model, we start with the end first: we identify what we are trying to achieve and only then determine the activities necessary to achieve it.

Here are a bunch of bad, but common reasons to do a research project:

  • To prove your own value.
  • To increase your organization’s prestige.
  • To advance an ideological agenda.
  • To provide political cover for a decision.

There is only one good reason to do research, and that is to try to find out something you didn’t know before.

Issue #5: Fragmentation

The worst part of the problem I just described is that it drives what research gets done – and what doesn’t get done. There is no common research agenda adopted by the entire field, which is a shame, because collective knowledge is pretty much the definition of a public good: if I increase my own knowledge, it’s very easy for me to increase your knowledge too. The practical consequences of this fragmentation are severe. It results in a concentration of research using readily available data sources (ignoring the fact that the creation of new data sources may be more valuable). It results in a concentration of research in geographies and communities that can afford it, because people don’t often pay for research that’s not about them. And it results in a concentration of research serving narrow interests: discipline-specific, organization-specific, methodology-specific. My biggest pet peeve is that research is almost never intentionally replicated – everybody’s reinventing the wheel, studying the same things over and over again in slightly different ways. A great example of a research study crying out for replication is the Arts Ripple Effect report, which I talked about earlier. The results of that study are now guiding the distribution of millions of dollars in annual arts funding. Are those results universal, or unique to the Greater Cincinnati region? We have no way to know.

Issue #6: Allocating resources

Everyone knows there’s been a trend in recent years towards more and more data collection at the level of the organization or artist. Organizations, especially small ones, complain all the time about being expected to do audience surveys, submit onerous paperwork, and so forth. And you know what, I agree with them! You might be surprised to hear me say that, but when you’re talking about organizations that have small budgets, no expertise to do this kind of work, and the funder who is requesting the information is not providing any assistance to get it…just take a risk! You make a small grant that goes bad, so what? You’re out a few thousand dollars. The sun will rise tomorrow.

As an example of what I’m talking about, I participated in a grant panel recently. I enjoyed the experience, and am glad I did it, but there’s one aspect of the experience that is relevant here. There were seven panelists, and we were all from out of town. Each of us spent, I’d say, roughly 40 hours reviewing applications in advance of the panel itself. Then we all got together for two full days in person to review these grants some more and talk about them and score them. We did this for 64 applications for up to $5,000 each, and in the end, 92% 94% were funded.

So consider this as a research exercise. The decision is who to give grants to, and how much. The data is the grant applications. The researchers are the review panel. What uncertainty is being reduced by this process? How much worse would the outcome have been if we’d just taken all the organizations, put them into Excel, run a random number generator, and distributed the dollars randomly up to $5,000 per organization? And I’m not saying this to make fun of this particular organization or single them out, because honestly it’s not uncommon to take this kind of approach to small-scale grantmaking. And yet if you compare it to ArtPlace’s first round of grants, theoretically they had thousands of projects to choose from, and they gave grants up to $1 million for creative placemaking projects – but there was no [open] review process; they just chose organizations to give grants to. So there’s a bit of a mismatch in the strategies we use to decide how to allocate resources.

There’s a concept called “expected value of information” described in a wonderful book called How to Measure Anything, by Douglas W. Hubbard. It’s a way of taking into account how much information matters to your decision-making process. In the book, Hubbard shares a couple of specific findings from his work as a consultant. He found that most variables have an information value of zero; in other words, we can study them all we want, but whatever the truth is is not going to change what we do, because they don’t matter enough in the grand scheme of things. And he also found that the things that matter the most, the kinds of things that really would change our decisions, often aren’t studied, because they’re perceived as too difficult to measure. So we need to ask ourselves how new information would actually change the decisions we make.

There is so much untapped potential in arts research. But it remains untapped because of all the issues described above. So what can we do about it?

First, we need a major field-building effort for arts research. Connecting researchers with each other through a virtual network/community of practice would help a lot. So would a centralized clearinghouse where all research can live, even if it’s behind a copyright firewall. The good news is that the National Endowment for the Arts has already been making some moves in this direction. The Endowment published a monograph a couple of months ago called “How Art Works,” the major focus of which was a so-called “system map” for the arts. But the document also had a pretty detailed research agenda for the NEA, not for the entire field, that lays out what the NEA’s Office of Research and Analysis is going to do over the next five years, and two of the items mentioned are exactly the two things I just talked about: a virtual research network and a centralized clearinghouse for arts research.

This new field that we’re building should be guided by a national research agenda that is collaboratively generated and directly tied to decisions of consequence. The missing piece from the research agenda in “How Art Works” is the tie to actual decisions. Instead it has categories, like cultural participation, and research projects can be sorted under those buckets. But it’s not enough for research to simply be about something – research should serve some purpose. What do we actually need to know in order to do our jobs better?

We should be asking researchers to spend less time generating new research and more time critically evaluating other people’s research. We need to generate lots more discussion about the research that is already produced. That’s the only way it’s going to enter the public consciousness. Each time we fail to do that, we are missing out on opportunities to increase knowledge. It will also raise our collective standards for research if we are engaging in a healthy debate about it. But realistically, in order for this to happen, field incentives are going to have to change – analyzing existing research will need to be seen as equally prestigious and worthy of funding as creating a new study. Of course, I would prefer if people are not evaluating the work of their direct competitors – but I’ll take what I can get at this point!

Every research effort should take into account the expected value of the information it will produce. Consider the risk involved in various types of grants made. What are you trying to achieve by giving out lots of small grants, if that’s what you’re doing? Maybe measure the effectiveness of the overall strategy instead of the success or failure of each grant. This is getting into hypothesis territory, but based on what I’ve seen so far I would guess that research on grant strategy is woefully underfunded, while research on the effectiveness or potential of specific grants is probably overfunded. We probably worry more than we need to about individual grants, but we don’t worry as much as we should about whether the ways in which we’re making decisions about which grants to support are the right ways to do that.

Finally, we should be open-sourcing research and working as a team. I’m talking about sharing not just finished products and final reports, but plans, data, methodologies as well. I’m talking about seeking multiple uses and potential partners at every point for the work we’re doing. This would make our work more effective by allowing us to leverage each other’s strengths – we’re not all experts at everything, after all! And it would cut down on duplicated effort and free up expensive people’s time to do work that moves the field forward.

I thank everyone for their time, and I’d love to take any questions or comments on these thoughts about the state of our research field.

(Enjoyed this post? Today is the last day of our campaign to make the next generation of Createquity possible. We’re thrilled to have reached our initial goal, but additional contributions are still welcome and will be put to good use in strengthening us for the future. Thank you for your support!)

1 Comment

Last Chance to Take Createquity to the Next Level!

Createquity readers, tomorrow is the final day in our Indiegogo funding campaign. Thanks to your generous contributions, as of this writing we have raised $7,385 from 93 funders toward our $10,000 goal. It’s been truly humbling to witness the number of people who care enough about high-quality information and analysis in the arts to contribute. And with just about 36 hours left in the campaign, it’s time to put the pedal to the metal to bring us over the top. If you believe we as a sector need better, data-driven advocacy, or simply appreciate Createquity as a resource for your work, please donate today!

One of the most gratifying things about this campaign so far has been seeing the wave of support we’ve received from people whose work is central to our field. Barry Hessenius, whose blog is another widely-read resource among arts managers, graced us last week with a completely unsolicited and glowing endorsement of this project:

I hope you will go to the Indiegogo site and support this effort.  I did….I can give you two good reasons why you might part with the cost of a couple of Starbuck’s half caffeine, double mocha, caramel, latte frappacinnos:  First:  Ian and the people he has assembled to help with his newest reinvention of his site are exactly the people we want to support in our field – young, smart, dedicated, committed people who are already making a contribution to the field to help make things better for everyone.  Supporting that alone ought to be worth ten or twenty bucks.  But Second, I can almost guarantee you that if you follow whatever Createquity does over the next year you will read two or more posts that you (you personally) will find of great value to what you are doing on your job.  That ought to be worth a few bucks, no?

And how often do you get to play Santa Claus in July?

Indeed, it’s been amazing to see the movers and shakers who find value in Createquity’s work. Every year, with the help of a pool of nominators, Barry compiles a list of the nonprofit arts sector’s 50 most powerful and influential leaders. More than a fifth of the 2013 list has contributed to our campaign so far. The show of support from our field has been extraordinary, with donations from star consultants like Holly Sidford (Helicon Collaborative), Alan Brown (WolfBrown), Adrian Ellis (AEA), Jerry Yoshitomi (MeaningMatters), Claudia Bach (AdvisArts), and Anne Gadwa Nicodemus (Metris Arts Consulting); arts organization leaders like Adam Huttler (Fractured Atlas), Laura Zucker (LA County Arts Commission), Mara Walker (Americans for the Arts), and Kemi Ilesanmi (The Laundromat Project); current and former foundation leaders like Kerry McCarthy (New York Community Trust), Angelique Power (Joyce Foundation), and Marian Godfrey (ret. Pew Charitable Trusts); and fellow arts thinkers and information mavens Doug McLennan (ArtsJournal), Nina Simon (Museum 2.0), Thomas Cott (You’ve Cott Mail), Andrew Taylor (The Artful Manager), and Diane Ragsdale (Jumper). The latter four have contributed to our campaign in particularly special ways: Thomas, Andrew, and Diane all were kind enough to record video testimonials for us (embedded below), and Nina is donating two rare signed copies of her classic read The Participatory Museum, which are available to donors at the $100 level. Grab ‘em fast!

I hope you agree with us that this is a pretty incredible list. Won’t you add your name to it and help us cross the finish line?

Leave a comment

[Createquity Reruns] Public Art and the Challenge of Evaluation

(Createquity’s summer rerun programming continues this week with a focus on arts research! This instant classic by Createquity Writing Fellow Katherine Gressel spread like wildfire when it was first published in January 2012, and remains our third-most popular post ever. It even brought us a bunch of new readers from Australia! [Long story.] While not a short read, it’s packed with useful information about how practitioners have gone about conceptualizing and evaluating one of the hardest beasts to measure – public art. -IDM)

Steve Powers, “Look Look Look,” Part of the “A Love Letter for You” project, commissioned by the Philadelphia Mural Arts Program, 2009-2010. http://www.aloveletterforyou.com

In the Spring/Summer 2011 issue of Public Art Review, Jack Becker writes, “There is a dearth of research efforts focusing on public art and its impact. The evidence is mostly anecdotal. Some attempts have focused specifically on economic impact, but this doesn’t tell the whole story, or even the most important stories.”

Becker’s statement gets at some of the main challenges in measuring the “impact” of a work of public art—a task which more often than not provokes grumbling from public art administrators. When asked how they know their work is successful, most organizations and artists that create art in the public realm are quick to cite things like people’s positive comments, or the fact that the artwork doesn’t get covered with graffiti or cause controversy.

We are much less likely to hear about systematic data gathered over a long time period—largely due to the seemingly complex, time-consuming, or futile nature of such a task. Unlike museums or performance spaces, public art traditionally doesn’t sell tickets, or attract “audiences” who can easily be counted, surveyed, or educated. A public artwork’s role in economic revitalization is difficult to separate from that of its overall surroundings. And as Becker suggests, economic indicators of success may leave out important factors like the intrinsic benefits of experiencing art in one’s everyday life.

However, public art administrators generally agree that some type of evaluation is key in not only making a case for support from funders, but in building a successful program. In the words of Chicago Public Art Group (CPAG) executive director Jon Pounds, evaluations can at the very least “help artists strengthen their skills…and address any problems that come up in programming.” Is there a reliable framework that can be the basis of all good public art evaluation? And what are some simple yet effective evaluation methods that most organizations can implement?

This article will explore some of the main challenges with public art evaluation, and then provide an overview of what has been done in this area so far with varying degrees of success. It builds upon my 2007 Columbia University Teachers College Arts Administration thesis, And Then What…? Measuring the Audience Impact of Community-Based Public Art.That study specifically dealt with the issue of measuring audience response to permanent community-based public art, and included interviews with a wide range of public artists and administrators.

This article will discuss evaluation more broadly—moving beyond audience response—and incorporate more recent interviews with leaders in the public art field. My goal was not to generate quantitative data on what people are doing in the field as a whole with evaluation (according to Liesel Fenner, director of Americans for the Arts’s Public Art Network, such data is not yet available, though it is a goal). Instead, I have reviewed recent literature on public art assessment, and interviewed a range of different types of organizations, from government-run “percent for art” and transit programs to grassroots community-based art organizations in New York City (where I am based) and other parts of the United States. I sought to find out whether evaluation is considered important, how much time is devoted to it, and the details of particularly innovative efforts.

The challenge of defining what we are actually evaluating

The term “public art” once referred to monumental sculptures celebrating religious or political leaders. It evolved during the mid-twentieth century to include art meant to speak for the “people” or advance social and political movements, as in the Mexican and WPA murals of the 1930s, or the early community murals of the 1960s-1970s civil rights movements. Today, “public art” can describe anything from ephemeral, participatory performances to illegal street art to internet-based projects. The intended results of various types of public art, and our capacity to measure them, are very different.

In the social science field, evaluation typically involves setting clear goals, or expected outcomes, connected to the main activities of a program or project. It also involves defining indicators that the outcomes have been met. This exercise often takes the form of a “theory of change.” Since there are so many types of public art, it is exceedingly difficult to develop one single “theory of change” for the whole field, but it may be helpful to use a recent definition of public art from the UK-based public art think tank Ixia: “A process of engaging artists’ ideas in the public realm.” This definition implies that public art will always occupy some kind of “public realm”–whether it is a physical place or otherwise-defined community—and require an “engagement” with the public that may or may not result in a tangible artwork as end result. This process and the reactions of the public must be evaluated along with whatever artistic product may come out of it.

The challenge of building a common framework for evaluation

In 2004, Ixia commissioned OPENspace, the research center for inclusive access to outdoor environments based at the Edinburgh College of Art and Heriot-Watt University, to research ways of evaluating public art, ultimately resulting in a comprehensive 2010 report, “Public Art: A Guide to Evaluation” (see a helpful summary by Americans for the Arts). The guide’s emphasis and content was shaped by feedback from Ixia’s Evaluation Seminars and fieldwork conducted by Ixia and consultants who have used its Evaluation Toolkit. Ixia provides the most comprehensive resources on evaluation that I have encountered, with two main evaluation tools, the evaluation matrix and the personal project analysis. These are helpful as a starting point for evaluating any project or program.

The matrix’s goal is to “capture a range of values that may need to be taken into account when considering the desirable or possible outcomes of engaging artists in the public realm.” It is meant to be filled out by various stakeholders during a project-planning stage, as well as at the midpoint and conclusion of a project.

Ixia’s “personal project analysis”is “a tool for process delivery that aims to assess how a project’s delivery is being put into practice.” I will not analyze it in detail here, except to say that something similar should also ideally be part of any organization’s evaluation plan, as it allows for assessing how well the project is being carried out.

Personal Project Analysis from Ixia’s “Public Art: A Guide to Evaluation”

Matrix from Ixia’s “Public Art: A Guide to Evaluation”

Ixia’s matrix identifies four main categories of values:

  1. Artistic Values [visual/aesthetic enjoyment, design quality, social activation, innovation/risk, host participation, challenge/critical debate]
  1. Social Values [community development, poverty and social inclusion, health and well being, crime and safety, interpersonal development, travel/access, and skills acquisition]
  1. Environmental Values [vegetation and wildlife, physical environment improvement, conservation, pollution and waste management-air, water and ground quality, and climate change and energy],
  1. Economic Values [marketing/place identity, regeneration, tourism, economic investment and output, resource use and recycling, education, employment, project management/sustainability, and value for money].

The matrix accounts for the fact that each public artwork’s values and desired outcomes will be different depending on the nature of the presenting organization, site, and audience.

It is unclear how widely these tools have been adopted in the UK since their publication, and I did not encounter anyone in the U.S. using them. Yet many organizations are employing a similar process of engaging various stakeholders during the project-planning phase to determine goals specific to each project, which relate to the categories in Ixia’s matrix. For example, most professionals I interviewed cited some type of “artistic” goals for the work. Some organizations prioritize presenting the highest quality art in public spaces, in which case the realization of an artist’s vision is top priority (representatives of New York City’s Percent for Art program described “Skilled craftsmanship” and “clarity of artistic vision” as key success factors, for example).

By contrast, organizations that include a youth education or community justice component may rank “social” or “economic” values higher. Groundswell Community Mural Project, an NYC-based nonprofit that creates mural projects with youth, asks all organizations that host mural projects (which may include schools, government agencies, and community-based organizations) in pre-surveys to choose their top desired project outcomes from a range of choices, as well as identify project-specific issues. Groundswell does have a well-developed theory of change behind all its projects, relating to the organization’s core mission to “beautify neighborhoods, engage youth in societal and personal transformation, and give expression to ideas and perspectives that are underrepresented in the public dialog.” However, some project-specific outcomes may be more environmental—for example, partnerships with the Trust for Public Land to integrate murals into new school playgrounds–while some relate to “crime and safety,” as in an ongoing partnership with the NYC Department of Transportation to install murals and signs at dangerous traffic intersections that educate the public about traffic safety.


Groundswell Community Mural Project, signs from “Traffic Safety Program,” a partnership between Groundswell, the Department of Transportation’s Safety Education program, and several NYC public elemenary schools. Lead artists Yana Dimitrova, Chris Soria, and Nicole Schulman worked with students to create these signs installed at locations identified as most in need of traffic signage.

Groundswell is just one example of many public art organizations that set goals at the outset of each individual project, based on each project’s particular site and community. While individual organizations may effectively evaluate their own projects this way, crafting a common theory of change for all public art may be an unrealistic expectation.

The challenge of reliable indicators and data collection

The Ixia report discusses the process by which indicators of public art’s ability to produce desired outcomes may be identified, with the following questions:

  1. Is it realistic to expect a public art project to influence the outcomes you are measuring?
  2. Is it likely that you can differentiate the impact of the public art project and processes from other influences, e.g., other local investment?
  3. Is it possible to conduct meaningful data on what matters in relation to the chosen indicators?

For example, in studies seeking to measure any kind of change, good data collection should always include a baseline—i.e., economic conditions or attitudes of people BEFORE the public art entered the picture. Data collection methods ideally should also be reliable, unbiased, and easily replicated.

The “Guide to Evaluation” does not go into detail about any concrete indicators of public art’s “impact.” Therefore, the matrix seems to be most useful as a guide to goal-setting. As the Americans for the Arts summary of this report points out, “Ixia directs users to [UK-based] government performance indicators as a baseline source, but that is where the discussion ends.”

Liesel Fenner of Americans for the Arts’s Public Art Network mentioned in an email to me that while PAN hopes to develop a comprehensive list of indicators in the future, which can be shared among public art presenters nationally, “developing quantitative indicators is the main obstacle.”

According to my interviews with both on-the-ground administrators and public art researchers, many busy arts administrators find the type of data collection recommended in Ixia’s guide difficult, costly and time-consuming. It can be a challenge to get artistic staff to buy into even basic evaluation; says one community arts administrator, “artists are paid for a their leadership in developing and delivering a strong project. Many artists don’t see as much value in evaluation because, in part, it comes in addition to the difficult work that they just accomplished.” It is also uncommon to spend precious training resources on something like quantitative evaluation techniques.

Some are of the opinion that even if significant time were spent on justifying public art’s existence by “proving” its practical usefulness, this would still be a losing battle that could lead to the withdrawal of support for public art, the production of bad art that panders merely to public needs, or both. One seasoned public art administrator asked me: “Is architecture evaluated this way? The same way public buildings need to exist, public art needs to exist. It’s people looking to weaken public art who are trying to ask these questions about its impact.”

The challenge of evaluating long-term, permanent installations

Glenn Weiss, former director of the Times Square Alliance Public Art Program and current director of Arts League Houston, posits that economic impact studies are “most possible with highly publicized, short-term projects like the Gates or large public art festivals.” Indeed, the New York City Mayor’s office published a detailed report on “an estimated $254 million in economic activity” that resulted from The Gates, a large installation in Central Park by internationally acclaimed artists Christo and Jeanne-Claude, based on data like increased park attendance and business at nearby hotels, restaurants, etc. However, most public art projects, even temporary ones, are not as monumental or heavily promoted as The Gates, making it difficult to prove that people come to a neighborhood, or frequent its businesses, primarily to see the public art.

Visitors crowd Christo and Jeanne-Claude’s “The Gates” (2005) in Central Park. Photo by Eric Carvin.

Weiss also believes that temporary festivals are generally easier to evaluate quantitatively than long-term public art projects. For example, during a finite event or installation, staff members can keep a count of attendees (some of the temporary public art projects I have encountered in my research, such as the FIGMENT annual participatory art festival on Governors Island and in various other U.S. cities, use attendance counts as a measure).

The few comprehensive studies connecting long-term, permanent public art to economic and community-wide impacts, conducted by research consultants and funded by specific grants, have led to somewhat inconclusive results. For example, An Assessment of Community Impact of the Philadelphia Department of Recreation Mural Arts Program (2002), led by Mark J. Stern and Susan C. Seifert of University of Pennsylvania’s Social Impact of the Arts Project (SIAP), cites the assumed community-wide benefits of murals outlined in MAP’s mission statement at the time of the study:

The creation of a mural can have social benefits for entire communities…Murals bring neighbors together in new ways and often galvanize them to undertake other community improvements, such as neighborhood clean-ups, community gardening, or organizing a town watch. Murals become focal points and symbols of community pride and inspiring reminders of the cooperation and dedication that made their creation possible.

Yet when asked to “use the best data available to document the impact that murals have had over the past decade on Philadelphia’s communities,” Stern and Seifert found that

this is a much more difficult task than one might imagine. First, there are significant conceptual problems involved in thinking through exactly how murals might have an impact on neighborhoods. Second, the quality of data available to test hypotheses concerning murals is limited. Finally, there are a number of methodological problems involved in using the right comparisons in assessing the potential impact of murals. For example, how far from a mural might we expect to see an impact? How long after a mural is painted might it take to see an effect and how long might that effect last?…Ultimately, this report concludes that these issues remain a significant impediment to understanding the role of murals.

By comparing data on murals to existing neighborhood quality of life data, Stern and Seifert considered murals’ connection to factors like community economic investment and indicators of more general neighborhood change (such as reduced litter or crime, or residents’ investment in other community organizing activities). The study also measured levels of community investment and involvement in murals. However, the scarce data available on these factors, according to the authors, are difficult to connect directly to public art in a cause and effect relationship. Stern and Seifert’s strongest finding was that murals may build “social capital,” or “networks of relationships” that can promote “individual and group well-being,” because of all the events surrounding mural production in which people can participate. It was more difficult to show a consistent relationship between murals and other theorized outcomes, such as ability to “inspire” passersby or serve as “amenities” for neighborhoods. The study recommends that “more systematic information on their physical characteristics and sites—‘before and after’—would provide a basis for identifying murals that become an amenity.”

A more recent 2009 report on Philadelphia’s commercial corridors by Econoconsult also demonstrated “some indication of a positive correlation” between the presence of murals and shopping corridor success. Murals are described here as “effective and cost efficient ways of replacing eyesores with symbols of care.” However, the report also adds the disclaimer that a positive correlation is not necessarily proof of the murals’ role as the primary cause of a neighborhood’s appeal.

So what can we assess most easily, and how?

My research revealed that quantitative data on short-term inputs and outputs of public art programs is frequently cited (sometimes inappropriately) as evidence of a program’s success in things like reports or funding proposals—for example, number of new projects completed in one year, number of youth or community partners served, or number of mural tour participants. However, in this article I am not really focusing on this type of reporting, as it does not address how public art impacts communities over time.

The good news is that there are several examples of indicators that are more easily measurable in certain types of public art situations, including permanent installations. These include:

  • Testimonies on the educational and social impact of collaborative public art projects, from youth and community participants and artists alike
  • Qualitative audience responses to public art, including whether or not the art provokes any type of discussion, debate, or controversy
  • How a public artwork is treated over time by a community, including whether it gets vandalized, and whether the community takes the initiative to repair or maintain it
  • Press coverage
  • The “use” of a public artwork by its hosts, e.g. in educational programs or marketing campaigns
  • Levels of audience engagement with public art via internet sites and other types of educational programming

Below I will summarize some helpful methods by which data is collected around all these indicators.

Mining the Press

Archiving press coverage of public art projects online is a common practice among organizations, as is presenting pithy press clippings and quotes in funding proposals and marketing materials as a means of demonstrating a project’s success. For researchers, studying articles (and increasingly, blog posts) on past projects can also provide rich documentation of artworks’ immediate effects, as well as points of comparisons. For example, the “comments” sections of online articles and blogs can generate interesting, often unsolicited feedback, albeit from a nonrandom sample.

One possible outcome of public art projects is controversy, which is not always considered a bad thing, despite now-infamous examples of projects like Richard Serra’s Tilted Arc being removed. For example, Sofia Maldonado’s 42nd Street Mural, presented in March 2010 by the Times Square Alliance, provoked extensive coverage on news programs and blogs. The mural’s un-idealized images of Latin American and Caribbean women based on the artist’s own heritage led some women’s and cultural advocacy organizations to call for its removal. The Alliance opted to leave the mural up, and has cited this project as evidence of the Alliance’s commitment to artists’ freedom of expression. The debates led Maldonado to reflect, “as an art piece it has accomplished its purpose: to establish a dialogue among its spectators.”

Sofia Maldonado, “42nd Street Mural,” 2010, Commissioned by the Times Square Alliance Public Art Program.

Site visits and “public art watch”

As an attempt to promote more sustained observation of completed works over time, public art historian Harriet Senie assigns her students in college and graduate level courses a final term paper project every semester that contains a

“public art watch”…For the duration of a semester, on different days of the week, at different times, students observe, eavesdrop, and engage the audience for a specific work of public art. Based on a questionnaire developed in class and modified for individual circumstances, they inquire about personal reactions to this work and to public art in general” (quoted in Sculpture Magazine).

Senie’s students also observe things like people’s interactions with an artwork, such as how often they stop and look up at it, take pictures in front of it, or use it as a meeting place.

Senie maintains that “Although far from ‘scientific,’ the information is based on direct observation over time—precisely what is in short supply for reviewers working on a deadline.” This approach towards challenging college students to think critically about public art has also been implemented in public art courses at NYU and Pratt Institute, and the aggregate results of student research over time are summarized in one of Senie’s longer publications.

I have not encountered any other organizations able to integrate this type of research into their regular operations; however, there may be opportunities to integrate direct observation into routine site visits to completed permanent public artworks.

In the NYC Percent for Art program, and its Public Art for Public Schools (PAPS) wing that commissions permanent art for new and renovated school buildings, staff members are expected to undertake periodic visits “to monitor the condition of artworks that have been commissioned,” according to PAPS director Tania Duvergne. Such “maintenance checks” can provide opportunities to survey building inhabitants or local residents about their opinions and use of the artworks.

Duvergne uses these “condition report” visits as opportunities to further her agency’s mission to “bridge connections between what teachers are already doing in their classrooms and their physical environments.” At each site, she tries to interview custodians, teachers, principals and students about whether the art is well treated, whether they know anything about the artwork (and are using the online resources available to them), and whether they want more information. Duvergne notes that many teachers use the public art in their teaching in some way, even if they do not know a lot about the artwork. While observing a public artwork during a site visit every few years is nowhere near as extensive and sustained observation as Senie’s class assignment, perhaps a similar survey and observation could be undertaken with a wide range of students and staff members over the course of a day.

Project participant and resident surveys

Organizations that create community-based public art usually have specific desired social, educational, or behavioral outcomes in project participants. Mural organizations Groundswell and Chicago Public Art Group describe thorough evaluation processes in which mural artists, youth, community partners and parents are all surveyed and sometimes interviewed before, during and after projects. Groundswell’s community partner post-project survey, for example, asks partners to rank their level of agreement about whether certain community-wide outcomes have been met, such as whether the mural increases the organization’s visibility, increases awareness of an identified issue, and improves community attitudes towards young people.

Groundswell’s theory of change (most recently honed in 2010 through focus groups with youth participants and community partners) articulates various clear desired outputs and outcomes for both youth and community partner organizations. This includes the development of “twenty-first century” life skills in teen mural participants. To measure this impact specifically, Groundswell has made it a priority to continue to track youth participants after they graduate, turn 21, and reach other checkpoints, according to Executive Director Amy Sananman. Groundswell recently hired an outside researcher to build a comprehensive database (using the free program SalesForce), in which participant data and survey results, and data on completed murals (such as whether any were graffitied, how many times they appeared in news articles, etc.) can be entered and compared to generate reports.

In 2006, Philadelphia’s Mural Arts Program conducted a community impact study using audience response questionnaires as a starting point. Then- special projects manager Lindsey Rosenberg employed college students, through partnerships with local universities, to conduct door-to-door surveys of all residents living within a mile radius of four murals. The murals differed by theme, neighborhood, and level of community involvement. The interns orally administered a multiple-choice questionnaire with questions ranging from general opinions of the murals to level of participation in making the murals to perceptions of changes in the neighborhood as a result of the murals. They then inputted the surveys into a computer database specifically created for this study by outside consultants. The database not only calculated percentages of each response to murals, but tracked correlations between these responses and census demographic data, including income level and home ownership.

This research project was different from prior MAP community impact studies in that it assumed that “what people perceive to be the impact of a mural is in itself valuable,” as much as external evidence of change.

In 2007, MAP shared some preliminary results of this endeavor with me to aid my thesis research. At the time the research seemed to generate some useful data on which murals were appreciated most in which neighborhoods, and the correlation between appreciation and community participation in the projects. However, since then I have not been able to gather any further information on this study, or find any published results. I did hear from MAP at the time of the study that only 25% of people who were approached actually took the surveys, indicating just one problematic aspect of conducting such research on a regular basis. The database was also costly.

Most recently, MAP is partnering (page 160) with the Philadelphia Department of Behavioral Health & Mental Retardation Services (DBH/MRS), community psychologists from Yale, and almost a dozen local community agencies and funders with core support from the Robert Wood Johnson Foundation, on “a multi-level, mixed methods comparative outcome trial known as the Porch Light Initiative. The Porch Light Initiative examines the impact of mural making as public art on individual and community recovery, healing, and transformation and utilizes a community-based participatory research (CBPR) framework.” Unfortunately, MAP declined my requests for more information on this new study.

Interviewing youth and community members can of course only generate observations and opinions, but Groundswell at least is also taking the step of also tracking what happens to participants after they complete a mural project. I am still not clear how to prove that any impacts on participants are a direct result of public art projects. Yet surveying project participants and community members about their feelings about a program or project, and how they think they were impacted by it, is one of the most do-able types of research (apart from the challenges of getting people to fill out surveys).

Community-based “proxies”

Groundswell director Amy Sananman has described some success in utilizing community partners as “proxies” for reporting on a mural’s local impact, effectively outsourcing some of the burden of data collection to other organizations. For example, the director of a nonprofit whose storefront has a Groundswell mural could report back to Groundswell on the extent to which local residents take care of the mural, how often people comment on it, etc.

PAPS, CPAG, and ArtBridge, an organization that commissions artwork for vinyl construction barrier banners, have described similar ideas for partnerships. ArtBridge hopes to implement a more formal process in which the owners of stores where its banners are installed can document changes like increased business due to public art. PAPS director Tania Duvergne also cites examples of “successful projects” in which public schools, on their own, designed art gallery displays or teaching curricula around their public art pieces, and shared this with PAPS on site visits.

There might be a danger in depending on community partner organization representatives to speak for the whole “community” or to provide reliable, accurate data. But if cooperative partners can be identified and regular reporting scheduled using consistent measurement tools, the burden of reporting on specific neighborhoods is lessened for the public art organization.

“Smart” Technology

Groundswell, ArtBridge, and MAP are all starting to utilize the new QR code smartphone application, which uses QR codes to direct public art site visitors to websites with more information about the art. Groundswell experimented this past summer with adding QR codes to a series of posters designed by its Voices Her’d Visionaries program to be hung in public schools to educate teens about healthy relationships. Groundswell can then track how many hits the website gets through the QR app. In general, web activity on public art sites is an easy quantitative measure of public interest.

Philadelphia’s Mural Arts Program has a “report damage” section on its website, where anyone who notices a mural in need of repair can alert MAP online. This is also a potential source for quantitative evidence of how many people notice and feel invested in murals.

Use of Interpretive Programming

Public art organizations are increasingly designing interpretive programming around completed artwork, from outdoor guided tours to curated “virtual” artwork displays. NYC’s Metropolitan Transit Authority’s Arts for Transit program provides downloadable podcasts about completed artworks on its website; other organizations include phone numbers to call for guided tours at public art sites themselves (as in many museum exhibits). Both in-person and virtual/phone tours can provide rich opportunities to track usage, collect informal feedback from participants, and solicit feedback via surveys. ArtBridge recently initiated its WALK program giving tours of its outdoor banner installations. After each tour, ArtBridge emails a link to a brief questionnaire to all tour participants, and offers a prize as an incentive for taking the survey.

A Philadelphia Mural Arts Program guided tour.

Concluding remarks: What next for evaluation?

While systematic, reliable quantitative analysis of public art’s impact at the neighborhood level remains challenging and undervalued in the field, new technologies as well as effective partnerships are making it increasingly feasible for public art organizations to assess factors such as audience engagement, benefits to participants, and community stewardship of completed public art works. The Ixia “Guide to Evaluation” offers a useful roadmap for approaching the evaluation of any type of public art project. At the same time, we should not forget the ability of art to affect people in ways that may seem intangible or even immeasurable, or, as Glenn Weiss puts it, “become part of a memory of a community, part of how a community sees itself.”

(Enjoyed this post? We’re raising funds through this Thursday to make the next generation of Createquity possible. We’re getting close, but need your help to cross the finish line. Please consider a tax-deductible donation today!)

Leave a comment