Creative Placemaking Has an Outcomes Problem

(Note: a follow-up to this post, “In Defense of Logic Models,” is now available here)

“I feel like whenever I talk to artists these days, I should be apologizing,” says Kevin Stolarick, Research Director for the Martin Prosperity Institute at the University of Toronto’s Rotman School of Management. To most in the arts community, Stolarick is better known as Richard Florida’s longtime right-hand man and research collaborator on his bestselling book, The Rise of the Creative Class. Stolarick, who first met Florida just after the academic had cashed the first check for the advance from Basic Books, proceeds to recount how the book’s success led to an explosion of interest from mayors all around the country wanting to redefine their cities as welcoming meccas for Florida’s new Starbucks-drinking, jeans-wearing idea people. Unfortunately, the mayors’ collective interpretation of the lessons from Florida’s book boiled down to, “all we need is to get us some gays and artists and a bike path or two, and our problems will be solved! The problem,” Stolarick tells us, a decade after The Rise of the Creative Class’s publication, “is that it’s a trap.”

This scene is unfolding in a basement auditorium in lower Manhattan, the site of a panel and presentation hosted by the Municipal Art Society of New York to give audiences the first public preview of the ArtPlace vibrancy indicators. ArtPlace, as many readers know, is a private-sector partnership among nearly a dozen leading foundations to support “creative placemaking,” a term invented by officials at the National Endowment for the Arts. Spearheaded by leadership from the NEA, the creation of ArtPlace is perhaps this Endowment’s, and by extension the Obama administration’s, signature achievement in the arts—despite the fact that it doesn’t distribute a cent of government money.

Stolarick’s presence at the event was appropriate, for in many ways it was The Rise of the Creative Class that made the current creative placemaking movement possible. For a time it was the kind of book that smart people buy for all of the other smart people they know – a genuine ideavirus. Florida, more than anyone else, was responsible for conflating creativity, innovation, and artistry in the popular imagination, and among the measures that he and Stolarick developed for the book was a “Bohemian index” associating the concentration of artists in a given metropolitan area with population and employment growth. Though the empirical claims in the book turned out to be built on shaky foundations, they were intuitive (and well-argued) enough that municipal leaders started taking notice. In fact, Carol Coletta, the current director of ArtPlace, was one of the first people to invite Florida to help put his ideas into practice in a real city context as co-organizer of 2003’s Memphis Manifesto Summit. Florida, Stolarick, and their associates became the first widely acknowledged spokespeople for the idea that a vibrant set of opportunities and amenities for creative expression could lead to regional economic prosperity.

But Florida wasn’t the only one drawing public attention to the economic power of the arts over the previous decade. Separately, the Social Impact of the Arts Project at the University of Pennsylvania has been studying the relationship between concentrations of cultural resources and various social and economic outcomes since 1994. As then-Associate Director of the Rockefeller Foundation, Joan Shigekawa commissioned a groundbreaking collaboration between SIAP and The Reinvestment Fund to study the dynamics of culture and urban revitalization, work whose influence can be seen clearly in much of the policy that Shigekawa has since helped develop as Senior Deputy Chairman of the NEA.

SIAP, which is led by Mark Stern and Susan Seifert, cites The Rise of the Creative Class frequently in its publications dating from that period, usually to position its approach in opposition to Florida’s. In fact, in 2008 SIAP published one of the most hilariously brutal program evaluations I’ve ever read, following the attempts of Florida’s Creative Class Group (CCG) to turn around three Knight Foundation communities by inspiring volunteer “catalysts” to drive toward the “4 T’s” of economic development (technology, talent, tolerance, and territorial assets). In that evaluation, Stern and Seifert offer a single overarching criticism: CCG forgot about its outcomes. Much like South Park’s Underpants Gnomes, the project team had a clear idea of what it was putting in to the process and what it hoped to get out of it, but a much vaguer sense of how it was going to get from Phase 1 to Phase 3.

Which brings me to my central point: despite all of the attention paid to this issue in the past year and a half, despite all of the new money that has been committed to the cause, creative placemaking still has an outcomes problem. As a field, we have not yet learned the lessons of the Underpants Gnomes. And until we do, I’m worried that we risk repeating Stolarick’s apology to practitioners a decade hence.

Leaving the dots unconnected

“When times were good,” Kevin Stolarick explains at the ArtPlace vibrancy indicators convening, it was easy for city councils, funders, and others to buy into the ideas in Florida’s book on the strength of his celebrity and qualitative arguments. But now that cities are facing more economic pressure, Stolarick continues, “they’re saying, ‘we need proof – and that’s going to take more than Richard Florida’s next book.’”

“Proof” is a word that seems to give creative placemakers hives these days. Less than two weeks prior to the ArtPlace event, I had participated in a webinar given by the NEA to introduce its Our Town Community Indicators Study. Our Town is the Endowment’s public-sector counterpart to ArtPlace – likewise the brainchild of Rocco Landesman, it granted some $6.6 million to communities for creative placemaking projects across the country in its inaugural round last year. The Community Indicators Study is a multiyear data collection effort whose chief purpose is to “advance public understanding of how creative placemaking strategies can strengthen communities.” Yet when, prompted by researchers who were listening in on the call, the NEA’s Chief of Staff, Jamie Bennett, asked the Deputy Director of NEA’s Office of Research and Analysis about causation vs. correlation, this is the exchange that resulted:

Bennett: …Are you going to in some way be able through this project to prove [for example] that arts had a direct impact in causing the crime rate to go down?

Shewfelt: A lot of the language I’ve used today has been very carefully chosen to avoid suggesting that we are trying to design a way to specifically address the causal relationship between creative placemaking and the outcomes we’re interested in.

As a matter of fact, the NEA has chosen to forgo a traditional evaluation of the Our Town grant program in favor of developing the aforementioned indicator system. The project will no doubt result in a lot of great data, but essentially no mechanism for connecting the Endowment’s investments in Our Town projects to the indicators one sees. A project could be entirely successful on its own terms but fail to move the needle in a meaningful way in its city or neighborhood. Or it could be caught up in a wave of transformation sweeping the entire community, and wrongly attribute that wave to its own efforts. There’s simply no way for us to tell. I hate to be the bearer of bad news, but we can’t accomplish the goal of “advancing understanding of how creative placemaking strategies can strengthen communities” without digging more deeply into the causal relationships that the NEA would prefer to avoid.

The vibrancy indicators that were the subject of the ArtPlace convening face a similar quandary. The purpose of the indicators is to help ArtPlace “understand the impact of [its] investments.” And what is that desired impact? During a webinar delivered to prospective applicants last fall, Coletta declared that “with ArtPlace, we aim to do nothing less than transform economic development in America…to awaken leaders who care about the future of their communities to the fact that they’re sitting on a pile of assets that can help them achieve their ambitions…and that asset is art.”

ArtPlace’s investments all have a singular focus on “vibrancy,” a concept defined in its guidelines as “attracting people, activities and value to a place and increasing the desire and the economic opportunity to thrive in a place.” While that was as specific as things got during ArtPlace’s first two rounds of grantmaking, the indicators project, which examines factors as diverse as cell phone use, population density, and home values, will go a long way toward concretizing ArtPlace’s primary lever of community transformation. Even so, ArtPlace doesn’t seem any more eager than the NEA to connect the activities of its grant recipients to the broader vibrancy indicators directly. Though the projects themselves are supposed to have a “transformative” impact on vibrancy, ArtPlace isn’t requiring its grantees to collect any data on how that impact is achieved. Furthermore, ArtPlace’s guidelines state clearly that the consortium has no plans to invest in research on creative placemaking beyond the vibrancy indicators themselves, despite its advocacy goals and a desire to “share the lessons [grantees] are learning to other communities across the U.S.”

To be clear, I don’t mean to question the value of research of the type ArtPlace and Our Town are leading. Efforts such as these, Fractured Atlas’s Archipelago data aggregation and visualization platform, Americans for the Arts’s National and Local Arts Index, Western States Arts Federation’s Creative Vitality Index, and others help to draw a clear picture of a community’s overall cultural and creative health and can serve as an essential tool within a broader research portfolio. But in order for those tools to really come alive in a grantmaking context, they have to be grounded in a clear and rigorous conceptual frame for the how the specific funded activities are going to make a difference, and then integrated into the actual process for selecting grant recipients. And that’s the part still missing from the vast majority of these efforts. In an upcoming article for the Grantmakers in the Arts Reader, Anne Gadwa Nicodemus (who co-authored the original Creative Placemaking white paper for the NEA with her mentor, Ann Markusen) writes, “it’s probably unreasonable to expect that a modest, one-year Our Town grant will move the needle, at least quickly….Because the geographic scale, time horizons, and desired outcomes vary across creative placemaking efforts, one-size-fits-all indicator systems may prove inappropriate.”

Without a clear and detailed theory of how and why creative placemaking is effective, policy and philanthropy to support creative placemaking is hobbled. Attempting to predict and judge impact based on indicator systems alone carries with it at least four problems:

• It doesn’t give a clear road map for project selection that will identify investments most likely to make a difference. Without previous research demonstrating causal interactions between grants given and differences made, it’s hard to know what effect a new grant will have – much less how to compare the potential effects of hundreds or (in ArtPlace’s case) thousands of competing investment opportunities.
• It doesn’t give us the tools to go back and analyze why certain projects did and didn’t work. Maybe a public artwork succeeds in drawing people to a neighborhood, but real estate values stay stagnant. Maybe development along a transit corridor was executed on schedule, but ridership is lower than expected. Broad, sector-level indicators will only tell us that the project didn’t work – not why.
• It doesn’t acknowledge the complex nature of economic ecosystems and the indirect role that arts projects play in them. Many economists agree that talented, highly educated individuals are key to community prosperity. But numerous considerations likely play into their decision to (re)locate in a particular place. When are the arts truly catalytic for a community, and when are they merely icing on the cake? Indicator systems would have no way of telling us on their own.
• It provides little insight on how to pursue arts-led economic development while avoiding the thorny problems of gentrification. Any thinking around policy interventions must acknowledge the possibility of negative impacts as well as positive ones. In the case of creative placemaking, an attendant worry is that longtime residents of transformed neighborhoods won’t have asked for the change, and may be adversely affected by it. To date, there is little shared understanding of how creative placemaking projects that benefit all community residents are distinguished from those that simply replace poorer residents with wealthier ones.

In her Reader article, Nicodemus writes that

The answer to the question “What is creative placemaking, really?” is that funders and practitioners are making it up in real time. We’ve entered an exciting period of experimentation, which makes sharing information absolutely critical.

In the interest of sharing information, then, I will report out below on some lessons I’ve learned from my own research on the topic over the past five years, as well as from a collaboration with ArtsWave, a funder supporting vibrancy through the arts in the Greater Cincinnati/Northern Kentucky region.

Toward a unified theory of creative placemaking: Filling in the blanks

The major deficiency of the Underpants Gnomes’ business plan was that they attempted to connect their activity (stealing underpants) with their intended impact (profit), without really considering the steps in between. To take an extreme example, if I start an organization called “Artists for World Peace” (there is such an organization, by the way), get some artists together to stand in solidarity, and put on a show, it would be unrealistic of me to expect world peace as the next logical result.

Yet most studies of the connection between the arts and economic development have attempted to measure the direct relationship between arts activities (whether single or in the aggregate) and economic outcomes. For example, the Social Impact of the Arts Project examined the correlation between cultural assets and poverty decline in Philadelphia, and a groundbreaking study by Steve Sheppard compared employment levels and real estate values in North Adams, MA before and after the opening of the Massachusetts Museum of Contemporary Art. These research efforts have done much to shape our collective understanding of urban revitalization through the arts. But they share in common an unfortunate tendency to gloss over the details of exactly how creative activities are responsible for making neighborhoods and communities more attractive, and therefore, more valuable. This gap is especially problematic when one tries to apply the lessons of these studies to a policy or grantmaking context, where the details of how projects are implemented can make all the difference in whether a particular intervention is successful or not.

When I was in graduate school, before I came into contact with any of the research above, I created a simple model of arts-led gentrification to illustrate the specific case of a neighborhood lent a young, “hip” reputation by newly relocated artists. This model is different from others I’ve seen in a few ways. First, it casts neighborhood development as an iterative process, starting with tourism on the local level among artists. In other words, the people who are going to be checking out the happenings in a struggling outpost of the city are not, by and large, yuppies – they are other artists who are colleagues of the ones living in that neighborhood. Second, it emphasizes the role of bars and restaurants as attractors for other neighborhood visitors (including yuppies), whose viability is only made possible by the modest foot traffic generated by arts activities. And finally, it places at the beginning of the process not just arts activities, but specific kinds of arts activities: visible, storefront spaces like galleries and performance venues that signal the presence of art and draw visitors to a particular location.

Three years later, some of the thinking reflected above found its way into my grantmaking strategy work with ArtsWave, an local arts agency based in Cincinnati, OH. First, some background: in late 2008, ArtsWave had commissioned a research initiative designed to develop an inclusive public conversation about the arts in the region. Based on hundreds of conversations, interviews, and focus groups with area residents, two key “ripple effect” benefits emerged as especially valued by citizens:

1. that the arts create a vibrant, thriving economy: neighborhoods are more lively, communities are revitalized, tourists are attracted to the area, etc…and
2. that the arts create a more connected community: diverse groups share common experiences, hear new perspectives, understand each other better.

To its immense credit, ArtsWave didn’t just sit on these results and continue in the status quo. Instead, the 83-year-old united arts fund underwent a total transformation, taking on a new name and organizational identity, and most importantly, adopting these two themes as the new goals for its grantmaking.

My task, starting in January 2011, was to assist ArtsWave in creating a new framework for funding arts & culture activities based upon the ability of organizations to create vibrancy and connect people in the region. With the help of a volunteer task force consisting of ArtsWave board members, staff, community leaders, and grantee organizations, we worked backwards from the idea of “vibrancy” and ended up with an extraordinarily complex theory of change. Here’s the part that specifically deals with cultural clusters and neighborhood economic development:

Some elements of this model will certainly look familiar, though with some new wrinkles added: evening and weekend hours for storefronts, for example, as well as decreased crime and improved physical spaces (in general, not just arts spaces). ArtsWave, however, extended the concept to apply to regional economic development as well:

Note here that the principal lever for the regional development model is that the Greater Cincinnati/Northern Kentucky region is “differentiated” through the arts. That is to say, it attracts people from outside of the region because it gains a (deserved) reputation for being a more interesting place to be than its peer cities. And what helps differentiate Cincinnati is something we call “extraordinary cultural experiences.” We attach a very specific definition to “extraordinary,” focusing on its literal meaning of “out of the ordinary.” For ArtsWave’s purposes, experiences are extraordinary if they are associated with one of the following:

• Events or productions with a national or international profile
• Events or productions that feature something uniquely special about the region
• Events or productions that feature innovative programming or presentation

Not only do experiences meeting the above criteria help to differentiate the Greater Cincinnati region in the eyes of tourists or prospective residents, they also contribute directly to ArtsWave’s notion of “vibrancy” (the green arrow in the diagram).

What this approach does is explicitly connect the activities of grantees to the broader community change that ArtsWave hopes to create. A key innovation that came out of this process was the distinction between “Sector Outcomes” (in blue) and “Grantee Outcomes” (in purple). We defined grantee outcomes as the farthest point out in the model to which individual organizations could reasonably be held accountable—and those outcomes feed back into the evaluation and selection process at the grant application stage. All other outcomes, the sector outcomes, are a reflection on ArtsWave’s overall strategy, rather than on any one particular investment. This allows us to “aggregate” impact from the level of the individual project to the level of the broader context.

The beauty of designing a model like this is that it allows each assumption embedded in each link on the causal chain to be tested, if necessary. Of course, it would be impractical to do so for every investment a grantmaker might make. But that isn’t necessary. In order to provide the kind of evidence that mayors and other officials are looking for, you only need a few examples to demonstrate replicability. But we have to be sure that those examples really do show the effects of intentional creative placemaking strategy, rather than just a lucky coincidence.

Where We Go From Here

Despite the challenges I discuss in the first part of this article, I’m heartened to see creative placemaking funders taking some positive steps toward a more rigorous theoretical foundation for their work. In particular, ArtPlace is beginning to move in this direction with a list of ten signals grantees can use to judge whether their projects are making a difference. The challenge will be to unpack those relationships with the same rigor as is currently applied to collecting data.

Meanwhile, we would love feedback on the models we have created to describe economic development through the arts. While we are hopeful they can help to move the conversation towards a deeper consideration of the complex mechanisms involved in creating place-based vibrancy, we readily acknowledge that they aren’t perfect. Do they accurately reflect creative placemaking goals and processes? Which aspects of the model are best backed up by existing research and which are shakiest? Which seem intuitively right but have not been studied in depth? What are we leaving out?

If you have comments, questions, or resources to offer, please leave a comment here or get in touch at ian.moss@fracturedatlas.org. And in the meantime, Fractured Atlas will be eagerly researching how emerging evaluation methods in other sectors, such as outcome mapping, most significant change technique, and complexity science, can potentially be applied to the arts.