One intersection of Data and art. Photo by Geoff Stearns.

One intersection of Data and art. Photo by Geoff Stearns.

For all of the predictions flying back and forth about what 2013 holds for the arts and culture sector in the United States, one of the few things we can say with near-certainty is that 2013 will be a year of major transition for the Cultural Data Project (CDP). Our sector’s largest-scale effort to quantify and streamline the “value” of arts and culture across different regions, the CDP is breaking off from its original home at the Pew Charitable Trusts to become a separate nonprofit. Until now, the CDP has been governed by a local Pennsylvania-based board, but it recently launched a national board and announced Beth Tuttle as their new CEO, who will lead a strategic planning process to further evolve its scope and impact. These shifts, according to the CDP, will “put [it] in an even stronger position to serve the arts and culture community” and its “increasingly large and diverse number of constituents.”

As arts organizations are increasingly pressed to demonstrate tangible benefits, this turning point in the CDP’s history provides an opportunity to look back on its trajectory and examine ways to measure the contributions of arts and culture to society.

For those unfamiliar with the CDP and how it came to be, here is a little refresher:

Background of the CDP

The CDP launched in Pennsylvania in 2004 thanks to the collaboration between the Pew Charitable Trusts and a number of local funders. Using the CDP platform, arts and cultural organizations create an annual Data Profile based on their financial audit and quantitative programmatic data, such as the number of exhibitions or workshops held. It is an online tool that aims to help arts and cultural organizations “improve their financial management and services to communities” by streamlining, storing and aggregating data on their financial and programmatic activities each year.

In grantseeking, it functions a little bit like the Common Application for undergraduate college admissions, which some funders have emulated through standardized application templates; once organizations create Data Profiles they can then submit customized reports to multiple funders. Arts organizations also have access to a variety of tools that illustrate trends in their performance from year to year, and can compare their data against other organizations in the region or across all states participating in the CDP.

The tools include:

  • A Program Revenue and Marketing Expense Report, which explores the relationship between marketing expenses and attendance figures
  • A Personnel Report, which outlines costs associated with staffing, salaries and benefits
  • A Contributed Revenue and Fundraising Expense Report, which examines how changes in fundraising expenses effect contributed revenue
  • The Financial Health Analysis, developed in partnership with the Nonprofit Finance Fund, which serves as a fiscal health “check up” and provides an overview of financial strengths, weaknesses and business dynamics

Users can tailor comparison reports according to detailed specifications. For example, if you’re a mid-sized theater company, you can see how your programming and marketing expenses compare to those of your peers.  You can also see how they compare against, say, all arts organizations that were founded within a certain time frame and target a specific constituent group. CDP Help Desk Staff assists in running those reports, and with cleaning and checking the data that goes into creating a Data Profile, which consists of nearly 300 questions and requires 15 to 30 hours to complete.

Why go through such a complex process? For one thing, funders get a standardized profile with grant applications that allows them to easily compare organizations. Accordingly, most funders who support the CDP require that nonprofits submit their Data Profiles and a customized report with their grant applications. Furthermore, many funders are interested in research, and the CDP provides a massive data bank. The more funders require their constituents to use the CDP, the bigger the ultimate payoff for researchers: a regional (and, perhaps one day, national) aggregate of information about arts and cultural organizations, updated annually.

As for the cultural organizations that have to complete the Data Profile, those applying to multiple grantmakers requiring CDP reports receive the benefit of being able to submit detailed financial information in one format. If they participate over several years and take advantage of the reports to track and compare their progress, they also gain a better picture of their own strengths. The drawback comes for organizations with limited capacity to devote to completing the Data Profile, or for those seeking to evaluate more qualitative elements of their work. But therein lie the possibilities.

The CDP’s Impact to Date: The Research Perspective

Thirteen states currently take part in the CDP, with sixteen more expressing interest. (A list of participating states with links to their CDP Web sites is available here.) In the near-decade since the CDP launched, how has it benefited the myriad cultural organizations that input data and the researchers who analyze it?

Most researchers to date have used CDP data to try to quantify the broad “impact” of the arts and culture sector in financial and programmatic terms, totaling up the number of people employed by arts and culture organizations in a given area, the number of public events, the number of attendees, etc.  A snippet from Cuyahoga County’s Strengthening Communities 2011 report provides one such example:

Excerpt from Strengthening Communities

The ability to quantify the scope and variety of the nonprofit arts and culture communities has proven invaluable to several advocacy campaigns. The Greater Philadelphia Cultural Alliance used CDP data to lead constituents across the state to successfully defeat a proposed tax on arts and culture activities. ArtServe Michigan’s Creative State Michigan Report, comprised mainly of CDP data, prompted the state’s Republican governor to propose a threefold increase in state arts funding.

Regardless of whether studies like these demonstrate that the arts generate economic growth, research resulting from the CDP has undoubtedly helped make the case for investment in the arts and culture and tell a more concrete story about people served. These efforts have been somewhat limited, however, by the “one size fits all” nature of the CDP’s construction. After all, attributes like financial health and the quantity of public events would be more accurately described as outputs rather than outcome or impact measures. The fact that CDP profiles are optional to fill out for organizations whose funders don’t require it also creates challenges for researchers who wish to generalize from CDP data to all nonprofit arts and culture organizations, since the CDP constitutes a nonrandom sample of such organizations. Recently, funders in California and New York have begun piloting small grant programs to support research projects using the CDP in their states; time will tell if these opportunities lead to a more diverse range of inquiries using the data.

The CDP’s Impact to Date: Arts Organizations’ Perspective

The CDP has been a boon to research and advocacy efforts, but cultural organizations themselves don’t always hear about that work, or take full advantage of the CDP’s resources. In 2012, the CDP conducted a survey of over 1,800 arts organizations charged with filling out a Data Profile every year. Arin Sullivan, the CDP’s Senior Associate for State-Based Activities, said that the survey found that 68 percent of respondents had never read a report that includes CDP data, like Cuyahoga County’s. This implies that researchers, and the CDP itself, need to close the feedback loop between research and the constituents being studied. In addition, according to Sullivan, the survey revealed that more than 40% of participating organizations have never run an annual, trend, or comparison report.

There are a number of possible reasons for this. Organizations simply may not be aware of the CDP reporting features. They may have limited staff capacity to run and analyze reports, particularly after devoting significant time to completing their Data Profile. Or they are aware of the reporting features, but don’t think they are worth using.

No matter what the reasons, these results suggest a range of perceptions about the CDP. To some organizations, it is simply a profile to complete in order to get a grant. To others, it is a resource that can help better inform their practice. The same survey that found nearly half of organizations don’t use CDP reporting tools also found that 45% of participants understood their own finances better as a result of completing the Profile. Of those respondents that did use CDP reports, 40% said it resulted in better transparency, 45% said they had a better sense of their progress and goals, and 56% said they had a better sense of their organization over time.  These relatively low percentages suggest that even organizations taking full advantage of CDP reports do not always find them of substantial benefit.

Yet for some organizations, the CDP has been quite useful indeed. DanceWorks Chicago, for example, changed its entire accounting system to align with the format of the CDP. According to CEO Andreas Böttcher, many small organizations have little expertise in accounting, and until the advent of the CDP had no clear template to follow when setting up their books. Using the CDP was “onerous at first,” he said, “but once you have it, you can build a business plan based on what you have to report.”

DanceWorks has also made use of comparison reports. Böttcher said they’ve shown funders, for example, that not only is DanceWorks unique in its programming (which provides a laboratory for artistic growth to dancers, choreographers, and directors), but it is one of only a few organizations of its size to offer health insurance to dancers. Comparison reports also affirmed other things the company was doing well, despite its relative youth: its 80 individual donors reflect a larger funding pool than those of comparable, older organizations, and its web-based donations are 300% higher.

Other organizations have used the CDP to better track programmatic information, such as their most popular services or types of events. One such organization is the Rhode Island Historical Society, whose Executive Director, Morgan Grefe, used the CDP as an opportunity to make a greater commitment to program evaluation.  According to Grefe, having to complete the Data Profile made program evaluation “the thing you can’t ignore instead of the thing you always put off.” She credits the CDP process with the Society’s increased attentiveness to more qualitative program data, which has had an effect on the organization’s work.

For example, staff members started tracking how many visitors to different historic sites used those sites as a “visitor’s center,” asking for brochures and other tourist information, but not participating in a tour of the building.  Tracking such data enabled the Society to illustrate to public and private funders that the sites hold value to the tourist industry as well as to visitors concerned primarily with historic preservation. It also allows Grefe to make programmatic decisions, such as recruiting volunteers to fulfill visitor center functions or investing in more brochures.

An increasingly data-oriented attitude also led to the Society to launch the Rhode Island History Online Directory Initiative (RHODI). The RHODI Project will be “a comprehensive and detailed survey of Rhode Island’s history and heritage sector, [providing] not only trustworthy data on which to base future grant-funded proposals for such activities as collections cataloging, capacity building advice, preservation projects, educational programming, a virtual museum, but also the much needed impetus for synergies and collaborations.”

Beyond the Numbers: Looking Ahead as The CDP Expands

As these examples indicate, the CDP has shown potential in establishing and tracking organizational success measures that can encourage stronger business operations, advocate for the arts, and guide grantmaking. In order for that success to solidify, however, the CDP must gain buy-in from a broader group of stakeholders, and address the fact that its current ability to gauge the arts’ “impact” is limited.

Of course, the CDP can’t be expected to track and measure everything related to arts and culture. Perhaps in recognition of this fact, and in an effort to integrate CDP data with other available resources, Southern Methodist University recently announced a partnership with the CDP to launch the National Center for Arts Research (NCAR). Among other things, NCAR plans to launch an interactive “dashboard” where “arts leaders will be able to enter information about their organizations and see how they compare to the highest performance standards in areas such as community engagement, earned and contributed revenue and balance sheet health.” Assuming the effort manages to avoid redundancies with existing CDP infrastructure, it may fulfill its vision to become “a catalyst for the transformation and sustainability of the national arts and cultural community.”

At this point in the CDP’s history, it’s worth asking how it can further engage arts organizations, and what role it can play in evaluating more qualitative trends in cultural activities, such as audience loyalty and the evolution of programming. Since the organizations participating in the CDP range from zoos to art galleries, attempting to evaluate these aspects in any standardized way may be an exercise in insanity. What’s the dance education equivalent of a blockbuster museum exhibition? Nevertheless, there may be opportunities for the CDP to help organizations draw connections between CDP data and other reports, and to provide them with a broader toolkit of resources.

For example, for more than 20 years the League of American Orchestras has compiled a report of repertoire played by its member orchestras, including a list of the top 10 most frequently performed works, composers, and soloists. Though sadly only published in dense PDFs, one could track the repertoire of the top orchestras and determine if programming is, for example, becoming more conservative over time. Those same orchestras could make careful use of their own CDP data to make connections between programming choices and financial health.

The community and economic development sector, meanwhile, has attempted to build evaluative tools into a shared measurement system with a platform called Success Measures,  which is smaller in scope but has certain similarities to the CDP.  Success Measures provides its members, most of whom are organizations with little experience or capacity for program evaluation, with a set of evaluative tools (surveys, interview protocols, observational checklists, and so forth).  They receive training on how to use the tools and then, using an online system similar to the CDP’s, upload, clean and analyze their data.

Participation in Success Measures is optional. By contrast, in each state where CDP operates, many funders require it as part of the application process, effectively enforcing participation. Given the CDP’s reach and the strong level of technical support it already provides, it is well poised to develop and disseminate additional tools. Suppose an organization was hoping to understand its audience retention rates, and had the option of accessing standardized survey tools as a CDP participant. The CDP would provide the organization with technical assistance in using the tools, in exchange for the survey results being added to the aggregate database. Such a system could not only strengthen the value of participating in the CDP, but also take a step toward a deeper understanding of “impact.” New tools — even optional ones — can motivate organizations to take a closer look at their activities, perhaps discovering how cultural choices affect their bottom line over time, or if their audience development efforts are truly paying off.

No matter how (and if) it chooses to expand, a few things about the CDP are clear:

  • It has propelled an ongoing conversation about the economic role of arts and culture
  • It may yet build the capacity of the field as a whole if it more directly engages the arts and culture organizations it aims to serve
  • To do so, it needs to clarify and draw greater attention to the existing benefits of using the system, and stay alert to organizations’ needs moving forward

With both for-profit and nonprofit sectors gravitating toward Big Data (and seeking more and more people to help them make sense of that data), the CDP can take a lead in ensuring that aggregate information is not only useful to researchers studying the arts and culture sector, but also to the organizations — large and small, vibrant and struggling — that are its engine.

(Talia Gibas is Manager, Arts for All at the Los Angeles County Arts Commission and a past Createquity Writing Fellow. Amanda Keil is a mezzo-soprano and writer based in New York City.)

  • Rohana Elias-Reyes

    Thanks for the article; it’s a great time to take the temperature of the Cultural Data Project as it becomes independent.

    However, I would say the post understates the drawbacks of the CDP: it is extremely time-consuming for cultural organizations to complete (30+ hours is an entire work week!), there is no added financial benefit for completing it, and although some organizations require the CDP report, it is in addition to their own requirements, not instead of. Additionally, my guess is that the benefits are overstated. For example, in the cited Cayahoga County report that used CDP data to note the numbers of different types of constituents served and so forth, I would bet that is the type of data already tracked prior to the existence of the CDP, certainly that is the case in New York City. Furthermore, for the smaller organizations that use the CDP for an accounting template, I’m sure they would benefit more from development workshops in those areas than the time-consuming trial by fire of the CDP, which was not designed for that purpose. Simply googling “nonprofit accounting template” might be equally beneficial in that area, or dropping by a bookstore. Finally, both of the above statements of benefit rely on anecdotal/qualitative, not statistical, information – exactly the opposite of what the CDP requires of the organizations that must fill out the form. In other words, as nice as those stories are, are these two organizations in any way representative of benefit to the 1800 organizations required to complete the CDP?

    In many cases, government funding organizations and foundations already request and have much of the data requested on the CDP form. The CDP model puts the entire burden of entering this data into a centralized database on the cultural organization. In other words the costs of creating this database fall entirely on the cultural organizations and the benefits to the cultural organizations are negligible at best. The stated benefit of a one stop report for funders doesn’t exist; it’s simply an extra report. However, arts organizations are compelled to do it or lose funding that they were receiving prior to the CDP’s launch.

    Using my own organization, New School Concerts, as a mini case study: Four years ago the NYC Department of Cultural Affairs began requiring the CDP and the following year NYSCA began requiring it. There was, as noted, a big increase in the number of hours needed to request those funds we had received at a much lower time cost in the past, and no funding increase tied to the extra labor. In fact, because my organization is housed within a university, I had to complete two separate CDP profiles to satisfy the different requirements of the NYC DCA and NYSCA* and neither organization accepted the CDP reports in lieu of their own. In fact, four years later, both organizations still require the CDP and their own financial forms. The labor put into the CDP is simply in addition to what was traditionally required.

    Finally, I would say that CDP should make a greater effort to find out if the project is beneficial to cultural organizations. I was unable to complete the automated survey I received, because it steered me toward making inaccurate statements and thus would only lead to flawed data. I am curious what the actual response rate to the survey sent to the 1800 organizations was, as that would inform my thinking regarding the percentage rates (low as they already are) quoted among respondent in terms of how organizations are benefitting from the CDP. I’d also love to know if the agencies and foundations that have been requiring the organizations they fund to fill out the CDP have actually been accessing and using any of the data, and if they have experienced any benefits.

    From my perspective of working with the CDP for four years now, despite how wonderfully responsive and above and beyond patient their staff is, the project itself appears to be a case of the emperor’s new clothes. People are very excited about the idea of a centralized repository of data on a broadly defined cultural sector, even if the results are more or less invisible.

    *I should state that my organization exists within a parent university, and that some of my challenges with completing the CDP form would not exist for stand-alone organizations.

    • John Carnwath

      I’m not sure if the redundancy of filling out the CDP profile in addition to the funders’ financial forms is to be blamed on CDP or on the funders. If applicants are required to fill out the CDP forms, shouldn’t the funders who participate in CDP be obliged to pull all of their financial info from there? I’ve certainly applied for grants from foundations that use the CDP reports instead of other financial forms, and when that’s the case I was certainly grateful not to have to worry about reporting the financials as the application deadline approached (even if entering the data into CDP is a drag initially).

    • Thanks Rohana and John. The prospect of funders asking for CDP reports in addition to, rather than as a replacement for, their own budget forms is certainly an alarming one. I would love to know more about the extent to which this is happening more broadly. But I think we need to be realistic as well about what the CDP can actually replace. It certainly doesn’t replace a narrative, which is where a lot of the work of preparing an application comes in. And it can’t replace a foregoing projected budget for the current year or the year ahead, since it’s all about verified historical data. For example, Rohana’s claim that the NYC DCA “still require[s] the CDP and their own financal forms” may be true in a literal sense, but none of the required forms are actually duplicative with the CDP. So, the common characterization of the CDP as a kind of “common app” for the arts (which appears in this piece and which I’ve used myself, on occasion) is kind of inaccurate. A better comparison would be to something like a common form for high school grade transcripts.

    • Aaron Andersen

      I agree completely with Rohanna. The CDP is building a valuable data set, but it adds a lot of extra burden onto cultural nonprofits that don’t receive any benefit in return.

      Ian, the CDP can’t capture budget information, true, and so it ends up reporting different data than the budget forms. But organizations used to be able to file their budgets and their prior year results in the same format, in the same excel spreadsheet, without having to maintain a parallel set of financial statements. That’s what more than half of the CDP is: a parallel set of financial statements. Ain’t nobody got time for that.

      The one virtue in the financial information that I can see is, that by enforcing a common format, you take away some discretion that organizations have to skew their financials by reporting things by activity instead of functional expense, showing an operating budget separate from an unrestricted capital fund instead of consolidating it as all unrestricted, or whatnot. You take away the discretion that organizations have to format and organize their own financial statements. But we’ve generally been able to rely on the 990 for that, which most of us had to complete, anyway.

      So then, what’s left that the CDP uniquely offers is a set of common operational metrics, like number of ticket buyers, etc… That’s useful, I suppose. But not to the organizations who have to reformat and input the data, because the vast majority of us are already collecting and reporting this data in order to attract funding.

  • Pingback: Looking for the culture in the Cultural Data Project | thousandfold echo()

  • The CDP seems like a beneficial tool that is used to strengthen arts and cultural organizations but I would like to see it broaden it’s research to provide more information on the artistic programing art organizations use to stay in business.
    Here is Michigan the CDP tells us that art organizations paid out almost 5 times as much in salaries and fringe benefits to their employees as they did to pay the artists and performers they present. They paid more in professional services and almost half as much in turn for communications than they did for artistic programing.
    To truly provide a full picture of the health of an industry a cultural assessment of the costs and benefits of the arts and cultural sector needs to also include and assess the earned revenue and expenses and budgets of their main source – those who produce the programing for the sector. The way the project is set up now the artists and performers seem like an after thought.

  • Pingback: Sequester highlights region’s economic dependence on government…D.C. budget not as rosy as it seems…The impact of the CDP on arts organizations [News, 3.8.13] | The Daily WRAG()

  • Pingback: Big Data Meets Art | ChangingMedia()

  • Craig Coogan

    I have had to fill out the CDP for several organizations over the past several years. While the concept of the CDP is fine, the devil is in the details. Yes, it’s a horrible process to fill out, and it shouldn’t be given the technological age we live in.

    In the states that I have filled this out the major funders have not yet signed on, so there is a total duplication of effort. Given that the organizations I have done this for are VERY small (1 was $300K, 1 $600K) – the commitment of staff resources is way out of whack for any benefit.

    The fundamental issue is that the CDP has chosen to come up with their own definition of financial metrics. It’s hard enough to train a board to read financials and show how they relate to the tax return – but using the CDP reporting would require a third way for a board to look at figures and that’s just a train wreck waiting to happen.

    Why not pull data from the 990? Do a partnership with Guidestar. Our CPA’s have the data already in that format. It’s THE standard that most funders look for anyway. If you then want to supplement with further detail, fine, but let’s at least use the same foundation that the entire not-for-profit sector is required to follow in the first place.

    • Thanks very much for your input on both the benefits and challenges of participating in the Cultural Data Project.

      The financial metrics used in the Data Profile are based on Generally Accepted Accounting Principles (GAAP) and were developed with nonprofit finance experts over a four-year planning process. As Craig mentioned, the data that the CDP collects is slightly different from the 990 and more comprehensive since it is a national effort to standardize data relevant to the arts and cultural industry specifically.

      Your feedback is extremely timely and useful as the CDP is about to embark in its next phase of strategic planning and its first as an independent nonprofit. Many of the comments made in response to this blog post are issues that CDP staff is aware of and sensitive to, and eager to explore moving ahead. With that being said, we are looking forward to continuing the dialogue on how the CDP can build community and become a national resource for cultural enterprise.

  • Pingback: Catalogue for Philanthropy: Greater Washington » The Cultural Data Project()

  • Pingback: Big Data in the arts and culture sector: background reading - Chris Unitt's blog()