Title: “New Data Directions for the Cultural Landscape: Toward a Better-Informed, Stronger Sector”

Author(s): Sarah Lee and Peter Linett

Publisher: Cultural Data Project

Year: 2013

URL: http://www.culturaldata.org/wp-content/uploads/new-data-directions-for-the-cultural-landscape-a-report-by-slover-linett-audience-research-for-the-cultural-data-project_final.pdf

Topics: research, data

Methods: Theory/assertion, informed by synthesis of comments from a CDP-hosted online forum of researchers (disclosure: I was one of them), results from CDP’s internal strategic planning survey, and a paper by Margaret Wyszomirksi (not available online) “to frame and inventory the cultural data landscape.”

What it says: “New Directions” was commissioned by the Cultural Data Project in connection with that organization’s transition from a foundation-housed initiative to an independent nonprofit, and the strategic planning process that followed. The report was intended to inform that process by situating CDP’s efforts within the larger context of data collection throughout the United States cultural sector. It notes a growing abundance of and interest in arts and cultural data, but identifies six factors that “may be limiting the sector from effectively incorporating data into decision-making processes.” The six factors [paraphrased] are:

  • Poor accessibility, quality, and comparability of cultural data (stemming from a decentralized infrastructure)
  • Norms about data collection and use, including low priority/importance assigned to the task of data collection in general
  • Lack of coordination and standardization among existing data collection efforts
  • Skill and resource capacity constraints among cultural nonprofits
  • Organizational culture dynamics that inhibit thoughtful decision-making
  • A paucity of vision and case studies regarding the successful use of data to drive decisions

To address these challenges, “New Directions” recommends coordinating leadership on cultural data, engaging program and artistic staff in conversations about data, shifting the frame from accountability to decision-making, developing a research and data collection agenda, developing data-related skills among organization staff, and improving the cultural data infrastructure.

What I think about it: While the work relies heavily on the impressions of a small number of experts, Lee and Linett make a number of good and important points. I particularly agree with the notion that what the world needs is not more data collection, but rather better skills and filters to apply to the data and research that’s already there. “New Directions” also calls out the “data first, research questions later” approach adopted by so many cultural institutions (including, arguably, the CDP itself) as an unhelpful norm, and makes veiled but unmistakable reference to the widespread practice of letting advocacy goals take priority over strong methodological standards. That said, the paper’s persistent focus on data rather than the broader concept of research ends up causing it to miss the forest for the trees in some respects. Despite acknowledging in places that new data collection isn’t always the most promising route to greater wisdom, it neglects to consider the role that literature review, calibrated probability assessment, and other approaches not involving primary data collection can play in informed decision-making. The paper focuses on challenges and strategies without the same level of attention to desired outcomes. Because of this, I felt excited by the direction “New Directions” was taking me, but frustrated that it didn’t map out more of the journey.

What it all means: Are we ready to declare a crisis in the field around its data collection practices? Are people who commission or carry out new research without thinking either about its direct tie to decision-making or its strategic place in the literature not just failing to add value, but doing the arts an active disservice? There’s an argument to be made that organizations shouldn’t be asked to collect any data at all, except insofar as it serves their strategic purposes and they know what they’re doing; that, instead, all data intended to serve the needs of the sector should be collected by knowledgeable third parties working under a clear and coordinated research agenda. While “New Directions” declines to do so for us, it’s interesting to imagine what a vastly improved cultural data infrastructure would actually look like, along with how we might get there.