“Old World Inspirational Sign,” by Flickr user Nicholas Raymond

It’s no secret that embarking on a professional career in the arts requires a degree of boldness in the face of economic uncertainty. The prevailing stereotype of the “starving artist” indicates that people do so anyway – either willing to forego comfort for creativity’s sake, possessing alternative income (such as a day job or family help), and/or confident that their talent and drive will see them through. But how long do they stick with it before throwing in the towel – or professionally shifting gears? As the Greek playwright Aeschylus wrote: “Necessity is stronger far than art.”

A study from 2016 assesses the odds of artistic longevity through the prism of academia: does a formal education enhance one’s chances of making it (and staying) in the arts? Using data gathered by Statistics Denmark, “Artistic education matters: survival in the arts occupations” analyzes more than 27,000 employment records between 1996 and 2012 across five categories of Danish artists: visual artists, choreographers and dancers, composers and musicians, film/stage actors and directors, and writers (including journalists).

Authors Trine Bille and Søren Jensen estimate the impact of a formal artistic education on the length of artists’ careers in each of these groups. (The definitions of “relevant education” and “relevant industry” for each arts group are specified in an appendix to the report.) Among their key findings:

• Formal education reduces the rate of attrition (i.e., abandoning an arts career) for musicians, actors and writers.
• The same correlation is not seen among visual artists and dancers, though these samples are smaller.
• Exit rates – especially early in a career – vary between artists in different fields.

Methods of Inquiry

The Bille/Jensen report leverages Statistics Denmark, an agency that collects a remarkable census of all Danish citizens covering employment, income, industry, age, and gender, among many other topics. Made possible via a personal identification number associated with every Danish citizen, this is arguably the most robust longitudinal dataset we’ve ever encountered at Createquity. For its purposes, “Artistic Education Matters” homes in on people ages 18 to 70 (excluding pensioners) who have a positive income primarily generated by work in one of the five defined arts categories over the 17-year time frame of the study. Exit rates are marked by a ‘‘definitive exit from the artistic labor market’’ – a shift in occupation that continues throughout the observed period without a return to arts employment.

Via a literature review, the authors point to several hurdles in the arts labor market, including an excess supply of artists, paltry average income, skewed income distribution, and a low overall survival rate, with just 20% of the subjects remaining in their fields after ten years. “Compared to other fields of employment,” they write, “the arts seem to be a risky business.”

Bille and Jensen’s conclusions focus on artistic survival rates – the odds of remaining in an arts profession – more than income levels. Indeed, they cite a host of cultural and economic literature indicating that artistic education does not have a significant impact on income, noting that many artists are self-taught and theorizing that “indefinable” factors – such as talent, creativity, and ambition – may contribute more to higher rates of payment than formal training does. (Despite this conclusion, Bille and Jensen perform their own analysis of income levels in the dataset and find that, at least for Danish musicians, actors, and writers, a relevant education actually does positively affect earnings. Income from self-employment is not included in the study, though, so we should take those results with a grain of salt.) While acknowledging that “higher income makes [artists] better able to live from their arts,” “Artistic Education Matters” concerns itself mainly with education’s effect on longevity – not financial success per se – in a chosen arts field.

Bille and Jensen employ the Cox model of regression analysis to to investigate what factors predict longevity in the marketplace as an artist, controlling for income level, relevant experience (i.e., working in the field in which the artists studied), other experience, additional employment, and demographic variables such as gender and age.

Motley Crews

Overall, across the five artist groups studied, only about 20% remain in their chosen occupation after 10 years. The report makes clear, though, that the impact of an arts degree varies considerably by discipline. Below we discuss which groups benefit most from a relevant education, from most to least:

• Writers: the authors note that “it is not possible to separate authors from journalists” in the dataset, and this is the largest group, with 14,943 subjects. About 20% have a relevant education: for them, the exit rate after five years is 20%, compared to more than 60% for those without a writing degree. Bille and Jensen note that journalism in particular “functions more like an ordinary labor market, where an education may have an important signaling effect to employers.”
• Actors: the sample of 3,813 “film, stage, and related actors and directors” shows this group to be most vulnerable to an early exit. While Bille and Jensen note that “an actor can get a job without any formal education or other experience,” 16% of the actors had in fact formally studied their craft. At the five-year point, only 45% of those with a relevant education had left the field, compared to 75% of those without.
• Musicians: the sample of 3,161 “composers, musicians, and singers” shows 34% with a music degree; among this group, 55% left after five years, compared to 70% for those without. The authors note relatively high barriers for entry in this field: “technical skills may have an impact on the survival.”
• Dancers: The group of “choreographers and dancers” has only 296 subjects; just 9% of them have formal dance training, and this is said to have “no impact on staying in the profession” (the five-year exit rate difference is less than five percentage points). However, the authors note the small dataset: “The problem is that there are very few observations for this group.”
• Visual Artists: Among 4,851 “sculptors, painters, and related artists” (including workers commercial fields like advertising) the authors report that just 2% have a formal arts education, “which means that most of these visual artists were autodidacts.” Based on this small sample, the formal education “seems to have no impact” on an early exit.

Bille and Jensen also cite data from a 2005 French study – Gender differentiated effect of time in performing arts professions (Coulangeon et al), which investigated survival function estimates for musicians, actors, and dancers in France from the mid-1980s to 2000. Both studies indicate that the most vulnerable period of attrition for all artist groups is in the two years following entry to the labor market; the Danish study, with a wider dataset, shows an even more dramatic early exit rate than the French study.

What Else Might Be Going On?

An obvious and tempting conclusion to draw from these results is that, yes, an arts degree is valuable, at least if one works in music, theater, or literature and if one’s goal is to stay in the profession for as long as possible. Yet for all its specificity, this analysis leaves several key questions unanswered and hypotheses unexplored. Among them are:

• Are these results simply an indication that people who bother to get a formal education in the arts are more committed to their chosen art form in the first place? Could there be a sunk-cost effect where people feel like they ought to stay in an arts field longer if they invested substantial time and resources getting trained in it? Especially since, for most, that probably also meant not getting a degree in some other field?
• Are the conclusions about visual artists and dancers – that formal education has little impact on longevity – simply a reflection of limited data for those groups or is there something more to this than first-glance results?
• Is it possible that people who survive the longest in the arts have alternative income streams (such as family help) that wouldn’t be reflected in this dataset?
• What is the relationship between survival in the arts and financial success? The authors note that a relevant education correlates with higher salaries for three of the arts groups – musicians, actors, and writers – and these are the same groups in which education is seen to increase longevity.
• What role does self-employment play in all this? Unfortunately, the authors chose not to include income from self-employment in the analysis due to challenges with the dataset, which could have skewed the results in unpredictable ways.
• Would we see these same types of results looking at similar data from a different country and/or a different time frame?

The Bille/Jensen study would have benefitted from a closer examination of these hypotheses as they relate to the Danish dataset. With its literature review and emphasis on longevity, this report adds a helpful lens to research gathered by Createquity last year in “The BFA’s Dance with Inequality” – which cast some doubt on the importance of arts degrees for lower-income students – as well as our 2016 article “Who Can Afford to Be a Starving Artist?” But because the study is light on analysis of socioeconomic factors and personal wealth – particularly the role that secondary income support plays in extending an artistic career – it doesn’t illuminate its subject as much as it could.

Then there are socio-political aspects such as government support for arts education: Denmark has a robust social safety net compared to, say, the United States, which may play into arts longevity. Getting a formal arts education in Denmark – where university education is free and living costs are buttressed by a system of grants – does not entail the equity barriers found in countries with high, generally unsubsidized university costs (especially at the graduate level).

Still, with an extraordinarily comprehensive national dataset, thoughtful analysis, and pinpointed conclusions, Bille and Jensen make a strong case for the connection between formal art education and longevity in an artistic career, especially for those working in music, theater/film, and literature. This should be good news to arts school students and grads who hope to spend their lives doing nothing else.

Demonstrators protest in front of the White House in support of Net Neutrality | Photo by Joseph Gruber via Creative Commons

Just days past his confirmation, Ajit Pai, the Trump administration’s pick for Federal Communications Commission chairman, is already rolling back regulations put in place by the Obama administration in 2015 to protect net neutrality and increase access to the Internet. Changes that have already been enacted include the removal of nine companies from the Lifeline subsidy program, former chairman Tom Wheeler’s initiative which reduced the cost of broadband access for low-income families; the FCC also put a stop to data-security rules enacted in October. These actions signal a rapid-fire change in the FCC’s direction and portend new battles over Internet access. Pai has yet to lay out a specific plan to reverse the FCC’s classification of broadband internet as a utility like electricity or water – one of the landmark decisions under Wheeler’s tenure – but he’s made clear that he sees that move as a “mistake” that has depressed growth in new broadband investment. Some critics consider the loss of a free, open, and affordable Internet one of the biggest potential threats to the arts, favoring corporate interests at best, with the looming possibility of censorship at worst.

Brits attempt to impose quality standards on art. Arts Council England has earmarked £2.7 million to implement Quality Metrics, a controversial process aimed at measuring the quality of art presented to the public by government grantees. Drawn from a series of evaluations by peers, audiences, and the grantees themselves, the system seeks to measure artistic quality across various art forms and types of arts organizations, and will be mandatory for all organizations receiving at least £250,000 per year in operating support from the Arts Council. The plan is set to roll out despite many concerns raised following an independent review of the pilot phase of the program, particularly regarding the use of a single set of metrics across a plethora of artistic disciplines and questions regarding feasibility, data ownership, and anonymity. Buy-in from artists has been equally lukewarm, with many expressing resistance to the very idea of quantifying the arts.

It’s getting even harder to make it in Hollywood. A recent episode of the NPR podcast Freakonomics examined America’s ailing visual effects industry, which has endured economic troubles as jobs continue to migrate out of Hollywood. Despite visual effects playing an increasingly large role in filmmaking (and obliterating trades like special effects in the process), multiple companies in the industry remain in dire economic straits. Their job attrition likely stems from producers and directors chasing tax rebates in neighboring states, and increasingly abroad, forcing many film jobs out of California and hastening the globalization of the industry. At least one Hollywood profession may be getting some help: the Los Angeles City Attorney brought charges last month against five casting workshops accused of using a pay-to-play scheme trading acting roles for cash. In announcing the charges following an investigation involving an undercover actor, the city cited the Krekorian Talent Scam Prevention Act, which bars casting agents from requiring actors to pay fees for auditions.

Libraries grapple between access and ownership. In an era of inevitable change for public libraries, some are relaxing or even doing away entirely with overdue fines, questioning whether the penalties ultimately hurt Americans who need libraries the most. The decision stands in stark contrast to recent crackdowns on overdue books in Alabama and Texas, in which authorities threatened delinquent borrowers with jail time in an effort to recover hundreds of thousands, even millions, of dollars in lost property. The US is not alone; in the UK, more than 25 million books are lost and unaccounted for in that country’s libraries according to industry sources. So, while releasing borrowers from fines may remove the economic barrier and increase libraries’ appeal for marginalized communities, it also inevitably means fewer titles to chose from.

Federal arts funding hangs in the balance. Arts organizations are gearing up for battle as the Trump administration continues to toy with the idea of cutting arts agencies such as the National Endowment for the Arts in first drafts of the federal budget. While these cuts have not yet been formally instigated, their possibility has spurred activists to flood congressional offices in opposition. Much attention is focusing on the small but politically significant cadre of Republican arts champions, including New Jersey congressman Leonard Lance and senators Shelley Moore Capito and Susan Collins, both of whom signed a letter of support organized by fellow senator Kirsten Gillibrand. The ramifications of losing these agencies would be most deeply felt in rural areas, which receive less support from state and municipal arts funding. Despite a gradual uptick in appropriations to state agencies dating from the recession, the biggest gains of recent years have been concentrated in populous states like Florida and California, while it’s one step forward two steps back in places like Iowa. Bigger cities may have the best chance for surviving a wholesale cut to the arts: Atlanta and New York are among those plotting ways to increase support at the local level by proposing dedicated arts and culture taxes, providing incentives to artists who live in particular cities or states, and bolstering public art programming.

MUSICAL CHAIRS / COOL JOBS:

• Director Craig Watson of the California Arts Council will step down from his role with the agency effective April 2017.
• Philadelphia’s William Penn Foundation has named Shawn McCaney as its new executive director. McCaney was previously director of Penn’s Creative Communities program.
• The Wallace Foundation named Giselle “Gigi” Antoni as its new director of learning and enrichment. Antoni had developed a national reputation as the leader of Dallas’s Big Thought arts education initiative.
• The Alaska-based Rasmuson Foundation has announced Alexandra McKay as its new vice president of programs.
• Seattle arts critic Jen Graves voluntarily resigned after more than a decade at The Stranger, stating that it’s “not a viable place for me to do the work I’ve always cared about.”
• Perhaps less voluntarily, the outspoken New York Times theater critic Charles Isherwood is looking for work. Despite the implosion of jobs in arts criticism, the Times intends to fill the full-time position.
• DataArts is seeking a senior research manager to lead teams in study design, data analysis and interpretation and the delivery of the organization’s research services.
• The Boston-based Barr Foundation is hiring an arts & creativity program officer.

NEW RESEARCH OF NOTE:

• The National Endowment for the Arts shared its latest installment of data on economic trends in arts and culture, produced in collaboration with the Bureau of Economic Analysis. Upshot: artists add value to the economy, but public funding for arts education is in a sharp decline.
• Out of 1,000 responses to a survey by the UK’s Guardian Teacher Network, 80% claimed their schools made or are planning to make cuts to the arts.
• New evidence suggests that formal artistic education (i.e. conservatory training) can have a positive impact on artists’ career sustainability, as can financial and business training. Of course, one must be able to afford such training; indeed, the Sutton Trust noted that British actors from wealthy backgrounds are more likely successful than those with modest upbringings. Meanwhile, college-educated white adults make more than college-educated black and Latino adults according to Demos and the Institute on Assets and Social Policy, challenging the assumption that higher education can neutralize racial wage gaps.
• Exponent Philanthropy reported that small foundations and individual donors are developing strategies to up their impact potential in grantmaking. But larger funders tend to rely on their peers as the most trusted source of knowledge, according to a Hewlett Foundation report.
• An evaluation of Arts Council England’s Catalyst program indicates it provided a significant kick-start needed to increase the fundraising capacity of grantees.
• Museums and other cultural attractions continue to face challenges. A new metric indicates that visitor confidence to US cultural organizations is experiencing a sharp decline. However, a recent review of the literature indicates that museums are developing new fundraising strategies by looking beyond wealthy socialites as sources of individual donor support. Meanwhile, the American Alliance of Museums, as it does each year, published its TrendsWatch report considering what the future might hold for the industry.
• A new Ipsos survey asked Canadians across the county to define their culture and its products in the digital age.
• A new study identified hip and arm movement as the mark of good dancing in women. A rebuttal from Slate’s Daniel Engber, however, questions the relevance of the study, deeming it science’s version of clickbait.
• Grantmakers in the Arts produced a review of the literature regarding the arts in medicine, with a specific focus on optimizing investments.
• Research from the University of Chicago indicates that it’s easier to have a negative attitude then to look on the bright side.
• New research suggests that, compared to other teens, ballet dancers experience greater rates of “psychological inflexibility,” leading to anxiety and depression. Dancing may contribute to a greater fear of failure and pressure to achieve a physical aesthetic, which may also lead to such symptoms.
• Violent video games are thought to be associated with negative behaviors. Could uplifting games elicit the opposite effect? A UNESCO-sponsored study indicates they could.
• The Kennedy Center has partnered with the National Institutes of Health to create Sound Health, an initiative that explores music’s effects on health and wellness. Fast Company interviewed author Daniel Levitin about his new book on a similar topic: the neuroscience of music, and how playing music at home impacts behavior, attention span and productivity. Levitin’s work indicates that music is no longer as prevalent in the home, perhaps due to increased screen time, and could be used to facilitate mental breaks from focused tasks. His findings contrast evidence that positions silence (differentiated from quiet, or ambient noise) as an underutilized productivity tool.
• An annual report on freedom of expression around the world released by Freemuse finds that violations of artists’ rights more than doubled in 2016.
• The University of Wisconsin found that people of color accounted for 22% of children’s books characters in 2016, a 13-percentage-point increase over the course of two decades.
• Despite the success of high-profile female artists like Adele and Beyoncé, women are, on the whole, seriously underrepresented on the top 40 charts.
• #OscarsSoYoung could be the latest hashtag criticizing the Academy Awards. A new report from USC’s Annenberg School for Communication and Journalism indicates that there were only two characters over 60 nominated over the past three years…and both were played by Michael Keaton. USC researchers also found that women directors working on the top-grossing films were unlikely to have released more than one film in the last decade.

In a previous Createquity Research Spotlight, we wrote about the “What Works” initiative in the UK, which borrows a policy evaluation methodology from the medical community and applies it to (among other things) the arts. In this case, HULA is developing metrics to evaluate arts and humanities using the discipline’s own tools.

HULA does this by repositioning the humanities as “an assemblage of craft practices,” whereby each craft embodies distinctive goals, logical methods,and results that are passed from master to apprentice over thousands of years. For example, in the craft of pottery-making, we could attribute a set of tools, techniques, and sequencing – a general sense of purposeful and procedural logic – that all contribute toward creating a beautiful or useful product. By organizing humanities as individual practices and crafts, we can start to identify different steps, logical patterns, and tools that each activity utilizes to produce an outcome.

In this research context, human development itself is the desired outcome. The HULA white paper thus explores three key research questions:

1. How do the humanities contribute to human development?
2. To what extent are the humanities effective in achieving this goal?
3. How can we measure the effects of the humanities anyway?

HULA defines “human development” in terms of education, subdividing that definition in two ways: 1) education as a system of institutions, maintained by the state to serve utilitarian purposes (such as cultivating civic service or civic responsibility); and 2) education as an individual act of instruction, which relates to personal development. In short, the teaching of humanities is likened to the practice of “crafts,” which also help advance education and contribute to human development.

To explore how different humanities practices contribute to these educational outcomes, HULA identifies various elements – or building blocks – that make up different humanities practices. All of these elements are then sorted and coded according to HULA’s proposed methodological framework; this allows us to see which elements are different or common to various humanities practices, and thus track how they lead to similar or dissimilar learning pathways toward the goal of human development.

In Figure 1 below, HULA defines four sequential stages of a learning pathway and attributes possible elements of a given craft to certain learning processes or outcomes: input, processing, and short-term and long-term results. In this example, the red arrows represent the pathway of a political philosophy instructor who engages students in close readings (verbal input) and logical debate (cognitive-analytical analysis), with the aim to encourage understanding of political concepts (the short-term goal), which in turn may enable students to become more civic-minded citizens (the long-term goal).

Figure 1. Example of a HULA Learning Pathway. Adapted from “Humanities and Liberal Arts Assessment White Paper”, by The HULA Research Team. 2015, p.15.

HULA’s application of coding schemes to different humanities activities requires a set of assumptions about each humanities practice or craft, including the elements it comprises, how it works, what it is trying to achieve, and which skills it develops. These assumptions draw on the implicit logic associated with the practice or craft. Once these elements are broken down, the study assigns the process and outcome advanced by each element. The elements are thus coded categorically along a structured framework of possible learning pathways, which allows us to track the progress and outcome of a given humanities practice or craft. To see how this coding process works in detail, the HULA study on 30 Years of Illinois Humanities Council Grant-Making features appendices of the “code structure” and “code sets” used to track the different methods and mechanisms followed by various humanities activities.

Does this sound complicated? Indeed, applying this methodology, with its myriad definitions and categorizations, requires users to absorb concepts that can be rather academic and abstract. This barrier could limit the adoption of HULA’s methodology beyond the realm of the academy.

Still, the research logic of the study is worth a deeper dive. The core concept of its design makes sense, especially when the terms of its application are simplified. Ultimately, by coding and analyzing humanities as a series of crafts – each of which has its own elemental purpose and logic –we have a new way of unpacking what each practice is really about, what elements it comprises, how it works, and towards which learning outcomes it steers.

If we can manage to make the language of the HULA model a bit more accessible, we just might have a promising methodology for assessing the value of the humanities – using evaluation tools drawn from the discipline itself, as opposed to metrics from other disciplines, which are often an imperfect fit.

Cover image: “Pottery” courtesy of Lennart Tange. via Flickr Creative Commons license.

My Console Collection by Flickr user Sarah

It’s widely reported that able-bodied young men, without college degrees are underemployed and unemployed in record numbers. Despite this hardship, one recent study has found that these young men are actually happier than their equivalents were 10 years ago. The source of their pleasure? Much of it may come from playing video games instead of working. The “real-world” jobs available to them do not provide the sense of achievement or community that can be found through gaming, so many of these young men are choosing to live at home, in a virtual reality (nearly three quarters of the drop in work hours for this group is accounted for by increased time spent playing video games). It seems like bad news, but perhaps the implications of this retreat from the workforce are not as dire as they seem: inventive researchers are working with gamers to find cures for disease.

Can the Quality of Art be Quantified? Arts Council England is betting on it. The government agency recently announced a plan to have all of its National Portfolio Organizations (NPO) that receive over £250k per year must adopt and adhere to the Quality Metrics program, a standardized measurement approach designed to consistently and meaningfully measure artistic quality. These grantees are required to participate in a number of annual evaluations and engage in regular peer review, regardless of art form and organizational structure. Despite significant concerns raised in a post-pilot evaluation of the platform, the program is moving forward – for now. The news has sparked quite a row from UK artists on Twitter, and even incoming ACE Chair Nicholas Serota has expressed skepticism. In other quantification news, a new algorithm predicts the likelihood that a book will become a bestseller and, thanks to Apple’s iBeacon, many of the world’s major museums are using big data in their attempts to improve their visitors’ experiences.

High Culture and Pop Culture Converge. BBC2 is dropping an unorthodox bomb in this autumn’s rating wars: high culture. The British television station will shelve its usual schedule of repeats, to air poetry, dance, and documentaries on Saturday evenings. This new focus on culture will feature contemporary programming rooted in traditional forms and narrative (for example, a twist on WH Auden and a production by a performance artist who merges hip-hop, poetry and theatre). Through the creation of a “cultural destination” for its viewers, BBC2 may well provide the UK’s artists and arts organizations with invaluable opportunities and exposure.

The Connected Future of Fine Art? We suppose it was only a matter of time before “hacking” would come for classical art forms.  In August, the Dutch National Ballet premiered Night Fall, a new ballet choreographed by Peter Leung – not for the stage, but for virtual reality (VR).  Viewers need only a VR-compatible device to experience the “goose bump-worthy” performance, the first of its kind, as technology enables the performers to embark on an instant global tour. Meanwhile, the Tate Britain launched the IK Prize-winning online initiative Recognition. The program employs artificial intelligence to match the Tate Britain’s iconic collection with photojournalism from the contemporary 24-hour news cycle. It is designed to provoke new questions about art and life.

Culture vs. Terrorism. In September, France’s President François Hollande stood in the Egyptian Galleries at New York City’s Metropolitan Museum of Art, and announced the formation of a $100 million fund to combat terrorist attacks on cultural sites in the Middle East. (He did not say how much his own government would be contributing to this “public-private partnership,” but did express hope that the Met’s donors would pitch in.) Hollande also referenced an upcoming (December 2016) conference hosted by the Louvre Abu Dhabi, which will focus on culture and terrorism. Although the preservation of cultural artifacts is integral to global human culture, it is interesting that France’s president advocated for the asylum of art works while its Prime Minister expressed reluctance to grant asylum to people.

MUSICAL CHAIRS / COOL JOBS

• Laura Sparks begins her term as the Cooper Union’s first female president in January.  Currently, she’s finishing her term as executive director of the William Penn Foundation; her replacement will be the foundation’s fifth head so far this decade.
• Heather Kim brings over 20 years of experience in higher education research to the Andrew W. Mellon Foundation, in the newly created role Director of Institutional Research.
• Sir Nicholas Serota will leave the Tate, after 28 years (!), to become the next chairman of Arts Council England. Will significant government cuts to the arts prove challenging for the “virtuoso fundraiser”?
• Just four months after being reappointed by David Cameron, BBC chair Rona Fairhead will step down. Her successor has not been named.
• The Center for Arts Education is hiring a Director of Advocacy and Engagement.
• The Center for Artistic Activism is hiring a part-time Non-Profit Manager.
• The New York Public Library’s Library of the Performing Arts is hiring a Deputy Director of Research and Collections Services.

NEW RESEARCH OF NOTE

• The National Endowment for the Arts and the Center for Cultural Innovation released a long-awaited report on trends and conditions affecting U.S. artists, an update of a major, decade-old study and a centerpiece of Chairman Jane Chu’s “Creativity Connects” program. Meanwhile, new arts data profiles published by the NEA offer state-by-state perspectives on Americans arts participation. The data highlights a north-south divide in American creativity, and reveals that the percentage of American adults who read literature fell to at least a three-decade low last year, after a “long, steady decline.”
• A Los Angeles County Arts Commission literature review on public engagement in the arts, and reports from the Paul Hamlyn Foundation’s Our Museum program provide resources on cultural equity and inclusion in museums and beyond.
• Research commissioned by the UK’s Association of Independent Museums, Arts Council England and the Welsh Government shows that introducing admissions fees does not affect diversity, but may cause attendance to fall.
• The latest annual report from the Strategic National Arts Alumni Project focuses on institutional connections, resources, and working across disciplines for arts alumni. And a recent study from artnet suggests that the institution from which an artist receives an MFA has implications for career “success.”
• The National Center for Arts Research released its most comprehensive report to date on national fundraising trends. Meanwhile, a new study published in the Public Performance and Management Review suggests that arts donors aren’t influenced by high attendance.
• Partners for Sacred Places has released the results of an evaluation of its pilot program to match artists to historic sacred spaces.
• John Sedgwick and Mike Pokorny’s research on financial risk in the film industry challenges conventional wisdom on the peripatetic nature of box office predictions. And new research from the Journal of Political Economy investigates movies as a shared experience. Unfortunately, the latest report from the University of California’s Annenberg School for Communication and Journalism reveals that Hollywood is all talk and no action when it comes to advancing diversity.
• A study published in the Journal of Experimental Psychology: Human Perception and Performance suggests a link between ballet and sensitivity to others. On the other hand, new research from Psychology, Public Policy, and Law reports that more people find lyrics threatening if they believe they are from a rap song, as opposed to a country ballad.
• Income inequality isn’t the only kind of inequality: using information from the National Center for Education Statistics, the New York Times reports that the educational inequality gap is narrowing for children entering kindergarten. And results of a new study published in the Review of Income and Wealth indicate that happiness inequality is on the decline.
• The Center for Effective Philanthropy released a comprehensive report on evaluation practices at foundations.
• A new study from the Indiana University-Purdue University Indianapolis – Center on Philanthropy explores thirty years of nonprofit research.
• Research from Australia’s Art Gallery of New South Wales found that viewing art relieves anxiety in dementia patients and helps them to “stay in the moment.”
• A new book from Eric Booth and Tricia Tunstall chronicles the growth of El Sistema-inspired music education programs around the world. Not everyone, however, is convinced.

The UK government has been asking just this question. Drawing from the experience of the medical community’s National Institute of Health and Care Excellence (NICE) model, which systematically assesses and synthesizes the cost-effectiveness of medical interventions to help the UK National Health Service (NHS) prioritize its public spending, the UK Cabinet office has envisioned its What Works initiative as a “NICE for social policy.”

Launched in March 2013, the What Works Network set out to evaluate the performance of public policies and programs using evidence collected throughout their implementation. For every policy or program, What Works tracks which social benefits that program has achieved and how much money those benefits cost per participant. What Works evaluations aim to help policymakers and practitioners improve their decision-making process by providing evidence and advising on which interventions offered the best value for money.

Generally speaking, the What Works methodology rates a policy or program on its effectiveness according to indicators that are specific to the sector in question. For example, the success of an education policy might be rated according to how many additional months of progress a student makes in a classroom. This rating becomes even more useful to practitioners when complemented by an estimate of how much money it costs per student to implement the policy.

Now in the Network’s third year of operation, seven “What Works Centres” and two affiliate members – What Works Scotland and the Public Policy Institute for Wales – have been established thus far across the UK, each focused on a particular area of policy. They monitor and evaluate interventions according to a standardized methodology in seven categories: educational achievement, local economic growth, crime reduction, health and social care, wellbeing, improved quality of life for older people, and early intervention for at-risk children. And yes, the What Works initiative is evaluating arts interventions within the broader context of these public policy areas.

For example, the What Works Centre for Well-Being analyzes the impacts of culture and sports on wellbeing according to the same four dimensions – satisfaction with life, happiness, worthwhileness, and anxiety – used by the National Statistics of the UK to assess personal wellbeing and subjective wellbeing. Similarly, the Sutton Trust and Education Endowment Foundation’s (EEF) Teaching and Learning Toolkit provides an easy-to-read ranking on the cost-effectiveness of arts participation for improving educational outcomes for students aged 5-16, relative to other interventions.

So what does What Works say about how the arts work? A report issued by the UK government in 2014 presented a selection of early findings from the six What Works Centres that had been active up to that date. Two projects relating to the arts were included. One project from the Centre for Local Economic Growth examined 36 evaluations covering the impact of major sport and culture projects on the local economy and found that the overall measurable impacts were rare, and small if they existed at all. Built facilities, however – with sporting facilities comprising the vast bulk of the evidence – might increase the value of properties in their immediate vicinity. The Sutton Trust-EEF Teaching and Learning Toolkit found that arts participation had a low positive impact on student education attainment (defined as “additional months progress you might expect pupils to make as a result of an approach being used in school, taking average pupil progress over a year as a benchmark”), but for much lower cost compared to some other learning interventions with similar impact, such as attending summer school or using teaching assistants. The Teaching & Learning Toolkit collects impact evidence from EEF projects covering 34 education topics, and impact results are regularly updated and summarized as they are collected.

What Works’s venture into evidence-based policy was still in its infancy at the time of the report’s publication. Since then, several new arts-related projects have been commissioned and evaluated. An “Act, Sing, Play” project sought to answer whether exposure to high quality music education was more cost-effective than drama participation for improving students’ literacy and math scores, and another “SHINE on Manchester” project assessed to what extent Saturday music education improved students’ literacy and math scores. While assessments of these two projects did not yield convincing evidence that participation in the arts helped achieve the designated outcomes of improved literacy and math scores, they also did not discount the possibility that arts participation might yield other positive outcomes.

Is evidence-based evaluation of public policy the wave of the future? In the United States, a loose alliance of several organizations would like to make it so. Results for America aims to spearhead smart policy changes at all government levels by encouraging the use of best available data, evidence and evaluation about what’s effective. The Laura and John Arnold Foundation (LJAF) is likewise active in this area, having recently absorbed the Coalition for Evidence-Based Policy, a Washington think tank that had success in advocating for government reforms, into its grantmaking. At this very moment, LJAF is holding a $15 million competition to encourage government and nonprofit organizations to implement highly effective programs, and Results for America just launched a global initiative “identifying the policies, programs and systems that governments are using to support the production and use of data and evidence” called Results for All.

In theory, this approach of collecting, synthesizing, and ranking evidence from a diverse range of policy and program evaluations will help make that evidence accessible to a wide audience – and that is undoubtedly a good thing. At the same time, paring down the impacts of policies and programs to cost-effectiveness might be challenging when goals are less readily quantifiable, or where effectiveness needs to be assessed according to more innovative or perhaps even abstract criteria. In such cases, less relevant targets might become more appealing to policymakers because they are cheaper or easier to tag with numbers, resulting in an oversimplified framework for measuring impact that displaces a true understanding of effectiveness. Arts and cultural policies arguably are particularly vulnerable to this risk, particularly given that we are only beginning to understand the true nature of their value to individuals and society. At Createquity, we don’t think it is impossible for the benefits of the arts to be assessed under a What-Works-style evaluation framework, but we do have to be careful that we are attempting to measure the right things – the things that arts are actually good for.

The relatively small and weak body of information and data on the impacts of arts/culture policies and programs shows that there are significant gaps and limitations – but also much room to grow – for What Works’s assessments of arts interventions going forward. In the meantime, we can do our part to contribute evidence to What Works inquiries by submitting tips, research and assessments of public policies to the relevant Centres. As of publication, EEF is seeking new education interventions to fund and evaluate, and welcomes applications from “projects that show promising evidence of having a measurable impact on attainment or a directly related outcome” until December 9.

Cover image: “Mathematica” courtesy of the Ivan T. via Flickr Creative Commons license.

“Atto primo ▼ Picnic” by BAAM Milano

Title: Culture Counts in Communities: A Framework for Measurement

Author(s): Maria Rosario Jackson and Joaquin Herranz

Publisher: The Urban Institute

Year: 2002

URL: http://www.urban.org/research/publication/culture-counts-communities

Topics: Cultural Indicators, Neighborhoods, Community, Cultural Vitality

Methods: Literature review, consultations (interviews and focus groups) with experts and community members

What it says: The Arts and Culture Indicators Project (ACIP) was launched in 1996. “Recognizing that arts and culture had too frequently been neglected in efforts to assess quality of life” the Rockefeller Foundation commissioned the Urban Institute to “explore the possibility of integrating arts and culture-related measures into neighborhood indicator systems.”

The researchers conducted interviews and focus groups with arts professionals and community residents, and reviewed the available literature. They found that there is neither much empirical data nor theoretical work on the ways in which arts and cultural participation contribute to social dynamics, and the data that is available primarily focuses on formal arts and culture venues. Since the existing research is insufficient as the basis for an indicator system, the authors propose principles and parameters for research and measurement that were developed through a series of workshops and conferences.

The authors present four “guiding principles” for indicator development in communities, which highlight the need to allow the communities that are being studied to define “arts,” “culture,” and “creativity” in ways that are appropriate for their community, the need to be open to broad definitions of participation and the multiplicity of meanings art can have simultaneously, and the fact that opportunities to participate require both arts and non-arts resources.

They go on to propose a conceptual framework that has four domains:

• presence (qualitative and quantitative cultural inventorying)
• participation
• impacts (contribution to community building outcomes)
• systems of support

Regarding the impact domain, the authors point out that the necessary fuzziness around the definition of arts, culture and creativity make it difficult to pinpoint their impact on community building outcomes.

While the data is considered inadequate to support any definitive conclusions, the authors identify “a list of important impacts that participation in arts, culture, and creativity at the neighborhood level may have” [emphasis added] based on their review of the literature. Directly or indirectly, the arts, culture, and creativity may contribute to

• supporting civic participation and social capital;
• catalyzing economic development;
• improving the built environment;
• promoting stewardship of place;
• augmenting public safety;
• preserving cultural heritage;
• bridging cultural/ethnic/racial boundaries;
• transmitting cultural values and history;
• creating group memory and group identity.

The authors acknowledge that their principles and conceptual framework are just a beginning and that further theoretical development and empirical research is necessary.

What I think about it: This is an important initiative/study that shifted the conversations about arts in communities by introducing expanded definitions of participation and including practitioners and community members in the research/definition of outcomes. It’s really at the beginning of the whole creative placemaking conversation.

Nonetheless, there are some weaknesses: The “Systems of Support” domain in the ACIP framework seems like a measure of outputs rather than outcomes. The same might be said about the mere “presence” of cultural opportunities. Moreover, in the discussion of potential impacts that the arts have on communities little attention is paid to the fact that arts and culture can also work in the opposite direction: they can increase cultural/ethnic/racial boundaries, obscure cultural values and history, etc.

What it all means: ACIP’s objective of including arts and culture in systems of quality of life indicators parallels our work on wellbeing at Createquity, but since they’re focused on neighborhoods it’s not clear that much of their work would carry over to assessments of wellbeing at the national level. For instance, ACIP’s desire to let the communities under investigation develop their own definitions of “arts,” “culture,” and “creativity,” and set their own indicators of cultural participation may make sense at the neighborhood level, but it would be extremely difficult to arrive at a national consensus on these matters through community consultations. Similarly, it is easier to imagine incorporating qualitative data that sheds light on local history and local cultural meanings in the context of neighborhoods than in national indicator systems.

Author(s): Sarah Lee and Peter Linett

Publisher: Cultural Data Project

Year: 2013

URL: http://www.culturaldata.org/wp-content/uploads/new-data-directions-for-the-cultural-landscape-a-report-by-slover-linett-audience-research-for-the-cultural-data-project_final.pdf

Topics: research, data

Methods: Theory/assertion, informed by synthesis of comments from a CDP-hosted online forum of researchers (disclosure: I was one of them), results from CDP’s internal strategic planning survey, and a paper by Margaret Wyszomirksi (not available online) “to frame and inventory the cultural data landscape.”

What it says: “New Directions” was commissioned by the Cultural Data Project in connection with that organization’s transition from a foundation-housed initiative to an independent nonprofit, and the strategic planning process that followed. The report was intended to inform that process by situating CDP’s efforts within the larger context of data collection throughout the United States cultural sector. It notes a growing abundance of and interest in arts and cultural data, but identifies six factors that “may be limiting the sector from effectively incorporating data into decision-making processes.” The six factors [paraphrased] are:

• Poor accessibility, quality, and comparability of cultural data (stemming from a decentralized infrastructure)
• Norms about data collection and use, including low priority/importance assigned to the task of data collection in general
• Lack of coordination and standardization among existing data collection efforts
• Skill and resource capacity constraints among cultural nonprofits
• Organizational culture dynamics that inhibit thoughtful decision-making
• A paucity of vision and case studies regarding the successful use of data to drive decisions

To address these challenges, “New Directions” recommends coordinating leadership on cultural data, engaging program and artistic staff in conversations about data, shifting the frame from accountability to decision-making, developing a research and data collection agenda, developing data-related skills among organization staff, and improving the cultural data infrastructure.

What I think about it: While the work relies heavily on the impressions of a small number of experts, Lee and Linett make a number of good and important points. I particularly agree with the notion that what the world needs is not more data collection, but rather better skills and filters to apply to the data and research that’s already there. “New Directions” also calls out the “data first, research questions later” approach adopted by so many cultural institutions (including, arguably, the CDP itself) as an unhelpful norm, and makes veiled but unmistakable reference to the widespread practice of letting advocacy goals take priority over strong methodological standards. That said, the paper’s persistent focus on data rather than the broader concept of research ends up causing it to miss the forest for the trees in some respects. Despite acknowledging in places that new data collection isn’t always the most promising route to greater wisdom, it neglects to consider the role that literature review, calibrated probability assessment, and other approaches not involving primary data collection can play in informed decision-making. The paper focuses on challenges and strategies without the same level of attention to desired outcomes. Because of this, I felt excited by the direction “New Directions” was taking me, but frustrated that it didn’t map out more of the journey.

What it all means: Are we ready to declare a crisis in the field around its data collection practices? Are people who commission or carry out new research without thinking either about its direct tie to decision-making or its strategic place in the literature not just failing to add value, but doing the arts an active disservice? There’s an argument to be made that organizations shouldn’t be asked to collect any data at all, except insofar as it serves their strategic purposes and they know what they’re doing; that, instead, all data intended to serve the needs of the sector should be collected by knowledgeable third parties working under a clear and coordinated research agenda. While “New Directions” declines to do so for us, it’s interesting to imagine what a vastly improved cultural data infrastructure would actually look like, along with how we might get there.

Photo by ed_needs_a_bicycle

(This is the third post in a series on Fractured Atlas’s capacity-building pilot initiative, Fractured Atlas as a Learning Organization. To read more about it, please check out Fractured Atlas as a Learning Organization: An Introduction.)

I don’t know about you, but I’ve always been a reluctant decision maker. When I go out to eat at a restaurant, I often drive my dinner mates crazy by asking the wait staff for recommendations and then just ordering what I was thinking about getting anyway. In high school, I used to agonize over what now seem like trivial choices like what topic I should choose for my English papers. Perhaps that’s why I’m so drawn to the science of decision-making. I’m somebody who likes options, who likes variety. As soon as I make a decision, I close myself off to a world of possibilities. So I want to be sure I’m making the right one!

As much as anything else, making the right decision has to do with accurately forecasting what will happen once you make it. The trouble is, a lot of times we are pretty uncertain about just what those consequences might be. In the previous post in this series, I talked about how we frequently use too narrow a range of possibilities when trying to make a prediction about something we’re not sure about. In part that’s because our society – and perhaps our brains – have trained us to think deterministically rather than probabilistically. What that means in plain English is that when we make a guess, we typically hone in on one option. If you’re a contestant on Jeopardy! and the category is Tom Cruise movies, you can’t chime in with “What is either Magnolia or Top Gun, I’m not sure?” You gotta pick one. And that’s true of decision making as well, isn’t it? In the end, there’s only so much you can hedge your bets. We only get one life to live (well, that we know about) and we are accustomed to choosing a single path from the many in front of us.

The thing is, though, if we want to make smarter decisions, we need to train ourselves to be a little more comfortable with ambiguity. Sure, that course you took last fall seems like a slam-dunk right choice in retrospect, but did it look that way before you decided to sign up? What if it hadn’t worked out the way you’d hoped? The funny thing about decisions is that sometimes you can make the right call and still have things turn out badly – or vice versa. Improving the decision-making process is a long game – it banks on the idea that, over time, and across many decisions, you’re going to come out ahead for approaching each one thoughtfully. And thoughtful decision making involves defining the assumptions and potential consequences of your decision carefully and considering the full range of possibilities associated with each.

Forecasters have long been using a tool called Monte Carlo simulations to help us do just that. The Monte Carlo method was invented by a physicist named Stanislaw Ulam and was originally used to help design atomic bombs. (The codename “Monte Carlo” comes from the legendary casino in Monaco, where Ulam’s uncle used to gamble.) More recently, the method, which typically involves running thousands (or sometimes tens or hundreds of thousands) of simulations on a model and then aggregating the results, has shown up in everything from computational biology to corporate finance. Data scientist and former New York Times blogger Nate Silver, who famously predicted the 2008 and 2012 presidential election results with near-perfect accuracy, relied on a sophisticated Monte Carlo model to do so.

Using Monte Carlo methods within a decision-making context enables us to make predictions about the consequences of those decisions, and thus better understand the opportunities and risks associated with them. At Fractured Atlas, where we’ve set up an internal team to experiment with this sort of decision modeling, we’ve begun to hone the process for bringing quantitative definition to our decision dilemmas and making predictive estimates. Below, I’ll lay out the steps we currently use to do this.

1. Define what your dilemma actually is. For many people, this exercise is fairly intuitive, but sometimes it takes a little bit of work to get your decision to a state where it can actually be modeled. Something like “what should I do with my life?” is not really at that place yet. But let’s say you’re deciding whether or not to move your office to a bigger space. That’s a very clear, specific dilemma that a process like this one can help resolve (with a bit of elbow grease).
2. Articulate what excites you, and conversely, what worries you about this decision. If we had perfect information all the time about what was going to happen in the future, decision making would never be a source of stress. In fact, it would be very boring! What causes us to worry about making the wrong decision is that hard decisions always carry some sort of risk, opportunity, or combination of the two. Translating your emotions about the decision into words will help you understand the criteria by which the decision will be judged successful or not, and what factors could affect the outcome.
3. Define the “central question” for the decision and identify the relevant variables affecting it. This central question needs to be understood in quantitative terms and usually boils down to some sort of cost-benefit equation: for example, will the amount of money I earn from the points on this credit card exceed the annual fee? Alternatively, the central question might weigh two options against each other: does a change in the business model for my program yield better results than the status quo? Once you know what your central question is and have identified the elements of the decision that you care about most, the next step is to define variables – things like the number of people attending your event, the expected cost of a new software feature, and the like. The variables should collectively work together to create an equation that ultimately answers the central question.
4. Estimate values and distributions for the variables. For each of the variables in the model, we need to give our model some sense of what the numbers might be. For example, you might be 90% confident that the proportion of folks who will buy a book after they hear you speak is somewhere between one-tenth and one-third. This is where the calibration training mentioned in the previous post comes in handy. If we’ve learned anything about estimating ranges, it’s that we need to make sure to make them wide enough!
5. Run the Monte Carlo simulation. We set up Excel to run 10,000 individual scenarios, each one a plausible real-life outcome according to the assumptions we’ve set up in our model. The model aggregates the results of each of those to tell us, out of those 10,000, what percentage ended up answering the central question a certain way. And now we have a prediction of how our decision is going to turn out!

I imagine this must all seem kind of abstract, so let’s run through the process again with a concrete example. Let’s say you run a rental space and you’re deciding whether or not to extend your hours later into the evening. You’re excited because a lot of customers have been requesting to rehearse at those hours, and there’s a reasonable expectation that the change is going to bring in more revenue. On the other hand, you’re worried that the fixed costs of staffing the space during the late-night hours, whether it’s being used or not, will outpace any incremental dollars you might bring in. The central question is one of simple costs and benefits: is extending the hours going to be worth it in the end?

Defining the incremental costs of the change should be fairly simple. Most of it will have to do with labor. If you pay your people hourly, it’s as simple as the additional hours you’ll have to pay out due to the extended schedule. You may want to account for the possibility that you’ll need to hire an extra person to handle the load; in that case the drain on your time to find that person should be considered as an upfront cost. If you want to get a little fancier, there may also be increased utility costs (electricity, etc.) that you’ll incur as a result of the increased usage of the space, as well as increased wear-and-tear that could be expressed as faster depreciation.

Defining the benefit side of the equation happens much the same way. Do you have only one rental rate for all your customers? Then it’s just a matter of estimating how many new customer-hours will be enabled by the schedule extension and calculating how much they will pay for those hours. Things get a little more interesting if you have lots of different rates for different customers and uses – e.g., rehearsal vs. recording, non-profit vs. for-profit, etc. In that case, you’ll want to segment those customers out by their different properties, and calculate how much new revenue is generated by each type.

(click to enlarge)

Ultimately, we end up with a set of estimates for the potential costs and benefits of following through with your decision. The Monte Carlo simulation can then calculate, out of those tens of thousands of scenarios, how often the decision turned out to be the right one. Basically, it’s offering you a recommendation on which path to choose – kind of like the waiter who confidently tells you to get the saltimbocca. Just don’t be like me and order the chicken anyway!

If you’d like to see the formulas and play with the numbers yourself in the example above, you can download a simple version of it here.

“I want change” by m.a.r.c.

Toward the end of the summer, bioethicist Peter Singer raised the hackles of art lovers everywhere with a New York Times op-ed that considered a hypothetical dilemma: should you donate to a charity that combats blindness in the developing world or should you spend that money instead on an art museum? After running through a cost-benefit analysis of each option, he determined that the charity addressing blindness “offers [donors] at least 10 times the value” of the museum.

Ouch.

To no one’s surprise, the arts community didn’t exactly roll out the welcome mat for the piece, calling Singer’s argument “a shocker,” “absurd,” and “tyrannical.” Another round of alarm ensued recently when none other than megaphilanthropist Bill Gates threw his support behind Singer’s thesis. The responses from our field to date have generally coalesced around two broad counter-arguments:

•  Why does it have to be “either/or”? Why can’t we support both? Singer forces a false choice in “assuming charitable giving is a zero sum game.” Weighing the value of saving a life against the value of donating to an art museum is comparing apples to oranges when “both are essential, and if either disappeared you’d be in bad shape.” We need a holistic approach to ensure we don’t “solv[e] Third World crises at the expense of fostering crises right here at home.” Just as we have “multiple passions in [our] lives,” donors can and should target multiple causes and direct their charitable dollars in a “proportionally prioritized” manner. Anyway, we can’t really be sure than curing blindness is more important than inspiring the next Jackson Pollock, and even if we were, concentrating all our resources with one or two tried and true nonprofits runs counter to the “messiness and power of America’s [decentralized] approach to charity.”

• Saving lives is all fine and good – but only if those lives have meaning. If we’re so concerned with making sure that people can see, shouldn’t we also try to make sure they have beautiful things to look at? Singer’s logic is dangerous because he fails to acknowledge the “creative outlet[s] and emotional oas[e]s that only art museum[s] can provide.” If all philanthropic dollars were channeled toward alleviating disease and poverty, arts and culture would languish, society would become monochromatic and dull, and life would cease to be worth living.

As satisfying as these rebuttals may feel to arts advocates, they unfortunately miss the point. The crucial assumptions behind Singer’s argument are that

1. “there are objective reasons for thinking we may be able to do more good in one [sector] than in another,” and
2. we have a moral obligation to make choices that do as much good as possible.

It’s important to understand this perspective in the context of “effective altruism,” a relatively nascent but growing area of applied ethics that has been featured more than once on this blog, not to mention a recent edition of This American Life. Besides Gates, fellow philanthropic heavyweight and past Hewlett Foundation President Paul Brest has declared himself a fan. “Effective altruists,” or EAs, are on a quest to “do good” by way of hard-nosed rationality. “Doing good” doesn’t mean recycling a little more, or occasionally doling out spare change to a beggar on the street. It doesn’t mean foregoing a high-powered corporate career to work for a nonprofit. It means taking the time to analyze how to do the most amount of good possible with the resources available – or, to use a more nerdy turn of phrase, to “[use] science and rational decision-making to help as many sentient beings” as they can.

Most funders are already in search of a big “bang for your buck,” but in trying to identify the objectively best causes to support, effective altruists stray from the conventional wisdom of mainstream philanthropy. EAs cast a global net when determining where to focus, and often settle on supporting causes in faraway parts of the world, the results of which they may never see in person. They also believe that while human lives are created equal, philanthropic causes are not. Those causes that can save or improve the most lives must take first priority.

How does this play out in practice? Let’s say you donate to the free medical clinic in your area. You do this for good reasons: you care about inequities in the American healthcare system, and want to give back to your community. You like the feeling you get when you walk by that clinic every day. Maybe you even know people who benefit from the services the clinic provides. The clinic gets its donation, and you get warm fuzzies. Everybody wins. Right?

Not so, an EA would counter. Despite your good intentions, your donation amounts to a near-waste of resources:

We understand the sentiment that ‘charity starts at home,’ and we used to agree with it, until we learned just how different U.S. charity is from charity aimed at the poorest people in the world. Helping people in the U.S. usually involves tackling extremely complex, poorly understood problems… In the poorest parts of the world, people suffer from very different problems…

We estimate that it costs [Givewell’s] top-rated international charity less than $2,500 to save a human life… Compare that with even the best U.S. programs… over $10,000 per child served, and their impact is encouraging but not overwhelming.

EAs advocate making evidence-based decisions even if they don’t resonate on an emotional or intuitive level:

Effective altruism is consistent with believing that giving benefits the giver, but it’s not consistent with making this the driving goal of giving. Effective altruists often take pride in their willingness to give (either time or money) based on arguments that others might find too intellectual or abstract, and their refusal to give suboptimally even when a pitch is emotionally compelling. The primary/driving goal is to help others, not to feel good about oneself.

If this approach leaves you with an empty feeling in the back of your throat, it is by design. “Opportunity costs” – the costs of choosing not to behave in a certain way – weigh heavily on EAs. Every time you make a donation, considering where your money could have gone is as important as considering where it will ultimately go (emphasis mine):

In the “Buy A Brushstroke” campaign, eleven thousand British donors gave a total of £550,000 to keep the famous painting “Blue Rigi” in a UK museum. If they had given that £550,000 to buy better sanitation systems in African villages instead, the latest statistics suggest it would have saved the lives of about one thousand two hundred people from disease…  Most of those 11,000 donors genuinely wanted to help people … But these people didn’t have the proper mental habits to realize that was the choice before them, and so a beautiful painting remains in a British museum and somewhere in the Third World a thousand people are dead.

Weighing choices isn’t limited to how we spend our money – it also applies to how we spend our time. Just as EAs dispute the notion that people should support whichever charities they feel “passionate” about, they question whether channeling those passions into a nonprofit or medical career is the best way to make a difference. Many suggest instead that people “earn to give,” saying they “might be better off…in a high-earning job and making a deliberate commitment to give a large portion of what [they] earn away.“ The organization 80,000 Hours, founded to “become the world’s number one source for advice on pursuing a career that truly makes a difference in an effective way,” elaborates,

Working at a non-profit can be a great way to make a difference. But it’s no guarantee. Amazingly, lots of non-profits probably have no impact. And do workers at [a] non-profit have more impact than the people who fund them? The researchers who push forward progress? The entrepreneurs who transform the economy? Policy makers? Maybe. No one stops to ask.

Putting ideas like these on the table is a great way to make those of us in the arts squirm. While there are echoes of the effective altruism movement in some recent trends within our field, like the “universal call” for better data on the impact of the arts and the pointed questions about who ultimately benefits from arts funding, the arts are chock-full of people – artists and arts administrators alike – who were drawn to their work by that same passion that EAs claim clouds our judgment. The idea of allowing cold rationality to dictate and limit our quest to “do good” flies in the face of our artistic sensibilities, and challenges the assumptions many of us made when we entered the nonprofit sector in the first place – even those of us who have a sincere desire to address social inequities.

Tempting as it may be, it would be short-sighted to dismiss the EA movement as the pet project of a bunch of aesthetically stunted curmudgeons. It’s hard to dispute the notion that we could improve the human condition if only we could get our act together and commit our resources to a data-driven approach. After all, the nonprofit darling of the moment, collective impact, is based on the same premise. What effective altruism does is counter our cause-specific argument for the arts with a dizzying moral appeal for cause agnosticism. And to be honest, it’s hard to see how the arts win if they play the game by the EAs’ rules. The “both/and” argument mentioned previously is unlikely to sway an effective altruist who weighs each decision as a choice between two different futures, one in which a museum gets funded and some lives get saved and one in which the museum struggles and more lives get saved. Even if the museum shut down completely, its patrons could probably find or create an alternative “creative outlet and emotional oasis,” while the people dying of malaria can’t very well make the mosquito nets themselves. The “we give lives meaning” argument likewise rings hollow when we’re talking about lending privileged lives (anyone living on more than $2 a day is privileged in a global context) a dose of incremental “meaning” at the expense of giving others a shot at basic survival. It also comes across as incredibly condescending to those others considering that they would likely never get the opportunity to visit or benefit from Singer’s hypothetical museum. In any case, art is hardly the only possible delivery mechanism for meaning. In the words of one effective altruist,

Trying to maximize the good I accomplish with both my hours and my dollars is an intellectually engaging challenge. It makes my life feel more meaningful and more important. It’s a way of trying to have an impact and significance beyond my daily experience. In other words, it meets the sort of non-material needs that many people have.

Whether the EA movement sputters or gathers steam, taking the time to engage with its principles, even critically, is a healthy exercise. The bottom line is that EAs may actually be onto something when they argue it’s possible to make a bigger dent in one sector than another. Rather than insisting otherwise or dodging the argument altogether, we could heed the call to examine how altruism really manifests in our work, particularly when examined through the lens of what benefits the people we engage, rather than what benefits our organizations or our donors. Might we, too, have objective reasons for thinking we may be able to do more “good” in one program, or with one population, than in another? Do we, too, have a moral obligation to maximize that good? How would that change how we operate and who we serve? Do we want to change how we operate?

If the effective altruism debate makes anything clear, it’s that to be able to make art, not to mention argue about it, is to be fortunate. Taking a hard look at our assumptions about what draws and keeps us to this work may not be easy, but if we squirm a little, so be it. In the grand scheme of things, a little squirming is a luxury too.

If you’ve been paying any attention at all to technology trends the past few years, you know that we live in the era of Big Data. All of those videos we upload to YouTube, hard drives we fill with government secrets (or cat photos, take your pick), and tweets we awkwardly punch out on touchscreen keyboards add up to a whole lot of gigabytes, the bulk of which are stored by someone, somewhere, indefinitely. By some estimates, human beings generate more data every two days than we did in the entire history of civilization prior to 2003 – and that was as of three years ago!

Indeed, these are exciting times for data nerds, and data nerds in the arts are no exception. Initiatives such as the Cultural Data Project, Southern Methodist University’s National Center for Arts Research, and the Americans for the Arts National Arts Index seek to collect or organize relevant indicators pertaining to everything from arts organizations’ financial health to audience reach and characteristics to long-term trends for musical instrument purchases.

Fractured Atlas is no stranger to data initiatives in the arts. Our Archipelago data visualization software is one of the largest such efforts, bringing together information on arts nonprofits, for-profits, fiscally sponsored projects, funding, audience distributions, and community context all in one place in the service of better understanding the arts ecosystem in a region. Facilitating data-driven decisions is a major long-term objective of Artful.ly, our just-launched cloud-based arts management tool, and a present-day reality for Spaces, our venue listing and booking service that can promote spaces with last-minute availability to users. Through our research advisory services work, we’ve helped funders such as ArtsWave organize their entire grantmaking process around principles of data-driven decision-making in order to further their philanthropic objectives. Everyone benefits when funders, organizations and individuals in the arts ecosystem make thoughtful decisions about resource allocation, setting up and responding to incentives, and more. At Fractured Atlas, we believe that data can and should be a crucial input into that thoughtful decision-making process, and we’ve been increasingly vocal in evangelizing for data-driven decision making throughout the arts and cultural sector.

There’s just one problem. Up until now, Fractured Atlas has not had any formal guidelines in place to ensure that we use data in our own decision making, with the result that our internal decisions – relating to management, marketing, strategy, and the like – have been guided primarily by managerial intuition. In a “doctor, heal thyself!” moment, we’ve agreed that is time for our practices to reflect our preaching, at both the program and institutional levels. In 2013, the scope of our operations, the size of the community we serve, and the financial stakes in our work demand informed analysis at a level of rigor that we have not historically practiced. (This directive was immortalized by our fearless leader Adam Huttler in the organization’s annual Strategic Priorities Memo with the colorful title, “Eating Our Data-Driven Dog Food.”)

So between now and next summer, Fractured Atlas is embarking on a pilot initiative to explore how we can use data and evidence to improve our decision-making process at all levels. We’re calling it Fractured Atlas as a Learning Organization, and through this and future blog posts, we’re giving you the opportunity to be a fly on the wall as use this process as a way of grappling with issues of organization identity, strategy, culture, and impact.

What Is a Learning Organization?

As I define it*, a learning organization is one for which information and strategy are joined at the hip. It is, quite literally, an organization that has successfully forged a culture of learning and integrated that culture into its decision-making process at all levels.

Why is this integration between information and strategy important? Because every organization operates in an environment of uncertainty about what is going to result from its decisions, and every decision we make on behalf of an organization is based on a prediction, whether explicitly articulated or not, about the results of that decision.

If you can reduce the uncertainty associated with your decisions, the chances that you will make the right decision will increase. Of particular interest here  are what I call decisions of consequence: dilemmas for which the consequences of making the wrong decision and uncertainty about the nature of the right decision are both high. So, how do you reduce that uncertainty? Why, through research, of course! Studying what has happened in the past can inform what is likely to happen in the future. Studying what has happened in other contexts can inform what is likely to happen in your context. And studying what is happening now can tell you whether your assumptions seem spot on or off by a mile.

In fact, I subscribe to the notion that research is only valuable insofar as it helps to answer a question that matters. I’m not the only one who thinks so, either: Jake Porway, the founder of a nonprofit that connects data scientists with social enterprises in need, wrote this past spring that “any data scientist worth their salary will tell you that you should start [a data project] with a question, NOT the data.” In fact, all of the excitement around Big Data notwithstanding, data divorced from strategy is not likely to be very useful.

A learning organization solves this problem by forging a powerful feedback loop between information and strategy, with each feeding the other and adapting in relation to the other. The more obvious implication of this symbiosis is that organizational decisions must adapt in response to new information, as discussed above. But the less obvious implication is no less important: information-gathering must be directed by the organization’s decision-making needs. Without that intimate connection, there are no real safeguards to prevent organizations from thinking they are making data-driven decisions without really putting much thought into either the data or the decisions.

More broadly, a learning organization develops a culture of seeking out and using information thoughtfully from the highest levels to the organization’s grassroots. The most effective organizations are conscious about the impact they are trying to achieve, and willing to be open-minded regarding the paths they take to maximizing that impact.

*Some readers may be familiar with the term “learning organization” as defined by MIT management scientist Peter Senge in his well-known 1990 book The Fifth Discipline. My use of the phrase is broadly in the same spirit as Senge’s, but he sets out a very specific formula for what constitutes a learning organization that I don’t make use of here.

Fractured Atlas as a Learning Organization

This fiscal year, which started in September and goes through next summer, we are undertaking a pilot project to put some of these principles into practice. The primary goal of the pilot is to develop a conceptual framework and a toolkit of situation-adaptable methods for reducing uncertainty about decisions of consequence. If we can reduce the uncertainty we have about those decisions through strategic measurement and information-gathering efforts, over time we’re likely to make better decisions that will in turn lead to better outcomes for Fractured Atlas and the people who benefit from our work.

As powerful as this idea is, it only works if we have a very concrete sense of what we’re trying to accomplish as an organization. While we’ve had a mission statement for some time now, the huge variety of programs and services Fractured Atlas offers is virtually impossible to fully capture in a single sentence. Accordingly, the first step in this process is to create a theory of change for every program at the organization, from which we’ll roll up an overall theory of change and logic model for the organization as a whole. This will allow us to define our overall goals as well as some key success metrics at various levels of operation, taking into account both Fractured Atlas’s mission objectives and its focus on developing programs that are sustainable with earned income.

Meanwhile, we’ve formed an internal task force to work on this project at a deeper level of engagement throughout the year. Affectionately called the Data-Driven D.O.G. Force (the “D.O.G.” stands for Data Over Gut), the group will meet every 6-8 weeks to receive calibrated probability assessment training, identify real-world decisions of consequence to use as case studies, and come up with measurement experiments to gather information relevant to those decisions. In doing so, we’ll be using a modified version of a methodology called Applied Information Economics as described in Douglas W. Hubbard’s book How to Measure Anything: Finding the Value of “Intangibles” in Business. One major advantage of AIE is that it explicitly takes into account the cost-benefit of measurement strategies by calculating something called the value of information, which we’ll be exploring further in a future post.

At the end, we’ll attempt to formalize a process for identifying decisions of consequence in the future and fitting measurement strategies to the situation at hand. We’ll also present some recommendations for building infrastructure in the form of ongoing data collection, to address those questions that are likely to be asked again and again. And through it all, I’ll be writing about it here – so that anyone who wants to can learn alongside us.

Learning in Context: Why Philosophy Matters as Much as Performance

Data-driven decision-making isn’t just about crunching numbers. It’s a practice that requires certain values in order to work. The hard part of being data-driven is not the “data” but the “driven” – you have to be willing to question your assumptions and actually change your behavior in response to the new information coming in. Put another way, a learning organization is, well, open to learning new things -even things that suggest that the way that we’re currently doing things isn’t working as well as it could, or that we’re missing important opportunities to increase our impact.

It’s much easier to attain that kind of open stance if we train ourselves to expect failure upfront. In general, organizations as well as people have a tendency to be far too risk-averse. Being a learning organization means embracing a culture of intentional experimentation and productive failure: we’re likely not going to hit upon the secret sauce the very first time we try something – or, sometimes, at all.

Being a learning organization similarly requires that we think about ourselves from a system perspective – how are we making a difference in light of what everyone else is doing? And how can our experiences shed light on those of others? That’s why we’re not just going down this path on our own and in private. If the specific activities of the pilot project turn out to be a big waste of time (and I can’t guarantee that they won’t), we won’t be able to hide that from you or the world. But even that would ultimately be a good thing – because, in true learning organization fashion, it would cause us to reconsider the limitations of a data-driven approach. Embracing change is hard, but one of the very best things about it is that it can allow us to extract just as much (if not more) value from failure as success.

For me, personally, this project is very exciting. Of course I’m eager to find out what we’ll learn. But more than that, Fractured Atlas as a Learning Organization is an opportunity for us to exercise leadership in a way that reaffirms our highest standards for ourselves and for the field. I’m looking forward to sharing our journey with you.