So, Tom Garvey’s takedown of Emily Glassberg Sands’s undergraduate thesis on sexism in theater is pretty much a must-read.
Now the ultra-articulate Sands had been in high gear from the very start of the conversation, but as I got closer to my concerns, she began to power-chatter at a nearly alarming rate. I kept trying to steer the conversation to what I thought should be the central question of her final chapter – were those closed shows actually more profitable than the male-written shows she was comparing them to? (Because profitability, or the lack thereof, is the reason producers close shows.) But every time I tried to phrase this question, Emily deflected it by claiming that I wouldn’t understand her even if she explained her method, that I wasn’t trained enough in statistics to comprehend what she was doing, etc., etc. Finally, I managed to blurt out the full question:
“Emily – do you or do you not have valid data on profitability?”
And suddenly the chatter ended abruptly; a long silence ensued. And ensued. For what seemed like minutes. “Emily? Emily, are you there?” I asked. “Yes, I’m still here,” she answered quietly, but said nothing more. “Can you answer my question?” I then ventured. “I’m still here,” she repeated. And again fell absolutely silent.
It gets more uncomfortable from there, in case you’re wondering. For those of you new to this, here’s the backstory: Emily Glassberg Sands, a May 2009 graduate of Princeton College, recently released the results of her undergraduate economics thesis, a study attempting to identify and quantify gender bias against female playwrights. Due to its topical content and a connection to Freakonomics author Steven D. Levitt through playwright Julia Jordan, her thesis has received quite a bit more attention than most of her peers’ do; for her part, Sands is starting a Ph.D. at Harvard in the fall.
I haven’t yet read Sands’s report, which can be downloaded here, and I’m not sure when I’ll have time to; I already have a bunch of studies on my plate for the Arts Policy Library, and clocking in at 173 dense pages, Opening the Curtain would not be an insignifcant addition to that pile. In the meantime, though, the back-and-forth about the study is piquing my interest quite a bit.
Garvey’s analysis is compelling because, from what I can tell, he is the only journalist who has actually taken the time to dig in to the study and try to understand its underlying methodology. (Frankly, he may well be the only journalist who’s actually read the whole thing.) Not having read the report myself, I can’t vouch for his analysis of her analysis, but he does seem to be asking the right questions. His investigation demonstrates the unfortunate truth about research studies: they’re a lot more subjective than we might like to believe. Everything from the way that the data is collected to the way it is analyzed to the way it is presented can color what the public eventually sees. And all too often, only the researcher is ever in a position to examine the process from start to finish. This is why the Givewell project is so valuable — it takes the time to separate meaningful, rigorous studies from those that merely claim to be. And that’s very much a part of what I hope to do with the Arts Policy Library.
On the other hand, Garvey’s post (which goes on to tell a story about Sands and her thesis that, while plausible, is almost completely speculative) and very aggressive questioning of Sands can be read in other ways, not least through a gender bias lens. He veers between alternately complimenting her on the rigor of the study and assailing her intellectual honesty, ultimately accusing her of perpetrating fraud. (It’s important to note that even if his “gotcha” is valid, it only pertains to one component of Sands’s multi-pronged analysis – and not the part that is getting talked about the most). What I find most interesting to consider, though, is the possibility that this is just a case of “we find what we expect to find.” After all, it’s perhaps not surprising that Garvey discovered things to be skeptical about in the study, considering that he admitted before reading it that he is “hostile” to the notion of sexism against female playwrights because he “often see[s] inferior plays produced by women (Sarah Ruhl, Lynn Nottage, Lydia R. Diamond) who seem to be favored by either the academy or the arbiters of political correctness.” Just as it’s not surprising that Jodi Schoenbrun Carter was so impressed with Sands’s work that she immediately volunteered to help her spread the word as widely as possible, when just two weeks before she had pledged to use her blog to discuss “the on-going struggle [she and her female] peers go through in both the commercial and nonprofit world.”
It’s hard for everyone to escape this bias, the ultimate bias — that our expectations influence our perceptions — and researchers are no exception. The only real way to fight it, as I learned in an ethics class this spring that was steeped in behavioral psychology research, is to ask oneself at every turn, “how could I be wrong?” It is only through naming our assumptions that we destroy their power to lead us astray.