The Case (Study) of Arsenic Life: How the Internet Can Make Science Better
At every step of the way, newer tools for conveying and analyzing science had better results than more traditional methods.
The microbe in question (NASA)
This weekend brought to a close the year-and-a-half-long arsenic-life saga. Or really I should say #arseniclife saga, as the Twitter hashtag that trailed every turn is itself a sort of metonym for the greater story -- not the story of a strange microbe that lived in Mono Lake, but of how science works, and sometimes doesn't, in these early times of online publishing, analysis, and, of course, chatter.
Here's how it all came to an end: On Sunday, while scientist Rosie Redfield spoke at the Joint Congress of Evolutionary Biology, two papers, one by her and her colleagues and another by a separate group, were published on the website of the journal Science. Together, the papers represent a summary refutation of the claims, first made in December of 2010, that the bacteria (GFAJ-1) could use arsenic to build its DNA, not phosphorous as is the case with all other life on Earth. As it turns out, GFAJ-1 is just like the rest of us -- phosphate-dependent -- and doesn't represent some strain of alien or separately evolved life here on Earth, as the existence of arsenic-based life implies. Though the papers were not set to be published until later this month, Science released them early -- during Redfield's talk -- in a move that surprised even Redfield herself, and garnered a cheer from the live audience in Ottawa.
This is all as it should be, right? As Redfield told the Washington Post's Marc Kaufman, "A very flawed paper was published and received an inordinate amount of publicity. ... Now refutations of the work by two independent research groups are appearing in the same high-profile journal, and the refutations are being well publicized. This is how science is supposed to work."
But there's something lacking in this. All is not well just because it has ended well. Perhaps that's true for the scientific core of the story, the narrow question of whether GFAJ-1 depends on phosphate. But the bigger story -- how the original, shoddy paper was hyped by NASA and Science beyond responsible levels; how the authors of that paper handled the criticism; right up until how Science kept the new papers and their authors under lock and key until last night, even while Redfield's paper was already available on arXiv.org; and, notably, how the open fora of science blogs and Twitter provided a platform for public analysis and scrutiny -- this story demonstrates the large problems with the process of how science is packaged and delivered, and how the tools of the web could be deployed to make that process better, smarter, and more efficient.
The problems showed their fanged teeth right from the get-go, on November 29, 2010, with the news arriving in the form of a NASA press release. Okay, okay I jest. No news really arrived in the press release. Rather, NASA announced that they *would be announcing later* "an astrobiology finding that will impact the search for evidence of extraterrestrial life."
This, predictably, had the entire Internet dying with anticipation. Could there be a more tantalizing press release? Jason Kottke asked, "Has NASA discovered extraterrestrial life?" Gawker followed suit: "Did NASA discover life on one of Saturn's moons?" Alexis Madrigal jumped in and ruined all the speculation-fun, tweeting, "I'm sad to quell some of the @kottke-induced excitement about possible extraterrestrial life. I've seen the Science paper. It's not that." But that wasn't anywhere near the end of it: Four days later -- once the paper was released, embargo lifted, science revealed -- the news of arsenic-based life (even arsenic-based life here on Earth) was still pretty dope. As Phil Plait of Bad Astronomy wrote, "First off, just to be straight and to dispel the rumors: this is not aliens on Titan, or Mars, or anywhere else. This bizarre life form was found right here on good ol' Earth. And don't be disappointed: this is still pretty cool news." Life, whatever that magical quality is, was more diverse, resilient, and possible than we had ever previously had reason to believe.
But pretty soon even the Earth-based claims began to fall apart, most prominently and thoroughly on Rosie Redfield's blog, RRResearch, where she wrote, NASA's shameful analysis of the alleged bacteria in the Mars meteorite made me very suspicious of their microbiology, an attitude that's only strengthened by my reading of this paper. Basically, it doesn't present ANY convincing evidence that arsenic has been incorporated into DNA (or any other biological molecule)." And then she proceeded to eviscerate the paper's claims. Six months later, she, along with seven others, published in Science their criticisms of the study, and the study's authors, led by Felisa Wolfe-Simon, responded at length. As Carl Zimmer wrote in Slate at the time:
For scientists who only get their information in print, this may be the first they've heard anything about a longstanding controversy that's come to be known over the past six months by its Twitter hashtag: #arseniclife.
For those of us who have been tracking #arseniclife since last Thanksgiving, however, today comes as an anticlimax. There's not much in the letters to Science that we haven't read before. In the past, scientists might have kept their thoughts to themselves, waiting for journals to decide when and how they could debate the merits of a study. But this time, they started talking right away, airing their criticisms on the Internet.
Now today, more than a year later, the fruits of some of those criticisms have been published on the Science website, pushed out from under their own embargo by Redfield's talk.
At every single step of the way in this long tale, there is a tension between the archaic, rule-bound process preferred by NASA and Science and the free-for-all, rapid-consumption ethos of science online. And at each step, the science and surrounding reporting coming from the latter looks smarter, cooler-headed, and more solid than that emanating from the older organs.
Begin with the press release which ramped up expectations beyond where the science could go, look at Alexis's tweet, which tried to bring them back down (and which teetered on the edge of acceptable embargo behavior). The whole process makes little sense: As EmbargoWatch's Ivan Oransky wrote at the time, "If the goal is to communicate the science accurately, and there's a lot of allegedly inaccurate coverage floating around out there about a study that is already peer-reviewed and proofed, and available in PDF form to reporters, can someone please explain why the best thing to do is to wait until Thursday to release the actual study?"
Then go from the press conference to the initial fall-out. Points go to Redfield and other scientists who debunked the study on their blogs and Twitter streams. Wolfe-Simon and her team, for their part, refused to wade in (that is, any more than they already had by, you know, publishing the paper). "Any discourse will have to be peer-reviewed in the same manner as our paper was, and go through a vetting process so that all discussion is properly moderated," Wolfe-Simon wrote to Zimmer. "The items you are presenting do not represent the proper way to engage in a scientific discourse and we will not respond in this manner." [Emphasis added]
Finally, take a look at the most recent incident, when Redfield had to walk a delicate line during her talk on this topic, not to *break an embargo of a paper that was already freely available on arXiv.org.* She ruminated on her blog in advance of her talk about the instructions Science had sent her:
Science asks authors to not initiate contact with the press about their publication, and to only talk to members of the press who have agreed to respect the embargo. Authors are free to present their data at conferences, but are asked to inform Science of this in advance.
All this seems a bit silly when applied to research results that have already been widely publicized, with the manuscript publicly available on the arXiv server (it's also on PubMed Central but not released yet). So I emailed Science for clarification. The response asked for what seem to be slightly tighter restrictions (to not mention that the paper is in press at Science, to not talk to the press after my presentation). These seem inappropriate, since this is a public-outreach talk and since a major focus of my presentation will be on how science is communicated.
Miraculously, (or, as Zimmer said, "weirdly,") midway through Redfield's talk, Science changed its mind, posting both Redfield's team's paper and another a half-hour into her talk, though they did so in a haphazard way, alerting only reporters who had specifically requested a heads-up about this story, not everyone who was under the embargo. Regardless of the details, Oransky said "Science did the right thing by releasing the paper early." (Though, it should be noted, that paper is still not universally available. Unless you have institutional or personal access to the pages of Science, let me direct you to ArXiv.)
And, finally, in this comparison of traditional science publishing and newer avenues, no greater stain exists than the mere fact of the study's publication that holy of holies, the peer-reviewed pages of Science. One source told Zimmer early on: "This paper should not have been published." The paper has not been retracted, just refuted, but there is no question that the peer reviewing of the web worked better than the closed peer reviewing of a journal -- that the vetting that should have happened before publication happened after. As Matthew Battles put it, "The scientists' onstage tensions, like the grumblings of colleagues elsewhere, are entirely healthy expressions of the human reality of science--but perhaps a NASA press conference was not their ideal setting. Results like Wolfe-Simon's would be better presented in a journal committed to open access. In future, NASA would do well to let the process play out before calling for the biology textbooks to be rewritten."
So much for "the proper way to engage in a scientific discourse."
Perhaps this is clear, but the reason to have "proper" methods of engagement is because they ostensibly will produce better science and better science journalism. But in this one case study, we can see how the opposite is true: The "proper" paths of engagement produced uninformed hype, poor science, and kept the sources -- both human and paper -- away from a conversation that was simmering with genuine enthusiasm and curiosity. The best science -- and the best science writing -- could come when we allow those natural levels of interest to have a field day with the research and researchers that are out there. The curious may never be satisfied, but at least they will have some science to sink their teeth into, rather than the vapors emitting from some press releases, press conferences, and papers whose authors shy away from the conversation.
Jonathan Eisen of UC Davis and open-science advocate put it to me this way: "If we think about what science is supposed to be, it's supposed to be about discovery and testing of hypotheses and ideas to explain the world around ... There is nothing in that system that says that it only works in this system that we have of peer reviewed journals. It is true that the system we have has done a decent job for years. And there is no doubt in my mind that the web, social media, and other novel forms of communication can enrich science."
We are still only at the very leading edge of what the new world of science could look like. But the case of arsenic life gives us some clues, and some reason to look forward, not withstanding any hype.
More From The Atlantic
