Recurring Decimals…..

Everything here is irrelevant

Opening up peer-review

with 2 comments

Wired magazine has an article on how the traditional ‘peer-review’ process for publishing scholarly scientific article is undergoing a gradual change, both through pressure from as well as to maximize the benefits of a digitally connected age.

“Peer review was brilliant when distribution was a problem and you had to be selective about what you could publish,” says Chris Surridge, managing editor of the online interdisciplinary journal PLoS ONE. But the Web has remapped the universe of scientific publishing – and as a result, peer review may finally get fixed.

The proof: In June, Nature began experimenting with a new method online. Authors submitting papers can choose a two-track process. While the work goes through the usual peer review drill, a preprint version gets posted on the Web. Anyone – even you – can comment, as long as you attach your name, affiliation, and email address. As of July, 25 articles had undergone this process, and the journal plans to issue a report late this year on how the test went. (Full disclosure: Wired editor in chief Chris Anderson participated in the project.) “The whole point of peer review is to help the editors select papers that are going to move science forward,” says Linda Miller, US executive editor of Nature and the Nature research journals (Nature Biotechnology, Nature Genetics, et cetera). “If there’s a better way, then why not? How could I say no?”

In other quarters, traditional peer review has already been abandoned. Physicists and mathematicians today mainly communicate via a Web site called arXiv. (The X is supposed to be the Greek letter chi; it’s pronounced “archive.” If you were a physicist, you’d find that hilarious.) Since 1991, arXiv has been allowing researchers to post prepublication papers for their colleagues to read. The online journal Biology Direct publishes any article for which the author can find three members of its editorial board to write reviews. (The journal also posts the reviews – author names attached.) And when PLoS ONE launches later this year, the papers on its site will have been evaluated only for technical merit – do the work right and acceptance is guaranteed. “Data becomes useful only if it’s shared,” Surridge says. “At the moment, our mechanisms for sharing information are the traditional journals, and if they’re hard to get into, data is completely lost.”

All these are certainly exciting new ways of sharing your data. At the same time, posting your data online in fact is really not very different from presenting your work at conferences in forms of seminars or posters, with the added advantage of being cost-effective from time and money point of view. Additionally, you are getting a much wider audience than you would at a conference and you may get to pick the brains of scientists from remote corners of the world. This could also potentially lead to better chances of discovering collaborators and fewer incidences of different groups repeating the same work, thereby optimizing research output.

Unfortunately, as the article points out, old habits are difficult to change. The major problem with this system will be the question of how to evaluate the value of a publication. An academic scientist is pretty much judged by his/her publication record – both quality and quantity. As any graduate student or post-doc knows, a first author Nature, Science or Cell paper is pretty much a sure-shot ticket to tenure-track position. The reason for that is the exclusiveness of the journals. An open review may be viewed as a dilution of the value of the final product.

But seriously: Who cares? An up-and-coming researcher can get more attention from the right experts by publishing something earthshaking on arXiv than by pushing it through the usual channels. Crazy ideas will get batted around in moderated forums, which is pretty much what the Internet is for. Eventually, printed journal articles will be quaint artifacts. Scientific papers will be living documents with data published on Web pages – commented on, linked to, and mirrored by labs doing the same work 6,000 miles away. Every research effort will have thousands of reviewers working in real time. Today’s undergrads have never thought about the world any differently – they’ve never functioned without IM and Wikipedia and arXiv, and they’re going to demand different kinds of review for different kinds of papers. It’s in their nature.

Eventually, I think the article is a bit too optimistic about the future of peer-reiew. But then ten years ago, I never really thought that going to the library to get a copy of a journal paper would become an antiquated practice (you get almost all papers in pdf format from journal websites nowadays) .

Written by BongoP'o'ndit

August 29, 2006 at 3:22 pm

Posted in Science

2 Responses

Subscribe to comments with RSS.

  1. These are interesting times, and there’s been a lot of good debate on peer review reform. It will happen, because the need for it is increasing…..

    good thing, this.


    August 30, 2006 at 8:47 pm

  2. interesting post — even 2.5 years on. it came up as an automatic link in my blog.

    you note: “The major problem with this system will be the question of how to evaluate the value of a publication.”

    i started discussing this questions here:

    comments welcome.

    –nikolaus kriegeskorte


    February 12, 2009 at 8:49 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: