Saturday, July 07, 2007

Referee reports in Nature Precedings ?

I was having a look at some of the bioinformatics manuscripts available in Precedings and I come upon this paper on "The Reproducibility of Lists of Differentially Expressed Genes in Microarray Studies". After the figures there is a letter to the editor with the response to the questions from the referees.

I could not find the paper published in a peer-reviewed journal and I wonder if this was intentional of maybe part of an (opt-in and maybe buggy) automatic procedure from Nature to have submitted papers appear in Precedings. If I was an editor of a bioinformatics/genomics journal I could now consider if this paper with these referee reports would be interesting to the journal and send an email to authors suggesting that if by some chance their paper gets reject my journal would be willing to publish it.

Deepak was recently saying that it would be good to have access to this type of information. Why was a paper rejected from some journal and published in another? Most manuscripts go through several editorial and referee evaluations before getting published. Biology Direct and now PLoS ONE (too some extent) capture this information. I have found that many times it is useful to read the referee comments in Biology Direct because it provides with several independent criticism that makes it easier to home in on good and bad parts of the work.

I am sure that this has come up before in the context of ArXive but wouldn't it be more efficient to have journal editors somehow fish out from a common pool what is more interesting and hight impact to their community instead of the current submission ladder that i assume a lot of people go through ? We would submit to a preprint server and tag the paper according to perceived audience (i.e cell biology, bioinformatics, etc). Editors would flag their interest for the paper and the authors would select one of the journals. You can imagine some of the dynamics that this could create with some journals only looking at manuscripts that have already been flagged by some other journals, etc.

The referee reports would be attached the paper and the editor would make a decision. If rejected the paper would be up again for editorial selection but with the previous information attached. Other journals could just decide to publish with those referee comments.

I think this is not far from what already happens within publishing houses. Referee reports can be passed around to other journals of the same publisher. This would make it more general. Although there are clear advantages to authors (fewer rounds of refereeing and quicker publishing), it would be hard to convince most publishers to such a scheme. For those publishing mostly journals with low rejection rates it would be beneficial since most likely the papers have been already refereed, but for those with high rejection rates it could feel like they would be giving away their work for free. Since it is really the work of the referees maybe it should be up to the referees to decide if the reports can be made public or not, period.