Friday, March 27, 2015

Scientific Reports partners with Research Square to unbundle peer-review

Apparently, Scientific Reports has sent out emails to their editorial board members about an upcoming trial for a new peer-review track. The email is available this link for now. According to the email: "a selection of authors submitting a biology manuscript to Scientific Reports will be able to opt-in to a fast-track peer-review service at an additional cost"

They claim that this service will speed up the response time and they commit to have the editorial decision and peer-review comments in 3 weeks from submission. To do this, they will partner with Research Square that offers a third-party peer-review system called Rubriq. This service has been previously covered in the news by The Economist and Nature. Rubriq appears to be essentially a web-based reviewing platform. Scientists can register on the system and get matched to submitted articles. Reviewing is paid on a per-article basis. The process is described well in their website and they have online an example report.

Some of the reactions to this partnership have been negative. See for example this twitter thread. One of the negative comments appears to be that Scientific Reports is trying to sell a fast track on top of their existing peer-review track. I honestly think this was just a bad PR move and the wrong focus on their email to editors. I assume Scientific Reports is working like PLOS ONE with many academic editors and academic reviewers that are not paid at all. They still need some editorial staff to make sure the papers move along the process which is what probably costs them money. Rubriq is currently charging $500-$650 per article although I assume they might have some cheaper deal with Scientific Reports for this trial. If this trial works out I can imagine that Scientific Reports could cut down on costs per paper significantly but in the long run unbundled peer review would probably actually hurt them.  If peer-reviewing is external and editorial decisions are based on scientific soundness then the journal becomes just a branded specialized blogging platform.

With the bad PR paid "fast-track" notion out of the way I think that the best discussion is really about the merit of unbundling peer review. What I don't like about it is that it sounds like Amazon Turk for peer review. I don't review articles because I get paid to do it. However, if I needed to make a living out of it I would not reject so many requests as I do now. I think Rubriq is currently paying $100 per referee report which, for now, is just an extra incentive. This extra incentive is apparently still very important has Rubriq found out when running a survey. If we imagine this sort of marketplace scaling up we would need to have some assurance that professional reviewing was up to some required standard. How do we define and evaluate these standards is really worth thinking about.

The positive aspects of third party peer review should be clear to anyone that has gone through the process. There is so much time wasted from having the same paper re-submitted to several different journals and getting reviewed by a different set of reviewers each time. Having the evaluation of the soundness and merit of research separate from dissemination would be a clear innovation in the scientific process. This would also make the publishing costs more transparent and probably would result in lower prices.

So, overall I think it is great the Scientific Reports is doing this trial. Many people have talked about third party peer review and paid peer review. We do want more transparency about the costs of publishing. Maybe it turns out that Amazon Turk for peer review is a bad model but if we don't try new things we won’t find out.

P.S - Dear NPG PR people, feel free to use the tittle in this post when you announce this partnership and put less of an emphasis on speed.  You’re welcome.