Scientific Reports partners with Research Square to unbundle peer-review

Apparently, Scientific Reports has sent out emails to their editorial board members about an upcoming trial for a new peer-review track. The email is available this link for now. According to the email: "a selection of authors submitting a biology manuscript to Scientific Reports will be able to opt-in to a fast-track peer-review service at an additional cost"

They claim that this service will speed up the response time and they commit to have the editorial decision and peer-review comments in 3 weeks from submission. To do this, they will partner with Research Square that offers a third-party peer-review system called Rubriq. This service has been previously covered in the news by The Economist and Nature. Rubriq appears to be essentially a web-based reviewing platform. Scientists can register on the system and get matched to submitted articles. Reviewing is paid on a per-article basis. The process is described well in their website and they have online an example report.

Some of the reactions to this partnership have been negative. See for example this twitter thread. One of the negative comments appears to be that Scientific Reports is trying to sell a fast track on top of their existing peer-review track. I honestly think this was just a bad PR move and the wrong focus on their email to editors. I assume Scientific Reports is working like PLOS ONE with many academic editors and academic reviewers that are not paid at all. They still need some editorial staff to make sure the papers move along the process which is what probably costs them money. Rubriq is currently charging $500-$650 per article although I assume they might have some cheaper deal with Scientific Reports for this trial. If this trial works out I can imagine that Scientific Reports could cut down on costs per paper significantly but in the long run unbundled peer review would probably actually hurt them.  If peer-reviewing is external and editorial decisions are based on scientific soundness then the journal becomes just a branded specialized blogging platform.

With the bad PR paid "fast-track" notion out of the way I think that the best discussion is really about the merit of unbundling peer review. What I don't like about it is that it sounds like Amazon Turk for peer review. I don't review articles because I get paid to do it. However, if I needed to make a living out of it I would not reject so many requests as I do now. I think Rubriq is currently paying $100 per referee report which, for now, is just an extra incentive. This extra incentive is apparently still very important has Rubriq found out when running a survey. If we imagine this sort of marketplace scaling up we would need to have some assurance that professional reviewing was up to some required standard. How do we define and evaluate these standards is really worth thinking about.

The positive aspects of third party peer review should be clear to anyone that has gone through the process. There is so much time wasted from having the same paper re-submitted to several different journals and getting reviewed by a different set of reviewers each time. Having the evaluation of the soundness and merit of research separate from dissemination would be a clear innovation in the scientific process. This would also make the publishing costs more transparent and probably would result in lower prices.

So, overall I think it is great the Scientific Reports is doing this trial. Many people have talked about third party peer review and paid peer review. We do want more transparency about the costs of publishing. Maybe it turns out that Amazon Turk for peer review is a bad model but if we don't try new things we won’t find out.

P.S - Dear NPG PR people, feel free to use the tittle in this post when you announce this partnership and put less of an emphasis on speed.  You’re welcome.

A Borg moment and the end of Friendfeed

Apparently Facebook finally decided to shutdown Friendfeed after several years of declining usage. I only found out because Neil, Deepak  and Cameron wrote posts about this. Although I was a heavy user I ended up moving with the crowd after the Facebook acquisition. For those that never used it but are familiar with Twitter or Facebook it might be hard to understand why some people like myself are so disappointed with it's decline. Friendfeed was simply leaps ahead of anything at the time as a mechanism for sharing information and organizing discussions around these shared items. In fact, although there has been no further development for 5 years it is still much better than Twitter for these things. As Neil mentioned in his post, it is hard to understand why this is the case. Maybe because comments were attached to a shared item and not limited to 140 characters so you could actually have meaningful discussions. Unlike forums the shared items were a feed/river so there was the same impression and emphasis on immediacy as twitter. However, recently commented items would jump up on your feed which would tend to foster discussions. It is possible that it only worked because those that joined were the right people at the right time. Maybe it would not scale with the trolls. We will never know.

For those that never used it I want to write down the best experience I ever had on Friendfeed. I was attending the ISMB conference in Toronto in 2008. The number of geeks at this conference is understandably high and there were many Friendfeed users attending. At the time Friendfeed had already introduced the notion of a "room" which was a separate public feed that anyone could join. Similar to tracking a hashtag on twitter. A feed for the conference was set up and many people at the conference joined and started participating. In fact, the feed is still available here so you can go have a look for the time being. This was the first time I really had the impression of connecting to a hive-mind. In this back channel tens of people were taking notes and giving comments about the several simultaneous talks. During keynotes you could even see, as the speaker was changing topics, different people would take up the slack of taking notes and commenting according to their own expertise. Unlike twitter these didn't feel like we were drowning  in a sea of uncoordinated messages. You could always focus your attention on just one thread (i.e. a shared item) and its comments at any time. It worked so well that we ended up using the notes to write up a conference report that got published in PLOS Comp Bio.

That community of scientists and other open science advocates moved on to Twitter after the Facebook acquisition. Twitter usage by scientists and in particular by prominent established scientists also really took off at around the same time. Although it serves a similar purpose Twitter really is more of a broadcasting mechanism than a discussion forum. It is a pity that a lesser solution won out. Still, the amount of open scientific discussions that are going on online these days is just phenomenal and a drastic change from my PhD days.