If peer-review is a fundamental aspect of how science is done then we in academia are required to act as reviewers as part of our jobs. I assume there is no real argument as to if peer-review is needed. Instead one can argue about when to do it, in the life-time of a research project, and who should do it. Do we do it before the results are made public (pre-publishing) or after (post-publication) ? Do we have dedicated reviewers or should acting scientists do this ? The current dominant form of peer-review is done by active scientists and before disclosure of the results. This is all done anonymously and hidden from view. There are many issues with this process as the many repeated peer-review rounds required when an article bounces from journal to journal. Another drawback is that nobody gets credit from doing the work and reversely nobody gets shunned from not contributing sufficiently to the process.This week, Alex Bateman (@Alexbateman1) directed me to the Publons company that is attempting to create reviewer profiles. Publons is attacking the problem in a couple of different ways: they mine journals that provide open peer reviews; they curate the journal's confirmation emails sent in by reviewers and they are apparently in talks with publishers to automate this process. The reviewer controls the degree of information displayed by the site. The minimum information shared is the journal name and the month which is what I expect most people will opt for. Alex had apparently kept the journal's emails acknowledging the receipt of his reviews going back for many years and he has created an extensive profile that demonstrates his contributions as a reviewer (and editor).
The company has been profiled last October in a Nature news article where Andrew Preston, one of the co-founders, states the aim of making peer-review a measurable research output. It is useful to have a verified account of our reviewing activities but I am not sure if Publons is the best way of getting there. Given that we have ORCID we could imagine that publishers would be able to jump over Publons and report reviewing activities directly to ORCID. On the other hand, Publons may serve as a focus point to get publishers to provide this information in a standardized and automated fashion. For now, the closed reviews are coming from the authors so that is why I assume the company has a reward program that has been giving out awards worth $3000 for the top reviewers in a given cycle. I do wish that companies like this one, that collect information without an obvious source of income would make their business plans more transparent. Even their terms and privacy statements are currently empty. Are we putting effort into something that will last, that will sell this information ?
So how many referee reports should we do per year ? I guess that we should aim to do at least as many as the articles we publish. The truth is that this probably varies widely and having some feedback and accountability will be good for the system. I keep all my referee reports on file so apparently since 2007 I have done 53 referee reports or about 7.5 per year. This has varied a lot with years where I have done as few as 2. With 24 articles published and only 2 years into a group-leader position I think I have been contributing well. I have set up my profile in Publons and sent in a couple of recent reviews to see how the process works. So far it has all been very straightforward and I will
give it a try for a while.