Monday, December 14, 2015

Replace journals with recommendation engines

There was another round of interesting discussions on twitter after Mike Eisen decided to scrub all journal tittles from his lab's publication list. Part of the discussion was summarized in this Nature news story. The general idea is that our science should not be evaluated by where it is published but should stand by its own merit. We all want this to happen but unfortunately we don't have infinite time to read papers. Megajournal and open access advocates often dismiss this problem. They will often say that journal rankings are not adequate filters and that we should be able to make our own opinions and to search for whatever we want to read. This is the line of argument that just drives me crazy. It is basically implying that any defence of journal rankings is an admission of inability to evaluate science. The biomedical scientific community is producing over 100 thousand articles every month. Any suggestion that we don't need some sort of filtering mechanism is in turn an admission that you are not aware of the extent of science that is being produced. If you are not scanning table of contents yourself, you are being feed suggestions by someone that does.

Imagine a world without any science journals. Just a single pot where all articles are deposited. I think that the spread of knowledge would slow down. I can barely keep track of advances and authors that are closely related to my work by using keyword searches. I would not think one day to just search for "clustered regularly interspaced short palindromic repeats" or to have a curious look into advances in cryoEM.  In the absence of good filters we would risk becoming even more isolated in our small little corners of science and miss out on cross-fertilization. We would tend to focus even more on the science of a few labs that we knew from past works or from personal contact. I would not know where to look at for important new discoveries in other fields that could impact my own. The current system of journals serve this role of trying to assign a piece of science to a target audience. If nothing else, journals can filter through self-selection of topics at submission for specific communities. Less specific journals try to promote the advances in science that should reach a broader audience. I think that we are not even aware of how much the current system of journals facilitates the exchange of information within and across fields. In my opinion, the best way and probably the only way to get rid of the current system is to replace it by something that can do the equivalent job.

One way to replace the current system, by something less frustrating, would be to use automated recommendation engines. I have tried Google Scholar recommendations and Pubchase and both work really well. If we want to get rid of journals we need to figure out a way for such automated systems to mimic the journal's transfer of knowledge within and across communities. I can easily imagine the steps needed to come up with article similarity metrics and clustering of users and so on. One can also easily imagine that the recommendation engines can react to user feedback such that a niche community will "bump up" - for example by click-trough counts - the perceived value of a piece of science to such an extent that it get's recommended to a wider community. This would require a hierarchical recommendation engine that is widely used. The biggest advantage of such a system would be that it can work post publication on top of megajournals. Scientists could stop focusing their energy on submitting to journal X and just  focus on producing good science that would spread widely. I am convinced that the fastest way to get to a world without journals is to come up with this replacement. If we really want to get rid of impact factors and journal rankings we need to start talking about what we will do instead.

One thing we won't be able to change - we don't have enough time to read all of the science in the world. Unfortunately we don't even have enough time to read all of the articles of job applicants. It is not hard to predict that any other solution that replaces journal rankings will too often used to make hiring decisions.