Opacity in scientific publication: Do journals discriminate?
Watson & Crick’s 1953 article in Nature revealing the double-helix structure of DNA was not peer reviewed. Many scientists claim this paper presents the most important discovery of the 20th century. The peer review system is what gives science integrity. Yet this paper was published based on the evaluation of the editors that it was obviously true.
One great discussion at the European Gender Summit in Brussels gathered editorial representatives from some of the most important scientific journals in the world, including the Lancet and Nature. That’s where I heard this story.
A pervasive theme at the Gender Summit was the connection between transparency and fairness, or between opacity and discrimination. When processes are not clear, open and transparent, prejudice comes into play, and the result can be unjust.
Where do these two topics overlap? Are there senses in which scientific publishing is secretive or opaque?
I think there are at least two areas where the guidelines are so vague that the (perhaps subconscious) biases of editors are likely to play a role.
The first of these involves journals that have professional staff members who carry out an initial sorting of submissions. Both of those journals report that they send out approximately 20% of submissions for peer review. How do they make that decision? They have scientific editors who work full time at the journal and these editors look at submissions and judge them on a number of criteria.
At the workshop I attended, a repeatedly used phrase was “impact.” That is, they identify articles likely to have high impact and send them out for review. Articles that are likely to have low impact, are returned. This is an implicit and nearly opaque process, and I wonder how it could be opened up.
It’s in the interest of the journals to only receive submissions that they find potentially interesting. How can one communicate the notion of “impact” or, even more difficult, the notion of “likely impact” to potential publishing scientists?
The second murky area I see involves the selection of reviewers. When an article is sent out, it is usually reviewed by three peers, and the opinion of those peers will be extremely important in the final determination of whether an article gets published. All editors are concerned to avoid conflicts of interest when they select reviewers.
But once we get beyond that, how does this work? Of course we try to identify “suitable” experts. But it is likely that the personal networks of the editors are extremely relevant. Given the power the reviewers have, perhaps a step in the right direction would be to remove the cover of anonymity.
Scientific publication has a hallowed status. At times, I think that is deserved. But at times, it’s just another old boys network. When it starts to look like that, we have to start thinking about new models. We’ll keep discussing this at the upcoming Munin conference. Maybe you’ll join us?