Peer Review in Crisis

EJIL and its sister publication, I-CON are peer-reviewed journals. This is a counter-cultural posture in an age which celebrates, for some very good reasons (and some less admirable), the freedom that self-publication on the internet provides. Our own very successful Blog, EJILTalk!, is an example of a highly interesting and useful form of self-publication and I-CONnect will be launched soon. There are surely others like ours. SSRN is a more ambiguous example, but even there, there are some diamonds in the rough, if you have the patience to do some heavy-duty prospecting and sifting. Be that as it may, SSRN is not just part of contemporary academic culture; it is a defining part, both reflective and constitutive.

There is a place, we maintain, for discernment in publication, including external referents. There are some weeks where the (electronic) mailman (in the form of ScholarOne) sends my way one or even more articles per day for both EJIL and I-CON. We need to select, not simply because the economy of a journal dictates such, but because we try to give our readers a certain guarantee of quality, even excellence. We know, too, that in many countries, publication in a selective, peer-reviewed journal plays an important role in appointment, promotion and tenure.

At the heart of such a system is, indeed, peer review. This institution is in serious crisis, which is evident in the functioning of both journals. I have discussed the issue with other Editors in other journals and the situation is the same elsewhere. I am, thus, taking the extraordinary step of publishing a similar editorial in both EJIL and I-CON.

At EJIL (and I-CON) we try to practise double-blind peer review: in principle, the reviewer should not know the identity of the author, and the author, obviously, is not privy to the identity of the reviewer. The double-blind principle is not always achievable. We do not have the resources to scour each and every article that goes out to review and excise from it all tell-tale signs, notably footnotes of the ‘see-my-treatment in…’. Some authors have a distinct voice which is impossible to conceal. And, as I explained in greater length in an earlier Editorial (‘Demystifying the EJIL Selection and Editorial Process’, at 22 EJIL (2011)), since we like each piece we publish to have had critical scrutiny by at least two sets of eyes, oftentimes one of the peers is myself; obviously I am aware of the identity of the author. In that case the double-blind principle will apply only to one of the reviews.

We give considerable thought to the selection of ‘peers’. We look for people who have expertise in the field and whose own publications meet our yardstick of excellence. We make liberal use of our own Scientific Advisory Board and Members of the Editorial Board. But given the volume and diversity of submissions we receive, even after our in-house screening which reduces the numbers considerably, we need to venture outside and turn to the legal academic community at large.

Why crisis? Simple enough: first, the difficulty of finding willing peer reviewers. Peer reviewing is a selfless task. Though we have taken to sending a little gift to external reviewers in the form of an Amazon Token, peer reviewing is an act of academic citizenship which demands sacrifice in time and mental energy. Sometimes our turn-down rate exceeds 50 per cent. Not infrequently we will get a refusal from the first and second and third reviewer to whom we turn, whilst the hapless and unsuspecting author is fuming at the length of time it takes EJIL to reach a decision. I find it particularly galling, yes – galling, when a published author in EJIL or I-CON refuses to review, having himself or herself previously enjoyed the fruit of the system. If ever there was a case of spitting into the plate whence one eats.

Second, not infrequently I receive reviews which are perfunctory, conclusory, poorly reasoned and hence appear arbitrary, effectively amounting to an ‘it’s bad because I say it is’ or ‘it is good because I liked it’. This is not only unhelpful to me as Editor, but raises doubts in my mind whether the reviewer has taken time to read and ponder the article carefully. In such cases, and the number is not trivial, the process has to begin again. Since I have been with EJIL throughout its life (though only recently as Editor-in-Chief) and with I-CON for some years now, I can also point quite distinctly to a deterioration in the situation. One can only speculate as regards the reason: Perhaps a proliferation of Journals with a corresponding increase in the number of times one is asked to review? Perhaps a change in culture where the very institution is eclipsed by self-publication, and  peer reviewing has fallen outside the basket of academic citizenship virtues? Perhaps everyone is more busy writing?

And yet, we really cannot do without peer review. We can rely on our ‘own resources’ – the gallant members of our Scientific Advisory Board and other members of the Board of Editors only so much.

What is to be done?

• We need, perhaps, to revisit the structure of incentives. When one is asked to peer review a prospective book, one is typically offered some free or discounted books by the publisher commissioning the review. Let’s see if we can work something out with OUP.
• We will be shortly opening a Register of potential reviewers, a roster which we can then dip into.
• In our letter of acceptance to authors who have submitted an article to I-CON or EJIL we plan to prominently emphasize the gratitude to the anonymous peer reviewer and emphasize the moral duty to commit to this duty of academic citizenship.

 We welcome other suggestions from our authors and readers.