Review This!

One of the hidden parts of the academic business is that of referee.  Academics tend only to value publications that make it through a review process, but who are the reviewers?  We are.*  That is, academics review the work of other academics.  The process varies, but the standard one for most respected journals in Political Science is double-blind refereeing.**  The reviewer does not know who the author is or authors are, and the author(s) does not know who the reviewers are.  I have had several people guess that I was their reviewer for much of their work, but they were wrong.  Guess the reviewer game is fun but tends not to be very successful.  Usually, there are two or three reviewers who provide their assessments to the editors of the journal, and the editors weight the various reviews and make a decision (sometimes they take forever to decide): accept, accept with minor revisions, revise and resubmit, or reject. Revise and resubmit (R&R) is often the best one can hope for, as reviewers and editors are often very picky, especially for the most visible and selective journals.

 *  This is one of the many big differences between normal academic journals and law reviews.  For law reviews, the law students do the reviewing.
** Increasingly (apparently), editors are "desk-rejecting" which means that they reject the piece before sending it out for review as the piece does not fit the journal or is obviously unworthy for the journal.  Reviewing books is usually blind in only one direction--that the book author does not know who is doing the reviewing, but it is relatively impossible to hide the identity of the author of a book length work.

So, for every one article submitted, there are a few academics who must agree to review it.  What do reviewers get paid?  Nada.  Why do they do it?  Because it is a professional obligation and because people think that editors will ultimately treat nicely the folks who are reliable reviewers and not treat nicely those who are not.  Whether the second motivation has any basis in reality, I don't know.  I have been on editorial boards but not an editor, so I cannot say how these folks respond to the unreliable. I do know that the reliable folks do get a certain kind of treatment--more reviews.

I have thought of myself as a reliable reviewer--meeting most deadlines and usually saying yes when asked.  I used to only say no if the piece was outside of my expertise or if it would be inappropriate for me to do the review.  What would make me an inappropriate reviewer?  Mostly if I had already reviewed that piece or similar work by the same author (I don't want to be the sole reviewer blocking a person's career).  How can I tell that if the process is blind?  Well, blind but not stupid.  Over the course of time, one gets asked often to review work that is close to one's own, but then one is familiar with the work in that area, so, ta da! not so blind.  Google also can make things less blind, although I never google to find the authors of a piece until after I have written the review and sent it off, and mostly not even then.

I am writing about reviews because today seems to be post about refereeing day.  See here for a good piece discussing rules of refereeing by Marc Bellemare.  Some very good suggestions that I will try to follow in the future.  I have been violating one of the rules lately--saying no.  Why?  Because I have said yes a lot recently,*** and am backed up.  As a result, I cannot credibly commit to doing the review in a timely fashion.  So, I say no.  I am also less willing these days to review stuff that is outside of my comfort zone, as I have realized that I cannot provide good reviews.  Given the ample opportunities to review stuff (especially after summers and winter holidays when folks finish stuff and send it off to journals)

***  I have said yes to a variety of writing commitments, journal reviews, and tenure reviews.  For the last, it requires one to read not just one article or book but many articles and perhaps more than one book.  Takes a lot of time with heaps of stakes involved--a person's career.

Phil Arena has a post on how not to be a good reviewer.  Such as rejecting stuff because it does not cite you or asking the person to not simplify.  For the former, it is a dick move.  I have recommended to folks to cite my work if it really is central to their work and only if my piece is published some place visible and even then only rarely and never as a reason to reject a piece.  For the latter, articles have word limits, so you cannot ask someone to add two more cases or explicate something that is not central to their argument.  A big difference between a book and an article is the latter is always going to be a snapshot and not a completed research agenda.

Young scholars cannot wait to get in the reviewer game.  Kind of like being eager to start shaving. Sure, it makes you feel like an adult, but once you get started, you are going to be doing it again and again and again for the rest of your career/life.  So, don't be so eager.  It will happen if you publish.