Thursday, December 02, 2010

Thom Brooks on "Guidelines on How to Referee"

At long last, I have completed a draft of "Guidelines on How to Referee." The guidelines offer practical advice on how to referee for academic journals and publishers. It is written in the same vein as my earlier "Publishing Advice for Graduate Students" that I know many have found useful. While there was little available on publishing advice when I wrote the earlier piece, there seems even less advice available on how to act as a referee. I hope this new set of guidelines fills a real gap and proves popular.

The abstract is:

"This essay offers clear practical advice on how to act as a referee when asked to review an article for an academic journal. The advice is also relevant for reviewing manuscript proposals for academic publishers. My advice is based on my experiences in editing an academic journal, the Journal of Moral Philosophy, and four book series. I will draw on these experiences throughout as illustrations. The structure of the advice is as follows. First, I will begin by saying a few words about the academic publishing industry. Secondly, I will discuss whether one should accept or decline an invitation to review. Thirdly, I will examine the question of what appropriate standard should be applied when reviewing submissions. Finally, I conclude with advice on how to draft a report before submitting it to an editor.

The essay is designed in much the same spirit as my earlier “Publishing Advice for Graduate Students” and my hope is that this new essay on refereeing advice will be found every bit as useful by colleagues and students."

The paper is available for free downloading here.

Comments and suggestions are most welcome! Please do share the link with others who may be interested in this topic.

UPDATE: My thanks to Larry Solum and his Legal Theory Blog for posting a link to this paper here!


Anonymous said...

Thanks for writing this. Your previous guide was very helpful as well. - Kris McDaniel

Thom Brooks said...

You are very welcome, Kris. I've been amazed at how popular my previous guide was. I hope that this new guide is helpful to the profession as well. If there is little good advice around on how to publish, then there is even less good advice on how to referee.

Mike Titelbaum said...

This is excellent—I'll definitely pass it on to graduate students I know when they referee for the first time. Since you said you're planning on making changes in the future, here are two issues you might want to address:

1. The experience you draw on is mostly from moral/political venues, but here's an issue that comes up on some of the more technical sides of philosophy: Technical papers sometimes involve proofs, which can be long and difficult to check (especially if they employ results from other sources). Even though no editor has ever made this explicit, I've always assumed it's part of my job as a referee to check the proofs. (Because who else is going to?) But perhaps you could put in some official word on this from a referee of a more technical journal.

2. Perhaps also a couple of words on a tricky judgment call: Sometimes I read a submission and think that while the argument is good, I have a response that would (in my mind at least) pretty convincingly defeat it. The question then is whether to reject the piece on those grounds, recommend a revision that responds to the objection, or just leave it to the literature to further that dialectic on its own. I don't think there's any universal right answer here, but something on which factors to weigh in making this decision might be good.

Thanks again for the work you've obviously put into this project!

Anonymous said...

Thank you, though I very much wish I had this a month ago as I am a student and just finished my first referee job.

I am relieved to see that I did not screw up too much, though I did give an extended summary of the paper.

For purposes of fixing up the draft: Page 9, second full paragraph, sentence four, "then" instead of "than". Nothing else stood out.

Thom Brooks said...

Many thanks for these wonderful comments. In response to Mike's comments, I don't think it is the responsibility of referees to copy-edit the paper as well: this is something for the publisher and editor to sort out if the piece were accepted.

On your other comment, I would think that if you had a knock down argument against a claim in the paper - whether or not it has been published - I would note this as a problem for the paper that may lead to its being rejected.

Mike Titelbaum said...

In my first point, I wasn't talking about about copy-editing the paper. I was talking about checking to make sure that any proofs in the paper are valid (i.e. checking the logical steps). To some it might seem like a no-brainer that this is part of the referee's job, but I know people who just assume the proofs go through if the conclusion sounds right. (And I certainly know of cases in which the publisher and editor did no checking at all!) So I thought it might be good to have a word on this in the guide.