Thursday, September 29, 2011

Journal Rankings for Philosophy

I've received many requests for some kind of journal rankings list from readers of my essay on publishing advice.

Readers will be aware of several different rankings of philosophy journals. These include the Australian Research Council's (ARC) now disused ERA rankings and European Science Foundation's (ESF) European Research Index for the Humanities (ERIH).  Plus, there have been different polls by Brian Leiter and the Brooks Blog (and this more comprehensive poll of 140+ journals). Other blogs discussing journal rankings include Certain DoubtsLemmings, Thoughts, Arguments, and Rants, and this.

Each metric has its limitations and such a discussion would merit a long blogpost of its own. Let me be clear from the beginning that I believe that journal rankings are the crudest of indicators. If you want to assess the quality of something, then read it.

What I propose here is a ranking of rankings. Journals will be grouped in tiers based upon various metrics. There is broad agreement between different lists and I don't believe this list will prove controversial. The journals that score best are those journals that have consistently ranked highly across the major studies both European (ERIH), International (ARC ERA), and major opinion polls of thousands of philosophers (Brooks Blog, Leiter Reports). We find wide consistency across most indicators, but taken together we can find a strong "core" that come out top again and again. Those that perform less well is often a result of inclusion on some indicators, but not others. Comments are most welcome and the list (with information on how data was collected) is below. Enjoy!

The official Brooks Blog Journal Rankings for Philosophy *

Rated 'A*' (maximum 25 points):
Journal of Philosophy
Philosophical Review
Philosophical Quarterly
Philosophical Studies
Philosophy and Phenomenological Research

Rated 'A' (20-24 points):

Analysis (24)
Australasian Journal of Philosophy (24)
Philosophy and Public Affairs (24)
Canadian Journal of Philosophy (23)
American Philosophical Quarterly (22)
Monist (22)

Rated 'B' (15-19):

European Journal of Philosophy (19)
Synthese (19)
Journal of the History of Philosophy (18)
Philosophers' Imprint (18)
Proceedings of the Aristotelian Society (18)
Ratio (18)
British Journal for the Philosophy of Science (17)
Journal of Political Philosophy (17)
Midwest Studies in Philosophy (17)
Philosophy of Science (17)
Journal of Ethics (16)
Journal of Moral Philosophy (16)
Pacific Philosophical Quarterly (16)
Philosophical Topics (16)
Utilitas (16)
Journal of Philosophical Logic (15)

Rated 'C' (10-14 points):

British Journal for the History of Philosophy (14)
Erkenntnis (14)
Mind and Language (14)

Kant-Studien (13)
Philosophy (13)
Ethical Theory and Moral Practice (12)
Philosophical Papers (12)
Phronesis (12)
Southern Journal of Philosophy (12)
Oxford Studies in Ancient Philosophy (11)
Review of Metaphysics (11)
Hume Studies (10)
Journal of Ethics and Social Philosophy (10)
Journal of Philosophical Research (10)

N/a ranked (9 or less points):

Metaphilosophy (9)
Philosophical Investigations (9)
History of Philosophy Quarterly (8)
International Journal of Philosophical Studies (8)
Philosophy Compass (7)
Notre Dame Philosophical Reviews (5)
Philosophia (4)

* Note on rankings:

I have weighted the journals in the following way:

ARC ERA list:
Journals are ranked A*, A, B, C. Points awarded: A* = 5, A = 4, B = 3, C = 2.

Brooks Blog list (1):
Journals ranked 1-50. Points awarded: #1-10 = 5, #11-20 = 4, #21-30 = 3, #31-40 = 2, #41-50 = 1.

List (2) ranks journals 1-50. Points awarded: #1-10 = 5, #11-20 = 4, #21-30 = 3, #31-40 = 2, #41-50 = 1, #51-143 = 0.

ERIH list:
Journals are (now) ranked Int1, Int2, Nat. Points awarded: Int1 = 5, Int2 = 4, Nat = 3.

Leiter Reports list:
General philosophy journals ranked only in top 20. Points awarded: #1-10 = 5, #11-20 = 4. Leiter has an additional list in ethics which raises complications. Journals are not double-counted and keep score if on general list. Points awarded: #1-10 = 5, #11-20 = 4, #21-30 = 3 where journals not on list 1. While this will cover general journals and journals that publish in ethics, there is need for a list in other areas especially mind/language and philosophy of science.

Notes: There are two lists for the Brooks Blog. List 2 is original list and surveys top 143 journals from a broad range. The top 50 in this poll were polled a second time in List 1.

QUERY FOR READERS: Do the weightings seem appropriate? What would you change? What journal rankings would you add?

UPDATE: Do readers believe the rankings are an accurate reflection of the field? Any surprises?


Doug Portmore said...

Hi Thom,

I think that Philosophers' Imprint is clearly better than either CJP or APQ; I would even say that Philosopher's Imprint is in a tier above CJP and APQ. This makes me suspicious of some of the other rankings.

Also, there's a typo. It's Philosophy and Phenomenological Research (not Review).

Ben said...

I'm not so familiar with the general journals, but my impression is that Phil Studies and Phil Quarterly - though both great journals - aren't in the same tier as J Phil or Phil Review (or perhaps even Mind/PPR, which I'd have put between these other two pairs).

That they are lumped together, while P&PA comes out a tier below Ethics (albeit at 24 to 25) seems odd to me.

Thom Brooks said...

Doug - Of course, all journals listed are amongst the best ranked in the field. The Philosophers' Index missed the top group because rankings, such as the ARC's ERA ranking (score = A) and ERIH (score = Int1), didn't score the journal in top rank. APQ was more often in cited journal rankings lists.

I should add that I don't think much can be inferred by one journal having 18 points and another 19 points, but something special about those journals scoring the maximum 25 points: these journals are ranked in the top group across all major rankings available (that I know of!).

Thom Brooks said...

Ben - Philosophy and Public Affairs (a journal I subscribe to - so I'm a particular fan of it) is only one point off the top group, so virtually a member. Journals such as Philosophical Studies and Philosophical Quarterly more consistently rank in the top group and so this is why they are included in my top group.

Obviously, my ranking is a "ranking of ranking": I'd be interested to know if (1) I should include any other journal rankings and if (2) I should amend my weighting criteria.

Of course, if these rankings seem suspicious, then this exercise is meant to cast some doubt on journal rankings in general. It is also meant to reflect the general standing of the leading journals across known major journal rankings. I am highly critical of how most lists have been drawn up and I thought a "ranking of rankings" would bring more attention to this important issue in our profession.

The next step? Perhaps a PGR on journal rankings organizing journals by tiers (A, B, C, etc)...

Jonathan said...

Hi Thom,

Ah rankings! So, two comments about the weightings. First, as you say, it's a ranking of the rankings but it seems that by including the Leiter Ethics list, this automatically skews this ranking of the rankings in favour of ethics-y journals over mind/language-y journals. I'd love to see what happens if that ethics ranking is dropped (if anything).

Second, it strikes me as a bit off (though I'm not quite sure why) that a 'bottom ranked' journal in the ERIH lists scores more than a 'bottom ranked' journal in the ARC list. I don't think it's obviously any *better* to give 1 point to the bottom rank of any ranking system and go from there, but it might look a bit different. Again, I'd be interested to see what happens.

Anyway, thanks for compiling this. It's always interesting to see how these things play out!


Thom Brooks said...

Jonathan - These are excellent points. In reply:
1. The weightings for ARC ERA and ERIH are of no consequence on this score. The ARC list is far more comprehensive with much greater coverage of journals. One consequence is that it has a large number of journals scoring "C" whereas the ERIH rankings doesn't include many of those journals (but it does include the top performing journals). I don't think any journal on my list received a "2" for a "C" on the ERA list.

2. Yes, this was something I worried about - but I scored Leiter poll journals in the following way. The first poll was of general journals and the second poll of ethics journals. If a journal was named on both, then I used its best ranking (giving scores for top half and bottom half of the few that made the final cut: note many, many journals were in the poll but didn't make final cut - they received 0 points). This coverage meant that some areas were left uncovered and it is an admitted worry. However, looking at the scores, the journals you'd expect to do particularly well if include M&L journals might be Nous, PPR, Mind, etc. -- and these all have earned the maximum score.

So I agree that more metrics should be added (created?) to make the list more inclusive, etc. But I'm unsure that this will make any substantive difference: those journals in top groups would likely stay in these groups (even if they might earn an extra point or two overall). Or so I would guess.

Ben said...

I made two observations earlier. What I found odd was not either one in isolation, but their conjunction.

Perhaps one way to put it is that I'd regard P&PA as closer to Ethics than Phil Studies is to J Phil.

So, if Phil Studies comes out as part of the very top tier (25/25), I'm surprised P&PA isn't also. Not necessarily saying Phil Studies should be less than 25, or that P&PA should be 25, but I would have thought one or the other.

Thom Brooks said...

Yes, but recall that Philosophy & Public Affairs has a nearly perfect score...

cristian said...

How would you place Politics, Philosophy & Economy in this ranking?

Thom Brooks said...

I think the journal is excellent and I'm a regular reader. I think it is very highly regarded. I have compiled a ranking of rankings: the ranking here DOES NOT reflect my personal views on which philosophy journals are best.

Those journals most often included and scoring consistently highly have made this list. If journals don't make the list, then this is perhaps all the more reason to question how these lists are drawn up in the first place.