Tuesday, January 18, 2011

The top philosophy journals: initial results

Readers will be aware of the philosophy journal poll I have been hosting here. The poll was comprehensive in that it covered over 140 philosophy journals, most of them suggestions by readers. These journals cover the full spectrum of the discipline. There have been more than 36,000 votes cast already and I believe we can draw some initial findings. Journals are each assigned a score: this is the percent (%) chance that voters will select this journal as their favourite if asked to choose between this journal and a second journal chosen at random.

The first finding is that there appears to be a top tier of philosophy journals -- this is not controversial -- that is relatively small -- this latter part may be more controversial.

From the poll, the top tier of philosophy journals appears to consist of the following publications:

1. Journal of Philosophy 87

2. Philosophical Review 84
3. Philosophy & Phenomenological Research 83
3. Nous 83 
5. Mind 82 
6. Ethics 80

I say that these appear to be the top tier as each were no. 1 or 2 at some point during the voting (unlike other journals). Each would be selected at least 80% of the time if paired with a second journal chosen at random.

A further finding is that the second tier of journals -- which we might classify as chosen at least 60-79% of the time when paired with a second journal chosen at random -- is perhaps surprsingly large. This second tier might consist of the following journals:

7. Philosophical Studies 79
8. Synthese 77
8. Philosophy & Public Affairs 77
10. Analysis 76
10. Philosophical Quarterly 76
10. American Philosophical Quarterly 76
10. Philosophers' Imprint 76
10. Monist 76
10. Canadian Journal of Philosophy 76
16. Journal of the History of Philosophy 75
16. Pacific Philosophical Quarterly 75
16. Midwest Studies in Philosophy 75
16. Proceedings of the Aristotelian Society 75
20. Australasian Journal of Philosophy 74
21. British Journal for the Philosophy of Science 73
21. European Journal of Philosophy 73
23. Erkenntnis 72
24. Philosophy of Science 71
25. Philosophy 70
25. History of Philosophy Quarterly 70
25. Ratio 70
28. Journal of Moral Philosophy 69
29. Oxford Studies in Ancient Philosophy 68
30. Notre Dame Philosophical Reviews 67
31. Philosophical Papers 67
32. Journal of Philosophical Logic 67
33. Journal of Philosophical Research 66
33. British Journal for the History of Philosophy 66
33. Utilitas 66
33. Mind and Language 66
33. Journal of Ethics 66
38. Southern Journal of Philosophy 65
39. Review of Metaphysics 64
39. Philosophical Investigations 64
39. Kant-Studien 64
42. Metaphilosophy 62
42. Philosophy Compass 62
42. Journal of Political Philosophy 62
42. Philosophical Topics 62
42. Philosophia 62
47. Hume Studies 61
47. Linguistics and Philosophy 61
49. Journal of Ethics & Social Philosophy 60

The next third tier of journals are those chosen about 50% of the time (from 40-60%)  where paired with a second journal chosen at random:

50. Phronesis 59 
51. Journal of the History of Ideas 58

51. Journal of Aesthetics and Art Criticism 58
53. Ethical Theory & Moral Practice 57
53. Philosophical Forum 57
53. Inquiry 57
56. Oxford Journal of Legal Studies 56
57. Political Theory 55
57. Social Theory & Practice 55
57. Philosophical Explorations 55
57. Journal of Social Philosophy 55
57. Economics & Philosophy 55
62. Law & Philosophy 54
62. dialectica 54
62. Public Affairs Quarterly 54
62. Acta Analytica 54
66. Social Philosophy & Policy 53
66. Theoria 53
66. Journal of Applied Philosophy 53
69. Faith and Philosophy 52
70. Political Studies 51
71. Journal of Value Inquiry 51
72. Harvard Law Review 50
73. Studies in History and Philosophy of Science 49
73. Philosophy & Public Policy Quarterly 49
73. Philosophical Psychology 49
76. Bioethics 48
76. International Journal of Philosophical Studies 48
78. Politics, Philosophy, Economics 47
78. Kantian Studies 47
79. History of Political Thought 44
80. Legal Theory 43
81. Hypatia 42
82. Philosophical Writings 41
82. southwest philosophy review 41
84. Apeiron 40
84. European Journal of Political Theory 40
84. American Journal of Bioethics 40

The remaining results for other journals are as follows:

87. Environmental Ethics 39
87. Logique et Analyse 39
87. Philosophy Today 39
90. Ratio Juris 38
90. Studies in History and Philosophy of Modern Physics 38
90. Business Ethics Quarterly 38
93. Journal of the British Society for Phenomenology 37
93. Ethical Perspectives 37
93. Public Reason 37
96. Hegel-Studien 36
97. Philosophy & Social Criticism 35
97. Res Publica 35
97. Philosophy in Review 35
97. Philo 35

101. Neuroethics 34
101. Ethics and Justice 34
103. Philosophy and Theology 33
104. International Journal of Applied Philosophy 32
105. Phenomenology and the Cognitive Sciences 32
106. Review of Politics 31
106. Jurisprudence 31
106. Research in Phenomenology 31
109. Journal of Philosophy of Education 30
109. Review Journal of Political Philosophy 30
109. Philosophy East and West 30
112. South African Journal of Philosophy 29
112. Kennedy Institute of Ethics Journal 29
114. Teaching Philosophy 28
114. Review Journal of Philosophy & Social Science 28
114. Critical Review of International Social and Political Philosophy 28
117. Journal of Global Ethics 27
117. APA Newsletters 27
119. Transactions of the C. S. Peirce Society 26
120. Bulletin of the Hegel Society of Great Britain 25
121. Adam Smith Review 23
121. Archiv fur Rechts- und Sozialphilosophie 23
121. Imprints: Egalitarian Theory and Practice 23
124, Theory and Research in Education 22
125. Polish Journal of Philosophy 21
125. Epoche 21
125. Fichte Studien 21
125. Symposium: Canadian Journal of Continental Philosophy 21
125. Asian Philosophy 21
130. Think 20
131. Archives de Philosophie du Droit 18
131. Collingwood & British Idealism Studies 18
131. Owl of Minerva 18
131. New Criminal Law Review 18
135. Journal of Indian Philosophy 17
136. Continental Philosophy Review 17
136. The European Legacy 17
138. Education, Citizenship, and Social Justice 15
139. Reason Papers 14
139. Associations 14
139. Journal of Indian Philosophy and Religion 14
142. Studia Philosophica Estonica 13
143. Derrida Today 5

Some further reflections. While there are several exceptions, it would be interesting to analyze any correlation between the age of a journal and its position in the rankings. There are several surprises on the list, this list does not correspond to my own opinions (I would have ranked many journals differently), and I do not believe that there is much difference between journals ranked closely together.

I also purposively put some selections in to see how they might play out. For example, I added Harvard Law Review out of curiosity and I was surprised to see of all journals exclusively publishing law and legal philosophy journals it appears to come second to the Oxford Journal of Legal Studies and above other choices. (I was surprised legal philosophy journals did not score much better.) I added several journals edited by political scientists, such as Political Studies, and was surprised to see they did not score as highly as I had thought. Roughly speaking, journals with a wider remit performed much better than journals with a more specific audience. I also added at least one journal, Ethics and Justice, that I believe is no longer in print. (Can readers correct me on this? I hope I am in error.) It scored 34% and came in at 101st.

What I will do shortly is create a new poll that will only have the top 50 philosophy journals from this poll roughly speaking. Expect to see this new link widely advertised shortly.

In the meantime, what do readers think we can take away from the results thus far? Have I missed anything?


Ben said...

I had a few goes at this and have one observation that might explain the poorer than expected showing for Political Studies - namely, we were asked to compare two journals *as philosophy journals*.

Had I been asked to compare Political Studies to, say, The Journal of Value Inquiry (which tied in your poll), I probably would have said that the latter is better qua philosophy journal, even if I regard the former as a better journal simpliciter.

Maybe the poll would have produced less surprising results if we'd been asked not to rank the journals as a whole but asked how we'd rank an average philosophical paper in each journal. (Though then I guess someone might complain that there are no philosophical papers in some.)

Also, I'm inclined to think that, when being asked to rank how good some journal is, qua philosophy journal, it's not unreasonable to favour a more general journal, because it will be of interest to a wider range of philosophers.

Anonymous said...

The results have already changed (in both scores and ordinal rankings) since you posted this. The top 10 now is:

Journal of Philosophy 86
Philosophical Review 84
Philosophy & Phenomenological Research 84
Nous 83
Mind 82
Philosophical Studies 80
Ethics 79
Synthese 77
Philosophy & Public Affairs 76
Journal of the History of Philosophy 76

One lesson to take from this shift (in just 20 minutes and perhaps a few hundred votes!) is something you've already noted: the exact ordinal rankings don't count for much, nor are the borders between the tiers all that precise.

In general (at least, as I checked in on the results over the last week), there appeared to be a "flattening" effect over time. Phil Review and J.Phil, e.g., had scores upwards of 95 for the first 10,000 votes or so, and the mean score of the top 25 was then much higher than it is now.

Anonymous said...

> There have been more than 36,000 votes

How is that possible? How can there be more than 36,000 people who are competent to evaluate philosophical journals?

Anonymous said...

In the updated survey, I think it would be helpful to specify what is meant by 'better journal'. For instance, I can see someone thinking that although Journal A publishes higher quality papers than Journal B, B is all-things-considered a better place to publish than A (perhaps because of its editorial practices). It would be nice if everyone used the same standard.

For the record, I'm most interested in finding out which journals are thought to publish the best papers, as there are other ways to find out about a journal's editorial practices, response time, and so on.

Jonathan Webber said...

I like this. But I wonder whether there's a bias towards the research interests of the respondent. Asked to compare a journal one knows from experience to be good with a journal one does not know well, I suspect most people would express a preference rather than not do so.

The journals that I would expect to do better than they do here are generally ethics, aesthetics, and phenomenology journals.

Of course, if this bias is there then that does not prevent the ranking being a useful guide to what the profession as a whole or some member of it picked at random is likely to think of your publications. But it does make it less useful as a guide to, for example, REF potential of publishing in a particular specialist journal (because the REF referee should be familiar with the field).

To some extent (though not wholly) this bias could be corrected for in the next round by making the 'I have no opinion on this' button less onerous (if you're not using the sub-categories it goes on to ask for, then there's no need for them).

Thom Brooks said...

In reply --
Ben: I see the point. When asked to choose between philosophy journals, certain politics journals which publish political philosophy might do less well. Perhaps the better question is "which journal is generally best for philosophical essays in your judgement?" although this seems a bit too wordy. I agree on your views on general journals and why these would generally score higher. Yet, these reasons are important to remember when we counter university bosses who then think general journals are generally better than specialist journals.

Anonymous 1 - quite right: there has been a flattening effect and I suspect this is because, well, I've got far too many journals listed. I will be using my recently published list of the top 50 to begin a new poll.

Anonymous 2 - yes, there were 36,000 votes...but any one person can (and should) vote more than once! So one vote is simply a choice between two randomly chosen journals. It would be helpful if voters voted on more than one pair so that preferences can be recorded on several journals, not just two.

Anonymous 3 - I agree and grateful for the advice. I will amend the question for the new poll accordingly...

Jon - sage words as always. First, I have no control over the "I have no idea which is best" button: this comes with the software I'm using. There will certainly be a bias amongst respondents and I think generally we tend to approve of what is familiar and disapprove of what is not -- and not just when it comes to choosing philosophy journals. So it'd be interesting to also see how the journal rankings compare with subscription rates: are those scoring best the oldest journals and/or the journals in the most libraries? It would be very helpful to know...

To all: new poll out very soon!

Anonymous said...

It is quite bizarre that the low-quality journal Philosophical Papers is much higher up than the top-journal Economics & Philosophy. Shows, I think, that a lot of votes are not based on proper knowledge of the journals ranked.



Roman said...

Bias effects do seem pretty clear; I'm a bit astounded that Inquiry didn't make it into the top 50; that seems insane to me in terms of quality, though of course Inquiry tends to publish spectacular articles on topics outside the absolute mainstream. How many people, after all, are competent to evaluate whether a paper on Heidegger is any good?

This explains to some extent the poor performance of all the specifically phenomenology/continental journals, though I can't imagine why "Continental Philosophy Review" fared worse than "Symposium." Is it because "Symposium" doesn't have the word "continental" as the first word of the title?

I am curious about what NDPR and Philosophy Compass are doing there. I have nothing against them and read them all the time; but it makes even less sense to compare them to standard journals than it does to compare, say, "Hume Studies" to "PPQ."

Thom Brooks said...

Anonymous - while I disagree with your opinion on the quality of Philosophical Papers, there were some surprising results. It would have been helpful to know, for instance, what field(s) voters worked in. If we control for fields, perhaps the results would be very different as perhaps most voting work in one field (favouring some journals over others) and not others.

Roman - yes, I was surprised on Inquiry as well. I included NDPR and Philosophy Compass to see how they'd rank against other journals. I found it very interesting that NDPR scored particularly highly.