Conference Acceptance Rates

Acceptance rates are one of the key ways of measuring the quality of conferences. I think it's time we collect that data for conferences relevant to visualization. I have put together a page for this, and have found some of that data. But I need your help to fill in the gaps and suggest other conferences that would be of interest.

I originally only wanted to include the VisWeek conferences, but then I figured it would make sense to extend the list a little bit. So currently, there is InfoVis, Vis, VAST, EuroVis, AVI, UIST, and CHI. I am open to suggestions for further conferences to include, but I want to keep focusing on visualization. So even though CHI and UIST are included, I don't want to expand into HCI much more.

There is also missing data. Especially for InfoVis, IEEE Xplore doesn't have the prefaces for conferences before 2005. Some of the early Vis conferences and a few others are also missing. Please take a look at the list and if you have access to proceedings for years where numbers are missing, send them to me or leave a comment below.

Given that this is about visualization conferences, I figured it would make sense to also visualize the data. The first visualization shows acceptance rates from 1990 to 2010. Not all of the missing lines are actually missing data, because the different conferences started at different times.

The next task is to look into impact factors for visualization journals. That information seems to be much harder to come by though, unfortunately. But it would be very worthwhile.

Follow this link to the acceptance rates data. The data is also available in CSV and JSON formats in a repository on github.

Posted by Robert Kosara on June 6, 2010.

---

Comments

Randall Hand says…

Does Acceptance Rate really correlate to Conference Quality? I've never heard a convincing argument of this.

Seems most of the conferences I know of with very low Acceptance Rates seem to suffer from the "in-crowd" syndrome, where they only accept papers from the same few people every year, no matter what they are.

Perhaps something connecting the papers & acceptance rates with the number of other papers using them as a reference (to find "foundation" papers) would be more useful.

Robert Kosara says…

There are certainly limits, a very low acceptance rate can be problematic. That's also the reason CHI didn't just decide to keep the number of papers constant and let the acceptance rate go down to 10% or so when submissions were increasing.

But it's still meaningful. There's no question that a conference with a 25% rate will have higher-quality papers than one with 40% or more. A rate around 20-25% is probably the best: it's selective while still leaving room for new ideas. And it's also a question how democratic and open the community is, and if it's controlled by a few powerful people.

Citation counts are another good metric, but much harder to come by. There's also a PageRank-like approach that looks at how highly cited the papers are that are citing your paper, etc. But the problem with that is getting the data, parsing references, etc.

Martin Theus says…

I am completely with you that extreme rates may be a hint that there is something wrong with the conference. But what makes a good conference? If it is just a "meeting" of the "best" papers, we are only generating CO2 by flying there. But seriously, meeting "the right" people, having the chance to bump into each others and exchange ideas is probably far more contributing to a good conference. The citation count is probably another thing you need to believe in, and often there is the correlation you have been talking about.

Good academic work is not necessarily bound to playing according to the commonly established system. Or vice versa, there is far too much happening in the community which just pays homage to the system. Both your suggested measures are tightly bound to the system - which may certainly be a good orientation to start off.

Anyways, great that you collected the data!

Tamara Munzner says…

I'm now back in my office after several weeks in Europe, and have access to my bookshelves again. Here's a bunch of the missing data:

conf, accept, submit, rate

infovis95, 18, 37, 49%
infovis96, 19, 34, 56%
infovis97, 16, 27, 59%
infovis98, 18, NA, NA
infovis99, 19, 47, 40%
infovis00, 20, NA, NA
infovis01, 22, 63, 35%
infovis02, 23, 78, 30%
infovis03, 29, 90, 32%
infovis04, 27, 89, 30%

vis90, 45, NA, NA
vis91, 41, NA, NA
vis92, 44, NA, NA
vis93, 41, NA, NA

vast06, 26, 60, 43%

uist91, 24, NA, NA

InfoVis98 preface has no submission numbers. Chairs Graham Wills and John Dill.
InfoVis 00 proceedings has no preface. Chairs Daniel Keim and Steve Roth.

Vis90 preface has no submission numbers. Chair Arie Kaufman.
Vis91 preface has no submission numbers. Chairs Greg Nielson and Larry Rosenblum.
Vis92 preface has no submission numbers. Chairs Arie Kaufman and Greg Nielson.
Vis92 preface has no submission numbers. Chairs Greg Nielson and Dan Bergeron.

UIST91 has no preface. Chair was Jock Mackinlay.

Depending on your motivation level, you could ping the chairs to see if they have records for the missing years. It would be good to have the data, I'm quite irritated that they didn't include it in the first place for the archival record.

Cheers, Tamara

Robert Kosara says…

Thank you so much! I'll update the page and data files right away.

Thanks also for the names of the chairs, I'm going to ask them for the missing acceptance rates. One problem I see with the move towards electronic proceedings is that they don't seem to include those prefaces (at least VisWeek 2009 didn't), and the digital libraries are sometimes missing pages.