Peer Review, Part 4: Good Reasons for Bad Papers
As a reviewer, you might sometimes ask yourself why people write so many bad papers. And why they bother submitting them. I certainly do. But where do they come from? Who submits bad papers? And why? It may come as a surprise, but there are good reasons to submit bad papers for review.
To Get Feedback
This is one you will hear advisors talk about quite a bit. “I didn’t expect the paper to be accepted, but I figured we’d get some useful feedback!” It’s not unreasonable to do this, though the question clearly is: when you think there is actually value in the feedback you’ll likely get from reviewers?
If you know that the paper is unfinished or even bad work, it makes no sense to send it in. But when you’re looking for direction or want to get an idea if you’re on the right track, reviews can be helpful. Of course, they can also be brutal.
The key is to act on the feedback, not just send the same bad or unfinished paper to another conference. I’ve seen people shop papers around without making any significant changes. In a field like visualization, where there is a lot of overlap in reviewers between different conferences, and between conferences and journals, this can bite you in the ass. I have no patience with people who do this, because they just waste the reviewers’ time. Respect the reviewers and the reviews, and work on your paper before you submit it again!
To Get It Out
There’s only so much you can do as an advisor to get a student to improve a paper before the deadline, short of just rewriting the whole thing. At some point, either the deadline is here, or you just decide that the remaining improvements are not worth the time. So send it in and hope for the best.
That’s not the best reason to submit a paper, to be sure, but it can help get a student back on track. It also provides him or her with a sense of accomplishment – even if that may be short-lived, only to be crushed by the negative reviews a few weeks later.
The realities of conference deadlines also contribute to this. A deadline can be a great motivator to get a lot of work done in a short amount of time, or to finish that project that has been dormant for months. But they also lead to a lack of reflection and time for refinement, which can sometimes be quite obvious. I once reviewed a paper where the first half was really well written, but the results, discussion, and conclusions had tons of typos, factual mistakes, and even notes the authors had put in for themselves to rewrite some parts. That paper did not get accepted.
To Just Give It A Shot
In theory, a paper should be submitted when it’s ready and its authors think it can’t be improved anymore. But the reality is that most people chase deadlines and need to get papers published at some reasonable rate if they want to advance in their jobs (or be considered relevant).
There are also small pieces of work that would be nice to publish: a class project, a master’s thesis, etc. Those are often overlooked in visualization, where it can be hard to find a place for something small but worthwhile. The short papers track at EuroVis might help with that, and there are always posters. A small contribution does not mean a bad paper, but the risk is certainly higher that it might be considered trivial.
Where to Draw the Line
The above was written with the assumption that the paper is (mostly) written by a student, and the advisor has a good sense of the quality of the paper. That might not always be the case, but I think it often is.
Getting work rejected is no fun, even when it is expected. There is a certain social contract between authors and reviewers though that should require authors to be very careful not to submit work they don’t reasonably believe could be accepted. If it’s obviously bad, don’t submit it.
Unfortunately, there is no good way to get feedback on a paper before it is ready. Sending it to colleagues might work, but they might never read it. And they will be much nicer when asked to review a paper for a colleague (or colleague’s student) than when writing an anonymous review.
Ultimately, there are still valid, if not always very good, reasons to submit papers that the authors know aren’t good enough.
This is part of a five-part series on peer review in visualization. One posting a day will be posted throughout this week.
Teaser image by bubbletea1, used under Creative Commons.
Posted by Robert Kosara on January 22, 2014. Filed under peer-review.