At a panel discussion at Vis 2006, we were blasted for raising the question, Is there Science in Visualization? A senior visualization researcher said that he was embarrassed that this question was being discussed, and that we were trying to push our way of doing things on the community. The panel was still a success, but this proved just how far we still have to go.
His arguments were not particularly strong: he argued that physicists would never ask that question (when there is a huge divide right now over how scientific string theory really is), and also said that we were just trying to present ourselves as role models (which we did not). But the point he made, and that a handful of people apparently agreed on, was to show how unhappy and uncomfortable some people are with discussing such a fundamental question.
If anything, we should all be embarrassed at how researchers in this field are not self-conscious and mature enough to deal with criticism. Technical people like to argue about topics that they think they care about, but that really don’t mean all that much to them. When it gets down to the things that hurt, this changes dramatically. Taken out of context, some of our comments could theoretically impact our potential for getting funding, and that would hurt. But despite some negative remarks, we all provided ideas what to do and how to improve – after all, that was the whole point of the panel.
And what is wrong with negative remarks, anyway? Can we not discuss what is wrong with our field? Do we have to pretend that everything is perfect, and reassure ourselves that we are doing great stuff? Visualization is still young, but when there are conferences with over 750 attendees, and many millions of funding for this type of research every year, we really are in a position where we should be able to question some of the basics of what we are doing.
Criticism, it seems, can only be dealt out anonymously, as a reviewer. And that criticism is only about the work of one author, not about more fundamental issues. Asking ourselves what we are doing, and why, seems to be outside of the comfort zone. This field clearly needs more people who are able and willing to think critically and talk, rather than more geeks.
What does all this mean for visualization criticism? It won’t be easy, that’s for sure. Not only does criticizing work of others mean putting yourself on the line (who are you to criticize? why don’t you do it better?), it also means fighting the resistance of those who feel threatened by criticism in general. So in order to keep the wrath at bay, we will need to be extra sensitive and positive, and sugarcoat the criticism as much as possible. Put in a more positive way, the criticism will have to be constructive and certainly not personal, like the remarks targetted at us.
The panel clearly was a success, though: we received the Best Panel Award, based on “attendance, the quality of presentation, the interaction with the audience, and the potential for impact on the field.”
Sounds like a very interesting panel. Are the slides available online?
I’m not familiar with information visualization research, however, I’m wondering if it wouldn’t be possible to use the same or similar techniques developed for user interface research?
BTW, is there any good place on the web where people can ask others for their (critical) opinion on information visualization (including simple graphs)?
Yes, the slides are available, I also updated the bibliography entry that's linked above.
As for research techniques: yes, some of the techniques used in HCI et al are used in InfoVis too, like user studies. But the question here is not just that, but what the actual basis for that research is, what we can do to improve it, etc. Colin Ware in particular has some very interesting ideas in that direction.
… I am working on including forums on this site, which should be ready to go at the beginning of next year.
Some of the reaction to our talk was based upon the provocativeness of the title. This, of course, was deliberate on our part to stir up discussion.
However, I agree that people are not often comfortable with discussing criticism. And many “hard scientists” are not comfortable with discussing the scientific rigor of their work—what they do, to them, is by definition scientific. A good many people would benefit from a history of science course for this very reason—it is hard to be take what you do as science for granted when you realize what we have allowed to be called “science” in the past.