Visualization Without Pictures

A question that I have been asked a few times recently is: what about visualization for the blind? Would it make sense to provide a textual representation? What about a representation that you can feel and touch, isn’t that a visualization? And is there perhaps more to visualization than just the pictures?

Visual and Non-Visual Visualization?

The word alone already makes it quite clear that visualization has to be visual. There are examples of using other modalities like sound, e.g., to explore a scatterplot, but those are clearly not visual; Andrew’s description of this sonification example as “non-visual visualization” is a rather obvious oxymoron.

But consider an even simpler example than an interactive sonification, what about a simple tangible scatterplot? It’s not hard to imagine how to do that, and there may be Braille displays that are able to do that. Is that a visualization?

Since that kind of representation would not be viewed but rather touched, it’s clearly not a visualization. The mechanisms are not entirely different between a visualization and a haptic representation, though. The key in both cases is spatial thinking, and that does not require vision. Blind people can navigate through cities and can imagine spatial relationships, so there is no reason why they would not be able to understand a haptic scatterplot. But there is still a difference between that and a visualization.

Our visual perception works very differently from our hearing, touch, and other senses. Not only is vision much more prominent than our other senses (just think about where you would locate your self: right behind your eyes), it is also an extremely powerful way of detecting patterns and quickly categorizing the world.

Visual representations have their own, specific rules. Colors, shapes, textures, orientations, etc. are very specific to vision. We can also look at a large number of things at the same time and pick up certain properties in parallel and within a fraction of a second (this is called preattentive perception). Hearing is very different: it is much more serial, with fewer different channels we can differentiate and a very specific idea of what patterns have to sound like to be pleasant (or even just bearable).

Munging all these things together and calling it all visualization is not just lazy and wrong, it also means ignoring what makes our senses special. Just like a painting and a piece of music are completely different things, we have to understand how different modalities work before we can make effective use of them.

Beyond the Pictures

But the bigger question is: if we take away the pictures, what remains? Is visualization simply a different modality of data analysis? What exactly does it mean to do visualization without pictures, anyway?

When Kaiser Fung talks about statistical thinking without actually showing specific formulas or data, he describes statistics without using statistics. I wonder if we would be able to write a similar book about visualization: not just compelling case studies that show real-world impact, but descriptions of using visual tools without showing pictures. How did the use of a scatterplot matrix help solve a particular problem? Why wouldn’t parallel coordinates have worked in this case? Why would a statistician have missed an important clue in this case?

Such a description requires an understanding of the underlying properties of different kinds of visualizations, how they relate to data, and what they can be used for. It also requires a language to talk about visual structures and what they mean, beyond just statistical terms. There are often structures that can be found in visualizations that do not cleanly map to statistical features, but that provide hints that guide further analysis.

I am convinced that there is more to visualization than just the pictures. But that does not mean that visualization can be done without pictures; quite the opposite. What we lack is an idea of the different levels at which visualization can be understood, and an effective way to communicate that. As we develop a clearer view of all the things that visualization is, we will be able to make a better case for it, and make people see the value of this field, even if they don’t (and don’t want to) know all the little details.

We can already do some of this: talking about visual structures in a visualization is nothing new. Perhaps one of the ways to get a deeper understanding of visualization is to put the images away for a while while we think about visualization without actually doing it.