Multi-touch Brushing for Parallel Coordinates

Interaction in visualization is incredibly important, but often more tedious than it needs to be. I have developed a new way of brushing in parallel coordinates that uses the multi-touch trackpads on Apple’s MacBook and MacBook Pro laptops for faster interaction. The video below demonstrates the technique, and the source code is available.

This is a little experiment that has gotten out of hand a bit. I recently submitted a paper to a conference that doesn’t let me attach additional materials, so I figured I’d put the video up on vimeo and link to it in the paper. I also posted a link on my secondary blog, which got posted on Twitter, and which ended up being blogged by some other people.

If you’ve used a MacBook or MacBook Pro, you know how simple and intuitive the two-finger scrolling is. Similarly, there are some interactions on the iPhone that are so obvious (like the pinch for zooming) that it’s tough to remember how we did things before touch screens. I wanted to create a similar experience for brushing in parallel coordinates.

The idea is to map the axes to locations on the trackpad, and use several fingers to indicate different kinds of brushing. A single finger selects an axis, two fingers let you brush on a single axis. With four fingers, you can brush on two axes at the same time, either on adjacent ones or ones that have other axes in between them. There is also a somewhat unusual brushing method called _angular brushing_, which is very tedious to use with a mouse. Using three fingers, it’s very quick and intuitive, and great for finding outliers in the data. You can also flip and rearrange axes.

There are a few details that make this a bit more complicated, because it’s hard to maintain the horizontal position during interaction. So there is a certain “stickiness” to the first axis, which keeps it selected even when the fingers move somewhere else. The right mix of stability and flexibility is not easy to find, but I think I’m getting close.

The trackpad is not just a good fit because it’s a lot more common than large touch screens (also on some non-Apple laptops), but because it solves an interesting problem you have when interacting with stuff on a touch screen: your fingers get in the way. That isn’t always a problem, but in the case of brushing, it is. You don’t want to have to change the brush, lift your hands, look at it, put down the hands, adjust, lift again, etc. With the interaction happening on a different surface (and in a convenient location), you can focus on the program’s response to what your hands are doing.

The video below demonstrates the technique. It’s still quite a different thing to actually try it out yourself, though. It allows for very fast exploration of data, unlike any interaction technique I’m aware of. Using your fingers is not very precise, but that’s not the goal here, anyway: it’s about getting a feeling for a new dataset.

You can watch the video in HD on the vimeo site. The inset in the lower right is meant as a guide to show what the fingers are doing, not as a part of the actual user interface. People have suggested showing the location of the fingers as an overlay, but when you watch the video you will see why this would be a bad idea: the fingers don’t have to line up with the axes, plus the floating dots would be distracting.

I have also posted the source code and there’s an executable under the downloads tab if you want to play with it. Be advised, though, that this is a pure research prototype that demonstrates the technique but doesn’t do anything useful. The data is hardcoded and the program only lets you brush and rearrange axes, nothing else. The program requires Mac OS X 10.6 “Snow Leopard” and a recent MacBook or MacBook Pro.