While working on the ZIPScribble map, I started to wonder how to untangle the beautifully scribbly lines, and finding the shortest path through all ZIP codes. In computer science, this is called the Travelling Salesman Problem (TSP), and so I decided to make this the Travelling Presidential Candidate Map.

The TSP is a very hard problem to solve, and would have taken forever for the over 37000 points on the map, even when using a very efficient algorithm. So I had the idea of using a Hilbert curve to get an approximation. A Hilbert curve is a recursive space-filling curve that provides a linear path through all locations in a square. Here is one level of it, in the next level each line segment is replaced by a more coplex curve, etc.

I was not the first one to have the idea of using a Hilbert curve for this task, and the existing work shows that this solution is typically only 75% optimal. But that is good enough for my purposes, and it proved quick enough to implement and run a rather naive implementation of (about 10 minutes on a 1.67GHz Apple PowerBook).

(Travelling Presidential Candidate Map PDF)

The Travelling Presidential Candidate Map (TPC) does not look nearly as interesting as the ZIPScribble. There are hardly any crossing lines, which makes the image appear a lot lighter and less interesting. It also loses quite a bit of the information in the original map, which nicely showed the borders between clusters of ZIP codes, which also defined state borders.

(Travelling Presidential Candidate Map Color PDF)

It does gain from adding more information though, like coloring the states. Its slightly flimsy appearance does not change, however. The thin lines also make it harder to see the colors, especially in the small "thumbnails" on this page.

(Travelling Presidential Candidate Map Color with Names PDF)

Finally, the map with the added borders to better see where the path crosses state lines. This is not quite as interesting as with the ZIPScribble map, since there are no emergent patterns, but still.

(Travelling Presidential Candidate Map Color with Names PDF)

This map is a lot more technical and lacks the slightly artsy appearance of the ZIPScribble Map. It does gain from added information, quite in contrast to the other one. The path through all ZIP codes is – not surprisingly – not all that interesting. If you are running for president, you might find it useful, though.

Enrico Bertini criticized the following image for being a bad visualization on his Visuale Blog:

The image was made for a poster and book about a poetry festival. The following describes how they did it:

We assigned a numerical value to every letter of the alphabet. Adding the values of all letters, one gets a number that represents the overall word. (For example, the number 99 would represent the word »poetry«.)

Using this system, an entire poem could be arranged on a circular path. The diameter of the circle is based on the length of the poem. So you can see the short poems in the centre of the poster, while the longer ones form the outer circles.

Red rings on the circular path represent a number. As many different words can share the same number (»poetry« shares the 99 with words like »thought« and »letters«), most rings represents different words. The thickness of the ring depends on the amount of words that share the same number.

The organized chaos in the image certainly looks great, and makes for a beautiful poster and book cover. Each poem was also identified in the book by its ring of the whole image, thus giving it a certain visual identity.

But is it a good visualization? Clearly not. Mapping words in such an arbitrary fashion destroys a lot of information (completely unrelated words are mapped to the same number), and the way they are arranged does not give us any clues about the contents of the poems either. The different red circles and the apparent pattern that most words are on the right of the image are simply an artifact of the encoding that puts most words into a small range of values, and a few longer ones further away.

Is it a visualization? It clearly is something visual created from data, but the answer to this question depends on your definition of visualization – which makes this an interesting example to ponder.

Visualization in the analytical sense requires that the data be mapped onto the screen so that it is possible to understand or read the data from the depiction. This is similar to what is called a bijective function in mathematics. A visualization can never be truly bijective of course, due to the limited range and resolution of all the visual parameters of a computer screen. But at least getting an idea of the patterns present in the data, and a good approximation of the original data is required in visualization.

So in terms of a mapping or function, the poetry poster is similar to a hash function, which only works in one direction, and has a very high likelihood of creating different output for different inputs. Like a fingerprint, two hash functions that are different tell you that the underlying data was different, but will not convey any information about what the original data actually contained (or what the person the finger belongs to is like).

These above image is conceptually similar to J.P. Lewis et al's VisualIDs, which create arbitrary (but unique) icons for different documents. They are also the opposite of informative art, which has the goal of being bijective and aesthetically pleasing at the same time.

A comparison of these images to visualization (Bertini mentions TextArc as the better way of doing this) therefore misses the point. It may be a bit elaborate for a simple icon, but it is still no more than one unique image per poem. Also looking at the archive of designs for earlier posters, it becomes even more obvious that the goal is not to visually analyze, but merely to represent.

Visual representations of time are particularly interesting, because they seem so logical. A point in time is a point in the visualization, an interval is a line. But things are not always that simple: planning and temporal uncertainty require more powerful visual tools. Sets of Possible Occurrences (SOPOs) are an example of a visual representation of time that is very flexible and powerful – and totally unintuitive.

The Technique

A SOPO diagram consists of two time axes, one for start time and one for end time. Consequently, any point in the diagram represents not a point in time, but an interval. A diagonal line can be drawn from the origin of the diagram, which represents all intervals with the same start and end time – i.e., all intervals with zero length, or points in time. The length of an interval is represented by its position, not its visual extent. Only intervals on and above the diagonal exist, any point below it would represent an interval that ends before it begins.

To more fully appreciate SOPOs, a little context is necessary. There is an area in artificial intelligence that deals with planning and temporal reasoning, and that entails the notion of temporal uncertainty. Most reasoning is centered around complex time annotations, which have not only one start and one end time, but an earliest and a latest start, and an earliest and a latest end. In addition, time annotations are often constrained in that they cannot be shorter or longer than a certain time, i.e., they have a minimum and maximum duration. In temporal reasoning, it is important to be able to not just consider one time annotation, but large numbers of them, and how they influence each other. That was the reason Jean-François Rit developed SOPOs in 1986 for the purpose of Propagating Temporal Constraints for Scheduling.

In a SOPO diagram, the length of an interval can be determined by its distance from the diagonal (measured parallel to any of the axes), so when we extend our point into a line parallel to the diagonal, we get a representation of all the intervals with exactly that duration (two, in the example below). This ranges from the interval [1,3] to [3,5], and anything in between. In other words, 1 is the earliest start of this set of intervals, 3 is the latest start, 3 is also the earliest end, and five the latest end. Since the (minimum and maximum) duration is 2, 3 cannot be both the beginning and the end, though – the point (3, 3) is not on the line.

But what if we wanted to extend our intervals by varying their possible durations? We simply extend our line into a square, covering a wide range of possible intervals. Starting on the lower left, and moving around the square clock-wise, we get the following corner intervals: [1,3], [1,5], [3,5], and [3,3]. The two intervals along the diagonal of the square have the same duration (2), while the upper left one has the longest (4), and the lower right one the shortest (0). Any interval between these extremes lies within our SOPO, and thus a wide range of possible start and end points, and of durations.

The exact point when something starts or ends is usually much less interesting than how long something takes. That can be a condition (if reading is above max-value for at least five minutes, do this), or a limit for an action (if patient's condition does not improve after a maximum of four hours, abort treatment and try something else). SOPOs can be constrained by cutting off the corners that are too close to or too far away from the diagonal, to set their minimum and maximum duration. This yields the following shape, which describes all intervals starting from 1 to 3, ending at 3 to 5, and being no shorter than one and no longer than three units.

The development of SOPOs was motivated by a landmark paper in 1983 concerned with Maintaining Knowledge about Temporal Intervals, by James F. Allen. Six pairs of relations between intervals were proposed there (like before-after, meets-is met by, etc.), as well as the symmetrical equals. SOPOs can not only visually express all these relations, they also allow the visual propagation of constrains from one interval to the next. The areas for before and after in the example below are determined by the horizontal and vertical axis, respectively, thus also reinforcing the meanings of the axes: horizontal for end (anything that is entirely before our interval has to end before it), and vertical for start (anything after the interval can only start after the end of that interval).

SOPOs as Visualization and User Interface

In his diploma thesis Time Shapes - A Visualization for Temporal Uncertainty in Planning, Peter Messner investigated the use of SOPOs for visualization of medical therapy plans (unofficially co-supervised by the author). A few changes were made to make SOPOs usable as an interactive visualization, like rotating the diagram, implementing diagonal scrolling, etc., and also to ease understanding of some of the connections between sub-plans and parent plans. The two images at the top of this article and below were taken from Messner's thesis, and some of the final discussion is also based on his findings.

Interesting things happen when plans consist of sub-plans that are performed in parallel (Plan D below) or in sequence (Plan C, the marked one below). The expanded Plan C does not visually contain its sub-plans, unless a triangular area is added that shows the containment. Something similar happens with parallel plans that occupy the same area, and are therefore impossible to see and hard to interact with (especially if they also contain further sub-structures).

The yellow triangle around Plan C above is entirely wrong in the context of SOPOs (it represents a completely different time specification), but it satisfies the need for spatial containment when representing temporal containment. It also adds visual clutter, though, and in the end did not add much to make understanding the diagram easier.

Interestingly, there is also an aspect of uncertainty that SOPOs cannot deal with: undefined parts of interval definitions (which were possible in the framework this was done in). While these can be shown when there are enough constraints (e.g., only the latest end is missing, but there is a maximum duration), unknown values pose a serious problem to any kind of visualization. How large should the SOPO be drawn if we do not know when it will end? Simply adding an indication for "uncertain edge" still means that an arbitrary value will have to be chosen and shown in the visualization.

Ultimately, the SOPOView was a failure, though that was not entirely unexpected. It was clear from the beginning that SOPOs were difficult to grasp, and especially with many of them shown in complex configurations, users would easily get lost. Messner performed a small user study, and the users actually did surprisingly well, considering the method and how much time they had.

Discussion

Why bother talking about SOPOs, when they are so unintuitive? While they will not replace the Gantt chart anytime soon, they have some interesting properties.

SOPOs represent something of a lost art, that of visual problem solving. By drawing those shapes, Rit was able to work with very complex temporal specifications in two dimensions rather than with just numbers or cluttered one-dimensional diagrams. The trained eye would also immediately see what the results of these operations were, and how the structure of the SOPO changed from one shape to the next. Powerful tools require training and practice, both of which are usually in short supply when demonstrating a visualization or testing it in a user study.

It would of course be silly to criticize SOPOs as a visualization, since that is not what they were intended to be. While they would ultimately be read visually, it was not Rit's intention to require a significant paradigm shift to simply explain sets of six numbers. SOPOs are almost write-only, they exist solely as an analytical tool, that draws its power from the peculiar way it represents the data visually. That way is very likely not useful as a general way of visualizing time, but it shows how specific problems can be solved using specific means.

Unintuitive as they are, SOPOs help us break out of the usual way we look at time, and make us aware of the many assumptions we make about visual representations of the fourth dimension. That alone makes studying them worthwile, even if their role is that of the ladder in Wittgenstein's metaphor: after we have reached a higher point with their help, we can leave them behind.