Politics

Swing States

2008-10-30T00:00:44-04:00

I always wondered how much those swing states actually swing. So I looked at the results of presidential elections over the last 100 years, and it's not easy to determine which states actually are swing states from just looking at their history. Rather, there seems to be a pattern of relative stability for a few election cycles, and then big, sweeping wins for one side.

The data for this chart was collected from the U.S. National Archives and Records Administration, which unfortunately does not provide this in a very usable format. The format also switches at some point, making things more work than necessary. I had originally collected the data in a year-by-state matrix, which turned out to be a poor choice. I used Hadley Wickham's reshape package for R to "melt" the data into a more useful format. That data was then fed to Tableau to produce this chart.

I chose a red that is quite a bit brighter than the blue to make the two colors easier to differentiate. Blue, of course, represents democrats, and red Republicans. There is also the Progressive Party of 1912 (not to be confused with the Progressive Party of 1924, but I still gave them the same color), as well as the "Dixiecrats" who only ran in 1948. They were all so short-lived that I didn't pay a lot of attention to them, but you can find them in the chart if you look closely.

(Click for larger image)

You can see big, sweeping wins where one party takes over from the other, like in 1932, 1964, and 1968, etc. Bear in mind though that each dot represents a state, not a fixed fraction of either the popular or the electoral vote (which can differ quite a bit, too). I have ideas for how to show these things, but haven't been able to do them in Tableau or Excel, and just don't have the time right now to write a program for this.

What is also interesting to see is how recently some states (like Alaska and Hawaii) became proper parts of the US, and that even "contiguous 48" states like Arizona and New Mexico were not represented 100 years ago. The District of Columbia is the only "state" to never change color, but there are a few that have fairly consistent records, like Vermont and Massachusetts.

The goal was to make a chart that would show the progression of state winners over time. The vertical time axis is not optimal, but due to the large number of states, there really is no other choice. This layout makes it possible to see each year as one unit, and also to follow each state separately (in the large version of the image, anyway).

So this is really more a starting point than a finished visualization. I don't think I really succeeded in showing the crucial structures here, and there is more information to be included (though I did not collect data on the number of electoral votes over time). The data is available below for you to try your hands on. Let me know what you come up with!

Data: Elections_1904-2004.zip

Debunking the Cent Smear

2008-10-24T23:52:05-04:00

A story is making the rounds recently that the Obama campaign has received many contributions with "odd" amounts (i.e., not whole dollars), which is supposedly proof that Obama was being funded by foreign money. Here is a quick look at the data, which shows some interesting patterns, but no evidence of foreign intervention.

The whole story is of course non-sensical: if people were really charging their foreign credit cards, they would still send whole dollar amounts, since amounts are always specified in the target currency. But the much stronger evidence that the argument is nonsense is in the following image (multiples of 10 are colored blue, multiples of 5 (which are not multiples of 10) are green. Of the more than two million contributions, almost 94% were whole numbers, so the 0 cents case is not shown below.

As you can see, the distribution is very uneven (unlike what you would expect from the result of currency conversion). Multiples of five (and thus "round" cents) are much more common than values in between. The most common amount, though, is .95 – strange perhaps, but definitely done on purpose. The number .01 stands out (for the winner, presumably), and .08 quite obviously because of the year (I've read of people contributing $20.08 every month and the Obama Store also sells a lot of swag for that amount). "Odd" amounts in between are also explained by a list of cent "attributions" to a variety of blogs – and by rounding (when you buy something and you round the amount up to some nice number, so the difference becomes a contribution).

Interestingly, McCain's data looks quite different. Of the roughly 400,000 contributions, less than 0.2% have fractional parts. The only strong pattern is at .50, most of the others seem rather random.

Getting the Data

Finding this data was much more difficult than expected. The FEC publishes campaign contribution data, and it is possible to download their reports as a large file. It took me a lot of time to finally figure out their horrible COBOL-style file format and be sure (because I thought I was just missing something) that they were only reporting whole dollars. I had to get the actual filing data (at the very bottom of the FTP page) and wade through another horrible format (which also changed over time) to finally get to the data. It is a mystery to me why they only report whole numbers, with the number of contributions, those cents add up.

Thanks to Robert Morton, who pointed me to the right place in a comment below. I have updated the charts with that data, which has changed the overall numbers a bit, but hasn't had an impact on the patterns.

The Chart

The chart was made in Excel this time, because I had trouble getting Numbers to show me the right axis labels. I used the stacked bar chart idea with three columns, two of which were zero in each row. This way, it was easy to get different colors for multiples of 5 and 10. If there is any interest, I can make the parsed data and the Excel file available.

Popular vs. Electoral Votes Using Stacked Bar Charts

2008-10-11T20:50:09-04:00

A few days ago, I looked at how the electoral college system amplifies the lead of the strongest candidate in a US presidential election. The way I made the chart (with the help of PhotoShop) created some interesting reactions, and finally led me to what I consider the best way to do it (using stacked bar charts). I also want to respond to a few comments about the kind of chart used and why I think it is the most effective way to show what it does.

My use of PhotoShop may have seemed silly, but I used it to stitch together the screenshots of the different parts of the chart (it is higher than my laptop screen), so it didn't seem so absurd to me. But there are of course much better ways to do this.

Jon Peltier wrote two postings on how to achieve the effect using overlapped bars (which are possible in Excel but not in Numbers), but making sure that the shorter bar is always visible. He also modified the technique to show thinner bars in front, so that the full length of both bars can be seen.

Jock Mackinlay asked me for the underlying data (which I later also added to the original posting) and made a similar chart in Tableau. He uses an interesting trick to add an additional series that is shown in front when the shorter bar would be hidden: if the value "in front" is greater than the one shown behind all other bars, that bar has the same length as the hidden one, otherwise it is zero.

Using Stacked Bar Charts

I'm using Mackinlay's idea to create the chart using stacked bar charts. Stacked bars are quite flexible, and I've used them to prototype a number of visualizations, including the Presidential Demographics applet (the key there was making parts of the bars invisible). They are also available in virtually any program that can draw charts, so this method should work with practically any program.

Here is a version of my table that shows the raw data (name, (popular) winner %, and electoral %) as well as the three columns that are going to be used for the stacked bar chart.

These three columns work like this: bar1 is green, and shows the electoral vote in case it is smaller than the popular vote (and it's zero otherwise); bar2 is blue, and shows the popular vote in both cases (meaning it's the same as the popular vote if bar1 is zero, or it's the difference between the popular and the electoral if it isn't); bar3 is green again, and shows the electoral vote in those cases where that is greater than the popular (i.e., the majority).

The formulas for these three are as follows: bar1: =IF(C2>B2,0,C2), bar2: =IF(D2>0,B2-C2,B2), and bar3: =IF(D2>0,0,C2-B2).

The resulting chart looks like this (again done in Numbers, certainly doable just as well in Excel, Tableau, or Open Office):

The only thing I did in PhotoShop (besides stitching) was to remove the third element from the legend. I also took up Jon Peltier's suggestion to only show the 50% and 100% lines, rather than shade the area behind the lower 50%. That makes for a cleaner chart that is easier to read, and focuses on the things I really wanted to do with this.

Making a Point with a Chart

The reason for making this chart were two questions: Were there cases where the electoral vote was less than the popular vote (and which were those)? Which candidates were pushed over the 50% mark by the "amplification" from the electoral college system (and how much was that)?

The whole point of this exercise was to make those cases stand out where the electoral vote was less than the popular one, and as I already described in my earlier posting, that was not doable with any other chart I tried. So in a way, this chart makes a point: it guides the viewer's attention to one specific criterion. It is not meant to be a generic chart to compare two series of numbers (that would be better done using pairs of bars).

Reader TV also commented on Jon Peltier's first posting that the chart went against the convention of the stacked bar chart that would have the blue and green bars be parts of a total that is shown by the total length of both bars. Though that works here too, because the green part can be seen as the amplification of the popular vote, so both add up to the effective votes that counted for a candidate.

Almost exactly two years ago, I wrote about the difference between visualization and information graphics being that one murmurs and the other opines. This chart has a clear message, and it is focused on answering these two particular questions. That is why it turned out to be a good idea to not show more than two vertical lines, because reading the precise percentages is just not a priority.

Making such decisions makes a chart more focused, and thus stronger. While we want to provide the reader with the means to see different information in a visualization, I believe that we also need to make a clear point. If we don't do that, the viewer is confused and lost, and is not given a well-defined starting point for his or her own exploration.

The Electoral College and Second Terms

2008-10-07T23:31:32-04:00

The Electoral College is a key aspect of the US presidential elections. Its mechanics and distribution of electors are crucial for presidential campaigns and determine the so-called battleground states – and possibly also distort the will of the people. I was interested this last effect, so I did a little analysis.

A presidential election in the US is essentially 51 separate elections (50 states plus the District of Columbia). All but two states have a winner-takes-all system, with Maine and Nebraska using a slightly more differentiated way of splitting up its delegates between the candidates. There are a number of consequences of this that I don't want to discuss in detail here, but what I was interested in was the boost this system gives to the strongest candidate.

There are two aspects to this. First, there is the relative majority: which candidate got the most votes? Splitting this up further, there is the popular vote (how many people voted for a particular candidate) and the electoral vote (how many electors voted for that candidate). My hypothesis was that the percentage of electoral votes the winner got would always be higher than the popular vote.

The other aspect is whether the candidate who wins is the candidate the absolute majority of people (i.e., more than 50%) voted for. In recent elections, with only two candidates from the two big parties, this has become almost synonymous with the previous question – any third-party candidate would only get a minuscule fraction of the popular vote and not a single electoral vote.

A Comparison Chart

So I came up with the following graphic to answer my question. The blue bars show the popular vote, the green ones electoral votes. Since I wanted to compare, I tried out a number of different configurations, but none made it easy to see the instances where the electoral vote would be smaller than the popular vote. So I ended up with a kind of stacking where the longer bar would be "behind" the shorter one. The idea was that the instances with electoral < popular would stand out.

As you can see, there were only three instances where the electoral percentage was lower than the popular one. The boost from the electoral system is quite astounding in many cases, easily adding 30 points and more to the popular vote.

The other thing the chart shows is where a candidate was elected with less than 50% of the popular vote. The shaded area marks the 50%, and you can see that there were quite a few presidents who where pushed across that mark by the electoral college system. The most recent is George W. Bush, but the list also includes Bill Clinton, Richard Nixon, and others.

This is really only meant to provide a data point for the discussion of the merits of the electoral system – the issue is far too complex to be boiled down to a few numbers. But I think this chart illustrates quite nicely what effect the current system has. For another data-centric discussion of how less than 1% change in popular vote could have changed the outcome of many of the past elections, see Mike Sheppard's How close were Presidential Elections?

Second Terms

Since I already had the data (which I scraped from Wikipedia), I got interested in looking at the second terms of presidents (or, in the case of FDR, in second, third, and fourth terms). Would a sitting president tend to gain or lose points? And what is the effect of the electoral college here? The following chart shows this data for presidents who got re-elected.

At first glance, it appears that most re-elected presidents did gain votes, and most of these gains were amplified by the electoral college (the losses, too). There are two notable exceptions, Andrew Jackson and Woodrow Wilson: in these two cases, a gain in one actually translated into a loss in the other. I have no explanation how this was possible, especially in Wilson's case.

What is missing here is data about sitting presidents who did not get re-elected. But since I was mostly interested in popular vs. electoral, I did not collect this data. I will work on such a comparison for a future posting.

Charting Challenges

What surprised me was how hard it was to produce a good chart for what I considered a simple dataset and question. Putting pairs of bars next to each other was entirely ineffective, there was way too much noise, even with ample spacing between the pairs (which also created a huge chart). Neither Excel nor Numbers would let me specify negative distances between the bars to make them slide behind each other. I'm a bit surprised that this is so difficult, I'm sure I've seen charts with overlapping bars.

So I ended up creating stacked bar charts, with a few additional columns of data to generate the needed numbers. While that wasn't very difficult, it did defeat the point of doing this visually: if I could just look at the sign of the difference between the electoral and popular percentages, why bother with a chart? It still does provide a good way to present the data, especially the amplification of the stronger candidate.

While Numbers doesn't have nearly the power of Excel, I really like its approach to spreadsheets. It also produces much nicer charts, in my humble opinion. What it does not do, however, is let me change the color of an individual element of a chart – I ended up doing those in PhotoShop. Also, while Numbers lets me draw arbitrary shapes, there is no snapping to chart elements, only their outlines. That makes adding information like the 50% shaded area much more difficult than necessary.

While both Excel and Numbers do provide a large variety of chart types and settings, a lot of manual work is still necessary to make a chart really informative. And many things that should be very simple to do in these programs (including such advanced features as histograms) still require a lot of tweaking and the use of tools like PhotoShop.

The source data for these charts is available.

A better version of the chart using stacked bar charts is also available.

Presidential Demographics, Part II

2008-08-23T19:41:03-04:00

Would McCain be the oldest US President? Would Obama be the youngest? Who was the youngest president? Were presidents younger in the past or older? What is the highest number of years a former president lived after leaving office? Who served the longest? Whose term was the shortest? The interactive visualization below lets you answer these and a few other questions.

If you do not see the applet below as showing you a number of gray and black lines, you need to activate or install Java. If you see everything pushed almost all the way to the right, reload the page. This is a bug in the Java plugin for the Mac.

Moving the mouse over the individual time lines will show you the name and other information about each president (if there is no reaction to mouse movements, click somewhere in the applet to give it the input focus). Clicking on a timeline will open that president's Wikipedia page (if your browser and popup blocker allow that).The lines that appear when mousing over the timelines are there to help compare dates, e.g., was President X already born when President Y took office?

Changing the alignment lets you ask different questions about the data: Who was the youngest president? Which president lived the longest? Who lived the longest after stepping down? You can also easily see the presidents who died in office – either from natural causes or because they were assassinated.

The two timelines in the lower right are the two candidates for 2008: John McCain and Barack Obama. The start of their terms is indicated, so you can compare them to all the other presidents.

You will perhaps notice the one timeline that is broken into two. Grover Cleveland served two non-consecutive terms, which I decided to show so that he does not look like two separate people.

The source code of this applet is available.

Expressive Visualization, Updated Presidents Chart

2007-11-30T00:29:17-05:00

I used the Presidential Demographics chart in my talk at the Impact of Social Data Visualization panel at InfoVis 2007, and got some interesting responses to that. There is some interest in printing this out, so I have made a new version of the chart that is now also available as a PDF. Stephen Few used Joseph Berk's term "interocular traumatic impact" – a visualization that hits you between the eyes – to describe it. And this is exactly what visualization can do extremely effectively: visual communication, and not just of data.

Visualization can tell a story, just like a comic/graphic novel or a silent movie can. Expressive visualization takes the idea of visualization as visual communication further, and uses almost exclusively visual means to make a point. And it doesn't even have to be dumbed down or prettied up for that purpose: a simple chart can be much more effective in delivering such a message.

Most of us have probably seen cases where a visualization made one point stand out, even if that may not have been the most important one. We need to learn to control the strength of a visualization, and make it work for us.

I am interested in finding more examples of visualizations that make a point – political or not. Any pointers would be appreciated.

A Nobel Prize for Charts

2007-10-12T07:51:29-04:00

The recipients of the Nobel Peace Prize 2007 are Al Gore and the UN Intergovernmental Panel on Climate Change. While the decision is undoubtedly a political one (not unlike this year's Nobel Literature Prize), Gore has made a huge impact with his documentary, An Inconvenient Truth. A large part of that comes from his use of graphs and charts.

The charts and the underlying data have of course been the subject of much scrutiny and ridicule, mostly from people who do not agree with the message of the movie. A thorough critique of these is certainly in order, but the fascinating thing is that these abstract charts help to further his story. Similar to Hans Rosling's talk (which I will keep linking to until everybody has watched it at least five times), the visual presentation of abstract data is what gets the message across - even if the representation is itself rather abstract.

Visualization and information graphics are excellent tools for presentation, and their impact is being felt in the world. Visualization has to step out of the ivory tower and become aware of its power. We need to get the message out!