Visualization Choice Influences Decisions
Can different ways of showing the same data lead to different decisions? And can those decisions be about something important, like continuing a clinical trial? A study published in 1999 shows that they can, and the way the data is represented does make a difference.
The Study
The study is very simple: a number of physicians at a medical research institute were shown data from a fictitious clinical trial and had to make a recommendation whether to continue or abort the trial. This is an important decision with real implications: if it becomes clear in a trial that the new treatment is ineffective, it is unethical (and damaging to the participants) to continue. The data was shown in one of four ways: a table, a bar chart, a pie chart, or an icon display. The table could either show the response rate (positive framing) or the failure rate (negative framing).
The results are surprising: the icon display (lower right in the image) resulted in significantly more correct responses than either of the other graphical displays. The negatively framed table also lead to significantly more abort decisions than the positive one. Pie charts and bar charts performed very similarly. The study also looked into other reasons for differences, like the academic rank of the participant (all were Assistant Professor or higher), but found no effect.
Criticism
Unfortunately, the study has one major flaw: it appears that the correct answer in all cases was to abort the study. Instead of testing for correct perception of the data, the study therefore measured how bad the study made the data look. A better design would have been to test both cases, to see which technique leads to more false positives vs. false negatives, etc.
That the icon display did so well is partially explained by the fact that this was about people, and physicians might be able to relate to the data more as being about people when they see each individual person represented as a discrete visual object. Whether this can be generalized is an interesting question, but I don't see why it would not. That opens up many interesting questions about how we think about data and how it should be represented. And even if it were not a general result, its impact in medicine is clearly important enough to justify some hard thinking about how medical data is visually represented.
Conclusions
This is an intriguing study, and a good example of the types of questions we need to ask in visualization. The results clearly tell us that there is more to the choice of representation than correct perception. Questions like this need to be asked about many other application areas as well, and we need to get a better understanding of the higher-level processes that cause these differences.
Linda S. Elting, Charles G. Martin, Scott B. Cantor, and Edward B. Rubenstein, Influence of data display formats on physician investigators' decisions to stop clinical trials: prospective trial with repeated measures, British Medical Journal, vol. 318, pp. 1527-1531, 1999.
Posted by Robert Kosara on May 17, 2011.