Edward Segel: interactive features should scream interactivity

In my presentation on e-learning research last week I made a short reference to Narrative Visualization: Telling Stories with Data, the famous article by Edward Segel and Jeffrey Heer. Today, Marije Rooze tweeted a link to this great video in which Segel shares his ideas on digital storytelling, data visualization, and interactivity. The most important thing according to Segel is that interactive features ‘scream interactivity’ to avoid a ‘click-and-seek experience’. He point out several ways to help the user: be explicit (‘click here’), make suggestions for actions, and give visual clues. The accompanying slides are online as well.

How e-learning research can benefit dataviz design

Yesterday I had the honor to speak at the Show me the Data 2012, a conference in Amsterdam that also featured talks by Alberto Cairo and Jan Willem Tulp. In my presentation I explained how in my opinion data visualization can benefit form research in multimedia learning. At the end, I called for more experiments with visualizations that combined interactivity with verbal narration. So, to put my money where my mouth is, I edited my talk a bit and created this interactive audio slideshow.

[slideshare id=12221551&doc=presentatiebasbroekhuizenshowmethedata290312-120330060052-phpapp01]

The video mentioned in the second last slide is The Value of Data Visualization.

Isotype Interactive

For a moment, I thought it was a coincidence. A few months ago I became fascinated by Isotype, the pictorial language created in the 1920s and 1930s. Within the span of a week, I found out that several people I work with actually had made the same discovery.

First I attended an impassioned lecture on ‘graphics with a cause’ by my colleague Yuri Engelhardt who compared Otto Neurath, the father of Isotype, with Hans Rosling, a modern protagonist of the power of visualization. A couple of days later I discovered that Eugene Tjoa, with whom I collaborate in several projects, was planning to revitalize the work of Gerd Arntz, the man who designed most of the Isotype icons.

Of course it was not coincidental at all. Sooner or later everyone interested in data visualization stumbles upon Isotype and is captivated by its clean icons and its clear principles of design. Neurath’s guidelines for visualizing data are as valuable today as they were when he wrote them down in 1936.

For obvious reasons, interactivity is not part of the Isotype cookbook. That’s what makes Eugene’s plans to modernize original Isotype productions so exciting. He will update the statistics of visualizations designed by Gerd Arntz and enhance them by adding interactive features. In my opinion, the challenge of this great experiment will be to keep the interactive design as clean and effective as the original visual design. And although Neurath never knew about interactive media like the web, he did write about combinations of Isotype images in exhibitions. With a little fantasy, this remark could be a good starting point for creating isotype interactives:

Every picture has to give a new impulse to attention, to conscious thought, to a desire for deeper knowledge. Interest has to be the guide between one picture and another. But it is possible to overdo things. “Less is more.” The teaching effect will be greater, the memory will be clearer, when only a small number of good pictures has been given, every one different from the other, and a the same time every one supporting the other.

(International picture language. The first rules of Isotype, 1936, p. 66)

A taxonomy of interactive tasks

Last month I wrote about different taxonomies of interactive features. Today Flowing Data links to a exciting article by Jeffrey Heer and Ben Shneiderman who propose a typology of 12 tasks grouped into three categories:

  1. data and view specification (visualize, filter, sort, and derive);
  2. view manipulation (select, navigate, coordinate, and organize); and
  3. analysis process and provenance (record, annotate, share, and guide).

The authors write:

These categories incorporate the critical tasks that enable iterative visual analysis, including visualization creation, interactive querying, multiview coordination, history, and collaboration. Validating and evolving this taxonomy is a community project that proceeds through feedback, critique, and refinement.

Datavis in documentary

Although this blog is about interactive infographics, I’m interested in other aspects of information visualization as well. Lately, as the previous post indicates, I have been studying Isotype as a forerunner of infographic design. Earlier I wrote about my fascination with animated infographics or explanimations. Logically, I was very happy with a presentation by Ekaterina Yudin that I attended last week.

To obtain her master’s degree in New Media at the University of Amsterdam, Ekaterina wrote a thesis on data visualization in documentary films. She analyzed modern work but also included films by the famous British documentary maker Paul Rotha. During her presentation Ekaterina showed some fine examples of the way Rotha used the pictorial language Isotype to explain economic and demographic processes. Unfortunately these early examples of Isotype on screen are not available on the web – unless you have access to BFI Screenonline.

The other movies she studied are more contemporary. Most are modern classics – at least in the visualization community – like An Inconvenient Truth (2006), Food, Inc. (2008), The Crisis of Credit (2009), Joy of Stats (2010), and Inside Job (2010). I.O.U.S.A. was new to me – I somehow managed to miss it when it came out. This documentary uses all kinds of animated charts and diagrams to explain how the exploding national debt of the United States brings us on the brink of a financial meltdown. Mind you: the film was released in 2008 before the financial system actually collapsed.

In all the examples mentioned above visualizations are used to explain something in a classic, linear narrative. However, as an increasing number of interactive documentaries demonstrates, on the web stories no longer have to be linear. In online documentaries, explains Ekaterina, visualizations can have a different purpose than explanation. Mostly they are used as a part of the interface, like a map or a timeline that helps to navigate the story. Examples are Gaza Sderot (2008), This Land (2009), and The Interview Project (2010).

Online documentaries rarely use truly interactive visualizations “where viewers can manipulate the data points on the graphic display or interface, introducing design decisions that do not apply to non-interactive media.” Ekaterina found only one: Collapsus (2010), a dramatic story about the forthcoming transition from fossil fuel to alternative fuels that combines interactivity with animation, fiction and documentary. This award-winning project was produced by Dutch cross media production outfit Submarine. Not coincidentally, the same company hired Ekaterina as the new producer of their online channel. I’m looking forward to future documentaries with lots of interactive infographics!

So much beautiful knowledge for free

Like most research the study of interactive infographics is interdisciplinary. I try to mix concepts and ideas from different fields ranging from communication theory to information visualization and cognitive psychology.

Obviously, interaction design is also high on the list. I don’t have any background in this field, so I’m very glad with Interaction-Design.org. The encyclopedia (actually it’s more like a textbook) at this website is a welcome introduction to the world of Human Computer Interaction and User Experience. For me, the chapters about Visual Representation (written by Alan Blackwell), Data Visualization (by Stephen Few), and the Philosophy of Interaction (Dag Svanaes) are also very relevant. For who wants to go more in depth, the site features a huge bibliography with references from all the relevant conferences on interaction design and adjacent fields.

But not only the newest insights are available online. Dustin Smith today links to some amazing books about data visualization from before the era of the computer. Among these works, that are made digital available by The Internet Archive, are two books by Willard Cope Brinton: Graphic Methods for Presenting Facts (1914) and Graphic Presentation (1939).

I would like to add the digitized work of Otto Neurath, the creator of Isotype. For me, this ‘picture language’ is the starting point of modern infographic design. Some of Neurath’s best work, including the Atlas Gesellschaft und Wirtschaft (1931) and International picture language. The first rules of Isotype (1936) is available in pdf. We are privileged to live in times that offer so much beautiful knowledge for free.

Kraftwagenbestand der Erde, from the atlas Gesellschaft und Wirtschaft (Society and Economy) by Otto Neurath and Gerd Arntz.

Explanation first, then overview, zoom, and the rest of it

In reply to my previous post about the functions of interactivity Eugene Tjoa advocates more attention to explanation in interactive infographics. I totally agree with him. Information visualization has a very fruitful influence on the design of infographics but most infovis techniques are developed for specialists, not for a general audience. Furthermore, there are no general accepted conventions (yet) about the interaction design of visualizations. And although recent research suggests that visual difficulties can stimulate engagement and active processing of information, I think it’s a good idea to at least initially give users of interactives some clues about what they are looking at and how they can play with it.

At the New York Times they call these clues the ‘annotation layer’ and they take them very seriously. At last year’s edition of the Eyeo Festival, Amanda Cox of The Times graphics department stated that “the annotation layer is the most important thing we do.” Without it, she explained, it’s like saying to your audience: “Here’s an interface, now go ahead and browse for the rest of your life.” If you missed her presentation, be sure to watch the video.

The BBC also often adds a layer of annotation to its interactives. Literally. A good example is their interactive map of deathly accidents on British roads between 1999 and 2008. On opening the page, the actual map is partly hidden behind an overlay with a text that explains what data is visualized and how it can be manipulated. After this ‘window’ is closed, smaller overlays indicate the three ‘manipulation modalities’: you can enter your postcode, zoom in on the map or specify a year.

But when it comes to explanation, even the BBC can’t compete with The LRA Crisis Tracker. The smooth design of this interactive map and timeline in a way conflicts with its sinister content. The website offers a live overview of the atrocities committed by the Lord’s Resistance Army in central Africa. Visitors can track the number of abductions, mutilations and killings in real time.

Maybe it’s because of the gravity of the data that so much attention is paid to clarifying them. On your first visit you are automatically presented an overlay with a movie that explains what the site and the data are about. After closing the video player or on subsequent visits, the map and the timeline present themselves with a small animation that highlights the different interactive options and gives you a feeling for the amount of data. It’s like the designers adapted the famous visualization mantra to: Explanation first, then overview, zoom, filter, and details-on-demand. Unfortunately the animation doesn’t run as smoothly anymore as it probably did when the site was launched. Sad but true, the likely cause is the increase of woeful incidents.

What’s the purpose of interactive features?

My research is about the effectiveness of interactives. One thing I’m interested in is the influence of the type and number of interactive features on the ability of an infographic to transfer information. But before you can study this influence, you first have to define what these ‘interactive features’ actually are. Different typologies are possible. For instance, one could look at the form of interactive elements, like buttons and sliders, but in my model of interactivity these are just images or representational modalities. I’m more interested in the functions of these elements, or what I’ve called their ‘manipulation modalities’, like browsing, showing, hiding, dragging, zooming, filtering, or adding data.

Of course, researchers have categorized the functions of interactive features before. Probably the most famous classification is the one Ben Shneiderman proposed in The Eyes Have It, his famous article about the Visual Information Seeking Mantra: overview, zoom, filter, details-on-demand, relate, history, and extract. Recently I found a taxonomy of interaction techniques by Yi, Kang, Stasko, and Jacko. Although their typology looks like the ones above, it’s actually of a higher level. The following categories of interaction aren’t based upon tasks or functions, but upon the intent of the user: with what purpose does he or she perform a certain interactive operation?

  • Select: mark something as interesting
  • Explore: show me something else
  • Reconfigure: show me a different arrangement
  • Encode: show me a different representation
  • Abstract/Elaborate: show me more or less detail
  • Filter: show me something conditionally
  • Connect: show me related items

The typology is based on a review of professional information visualizations articles and tools and clearly focuses on the visualization of data. It would be interesting to test its usefulness with regard to interactive infographics in general, for example by analyzing a corpus of popular interactives on the web. According to the authors the categories are not exhaustive: “Some techniques are difficult to classify and do not quite fit into any one of the categories.” A quick try on What employees say, an interactive about the Fortune’s annual ranking of the best companies to work for created for CNN by Infographics.com, indicates that the opposite is true as well: in some cases one single click will perform several of the functions from the list above. Whether that says more about the typology or about the interactive, I’m not yet sure.

Visualization in the data journalism workflow

One of the reasons for the increasing interest in interactive infographics is the fast growing popularity of data journalism. Although a simple search on Amazon shows that computer aided reporting has been around since the publication of Philip Meyer’s Precision Journalism in 1969, it is undeniable that data is the new buzzword in the newsroom.

This renewed enthusiasm is powered by the rapid proliferation of accessible numeric data on the web in combination with a growing number of tools to make sense of them without a degree in computer science or statistics. In the slipstream of the open data movement journalists around the world try to tell stories based upon numbers. If you’re new to the concept, read this explanation on the Guradian’s datablog, or watch Geoff McGhee’s outstanding interactive documentary.

Although I wouldn’t dare to call myself an expert in the field, I try to stimulate my students to experiment with data driven journalism. To get them on track, I offer them a simple workflow inspired by Paul Bradshaw’s Inverted Pyramid of Data Journalism:

  1. Start with a relevant question
    (Why not pick one from the classic list of who, what, when, where, why, how, or how much?)
  2. Find and clean the data you need to answer this question
    (If they’re not available anywhere, maybe you can create them yourself?)
  3. Analyse the data. Look for:
    outliers (the best, the worst, the highest, the lowest),
    differences (then/now, here/there, we/them), and
    patterns (that weekly spike, the clustering in that region, the relation between those two variables)
  4. Explain your findings
    (Data almost never answer the question why, so grab your phone and try to find some reasons)
  5. Tell your story
    (What is the best genre to report your findings? How are you going to present your data?)

So what’s missing? Indeed, contrary to the inverted pyramid, there’s no separate step that says: visualize your data. That’s because in my opinion there are two different types of visualization involved in the process. Firstly, in stage three, you use all kinds of visual tools to analyze your data. These visualizations don’t have to be very sophisticated or well designed, as long as they help you to find your answers. Secondly, in stage five, you create a visualization that helps you to tell the story. That one has a different function and its visual and functional design should meet much higher standards.

In my classes, I simply refer to these different visualizations as type 1 and type 2. But after reading Tom Steinberg’s post from last week I think about renaming them to Answer Visualizations and Story Visualizations. Maybe these concepts can prevent starting data journalists from just presenting their own visual tools to the audience. Because in my humble opinion, that’s what a lot of them do. And that’s a pity, because on their own, without proper context, these ‘answer visualizations’ don’t tell any story at all.

Datavis in motion: pitfalls and possibilities

This blog has been hibernating for far too long. I have promised myself many times to revive it, but there always were other and more urgent things to do: preparing workshops, teaching classes, reading theses, and, last but not least, changing jobs. But today a very nice e-mail from a Norwegian student told me that there are actually some people out there who look forward to new posts and that really is the best incentive you can wish for. So no more excuses, time to get interactive again!

Plenty to write about for sure. Interactive infographics are in the spotlight more then ever, data visualization really is in motion. That is exactly the name of a seminar I have organized together with Eugène Tjoa and Statistics Netherlands. On February 3th we will discuss the pitfalls and possibilities of interactive data visualizations. Among the speakers are Jerry Vermanen, who created a platform for Dutch data journalists, Thomas Clever, one of the founders of design bureau CLEVER°FRANKE, and freelance information visualizer Jan Willem Tulp.

No datavis conference without a datavis challenge. We provided three sets of data (about changes in the dwelling stock, consumer confidence and agriculture) and asked participants to use one or more of these sets to create an interactive visualization for a general public. We explicitly also invited designers with little experience.

The submission deadline closed last week so you can no longer participate. But you can take a look at the submissions. It’s fascinating to see how different designers, with different tools and techniques have created complete different interactives based on the same data. The jury has already made up its mind but feel free to share your thoughts. Which design do you like best? And, most important to me, what do you think about the use of interactivity?