design, cities, physical & social interaction, play

Leapfroglog design, cities, physical & social interaction, play

Playing With Complexity — Slides and Notes for My NLGD Festival of Games Talk

When the NLGD Foundation invited me to speak at their anual Festival of Games I asked them what they would like me to discuss. “Anything you like,” was what they said, essentially. I decided to submit an abstract dealing with data visualization. I had been paying more and more attention to this field, but was unsuccessful in relating it the other themes running through my work, most notably play. So I thought I’d force myself to tackle this issue by promising to speak about it. Often a good strategy, I’ve found. If it worked out this time I leave for you to judge.

In brief, in the presentation I argue two things: one — that the more sophisticated applications of interactive data visualization resemble games and toys in many ways, and two — that game design can contribute to the solutions to several design issues I have detected in the field of data visualization.

Below are the notes for the talk, slightly edited, and with references included. The full deck of slides, which includes credits for all the images used, is up on SlideShare.

Hello everyone, my name is Kars Alfrink. I am a Dutch interaction designer and I work freelance. At the moment I work in Copenhagen, but pretty soon I will be back here in Utrecht, my lovely hometown.

In my work I focus on three areas: mobility, social interactions, and play. Here is an example of my work: These are storyboards that explore possible applications of multitouch technology in a gated community. Using these technologies I tried to compensate for the negative effects a gated community has on the build-up of social capital. I also tried to balance ‘being-in-the-screen’ with ‘being-in-the-world’ — multitouch technologies tend to be very attention-absorbing, but in built environments this is often not desirable.1

I am not going to talk about multitouch though. Today’s topic is data visualization and what opportunities there are for game designers in that field. My talk is roughly divided in three parts. First, I will briefly describe what I think data visualization is. Next, I will look at some applications beyond the very obvious. Third and last, I will discuss some design issues involved with data visualization. For each of these issues, I will show how game design can contribute.

Right, let’s get started.

1. What It Is

Data visualization is admittedly a broad term, that can be used for many things. There are also closely related terms, such as information visualization, and information design. Data visualization seems to be the most common term though, often shortened to dataviz. One important qualification is that I am focused on visualizations that involve at least some level of interactivity.

Making Visible the Invisible

In his book The Ghost Map, Steven Berlin Johnson writes about a cholera outbreak in Victorian London. One of the book’s central protagonists, Dr. John Snow, creates a map of the cholera cases to help show that the source of the outbreak is a water pump. Here’s part of that map. I’ve marked the location of the pump with a red circle. For each case, Snow drew a black dash at the location of its residence. You can see clearly that the cases diminish the further you move away from the pump. You have to remember that in those days, people still believed that diseases such as cholera and the Black Death were spread through the air — the so-called miasma theory. The germ theory had not yet been definitively established.

This is an excellent example of showing things that cannot be seen. This is one of the defining qualities of data visualization for me.

Many Shapes and Sizes

Showing the invisible can be done in many ways, and information design goes back a long way — in the caves of Lascaux there is a map of the heavens dating back to 16,500 BC. Over the years, many ways to visualize data have been developed. You’ve got maps, charts, and graphs of all shapes and sizes. The point is, each data-set requires a specific display. Also, although there have been many established patterns, it is of course completely legitimate to experiment with new visualizations.

A Medium, Not a Technique

This is something the people at San Francisco-based design and technology studio Stamen often say. Data visualization is not just a technique — a way of doing things. It is also a medium — a way of conveying messages. From their site comes this quote:2

“It’s not quite honesty–you can lie with charts and graphs as easily as with words or pictures. It’s not quite accidental discovery–this is pretty technical stuff, and deliberate choices are made at every step along the way.”

For instance, Stamen have created something called Oakland Crimespotting. On this site you can explore a map of crimes committed in Oakland. When outsiders see this, they tend to react with shock, saying they would never want to live in Oakland. This was never the intention of the Stamen people though, and if you look closely at the patterns in the data, you discover that conventional wisdom about such things as which neighborhood is the most dangerous is often wrong.

This is an example of an argument made using data visualization. In the gaming world, I think this is mirrored by concept of procedural rhetoric as described by Ian Bogost: Using game models to convince people of your point of view. Both show that technology is not neutral, that neither games nor data visualization are just techniques, but media.

2. Non-Obvious Applications

Hopefully that has clarified what I mean when I say “data visualization”. To make things more concrete I’d like to look at two broad areas of application, beyond the well-known information graphics as we encounter them in print media. The first is the idea of making complex things more understandable. The second is this idea of improving people’s lives through data visualizations.

Understanding Complexity

I think we can all agree the world is becoming more complex. But in this case I am talking about a specific kind of complexity known as organized complexity. This is when in a system — be it biological, technological, economic or something else — a relatively small number of parts interact in a nontrivial way. Such systems display behavior that is not carried by the parts, so the organized character of the system emerges, without top-down control.

Many of the systems we find ourselves surrounded with these days are like this. The problem is that old-style models from things like statistics aren’t useful when dealing with them. What we can do is show the data that results from such systems. We can then literally see the emergent patterns. Going one step further, by making these visualizations interactive, we can allow people to play with complexity — exploring the space of possibilities (a term that should be familiar to game designers).

For example, this is a piece by Ben Fry, one of the initiators of the open source project Processing. It is a visualization of traffic in Los Angeles. It is meant to be manipulated with a gestural interface known as G-Speak (which was developed from initial concepts for the film Minority Report). The many degrees of freedom people have with a visualization like this allows them to get a deep understanding of the complex system that is urban traffic.

Clearly, game-like interactions are very appropriate for these kinds of visualizations.

Happiness Hacking

A hack is a clever solution to a problem, often in the form of a shortcut. I first came across the idea of happiness hacking on Jane McGonigal’s blog. She’s working towards the “goal of using new scientific research on well-being to develop technological systems that actually improve quality of life”.3 There’s a lot more to her crusade, I encourage you to check out her ETech presentation on the topic.

This idea of happiness hacking is extendible to other areas too. For instance, there is a whole subculture of lifehackers who are mostly focused on boosting their productivity using clever tricks. When I was writing this presentation, I used a hack called ten-plus-two-times-five, which was invented by life hacking guru Merlin Mann. The idea is that you set a timer for ten minutes, during which you only focus on work. Then, as a reward, you get two minutes to play, and then it’s back to work again. Repeat this five times for an hour of being productive for 50 minutes. Not bad. It works for me!

What this has to do with data visualization is perhaps best exemplified by the Apgar score. This is a way of measuring the condition of a baby after birth. When this scoring was introduced after its invention in 1952, the mortality of newborns dramatically decreased. This was because there were now numbers that could be collected and compared. Doctors could now measure what effect different procedures had on the health of babies. Measuring things changes them.

So now imagine you could quantify things that relate to quality of life — to happiness — and visualize those.

This can be done on the individual level. RescueTime is a web application that keeps track of what you do on your computer. You can tell it what types of activities you find productive or not, and it’ll visualize how you’re doing over time. This is what Matt Jones calls personal informatics, and there are more and more examples popping up. An older one being BodyMedia’s SenseWear Weight Management System. Each of these is an example of happiness hacking using data visualization. Like in games, it allows people to manage their resources.

By combining individual data-sets, we will be able to get a sense of collective patterns. We are all participants in complex systems ourselves, after all, and we aren’t particularly good at perceiving the high-order consequences of our actions. Matt Jones has this to say about it:4

“The overlays of these patterns with those of others are a new kind of feedback we haven’t had at any scale before. And we do flock well. So perhaps that’s how we will learn and change our behaviours… in a “supercontext” if you will…”

Supercontext is a term taken from the comic book series The Invisibles. There, it is a state-of-being humanity is evolving towards, where all is ambiguous and non-binary.

There aren’t many true collectively intelligent data visualizations out there yet.5 I am confident we will see them arise though. When they do, they will require game design. Compared to interaction designers, game designers are much more comfortable with complexity and indirect control, which is exactly what these systems will require. Wouldn’t you like to be involved with bringing joy to the world?

3. Design Issues

In this last section, I’d like to look at some design issues with data visualization. There is an ongoing debate about the role of aesthetics — is it important displays look pretty, or not? Also, interactive data visualizations tend to employ unorthodox interaction styles. Should utility trump explorability? Finally, is it enough to describe past data or should we also predict the future? I’ll show that for each of these questions, game design can contribute to the answer.

The Role of Aesthetics

In the fall of last year, Stamen launched Twitter Blocks. For those of you who are not on Twitter: It is a so-called microblogging service, where you leave short messages about what you are doing for your friends and followers. You can do this using web, IM, SMS or other channels. This is what my Twitter stream looks like:

Blocks was created by Stamen for Twitter. It is a 3D visualization of your friends’ status messages. From each friend’s message another stream is drawn. So you get to see your friends’ friends too. I guess it sounds complicated if you don’t use the service. The point is that this way you get to have a sense of what’s happening on Twitter beyond your own stream. Anyway, here’s what it looks like:

When this launched, it was met with quite a bit of criticism. A lot of it basically boiled down to: “This looks pretty, but it’s useless.” The prevalent idea amongst people still seems to be that aesthetics is subordinate to utility. I thought we had gotten over this, designers in fields such as product design and architecture have satisfactorily proven that if something is pretty, it is perceived to work better.

Andrew Vande Moere writes in a paper about this very topic (PDF):

“[…] information visualization should be enriched with the principles of creative design and art, to develop valuable data representations that address the emotional experience and engagement of users, instead of solely focusing on task effectiveness metrics […]”

The point being, aesthetics are ‘useful’ in their own right. They can make the same functional experience be perceived as wholly different. I believe you as game designers understand this very deeply. The role of fiction, dressing etc. in games is prominent. No-one would argue that a game with the same mechanics but drastically different aesthetics is the same game. It is not. Properly executed aesthetics are vital to the experience.

Utility Versus Frivolity

Let’s examine the criticisms received by Twitter Blocks a bit more. A lot of people complained about its uselessness. One of the designers of Blocks is Tom Carden. He writes in a response to the criticism on his blog:

“[…] why not also accept that some things might just be for entertainment and ask “am I having fun” once in a while instead of looking for a problem to be solved or an important statement to be read? Some things just are.”

The irony is that Twitter itself isn’t exactly a hardcore utilitarian experience. To a large extent it can be seen as a toy, and a lot of what people do with it is very frivolous.

In game design there have been big advances in the understanding of why people play. There is for instance a model called PENS — the player experience of need satisfaction — which shows that one flavor of fun results from a feeling of autonomy — the feeling of freedom to explore something on your own terms. I think Twitter Blocks’ fun is like this.

An Expressive Language

Blocks’ interface is unorthodox, it is exploratory. Other examples I’ve shown before have this same quality — interactions that might need some getting used to, but once you do are quite powerful. I tend to think of this as learning an expressive language, and games are particularly good at it.

Take Portal, for instance, where you are gradually taught the many uses of one single mechanic. The wonderful thing about Portal is that it expands your thinking in an almost subliminal way. Some people have gone as far as calling it a subversive game.

“[…] games are [not] a way to convey and directly put content in players’ brain but rather that the cognitive processes mobilized when playing games can create relevant routines that may possibly be transfered to other activities […]”

This is Nicolas Nova writing about a presentation by Ben Cerveny at Picnic 07. Ben is an advisor to Stamen. He talks a lot about the transformative potential of games.

My point here is that exploratory interactions not only contribute to a fun experience but also facilitate a kind of deep understanding. People gradually master an expressive language that allows them to think about complex things in a new way.

From Description to Prediction

Let’s move on to a slightly different topic. A while ago my friend Alexandra, who’s a designer and also CEO of physical computing studio Tinker.it, blogged about her disappointment with the current wave of data visualizations. She’s annoyed that most displays don’t go further than showing you the data.

For instance, this is my carbon output from flights over the past year or so. It is calculated by Dopplr — a social tool for optimizing travel. (I apologize for the horrendous output and promise to improve over the coming year.) The issue here is that from this display I can only say: “Wow, that’s a lot.” There are two ways to improve this: One — extrapolate the data into the future, giving me a sense of what my actions might result in on the long term. And two — offer me ways to mitigate the effects of my actions. Let’s start by focussing on the first.

Alexandra writes:6

“I would like to see us move towards a world full of little everyday objects that give me a glimpse into the future if I keep doing things the way I do, total yearly bills based on my current usage, predictions about how much I’ll have to spend on food and how much weight I’ll gain if I keep at the current pace.”

In stead of only describing the status quo, Alexandra would like data visualizations to also predict the future. I like this idea, because it is a non-prescriptive way of motivating people to change their behavior.

What would we need to do this? We would need to start using models in addition to data. Models of the phenomena we are showing data of, so that we can generate possible future outcomes. Therein though lies a danger — what if the predictions turn out to be wrong?

The Ludic Fallacy

In his book The Black Swan, Nassim Nicholas Taleb describes what he calls the ludic fallacy — “the misuse of games to model real-life situations”. In other words it is mistaking the map for the territory. The problems with predicting the future using models are many: It is impossible to know everything (to have all the data), it is very hard to account for amplified effects of small changes (a.k.a. the butterfly effect), and any model we employ can only be based on prior experience.

As Yogi Berra reportedly said:

“It is tough to make predictions, especially about the future.”

It is important to take the ludic fallacy into account when designing predictive data visualizations. It should be made clear to people that they are looking at a simplification of reality. For instance, perhaps it is wise to show ranges in stead of end points. In this way you communicate a certain level of uncertainty.

A Hybrid Approach

There is promise in this hybrid approach though. Data visualization traditionally draws displays of data about things that have happened. From the visualization one can start seeing patterns. Games have traditionally employed models to generate fictional realities. Any pattern that might arise is encoded in the model. By combining these two, we arrive at data visualizations that describe what has happened, and predict what could happen.

Finally, because we have a model, we can now also make the possible effects of mitigating actions perceivable. Imagine having a display of my past and future carbon, where I can play with different ways of offsetting it — flying less, planting trees, etc. The visualization would show me in real-time how much each method would affect my output, and perhaps relate it to financials too, so that I can figure out what the most cost-effective way of offsetting is.

I think this is an obvious type of ‘new’ data visualization that can benefit from game designers’ involvement.

I hope I have shown today that data visualization is a field that offers interesting opportunities. It can clearly benefit from the experience of game designers. If you do decide to get involved with data visualizations, I am looking forward to seeing your creations!

Thank you.

  1. For more background on this project please see this older blog post. More examples of my recent work can be found in my portfolio. []
  2. Taken from the page Data Visualization. []
  3. Quote taken from the blog post Work, Work, Work – How I Spent My 2007, or, a Year in Review []
  4. This quote is taken from a long but excellent interview with Matt Jones on Ryan Freitas’s blog which discusses personal informatics amongst many other things. []
  5. One example I stumbled across just before delivering this talk is Citysense. []
  6. This quote is taken from her blog post Prescriptive or predictive information visualisation?. []

14 Comments

links from TechnoratiPlaying with Complexity”(…) in the presentation I argue two things: one — that the more sophisticated applications of interactive data visualization resemble games and toys in many ways, and two — that game design can contribute to the solutions to several design issues

Posted by InfoDesign: Understanding by Design - News on 30 July 2008 @ 12am

Kramer auto Pingback

[…] Playing With Complexity found: 09:31am July 9, 2008 Kars Alfrink’s slides from a talk for the NLGD Foundation, on data visualization, aesthetics, and usefulness. […]

Posted by m. migurski / snippets on 10 July 2008 @ 1am

[…] Playing With Complexity — slides and notes for my NLGD Festival of Games talk (Leapfroglog) “the more sophisticated applications of interactive data visualization resemble games and toys in many ways, and… game design can contribute to the solutions to several design issues … in the field of data visualization.” Kars on top form at NLGD. (tags: design interaction play games playfulness dataviz infoviz visualisation ixd) […]

Posted by Infovore » links for 2008-07-09 on 10 July 2008 @ 1am

Lots of weekend reading for me here.

The problem of predicting the future is age old. Otto Scharmer and co of MIT write on it extensively. One of his colleagues worked with Shell on scenario building. Exploring the complexity rather than predicting.

This week I was checking on the definitions of final and efficient causes. I think sometimes we get fixated on believing something must or will happen rather than looking at the motivations of everything (including non human entities) and working out what we would like to bring about.

Great post. Happy homework for me.

Posted by Jo on 11 July 2008 @ 4pm

Kramer auto Pingback

[…] Playing With Complexity — slides and notes for my NLGD Festival of Games talk (Leapfroglog)  - 10-jul-2008 […]

Posted by target_is_new: Joost mist nog wat warmte on 13 July 2008 @ 5am

Gee, thanks Jo, I am glad I was able to provide you with some food for thought.

I see Mr. Scharmer has (co-)written a few books. Any one in particular you can recommend?

Posted by Kars on 14 July 2008 @ 9am

Kramer auto Pingback

[…] Playing with complexity (Click on the titel to view the pdf of the presentation) (Click here to view the notes of the presentation ) Typically, interaction designers focus on making things useful, whereas game designers are […]

Posted by NLGD Foundation | Festival of Games - Kars Alfrink on 21 July 2008 @ 5pm

[…] (a.k.a Leapfrog) brilliant slides and notes on data visualization and information design: Playing with complexity. No Comments, Comment or […]

Posted by Playing With Complexity | idiologie on 30 July 2008 @ 10am

Kramer auto Pingback

[…] Think of these applications of the Ennea and Kombolói-inspired device as variations on Jane McGonigal’s happiness hacking which Kars Alfrink discusses with regards to personal informatics and social interactions. […]

Posted by robertogreco {tumblr} - Ethan Zuckerman writes* about a project from the... on 31 July 2008 @ 8am

“It is important to take the ludic fallacy into account when designing predictive data visualizations. It should be made clear to people that they are looking at a simplification of reality. For instance, perhaps it is wise to show ranges in stead of end points. In this way you communicate a certain level of uncertainty.

A Hybrid Approach

There is promise in this hybrid approach though. Data visualization traditionally draws displays of data about things that have happened. From the visualization one can start seeing patterns. Games have traditionally employed models to generate fictional realities. Any pattern that might arise is encoded in the model. By combining these two, we arrive at data visualizations that describe what has happened, and predict what could happen.”

One model many people use to predict the future is astrology. The stars and planets were in THIS configuration when you were born, and today they are configured like THIS (history); therefore, THIS will happen (the future). Because of this logical error, astrology is largely written off. But we know that all astronomical bodies exert a certain quantifiable gravitational force, however insignificant when measured at a distance, and we can certainly measure the effects these same forces have on us from the larger (the sun) and nearer (the moon) bodies. Lives depend on it! Because the future ranges from the certain (tomorrow the sun will rise) to the uncertain (tomorrow it will rain), we need all the help we can get in trying to predict it. Rather than discarding a set of information for its lack of certainty, we should examine what it may say about TENDENCY.

Thanks for a thought-provoking piece.

Posted by Gary Newman on 5 August 2008 @ 8pm

Thanks for dropping by and commentin Gary. I enjoyed your astrology example.

Posted by Kars on 6 August 2008 @ 10am

[…] be fostered and leveraged in areas besides gaming too, IMHO. This point was at the core of my Playing With Complexity […]

Posted by Leapfroglog - Playing with emergence is like gardening on 22 August 2008 @ 3pm

Kramer auto Pingback

[…] What I did like, was Aleks’ mentioning the new wave of computational dataviz as a an example of playful acts on the web. But that will probably come as no surprise. […]

Posted by Matt Balara - Freelance Designer : “ dConstruct Round-up” Sep. 10th, 2008 on 19 September 2008 @ 5pm

Kramer auto Pingback

[…] Leapfroglog – Playing With Complexity — slides and notes for my NLGD Festival of Games talk SAVE […]

Posted by sschraml's subscriptions on Delicious on 8 October 2008 @ 10am