Playing With Complexity — slides and notes for my NLGD Festival of Games talk

When the NLGD Foun­da­tion invit­ed me to speak at their anu­al Fes­ti­val of Games I asked them what they would like me to dis­cuss. “Any­thing you like,” was what they said, essen­tial­ly. I decid­ed to sub­mit an abstract deal­ing with data visu­al­iza­tion. I had been pay­ing more and more atten­tion to this field, but was unsuc­cess­ful in relat­ing it the oth­er themes run­ning through my work, most notably play. So I thought I’d force myself to tack­le this issue by promis­ing to speak about it. Often a good strat­e­gy, I’ve found. If it worked out this time I leave for you to judge.

In brief, in the pre­sen­ta­tion I argue two things: one — that the more sophis­ti­cat­ed appli­ca­tions of inter­ac­tive data visu­al­iza­tion resem­ble games and toys in many ways, and two — that game design can con­tribute to the solu­tions to sev­er­al design issues I have detect­ed in the field of data visu­al­iza­tion.

Below are the notes for the talk, slight­ly edit­ed, and with ref­er­ences includ­ed. The full deck of slides, which includes cred­its for all the images used, is up on SlideShare.

Hel­lo every­one, my name is Kars Alfrink. I am a Dutch inter­ac­tion design­er and I work free­lance. At the moment I work in Copen­hagen, but pret­ty soon I will be back here in Utrecht, my love­ly home­town.

In my work I focus on three areas: mobil­i­ty, social inter­ac­tions, and play. Here is an exam­ple of my work: These are sto­ry­boards that explore pos­si­ble appli­ca­tions of mul­ti­touch tech­nol­o­gy in a gat­ed com­mu­ni­ty. Using these tech­nolo­gies I tried to com­pen­sate for the neg­a­tive effects a gat­ed com­mu­ni­ty has on the build-up of social cap­i­tal. I also tried to bal­ance ‘being-in-the-screen’ with ‘being-in-the-world’ — mul­ti­touch tech­nolo­gies tend to be very atten­tion-absorb­ing, but in built envi­ron­ments this is often not desir­able.1

I am not going to talk about mul­ti­touch though. Today’s top­ic is data visu­al­iza­tion and what oppor­tu­ni­ties there are for game design­ers in that field. My talk is rough­ly divid­ed in three parts. First, I will briefly describe what I think data visu­al­iza­tion is. Next, I will look at some appli­ca­tions beyond the very obvi­ous. Third and last, I will dis­cuss some design issues involved with data visu­al­iza­tion. For each of these issues, I will show how game design can con­tribute.

Right, let’s get start­ed.

1. What It Is

Data visu­al­iza­tion is admit­ted­ly a broad term, that can be used for many things. There are also close­ly relat­ed terms, such as infor­ma­tion visu­al­iza­tion, and infor­ma­tion design. Data visu­al­iza­tion seems to be the most com­mon term though, often short­ened to dataviz. One impor­tant qual­i­fi­ca­tion is that I am focused on visu­al­iza­tions that involve at least some lev­el of inter­ac­tiv­i­ty.

Making Visible the Invisible

In his book The Ghost Map, Steven Berlin John­son writes about a cholera out­break in Vic­to­ri­an Lon­don. One of the book’s cen­tral pro­tag­o­nists, Dr. John Snow, cre­ates a map of the cholera cas­es to help show that the source of the out­break is a water pump. Here’s part of that map. I’ve marked the loca­tion of the pump with a red cir­cle. For each case, Snow drew a black dash at the loca­tion of its res­i­dence. You can see clear­ly that the cas­es dimin­ish the fur­ther you move away from the pump. You have to remem­ber that in those days, peo­ple still believed that dis­eases such as cholera and the Black Death were spread through the air — the so-called mias­ma the­o­ry. The germ the­o­ry had not yet been defin­i­tive­ly estab­lished.

This is an excel­lent exam­ple of show­ing things that can­not be seen. This is one of the defin­ing qual­i­ties of data visu­al­iza­tion for me.

Many Shapes and Sizes

Show­ing the invis­i­ble can be done in many ways, and infor­ma­tion design goes back a long way — in the caves of Las­caux there is a map of the heav­ens dat­ing back to 16,500 BC. Over the years, many ways to visu­al­ize data have been devel­oped. You’ve got maps, charts, and graphs of all shapes and sizes. The point is, each data-set requires a spe­cif­ic dis­play. Also, although there have been many estab­lished pat­terns, it is of course com­plete­ly legit­i­mate to exper­i­ment with new visu­al­iza­tions.

A Medium, Not a Technique

This is some­thing the peo­ple at San Fran­cis­co-based design and tech­nol­o­gy stu­dio Sta­men often say. Data visu­al­iza­tion is not just a tech­nique — a way of doing things. It is also a medi­um — a way of con­vey­ing mes­sages. From their site comes this quote:2

It’s not quite honesty–you can lie with charts and graphs as eas­i­ly as with words or pic­tures. It’s not quite acci­den­tal discovery–this is pret­ty tech­ni­cal stuff, and delib­er­ate choic­es are made at every step along the way.”

For instance, Sta­men have cre­at­ed some­thing called Oak­land Crimespot­ting. On this site you can explore a map of crimes com­mit­ted in Oak­land. When out­siders see this, they tend to react with shock, say­ing they would nev­er want to live in Oak­land. This was nev­er the inten­tion of the Sta­men peo­ple though, and if you look close­ly at the pat­terns in the data, you dis­cov­er that con­ven­tion­al wis­dom about such things as which neigh­bor­hood is the most dan­ger­ous is often wrong.

This is an exam­ple of an argu­ment made using data visu­al­iza­tion. In the gam­ing world, I think this is mir­rored by con­cept of pro­ce­dur­al rhetoric as described by Ian Bogost: Using game mod­els to con­vince peo­ple of your point of view. Both show that tech­nol­o­gy is not neu­tral, that nei­ther games nor data visu­al­iza­tion are just tech­niques, but media.

2. Non-Obvious Applications

Hope­ful­ly that has clar­i­fied what I mean when I say “data visu­al­iza­tion”. To make things more con­crete I’d like to look at two broad areas of appli­ca­tion, beyond the well-known infor­ma­tion graph­ics as we encounter them in print media. The first is the idea of mak­ing com­plex things more under­stand­able. The sec­ond is this idea of improv­ing people’s lives through data visu­al­iza­tions.

Understanding Complexity

I think we can all agree the world is becom­ing more com­plex. But in this case I am talk­ing about a spe­cif­ic kind of com­plex­i­ty known as orga­nized com­plex­i­ty. This is when in a sys­tem — be it bio­log­i­cal, tech­no­log­i­cal, eco­nom­ic or some­thing else — a rel­a­tive­ly small num­ber of parts inter­act in a non­triv­ial way. Such sys­tems dis­play behav­ior that is not car­ried by the parts, so the orga­nized char­ac­ter of the sys­tem emerges, with­out top-down con­trol.

Many of the sys­tems we find our­selves sur­round­ed with these days are like this. The prob­lem is that old-style mod­els from things like sta­tis­tics aren’t use­ful when deal­ing with them. What we can do is show the data that results from such sys­tems. We can then lit­er­al­ly see the emer­gent pat­terns. Going one step fur­ther, by mak­ing these visu­al­iza­tions inter­ac­tive, we can allow peo­ple to play with com­plex­i­ty — explor­ing the space of pos­si­bil­i­ties (a term that should be famil­iar to game design­ers).

For exam­ple, this is a piece by Ben Fry, one of the ini­tia­tors of the open source project Pro­cess­ing. It is a visu­al­iza­tion of traf­fic in Los Ange­les. It is meant to be manip­u­lat­ed with a ges­tur­al inter­face known as G-Speak (which was devel­oped from ini­tial con­cepts for the film Minor­i­ty Report). The many degrees of free­dom peo­ple have with a visu­al­iza­tion like this allows them to get a deep under­stand­ing of the com­plex sys­tem that is urban traf­fic.

Clear­ly, game-like inter­ac­tions are very appro­pri­ate for these kinds of visu­al­iza­tions.

Happiness Hacking

A hack is a clever solu­tion to a prob­lem, often in the form of a short­cut. I first came across the idea of hap­pi­ness hack­ing on Jane McGonigal’s blog. She’s work­ing towards the “goal of using new sci­en­tif­ic research on well-being to devel­op tech­no­log­i­cal sys­tems that actu­al­ly improve qual­i­ty of life”.3 There’s a lot more to her cru­sade, I encour­age you to check out her ETech pre­sen­ta­tion on the top­ic.

This idea of hap­pi­ness hack­ing is extendible to oth­er areas too. For instance, there is a whole sub­cul­ture of life­hack­ers who are most­ly focused on boost­ing their pro­duc­tiv­i­ty using clever tricks. When I was writ­ing this pre­sen­ta­tion, I used a hack called ten-plus-two-times-five, which was invent­ed by life hack­ing guru Mer­lin Mann. The idea is that you set a timer for ten min­utes, dur­ing which you only focus on work. Then, as a reward, you get two min­utes to play, and then it’s back to work again. Repeat this five times for an hour of being pro­duc­tive for 50 min­utes. Not bad. It works for me!

What this has to do with data visu­al­iza­tion is per­haps best exem­pli­fied by the Apgar score. This is a way of mea­sur­ing the con­di­tion of a baby after birth. When this scor­ing was intro­duced after its inven­tion in 1952, the mor­tal­i­ty of new­borns dra­mat­i­cal­ly decreased. This was because there were now num­bers that could be col­lect­ed and com­pared. Doc­tors could now mea­sure what effect dif­fer­ent pro­ce­dures had on the health of babies. Mea­sur­ing things changes them.

So now imag­ine you could quan­ti­fy things that relate to qual­i­ty of life — to hap­pi­ness — and visu­al­ize those.

This can be done on the indi­vid­ual lev­el. Res­cue­Time is a web appli­ca­tion that keeps track of what you do on your com­put­er. You can tell it what types of activ­i­ties you find pro­duc­tive or not, and it’ll visu­al­ize how you’re doing over time. This is what Matt Jones calls per­son­al infor­mat­ics, and there are more and more exam­ples pop­ping up. An old­er one being Body­Media’s SenseWear Weight Man­age­ment Sys­tem. Each of these is an exam­ple of hap­pi­ness hack­ing using data visu­al­iza­tion. Like in games, it allows peo­ple to man­age their resources.

By com­bin­ing indi­vid­ual data-sets, we will be able to get a sense of col­lec­tive pat­terns. We are all par­tic­i­pants in com­plex sys­tems our­selves, after all, and we aren’t par­tic­u­lar­ly good at per­ceiv­ing the high-order con­se­quences of our actions. Matt Jones has this to say about it:4

The over­lays of these pat­terns with those of oth­ers are a new kind of feed­back we haven’t had at any scale before. And we do flock well. So per­haps that’s how we will learn and change our behav­iours… in a “super­con­text” if you will…”

Super­con­text is a term tak­en from the com­ic book series The Invis­i­bles. There, it is a state-of-being human­i­ty is evolv­ing towards, where all is ambigu­ous and non-bina­ry.

There aren’t many true col­lec­tive­ly intel­li­gent data visu­al­iza­tions out there yet.5 I am con­fi­dent we will see them arise though. When they do, they will require game design. Com­pared to inter­ac­tion design­ers, game design­ers are much more com­fort­able with com­plex­i­ty and indi­rect con­trol, which is exact­ly what these sys­tems will require. Wouldn’t you like to be involved with bring­ing joy to the world?

3. Design Issues

In this last sec­tion, I’d like to look at some design issues with data visu­al­iza­tion. There is an ongo­ing debate about the role of aes­thet­ics — is it impor­tant dis­plays look pret­ty, or not? Also, inter­ac­tive data visu­al­iza­tions tend to employ unortho­dox inter­ac­tion styles. Should util­i­ty trump explorabil­i­ty? Final­ly, is it enough to describe past data or should we also pre­dict the future? I’ll show that for each of these ques­tions, game design can con­tribute to the answer.

The Role of Aesthetics

In the fall of last year, Sta­men launched Twit­ter Blocks. For those of you who are not on Twit­ter: It is a so-called microblog­ging ser­vice, where you leave short mes­sages about what you are doing for your friends and fol­low­ers. You can do this using web, IM, SMS or oth­er chan­nels. This is what my Twit­ter stream looks like:

Blocks was cre­at­ed by Sta­men for Twit­ter. It is a 3D visu­al­iza­tion of your friends’ sta­tus mes­sages. From each friend’s mes­sage anoth­er stream is drawn. So you get to see your friends’ friends too. I guess it sounds com­pli­cat­ed if you don’t use the ser­vice. The point is that this way you get to have a sense of what’s hap­pen­ing on Twit­ter beyond your own stream. Any­way, here’s what it looks like:

When this launched, it was met with quite a bit of crit­i­cism. A lot of it basi­cal­ly boiled down to: “This looks pret­ty, but it’s use­less.” The preva­lent idea amongst peo­ple still seems to be that aes­thet­ics is sub­or­di­nate to util­i­ty. I thought we had got­ten over this, design­ers in fields such as prod­uct design and archi­tec­ture have sat­is­fac­to­ri­ly proven that if some­thing is pret­ty, it is per­ceived to work bet­ter.

Andrew Vande Moere writes in a paper about this very top­ic (PDF):

[…] infor­ma­tion visu­al­iza­tion should be enriched with the prin­ci­ples of cre­ative design and art, to devel­op valu­able data rep­re­sen­ta­tions that address the emo­tion­al expe­ri­ence and engage­ment of users, instead of sole­ly focus­ing on task effec­tive­ness met­rics […]”

The point being, aes­thet­ics are ‘use­ful’ in their own right. They can make the same func­tion­al expe­ri­ence be per­ceived as whol­ly dif­fer­ent. I believe you as game design­ers under­stand this very deeply. The role of fic­tion, dress­ing etc. in games is promi­nent. No-one would argue that a game with the same mechan­ics but dras­ti­cal­ly dif­fer­ent aes­thet­ics is the same game. It is not. Prop­er­ly exe­cut­ed aes­thet­ics are vital to the expe­ri­ence.

Utility Versus Frivolity

Let’s exam­ine the crit­i­cisms received by Twit­ter Blocks a bit more. A lot of peo­ple com­plained about its use­less­ness. One of the design­ers of Blocks is Tom Car­den. He writes in a response to the crit­i­cism on his blog:

[…] why not also accept that some things might just be for enter­tain­ment and ask “am I hav­ing fun” once in a while instead of look­ing for a prob­lem to be solved or an impor­tant state­ment to be read? Some things just are.”

The irony is that Twit­ter itself isn’t exact­ly a hard­core util­i­tar­i­an expe­ri­ence. To a large extent it can be seen as a toy, and a lot of what peo­ple do with it is very friv­o­lous.

In game design there have been big advances in the under­stand­ing of why peo­ple play. There is for instance a mod­el called PENSthe play­er expe­ri­ence of need sat­is­fac­tion — which shows that one fla­vor of fun results from a feel­ing of auton­o­my — the feel­ing of free­dom to explore some­thing on your own terms. I think Twit­ter Blocks’ fun is like this.

An Expressive Language

Blocks’ inter­face is unortho­dox, it is explorato­ry. Oth­er exam­ples I’ve shown before have this same qual­i­ty — inter­ac­tions that might need some get­ting used to, but once you do are quite pow­er­ful. I tend to think of this as learn­ing an expres­sive lan­guage, and games are par­tic­u­lar­ly good at it.

Take Por­tal, for instance, where you are grad­u­al­ly taught the many uses of one sin­gle mechan­ic. The won­der­ful thing about Por­tal is that it expands your think­ing in an almost sub­lim­i­nal way. Some peo­ple have gone as far as call­ing it a sub­ver­sive game.

[…] games are [not] a way to con­vey and direct­ly put con­tent in play­ers’ brain but rather that the cog­ni­tive process­es mobi­lized when play­ing games can cre­ate rel­e­vant rou­tines that may pos­si­bly be trans­fered to oth­er activ­i­ties […]”

This is Nico­las Nova writ­ing about a pre­sen­ta­tion by Ben Cer­ve­ny at Pic­nic 07. Ben is an advi­sor to Sta­men. He talks a lot about the trans­for­ma­tive poten­tial of games.

My point here is that explorato­ry inter­ac­tions not only con­tribute to a fun expe­ri­ence but also facil­i­tate a kind of deep under­stand­ing. Peo­ple grad­u­al­ly mas­ter an expres­sive lan­guage that allows them to think about com­plex things in a new way.

From Description to Prediction

Let’s move on to a slight­ly dif­fer­ent top­ic. A while ago my friend Alexan­dra, who’s a design­er and also CEO of phys­i­cal com­put­ing stu­dio Tinker.it, blogged about her dis­ap­point­ment with the cur­rent wave of data visu­al­iza­tions. She’s annoyed that most dis­plays don’t go fur­ther than show­ing you the data.

For instance, this is my car­bon out­put from flights over the past year or so. It is cal­cu­lat­ed by Dopplr — a social tool for opti­miz­ing trav­el. (I apol­o­gize for the hor­ren­dous out­put and promise to improve over the com­ing year.) The issue here is that from this dis­play I can only say: “Wow, that’s a lot.” There are two ways to improve this: One — extrap­o­late the data into the future, giv­ing me a sense of what my actions might result in on the long term. And two — offer me ways to mit­i­gate the effects of my actions. Let’s start by focussing on the first.

Alexan­dra writes:6

I would like to see us move towards a world full of lit­tle every­day objects that give me a glimpse into the future if I keep doing things the way I do, total year­ly bills based on my cur­rent usage, pre­dic­tions about how much I’ll have to spend on food and how much weight I’ll gain if I keep at the cur­rent pace.”

In stead of only describ­ing the sta­tus quo, Alexan­dra would like data visu­al­iza­tions to also pre­dict the future. I like this idea, because it is a non-pre­scrip­tive way of moti­vat­ing peo­ple to change their behav­ior.

What would we need to do this? We would need to start using mod­els in addi­tion to data. Mod­els of the phe­nom­e­na we are show­ing data of, so that we can gen­er­ate pos­si­ble future out­comes. There­in though lies a dan­ger — what if the pre­dic­tions turn out to be wrong?

The Ludic Fallacy

In his book The Black Swan, Nas­sim Nicholas Taleb describes what he calls the ludic fal­la­cy — “the mis­use of games to mod­el real-life sit­u­a­tions”. In oth­er words it is mis­tak­ing the map for the ter­ri­to­ry. The prob­lems with pre­dict­ing the future using mod­els are many: It is impos­si­ble to know every­thing (to have all the data), it is very hard to account for ampli­fied effects of small changes (a.k.a. the but­ter­fly effect), and any mod­el we employ can only be based on pri­or expe­ri­ence.

As Yogi Berra report­ed­ly said:

It is tough to make pre­dic­tions, espe­cial­ly about the future.”

It is impor­tant to take the ludic fal­la­cy into account when design­ing pre­dic­tive data visu­al­iza­tions. It should be made clear to peo­ple that they are look­ing at a sim­pli­fi­ca­tion of real­i­ty. For instance, per­haps it is wise to show ranges in stead of end points. In this way you com­mu­ni­cate a cer­tain lev­el of uncer­tain­ty.

A Hybrid Approach

There is promise in this hybrid approach though. Data visu­al­iza­tion tra­di­tion­al­ly draws dis­plays of data about things that have hap­pened. From the visu­al­iza­tion one can start see­ing pat­terns. Games have tra­di­tion­al­ly employed mod­els to gen­er­ate fic­tion­al real­i­ties. Any pat­tern that might arise is encod­ed in the mod­el. By com­bin­ing these two, we arrive at data visu­al­iza­tions that describe what has hap­pened, and pre­dict what could hap­pen.

Final­ly, because we have a mod­el, we can now also make the pos­si­ble effects of mit­i­gat­ing actions per­ceiv­able. Imag­ine hav­ing a dis­play of my past and future car­bon, where I can play with dif­fer­ent ways of off­set­ting it — fly­ing less, plant­i­ng trees, etc. The visu­al­iza­tion would show me in real-time how much each method would affect my out­put, and per­haps relate it to finan­cials too, so that I can fig­ure out what the most cost-effec­tive way of off­set­ting is.

I think this is an obvi­ous type of ‘new’ data visu­al­iza­tion that can ben­e­fit from game design­ers’ involve­ment.

I hope I have shown today that data visu­al­iza­tion is a field that offers inter­est­ing oppor­tu­ni­ties. It can clear­ly ben­e­fit from the expe­ri­ence of game design­ers. If you do decide to get involved with data visu­al­iza­tions, I am look­ing for­ward to see­ing your cre­ations!

Thank you.

  1. For more back­ground on this project please see this old­er blog post. More exam­ples of my recent work can be found in my port­fo­lio. []
  2. Tak­en from the page Data Visu­al­iza­tion. []
  3. Quote tak­en from the blog post Work, Work, Work — How I Spent My 2007, or, a Year in Review []
  4. This quote is tak­en from a long but excel­lent inter­view with Matt Jones on Ryan Freitas’s blog which dis­cuss­es per­son­al infor­mat­ics amongst many oth­er things. []
  5. One exam­ple I stum­bled across just before deliv­er­ing this talk is City­sense. []
  6. This quote is tak­en from her blog post Pre­scrip­tive or pre­dic­tive infor­ma­tion visu­al­i­sa­tion?. []

Published by

Kars Alfrink

Kars is a designer, researcher and educator focused on emerging technologies, social progress and the built environment.

14 thoughts on “Playing With Complexity — slides and notes for my NLGD Festival of Games talk”

  1. Lots of week­end read­ing for me here.

    The prob­lem of pre­dict­ing the future is age old. Otto Scharmer and co of MIT write on it exten­sive­ly. One of his col­leagues worked with Shell on sce­nario build­ing. Explor­ing the com­plex­i­ty rather than pre­dict­ing.

    This week I was check­ing on the def­i­n­i­tions of final and effi­cient caus­es. I think some­times we get fix­at­ed on believ­ing some­thing must or will hap­pen rather than look­ing at the moti­va­tions of every­thing (includ­ing non human enti­ties) and work­ing out what we would like to bring about.

    Great post. Hap­py home­work for me.

  2. Gee, thanks Jo, I am glad I was able to pro­vide you with some food for thought.

    I see Mr. Scharmer has (co-)written a few books. Any one in par­tic­u­lar you can rec­om­mend?

  3. It is impor­tant to take the ludic fal­la­cy into account when design­ing pre­dic­tive data visu­al­iza­tions. It should be made clear to peo­ple that they are look­ing at a sim­pli­fi­ca­tion of real­i­ty. For instance, per­haps it is wise to show ranges in stead of end points. In this way you com­mu­ni­cate a cer­tain lev­el of uncer­tain­ty.

    A Hybrid Approach

    There is promise in this hybrid approach though. Data visu­al­iza­tion tra­di­tion­al­ly draws dis­plays of data about things that have hap­pened. From the visu­al­iza­tion one can start see­ing pat­terns. Games have tra­di­tion­al­ly employed mod­els to gen­er­ate fic­tion­al real­i­ties. Any pat­tern that might arise is encod­ed in the mod­el. By com­bin­ing these two, we arrive at data visu­al­iza­tions that describe what has hap­pened, and pre­dict what could hap­pen.”

    One mod­el many peo­ple use to pre­dict the future is astrol­o­gy. The stars and plan­ets were in THIS con­fig­u­ra­tion when you were born, and today they are con­fig­ured like THIS (his­to­ry); there­fore, THIS will hap­pen (the future). Because of this log­i­cal error, astrol­o­gy is large­ly writ­ten off. But we know that all astro­nom­i­cal bod­ies exert a cer­tain quan­tifi­able grav­i­ta­tion­al force, how­ev­er insignif­i­cant when mea­sured at a dis­tance, and we can cer­tain­ly mea­sure the effects these same forces have on us from the larg­er (the sun) and near­er (the moon) bod­ies. Lives depend on it! Because the future ranges from the cer­tain (tomor­row the sun will rise) to the uncer­tain (tomor­row it will rain), we need all the help we can get in try­ing to pre­dict it. Rather than dis­card­ing a set of infor­ma­tion for its lack of cer­tain­ty, we should exam­ine what it may say about TENDENCY.

    Thanks for a thought-pro­vok­ing piece.

Comments are closed.