Playful Design for Workplace Change Management’ at PLAYTrack conference 2017 in Aarhus

Lase defender collab at FUSE

At the end of last year I was invit­ed to speak at the PLAY­Track con­fer­ence in Aarhus about the work­place change man­age­ment games made by Hub­bub. It turned out to be a great oppor­tu­ni­ty to recon­nect with the play research community. 

I was very much impressed by the pro­gram assem­bled by the organ­is­ers. Peo­ple came from a wide range of dis­ci­plines and cru­cial­ly, there was ample time to dis­cuss and reflect on the mate­ri­als pre­sent­ed. As I tweet­ed after­wards, this is a thing that most con­fer­ence organ­is­ers get wrong.

I was par­tic­u­lar­ly inspired by the work of Ben­jamin Mardell and Mara Krechevsky at Harvard’s Project ZeroMak­ing Learn­ing Vis­i­ble looks like a great resource for any­one who teach­es. Then there was Reed Stevens from North­west­ern Uni­ver­si­ty whose project FUSE is one of the most sol­id exam­ples of play­ful learn­ing for STEAM I’ve seen thus far. I was also fas­ci­nat­ed by Cia­ra Laverty’s work at PEDAL on observ­ing par­ent-child play. Miguel Sicart deliv­ered anoth­er great provo­ca­tion on the dark side of play­ful design. And final­ly I was delight­ed to hear about and expe­ri­ence for myself some of Amos Blan­ton’s work at the LEGO Foun­da­tion. I should also call out Ben Fin­cham’s many provoca­tive con­tri­bu­tions from the audience.

The abstract for my talk is below, which cov­ers most of what I talked about. I tried to give peo­ple a good sense of: 

  • what the games con­sist­ed of,
  • what we were aim­ing to achieve,
  • how both the fic­tion and the play­er activ­i­ties sup­port­ed these goals,
  • how we made learn­ing out­comes vis­i­ble to our play­ers and clients,
  • and final­ly how we went about design­ing and devel­op­ing these games.

Both projects have sol­id write-ups over at the Hub­bub web­site, so I’ll just point to those here: Code 4 and Rip­ple Effect.

In the final sec­tion of the talk I spent a bit of time reflect­ing on how I would approach projects like this today. After all, it has been sev­en years since we made Code 4, and four years since Rip­ple Effect. That’s ages ago and my per­spec­tive has def­i­nite­ly changes since we made these.

Participatory design

First of all, I would get even more seri­ous about co-design­ing with play­ers at every step. I would recruit rep­re­sen­ta­tives of play­ers and invest them with real influ­ence. In the projects we did, the pri­ma­ry vehi­cle for play­er influ­ence was through playtest­ing. But this is nec­es­sar­i­ly lim­it­ed. I also won’t pre­tend this is at all easy to do in a com­mer­cial context. 

But, these games are ulti­mate­ly about improv­ing work­er pro­duc­tiv­i­ty. So how do we make it so that work­ers share in the real-world prof­its yield­ed by a suc­cess­ful cul­ture change?

I know of the exis­tence of par­tic­i­pa­to­ry design but from my expe­ri­ence it is not a com­mon approach in the indus­try. Why?

Value sensitive design

On a relat­ed note, I would get more seri­ous about what val­ues are sup­port­ed by the sys­tem, in whose inter­est they are and where they come from. Ear­ly field research and work­shops with audi­ence do sur­face some val­ues but val­ues from cus­tomer rep­re­sen­ta­tives tend to dom­i­nate. Again, the com­mer­cial con­text we work in is a poten­tial challenge. 

I know of val­ue sen­si­tive design, but as with par­tic­i­pa­to­ry design, it has yet to catch on in a big way in the indus­try. So again, why is that?

Disintermediation

One thing I con­tin­ue to be inter­est­ed in is to reduce the com­plex­i­ty of a game system’s phys­i­cal affor­dances (which includes its code), and to push even more of the sub­stance of the game into those social allowances that make up the non-mate­r­i­al aspects of the game. This allows for spon­ta­neous rene­go­ti­a­tion of the game by the play­ers. This is dis­in­ter­me­di­a­tion as a strat­e­gy. David Kanaga’s take on games as toys remains huge­ly inspi­ra­tional in this regard, as does Bernard De Koven’s book The Well Played Game.

Gamefulness versus playfulness

Code 4 had more focus on sat­is­fy­ing the need for auton­o­my. Rip­ple Effect had more focus on com­pe­tence, or in any case, it had less empha­sis on auton­o­my. There was less room for ‘play’ around the core dig­i­tal game. It seems to me that mas­ter­ing a sub­jec­tive sim­u­la­tion of a sub­ject is not nec­es­sar­i­ly what a work­place game for cul­ture change should be aim­ing for. So, less game­ful design, more play­ful design.

Adaptation

Final­ly, the agency mod­el does not enable us to stick around for the long haul. But work­place games might be bet­ter suit­ed to a set­up where things aren’t thought of as a one-off project but more of an ongo­ing process. 

In How Build­ings Learn, Stew­art Brand talks about how archi­tects should revis­it build­ings they’ve designed after they are built to learn about how peo­ple are actu­al­ly using them. He also talks about how good build­ings are build­ings that its inhab­i­tants can adapt to their needs. What does that look like in the con­text of a game for work­place cul­ture change?


Play­ful Design for Work­place Change Management

Code 4 (2011, com­mis­sioned by the Tax Admin­is­tra­tion of the Nether­lands) and Rip­ple Effect (2013, com­mis­sioned by Roy­al Dutch Shell) are both games for work­place change man­age­ment designed and devel­oped by Hub­bub, a bou­tique play­ful design agency which oper­at­ed from Utrecht, The Nether­lands and Berlin, Ger­many between 2009 and 2015. These games are exam­ples of how a goal-ori­ent­ed seri­ous game can be used to encour­age play­ful appro­pri­a­tion of work­place infra­struc­ture and social norms, result­ing in an open-end­ed and cre­ative explo­ration of new and inno­v­a­tive ways of working.

Seri­ous game projects are usu­al­ly com­mis­sioned to solve prob­lems. Solv­ing the prob­lem of cul­tur­al change in a straight­for­ward man­ner means view­ing games as a way to per­suade work­ers of a desired future state. They typ­i­cal­ly take videogame form, sim­u­lat­ing the desired new way of work­ing as deter­mined by man­age­ment. To play the game well, play­ers need to mas­ter its sys­tem and by extension—it is assumed—learning happens.

These games can be be enjoy­able expe­ri­ences and an improve­ment on pre­vi­ous forms of work­place learn­ing, but in our view they decrease the pos­si­bil­i­ty space of poten­tial work­place cul­tur­al change. They dimin­ish work­er agency, and they waste the cre­ative and inno­v­a­tive poten­tial of involv­ing them in the inven­tion of an improved work­place culture. 

We instead choose to view work­place games as an oppor­tu­ni­ty to increase the space of pos­si­bil­i­ty. We resist the temp­ta­tion to bake the desired new way of work­ing into the game’s phys­i­cal and dig­i­tal affor­dances. Instead, we leave how to play well up to the play­ers. Since these games are team-based and col­lab­o­ra­tive, play­ers need to nego­ti­ate their way of work­ing around the game among them­selves. In addi­tion, because the games are dis­trib­uted in time—running over a num­ber of weeks—and are playable at play­er dis­cre­tion dur­ing the work­day, play­ers are giv­en license to appro­pri­ate work­place infra­struc­ture and sub­vert social norms towards in-game ends.

We tried to make learn­ing tan­gi­ble in var­i­ous ways. Because the games at the core are web appli­ca­tions to which play­ers log on with indi­vid­ual accounts we were able to col­lect data on play­er behav­iour. To guar­an­tee pri­va­cy, employ­ers did not have direct access to game data­bas­es and only received anonymised reports. We took respon­si­bil­i­ty for play­er learn­ing by facil­i­tat­ing coach­ing ses­sions in which they could safe­ly reflect on their game expe­ri­ences. Round­ing out these efforts, we con­duct­ed sur­veys to gain insight into the play­er expe­ri­ence from a more qual­i­ta­tive and sub­jec­tive perspective.

These games offer a mod­el for a rea­son­ably demo­c­ra­t­ic and eth­i­cal way of doing game-based work­place change man­age­ment. How­ev­er, we would like to see efforts that fur­ther democ­ra­tise their design and development—involving work­ers at every step. We also wor­ry about how games can be used to cre­ate the illu­sion of work­er influ­ence while at the same time soft­ware is deployed through­out the work­place to lim­it their agency. 

Our exam­ples may be inspir­ing but because of these devel­op­ments we feel we can’t con­tin­ue this type of work with­out seri­ous­ly recon­sid­er­ing our cur­rent process­es, tech­nol­o­gy stacks and busi­ness practices—and ulti­mate­ly whether we should be mak­ing games at all.

Books I’ve read in 2017

Return­ing to what is some­thing of an annu­al tra­di­tion, these are the books I’ve read in 2017. I set myself the goal of get­ting to 36 and man­aged 38 in the end. They’re list­ed below with some com­men­tary on par­tic­u­lar­ly mem­o­rable or oth­er­wise note­wor­thy reads. To make things a bit more user friend­ly I’ve gone with four broad buck­ets although as you’ll see with­in each the picks range across gen­res and subjects.

Fiction

I always have one piece of fic­tion or nar­ra­tive non-fic­tion going. I have a long-stand­ing ‘project’ of read­ing cult clas­sics. I can’t set­tle on a top pick for the first cat­e­go­ry so it’s going to have to be a tie between Lowry’s alco­hol-drenched tale of lost love in pre-WWII Mex­i­co, and Salter’s unmatched lyri­cal prose treat­ment of a young couple’s liaisons as imag­ined by a lech­er­ous recluse in post-WWII France.

When I feel like some­thing lighter I tend to seek out sci-fi writ­ten from before I was born. (Con­tem­po­rary sci-fi more often than not dis­ap­points me with its lack of imag­i­na­tion, or worse, nos­tal­gia for futures past. I’m look­ing at you, Cline.) My top pick here would be the Stru­gatsky broth­ers, who blew me away with their weird tale of a world for­ev­er changed by the inex­plic­a­ble vis­it by some­thing tru­ly alien.

I’ve also con­tin­ued to seek out works by women, although I’ve been less strict with myself in this depart­ment than pre­vi­ous years. Here I’m ashamed to admit it took me this long to final­ly read any­thing by Woolf because Mrs Dal­loway is every bit as good as they say it is. I rec­om­mend seek­ing out the anno­tat­ed Pen­guin addi­tion for addi­tion­al insights into the many things she references.

I’ve also some­times picked up a new­er book because it popped up on my radar and I was just real­ly excit­ed about read­ing it. Most notably Dolan’s retelling of the Ili­ad in all its glo­ri­ous, sad and gory detail, updat­ed for today’s sensibilities.

Literary non-fiction

Each time I read a nar­ra­tive treat­ment of his­to­ry or cur­rent affairs I feel like I should be doing more of it. All of these are rec­om­mend­ed but Kapuś­cińs­ki tow­ers over all with his heart-wrench­ing first-per­son account of the Iran­ian revolution.

Non-fiction

A few books on design and tech­nol­o­gy here, although most of my ‘pro­fes­sion­al’ read­ing was con­fined to aca­d­e­m­ic papers this year. I find those to be a more effec­tive way of get­ting a han­dle on a par­tic­u­lar sub­ject. Books pub­lished on my méti­er are noto­ri­ous­ly fluffy. I’ll point out Löw­gren for a tough but reward­ing read on how to do inter­ac­tion design in a non-dog­mat­ic but reflec­tive way.

I got into left­ist pol­i­tics quite heav­i­ly this year and tried to edu­cate myself a bit on con­tem­po­rary anti-cap­i­tal­ist think­ing. Fisher’s book is a most inter­est­ing and also amus­ing diag­no­sis of the cur­rent polit­i­cal and eco­nom­ic world sys­tem through a cul­tur­al lens. It’s a shame he’s no longer with us, I won­der what he would have made of recent events.

Game books

I decid­ed to work my way through a bunch of role­play­ing game books all ‘pow­ered by the apoc­a­lypse’ – a fam­i­ly of games which I have been aware of for quite a while but haven’t had the oppor­tu­ni­ty to play myself. I like read­ing these because I find them odd­ly inspi­ra­tional for pro­fes­sion­al pur­pos­es. But I will point to the orig­i­nal Apoc­a­lypse World as the one must-read as Bak­er remains one of the design­ers I am absolute­ly in awe of for the ways in which he man­ages to com­bine sys­tem and fic­tion in tru­ly inven­tive ways.

  • The Per­ilous Wilds, Jason Lutes
  • Urban Shad­ows: Polit­i­cal Urban Fan­ta­sy Pow­ered by the Apoc­a­lypse, Andrew Medeiros
  • Dun­geon World, Sage LaTorra
  • Apoc­a­lypse World, D. Vin­cent Baker

Poetry

I don’t usu­al­ly read poet­ry for rea­sons sim­i­lar to how I basi­cal­ly stopped read­ing comics ear­li­er: I can’t seem to find a good way of dis­cov­er­ing worth­while things to read. The col­lec­tion below was a gift, and a delight­ful one.

As always, I wel­come sug­ges­tions for what to read next. I’m shoot­ing for 36 again this year and plan to pro­ceed rough­ly as I’ve been doing lately—just mean­der from book to book with a bias towards works that are non-anglo, at least as old as I am, and prefer­ably weird or inventive. 

Pre­vi­ous years: 2016, 2015, 2011, 2009.

Prototyping the Useless Butler: Machine Learning for IoT Designers

ThingsCon Amsterdam 2017, photo by nunocruzstreet.com
ThingsCon Ams­ter­dam 2017, pho­to by nunocruzstreet.com

At ThingsCon Ams­ter­dam 2017, Péter and I ran a sec­ond iter­a­tion of our machine learn­ing work­shop. We improved on our first attempt at TU Delft in a num­ber of ways.

  • We pre­pared exam­ple code for com­mu­ni­cat­ing with Wek­ina­tor from a wifi con­nect­ed Arduino MKR1000 over OSC.
  • We cre­at­ed a pre­de­fined bread­board setup.
  • We devel­oped three exer­cis­es, one for each type of Wek­ina­tor out­put: regres­sion, clas­si­fi­ca­tion and dynam­ic time warping.

In con­trast to the first ver­sion, we had two hours to run through the whole thing, in stead of a day… So we had to cut some cor­ners, and dou­bled down on walk­ing par­tic­i­pants through a num­ber of exer­cis­es so that they would come out of it with some read­i­ly applic­a­ble skills. 

We dubbed the work­shop ‘pro­to­typ­ing the use­less but­ler’, with thanks to Philip van Allen for the sug­ges­tion to frame the exer­cis­es around build­ing some­thing non-pro­duc­tive so that the focus was shift­ed to play and exploration.

All of the code, the cir­cuit dia­gram and slides are over on GitHub. But I’ll sum­marise things here.

  1. We spent a very short amount of time intro­duc­ing machine learn­ing. We used Google’s Teach­able Machine as an exam­ple and con­trast­ed reg­u­lar pro­gram­ming with using machine learn­ing algo­rithms to train mod­els. The point was to pro­vide folks with just enough con­cep­tu­al scaf­fold­ing so that the rest of the work­shop would make sense.
  2. We then intro­duced our ‘tool­chain’ which con­sists of Wek­ina­tor, the Arduino MKR1000 mod­ule and the OSC pro­to­col. The aim of this tool­chain is to allow design­ers who work in the IoT space to get a feel for the mate­r­i­al prop­er­ties of machine learn­ing through hands-on tin­ker­ing. We tried to cre­ate a tool­chain with as few mov­ing parts as pos­si­ble, because each addi­tion­al com­po­nent would intro­duce anoth­er point of fail­ure which might require debug­ging. This tool­chain would enable design­ers to either use machine learn­ing to rapid­ly pro­to­type inter­ac­tive behav­iour with min­i­mal or no pro­gram­ming. It can also be used to pro­to­type prod­ucts that expose inter­ac­tive machine learn­ing fea­tures to end users. (For a spec­u­la­tive exam­ple of one such prod­uct, see Bjørn Kar­man­n’s Objec­ti­fi­er.)
  3. Par­tic­i­pants were then asked to set up all the required parts on their own work­sta­tion. A list can be found on the Use­less But­ler GitHub page.
  4. We then pro­ceed­ed to build the cir­cuit. We pro­vid­ed all the com­po­nents and showed a Fritz­ing dia­gram to help peo­ple along. The basic idea of this cir­cuit, the epony­mous use­less but­ler, was to have a suf­fi­cient­ly rich set of inputs and out­puts with which to play, that would suit all three types of Wek­ina­tor out­put. So we set­tled on a pair of pho­tore­sis­tors or LDRs as inputs and an RGB LED as output.
  5. With the pre­req­ui­sites installed and the cir­cuit built we were ready to walk through the exam­ples. For regres­sion we mapped the con­tin­u­ous stream of read­ings from the two LDRs to three out­puts, one each for the red, green and blue of the LED. For clas­si­fi­ca­tion we put the state of both LDRs into one of four cat­e­gories, each switch­ing the RGB LED to a spe­cif­ic col­or (cyan, magen­ta, yel­low or white). And final­ly, for dynam­ic time warp­ing, we asked Wek­ina­tor to recog­nise one of three ges­tures and switch the RGB LED to one of three states (red, green or off).

When we reflect­ed on the work­shop after­wards, we agreed we now have a proven con­cept. Par­tic­i­pants were able to get the tool­chain up and run­ning and could play around with iter­a­tive­ly train­ing and eval­u­at­ing their mod­el until it behaved as intended. 

How­ev­er, there is still quite a bit of room for improve­ment. On a prac­ti­cal note, quite a bit of time was tak­en up by the build­ing of the cir­cuit, which isn’t the point of the work­shop. One way of deal­ing with this is to bring those to a work­shop pre-built. Doing so would enable us to get to the machine learn­ing quick­er and would open up time and space to also engage with the par­tic­i­pants about the point of it all. 

We’re keen on bring­ing this work­shop to more set­tings in future. If we do, I’m sure we’ll find the oppor­tu­ni­ty to improve on things once more and I will report back here.

Many thanks to Iskan­der and the rest of the ThingsCon team for invit­ing us to the conference.

ThingsCon Amsterdam 2017, photo by nunocruzstreet.com
ThingsCon Ams­ter­dam 2017, pho­to by nunocruzstreet.com

Design and machine learning – an annotated reading list

Ear­li­er this year I coached Design for Inter­ac­tion mas­ter stu­dents at Delft Uni­ver­si­ty of Tech­nol­o­gy in the course Research Method­ol­o­gy. The stu­dents organ­ised three sem­i­nars for which I pro­vid­ed the claims and assigned read­ing. In the sem­i­nars they argued about my claims using the Toul­min Mod­el of Argu­men­ta­tion. The read­ings served as sources for back­ing and evidence.

The claims and read­ings were all relat­ed to my nascent research project about machine learn­ing. We delved into both design­ing for machine learn­ing, and using machine learn­ing as a design tool.

Below are the read­ings I assigned, with some notes on each, which should help you decide if you want to dive into them yourself.

Hebron, Patrick. 2016. Machine Learn­ing for Design­ers. Sebastopol: O’Reilly.

The only non-aca­d­e­m­ic piece in this list. This served the pur­pose of get­ting all stu­dents on the same page with regards to what machine learn­ing is, its appli­ca­tions of machine learn­ing in inter­ac­tion design, and com­mon chal­lenges encoun­tered. I still can’t think of any oth­er sin­gle resource that is as good a start­ing point for the sub­ject as this one.

Fiebrink, Rebec­ca. 2016. “Machine Learn­ing as Meta-Instru­ment: Human-Machine Part­ner­ships Shap­ing Expres­sive Instru­men­tal Cre­ation.” In Musi­cal Instru­ments in the 21st Cen­tu­ry, 14:137–51. Sin­ga­pore: Springer Sin­ga­pore. doi:10.1007/978–981–10–2951–6_10.

Fiebrink’s Wek­ina­tor is ground­break­ing, fun and inspir­ing so I had to include some of her writ­ing in this list. This is most­ly of inter­est for those look­ing into the use of machine learn­ing for design and oth­er cre­ative and artis­tic endeav­ours. An impor­tant idea explored here is that tools that make use of (inter­ac­tive, super­vised) machine learn­ing can be thought of as instru­ments. Using such a tool is like play­ing or per­form­ing, explor­ing a pos­si­bil­i­ty space, engag­ing in a dia­logue with the tool. For a tool to feel like an instru­ment requires a tight action-feed­back loop.

Dove, Gra­ham, Kim Hal­skov, Jodi For­l­izzi, and John Zim­mer­man. 2017. UX Design Inno­va­tion: Chal­lenges for Work­ing with Machine Learn­ing as a Design Mate­r­i­al. The 2017 CHI Con­fer­ence. New York, New York, USA: ACM. doi:10.1145/3025453.3025739.

A real­ly good sur­vey of how design­ers cur­rent­ly deal with machine learn­ing. Key take­aways include that in most cas­es, the appli­ca­tion of machine learn­ing is still engi­neer­ing-led as opposed to design-led, which ham­pers the cre­ation of non-obvi­ous machine learn­ing appli­ca­tions. It also makes it hard for design­ers to con­sid­er eth­i­cal impli­ca­tions of design choic­es. A key rea­son for this is that at the moment, pro­to­typ­ing with machine learn­ing is pro­hib­i­tive­ly cumbersome.

Fiebrink, Rebec­ca, Per­ry R Cook, and Dan True­man. 2011. “Human Mod­el Eval­u­a­tion in Inter­ac­tive Super­vised Learn­ing.” In, 147. New York, New York, USA: ACM Press. doi:10.1145/1978942.1978965.

The sec­ond Fiebrink piece in this list, which is more of a deep dive into how peo­ple use Wek­ina­tor. As with the chap­ter list­ed above this is required read­ing for those work­ing on design tools which make use of inter­ac­tive machine learn­ing. An impor­tant find­ing here is that users of intel­li­gent design tools might have very dif­fer­ent cri­te­ria for eval­u­at­ing the ‘cor­rect­ness’ of a trained mod­el than engi­neers do. Such cri­te­ria are like­ly sub­jec­tive and eval­u­a­tion requires first-hand use of the mod­el in real time. 

Bostrom, Nick, and Eliez­er Yud­kowsky. 2014. “The Ethics of Arti­fi­cial Intel­li­gence.” In The Cam­bridge Hand­book of Arti­fi­cial Intel­li­gence, edit­ed by Kei­th Frank­ish and William M Ram­sey, 316–34. Cam­bridge: Cam­bridge Uni­ver­si­ty Press. doi:10.1017/CBO9781139046855.020.

Bostrom is known for his some­what crazy but thought­pro­vok­ing book on super­in­tel­li­gence and although a large part of this chap­ter is about the ethics of gen­er­al arti­fi­cial intel­li­gence (which at the very least is still a way out), the first sec­tion dis­cuss­es the ethics of cur­rent “nar­row” arti­fi­cial intel­li­gence. It makes for a good check­list of things design­ers should keep in mind when they cre­ate new appli­ca­tions of machine learn­ing. Key insight: when a machine learn­ing sys­tem takes on work with social dimensions—tasks pre­vi­ous­ly per­formed by humans—the sys­tem inher­its its social requirements.

Yang, Qian, John Zim­mer­man, Aaron Ste­in­feld, and Antho­ny Toma­sic. 2016. Plan­ning Adap­tive Mobile Expe­ri­ences When Wire­fram­ing. The 2016 ACM Con­fer­ence. New York, New York, USA: ACM. doi:10.1145/2901790.2901858.

Final­ly, a feet-in-the-mud explo­ration of what it actu­al­ly means to design for machine learn­ing with the tools most com­mon­ly used by design­ers today: draw­ings and dia­grams of var­i­ous sorts. In this case the focus is on using machine learn­ing to make an inter­face adap­tive. It includes an inter­est­ing dis­cus­sion of how to bal­ance the use of implic­it and explic­it user inputs for adap­ta­tion, and how to deal with infer­ence errors. Once again the lim­i­ta­tions of cur­rent sketch­ing and pro­to­typ­ing tools is men­tioned, and relat­ed to the need for design­ers to devel­op tac­it knowl­edge about machine learn­ing. Such tac­it knowl­edge will only be gained when design­ers can work with machine learn­ing in a hands-on manner.

Supplemental material

Floyd, Chris­tiane. 1984. “A Sys­tem­at­ic Look at Pro­to­typ­ing.” In Approach­es to Pro­to­typ­ing, 1–18. Berlin, Hei­del­berg: Springer Berlin Hei­del­berg. doi:10.1007/978–3–642–69796–8_1.

I pro­vid­ed this to stu­dents so that they get some addi­tion­al ground­ing in the var­i­ous kinds of pro­to­typ­ing that are out there. It helps to pre­vent reduc­tive notions of pro­to­typ­ing, and it makes for a nice com­ple­ment to Buxton’s work on sketch­ing.

Ble­vis, E, Y Lim, and E Stolter­man. 2006. “Regard­ing Soft­ware as a Mate­r­i­al of Design.”

Some of the papers refer to machine learn­ing as a “design mate­r­i­al” and this paper helps to under­stand what that idea means. Soft­ware is a mate­r­i­al with­out qual­i­ties (it is extreme­ly mal­leable, it can sim­u­late near­ly any­thing). Yet, it helps to con­sid­er it as a phys­i­cal mate­r­i­al in the metaphor­i­cal sense because we can then apply ways of design think­ing and doing to soft­ware programming.

Status update

This is not exact­ly a now page, but I thought I would write up what I am doing at the moment since last report­ing on my sta­tus in my end-of-year report.

The major­i­ty of my work­days are spent doing free­lance design con­sult­ing. My pri­ma­ry gig has been through Eend at the Dutch Vic­tim Sup­port Foun­da­tion, where until very recent­ly I was part of a team build­ing online ser­vices. I helped out with prod­uct strat­e­gy, set­ting up a lean UX design process, and get­ting an inte­grat­ed agile design and devel­op­ment team up and run­ning. The first ser­vices are now ship­ping so it is time for me to move on, after 10 months of very grat­i­fy­ing work. I real­ly enjoy work­ing in the pub­lic sec­tor and I hope to be doing more of it in future.

So yes, this means I am avail­able and you can hire me to do strat­e­gy and design for soft­ware prod­ucts and ser­vices. Just send me an email.

Short­ly before the Dutch nation­al elec­tions of this year, Iskan­der and I gath­ered a group of fel­low tech work­ers under the ban­ner of “Tech Sol­i­dar­i­ty NL to dis­cuss the con­cern­ing lurch to the right in nation­al pol­i­tics and what our field can do about it. This has devel­oped into a small but active com­mu­ni­ty who gath­er month­ly to edu­cate our­selves and devel­op plans for col­lec­tive action. I am get­ting a huge boost out of this. Fig­ur­ing out how to be a left­ist in this day and age is not easy. The only way to do it is to prac­tice and for that reflec­tion with peers is invalu­able. Build­ing and facil­i­tat­ing a group like this is huge­ly edu­ca­tion­al too. I have learned a lot about how a com­mu­ni­ty is boot-strapped and nurtured.

If you are in the Nether­lands, your pol­i­tics are left of cen­ter, and you work in tech­nol­o­gy, con­sid­er your­self invit­ed to join.

And final­ly, the last major thing on my plate is a con­tin­u­ing effort to secure a PhD posi­tion for myself. I am get­ting great sup­port from peo­ple at Delft Uni­ver­si­ty of Tech­nol­o­gy, in par­tic­u­lar Gerd Kortuem. I am focus­ing on inter­net of things prod­ucts that have fea­tures dri­ven by machine learn­ing. My ulti­mate aim is to devel­op pro­to­typ­ing tools for design and devel­op­ment teams that will help them cre­ate more inno­v­a­tive and more eth­i­cal solu­tions. The first step for this will be to con­duct field research inside com­pa­nies who are cre­at­ing such prod­ucts right now. So I am reach­ing out to peo­ple to see if I can secure a rea­son­able amount of poten­tial col­lab­o­ra­tors for this, which will go a long way in prov­ing the fea­si­bil­i­ty of my whole plan.

If you know of any com­pa­nies that devel­op con­sumer-fac­ing prod­ucts that have a con­nect­ed hard­ware com­po­nent and make use of machine learn­ing to dri­ve fea­tures, do let me know.

That’s about it. Free­lance UX con­sult­ing, left­ist tech-work­er organ­is­ing and design-for-machine-learn­ing research. Quite hap­py with that mix, really.

Hybrid Writing for Conversational Interfaces’ workshop

On May 24 of this year, Niels ’t Hooft and myself ran a work­shop titled ‘Hybrid Writ­ing for Con­ver­sa­tion­al Inter­faces’ at TU Delft. Our aim was twofold: teach stu­dents about writ­ing char­ac­ters and dia­log, and teach them how to pro­to­type chat interfaces. 

We spent a day with rough­ly thir­ty indus­tri­al design stu­dents alter­nat­ing between bits of the­o­ry, writ­ing exer­cis­es, instruc­tions on how to use Twine (our pro­to­typ­ing tool of choice) and closed out with a small project and a show and tell. 

I was very pleased to see pro­to­types with quite a high lev­el of com­plex­i­ty and sophis­ti­ca­tion at the end of the day. And through­out, I could tell stu­dents were enjoy­ing them­selves writ­ing and build­ing inter­ac­tive conversations.

Here’s a rough out­line of how the work­shop was structured.

  1. After briefly intro­duc­ing our­selves, Niels pre­sent­ed a mini-lec­ture on inter­ac­tive fic­tion. A high­light for me was a two-by-two of the ways in which fic­tion and soft­ware can intersect.

Four types of software-fiction hybrids

  1. I then took over and did a show and tell of the absolute basics of using Twine. Things like cre­at­ing pas­sages, link­ing them, cre­at­ing branch­es and test­ing and pub­lish­ing your story.
  2. The first exer­cise after this was for stu­dents to take what they just learned about Twine and try to cre­ate a very sim­ple inter­ac­tive story.
  3. After a cof­fee break, Niels then pre­sent­ed his sec­ond mini-lec­ture on the very basics of writ­ing. With a par­tic­u­lar focus on writ­ing char­ac­ters and dia­log. This includ­ed a handy cheat­sheet for things to con­sid­er while writing.

A cheatsheet for writing dialog

  1. In our sec­ond exer­cise stu­dents worked in pairs. They first each cre­at­ed a char­ac­ter, which they then described to each oth­er. They then first planned out the struc­ture of an encounter between these two char­ac­ters. And final­ly they col­lab­o­ra­tive­ly wrote the dia­logue for this encounter. They were required to stick to Hol­ly­wood for­mat­ting. Niels and I then did a read­ing of a few (to great amuse­ment of all present) to close out the morn­ing sec­tion of the workshop.
  2. After lunch Niels pre­sent­ed his third and final mini-lec­ture of the day, on con­ver­sa­tion­al inter­faces, rely­ing heav­i­ly on the great work of our friend Alper in his book on the subject.
  3. I then took over for the sec­ond show and tell. Here we ramped up the chal­lenge and intro­duced the Twine Tex­ting Project – a frame­work for pro­to­typ­ing con­ver­sa­tion­al inter­faces in Twine. On GitHub, you can find the starter file I had pre­pared for this section.
  4. The third and final exer­cise of the day was for stu­dents to take what they learned about writ­ing dia­log, and pro­to­typ­ing chat inter­faces, and to build an inter­ac­tive pro­to­type of a con­ver­sa­tion­al inter­face or inter­ac­tive fic­tion in chat for­mat. They could either build off of the dia­log they have cre­at­ed in the pre­vi­ous exer­cise, or start from scratch.
  5. We fin­ished the day with demos, where put the Twine sto­ry on the big screen and as a group chose what options to select. After each demo the cre­ator would open up the Twine file and walk us through how they had built it. It was pret­ty cool to see how many stu­dents had put what they had learned to very cre­ative uses.

Reflect­ing on the work­shop after­wards, we felt the struc­ture was nice­ly bal­anced between the­o­ry and prac­tice. The dif­fi­cul­ty lev­el was such that stu­dents did learn some new things which they could incor­po­rate into future projects, but still built on skills they had already acquired. The choice for Twine worked out well too since it is high­ly acces­si­ble. Non-tech­ni­cal stu­dents man­aged to cre­ate some­thing inter­ac­tive, and more advanced stu­dents could apply what they knew about code to pro­duce more sophis­ti­cat­ed prototypes. 

For future work­shops we did feel we could improve on build­ing a bridge between the writ­ing for inter­ac­tive fic­tion and writ­ing for con­ver­sa­tion­al inter­faces of soft­ware prod­ucts and ser­vices. This would require some adap­ta­tion of the mini lec­tures and a slight­ly dif­fer­ent empha­sis in the exer­cis­es. The key would be to have stu­dents imag­ine exist­ing prod­ucts and ser­vices as char­ac­ters, and to then write dia­log for inter­ac­tions and pro­to­type them. For a future iter­a­tion of the work­shop, this would be worth explor­ing further.

Many thanks to Ianus Keller for invit­ing us to teach this work­shop at IDE Acad­e­my.

Curiosity is our product

A few weeks ago I facil­i­tat­ed a dis­cus­sion on ‘advo­ca­cy in a post-truth era’ at the Euro­pean Dig­i­tal Rights Initiative’s annu­al gen­er­al assem­bly. And last night I was part of a dis­cus­sion on fake news at a behav­iour design meet­up in Ams­ter­dam. This was a good occa­sion to pull togeth­er some of my notes and fig­ure out what I think is true about the ‘fake news’ phenomenon.

There is plen­ty of good writ­ing out there explor­ing the his­to­ry and cur­rent state of post-truth polit­i­cal culture. 

Kellyanne Conway’s “alter­na­tive facts” and Michael Gove’s “I think peo­ple have had enough of experts” are just two exam­ples of the right’s appro­pri­a­tion of what I would call epis­te­mo­log­i­cal rel­a­tivism. Post-mod­ernism was fun while it worked to advance our left­ist agen­da. But now that the tables are turned we’re not enjoy­ing it quite as much any­more, are we?

Part of the fact-free pol­i­tics play­book goes back at least as far as big tobacco’s efforts to dis­cred­it the anti-smok­ing lob­by. “Doubt is our prod­uct” still applies to mod­ern day reac­tionary move­ments such as cli­mate change deniers and anti-vaxers.

The dou­ble wham­my of news indus­try com­mer­cial­i­sa­tion and inter­net plat­form con­sol­i­da­tion has cre­at­ed fer­tile ground for coor­di­nat­ed efforts by var­i­ous groups to turn the sow­ing of doubt all the way up to eleven.

There is Russia’s “fire­hose of false­hood” which sends a high vol­ume of mes­sages across a wide range of chan­nels with total dis­re­gard for truth or even con­sis­ten­cy in a rapid, con­tin­u­ous and repet­i­tive fash­ion. They seem to be hav­ing fun desta­bil­is­ing west­ern democ­ra­cies — includ­ing the Nether­lands — with­out any appar­ent end-goal in mind.

And then there is the out­rage mar­ket­ing lever­aged by trolls both minor and major. Piss­ing off main­stream media builds an audi­ence on the fringes and in the under­ground. Jour­nal­ists are held hostage by fig­ures such as Milo because they depend on sto­ries that trig­ger strong emo­tions for dis­tri­b­u­tion, eye­balls, clicks and ulti­mate­ly revenue. 

So, giv­en all of this, what is to be done? First some bad news. Facts, the weapon of choice for lib­er­als, don’t appear to work. This is empir­i­cal­ly evi­dent from recent events, but it also appears to be borne out by psychology. 

Facts are often more com­pli­cat­ed than the untruths they are sup­posed to counter. It is also eas­i­er to remem­ber a sim­ple lie than a com­pli­cat­ed truth. Com­pli­cat­ing mat­ters fur­ther, facts tend to be bor­ing. Final­ly, and most inter­est­ing­ly, there is some­thing called the ‘back­fire effect’: we become more entrenched in our views when con­front­ed with con­tra­dict­ing facts, because they are threat­en­ing to our group identities.

More bad news. Giv­en the speed at which false­hoods spread through our net­works, fact-check­ing is use­less. Fact-check­ing is after-the-fact-check­ing. Worse, when media fact-check false­hoods on their front pages they are sim­ply pro­vid­ing even more air­time to them. From a strate­gic per­spec­tive, when you debunk, you allow your­self to be cap­tured by your opponent’s frame, and you’re also on the defen­sive. In Boy­di­an terms you are caught in their OODA loop, when you should be work­ing to take back the ini­tia­tive, and you should be offer­ing an alter­na­tive narrative. 

I am not hope­ful main­stream media will save us from these dynam­ics giv­en the real­i­ties of the busi­ness mod­els they oper­ate inside of. Jour­nal­ists inside of these organ­i­sa­tions are typ­i­cal­ly over­worked, just hold­ing on for dear life and churn­ing out sto­ries at a rapid clip. In short, there is no time to ori­ent and manoeu­vre. For bad-faith actors, they are sit­ting ducks.

What about lit­er­a­cy? If only peo­ple knew about chur­nal­ism, the atten­tion econ­o­my, and fil­ter bub­bles ‘they’ would become immune to the lies ped­dled by reac­tionar­ies and return to the lib­er­al fold. Per­son­al­ly I find these claims high­ly uncon­vinc­ing not to men­tion condescending. 

My cur­rent work­ing the­o­ry is that we, all of us, buy into the sto­ries that acti­vate one or more of our group iden­ti­ties, regard­less of wether they are fact-based or out­right lies. This is called ‘moti­vat­ed rea­son­ing’. Since this is a fact of psy­chol­o­gy, we are all sus­cep­ti­ble to it, includ­ing lib­er­als who are sup­pos­ed­ly defend­ers of fact-based reasoning.

Seri­ous­ly though, what about lit­er­a­cy? I’m sor­ry, no. There is evi­dence that sci­en­tif­ic lit­er­a­cy actu­al­ly increas­es polar­i­sa­tion. Moti­vat­ed rea­son­ing trumps fac­tu­al knowl­edge you may have. The same research shows how­ev­er that curios­i­ty in turn trumps moti­vat­ed rea­son­ing. The way I under­stand the dis­tinc­tion between lit­er­a­cy and curios­i­ty is that the for­mer is about knowl­edge while the lat­ter is about atti­tude. Moti­vat­ed rea­son­ing isn’t coun­ter­act­ed by know­ing stuff, but by want­i­ng to know stuff.

This is a mixed bag. Offer­ing facts is com­par­a­tive­ly easy. Spark­ing curios­i­ty requires sto­ry­telling which in turn requires imag­i­na­tion. If we’re pre­sent­ed with a fact we are not invit­ed to ask ques­tions. How­ev­er, if we are pre­sent­ed with ques­tions and those ques­tions are wrapped up in sto­ries that cre­ate emo­tion­al stakes, some of the views we hold might be destabilised.

In oth­er words, if doubt is the prod­uct ped­dled by our oppo­nents, then we should start traf­fick­ing in curiosity.

Further reading

Machine Learning for Designers’ workshop

On Wednes­day Péter Kun, Hol­ly Rob­bins and myself taught a one-day work­shop on machine learn­ing at Delft Uni­ver­si­ty of Tech­nol­o­gy. We had about thir­ty master’s stu­dents from the indus­tri­al design engi­neer­ing fac­ul­ty. The aim was to get them acquaint­ed with the tech­nol­o­gy through hands-on tin­ker­ing with the Wek­ina­tor as cen­tral teach­ing tool.

Photo credits: Holly Robbins
Pho­to cred­its: Hol­ly Robbins

Background

The rea­son­ing behind this work­shop is twofold. 

On the one hand I expect design­ers will find them­selves work­ing on projects involv­ing machine learn­ing more and more often. The tech­nol­o­gy has cer­tain prop­er­ties that dif­fer from tra­di­tion­al soft­ware. Most impor­tant­ly, machine learn­ing is prob­a­bilis­tic in stead of deter­min­is­tic. It is impor­tant that design­ers under­stand this because oth­er­wise they are like­ly to make bad deci­sions about its application. 

The sec­ond rea­son is that I have a strong sense machine learn­ing can play a role in the aug­men­ta­tion of the design process itself. So-called intel­li­gent design tools could make design­ers more effi­cient and effec­tive. They could also enable the cre­ation of designs that would oth­er­wise be impos­si­ble or very hard to achieve.

The work­shop explored both ideas.

Photo credits: Holly Robbins
Pho­to cred­its: Hol­ly Robbins

Format

The struc­ture was rough­ly as follows: 

In the morn­ing we start­ed out pro­vid­ing a very broad intro­duc­tion to the tech­nol­o­gy. We talked about the very basic premise of (super­vised) learn­ing. Name­ly, pro­vid­ing exam­ples of inputs and desired out­puts and train­ing a mod­el based on those exam­ples. To make these con­cepts tan­gi­ble we then intro­duced the Wek­ina­tor and walked the stu­dents through get­ting it up and run­ning using basic exam­ples from the web­site. The final step was to invite them to explore alter­na­tive inputs and out­puts (such as game con­trollers and Arduino boards).

In the after­noon we pro­vid­ed a design brief, ask­ing the stu­dents to pro­to­type a data-enabled object with the set of tools they had acquired in the morn­ing. We assist­ed with tech­ni­cal hur­dles where nec­es­sary (of which there were more than a few) and closed out the day with demos and a group dis­cus­sion reflect­ing on their expe­ri­ences with the technology.

Photo credits: Holly Robbins
Pho­to cred­its: Hol­ly Robbins

Results

As I tweet­ed on the way home that evening, the results were… interesting. 

Not all groups man­aged to put some­thing togeth­er in the admit­ted­ly short amount of time they were pro­vid­ed with. They were most often stymied by get­ting an Arduino to talk to the Wek­ina­tor. Max was often picked as a go-between because the Wek­ina­tor receives OSC mes­sages over UDP, where­as the quick­est way to get an Arduino to talk to a com­put­er is over ser­i­al. But Max in my expe­ri­ence is a fick­le beast and would more than once crap out on us.

The groups that did build some­thing main­ly assem­bled pro­to­types from the exam­ples on hand. Which is fine, but since we were main­ly work­ing with the exam­ples from the Wek­ina­tor web­site they tend­ed towards the inter­ac­tive instru­ment side of things. We were hop­ing for explo­rations of IoT prod­uct con­cepts. For that more hand-rolling was required and this was only achiev­able for the stu­dents on the high­er end of the tech­ni­cal exper­tise spec­trum (and the more tena­cious ones).

The dis­cus­sion yield­ed some inter­est­ing insights into men­tal mod­els of the tech­nol­o­gy and how they are affect­ed by hands-on expe­ri­ence. A com­ment I heard more than once was: Why is this con­sid­ered learn­ing at all? The Wek­ina­tor was not per­ceived to be learn­ing any­thing. When chal­lenged on this by reit­er­at­ing the under­ly­ing prin­ci­ples it became clear the black box nature of the Wek­ina­tor ham­pers appre­ci­a­tion of some of the very real achieve­ments of the tech­nol­o­gy. It seems (for our stu­dents at least) machine learn­ing is stuck in a grey area between too-high expec­ta­tions and too-low recog­ni­tion of its capabilities.

Next steps

These results, and oth­ers, point towards some obvi­ous improve­ments which can be made to the work­shop for­mat, and to teach­ing design stu­dents about machine learn­ing more broadly. 

  1. We can improve the toolset so that some of the heavy lift­ing involved with get­ting the var­i­ous parts to talk to each oth­er is made eas­i­er and more reliable.
  2. We can build exam­ples that are geared towards the prac­tice of design­ing IoT prod­ucts and are ready for adap­ta­tion and hacking.
  3. And final­ly, and prob­a­bly most chal­leng­ing­ly, we can make the work­ings of machine learn­ing more trans­par­ent so that it becomes eas­i­er to devel­op a feel for its capa­bil­i­ties and shortcomings.

We do intend to improve and teach the work­shop again. If you’re inter­est­ed in host­ing one (either in an edu­ca­tion­al or pro­fes­sion­al con­text) let me know. And stay tuned for updates on this and oth­er efforts to get design­ers to work in a hands-on man­ner with machine learning.

Spe­cial thanks to the bril­liant Ianus Keller for con­nect­ing me to Péter and for allow­ing us to pilot this crazy idea at IDE Acad­e­my.

References

Sources used dur­ing prepa­ra­tion and run­ning of the workshop:

  • The Wek­ina­tor – the UI is infu­ri­at­ing­ly poor but when it comes to get­ting start­ed with machine learn­ing this tool is unmatched.
  • Arduino – I have become par­tic­u­lar­ly fond of the MKR1000 board. Add a lithi­um-poly­mer bat­tery and you have every­thing you need to pro­to­type IoT products.
  • OSC for ArduinoCNMAT’s imple­men­ta­tion of the open sound con­trol (OSC) encod­ing. Key puz­zle piece for get­ting the above two tools talk­ing to each other.
  • Machine Learn­ing for Design­ers – my pre­ferred intro­duc­tion to the tech­nol­o­gy from a design­er­ly perspective.
  • A Visu­al Intro­duc­tion to Machine Learn­ing – a very acces­si­ble visu­al expla­na­tion of the basic under­pin­nings of com­put­ers apply­ing sta­tis­ti­cal learning.
  • Remote Con­trol Theremin – an exam­ple project I pre­pared for the work­shop demo­ing how to have the Wek­ina­tor talk to an Arduino MKR1000 with OSC over UDP.

Design × AI coffee meetup

If you work in the field of design or arti­fi­cial intel­li­gence and are inter­est­ed in explor­ing the oppor­tu­ni­ties at their inter­sec­tion, con­sid­er your­self invit­ed to an infor­mal cof­fee meet­up on Feb­ru­ary 15, 10am at Brix in Amsterdam.

Erik van der Plui­jm and myself have for a while now been car­ry­ing on a con­ver­sa­tion about AI and design and we felt it was time to expand the cir­cle a bit. We are very curi­ous who else out there shares our excitement.

Ques­tions we are mulling over include: How does the design process change when cre­at­ing intel­li­gent prod­ucts? And: How can teams col­lab­o­rate with intel­li­gent design tools to solve prob­lems in new and inter­est­ing ways?

Any­way, lots to chew on.

No need to sign up or any­thing, just show up and we’ll see what happens.

High-skill robots, low-skill workers

Some notes on what I think I under­stand about tech­nol­o­gy and inequality.

Let’s start with an obvi­ous big ques­tion: is tech­nol­o­gy destroy­ing jobs faster than they can be replaced? On the long term the evi­dence isn’t strong. Humans always appear to invent new things to do. There is no rea­son this time around should be any different.

But in the short term tech­nol­o­gy has con­tributed to an evap­o­ra­tion of mid-skilled jobs. Parts of these jobs are auto­mat­ed entire­ly, parts can be done by few­er peo­ple because of high­er pro­duc­tiv­i­ty gained from tech.

While pro­duc­tiv­i­ty con­tin­ues to grow, jobs are lag­ging behind. The year 2000 appears to have been a turn­ing point. “Some­thing” hap­pened around that time. But no-one knows exact­ly what. 

My hunch is that we’ve seen an emer­gence of a new class of pseu­do-monop­o­lies. Oli­gop­o­lies. And this is com­pound­ed by a ‘win­ner takes all’ dynam­ic that tech­nol­o­gy seems to produce. 

Oth­ers have point­ed to glob­al­i­sa­tion but although this might be a con­tribut­ing fac­tor, the evi­dence does not sup­port the idea that it is the major cause.

So what are we left with?

His­tor­i­cal­ly, look­ing at pre­vi­ous tech­no­log­i­cal upsets, it appears edu­ca­tion makes a big dif­fer­ence. Peo­ple neg­a­tive­ly affect­ed by tech­no­log­i­cal progress should have access to good edu­ca­tion so that they have options. In the US the access to high qual­i­ty edu­ca­tion is not equal­ly divided.

Appar­ent­ly fam­i­ly income is asso­ci­at­ed with edu­ca­tion­al achieve­ment. So if your fam­i­ly is rich, you are more like­ly to become a high skilled indi­vid­ual. And high skilled indi­vid­u­als are priv­i­leged by the tech economy.

And if Piket­ty’s is right, we are approach­ing a real­i­ty in which mon­ey made from wealth ris­es faster than wages. So there is a feed­back loop in place which only exac­er­bates the situation.

One more bul­let: If you think trick­le-down eco­nom­ics, increas­ing the size of the pie will help, you might be mis­tak­en. It appears social mobil­i­ty is helped more by decreas­ing inequal­i­ty in the dis­tri­b­u­tion of income growth.

So some pre­lim­i­nary con­clu­sions: a pro­gres­sive tax on wealth won’t solve the issue. The edu­ca­tion sys­tem will require reform, too. 

I think this is the cen­tral irony of the whole sit­u­a­tion: we are work­ing hard to teach machines how to learn. But we are neglect­ing to improve how peo­ple learn.