PhD update – January 2019

Thought I’d post a quick update on my PhD. Since my pre­vi­ous post almost five months have passed. I’ve been devel­op­ing my plan fur­ther, for which you’ll find an updat­ed descrip­tion below. I’ve also put togeth­er my very first con­fer­ence paper, co-authored with my super­vi­sor Gerd Kortuem. It’s a case study of the MX3D smart bridge for Design­ing Inter­ac­tive Sys­tems 2019. We’ll see if it gets accept­ed. But in any case, writ­ing some­thing has been huge­ly edu­ca­tion­al. And once I final­ly fig­ured out what the hell I was doing, it was sort of fun as well. Still kind of a trip to be paid to do this kind of work. Look­ing ahead, I am set­ting goals for this year and the near­er term as well. It’s all very rough still but it will like­ly involve research through design as a method and maybe object ori­ent­ed ontol­ogy as a the­o­ry. All of which will serve to oper­a­tionalise and eval­u­ate the use­ful­ness of the “con­testa­bil­i­ty” con­cept in the con­text of smart city infra­struc­ture. To be continued—and I wel­come all your thoughts!


Design­ing Smart City Infra­struc­ture for Con­testa­bil­i­ty

The use of infor­ma­tion tech­nol­o­gy in cities increas­ing­ly sub­jects cit­i­zens to auto­mat­ed data col­lec­tion, algo­rith­mic deci­sion mak­ing and remote con­trol of phys­i­cal space. Cit­i­zens tend to find these sys­tems and their out­comes hard to under­stand and pre­dict [1]. More­over, the opac­i­ty of smart urban sys­tems pre­cludes full cit­i­zen­ship and obstructs people’s ‘right to the city’ [2].

A com­mon­ly pro­posed solu­tion is to improve cit­i­zens under­stand­ing of sys­tems by mak­ing them more open and trans­par­ent [3]. For exam­ple, GDPR pre­scribes people’s right to expla­na­tion of auto­mat­ed deci­sions they have been sub­ject­ed to. For anoth­er exam­ple, the city of Ams­ter­dam offers a pub­licly acces­si­ble reg­is­ter of urban sen­sors, and is com­mit­ted to open­ing up all the data they col­lect.

How­ev­er, it is not clear that open­ness and trans­paren­cy in and of itself will yield the desired improve­ments in under­stand­ing and gov­ern­ing of smart city infra­struc­tures [4]. We would like to sug­gest that for a sys­tem to per­ceived as account­able, peo­ple must be able to con­test its workings—from the data it col­lects, to the deci­sions it makes, all the way through to how those deci­sions are act­ed on in the world.

The lead­ing research ques­tion for this PhD there­fore is how to design smart city infrastructure—urban sys­tems aug­ment­ed with inter­net-con­nect­ed sens­ing, pro­cess­ing and actu­at­ing capabilities—for con­testa­bil­i­ty [5]: the extent to which a sys­tem sup­ports the abil­i­ty of those sub­ject­ed to it to oppose its work­ings as wrong or mis­tak­en.

Ref­er­ences

  1. Bur­rell, Jen­na. “How the machine ‘thinks’: Under­stand­ing opac­i­ty in machine learn­ing algo­rithms.” Big Data & Soci­ety 3.1 (2016): 2053951715622512.
  2. Kitchin, Rob, Pao­lo Car­dul­lo, and Cesare Di Feli­cianto­nio. “Cit­i­zen­ship, Jus­tice and the Right to the Smart City.” (2018).
  3. Abdul, Ashraf, et al. “Trends and tra­jec­to­ries for explain­able, account­able and intel­li­gi­ble sys­tems: An hci research agen­da.” Pro­ceed­ings of the 2018 CHI Con­fer­ence on Human Fac­tors in Com­put­ing Sys­tems. ACM, 2018.
  4. Anan­ny, Mike, and Kate Craw­ford. “See­ing with­out know­ing: Lim­i­ta­tions of the trans­paren­cy ide­al and its appli­ca­tion to algo­rith­mic account­abil­i­ty.” New Media & Soci­ety 20.3 (2018): 973–989.
  5. Hirsch, Tad, et al. “Design­ing con­testa­bil­i­ty: Inter­ac­tion design, machine learn­ing, and men­tal health.” Pro­ceed­ings of the 2017 Con­fer­ence on Design­ing Inter­ac­tive Sys­tems. ACM, 2017.

Books I’ve read in 2018

Goodreads tells me I’ve read 48 books in 2018. I set myself the goal of 36 so it looks like I beat it hand­i­ly. But includ­ed in that count are quite a few role­play­ing game books and comics. If I dis­card those I’m left with 28 titles. Still a decent amount but noth­ing par­tic­u­lar­ly remark­able. Below are a few lists and some notes to go with them.

Most of the non-fic­tion is some­where on the inter­sec­tion of design, tech­nol­o­gy and Left pol­i­tics. A lot of this read­ing was dri­ven by my desire to devel­op some kind of men­tal frame­work for the work we were doing with Tech Sol­i­dar­i­ty NL. More recently—since I start­ed my PhD—I’ve most­ly been read­ing text­books on research method­ol­o­gy. Hid­den from this list is the aca­d­e­m­ic papers I’ve start­ed con­sum­ing as part of this new job. I should fig­ure out a way of shar­ing some of that here or else­where as well.

I took a break from tech­nol­o­gy and indulged in a deep dive into the his­to­ry of the thir­ty year’s war with a mas­sive non-fic­tion treat­ment as well as a clas­sic picaresque set in the same time peri­od. While read­ing these I was tran­si­tion­ing into my new role as a father of twin boys. Some­what relat­ed was a brief his­to­ry of The Nether­lands, which I’ve start­ed rec­om­mend­ing to for­eign­ers who are strug­gling to under­stand our idio­syn­crat­ic lit­tle nation and go beyond super­fi­cial­i­ties.

Then there’s the fic­tion, which in the begin­ning of the year con­sist­ed of high­brow weird and his­tor­i­cal nov­els but then ven­tured into clas­sic fan­ta­sy and (utopi­an) sci-fi ter­ri­to­ry. Again, most­ly because of a jus­ti­fi­able desire for some escapism in the sleep deprived evenings and nights.

Hav­ing men­tioned the arrival of our boys a few times it should come as no sur­prise that I also read a cou­ple of par­ent­ing books. These were more than enough for me and real­ly to be hon­est I think par­ent­ing is a thing best learned through prac­tice. Espe­cial­ly if you’re rais­ing two babies at once.

So that’s it. I’ve set myself the mod­est goal of 24 books for this year because I’m quite sure most of my read­ing will be papers and such. Here’s to a year of what I expect will be many more late night and ear­ly morn­ing read­ing ses­sions of escapist weird fic­tion.

Pre­vi­ous years: 2017, 2016, 2015, 2011, 2009.

Unboxing’ at Behavior Design Amsterdam #16

Below is a write-up of the talk I gave at the Behav­ior Design Ams­ter­dam #16 meet­up on Thurs­day, Feb­ru­ary 15, 2018.

'Pandora' by John William Waterhouse (1896)
‘Pan­do­ra’ by John William Water­house (1896)

I’d like to talk about the future of our design prac­tice and what I think we should focus our atten­tion on. It is all relat­ed to this idea of com­plex­i­ty and open­ing up black box­es. We’re going to take the scenic route, though. So bear with me.

Software Design

Two years ago I spent about half a year in Sin­ga­pore.

While there I worked as prod­uct strate­gist and design­er at a start­up called ARTO, an art rec­om­men­da­tion ser­vice. It shows you a ran­dom sam­ple of art­works, you tell it which ones you like, and it will then start rec­om­mend­ing pieces it thinks you like. In case you were won­der­ing: yes, swip­ing left and right was involved.

We had this inter­est­ing prob­lem of ingest­ing art from many dif­fer­ent sources (most­ly online gal­leries) with meta­da­ta of wild­ly vary­ing lev­els of qual­i­ty. So, using meta­da­ta to fig­ure out which art to show was a bit of a non-starter. It should come as no sur­prise then, that we start­ed look­ing into machine learning—image pro­cess­ing in par­tic­u­lar.

And so I found myself work­ing with my engi­neer­ing col­leagues on an art rec­om­men­da­tion stream which was dri­ven at least in part by machine learn­ing. And I quick­ly realised we had a prob­lem. In terms of how we worked togeth­er on this part of the prod­uct, it felt like we had tak­en a bunch of steps back in time. Back to a way of col­lab­o­rat­ing that was less inte­grat­ed and less respon­sive.

That’s because we have all these nice tools and tech­niques for design­ing tra­di­tion­al soft­ware prod­ucts. But soft­ware is deter­min­is­tic. Machine learn­ing is fun­da­men­tal­ly dif­fer­ent in nature: it is prob­a­bilis­tic.

It was hard for me to take the lead in the design of this part of the prod­uct for two rea­sons. First of all, it was chal­leng­ing to get a first-hand feel of the machine learn­ing fea­ture before it was imple­ment­ed.

And sec­ond of all, it was hard for me to com­mu­ni­cate or visu­alise the intend­ed behav­iour of the machine learn­ing fea­ture to the rest of the team.

So when I came back to the Nether­lands I decid­ed to dig into this prob­lem of design for machine learn­ing. Turns out I opened up quite the can of worms for myself. But that’s okay.

There are two rea­sons I care about this:

The first is that I think we need more design-led inno­va­tion in the machine learn­ing space. At the moment it is engi­neer­ing-dom­i­nat­ed, which doesn’t nec­es­sar­i­ly lead to use­ful out­comes. But if you want to take the lead in the design of machine learn­ing appli­ca­tions, you need a firm han­dle on the nature of the tech­nol­o­gy.

The sec­ond rea­son why I think we need to edu­cate our­selves as design­ers on the nature of machine learn­ing is that we need to take respon­si­bil­i­ty for the impact the tech­nol­o­gy has on the lives of peo­ple. There is a lot of talk about ethics in the design indus­try at the moment. Which I con­sid­er a pos­i­tive sign. But I also see a reluc­tance to real­ly grap­ple with what ethics is and what the rela­tion­ship between tech­nol­o­gy and soci­ety is. We seem to want easy answers, which is under­stand­able because we are all very busy peo­ple. But hav­ing spent some time dig­ging into this stuff myself I am here to tell you: There are no easy answers. That isn’t a bug, it’s a fea­ture. And we should embrace it.

Machine Learning

At the end of 2016 I attend­ed ThingsCon here in Ams­ter­dam and I was intro­duced by Ianus Keller to TU Delft PhD researcher Péter Kun. It turns out we were both inter­est­ed in machine learn­ing. So with encour­age­ment from Ianus we decid­ed to put togeth­er a work­shop that would enable indus­tri­al design mas­ter stu­dents to tan­gle with it in a hands-on man­ner.

About a year lat­er now, this has grown into a thing we call Pro­to­typ­ing the Use­less But­ler. Dur­ing the work­shop, you use machine learn­ing algo­rithms to train a mod­el that takes inputs from a net­work-con­nect­ed arduino’s sen­sors and dri­ves that same arduino’s actu­a­tors. In effect, you can cre­ate inter­ac­tive behav­iour with­out writ­ing a sin­gle line of code. And you get a first hand feel for how com­mon appli­ca­tions of machine learn­ing work. Things like regres­sion, clas­si­fi­ca­tion and dynam­ic time warp­ing.

The thing that makes this work­shop tick is an open source soft­ware appli­ca­tion called Wek­ina­tor. Which was cre­at­ed by Rebec­ca Fiebrink. It was orig­i­nal­ly aimed at per­form­ing artists so that they could build inter­ac­tive instru­ments with­out writ­ing code. But it takes inputs from any­thing and sends out­puts to any­thing. So we appro­pri­at­ed it towards our own ends.

You can find every­thing relat­ed to Use­less But­ler on this GitHub repo.

The think­ing behind this work­shop is that for us design­ers to be able to think cre­ative­ly about appli­ca­tions of machine learn­ing, we need a gran­u­lar under­stand­ing of the nature of the tech­nol­o­gy. The thing with design­ers is, we can’t real­ly learn about such things from books. A lot of design knowl­edge is tac­it, it emerges from our phys­i­cal engage­ment with the world. This is why things like sketch­ing and pro­to­typ­ing are such essen­tial parts of our way of work­ing. And so with use­less but­ler we aim to cre­ate an envi­ron­ment in which you as a design­er can gain tac­it knowl­edge about the work­ings of machine learn­ing.

Sim­ply put, for a lot of us, machine learn­ing is a black box. With Use­less But­ler, we open the black box a bit and let you peer inside. This should improve the odds of design-led inno­va­tion hap­pen­ing in the machine learn­ing space. And it should also help with ethics. But it’s def­i­nite­ly not enough. Knowl­edge about the tech­nol­o­gy isn’t the only issue here. There are more black box­es to open.

Values

Which brings me back to that oth­er black box: ethics. Like I already men­tioned there is a lot of talk in the tech indus­try about how we should “be more eth­i­cal”. But things are often reduced to this notion that design­ers should do no harm. As if ethics is a prob­lem to be fixed in stead of a thing to be prac­ticed.

So I start­ed to talk about this to peo­ple I know in acad­e­mia and more than once this thing called Val­ue Sen­si­tive Design was men­tioned. It should be no sur­prise to any­one that schol­ars have been chew­ing on this stuff for quite a while. One of the ear­li­est ref­er­ences I came across, an essay by Batya Fried­man in Inter­ac­tions is from 1996! This is a les­son to all of us I think. Pay more atten­tion to what the aca­d­e­mics are talk­ing about.

So, at the end of last year I dove into this top­ic. Our host Iskan­der Smit, Rob Mai­jers and myself coor­di­nate a grass­roots com­mu­ni­ty for tech work­ers called Tech Sol­i­dar­i­ty NL. We want to build tech­nol­o­gy that serves the needs of the many, not the few. Val­ue Sen­si­tive Design seemed like a good thing to dig into and so we did.

I’m not going to dive into the details here. There’s a report on the Tech Sol­i­dar­i­ty NL web­site if you’re inter­est­ed. But I will high­light a few things that val­ue sen­si­tive design asks us to con­sid­er that I think help us unpack what it means to prac­tice eth­i­cal design.

First of all, val­ues. Here’s how it is com­mon­ly defined in the lit­er­a­ture:

A val­ue refers to what a per­son or group of peo­ple con­sid­er impor­tant in life.”

I like it because it’s com­mon sense, right? But it also makes clear that there can nev­er be one mono­lith­ic def­i­n­i­tion of what ‘good’ is in all cas­es. As we design­ers like to say: “it depends” and when it comes to val­ues things are no dif­fer­ent.

Per­son or group” implies there can be var­i­ous stake­hold­ers. Val­ue sen­si­tive design dis­tin­guish­es between direct and indi­rect stake­hold­ers. The for­mer have direct con­tact with the tech­nol­o­gy, the lat­ter don’t but are affect­ed by it nonethe­less. Val­ue sen­si­tive design means tak­ing both into account. So this blows up the con­ven­tion­al notion of a sin­gle user to design for.

Var­i­ous stake­hold­er groups can have com­pet­ing val­ues and so to design for them means to arrive at some sort of trade-off between val­ues. This is a cru­cial point. There is no such thing as a per­fect or objec­tive­ly best solu­tion to eth­i­cal conun­drums. Not in the design of tech­nol­o­gy and not any­where else.

Val­ue sen­si­tive design encour­ages you to map stake­hold­ers and their val­ues. These will be dif­fer­ent for every design project. Anoth­er approach is to use lists like the one pic­tured here as an ana­lyt­i­cal tool to think about how a design impacts var­i­ous val­ues.

Fur­ther­more, dur­ing your design process you might not only think about the short-term impact of a tech­nol­o­gy, but also think about how it will affect things in the long run.

And sim­i­lar­ly, you might think about the effects of a tech­nol­o­gy not only when a few peo­ple are using it, but also when it becomes wild­ly suc­cess­ful and every­body uses it.

There are tools out there that can help you think through these things. But so far much of the work in this area is hap­pen­ing on the aca­d­e­m­ic side. I think there is an oppor­tu­ni­ty for us to cre­ate tools and case stud­ies that will help us edu­cate our­selves on this stuff.

There’s a lot more to say on this but I’m going to stop here. The point is, as with the nature of the tech­nolo­gies we work with, it helps to dig deep­er into the nature of the rela­tion­ship between tech­nol­o­gy and soci­ety. Yes, it com­pli­cates things. But that is exact­ly the point.

Priv­i­leg­ing sim­ple and scal­able solu­tions over those adapt­ed to local needs is social­ly, eco­nom­i­cal­ly and eco­log­i­cal­ly unsus­tain­able. So I hope you will join me in embrac­ing com­plex­i­ty.

Starting a PhD

Today is the first offi­cial work day of my new doc­tor­al researcher posi­tion at Delft Uni­ver­si­ty of Tech­nol­o­gy. After more than two years of lay­ing the ground work, I’m start­ing out on a new chal­lenge.

I remem­ber sit­ting out­side a Jew­el cof­fee bar in Sin­ga­pore1 and going over the var­i­ous options for what­ev­er would be next after shut­ting down Hub­bub. I knew I want­ed to delve into the impact of machine learn­ing and data sci­ence on inter­ac­tion design. And large­ly through process of elim­i­na­tion I felt the best place for me to do so would be inside of acad­e­mia.

Back in the Nether­lands, with help from Ianus Keller, I start­ed mak­ing inroads at TU Delft, my first choice for this kind of work. I had vis­it­ed it on and off over the years, coach­ing stu­dents and doing guest lec­tures. I’d felt at home right away.

There were quite a few twists and turns along the way but now here we are. Start­ing this month I am a doc­tor­al can­di­date at Delft Uni­ver­si­ty of Technology’s fac­ul­ty of Indus­tri­al Design Engi­neer­ing.

My research is pro­vi­sion­al­ly titled ‘Intel­li­gi­bil­i­ty and Trans­paren­cy of Smart Pub­lic Infra­struc­tures: A Design Ori­ent­ed Approach’. Its main object of study is the MX3D smart bridge. My super­vi­sors are Gerd Kortuem and Neelke Doorn. And it’s all part of the NWO-fund­ed project ‘BRIdg­ing Data in the built Envi­ron­ment (BRIDE)’.

Below is a first rough abstract of the research. But in the months to come this is like­ly to change sub­stan­tial­ly as I start ham­mer­ing out a prop­er research plan. I plan to post the occa­sion­al update on my work here, so if you’re inter­est­ed your best bet is prob­a­bly to do the old RSS thing. There’s social media too, of course. And I might set up a newslet­ter at some point. We’ll see.

If any of this res­onates, do get in touch. I’d love to start a con­ver­sa­tion with as many peo­ple as pos­si­ble about this stuff.

Intel­li­gi­bil­i­ty and Trans­paren­cy of Smart Pub­lic Infra­struc­tures: A Design Ori­ent­ed Approach

This phd will explore how design­ers, tech­nol­o­gists, and cit­i­zens can uti­lize rapid urban man­u­fac­tur­ing and IoT tech­nolo­gies for design­ing urban space that express­es its intel­li­gence from the inter­sec­tion of peo­ple, places, activ­i­ties and tech­nol­o­gy, not mere­ly from the pres­ence of cut­ting-edge tech­nol­o­gy. The key ques­tion is how smart pub­lic infra­struc­ture, i.e. data-dri­ven and algo­rithm-rich pub­lic infra­struc­tures, can be under­stood by lay-peo­ple.

The design-ori­ent­ed research will uti­lize a ‘research through design’ approach to devel­op a dig­i­tal expe­ri­ence around the bridge and the sur­round­ing urban space. Dur­ing this extend­ed design and mak­ing process the phd stu­dent will con­duct empir­i­cal research to inves­ti­gate design choic­es and their impli­ca­tions on (1) new forms of par­tic­i­pa­to­ry data-informed design process­es, (2) the tech­nol­o­gy-medi­at­ed expe­ri­ence of urban space, (3) the emerg­ing rela­tion­ship between res­i­dents and “their” bridge, and (4) new forms of data-informed, cit­i­zen led gov­er­nance of pub­lic space.

  1. My Foursquare his­to­ry and 750 Words archive tell me this was on Sat­ur­day, Jan­u­ary 16, 2016. []

An Introduction to Value Sensitive Design

Phnom Bakheng
Phnom Bakheng

At a recent Tech Sol­i­dar­i­ty NL meet­up we dove into Val­ue Sen­si­tive Design. This approach had been on my radar for a while so when we con­clud­ed that for our com­mu­ni­ty it would be use­ful to talk about how to prac­tice eth­i­cal design and devel­op­ment of tech­nol­o­gy, I fig­ured we should check it out.

Val­ue Sen­si­tive Design has been around for ages. The ear­li­est arti­cle I came across is by Batya Fried­man in a 1996 edi­tion of Inter­ac­tions mag­a­zine. Iron­i­cal­ly, or trag­i­cal­ly, I must say I have only heard about the approach from aca­d­e­mics and design the­o­ry nerds. In indus­try at large, Val­ue Sen­si­tive Design appears to be—to me at least—basically unknown. (A recent excep­tion would be this inter­est­ing mar­riage of design sprints with Val­ue Sen­si­tive Design by Cen­ny­dd Bowles.)

For the meet­up, I read a hand-full of papers and cob­bled togeth­er a deck which attempts to sum­marise this ’framework’—the term favoured by its main pro­po­nents. I went through it and then we had a spir­it­ed dis­cus­sion of how its ideas apply to our dai­ly prac­tice. A report of all of that can be found over at the Tech Sol­i­dar­i­ty NL web­site.

Below, I have attempt­ed to pull togeth­er the most salient points from what is a rather dense twen­ty-plus-slides deck. I hope it is of some use to those pro­fes­sion­al design­ers and devel­op­ers who are look­ing for bet­ter ways of build­ing tech­nol­o­gy that serves the inter­est of the many, not the few.

What fol­lows is most­ly adapt­ed from the chap­ter “Val­ue Sen­si­tive Design and Infor­ma­tion Sys­tems” in Human–computer inter­ac­tion in man­age­ment infor­ma­tion sys­tems: Foun­da­tions. All quotes are from there unless oth­er­wise not­ed.

Background

The depar­ture point is the obser­va­tion that “there is a need for an over­ar­ch­ing the­o­ret­i­cal and method­olog­i­cal frame­work with which to han­dle the val­ue dimen­sions of design work.” In oth­er words, some­thing that accounts for what we already know about how to deal with val­ues in design work in terms of the­o­ry and con­cepts, as well as meth­ods and tech­niques.

This is of course not a new con­cern. For exam­ple, famed cyber­neti­cist Nor­bert Wiener argued that tech­nol­o­gy could help make us bet­ter human beings, and cre­ate a more just soci­ety. But for it to do so, he argued, we have to take con­trol of the tech­nol­o­gy.

We have to reject the “wor­ship­ing [of] the new gad­gets which are our own cre­ation as if they were our mas­ters.” (Wiener 1953)

We can find many more sim­i­lar argu­ments through­out the his­to­ry of infor­ma­tion tech­nol­o­gy. Recent­ly such con­cerns have flared up in indus­try as well as soci­ety at large. (Not always for the right rea­sons in my opin­ion, but that is some­thing we will set aside for now.)

To address these con­cerns, Val­ue Sen­si­tive Design was devel­oped. It is “a the­o­ret­i­cal­ly ground­ed approach to the design of tech­nol­o­gy that accounts for human val­ues in a prin­ci­pled and com­pre­hen­sive man­ner through­out the design process.” It has been applied suc­cess­ful­ly for over 20 years.

Defining Values

But what is a val­ue? In the lit­er­a­ture it is defined as “what a per­son or group of peo­ple con­sid­er impor­tant in life.” I like this def­i­n­i­tion because it is easy to grasp but also under­lines the slip­pery nature of val­ues. Some things to keep in mind when talk­ing about val­ues:

  • In a nar­row sense, the word “val­ue” refers sim­ply to the eco­nom­ic worth of an object. This is not the mean­ing employed by Val­ue Sen­si­tive Design.
  • Val­ues should not be con­flat­ed with facts (the “fact/value dis­tinc­tion”) espe­cial­ly inso­far as facts do not log­i­cal­ly entail val­ue.
  • Is” does not imply “ought” (the nat­u­ral­is­tic fal­la­cy).
  • Val­ues can­not be moti­vat­ed only by an empir­i­cal account of the exter­nal world, but depend sub­stan­tive­ly on the inter­ests and desires of human beings with­in a cul­tur­al milieu. (So con­trary to what some right-wingers like to say: “Facts do care about your feel­ings.”)

Investigations

Let’s dig into the way this all works. “Val­ue Sen­si­tive Design is an iter­a­tive method­ol­o­gy that inte­grates con­cep­tu­al, empir­i­cal, and tech­ni­cal inves­ti­ga­tions.” So it dis­tin­guish­es between three types of activ­i­ties (“inves­ti­ga­tions”) and it pre­scribes cycling through these activ­i­ties mul­ti­ple times. Below are list­ed ques­tions and notes that are rel­e­vant to each type of inves­ti­ga­tion. But in brief, this is how I under­stand them:

  1. Defin­ing the spe­cif­ic val­ues at play in a project;
  2. Observ­ing, mea­sur­ing, and doc­u­ment­ing people’s behav­iour and the con­text of use;
  3. Analysing the ways in which a par­tic­u­lar tech­nol­o­gy sup­ports or hin­ders par­tic­u­lar val­ues.

Conceptual Investigations

  • Who are the direct and indi­rect stake­hold­ers affect­ed by the design at hand?
  • How are both class­es of stake­hold­ers affect­ed?
  • What val­ues are impli­cat­ed?
  • How should we engage in trade-offs among com­pet­ing val­ues in the design, imple­men­ta­tion, and use of infor­ma­tion sys­tems (e.g., auton­o­my vs. secu­ri­ty, or anonymi­ty vs. trust)?
  • Should moral val­ues (e.g., a right to pri­va­cy) have greater weight than, or even trump, non-moral val­ues (e.g., aes­thet­ic pref­er­ences)?

Empirical Investigations

  • How do stake­hold­ers appre­hend indi­vid­ual val­ues in the inter­ac­tive con­text?
  • How do they pri­ori­tise com­pet­ing val­ues in design trade-offs?
  • How do they pri­ori­tise indi­vid­ual val­ues and usabil­i­ty con­sid­er­a­tions?
  • Are there dif­fer­ences between espoused prac­tice (what peo­ple say) com­pared with actu­al prac­tice (what peo­ple do)?

And, specif­i­cal­ly focus­ing on organ­i­sa­tions:

  • What are organ­i­sa­tions’ moti­va­tions, meth­ods of train­ing and dis­sem­i­na­tion, reward struc­tures, and eco­nom­ic incen­tives?

Technical Investigations

Not a list of ques­tions here, but some notes:

Val­ue Sen­si­tive Design takes the posi­tion that tech­nolo­gies in gen­er­al, and infor­ma­tion and com­put­er tech­nolo­gies in par­tic­u­lar, have prop­er­ties that make them more or less suit­able for cer­tain activ­i­ties. A giv­en tech­nol­o­gy more read­i­ly sup­ports cer­tain val­ues while ren­der­ing oth­er activ­i­ties and val­ues more dif­fi­cult to realise.

Tech­ni­cal inves­ti­ga­tions involve the proac­tive design of sys­tems to sup­port val­ues iden­ti­fied in the con­cep­tu­al inves­ti­ga­tion.

Tech­ni­cal inves­ti­ga­tions focus on the tech­nol­o­gy itself. Empir­i­cal inves­ti­ga­tions focus on the indi­vid­u­als, groups, or larg­er social sys­tems that con­fig­ure, use, or are oth­er­wise affect­ed by the tech­nol­o­gy.

Significance

Below is a list of things that make Val­ue Sen­si­tive Design dif­fer­ent from oth­er approach­es, par­tic­u­lar­ly ones that pre­ced­ed it such as Com­put­er-Sup­port­ed Coop­er­a­tive Work and Par­tic­i­pa­to­ry Design.

  1. Val­ue Sen­si­tive Design seeks to be proac­tive
  2. Val­ue Sen­si­tive Design enlarges the are­na in which val­ues arise to include not only the work place
  3. Val­ue Sen­si­tive Design con­tributes a unique method­ol­o­gy that employs con­cep­tu­al, empir­i­cal, and tech­ni­cal inves­ti­ga­tions, applied iter­a­tive­ly and inte­gra­tive­ly
  4. Val­ue Sen­si­tive Design enlarges the scope of human val­ues beyond those of coop­er­a­tion (CSCW) and par­tic­i­pa­tion and democ­ra­cy (Par­tic­i­pa­to­ry Design) to include all val­ues, espe­cial­ly those with moral import.
  5. Val­ue Sen­si­tive Design dis­tin­guish­es between usabil­i­ty and human val­ues with eth­i­cal import.
  6. Val­ue Sen­si­tive Design iden­ti­fies and takes seri­ous­ly two class­es of stake­hold­ers: direct and indi­rect.
  7. Val­ue Sen­si­tive Design is an inter­ac­tion­al the­o­ry
  8. Val­ue Sen­si­tive Design builds from the psy­cho­log­i­cal propo­si­tion that cer­tain val­ues are uni­ver­sal­ly held, although how such val­ues play out in a par­tic­u­lar cul­ture at a par­tic­u­lar point in time can vary con­sid­er­ably

[ad 4] “By moral, we refer to issues that per­tain to fair­ness, jus­tice, human wel­fare and virtue, […] Val­ue Sen­si­tive Design also accounts for con­ven­tions (e.g., stan­dard­i­s­a­tion of pro­to­cols) and per­son­al val­ues”

[ad 5] “Usabil­i­ty refers to char­ac­ter­is­tics of a sys­tem that make it work in a func­tion­al sense, […] not all high­ly usable sys­tems sup­port eth­i­cal val­ues”

[ad 6] “Often, indi­rect stake­hold­ers are ignored in the design process.”

[ad 7] “val­ues are viewed nei­ther as inscribed into tech­nol­o­gy (an endoge­nous the­o­ry), nor as sim­ply trans­mit­ted by social forces (an exoge­nous the­o­ry). […] the inter­ac­tion­al posi­tion holds that while the fea­tures or prop­er­ties that peo­ple design into tech­nolo­gies more read­i­ly sup­port cer­tain val­ues and hin­der oth­ers, the technology’s actu­al use depends on the goals of the peo­ple inter­act­ing with it. […] through human inter­ac­tion, tech­nol­o­gy itself changes over time.”

[ad 8] “the more con­crete­ly (act-based) one con­cep­tu­alis­es a val­ue, the more one will be led to recog­nis­ing cul­tur­al vari­a­tion; con­verse­ly, the more abstract­ly one con­cep­tu­alis­es a val­ue, the more one will be led to recog­nis­ing uni­ver­sals”

How-To

Val­ue Sen­si­tive Design doesn’t pre­scribe a par­tic­u­lar process, which is fine by me, because I believe strong­ly in tai­lor­ing your process to the par­tic­u­lar project at hand. Part of being a thought­ful design­er is design­ing a project’s process as well. How­ev­er, some guid­ance is offered for how to pro­ceed in most cas­es. Here’s a list, plus some notes.

  1. Start with a val­ue, tech­nol­o­gy, or con­text of use
  2. Iden­ti­fy direct and indi­rect stake­hold­ers
  3. Iden­ti­fy ben­e­fits and harms for each stake­hold­er group
  4. Map ben­e­fits and harms onto cor­re­spond­ing val­ues
  5. Con­duct a con­cep­tu­al inves­ti­ga­tion of key val­ues
  6. Iden­ti­fy poten­tial val­ue con­flicts
  7. Inte­grate val­ue con­sid­er­a­tions into one’s organ­i­sa­tion­al struc­ture

[ad 1] “We sug­gest start­ing with the aspect that is most cen­tral to your work and inter­ests.”

[ad 2] “direct stake­hold­ers are those indi­vid­u­als who inter­act direct­ly with the tech­nol­o­gy or with the technology’s out­put. Indi­rect stake­hold­ers are those indi­vid­u­als who are also impact­ed by the sys­tem, though they nev­er inter­act direct­ly with it. […] With­in each of these two over­ar­ch­ing cat­e­gories of stake­hold­ers, there may be sev­er­al sub­groups. […] A sin­gle indi­vid­ual may be a mem­ber of more than one stake­hold­er group or sub­group. […] An organ­i­sa­tion­al pow­er struc­ture is often orthog­o­nal to the dis­tinc­tion between direct and indi­rect stake­hold­ers.”

[ad 3] “one rule of thumb in the con­cep­tu­al inves­ti­ga­tion is to give pri­or­i­ty to indi­rect stake­hold­ers who are strong­ly affect­ed, or to large groups that are some­what affect­ed […] Attend to issues of tech­ni­cal, cog­ni­tive, and phys­i­cal com­pe­ten­cy. […] per­sonas have a ten­den­cy to lead to stereo­types because they require a list of “social­ly coher­ent” attrib­ut­es to be asso­ci­at­ed with the “imag­ined indi­vid­ual.” […] we have devi­at­ed from the typ­i­cal use of per­sonas that maps a sin­gle per­sona onto a sin­gle user group, to allow for a sin­gle per­sona to map onto to mul­ti­ple stake­hold­er groups”

[ad 4] “In some cas­es, the cor­re­spond­ing val­ues will be obvi­ous, but not always.”

[ad 5] “the philo­soph­i­cal onto­log­i­cal lit­er­a­ture can help pro­vide cri­te­ria for what a val­ue is, and there­by how to assess it empir­i­cal­ly.”

[ad 6] “val­ue con­flicts should usu­al­ly not be con­ceived of as “either/or” sit­u­a­tions, but as con­straints on the design space.”

[ad 7] “In the real world, of course, human val­ues (espe­cial­ly those with eth­i­cal import) may col­lide with eco­nom­ic objec­tives, pow­er, and oth­er fac­tors. How­ev­er, even in such sit­u­a­tions, Val­ue Sen­si­tive Design should be able to make pos­i­tive con­tri­bu­tions, by show­ing alter­nate designs that bet­ter sup­port endur­ing human val­ues.”

Considering Values

Human values with ethical import often implicated in system design
Human val­ues with eth­i­cal import often impli­cat­ed in sys­tem design

This table is a use­ful heuris­tic tool for val­ues that might be con­sid­ered. The authors note that it is not intend­ed as a com­plete list of human val­ues that might be impli­cat­ed. Anoth­er more elab­o­rate tool of a sim­i­lar sort are the Envi­sion­ing Cards.

For the ethics nerds, it may be inter­est­ing to note that most of the val­ues in this table hinge on the deon­to­log­i­cal and con­se­quen­tial­ist moral ori­en­ta­tions. In addi­tion, the authors have chose sev­er­al oth­er val­ues relat­ed to sys­tem design.

Interviewing Stakeholders

When doing the empir­i­cal inves­ti­ga­tions you’ll prob­a­bly rely on stake­hold­er inter­views quite heav­i­ly. Stake­hold­er inter­views shouldn’t be a new thing to any design pro­fes­sion­al worth their salt. But the authors do offer some prac­ti­cal point­ers to keep in mind.

First of all, keep the inter­view some­what open-end­ed. This means con­duct­ing a semi-struc­tured inter­view. This will allow you to ask the things you want to know, but also cre­ates the oppor­tu­ni­ty for new and unex­pect­ed insights to emerge.

Laddering—repeatedly ask­ing the ques­tion “Why?” can get you quite far.

The most impor­tant thing, before inter­view­ing stake­hold­ers, is to have a good under­stand­ing of the sub­ject at hand. Demar­cate it using cri­te­ria that can be explained to out­siders. Use descrip­tions of issues or tasks for par­tic­i­pants to engage in, so that the sub­ject of the inves­ti­ga­tion becomes more con­crete.

Technical Investigations

Two things I find inter­est­ing here. First of all, we are encour­aged to map the rela­tion­ship between design trade-offs, val­ue con­flicts and stake­hold­er groups. The goal of this exer­cise is to be able to see how stake­hold­er groups are affect­ed in dif­fer­ent ways.

The sec­ond use­ful sug­ges­tion for tech­ni­cal inves­ti­ga­tions is to build flex­i­bil­i­ty into a prod­uct or service’s tech­ni­cal infra­struc­ture. The rea­son for this is that over time, new val­ues and val­ue con­flicts can emerge. As design­ers we are not always around any­more once a sys­tem is deployed so it is good prac­tice to enable the stake­hold­ers to adapt our design to their evolv­ing needs. (I was very much remind­ed of the approach advo­cat­ed by Stew­art Brand in How Build­ings Learn.)

Conclusion

When dis­cussing mat­ters of ethics in design with peers I often notice a reluc­tance to widen the scope of our prac­tice to include these issues. Fre­quent­ly, folks argue that since it is impos­si­ble to fore­see all the poten­tial con­se­quences of design choic­es, we can’t pos­si­bly be held account­able for all the ter­ri­ble things that can hap­pen as a result of a new tech­nol­o­gy being intro­duced into soci­ety.

I think that’s a mis­un­der­stand­ing of what eth­i­cal design is about. We may not always be direct­ly respon­si­ble for the con­se­quences of our design (both good and bad). But we are respon­si­ble for what we choose to make part of our con­cerns as we prac­tice design. This should include the val­ues con­sid­ered impor­tant by the peo­ple impact­ed by our designs.

In the 1996 arti­cle men­tioned at the start of this post, Fried­man con­cludes as fol­lows:

As with the tra­di­tion­al cri­te­ria of reli­a­bil­i­ty, effi­cien­cy, and cor­rect­ness, we do not require per­fec­tion in val­ue-sen­si­tive design, but a com­mit­ment. And progress.” (Fried­man 1996)

I think that is an apt place to end it here as well.

References

  • Fried­man, Batya. “Val­ue-sen­si­tive design.” inter­ac­tions 3.6 (1996): 16–23.
  • Fried­man, Batya, Peter Kahn, and Alan Born­ing. “Val­ue sen­si­tive design: The­o­ry and meth­ods.” Uni­ver­si­ty of Wash­ing­ton tech­ni­cal report (2002): 02–12.
  • Le Dan­tec, Christo­pher A., Eri­ka She­han Poole, and Susan P. Wyche. “Val­ues as lived expe­ri­ence: evolv­ing val­ue sen­si­tive design in sup­port of val­ue dis­cov­ery.” Pro­ceed­ings of the SIGCHI con­fer­ence on human fac­tors in com­put­ing sys­tems. ACM, 2009.
  • Born­ing, Alan, and Michael Muller. “Next steps for val­ue sen­si­tive design.” Pro­ceed­ings of the SIGCHI con­fer­ence on human fac­tors in com­put­ing sys­tems. ACM, 2012.
  • Frei­d­man, B., P. Kahn, and A. Born­ing. “Val­ue sen­si­tive design and infor­ma­tion sys­tems.” Human–computer inter­ac­tion in man­age­ment infor­ma­tion sys­tems: Foun­da­tions (2006): 348–372.

Playful Design for Workplace Change Management’ at PLAYTrack conference 2017 in Aarhus

Lase defender collab at FUSE

At the end of last year I was invit­ed to speak at the PLAY­Track con­fer­ence in Aarhus about the work­place change man­age­ment games made by Hub­bub. It turned out to be a great oppor­tu­ni­ty to recon­nect with the play research com­mu­ni­ty.

I was very much impressed by the pro­gram assem­bled by the organ­is­ers. Peo­ple came from a wide range of dis­ci­plines and cru­cial­ly, there was ample time to dis­cuss and reflect on the mate­ri­als pre­sent­ed. As I tweet­ed after­wards, this is a thing that most con­fer­ence organ­is­ers get wrong.

I was par­tic­u­lar­ly inspired by the work of Ben­jamin Mardell and Mara Krechevsky at Harvard’s Project ZeroMak­ing Learn­ing Vis­i­ble looks like a great resource for any­one who teach­es. Then there was Reed Stevens from North­west­ern Uni­ver­si­ty whose project FUSE is one of the most sol­id exam­ples of play­ful learn­ing for STEAM I’ve seen thus far. I was also fas­ci­nat­ed by Cia­ra Laverty’s work at PEDAL on observ­ing par­ent-child play. Miguel Sicart deliv­ered anoth­er great provo­ca­tion on the dark side of play­ful design. And final­ly I was delight­ed to hear about and expe­ri­ence for myself some of Amos Blan­ton’s work at the LEGO Foun­da­tion. I should also call out Ben Fin­cham’s many provoca­tive con­tri­bu­tions from the audi­ence.

The abstract for my talk is below, which cov­ers most of what I talked about. I tried to give peo­ple a good sense of:

  • what the games con­sist­ed of,
  • what we were aim­ing to achieve,
  • how both the fic­tion and the play­er activ­i­ties sup­port­ed these goals,
  • how we made learn­ing out­comes vis­i­ble to our play­ers and clients,
  • and final­ly how we went about design­ing and devel­op­ing these games.

Both projects have sol­id write-ups over at the Hub­bub web­site, so I’ll just point to those here: Code 4 and Rip­ple Effect.

In the final sec­tion of the talk I spent a bit of time reflect­ing on how I would approach projects like this today. After all, it has been sev­en years since we made Code 4, and four years since Rip­ple Effect. That’s ages ago and my per­spec­tive has def­i­nite­ly changes since we made these.

Participatory design

First of all, I would get even more seri­ous about co-design­ing with play­ers at every step. I would recruit rep­re­sen­ta­tives of play­ers and invest them with real influ­ence. In the projects we did, the pri­ma­ry vehi­cle for play­er influ­ence was through playtest­ing. But this is nec­es­sar­i­ly lim­it­ed. I also won’t pre­tend this is at all easy to do in a com­mer­cial con­text.

But, these games are ulti­mate­ly about improv­ing work­er pro­duc­tiv­i­ty. So how do we make it so that work­ers share in the real-world prof­its yield­ed by a suc­cess­ful cul­ture change?

I know of the exis­tence of par­tic­i­pa­to­ry design but from my expe­ri­ence it is not a com­mon approach in the indus­try. Why?

Value sensitive design

On a relat­ed note, I would get more seri­ous about what val­ues are sup­port­ed by the sys­tem, in whose inter­est they are and where they come from. Ear­ly field research and work­shops with audi­ence do sur­face some val­ues but val­ues from cus­tomer rep­re­sen­ta­tives tend to dom­i­nate. Again, the com­mer­cial con­text we work in is a poten­tial chal­lenge.

I know of val­ue sen­si­tive design, but as with par­tic­i­pa­to­ry design, it has yet to catch on in a big way in the indus­try. So again, why is that?

Disintermediation

One thing I con­tin­ue to be inter­est­ed in is to reduce the com­plex­i­ty of a game system’s phys­i­cal affor­dances (which includes its code), and to push even more of the sub­stance of the game into those social allowances that make up the non-mate­r­i­al aspects of the game. This allows for spon­ta­neous rene­go­ti­a­tion of the game by the play­ers. This is dis­in­ter­me­di­a­tion as a strat­e­gy. David Kanaga’s take on games as toys remains huge­ly inspi­ra­tional in this regard, as does Bernard De Koven’s book The Well Played Game.

Gamefulness versus playfulness

Code 4 had more focus on sat­is­fy­ing the need for auton­o­my. Rip­ple Effect had more focus on com­pe­tence, or in any case, it had less empha­sis on auton­o­my. There was less room for ‘play’ around the core dig­i­tal game. It seems to me that mas­ter­ing a sub­jec­tive sim­u­la­tion of a sub­ject is not nec­es­sar­i­ly what a work­place game for cul­ture change should be aim­ing for. So, less game­ful design, more play­ful design.

Adaptation

Final­ly, the agency mod­el does not enable us to stick around for the long haul. But work­place games might be bet­ter suit­ed to a set­up where things aren’t thought of as a one-off project but more of an ongo­ing process.

In How Build­ings Learn, Stew­art Brand talks about how archi­tects should revis­it build­ings they’ve designed after they are built to learn about how peo­ple are actu­al­ly using them. He also talks about how good build­ings are build­ings that its inhab­i­tants can adapt to their needs. What does that look like in the con­text of a game for work­place cul­ture change?


Play­ful Design for Work­place Change Man­age­ment

Code 4 (2011, com­mis­sioned by the Tax Admin­is­tra­tion of the Nether­lands) and Rip­ple Effect (2013, com­mis­sioned by Roy­al Dutch Shell) are both games for work­place change man­age­ment designed and devel­oped by Hub­bub, a bou­tique play­ful design agency which oper­at­ed from Utrecht, The Nether­lands and Berlin, Ger­many between 2009 and 2015. These games are exam­ples of how a goal-ori­ent­ed seri­ous game can be used to encour­age play­ful appro­pri­a­tion of work­place infra­struc­ture and social norms, result­ing in an open-end­ed and cre­ative explo­ration of new and inno­v­a­tive ways of work­ing.

Seri­ous game projects are usu­al­ly com­mis­sioned to solve prob­lems. Solv­ing the prob­lem of cul­tur­al change in a straight­for­ward man­ner means view­ing games as a way to per­suade work­ers of a desired future state. They typ­i­cal­ly take videogame form, sim­u­lat­ing the desired new way of work­ing as deter­mined by man­age­ment. To play the game well, play­ers need to mas­ter its sys­tem and by extension—it is assumed—learning hap­pens.

These games can be be enjoy­able expe­ri­ences and an improve­ment on pre­vi­ous forms of work­place learn­ing, but in our view they decrease the pos­si­bil­i­ty space of poten­tial work­place cul­tur­al change. They dimin­ish work­er agency, and they waste the cre­ative and inno­v­a­tive poten­tial of involv­ing them in the inven­tion of an improved work­place cul­ture.

We instead choose to view work­place games as an oppor­tu­ni­ty to increase the space of pos­si­bil­i­ty. We resist the temp­ta­tion to bake the desired new way of work­ing into the game’s phys­i­cal and dig­i­tal affor­dances. Instead, we leave how to play well up to the play­ers. Since these games are team-based and col­lab­o­ra­tive, play­ers need to nego­ti­ate their way of work­ing around the game among them­selves. In addi­tion, because the games are dis­trib­uted in time—running over a num­ber of weeks—and are playable at play­er dis­cre­tion dur­ing the work­day, play­ers are giv­en license to appro­pri­ate work­place infra­struc­ture and sub­vert social norms towards in-game ends.

We tried to make learn­ing tan­gi­ble in var­i­ous ways. Because the games at the core are web appli­ca­tions to which play­ers log on with indi­vid­ual accounts we were able to col­lect data on play­er behav­iour. To guar­an­tee pri­va­cy, employ­ers did not have direct access to game data­bas­es and only received anonymised reports. We took respon­si­bil­i­ty for play­er learn­ing by facil­i­tat­ing coach­ing ses­sions in which they could safe­ly reflect on their game expe­ri­ences. Round­ing out these efforts, we con­duct­ed sur­veys to gain insight into the play­er expe­ri­ence from a more qual­i­ta­tive and sub­jec­tive per­spec­tive.

These games offer a mod­el for a rea­son­ably demo­c­ra­t­ic and eth­i­cal way of doing game-based work­place change man­age­ment. How­ev­er, we would like to see efforts that fur­ther democ­ra­tise their design and development—involving work­ers at every step. We also wor­ry about how games can be used to cre­ate the illu­sion of work­er influ­ence while at the same time soft­ware is deployed through­out the work­place to lim­it their agency.

Our exam­ples may be inspir­ing but because of these devel­op­ments we feel we can’t con­tin­ue this type of work with­out seri­ous­ly recon­sid­er­ing our cur­rent process­es, tech­nol­o­gy stacks and busi­ness practices—and ulti­mate­ly whether we should be mak­ing games at all.

Books I’ve read in 2017

Return­ing to what is some­thing of an annu­al tra­di­tion, these are the books I’ve read in 2017. I set myself the goal of get­ting to 36 and man­aged 38 in the end. They’re list­ed below with some com­men­tary on par­tic­u­lar­ly mem­o­rable or oth­er­wise note­wor­thy reads. To make things a bit more user friend­ly I’ve gone with four broad buck­ets although as you’ll see with­in each the picks range across gen­res and sub­jects.

Fiction

I always have one piece of fic­tion or nar­ra­tive non-fic­tion going. I have a long-stand­ing ‘project’ of read­ing cult clas­sics. I can’t set­tle on a top pick for the first cat­e­go­ry so it’s going to have to be a tie between Lowry’s alco­hol-drenched tale of lost love in pre-WWII Mex­i­co, and Salter’s unmatched lyri­cal prose treat­ment of a young couple’s liaisons as imag­ined by a lech­er­ous recluse in post-WWII France.

When I feel like some­thing lighter I tend to seek out sci-fi writ­ten from before I was born. (Con­tem­po­rary sci-fi more often than not dis­ap­points me with its lack of imag­i­na­tion, or worse, nos­tal­gia for futures past. I’m look­ing at you, Cline.) My top pick here would be the Stru­gatsky broth­ers, who blew me away with their weird tale of a world for­ev­er changed by the inex­plic­a­ble vis­it by some­thing tru­ly alien.

I’ve also con­tin­ued to seek out works by women, although I’ve been less strict with myself in this depart­ment than pre­vi­ous years. Here I’m ashamed to admit it took me this long to final­ly read any­thing by Woolf because Mrs Dal­loway is every bit as good as they say it is. I rec­om­mend seek­ing out the anno­tat­ed Pen­guin addi­tion for addi­tion­al insights into the many things she ref­er­ences.

I’ve also some­times picked up a new­er book because it popped up on my radar and I was just real­ly excit­ed about read­ing it. Most notably Dolan’s retelling of the Ili­ad in all its glo­ri­ous, sad and gory detail, updat­ed for today’s sen­si­bil­i­ties.

Literary non-fiction

Each time I read a nar­ra­tive treat­ment of his­to­ry or cur­rent affairs I feel like I should be doing more of it. All of these are rec­om­mend­ed but Kapuś­cińs­ki tow­ers over all with his heart-wrench­ing first-per­son account of the Iran­ian rev­o­lu­tion.

Non-fiction

A few books on design and tech­nol­o­gy here, although most of my ‘pro­fes­sion­al’ read­ing was con­fined to aca­d­e­m­ic papers this year. I find those to be a more effec­tive way of get­ting a han­dle on a par­tic­u­lar sub­ject. Books pub­lished on my méti­er are noto­ri­ous­ly fluffy. I’ll point out Löw­gren for a tough but reward­ing read on how to do inter­ac­tion design in a non-dog­mat­ic but reflec­tive way.

I got into left­ist pol­i­tics quite heav­i­ly this year and tried to edu­cate myself a bit on con­tem­po­rary anti-cap­i­tal­ist think­ing. Fisher’s book is a most inter­est­ing and also amus­ing diag­no­sis of the cur­rent polit­i­cal and eco­nom­ic world sys­tem through a cul­tur­al lens. It’s a shame he’s no longer with us, I won­der what he would have made of recent events.

Game books

I decid­ed to work my way through a bunch of role­play­ing game books all ‘pow­ered by the apoc­a­lypse’ – a fam­i­ly of games which I have been aware of for quite a while but haven’t had the oppor­tu­ni­ty to play myself. I like read­ing these because I find them odd­ly inspi­ra­tional for pro­fes­sion­al pur­pos­es. But I will point to the orig­i­nal Apoc­a­lypse World as the one must-read as Bak­er remains one of the design­ers I am absolute­ly in awe of for the ways in which he man­ages to com­bine sys­tem and fic­tion in tru­ly inven­tive ways.

  • The Per­ilous Wilds, Jason Lutes
  • Urban Shad­ows: Polit­i­cal Urban Fan­ta­sy Pow­ered by the Apoc­a­lypse, Andrew Medeiros
  • Dun­geon World, Sage LaTor­ra
  • Apoc­a­lypse World, D. Vin­cent Bak­er

Poetry

I don’t usu­al­ly read poet­ry for rea­sons sim­i­lar to how I basi­cal­ly stopped read­ing comics ear­li­er: I can’t seem to find a good way of dis­cov­er­ing worth­while things to read. The col­lec­tion below was a gift, and a delight­ful one.

As always, I wel­come sug­ges­tions for what to read next. I’m shoot­ing for 36 again this year and plan to pro­ceed rough­ly as I’ve been doing lately—just mean­der from book to book with a bias towards works that are non-anglo, at least as old as I am, and prefer­ably weird or inven­tive.

Pre­vi­ous years: 2016, 2015, 2011, 2009.

Prototyping the Useless Butler: Machine Learning for IoT Designers

ThingsCon Amsterdam 2017, photo by nunocruzstreet.com
ThingsCon Ams­ter­dam 2017, pho­to by nunocruzstreet.com

At ThingsCon Ams­ter­dam 2017, Péter and I ran a sec­ond iter­a­tion of our machine learn­ing work­shop. We improved on our first attempt at TU Delft in a num­ber of ways.

  • We pre­pared exam­ple code for com­mu­ni­cat­ing with Wek­ina­tor from a wifi con­nect­ed Arduino MKR1000 over OSC.
  • We cre­at­ed a pre­de­fined bread­board set­up.
  • We devel­oped three exer­cis­es, one for each type of Wek­ina­tor out­put: regres­sion, clas­si­fi­ca­tion and dynam­ic time warp­ing.

In con­trast to the first ver­sion, we had two hours to run through the whole thing, in stead of a day… So we had to cut some cor­ners, and dou­bled down on walk­ing par­tic­i­pants through a num­ber of exer­cis­es so that they would come out of it with some read­i­ly applic­a­ble skills.

We dubbed the work­shop ‘pro­to­typ­ing the use­less but­ler’, with thanks to Philip van Allen for the sug­ges­tion to frame the exer­cis­es around build­ing some­thing non-pro­duc­tive so that the focus was shift­ed to play and explo­ration.

All of the code, the cir­cuit dia­gram and slides are over on GitHub. But I’ll sum­marise things here.

  1. We spent a very short amount of time intro­duc­ing machine learn­ing. We used Google’s Teach­able Machine as an exam­ple and con­trast­ed reg­u­lar pro­gram­ming with using machine learn­ing algo­rithms to train mod­els. The point was to pro­vide folks with just enough con­cep­tu­al scaf­fold­ing so that the rest of the work­shop would make sense.
  2. We then intro­duced our ‘tool­chain’ which con­sists of Wek­ina­tor, the Arduino MKR1000 mod­ule and the OSC pro­to­col. The aim of this tool­chain is to allow design­ers who work in the IoT space to get a feel for the mate­r­i­al prop­er­ties of machine learn­ing through hands-on tin­ker­ing. We tried to cre­ate a tool­chain with as few mov­ing parts as pos­si­ble, because each addi­tion­al com­po­nent would intro­duce anoth­er point of fail­ure which might require debug­ging. This tool­chain would enable design­ers to either use machine learn­ing to rapid­ly pro­to­type inter­ac­tive behav­iour with min­i­mal or no pro­gram­ming. It can also be used to pro­to­type prod­ucts that expose inter­ac­tive machine learn­ing fea­tures to end users. (For a spec­u­la­tive exam­ple of one such prod­uct, see Bjørn Karmann’s Objec­ti­fi­er.)
  3. Par­tic­i­pants were then asked to set up all the required parts on their own work­sta­tion. A list can be found on the Use­less But­ler GitHub page.
  4. We then pro­ceed­ed to build the cir­cuit. We pro­vid­ed all the com­po­nents and showed a Fritz­ing dia­gram to help peo­ple along. The basic idea of this cir­cuit, the epony­mous use­less but­ler, was to have a suf­fi­cient­ly rich set of inputs and out­puts with which to play, that would suit all three types of Wek­ina­tor out­put. So we set­tled on a pair of pho­tore­sis­tors or LDRs as inputs and an RGB LED as out­put.
  5. With the pre­req­ui­sites installed and the cir­cuit built we were ready to walk through the exam­ples. For regres­sion we mapped the con­tin­u­ous stream of read­ings from the two LDRs to three out­puts, one each for the red, green and blue of the LED. For clas­si­fi­ca­tion we put the state of both LDRs into one of four cat­e­gories, each switch­ing the RGB LED to a spe­cif­ic col­or (cyan, magen­ta, yel­low or white). And final­ly, for dynam­ic time warp­ing, we asked Wek­ina­tor to recog­nise one of three ges­tures and switch the RGB LED to one of three states (red, green or off).

When we reflect­ed on the work­shop after­wards, we agreed we now have a proven con­cept. Par­tic­i­pants were able to get the tool­chain up and run­ning and could play around with iter­a­tive­ly train­ing and eval­u­at­ing their mod­el until it behaved as intend­ed.

How­ev­er, there is still quite a bit of room for improve­ment. On a prac­ti­cal note, quite a bit of time was tak­en up by the build­ing of the cir­cuit, which isn’t the point of the work­shop. One way of deal­ing with this is to bring those to a work­shop pre-built. Doing so would enable us to get to the machine learn­ing quick­er and would open up time and space to also engage with the par­tic­i­pants about the point of it all.

We’re keen on bring­ing this work­shop to more set­tings in future. If we do, I’m sure we’ll find the oppor­tu­ni­ty to improve on things once more and I will report back here.

Many thanks to Iskan­der and the rest of the ThingsCon team for invit­ing us to the con­fer­ence.

ThingsCon Amsterdam 2017, photo by nunocruzstreet.com
ThingsCon Ams­ter­dam 2017, pho­to by nunocruzstreet.com

Design and machine learning – an annotated reading list

Ear­li­er this year I coached Design for Inter­ac­tion mas­ter stu­dents at Delft Uni­ver­si­ty of Tech­nol­o­gy in the course Research Method­ol­o­gy. The stu­dents organ­ised three sem­i­nars for which I pro­vid­ed the claims and assigned read­ing. In the sem­i­nars they argued about my claims using the Toul­min Mod­el of Argu­men­ta­tion. The read­ings served as sources for back­ing and evi­dence.

The claims and read­ings were all relat­ed to my nascent research project about machine learn­ing. We delved into both design­ing for machine learn­ing, and using machine learn­ing as a design tool.

Below are the read­ings I assigned, with some notes on each, which should help you decide if you want to dive into them your­self.

Hebron, Patrick. 2016. Machine Learn­ing for Design­ers. Sebastopol: O’Reilly.

The only non-aca­d­e­m­ic piece in this list. This served the pur­pose of get­ting all stu­dents on the same page with regards to what machine learn­ing is, its appli­ca­tions of machine learn­ing in inter­ac­tion design, and com­mon chal­lenges encoun­tered. I still can’t think of any oth­er sin­gle resource that is as good a start­ing point for the sub­ject as this one.

Fiebrink, Rebec­ca. 2016. “Machine Learn­ing as Meta-Instru­ment: Human-Machine Part­ner­ships Shap­ing Expres­sive Instru­men­tal Cre­ation.” In Musi­cal Instru­ments in the 21st Cen­tu­ry, 14:137–51. Sin­ga­pore: Springer Sin­ga­pore. doi:10.1007/978–981–10–2951–6_10.

Fiebrink’s Wek­ina­tor is ground­break­ing, fun and inspir­ing so I had to include some of her writ­ing in this list. This is most­ly of inter­est for those look­ing into the use of machine learn­ing for design and oth­er cre­ative and artis­tic endeav­ours. An impor­tant idea explored here is that tools that make use of (inter­ac­tive, super­vised) machine learn­ing can be thought of as instru­ments. Using such a tool is like play­ing or per­form­ing, explor­ing a pos­si­bil­i­ty space, engag­ing in a dia­logue with the tool. For a tool to feel like an instru­ment requires a tight action-feed­back loop.

Dove, Gra­ham, Kim Hal­skov, Jodi For­l­izzi, and John Zim­mer­man. 2017. UX Design Inno­va­tion: Chal­lenges for Work­ing with Machine Learn­ing as a Design Mate­r­i­al. The 2017 CHI Con­fer­ence. New York, New York, USA: ACM. doi:10.1145/3025453.3025739.

A real­ly good sur­vey of how design­ers cur­rent­ly deal with machine learn­ing. Key take­aways include that in most cas­es, the appli­ca­tion of machine learn­ing is still engi­neer­ing-led as opposed to design-led, which ham­pers the cre­ation of non-obvi­ous machine learn­ing appli­ca­tions. It also makes it hard for design­ers to con­sid­er eth­i­cal impli­ca­tions of design choic­es. A key rea­son for this is that at the moment, pro­to­typ­ing with machine learn­ing is pro­hib­i­tive­ly cum­ber­some.

Fiebrink, Rebec­ca, Per­ry R Cook, and Dan True­man. 2011. “Human Mod­el Eval­u­a­tion in Inter­ac­tive Super­vised Learn­ing.” In, 147. New York, New York, USA: ACM Press. doi:10.1145/1978942.1978965.

The sec­ond Fiebrink piece in this list, which is more of a deep dive into how peo­ple use Wek­ina­tor. As with the chap­ter list­ed above this is required read­ing for those work­ing on design tools which make use of inter­ac­tive machine learn­ing. An impor­tant find­ing here is that users of intel­li­gent design tools might have very dif­fer­ent cri­te­ria for eval­u­at­ing the ‘cor­rect­ness’ of a trained mod­el than engi­neers do. Such cri­te­ria are like­ly sub­jec­tive and eval­u­a­tion requires first-hand use of the mod­el in real time.

Bostrom, Nick, and Eliez­er Yud­kowsky. 2014. “The Ethics of Arti­fi­cial Intel­li­gence.” In The Cam­bridge Hand­book of Arti­fi­cial Intel­li­gence, edit­ed by Kei­th Frank­ish and William M Ram­sey, 316–34. Cam­bridge: Cam­bridge Uni­ver­si­ty Press. doi:10.1017/CBO9781139046855.020.

Bostrom is known for his some­what crazy but thought­pro­vok­ing book on super­in­tel­li­gence and although a large part of this chap­ter is about the ethics of gen­er­al arti­fi­cial intel­li­gence (which at the very least is still a way out), the first sec­tion dis­cuss­es the ethics of cur­rent “nar­row” arti­fi­cial intel­li­gence. It makes for a good check­list of things design­ers should keep in mind when they cre­ate new appli­ca­tions of machine learn­ing. Key insight: when a machine learn­ing sys­tem takes on work with social dimensions—tasks pre­vi­ous­ly per­formed by humans—the sys­tem inher­its its social require­ments.

Yang, Qian, John Zim­mer­man, Aaron Ste­in­feld, and Antho­ny Toma­sic. 2016. Plan­ning Adap­tive Mobile Expe­ri­ences When Wire­fram­ing. The 2016 ACM Con­fer­ence. New York, New York, USA: ACM. doi:10.1145/2901790.2901858.

Final­ly, a feet-in-the-mud explo­ration of what it actu­al­ly means to design for machine learn­ing with the tools most com­mon­ly used by design­ers today: draw­ings and dia­grams of var­i­ous sorts. In this case the focus is on using machine learn­ing to make an inter­face adap­tive. It includes an inter­est­ing dis­cus­sion of how to bal­ance the use of implic­it and explic­it user inputs for adap­ta­tion, and how to deal with infer­ence errors. Once again the lim­i­ta­tions of cur­rent sketch­ing and pro­to­typ­ing tools is men­tioned, and relat­ed to the need for design­ers to devel­op tac­it knowl­edge about machine learn­ing. Such tac­it knowl­edge will only be gained when design­ers can work with machine learn­ing in a hands-on man­ner.

Supplemental material

Floyd, Chris­tiane. 1984. “A Sys­tem­at­ic Look at Pro­to­typ­ing.” In Approach­es to Pro­to­typ­ing, 1–18. Berlin, Hei­del­berg: Springer Berlin Hei­del­berg. doi:10.1007/978–3–642–69796–8_1.

I pro­vid­ed this to stu­dents so that they get some addi­tion­al ground­ing in the var­i­ous kinds of pro­to­typ­ing that are out there. It helps to pre­vent reduc­tive notions of pro­to­typ­ing, and it makes for a nice com­ple­ment to Buxton’s work on sketch­ing.

Ble­vis, E, Y Lim, and E Stolter­man. 2006. “Regard­ing Soft­ware as a Mate­r­i­al of Design.”

Some of the papers refer to machine learn­ing as a “design mate­r­i­al” and this paper helps to under­stand what that idea means. Soft­ware is a mate­r­i­al with­out qual­i­ties (it is extreme­ly mal­leable, it can sim­u­late near­ly any­thing). Yet, it helps to con­sid­er it as a phys­i­cal mate­r­i­al in the metaphor­i­cal sense because we can then apply ways of design think­ing and doing to soft­ware pro­gram­ming.

Status update

This is not exact­ly a now page, but I thought I would write up what I am doing at the moment since last report­ing on my sta­tus in my end-of-year report.

The major­i­ty of my work­days are spent doing free­lance design con­sult­ing. My pri­ma­ry gig has been through Eend at the Dutch Vic­tim Sup­port Foun­da­tion, where until very recent­ly I was part of a team build­ing online ser­vices. I helped out with prod­uct strat­e­gy, set­ting up a lean UX design process, and get­ting an inte­grat­ed agile design and devel­op­ment team up and run­ning. The first ser­vices are now ship­ping so it is time for me to move on, after 10 months of very grat­i­fy­ing work. I real­ly enjoy work­ing in the pub­lic sec­tor and I hope to be doing more of it in future.

So yes, this means I am avail­able and you can hire me to do strat­e­gy and design for soft­ware prod­ucts and ser­vices. Just send me an email.

Short­ly before the Dutch nation­al elec­tions of this year, Iskan­der and I gath­ered a group of fel­low tech work­ers under the ban­ner of “Tech Sol­i­dar­i­ty NL to dis­cuss the con­cern­ing lurch to the right in nation­al pol­i­tics and what our field can do about it. This has devel­oped into a small but active com­mu­ni­ty who gath­er month­ly to edu­cate our­selves and devel­op plans for col­lec­tive action. I am get­ting a huge boost out of this. Fig­ur­ing out how to be a left­ist in this day and age is not easy. The only way to do it is to prac­tice and for that reflec­tion with peers is invalu­able. Build­ing and facil­i­tat­ing a group like this is huge­ly edu­ca­tion­al too. I have learned a lot about how a com­mu­ni­ty is boot-strapped and nur­tured.

If you are in the Nether­lands, your pol­i­tics are left of cen­ter, and you work in tech­nol­o­gy, con­sid­er your­self invit­ed to join.

And final­ly, the last major thing on my plate is a con­tin­u­ing effort to secure a PhD posi­tion for myself. I am get­ting great sup­port from peo­ple at Delft Uni­ver­si­ty of Tech­nol­o­gy, in par­tic­u­lar Gerd Kortuem. I am focus­ing on inter­net of things prod­ucts that have fea­tures dri­ven by machine learn­ing. My ulti­mate aim is to devel­op pro­to­typ­ing tools for design and devel­op­ment teams that will help them cre­ate more inno­v­a­tive and more eth­i­cal solu­tions. The first step for this will be to con­duct field research inside com­pa­nies who are cre­at­ing such prod­ucts right now. So I am reach­ing out to peo­ple to see if I can secure a rea­son­able amount of poten­tial col­lab­o­ra­tors for this, which will go a long way in prov­ing the fea­si­bil­i­ty of my whole plan.

If you know of any com­pa­nies that devel­op con­sumer-fac­ing prod­ucts that have a con­nect­ed hard­ware com­po­nent and make use of machine learn­ing to dri­ve fea­tures, do let me know.

That’s about it. Free­lance UX con­sult­ing, left­ist tech-work­er organ­is­ing and design-for-machine-learn­ing research. Quite hap­py with that mix, real­ly.