Contestable Infrastructures” at Beyond Smart Cities Today

I’ll be at Beyond Smart Cities Today the next cou­ple of days (18–19 Sep­tem­ber). Below is the abstract I sub­mit­ted, plus a bib­li­og­ra­phy of some of the stuff that went into my think­ing for this and relat­ed mat­ters that I won’t have the time to get into.

In the actu­al­ly exist­ing smart city, algo­rith­mic sys­tems are increas­ing­ly used for the pur­pos­es of auto­mat­ed deci­sion-mak­ing, includ­ing as part of pub­lic infra­struc­ture. Algo­rith­mic sys­tems raise a range of eth­i­cal con­cerns, many of which stem from their opac­i­ty. As a result, pre­scrip­tions for improv­ing the account­abil­i­ty, trust­wor­thi­ness and legit­i­ma­cy of algo­rith­mic sys­tems are often based on a trans­paren­cy ide­al. The think­ing goes that if the func­tion­ing and own­er­ship of an algo­rith­mic sys­tem is made per­ceiv­able, peo­ple under­stand them and are in turn able to super­vise them. How­ev­er, there are lim­its to this approach. Algo­rith­mic sys­tems are com­plex and ever-chang­ing socio-tech­ni­cal assem­blages. Ren­der­ing them vis­i­ble is not a straight­for­ward design and engi­neer­ing task. Fur­ther­more such trans­paren­cy does not nec­es­sar­i­ly lead to under­stand­ing or, cru­cial­ly, the abil­i­ty to act on this under­stand­ing. We believe legit­i­mate smart pub­lic infra­struc­ture needs to include the pos­si­bil­i­ty for sub­jects to artic­u­late objec­tions to pro­ce­dures and out­comes. The result­ing “con­testable infra­struc­ture” would cre­ate spaces that open up the pos­si­bil­i­ty for express­ing con­flict­ing views on the smart city. Our project is to explore the design impli­ca­tions of this line of rea­son­ing for the phys­i­cal assets that cit­i­zens encounter in the city. Because after all, these are the per­ceiv­able ele­ments of the larg­er infra­struc­tur­al sys­tems that recede from view.

  • Alkhat­ib, A., & Bern­stein, M. (2019). Street-Lev­el Algo­rithms. 1–13. https://doi.org/10.1145/3290605.3300760
  • Anan­ny, M., & Craw­ford, K. (2018). See­ing with­out know­ing: Lim­i­ta­tions of the trans­paren­cy ide­al and its appli­ca­tion to algo­rith­mic account­abil­i­ty. New Media and Soci­ety, 20(3), 973–989. https://doi.org/10.1177/1461444816676645
  • Cen­ti­vany, A., & Glushko, B. (2016). “Pop­corn tastes good”: Par­tic­i­pa­to­ry pol­i­cy­mak­ing and Reddit’s “AMAged­don.” Con­fer­ence on Human Fac­tors in Com­put­ing Sys­tems — Pro­ceed­ings, 1126–1137. https://doi.org/10.1145/2858036.2858516
  • Craw­ford, K. (2016). Can an Algo­rithm be Ago­nis­tic? Ten Scenes from Life in Cal­cu­lat­ed Publics. Sci­ence Tech­nol­o­gy and Human Val­ues, 41(1), 77–92. https://doi.org/10.1177/0162243915589635
  • DiS­al­vo, C. (2010). Design, Democ­ra­cy and Ago­nis­tic Plu­ral­ism. Pro­ceed­ings of the Design Research Soci­ety Con­fer­ence, 366–371.
  • Hilde­brandt, M. (2017). Pri­va­cy As Pro­tec­tion of the Incom­putable Self: Ago­nis­tic Machine Learn­ing. SSRN Elec­tron­ic Jour­nal, 1–33. https://doi.org/10.2139/ssrn.3081776
  • Jack­son, S. J., Gille­spie, T., & Payette, S. (2014). The Pol­i­cy Knot: Re-inte­grat­ing Pol­i­cy, Prac­tice and Design. CSCW Stud­ies of Social Com­put­ing, 588–602. https://doi.org/10.1145/2531602.2531674
  • Jew­ell, M. (2018). Con­test­ing the deci­sion: liv­ing in (and liv­ing with) the smart city. Inter­na­tion­al Review of Law, Com­put­ers and Tech­nol­o­gy. https://doi.org/10.1080/13600869.2018.1457000
  • Lind­blom, L. (2019). Con­sent, Con­testa­bil­i­ty, and Unions. Busi­ness Ethics Quar­ter­ly. https://doi.org/10.1017/beq.2018.25
  • Mit­tel­stadt, B. D., Allo, P., Tad­deo, M., Wachter, S., & Flori­di, L. (2016). The ethics of algo­rithms: Map­ping the debate. Big Data & Soci­ety, 3(2), 205395171667967. https://doi.org/10.1177/2053951716679679
  • Van de Poel, I. (2016). An eth­i­cal frame­work for eval­u­at­ing exper­i­men­tal tech­nol­o­gy. Sci­ence and Engi­neer­ing Ethics, 22(3), 667–686. https://doi.org/10.1007/s11948-015‑9724-3

Contestable Infrastructures: Designing for Dissent in Smart Public Objects” at We Make the City 2019

Thi­js Turèl of AMS Insti­tute and myself pre­sent­ed a ver­sion of the talk below at the Cities for Dig­i­tal Rights con­fer­ence on June 19 in Ams­ter­dam dur­ing the We Make the City fes­ti­val. The talk is an attempt to artic­u­late some of the ideas we both have been devel­op­ing for some time around con­testa­bil­i­ty in smart pub­lic infra­struc­ture. As always with this sort of thing, this is intend­ed as a con­ver­sa­tion piece so I wel­come any thoughts you may have.


The basic mes­sage of the talk is that when we start to do auto­mat­ed deci­sion-mak­ing in pub­lic infra­struc­ture using algo­rith­mic sys­tems, we need to design for the inevitable dis­agree­ments that may arise and fur­ther­more, we sug­gest there is an oppor­tu­ni­ty to focus on design­ing for such dis­agree­ments in the phys­i­cal objects that peo­ple encounter in urban space as they make use of infra­struc­ture.

We set the scene by show­ing a num­ber of exam­ples of smart pub­lic infra­struc­ture. A cyclist cross­ing that adapts to weath­er con­di­tions. If it’s rain­ing cyclists more fre­quent­ly get a green light. A pedes­tri­an cross­ing in Tilburg where elder­ly can use their mobile to get more time to cross. And final­ly, the case we are involved with our­selves: smart EV charg­ing in the city of Ams­ter­dam, about which more lat­er.

Image cred­its: Vat­ten­fall, Fietsfan010, De Nieuwe Draai

We iden­ti­fy three trends in smart pub­lic infra­struc­ture: (1) where pre­vi­ous­ly algo­rithms were used to inform pol­i­cy, now they are employed to per­form auto­mat­ed deci­sion-mak­ing on an indi­vid­ual case basis. This rais­es the stakes; (2) dis­trib­uted own­er­ship of these sys­tems as the result of pub­lic-pri­vate part­ner­ships and oth­er com­plex col­lab­o­ra­tion schemes leads to unclear respon­si­bil­i­ty; and final­ly (3) the increas­ing use of machine learn­ing leads to opaque deci­sion-mak­ing.

These trends, and algo­rith­mic sys­tems more gen­er­al­ly, raise a num­ber of eth­i­cal con­cerns. They include but are not lim­it­ed to: the use of induc­tive cor­re­la­tions (for exam­ple in the case of machine learn­ing) leads to unjus­ti­fied results; lack of access to and com­pre­hen­sion of a system’s inner work­ings pro­duces opac­i­ty, which in turn leads to a lack of trust in the sys­tems them­selves and the organ­i­sa­tions that use them; bias is intro­duced by a num­ber of fac­tors, includ­ing devel­op­ment team prej­u­dices, tech­ni­cal flaws, bad data and unfore­seen inter­ac­tions with oth­er sys­tems; and final­ly the use of pro­fil­ing, nudg­ing and per­son­al­i­sa­tion leads to dimin­ished human agency. (We high­ly rec­om­mend the arti­cle by Mit­tel­stadt et al. for a com­pre­hen­sive overview of eth­i­cal con­cerns raised by algo­rithms.)

So for us, the ques­tion that emerges from all this is: How do we organ­ise the super­vi­sion of smart pub­lic infra­struc­ture in a demo­c­ra­t­ic and law­ful way?

There are a num­ber of exist­ing approach­es to this ques­tion. These include legal and reg­u­la­to­ry (e.g. the right to expla­na­tion in the GDPR); audit­ing (e.g. KPMG’s AI in Con­trol” method, BKZ’s transparantielab); pro­cure­ment (e.g. open source claus­es); insourc­ing (e.g. GOV.UK) and design and engi­neer­ing (e.g. our own work on the trans­par­ent charg­ing sta­tion).

We feel there are two impor­tant lim­i­ta­tions with these exist­ing approach­es. The first is a focus on pro­fes­sion­als and the sec­ond is a focus on pre­dic­tion. We’ll dis­cuss each in turn.

Image cred­its: Cities Today

First of all, many solu­tions tar­get a pro­fes­sion­al class, be it accoun­tants, civ­il ser­vants, super­vi­so­ry boards, as well as tech­nol­o­gists, design­ers and so on. But we feel there is a role for the cit­i­zen as well, because the super­vi­sion of these sys­tems is sim­ply too impor­tant to be left to a priv­i­leged few. This role would include iden­ti­fy­ing wrong­do­ing, and sug­gest­ing alter­na­tives.

There is a ten­sion here, which is that from the per­spec­tive of the pub­lic sec­tor one should only ask cit­i­zens for their opin­ion when you have the inten­tion and the resources to actu­al­ly act on their sug­ges­tions. It can also be a chal­lenge to iden­ti­fy legit­i­mate con­cerns in the flood of feed­back that can some­times occur. From our point of view though, such con­cerns should not be used as an excuse to not engage the pub­lic. If cit­i­zen par­tic­i­pa­tion is con­sid­ered nec­es­sary, the focus should be on free­ing up resources and set­ting up struc­tures that make it fea­si­ble and effec­tive.

The sec­ond lim­i­ta­tion is pre­dic­tion. This is best illus­trat­ed with the Collinridge dilem­ma: in the ear­ly phas­es of new tech­nol­o­gy, when a tech­nol­o­gy and its social embed­ding are still mal­leable, there is uncer­tain­ty about the social effects of that tech­nol­o­gy. In lat­er phas­es, social effects may be clear but then often the tech­nol­o­gy has become so well entrenched in soci­ety that it is hard to over­come neg­a­tive social effects. (This sum­ma­ry is tak­en from an excel­lent van de Poel arti­cle on the ethics of exper­i­men­tal tech­nol­o­gy.)

Many solu­tions dis­re­gard the Collingridge dilem­ma and try to pre­dict and pre­vent adverse effects of new sys­tems at design-time. One exam­ple of this approach would be val­ue-sen­si­tive design. Our focus in stead is on use-time. Con­sid­er­ing the fact that smart pub­lic infra­struc­ture tends to be devel­oped on an ongo­ing basis, the ques­tion becomes how to make cit­i­zens a part­ner in this process. And even more specif­i­cal­ly we are inter­est­ed in how this can be made part of the design of the “touch­points” peo­ple actu­al­ly encounter in the streets, as well as their back­stage process­es.

Why do we focus on these phys­i­cal objects? Because this is where peo­ple actu­al­ly meet the infra­struc­tur­al sys­tems, of which large parts recede from view. These are the places where they become aware of their pres­ence. They are the prover­bial tip of the ice­berg.

Image cred­its: Sagar Dani

The use of auto­mat­ed deci­sion-mak­ing in infra­struc­ture reduces people’s agency. For this rea­son, resources for agency need to be designed back into these sys­tems. Fre­quent­ly the answer to this ques­tion is premised on a trans­paren­cy ide­al. This may be a pre­req­ui­site for agency, but it is not suf­fi­cient. Trans­paren­cy may help you become aware of what is going on, but it will not nec­es­sar­i­ly help you to act on that knowl­edge. This is why we pro­pose a shift from trans­paren­cy to con­testa­bil­i­ty. (We can high­ly rec­om­mend Anan­ny and Crawford’s arti­cle for more on why trans­paren­cy is insuf­fi­cient.)

To clar­i­fy what we mean by con­testa­bil­i­ty, con­sid­er the fol­low­ing three exam­ples: When you see the lights on your router blink in the mid­dle of the night when no-one in your house­hold is using the inter­net you can act on this knowl­edge by yank­ing out the device’s pow­er cord. You may nev­er use the emer­gency brake in a train but its pres­ence does give you a sense of con­trol. And final­ly, the cash reg­is­ter receipt pro­vides you with a view into both the pro­ce­dure and the out­come of the super­mar­ket check­out pro­ce­dure and it offers a resource with which you can dis­pute them if some­thing appears to be wrong.

Image cred­its: Aangifte­doen, source unknown for remain­der

None of these exam­ples is a per­fect illus­tra­tion of con­testa­bil­i­ty but they hint at some­thing more than trans­paren­cy, or per­haps even some­thing whol­ly sep­a­rate from it. We’ve been inves­ti­gat­ing what their equiv­a­lents would be in the con­text of smart pub­lic infra­struc­ture.

To illus­trate this point fur­ther let us come back to the smart EV charg­ing project we men­tioned ear­li­er. In Ams­ter­dam, pub­lic EV charg­ing sta­tions are becom­ing “smart” which in this case means they auto­mat­i­cal­ly adapt the speed of charg­ing to a num­ber of fac­tors. These include grid capac­i­ty, and the avail­abil­i­ty of solar ener­gy. Addi­tion­al fac­tors can be added in future, one of which under con­sid­er­a­tion is to give pri­or­i­ty to shared cars over pri­vate­ly owned cars. We are involved with an ongo­ing effort to con­sid­er how such charg­ing sta­tions can be redesigned so that peo­ple under­stand what’s going on behind the scenes and can act on this under­stand­ing. The moti­va­tion for this is that if not designed care­ful­ly, the opac­i­ty of smart EV charg­ing infra­struc­ture may be detri­men­tal to social accep­tance of the tech­nol­o­gy. (A first out­come of these efforts is the Trans­par­ent Charg­ing Sta­tion designed by The Incred­i­ble Machine. A fol­low-up project is ongo­ing.)

Image cred­its: The Incred­i­ble Machine, Kars Alfrink

We have iden­ti­fied a num­ber of dif­fer­ent ways in which peo­ple may object to smart EV charg­ing. They are list­ed in the table below. These types of objec­tions can lead us to fea­ture require­ments for mak­ing the sys­tem con­testable.

Because the list is pre­lim­i­nary, we asked the audi­ence if they could imag­ine addi­tion­al objec­tions, if those exam­ples rep­re­sent­ed new cat­e­gories, and if they would require addi­tion­al fea­tures for peo­ple to be able to act on them. One par­tic­u­lar­ly inter­est­ing sug­ges­tion that emerged was to give local com­mu­ni­ties con­trol over the poli­cies enact­ed by the charge points in their vicin­i­ty. That’s some­thing to fur­ther con­sid­er the impli­ca­tions of.

And that’s where we left it. So to sum­marise:

  1. Algo­rith­mic sys­tems are becom­ing part of pub­lic infra­struc­ture.
  2. Smart pub­lic infra­struc­ture rais­es new eth­i­cal con­cerns.
  3. Many solu­tions to eth­i­cal con­cerns are premised on a trans­paren­cy ide­al, but do not address the issue of dimin­ished agency.
  4. There are dif­fer­ent cat­e­gories of objec­tions peo­ple may have to an algo­rith­mic system’s work­ings.
  5. Mak­ing a sys­tem con­testable means cre­at­ing resources for peo­ple to object, open­ing up a space for the explo­ration of mean­ing­ful alter­na­tives to its cur­rent imple­men­ta­tion.

Starting a PhD

Today is the first offi­cial work day of my new doc­tor­al researcher posi­tion at Delft Uni­ver­si­ty of Tech­nol­o­gy. After more than two years of lay­ing the ground work, I’m start­ing out on a new chal­lenge.

I remem­ber sit­ting out­side a Jew­el cof­fee bar in Sin­ga­pore1 and going over the var­i­ous options for what­ev­er would be next after shut­ting down Hub­bub. I knew I want­ed to delve into the impact of machine learn­ing and data sci­ence on inter­ac­tion design. And large­ly through process of elim­i­na­tion I felt the best place for me to do so would be inside of acad­e­mia.

Back in the Nether­lands, with help from Ianus Keller, I start­ed mak­ing inroads at TU Delft, my first choice for this kind of work. I had vis­it­ed it on and off over the years, coach­ing stu­dents and doing guest lec­tures. I’d felt at home right away.

There were quite a few twists and turns along the way but now here we are. Start­ing this month I am a doc­tor­al can­di­date at Delft Uni­ver­si­ty of Technology’s fac­ul­ty of Indus­tri­al Design Engi­neer­ing.

My research is pro­vi­sion­al­ly titled ‘Intel­li­gi­bil­i­ty and Trans­paren­cy of Smart Pub­lic Infra­struc­tures: A Design Ori­ent­ed Approach’. Its main object of study is the MX3D smart bridge. My super­vi­sors are Gerd Kortuem and Neelke Doorn. And it’s all part of the NWO-fund­ed project ‘BRIdg­ing Data in the built Envi­ron­ment (BRIDE)’.

Below is a first rough abstract of the research. But in the months to come this is like­ly to change sub­stan­tial­ly as I start ham­mer­ing out a prop­er research plan. I plan to post the occa­sion­al update on my work here, so if you’re inter­est­ed your best bet is prob­a­bly to do the old RSS thing. There’s social media too, of course. And I might set up a newslet­ter at some point. We’ll see.

If any of this res­onates, do get in touch. I’d love to start a con­ver­sa­tion with as many peo­ple as pos­si­ble about this stuff.

Intel­li­gi­bil­i­ty and Trans­paren­cy of Smart Pub­lic Infra­struc­tures: A Design Ori­ent­ed Approach

This phd will explore how design­ers, tech­nol­o­gists, and cit­i­zens can uti­lize rapid urban man­u­fac­tur­ing and IoT tech­nolo­gies for design­ing urban space that express­es its intel­li­gence from the inter­sec­tion of peo­ple, places, activ­i­ties and tech­nol­o­gy, not mere­ly from the pres­ence of cut­ting-edge tech­nol­o­gy. The key ques­tion is how smart pub­lic infra­struc­ture, i.e. data-dri­ven and algo­rithm-rich pub­lic infra­struc­tures, can be under­stood by lay-peo­ple.

The design-ori­ent­ed research will uti­lize a ‘research through design’ approach to devel­op a dig­i­tal expe­ri­ence around the bridge and the sur­round­ing urban space. Dur­ing this extend­ed design and mak­ing process the phd stu­dent will con­duct empir­i­cal research to inves­ti­gate design choic­es and their impli­ca­tions on (1) new forms of par­tic­i­pa­to­ry data-informed design process­es, (2) the tech­nol­o­gy-medi­at­ed expe­ri­ence of urban space, (3) the emerg­ing rela­tion­ship between res­i­dents and “their” bridge, and (4) new forms of data-informed, cit­i­zen led gov­er­nance of pub­lic space.

  1. My Foursquare his­to­ry and 750 Words archive tell me this was on Sat­ur­day, Jan­u­ary 16, 2016. []