Towards a realist AI design practice?

This is a ver­sion of the open­ing state­ment I con­tributed to the pan­el “Evolv­ing Per­spec­tives on AI and Design” at the Design & AI sym­po­sium that was part of Dutch Design Week 2024. I had the plea­sure of join­ing Iohan­na Nicen­boim and Jesse Ben­jamin on stage to explore what could be called the post-GenAI pos­si­bil­i­ty space for design. Thanks also to Math­ias Funk for moderating. 

The slide I displayed:

My state­ment:

1. There’s a lot of mag­i­cal think­ing in the AI field today. It assumes intel­li­gence is latent in the struc­ture of the inter­net. Metaphors like AGI and super­in­tel­li­gence are mag­i­cal in nature. AI prac­tice is also very secre­tive. It relies on demon­stra­tions. This leads to a lack of rig­or and polit­i­cal account­abil­i­ty (cf. Gilbert & Lam­bert in Ven­ture­Beat, 2023).
2. Design in its ide­al­ist mode is eas­i­ly fooled by such mag­ic. For exam­ple, in a recent report, the Dutch Court of Audit states that 35% of gov­ern­ment AI sys­tems are not known to meet expec­ta­tions (cf. Raji et al., 2022).
3. What is need­ed is design in a real­ist mode. Real­ism focus­es on who does what to whom in whose inter­est (cf. Geuss, 2008, 23 in von Busch & Palmås, 2023). Applied to AI the ques­tion becomes who gets to do AI to whom? This isn’t to say we should con­sid­er AI tech­nolo­gies com­plete­ly inert. It medi­ates our being in the world (Ver­beek, 2021). But we should also not con­sid­er it an inde­pen­dent force that’s just drag­ging us along.
4. The chal­lenge is to steer a path between, on the one hand, whole­sale cyn­i­cal rejec­tion and naive, opti­mistic, uncon­di­tion­al embrace, on the oth­er hand.
5. In my own work, what that looks like is to use design to make things that allow me to go into sit­u­a­tions where peo­ple are build­ing and using AI sys­tems. And to use those things as instru­ments to ask ques­tions relat­ed to human auton­o­my, social con­trol, and col­lec­tive free­dom in the face of AI.
6. The exam­ple shown is an ani­mat­ed short depict­ing a design fic­tion sce­nario involv­ing intel­li­gent cam­era cars used for pol­i­cy exe­cu­tion in urban pub­lic space. I used this video to talk to civ­il ser­vants about the chal­lenges fac­ing gov­ern­ments who want to ensure cit­i­zens remain in con­trol of the AI sys­tems they deploy (cf. Alfrink et al., 2023).
7. Why is this real­ist? Because the work looks at how some groups of peo­ple use par­tic­u­lar forms of actu­al­ly exist­ing AI to do things to oth­er peo­ple. The work also fore­grounds the com­pet­ing inter­ests that are at stake. And it frames AI as nei­ther ful­ly autonomous nor ful­ly pas­sive, but as a thing that medi­ates peo­ples’ per­cep­tions and actions.
8. There are more exam­ples besides this. But I will stop here. I just want to reit­er­ate that I think we need a real­ist approach to the design of AI.