At CSCW 2024, back in November of last year, we* ran a workshop titled “From Stem to Stern: Contestability Along AI Value Chains.” With it, we wanted to address a gap in contestable AI research. Current work focuses mainly on contesting specific AI decisions or outputs (for example, appealing a decision made by an automated content moderation system). But we should also look at contestability across the entire AI value chain—from raw material extraction to deployment and impact (think, for example, of data center activists opposing the construction of new hyperscales). We aimed to explore how different stakeholders can contest AI systems at various points in this chain, considering issues like labor conditions, environmental impact, and data collection practices often overlooked in contestability discussions.
The workshop mixed presentations with hands-on activities. In the morning, researchers shared their work through short talks, both in person and online. The afternoon focused on mapping out where and how people can contest AI systems, from data collection to deployment, followed by detailed discussions of the practical challenges involved. We had both in-person and online participants, requiring careful coordination between facilitators. We wrapped up by synthesizing key insights and outlining future research directions.
I was responsible for being a remote facilitator most of the day. But Mireia and I also prepared and ran the first group activity, in which we mapped a typical AI value chain. I figured I might as well share the canvas we used for that here. It’s not rocket science, but it held up pretty well, so maybe some other people will get some use out of it. The canvas was designed to offer a fair bit of scaffolding for thinking through what decision points there are along the chain that are potentially value-laden.
Here’s how the activity worked: We covered about 50 minutes doing a structured mapping exercise where participants identified potential contestation points along an AI value chain, using ChatGPT as an example case. The activity used a Miro board with a preliminary map showing different stages of AI development (infrastructure setup, data management, AI development, etc.). Participants first brainstormed individually for 10 minutes, adding value-laden decisions and noting stakeholders, harms, benefits, and values at stake. They then collaborated to reorganize and discuss the map for 15 minutes. The activity concluded with participants using dot voting (3 votes each) to identify the most impactful contestation sites, which were then clustered and named to feed into the next group activity.
The activity design drew from two main influences: typical value chain mapping methodologies (e.g., Mapping Actors along Value Chains, 2017), which usually emphasize tracking actors, flows, and contextual factors, and Wardley mapping (Wardley, 2022), which is characterized by the idea of a structured progression along an x‑axis with an additional dimension on the y‑axis.
The canvas design aimed to make AI system development more tangible by breaking it into clear phases (from infrastructure through governance) while considering visibility and materiality through the y‑axis. We ultimately chose to use a familiar system (ChatGPT). This, combined with the activity’s structured approach, helped participants identify concrete opportunities for intervention and contestation along the AI value chain, which we could build on during the rest of the workshop.
I got a lot out of this workshop. Some of the key takeaways that emerged out of the activities and discussions include:
There’s a disconnect between legal and technical communities, from basic terminology differences to varying conceptions of key concepts like explainability, highlighting the need for translation work between disciplines.
We need to move beyond individual grievance models to consider collective contestation and upstream interventions in the AI supply chain.
We also need to shift from reactive contestation to proactive design approaches that build in contestability from the start.
By virtue of being hybrid, we were lucky enough to have participants from across the globe. This helped drive home to me the importance of including Global South perspectives and considering contestability beyond Western legal frameworks. We desperately need a more inclusive and globally-minded approach to AI governance.
Many thanks to all the workshop co-organizers for having me as part of the team and to Agathe and Yulu, in particular, for leading the effort.
In my thesis, I use autonomy to build the normative case for contestability. It so happens that this year’s theme at the Delft Design for Values Institute is also autonomy. On October 15, 2024, I participated in a panel discussion on autonomy to kick things off. I collected some notes on autonomy that go beyond the conceptualization I used in my thesis. I thought it might be helpful and interesting to collect some of them here in adapted form.
The notes I brought included, first of all, a summary of the ecumenical conceptualization of autonomy concerning automated decision-making systems offered by Alan Rubel, Clinton Castro, and Adam Pham (2021). They conceive of autonomy as effective self-governance. To be autonomous, we need authentic beliefs about our circumstances and the agency to act on our plans. Regarding algorithmic systems, they offer this notion of a reasonable endorsement test—the degree to which a system can be said to respect autonomy depends on its reliability, the stakes of its outputs, the degree to which subjects can be held responsible for inputs, and the distribution of burdens across groups.
Second, I collected some notes from several pieces by James Muldoon, which get into notions of freedom and autonomy that were developed in socialist republican thought by the likes of Luxemburg, Kautsky, and Castoriadis (2020, 2021a, 2021b). This story of autonomy is sociopolitical rather than moral. This approach is quite appealing for someone interested in non-ideal theory in a realist mode like myself. The account of autonomy Muldoon offers is one where individual autonomy hinges on greater group autonomy and stronger bonds of association between those producing and consuming technologies. Freedom is conceived of as collective self-determination.
And then third and finally, there’s this connected idea of relational autonomy, which to a degree is part of the account offered by Rubel et al., but in the conceptions here more radical in how it seeks to create distance from liberal individualism (e.g., Christman, 2004; Mhlambi & Tiribelli, 2023; Westlund, 2009). In this, individual capacity for autonomous choice is shaped by social structures. So freedom becomes realized through networks of care, responsibility, and interdependence.
That’s what I am interested in: accounts of autonomy that are not premised on liberal individualism and that give us some alternative handle on the problem of the social control of technology in general and of AI in particular.
From my point of view, the implications of all this for design and AI include the following.
First, to make a fairly obvious but often overlooked point, the degree to which a given system impacts people’s autonomy depends on various factors. It makes little sense to make blanket statements about AI destroying our autonomy and so on.
Second, in value-sensitive design terms, you can think about autonomy as a value to be balanced against others—in the case where you take the position that all values can be considered equally important, at least in principle. Or you can consider autonomy more like a precondition for people to live with technology in concordance with their values, making autonomy take precedence over other values. The sociopolitical and relational accounts above point in this direction.
Third, suppose you buy into the radical democratic idea of technology and autonomy. In that case, it follows that it makes little sense to admonish individual designers about respecting others’ autonomy. They may be asked to privilege technologies in their designs that afford individual and group autonomy. But designers also need organization and emancipation more often than not. So it’s about building power. The power of workers inside the organizations that develop technologies and the power of communities that “consume” those same technologies.
With AI, the fact is that, in reality, in the cases I look at, the communities that AI is brought to bear on have little say in the matter. The buyers and deployers of AI could and should be made more accountable to the people subjected to AI.
This is a version of the opening statement I contributed to the panel “Evolving Perspectives on AI and Design” at the Design &AI symposium that was part of Dutch Design Week 2024. I had the pleasure of joining Iohanna Nicenboim and Jesse Benjamin on stage to explore what could be called the post-GenAI possibility space for design. Thanks also to Mathias Funk for moderating.
The slide I displayed:
My statement:
There’s a lot of magical thinking in the AI field today. It assumes intelligence is latent in the structure of the internet. Metaphors like AGI and superintelligence are magical in nature. AI practice is also very secretive. It relies on demonstrations. This leads to a lack of rigor and political accountability (cf. Gilbert & Lambert in VentureBeat, 2023).
Design in its idealist mode is easily fooled by such magic. For example, in a recent report, the Dutch Court of Audit states that 35% of government AI systems are not known to meet expectations (cf. Raji et al., 2022).
What is needed is design in a realist mode. Realism focuses on who does what to whom in whose interest (cf. Geuss, 2008, 23 in von Busch & Palmås, 2023). Applied to AI the question becomes who gets to do AI to whom? This isn’t to say we should consider AI technologies completely inert. It mediates our being in the world (Verbeek, 2021). But we should also not consider it an independent force that’s just dragging us along.
The challenge is to steer a path between, on the one hand, wholesale cynical rejection and naive, optimistic, unconditional embrace, on the other hand.
In my own work, what that looks like is to use design to make things that allow me to go into situations where people are building and using AI systems. And to use those things as instruments to ask questions related to human autonomy, social control, and collective freedom in the face of AI.
The example shown is an animated short depicting a design fiction scenario involving intelligent camera cars used for policy execution in urban public space. I used this video to talk to civil servants about the challenges facing governments who want to ensure citizens remain in control of the AI systems they deploy (cf. Alfrink et al., 2023).
Why is this realist? Because the work looks at how some groups of people use particular forms of actually existing AI to do things to other people. The work also foregrounds the competing interests that are at stake. And it frames AI as neither fully autonomous nor fully passive, but as a thing that mediates peoples’ perceptions and actions.
There are more examples besides this. But I will stop here. I just want to reiterate that I think we need a realist approach to the design of AI.
In the first half of this year, I’ve presented several versions of a brief talk on participatory AI. I figured I would post an amalgam of these to the blog for future reference. (Previously, on the blog, I posted a brief lit review on the same topic; this talk builds on that.)
So, to start, the main point of this talk is that many participatory approaches to AI don’t engage deeply with the specifics of the technology. One such specific is the translation work engineers do to make a problem “learnable” by a machine (Kang, 2023). From this perspective, the main question to ask becomes, how does translation happen in our specific projects? Should citizens be involved in this translation work? If so, how to achieve this?
Before we dig into the state of participatory AI, let’s begin by clarifying why we might want to enable participation in the first place. A common motivation is a lack of democratic control over AI systems. (This is particularly concerning when AI systems are used for government policy execution. These are the systems I mostly look at in my own research.) And so the response is to bring the people into the development process, and to let them co-decide matters.
In these cases, participation can be understood as an enabler of democratic agency, i.e., a way for subjects to legitimate the use of AI systems (cf. Peter, 2020 in Rubel et al., 2021). Peter distinguishes two pathways: a normative one and a democratic one. Participation can be seen as an example of the democratic pathway to legitimation. A crucial detail Peter mentions here, which is often overlooked in participatory AI literature, is that normative constraints must limit the democratic pathway to avoid arbitrariness.
So, what is the state of participatory AI research and practice? I will look at each in turn next.
As mentioned, I previously posted on the state of participatory AI research, so I won’t repeat that in full here. (For the record, I reviewed Birhane et al. (2022), Bratteteig & Verne (2018), Delgado et al. (2023), Ehsan & Riedl (2020), Feffer et al. (2023), Gerdes (2022), Groves et al. (2023), Robertson et al. (2023), Sloane et al. (2020), and Zytko et al. (2022).) Elements that jump out include:
Superficial and unrepresentative involvement.
Piecemeal approaches that have minimal impact on decision-making.
Participants with a consultative role rather than that of active decision-makers.
A lack of bridge-builders between stakeholder perspectives.
Participation washing and exploitative community involvement.
Struggles with the dynamic nature of technology over time.
Discrepancies between the time scales for users to evaluate design ideas versus the pace at which systems are developed.
A demand for participation to enhance community knowledge and to actually empower them.
Taking a step back, if I were to evaluate the state of the scientific literature on participatory AI, it strikes me that many of these issues are not new to AI. They have been present in participatory design more broadly for some time already. Many of these issues are also not necessarily specific to AI. The ones I would call out include the issues related to AI system dynamism, time scales of participation versus development, and knowledge gaps between various actors in participatory processes (and, relatedly, the lack of bridge-builders).
Framework for Meaningful Stakeholder Involvement is aimed at businesses, organizations, and institutions that use AI. It focuses on human rights, ethical assessment, and compliance. It aims to be a tool for planning, delivering, and evaluating stakeholder engagement effectively, emphasizing three core elements: Shared Purpose, Trustworthy Process, and Visible Impact.
Democratizing AI frames public participation in AI development as a way to add legitimacy and accountability and to help prevent harmful impacts. It outlines risks associated with AI, including biased outcomes, opaque decision-making processes, and designers lacking real-world impact awareness. Causes for ineffective participation include unidirectional communication, socioeconomic barriers, superficial engagement, and ineffective third-party involvement. The report uses environmental law as a reference point and offers eight guidelines for meaningful public participation in AI.
Taking stock of these reports, we can say that the building blocks for the overall process are available to those seriously looking. The challenges facing participatory AI are, on the one hand, economic and political. On the other hand, they are related to the specifics of the technology at hand. For the remainder of this piece, let’s dig into the latter a bit more.
Let’s focus on translation work done by engineers during model development.
For this, I build on work by Kang (2023), which focuses on the qualitative analysis of how phenomena are translated into ML-compatible forms, paying specific attention to the ontological translations that occur in making a problem learnable. Translation in ML means transforming complex qualitative phenomena into quantifiable and computable forms. Multifaceted problems are converted into a “usable quantitative reference” or “ground truth.” This translation is not a mere representation of reality but a reformulation of a problem into mathematical terms, making it understandable and processable by ML algorithms. This transformation involves a significant amount of “ontological dissonance,” as it mediates and often simplifies the complexity of real-world phenomena into a taxonomy or set of classes for ML prediction. The process of translating is based on assumptions and standards that may alter the nature of the ML task and introduce new social and technical problems.
So what? I propose we can use the notion of translation as a frame for ML engineering. Understanding ML model engineering as translation is a potentially useful way to analyze what happens at each step of the process: What gets selected for translation, how the translation is performed, and what the resulting translation consists of.
So, if we seek to make participatory AI engage more with the technical particularities of ML, we could begin by identifying translations that have happened or might happen in our projects. We could then ask to what extent these acts of translation are value-laden. For those that are, we could think about how to communicate these translations to a lay audience. A particular challenge I expect we will be faced with is what the meaningful level of abstraction for citizen participation duringAI development is. We should also ask what the appropriate ‘vehicle’ for citizen participation will be. And we should seek to move beyond small-scale, one-off, often unrepresentative forms of direct participation.
Bibliography
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022). Power to the People? Opportunities and Challenges for Participatory AI. Equity and Access in Algorithms, Mechanisms, and Optimization, 1–8. https://doi.org/10/grnj99
Bratteteig, T., & Verne, G. (2018). Does AI make PD obsolete?: Exploring challenges from artificial intelligence to participatory design. Proceedings of the 15th Participatory Design Conference: Short Papers, Situated Actions, Workshops and Tutorial — Volume 2, 1–5. https://doi.org/10/ghsn84
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023). The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 1–23. https://doi.org/10/gs8kvm
Ehsan, U., & Riedl, M. O. (2020). Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach. In C. Stephanidis, M. Kurosu, H. Degen, & L. Reinerman-Jones (Eds.), HCI International 2020—Late Breaking Papers: Multimodality and Intelligence (pp. 449–466). Springer International Publishing. https://doi.org/10/gskmgf
Feffer, M., Skirpan, M., Lipton, Z., & Heidari, H. (2023). From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 38–48. https://doi.org/10/gs8kvx
Gerdes, A. (2022). A participatory data-centric approach to AI Ethics by Design. Applied Artificial Intelligence, 36(1), 2009222. https://doi.org/10/gs8kt4
Groves, L., Peppin, A., Strait, A., & Brennan, J. (2023). Going public: The role of public participation approaches in commercial AI labs. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 1162–1173. https://doi.org/10/gs8kvs
Kang, E. B. (2023). Ground truth tracings (GTT): On the epistemic limits of machine learning. Big Data & Society, 10(1), 1–12. https://doi.org/10/gtfgvx
Peter, F. (2020). The Grounds of Political Legitimacy. Journal of the American Philosophical Association, 6(3), 372–390. https://doi.org/10/grqfhn
Robertson, S., Nguyen, T., Hu, C., Albiston, C., Nikzad, A., & Salehi, N. (2023). Expressiveness, Cost, and Collectivism: How the Design of Preference Languages Shapes Participation in Algorithmic Decision-Making. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10/gr6q2t
Rubel, A., Castro, C., & Pham, A. K. (2021). Algorithms and autonomy: The ethics of automated decision systems. Cambridge University Press.
Sloane, M., Moss, E., Awomolo, O., & Forlano, L. (2020). Participation is not a Design Fix for Machine Learning. arXiv:2007.02423 [Cs]. http://arxiv.org/abs/2007.02423
Zytko, D., J. Wisniewski, P., Guha, S., P. S. Baumer, E., & Lee, M. K. (2022). Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains. Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, 1–4. https://doi.org/10/gs8kv6
I am writing this final PhD update as a freshly minted doctor. On Thursday, May 23, 2024, I successfully defended my thesis, ‘Contestable Artificial Intelligence: Constructive Design Research for Public Artificial Intelligence Systems that are Open and Responsive to Dispute.’
I started the PhD on September 1, 2018 (read the very first update posted on that day here). So, that’s five years, eight months, 23 days from start to finish. It has been quite the journey, and I feel happy and relieved to have completed it. I am proud of the work embodied in the thesis. Most of all, I am thankful for the transformative learning experience, none of which would have been possible without the support of my supervisors Gerd, Ianus, and Neelke.
On the day itself, I was honored to have as my external committee members professors Dignum, Löwgren, van Zoonen, and van de Poel, professor Voûte as the chair, and Joost and Mireia as my paranymphs.
Me, with a copy of the thesis, shortly before starting the layperson’s talk. Photo: Roy Borghouts.
Recent events
Reviewing my notes since the last update, below are some more notable things that happened in the past eight months.
I ran a short workshop on AI Pedagogy Through A Design Lens, together with Hosana Morales, at the TU Delft spring symposium on AI education. Read the post.
A story about my research was published on the TU Delft industrial design engineering website in the run-up to my defense on May 14, 2024. Read the story.
I updated and ran the fifth and final iteration of the AI& Society industrial design engineering master elective course from February 28 through April 10, 2024. A previous version is documented here, which I plan to update sometime in the near future.
I gave a talk titled Contestable AI: Designing for Human Autonomy at the Amsterdam UX meetup on February 21, 2024. Download the slides.
The outcomes of a design sprint on tools for third-party scrutiny, organized by the Responsible Sensing Lab, which took inspiration from my research, were published on December 7, 2023. Read the report.
I was interviewed by Mireia Yurrita Semperena for a DCODE podcast episode titled Beyond Values in Algorithmic Design, published November 6, 2023. Listen to the episode.
Together with Claudio Sarra and Marco Almada, I hosted an online seminar titled Building Contestable Systems on October 26, 2023. Read the thread.
A paper I co-authored titled When ‘Doing Ethics’ Meets Public Procurement of Smart City Technology – an Amsterdam Case Study, was presented by first author Mike de Kreek at IASDR 2023 on October 9–13. Read the paper.
Looking ahead
I will continue at TU Delft as a postdoctoral researcher and will stay focused on design, AI, and politics, but I will try to evolve my research into something that builds on my thesis work but adds a new angle.
The Envisioning Contestability Loops article mentioned in previous updates is now in press with She Ji, which I am very pleased about. It should be published “soon.”
I’ve been thinking alot about civic participation in machine learning systems development. In particular, involving non-experts in the potentially value-laden translation work from specifications that engineers do when they build their models. Below is a summary of a selection of literature I found on the topic, which may serve as a jumping-off point for future research.
Abstract
The literature on participatory artificial intelligence (AI) reveals a complex landscape marked by challenges and evolving methodologies. Feffer et al. (2023) critique the reduction of participation to computational mechanisms that only approximate narrow moral values. They also note that engagements with stakeholders are often superficial and unrepresentative. Groves et al. (2023) identify significant barriers in commercial AI labs, including high costs, fragmented approaches, exploitation concerns, lack of transparency, and contextual complexities. These barriers lead to a piecemeal approach to participation with minimal impact on decision-making in AI labs. Delgado et al. (2023) observe that participatory AI involves stakeholders mostly in a consultative role without integrating them as active decision-makers throughout the AI design lifecycle.
Gerdes (2022) proposes a data-centric approach to AI ethics and underscores the need for interdisciplinary bridge builders to reconcile different stakeholder perspectives. Robertson et al. (2023) explore participatory algorithm design, emphasizing the need for preference languages that balance expressiveness, cost, and collectivism—Sloane et al. (2020) caution against “participation washing” and the potential for exploitative community involvement. Bratteteig & Verne (2018) highlight AI’s challenges to traditional participatory design (PD) methods, including unpredictable technological changes and a lack of user-oriented evaluation. Birhane et al. (2022) call for a clearer understanding of meaningful participation, advocating for a shift towards vibrant, continuous engagement that enhances community knowledge and empowerment. The literature suggests a pressing need for more effective, inclusive, and empowering participatory approaches in AI development.
Bibliography
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022). Power to the People? Opportunities and Challenges for Participatory AI. Equity and Access in Algorithms, Mechanisms, and Optimization, 1–8. https://doi.org/10/grnj99
Bratteteig, T., & Verne, G. (2018). Does AI make PD obsolete?: Exploring challenges from artificial intelligence to participatory design. Proceedings of the 15th Participatory Design Conference: Short Papers, Situated Actions, Workshops and Tutorial — Volume 2, 1–5. https://doi.org/10/ghsn84
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023). The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, 1–23. https://doi.org/10/gs8kvm
Ehsan, U., & Riedl, M. O. (2020). Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach. In C. Stephanidis, M. Kurosu, H. Degen, & L. Reinerman-Jones (Eds.), HCI International 2020—Late Breaking Papers: Multimodality and Intelligence (pp. 449–466). Springer International Publishing. https://doi.org/10/gskmgf
Feffer, M., Skirpan, M., Lipton, Z., & Heidari, H. (2023). From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 38–48. https://doi.org/10/gs8kvx
Gerdes, A. (2022). A participatory data-centric approach to AI Ethics by Design. Applied Artificial Intelligence, 36(1), 2009222. https://doi.org/10/gs8kt4
Groves, L., Peppin, A., Strait, A., & Brennan, J. (2023). Going public: The role of public participation approaches in commercial AI labs. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 1162–1173. https://doi.org/10/gs8kvs
Robertson, S., Nguyen, T., Hu, C., Albiston, C., Nikzad, A., & Salehi, N. (2023). Expressiveness, Cost, and Collectivism: How the Design of Preference Languages Shapes Participation in Algorithmic Decision-Making. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10/gr6q2t
Sloane, M., Moss, E., Awomolo, O., & Forlano, L. (2020). Participation is not a Design Fix for Machine Learning. arXiv:2007.02423 [Cs]. http://arxiv.org/abs/2007.02423
Zytko, D., J. Wisniewski, P., Guha, S., P. S. Baumer, E., & Lee, M. K. (2022). Participatory Design of AI Systems: Opportunities and Challenges Across Diverse Users, Relationships, and Application Domains. Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, 1–4. https://doi.org/10/gs8kv6
I’m back again with another Ph.D. update. Five years after I started in Delft, we are nearing the finish line on this whole thing. But before we look ahead, let’s review notable events since the previous update in March 2023.
Occurrences
I presented our framework, Contestable AI by Design, at the annual NWOICT Open conference, which, for the first time, had an entire track dedicated to HCI research in the Netherlands. It was an excellent opportunity to meet fellow researchers from other Dutch institutions. The slides are available as PDF at contestable.ai.
I visited Hamburg to present our paper, Contestable Camera Cars, at CHI 2023. We also received a Best Paper award, which I am, of course, very pleased with. The conference was equal parts inspiring and overwhelming. The best part of it was meeting in-person researchers who shared my interests.
Also, at CHI, I was interviewed about my research by Mike Green for his podcast Understanding Users. You can listen to it here. It is always good practice to try and lay out some of my arguments spontaneously live.
In June, I joined a panel at a BOLD Cities “talk show” to discuss the design of smart city systems for contestability. It was quite an honor to be on the same panel as Eefje Cuppen, director of the Rathenau Institute. This event was great because we had technological, design, political, and policy perspectives. Several guests argued for the need to reinvigorate representative democracy and give a more prominent role to elected politicians in setting technology policy. A report is available here.
In August, the BRIDE project had its closing event. This is the NWO research project that partially funded my Ph.D. The event was an excellent opportunity to reflect on our work together over the past years. I took the opportunity to revisit the work of Saskia Sassen on cityness and to think through some of the implications of my work on contestability for the field of smart urbanism. The slides are available at contestable.ai.
Finally, last week, a short opinion piece that lays out the argument for contestable AI in what I hope is a reasonably accessible manner, was published on the TU Delft website.
Eefje Cuppen and I being interviewed by Inge Janse at the BOLD Cities talk show on June 22, 2023—photo by Tiffany Konings.
Envisioning Contestability Loops
Throughout this, I have been diligently chipping away at my final publication, “Envisioning Contestability Loops: Evaluating the Agonistic Arena as a Generative Metaphor for Public AI.” I had a great time collaborating with Leon de Korte on an infographic of part of my design framework.
We took this infographic on a tour of Dutch interaction design agencies and conducted concept design workshops. I enjoyed returning to practice and sharing the work of the past couple of years with peers in practice. My friends at Eend wrote a nice blog post about it.
The analysis of the outcomes of these workshops forms the basis for the article, in which I explore the degree to which the guiding concept (generative metaphor) behind contestable AI, which I have dubbed the “Agonistic Arena” is a productive one for design practitioners. Spoilers: It is, but competing metaphors are also at play in the public AI design space.
The manuscript is close to completion. As usual, putting something like this together is a heavy but gratifying lift. I look forward to sharing the results and the underlying infographic with the broader world.
Are we there yet?
Looking ahead, I will be on a panel alongside the great Julian Bleecker and a host of others at the annual TU Delft Design &AI symposium in October.
But aside from that, I will keep my head down and focus on completing my thesis. The aim is to hand it in by the end of November. So, two more months on the clock. Will I make it? Let’s find out!
Hello again, and welcome to another update on my Ph.D. research progress. I will briefly run down the things that happened since the last update, what I am currently working on, and some notable events on the horizon.
Recent happenings
CHI 2023 paper
Stills from Contestable Camera Cars concept video.
First off, the big news is that the paper I submitted to CHI 2023 was accepted. This is a big deal for me because HCI is the core field I aim to contribute to, and CHI is its flagship conference.
Here’s the full citation:
Alfrink, K., Keller, I., Doorn, N., & Kortuem, G. (2023). Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute. https://doi.org/10/jwrx
I have had several papers rejected in the past (CHI is notoriously hard to get accepted at), so I feel vindicated. The paper is already available as an arXiv preprint, as is the concept video that forms the core of the study I report on (many thanks to my pal Simon for collaborating on this with me). CHI 2023 happens in late April. I will be riding a train over there to present the paper in person. Very much looking forward to that.
Contestable Camera Cars concept video.
Responsible Sensing Lab anniversary event
I briefly presented my research at the Responsible Sensing Lab anniversary event on February 16. The whole event was quite enjoyable, and I got some encouraging responses to my ideas afterward which is always nice. The event was recorded in full. My appearance starts around the 1:47:00 mark.
It me. (Credit: Responsible Sensing Lab.)
Video of my contribution. (Pakhuis de Zwijger / Responsible Sensing Lab.)
Tweeting, tooting, blogging
I have been getting back into the habit of tweeting, tooting, and even the occasional spot of blogging on this website again. As the end of my Ph.D. nears, I figured it might be worth it to engage more actively with “the discourse,” as they say. I mostly share stuff I read that is related to my research and that I find interesting. Although, of course, posts related to my twin sons’ music taste and struggles with university bureaucracy always win out in the end. (Yes, I am aware my timing is terrible, seeing as how we have basically finally concluded social media was a bad idea after all.)
Current activities
Envisioning Contestability Loops
At the moment, the majority of my time is taken up by conducting a final study (working title: “Envisioning Contestability Loops”). I am excited about this one because I get to once again collaborate with a professional designer on an artifact, in this case, a visual explanation of my framework, and use the result as a research instrument to dig into, in this case, the strengths and weaknesses of contestability as a generative metaphor for the design of public AI.
Thesis
In parallel, I have begun to put together my thesis. It is paper-based, but of course, the introductory and concluding chapters require some thought still.
The aim is to have both the final article and thesis finished by the end of summer and then begin the arduous process of getting a date for my defense, assembling a committee, etc.
Agonistic Machine Vision Development
In the meantime, I am also mentoring Laura, another brilliant master graduation student. Her project, titled “Agonistic Machine Vision Development,” builds on my previous research. In particular, one of the challenges I identified in Contestable Camera Cars, that of the differential in information position between citizens and experts when they collaborate in participatory machine learning sessions. It’s very gratifying to see others do design work that pushes these ideas further.
Upcoming events
So yeah, like I already mentioned, I will be speaking at CHI 2023, which takes place on 23–28 April in Hamburg. The schedule says I am presenting on April 25 as part of the session on “AI Trust, Transparency and Fairness”, which includes some excellent-looking contributions.
And before that, I will be at ICT.OPEN in Utrecht on April 20 to present briefly on the Contestable AI by Design framework as part of the CHINL track. It should be fun.
…
That’s it for this update. Maybe, by the time the next one rolls around, I will be able to share a date for my defense. But let’s not jinx it.
Tensions in the professional field of design. (PDF)
Note that these tensions are independent of each other. The diagram does not imply two “sides” of design. At any given moment, a design activity can be plotted on each axis independently. This is also not an exhaustive list of tensions. Finally, Dorst claims these tensions are irreconcilable.
The original passage:
Contemporary developments in design can be described and understood in much the same way. The professional field that we so easily label ‘design’ is complex, and full of inner contradictions. These inner tensions feed the discussions in the field. To name a few: (1) the objectives of design and the motivation of designers can range from commercial success to the common good. (2) The role and position of the designer can be as an autonomous creator, or as a problem solver in-service to the client. (3) The drive of the designer can be idealistic, or it can be more pragmatic (4) The resulting design can be a ‘thing’, but also immaterial (5) The basis for the process of designing can be intuitive, or based on knowledge and research… Etcetera… The development of the design disciplines can be traced along these lines of tension — with designers in different environments and times changing position relative to these fundamental paradoxes, but never resolving them. Ultimately, the real strength and coherence of design as a field of professions comes from recognizing these contradictions, and the dynamics of the field is a result of continuous experimentation along the rifts defined by them. Rather than a common set of practices and skills that designers might have [Cross, 1990] it is these inner contradictions in design that define its culture, its mentality. Design research should be an active force in these discussions, building bridges between them where possible. Not to resolve them into a monolithic Science of Design, but advancing the discussion in this dynamically shifting set of relations.
Dorst, K. (2016, June 27). Design practice and design research: Finally together? Proceedings of DRS 2016. Design Research Society 50th Anniversary Conference, Brighton, UK. https://www.drs2016.org/212
Below are some choice quotes on “citizen participation” from chapter 8 of The End of the End of History, a recommended book on our recent global political history. I feel like many of us in the participatory technology design space are complicit in these practices to some extent. I continue to grapple with alternative models of mass democratic control over technology.
The Center-Left will propose a range of measures designed to promote “civic engagement” or “community participation.”
Citizens’ summits, juries and panels all aim at participation rather than power, at the technocratic incorporation of the people into politics in order to manage away conflict.
Likewise the popularity of deliberative modes of engagement, deliberative stakeholder events or workshops are characteristic tools of technocratic do-gooders as they create the simulacrum of a democratic process in which people are assembled to provide an ostensibly collective solution to a problem, but decisions lack a binding quality or have already been taken in advance.
Though unable to gain traction at a transnational level, the Left may find some success in municipal politics, following the 2010s example of Barcelona.
Sidestepping […] animus toward Big Tech companies, [tech solutionism (Morozov, 2013) and the ideology of ease (Greenfield, 2017)] may come to be applied to non-market activities, such as solving community problems, perhaps at the level of municipal government.
Sovereign, national politics – which neoliberalism was designed to defang – will remain beyond the grasp of the Left. Progressives will prefer instead to operate at the municipal, the everyday or the supranational level – precisely the arena to which neoliberalism sought to displace politics, to where it could do no harm.
