Based on a talk delivered at the AI & Algorithm Seminar hosted by the Dutch Data Protection Authority on 25 March 2026 at ECC Leiden.
I am a postdoctoral researcher in Knowledge & Intelligence Design at TU Delft. I study how AI systems can be designed to be open and responsive to dispute, especially when they are used in public settings like cities. I call this program “contestable AI” (Alfrink et al., 2022). I have a background in interaction design, and my work sits at the intersection of design research, human-computer interaction, and philosophy of technology. The question driving my work: how can we use design to ensure people’s autonomy in an age of increasingly pervasive AI?
Human autonomy in the age of AI
Autonomy means having the effective capacity to govern your own life. It rests on two conditions: authenticity and agency (Prunkl, 2022). Authenticity means holding beliefs that are free from manipulative influences. Agency means being able to act on those beliefs with meaningful options available to you.
AI technologies can threaten both. They threaten authenticity through manipulation, adaptive preference formation, and deception. They undermine agency by restricting opportunities, limiting freedoms, diminishing our competence when we outsource tasks, and imposing paternalistic interventions against our choices.
Common approaches fall short
Ethical and rights-based approaches to AI are necessary but structurally limited (Alfrink, 2025a). They protect individual autonomy from above. They do not create the conditions for collective autonomy from below.
They focus on individuals, not collectives, but our capacity for self-governance also depends on institutions and social norms. They assume a universal definition of autonomy, but its meaning varies per context and is reshaped by the very technologies it is applied to, not least by the act of calling them “AI” (Suchman, 2023). They try to derive interventions from abstract norms, but if autonomy must be negotiated in situ, we need to involve people in the design process. And they can be dominated by majority norms, overlooking the plurality of autonomies that inclusive design should support.
Arenas, not guardrails
Contestability Loops for Public AI infographic.
These approaches give us guardrails, but not arenas. What we need are agonistic arenas for public AI (Alfrink et al., 2024): spaces where hidden engineering choices become visible and debatable, and where affected people can contest the trade-offs that shape their lives. Let me show you what that looks like.
Vision Model Macroscope
What follows is an example from a study in which we try to make machine learning engineering public. This work is part of the Human Values for Smarter Cities project.
The setting
Scan vehicle (photo: Robin Utrecht).
In Amsterdam and elsewhere, cities use scan vehicles equipped with computer vision to detect objects relevant to policy execution, such as misplaced trash. When the system detects trash, it reports the location to municipal workers who must verify the detection before dispatching cleanup crews. (Note: This is a hypothetical example not actually currently implemented in Amsterdam but previously explored in a pilot.)
This seemingly straightforward process involves real trade-offs. Set the detection threshold too low and workers waste time on false alarms. Set it too high and genuine problems go unaddressed. The confidence threshold is one of many engineering decisions that embed value judgments about whose time matters and which neighborhoods get attention. Engineers make these decisions, often without realizing their political dimensions (Kang, 2023).
What we built
An example macroscope. Here & There, BERG London, 2009.
We worked with design agency CLEVER°FRANKE in a nine-day design sprint to create a research prototype for focus group settings: sessions where participants gather around a tablet and collectively explore how machine learning trade-offs play out in practice.
We called it a “macroscope” because, following John Thackara (2005), it helps people see what the aggregation of many small actions looks like when added together, while maintaining a connection to the details.
How it works
Vision Model Macroscope interface. Design by CLEVER°FRANKE.
The core interaction is a confidence limit slider. Every detection by the computer vision model gets a confidence score between 0 and 1. The question is: from what threshold should the system treat a detection as true?
The macroscope then shows the consequences of that choice across multiple scales. Individual detections with scan car footage. A summary of how detections flow through the review process. Maps showing whether consequences are equally distributed across neighborhoods. The user can move between the micro and the macro to build intuition for the trade-offs.
What we found
Focus group session in progress.
We ran four focus groups with 19 municipal workers and analyzed the transcripts for recurring themes.
Participants adopted diverse perspectives: workload, fairness across neighborhoods, cost, city reputation, legal liability. Their arguments were sometimes contradictory, revealing genuine complexity. They went beyond the given trade-off, questioning the system design itself, rethinking how humans and the system should collaborate, and raising the temporal dimension of how these systems evolve.
The tool played a clear role in shaping these conversations. But it also posed challenges: it required significant familiarization time, and some elements caused confusion.
What this example reveals
My research is constructive. I make things to find things out. The prototype is a research instrument. The unit of analysis is practices, the doings that humans and machines co-perform together.
But the goal is not to design technology for a specific set of values. It is to expand what is subject to debate, and who gets to participate. The point is not to concentrate power in the designer who picks the “right” values. It is to create the conditions for local democratic deliberation about technology. This is a politics-first, not ethics-first, approach to design (van Maanen, 2022).
A value-sensitive design approach asks: what values should this system embody? I ask: which aspects of this technology are currently hidden from public view? And how can we make them visible and debatable?
Designing AI for autonomy
Taking autonomy seriously requires three moves: from individuals to collectives, from applications to infrastructure, and from abstract principles to actual power relations (Alfrink, 2025b).
First, design for groups, not just individuals. If autonomy is relational, it hinges on collective self-determination. We need to support genuine deliberation and build capacity for ongoing engagement, not one-off consultations.
Second, look beyond applications to infrastructure. The infrastructures AI depends on also shape what is possible. Data centers, compute resources, platform dependencies: these condition the autonomy of everyone who builds on, or is affected by, the systems they enable. Design must engage with these deeper layers.
Third, start from actual power relations, not abstract principles. A realist approach begins with an analysis of real historical institutions. It asks: who does what to whom, and for whose benefit? (Srnicek, 2022)
Takeaway
Designing AI for autonomy means treating it as a political project, not just a technical one.
We need to move from viewing AI as an autonomous entity to recognizing it as a socially situated system embedded within human practices and relations. Rather than imposing universal definitions of autonomy, we must engage in contextual negotiations of what autonomy means across different settings and communities.
This demands both technical design features that preserve human agency and inclusive design processes that acknowledge the plurality of autonomies.
References
Alfrink, K. (2025a). Designing Artificial Intelligence for Autonomy. In U. Pesch (Ed.), Design for Human Autonomy(pp. 18–19). Delft Design for Values Institute.
Alfrink, K. (2025b). People’s Compute: Design and the Politics of AI Infrastructures [Position paper]. OSF Preprints. https://doi.org/10.31219/osf.io/uaewn_v2
Alfrink, K., Keller, I., Kortuem, G., & Doorn, N. (2022). Contestable AI by Design: Towards a Framework. Minds and Machines, 33(4), 613–639. https://doi.org/10/gqnjcs
Alfrink, K., Keller, I., Yurrita Semperena, M., Bulygin, D., Kortuem, G., & Doorn, N. (2024). Envisioning Contestability Loops: Evaluating the Agonistic Arena as a Generative Metaphor for Public AI. She Ji: The Journal of Design, Economics, and Innovation, 10(1), 53–93. https://doi.org/10/gtzwft
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10/fswdcx
Kang, E. B. (2023). Ground truth tracings (GTT): On the epistemic limits of machine learning. Big Data & Society, 10(1), 1–12. https://doi.org/10/gtfgvx
Prunkl, C. (2022). Human autonomy in the age of artificial intelligence. Nature Machine Intelligence, 4(2), 99–101. https://doi.org/10/gsd2rt
Srnicek, N. (2022). Data, Compute, Labour. In M. Graham & F. Ferrari (Eds.), Digital Work in the Planetary Market(pp. 241–261). MIT Press.
Suchman, L. (2023). The uncontroversial ‘thingness’ of AI. Big Data & Society, 10(2), 1–5. https://doi.org/10/gs6q9w
Thackara, J. (2005). In The Bubble: Designing In A Complex World. MIT Press.
van Maanen, G. (2022). AI Ethics, Ethics Washing, and the Need to Politicize Data Ethics. Digital Society, 1(2), 9. https://doi.org/10/g833qb
