Why is trust so important when citizens encounter new technology?
Trust is something every organisation wants to inspire, but it plays a much more fundamental role in the welfare state—including economically. When citizens trust public authorities to make decisions about disability pensions, assistive equipment or medical treatment, for example, fewer resources are spent dealing with complaints, conflicts, fraud and interventions that fail. At the same time, it is important to recognise that trust and distrust are not evenly distributed across the public sector.
In Denmark, trust in institutions such as the healthcare system is relatively high. But in disability services and support for vulnerable children and families, many people are in conflict with the system. Here, digitalisation can add another layer to existing distrust.
How might AI contribute to marginalisation?
AI covers a range of technologies with different implications. Today, much of the focus is on tools based on large language models, such as chatbots and applications that write meeting minutes. Models with access to extensive databases of legislation and guidance are also being developed to make complex information easier to understand. At the same time, Denmark has a political goal of using AI to free up the equivalent of 30,000 full-time staff across the welfare state. This includes the hope that chatbots, for example, can replace staff answering telephone enquiries and reduce the time spent on documentation.
For many people, however, accessing public services digitally is already a challenge. There is a risk that it will become harder to reach a member of staff who can help them navigate legislation and guidance. Greater automation could therefore deepen inequalities in access to benefits and services.
The implications of more specialised predictive models based on machine learning are potentially even more serious. There are many international examples of bias, where models disadvantage people from particular socioeconomic backgrounds and vulnerable groups have systematically lost rights or benefits. In the Netherlands, for example, more than 20,000 people were wrongly identified as committing benefit fraud, with severe social consequences.
What does this mean for the legitimacy of the welfare state?
When predictive models are used to inform decisions by public authorities, it can become harder for people to understand how those decisions were reached and where errors may have occurred. But trust is about more than avoiding mistakes. People may distrust a decision even when it is correct, and declining trust undermines the legitimacy of the welfare state. This can lead to conflicts, complaints and media controversies that make AI tools less effective, particularly in services where close cooperation with citizens is essential.
Trust is also challenged when people experience data-driven administration as surveillance. When the Danish tax authorities began developing predictive models to detect fraud, concerns arose about whether they were monitoring what people sold on DBA (a Danish online marketplace). In a sense, the welfare state's collection of data about citizens is part of its duty of care, because it helps ensure fair decisions. But the boundary between care and surveillance is fragile. Conversely, there are also examples of AI helping to build trust—for instance, by allowing people to ask anonymously how changes to their preliminary tax assessment might affect them.
Predictive models generate individual assessments by statistically grouping people with others who share selected characteristics, in a way like how Netflix or Spotify recommends content. Even if a decision is justified on paper, it may not feel that way if people are not recognised as experts in their own lives, with first-hand knowledge of their circumstances. An encounter with a citizen will always reveal information beyond what an algorithm has been trained on. That is why people must be able both to challenge decisions by public authorities and to get an explanation of why a particular decision was reached.
Why do major AI experiments fail?
Some of the projects I have studied, including pioneering trials of predictive AI in casework by Danish municipalities, have failed partly because of a limited understanding of the legal requirements and whether the proposed uses were lawful, because there was no precedent to draw on. This is now changing with the EU's AI Act, the new government's work to establish a legal basis for using AI, and statements from the Danish Data Protection Agency. Fundamentally, though, there has been a strong focus on the potential benefits, without always giving citizens' rights the same attention.
Some projects have faced strong criticism from the media, citizens and professional organisations. That makes it important to involve stakeholders when defining the problems, rather than simply viewing citizens and staff as future users. Once a media storm takes hold, it is difficult to create space for mutual learning and a shared understanding of the problem.
What needs to change?
We need to recognise that any system we develop will have unintended consequences. So this is about more than developing good technology. It also requires leadership, ongoing organisational learning and adjustments as staff and citizens gain experience with the systems.
There is also a widespread tendency to greatly overestimate whether and how AI can improve efficiency. If efficiency becomes the overriding goal, it can overshadow good projects that could make a real difference to the quality of services. Therefore, the starting point should be people and their needs.
Citizens and staff should be involved in defining the problem, rather than simply being treated as users. In some areas, the underlying problems may have more to do with distrust, a lack of time and the complexity of the Danish welfare state and may call for solutions other than technology.
