Artificial intelligence comes from machines - and precisely for that reason, it isn't automatically fair. Doctoral researcher Lena Krieger works at Forschungszentrum Jülich on new methods to change that. Her goal: algorithms that treat all people equally, regardless of ethnicity, gender, or other sensitive attributes.
Ms. Krieger, you focus on fairness in artificial intelligence. Why is this an issue - isn't AI supposed to be neutral?
The main challenge with AI models is that they are highly dependent on the quality of the training data. Unfortunately, minorities are often underrepresented in these datasets, which leads to the amplification of biases in the model. A well-known example highlighting the urgency of fairness is the COMPAS study: a model designed to predict recidivism rates of alleged offenders to assist in sentencing decisions. Upon closer examination, it became clear that people of color were frequently assigned significantly higher risk scores than white individuals, regardless of their crimes. The model drew conclusions from variables that should not have had any influence. Fair AI approaches address exactly this problem - they counteract discrimination by, for example, preventing sensitive attributes like ethnicity or gender from influencing decisions.
You and your colleagues developed the algorithm FairDen. What exactly does it do, and what sets it apart from others?
In our project, we worked with density-based clustering. Clustering means automatically grouping data points based on their similarity. Unlike many traditional machine learning methods, this approach does not require labels - that is, pre-assigned categories. The structure emerges directly from the data. In density-based clustering, areas with many data points are recognized as clusters, separated by zones with fewer points.
Our algorithm - FairDen - takes this a step further by simultaneously considering fairness. It ensures that certain characteristics - so-called sensitive attributes, such as gender or origin - are distributed as equally as possible within the clusters. This creates clusters that both reflect the natural data structure and avoid biases and imbalances.
What kinds of applications is FairDen intended for?
FairDen can be used in a wide range of applications - anywhere data points need to be grouped. For example, it can help gain an overview of large datasets. The key requirement is that there is at least one sensitive attribute, such as gender. One example is the composition of school classes: it could ensure that the ethnic distribution is as balanced as possible while also grouping children from the same region in the same class.
What challenges remain, and what are the opportunities for improvement?
Of course, FairDen is only a first step toward fair AI. Many questions remain open. I am particularly interested in the intersection of explainability and fairness - so-called actionable recourse. This means not only explaining to users why a model produced a particular outcome but also showing them what actions they can take. For example, if someone applies for a loan and is denied, it's not helpful to say, 'It would be better if you were five years younger.' Instead, the explanation should be something like: 'If you reduce your existing debts, your chances will improve.' These actionable insights are important so that people understand how decisions are made - and what they themselves can do to influence them.
About the Researcher
Lena Krieger is a doctoral researcher at the Institute for Data Analytics and Machine Learning (IAS-8) at Forschungszentrum Jülich. She is supervised by Prof. Dr. Ira Assent (Forschungszentrum Jülich & Aarhus University, Denmark) and Prof. Dr. Thomas Seidl (LMU Munich). Her research focuses on the explainability and fairness of machine learning models.
