Artificial intelligence is increasingly being used by social service agencies to handle cases involving child abuse, intimate partner violence, and elder abuse. But a University of Iowa researcher and former social worker says the agencies need to establish safeguards to ensure that AI technology does not lead to bias against vulnerable or disadvantaged populations.
In a new study, Aislinn Conrad, associate professor in the School of Social Work, says AI systems can be useful, especially for tasks such as sifting through information to identify people considered at risk or to help agencies decide when intervention may be needed. But she warns that the technology could remove the human element that she says is critical to making informed, impartial decisions — especially when those decisions can lead to families being disrupted or marginalized.
In the study, Conrad recommends that agencies put survivors at the center of decisions when AI is being used. In particular, she recommends:
- Put people before automation: Families and survivors should have meaningful input into how AI systems are used. They need to understand why they've been flagged by an algorithm, and they should be able to contest decisions.
- Check AI for bias: Before agencies deploy an AI system, they need to audit it for fairness and then regularly review the system for unfair or discriminatory outcomes.
- Use AI technology to reduce surveillance: AI systems should be used to identify needed resources, such as housing, childcare, and services, rather than to increase electronic surveillance.
"AI can make us more efficient and effective, but only if we're willing to do the hard work of developing responsive systems that keep families safe without reproducing bias," Conrad says. "That's why I believe AI-safeguard frameworks are so critical right now."
When she was a social worker, Conrad performed child protective services investigations and helped children in foster care safely return to their families. Her territory covered a rural region in south-central Kansas where drug use and poverty were prevalent. Through this work she became interested in how social service agencies monitor people in the system.
"I saw the link between poverty and abuse over and over, and I began wondering, 'Is it really that poor kids are just more likely to be abused? Or is it a surveillance issue because they come to the attention of social services more often?'" Conrad says.
Her recommendations are timely for Iowa: In February, the state's Department of Health and Human Services partnered with Google to develop VISION, a new information system that uses AI to modernize how child welfare agencies manage cases and data.
Conrad says VISION or other AI-enhanced systems could lead to information being unintentionally misused. Data such as public assistance, housing instability, or previous contacts with social service agencies could be flagged as signs of "risk" in an algorithm. As a result, families already facing an economic or social hardship may face more scrutiny.
"When historical data shows that poor families and families of color have been disproportionately surveilled by child welfare systems, the algorithm learns that pattern and repeats it at scale, and does so invisibly," Conrad says. "A 'neutral' algorithm trained on biased data is just biased automation."
On the positive side, AI technology might be trained to identify communities where housing instability, caregiver stress, or inadequate services are severe. Agencies could then direct resources toward those needs.
"The goal shouldn't be more monitoring of vulnerable families," Conrad says. "It should be whether the technology genuinely improves access to services and outcomes for families."
The study, "Centering survivor agency in AI for violence prevention," was published online Oct. 9 in the Journal of the Society for Social Work and Research.
Conrad is the study's sole author. Her research was self-funded.
