Artificial intelligence can write text messages encouraging physical activity that most older adults consider appropriate and good quality, but their feelings about AI-and if they know AI wrote the message-impact their response, a new study in the Journals of Gerontology suggests.
The research is an important first step in helping health programs use AI to support large-scale behavior change, said lead author Allyson Tabaczynski, postdoctoral research fellow at the University of Michigan School of Kinesiology.
Tabaczynski and colleagues at U-M and Penn State University asked 630 adults aged 40 and older to read 80 AI-written text messages designed to motivate people to move more and sit less. Participants flagged any messages for cultural insensitivity and overall quality.
Key takeaways:
- The results were encouraging. Of nearly 50,000 ratings, only about 5% were flagged as culturally insensitive and roughly 6% had quality problems.
- Knowing the texts were written by AI and feeling more positive about AI was linked to people flagging more messages as culturally insensitive.
- Messages that emphasized sitting less (compared to moving more) or that described preparing for activity (compared to performing physical activity) received more low-quality ratings.
The most interesting finding was that even people who liked AI didn't let it off the hook-even when they knew beforehand that AI wrote the prompt, Tabaczynski said.
"Initially, I thought this was a little counterintuitive," she said. "If you have a more positive attitude toward AI, you might also just have more general knowledge of some of the biases or limitations that AI can have in its output or in its training data."
Half of the participants were told beforehand that the messages were AI-generated, and this group also rated more of the messages as possibly culturally insensitive when they had more positive attitudes toward AI.
When participants raised quality issues, the problem typically wasn't overt offensiveness but relevance. Some messages simply didn't fit a person's lifestyle or might not fit someone else's culture-for instance, a message suggesting dancing ("I don't dance") or advising people to stand for their morning coffee ("I don't drink coffee").
Those responses, Tabaczynski said, suggest AI messaging may be broadly acceptable while still needing better tailoring to individuals.
And that's not as easy as writing a message and pressing send. The team went through about 18 rounds of internal review, iterating on prompts and checking outputs to ensure the messages were evidence-based, varied and appropriate for the target audience.
AI could make this scalable, but the recipients still have to be willing to engage with the messages. The bottom line, Tabaczynski said, is that people's perception of AI matters.
"If someone is receiving a health intervention that uses AI, their perceptions of AI are going to impact how they're evaluating or responding to that intervention," she said. "So it's something that researchers and interventionists have to take into account as they're designing their interventions with this technology."
Study co-authors include: Yingjia Liu, Lizbeth Benson and David Conroy of U-M and Saeed Abdullah of Penn State.
The study was funded by the U-M Roybal Center, which is supported by the National Institute on Aging of the National Institutes of Health under Award Number P30AG086637. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
