AI Biases Shape Public View of History

As members of the public increasingly turn to AI chatbots to understand their world, even subtle latent biases in the underlying models could affect public understanding of the present—and past. Daniel Karell and colleagues explored the effects of both unintentional and intentional political biases in LLMs by asking 1,912 research participants to read GPT-4o and Wikipedia summaries of two 20th century historical events: the 1919 Seattle General Strike and the 1968 Third World Liberation Front student protests calling for greater ethnic minority representation in academia, which led the establishment of Ethnic Studies departments. Some AI summaries were explicitly generated with a liberal or conservative framing; others were produced with the model's default framing. After reading summaries, participants were asked to weigh in on issues related to the events, like the appropriateness of labor strikes and the use of curricula to advance social justice causes. Responses were graded on a five-point scale, where 1 is extremely conservative and 5 is extremely liberal. The authors found that the AI summaries with default framing and the AI summaries with liberal framing led to more liberal opinions compared to the Wikipedia summaries (Wikipedia text average 3.47, default LLM summary average 3.57, liberal LLM summary 3.67). Summaries with conservative framing led to slightly more conservative opinions compared to Wikipedia (conservative LLM summary average 3.36), although this effect was statistically significant only for those who already held conservative opinions. According to the authors, people seeking unbiased information from LLMs may be subtly influenced by the models' latent biases, and this may have consequences for society writ large.