Osaka, Japan - In recent years, strong criticism of the mass media has become widespread online in Japan. Against this backdrop, researchers at the University of Osaka examined whether direct survey questions may make negative attitudes toward the news media appear stronger than they actually are. In two web-based randomized experiments, agreement with the statement "the mass media are harmful to society" was higher when respondents were asked directly than when the same view was measured indirectly using a list experiment.
Across the two studies combined, 45.1% of respondents agreed with the statement in the direct-question condition. By contrast, the list experiment yielded an estimated agreement rate of 29.7%. The gap of 15.4 percentage points was statistically significant, and the same directional pattern appeared in both studies.
The findings suggest that, in social settings where criticizing the media is easy to express, direct questions may capture not only private opinion but also socially shaped response tendencies. In other words, negative attitudes toward the media may sometimes be overstated rather than hidden.
To test this, Professor Asako Miura of the Graduate School of Human Sciences at the University of Osaka conducted two online survey experiments in Japan. Some respondents were asked directly whether they thought the mass media were harmful to society, while others answered a list experiment designed to reduce pressure to state that view explicitly.
The study does not suggest that criticism of the media is insincere. Rather, it shows that the way a question is asked can affect how strongly such criticism appears in survey results. The paper also notes that both studies used nonprobability online samples, so the findings should not be treated as a direct estimate of opinion in Japan as a whole.
Professor Miura commented: "Seeing consistently high levels of media distrust in questionnaire surveys, I wondered whether the way we asked the question might be shaping the results."
