As digital tools become more common in teamwork, many organizations hope they can help teams collaborate more effectively. But a new study suggests that some uses of tools— such as providing real-time feedback on collaboration via specific metrics — do not necessarily improve how teams work and in some cases may even make performance worse. The study, " Exploring the Effects of Real-Time Feedback on Collaborative Processes to Enhance Collective Intelligence in Teams ," by researchers at Carnegie Mellon University and the University of Dayton, appears in Collective Intelligence. The researchers tested whether showing teams real-time indicators of collaboration quality would help them work together more effectively on a dynamic online search-and-rescue task.
"Real-time feedback can seem like an obvious way to improve teamwork, but our findings show that teams do not simply use these signals in the way designers intend," says Anita Williams Woolley, Professor of Organizational Behavior at Carnegie Mellon University's Tepper School of Business and a coauthor of the study. "When a metric becomes the focus of attention, teams may focus too much on one behavior and overlook what matters most for effective coordination."
Collaboration has become the engine of innovation in many areas of society. In the global workplace, more than 65 percent of work is organized around teams, and in the United States, more than 84 percent of workers collaborate with colleagues. These arrangements have been driven by technology and workforce changes brought by the COVID-19 pandemic. With these changes comes an urgent need to understand what contributes to the quality of these processes.
In two experiments, teams of four worked on an online search-and-rescue task. In one study, some teams received real-time feedback on three collaboration metrics at once: members' level of effort, effective use of member skills, and coordination strategy. In a second study, teams received feedback only on members' overall effort, the metric most strongly associated with performance in prior work. Researchers compared those teams with control groups that received no real-time feedback on collaboration.
The results were striking. Showing teams live feedback on all three metrics changed behavior but did not improve overall performance. Narrowing the feedback to a single metric, effort, made things worse: teams focused more heavily on that visible signal, but their overall performance declined as attention to coordination strategy suffered.
"Our findings show that metrics that are useful for diagnosing teamwork are not automatically useful for improving it in the moment," says first author Chase McDonald. "Better interventions may need to support reflection and adaptation, not just immediate reaction."
The findings suggest that organizations should be cautious about using real-time dashboards or digital nudges as simple fixes for collaboration problems. Rather than continuous feedback during task execution, the researchers suggest that teams may benefit more from well-timed, reflective interventions that help them interpret what happened and adjust more thoughtfully.
The study was funded by the Defense Advanced Research Projects Agency and by the National Science Foundation (NSF) and NSF's AI Institute for Societal Decision Making.
