So-called "driverless" cars often have human operators remotely controlling the vehicles to help navigate tricky driving situations and avoid accidents. But this setup poses a number of challenges. How do you ensure the operators stay alert? And what happens when operators are asked to monitor multiple vehicles at once, particularly in dynamic environments? Researchers are proposing a new approach to this remote-operation system that addresses these concerns and could serve as a blueprint for the future of remotely operated vehicles.
"There are automobiles that are largely autonomous but that actually have remote human operators who provide guidance or assistance when a vehicle encounters a complex scenario," says Jing Feng, a professor of human factors psychology at NC State whose work focuses on human attention, driver behavior and transportation safety. "The goal here is to ensure that the vehicle navigates the situation successfully, mitigating any risk of harm to passengers, other vehicles, and so on. Recent news stories suggest that Waymo taxis are an example of this.
"However, these systems face three connected challenges: maintaining vigilance, ensuring operators are not tasked with overseeing more vehicles than they can handle, and ensuring timely intervention by operators in dynamic environments," Feng says. "And these challenges are not purely academic thought exercises. They are important considerations for an emerging industry, and the regulators who are tasked with overseeing that industry.
"We are proposing a new approach to these remote-operation systems that can help address those challenges," adds Feng, who is corresponding author of a paper that describes the proposed approach.
Defining Remote Operation Challenges
Maintaining vigilance is a challenge for remote operators because when things are going smoothly, there is little need for intervention and it becomes boring.
"This is important because if a person becomes disengaged, and is no longer alert, they are going to miss things that are happening which they need to respond to," says Feng.
The second challenge is about managing cognitive workload - you don't want the remote operator to become overwhelmed.
"For example, research shows that remote operators can monitor two vehicles very well, but their performance declines significantly when asked to monitor four vehicles - it's simply too mentally taxing," says Feng. "There are also issues related to which vehicles an operator is asked to monitor. For example, if a remote operator is asked to monitor two vehicles that are in close proximity to each other, this can increase the likelihood of an accident if both vehicles experience an unexpected event at the same time - such as a third vehicle behaving erratically."
And challenges associated with maintaining vigilance and managing cognitive workload can be exacerbated when vehicles are in dynamic environments - such as highways or city streets.
"If people are disengaged, or if people are overwhelmed by monitoring too many vehicles, it becomes more difficult to respond to unexpected situations in a timely way," says Feng.
A Proactive Approach to Remote Operation
Current remote-operation systems largely take a reactive approach, meaning human operators generally don't intervene in the operation of a vehicle until the autonomous system sends a prompt that alerts them to a problem.
"We are proposing a proactive approach to remote operation," says Feng. "This would give remote operators more system information, and would also require closer monitoring, but would allow operators to better anticipate problems. This approach should improve an operator's situational awareness leading to faster response times and less likelihood that the operator will make a bad decision.
"Our proposed approach draws on extensive research into maintaining alertness, situational awareness, the behavior of conventional drivers and the behavior of drivers who are operating semi-autonomous vehicles. The next step is to put our approach to the test, ideally using remote operators overseeing multiple vehicles in simulations or controlled environments.
"There are a host of research questions to explore here," says Feng. "How many vehicles can an operator monitor safely? How long can an operator's shift be before they lose focus? What information is necessary to maximize situational awareness and anticipation? It's an exciting research area.
"We're currently exploring some of these research questions with funding from the North Carolina Department of Transportation, to better understand policy and regulatory approaches related to the remote operation of robotaxis," Feng says.
The paper, "Proactive Remote Operation of Automated Vehicles: Supporting Human Controllability," is a chapter in the recently published Handbook of Human-Centered Artificial Intelligence. The paper was co-authored by Xiaolu Bai, a Ph.D. graduate from NC State; Yunan Liu, an associate professor in NC State's Edward P. Fitts Department of Industrial and Systems Engineering; and Chris Cunningham, associate director of NC State's Institute for Transportation Research and Education.
