Road To Self-driving Future

Cameras, satnav, laser, radar and sensors. To earn the prefix 'self-driving', or 'autonomous', vehicles need a range of technologies capable of both sensing and evaluating the complex and unpredictable traffic environment.

Self-driving capability is graded on a five-point scale set by the Society of Automotive Engineers (see box). Many new cars are at least at the lower end of the scale, with features such as cruise control, ultrasonic sensors for close distance assessment and built-in satnav . The higher up the scale, the higher demands are on decision-making and 'judgement'. This means that AI and machine learning are a prerequisite for reaching the higher levels because only then can the car evaluate and act safely.

No rest breaks
Of the vehicles currently in use, the robot taxis in some major US and Chinese cities are the closest to being fully autonomous, i.e. level five. Otherwise, most are still being tested in controlled environments. Other commercial vehicles, such as buses and lorries, could be next to be released on a large scale, according to Viktor Larsson, a mathematics researcher who develops computer vision algorithms.

"Such vehicles operate regular routes and are expensive, which makes it worth fitting them with more sensors. And they can run around the clock - an automated driver doesn't need a rest break... They don't necessarily have to work all the time either. If a problem were to arise, an operator can jump in and control the vehicle remotely," he says.

Cameras not only cheaper
Car manufacturers prioritise different technologies, for example Tesla only uses cameras while its competitor Waymo also uses radar and lidar, i.e. laser-based distance measurement. All, however, use one or more cameras.

"Lidar is admittedly much better at measuring distance than cameras. It is very important to know that the road is clear. But because cameras are so much cheaper, car manufacturers try to solve the issue using them instead."

Cameras are not merely cheaper, however. When the car is to step up its intelligence and start drawing conclusions, its software needs to be fed with images. Is the child running on the pavement heading into the street? What does the squiggle on the sign mean?

"Therefore, a camera would still be needed, even if the price of lidar and radar were to fall."

Moreover, in a military context, cameras are preferable because they do not emit any energy and are therefore difficult to detect.
Acquired by Apple and Meta
Mathematicians at LTH have been working with computer vision for decades, long before machine learning and self-driving cars were on the agenda, and are behind several spin-off companies such as Spiideo, Cognimatics (acquired by Axis), Mapillary (acquired by Meta) and Polar Rose (acquired by Apple).

Thanks to the technological shift towards AI and machine learning, the field has exploded and there are now several hundred research teams around the world, both in academia and, more recently, business, working in the field.

How the camera gets depth perception
Machine learning can now be used to solve problems where a solution was once inconceivable - such as calculating distances to all objects in the camera view from just one image.

Traditionally, this has been solved by using two cameras, which is based on the same principle as us having two eyes. By examining the differences in the two images, both we and the cameras gain depth perception. Things that are closer to us move more between images than things that are farther away.

"Humans can still guess the distance even with one eye closed, thanks to acquired knowledge of the world around us."

It is now possible to create deep neural networks that, like the human brain, can estimate 3D depth from just one image. Apart from the fact that one camera is cheaper than two, stereo systems - meaning two cameras - are often very sensitive to calibration errors, i.e. how the cameras are positioned in relation to each other.

The ability to create your own maps
Viktor Larsson works primarily on developing new methods to create 3D reconstructions of reality, allowing more accurate positioning.

Positioning has traditionally been solved using GPS. This technology is also needed in self-driving cars, but as the resolution can sometimes fail at five to ten metres, more precise positioning is needed if the vehicle is to operate fully autonomously.

"GPS will also not work if you are driving through a tunnel or on narrow streets lined with tall buildings which the satellite signals bounce off. It is therefore important to be able to position the vehicle with the help of other sensors."

Ideally, maps should be created using sensor data collected by the vehicles themselves. Relying instead on manually collected information creates a risk that the maps will quickly become outdated and therefore less useful.

"That's why it's important to be able to update the maps with user-collected data, so that changes in the environment, such as new shop signs, roadworks or seasonal variations in vegetation, are directly reflected in the map."

Monotonous offices and urban environments are challenging
One of the difficulties is that our cities and indoor environments are full of repetitive structures and elements, which can lead to uncertainty in the algorithms, Viktor Larsson explains.

"Many buildings are symmetrical, so it can be difficult to tell which side of the building is shown in an image of the façade. Similar problems exist indoors, where many buildings have similar layouts on different floors, and many offices look very similar. Part of my research involves developing new methods to better address these problems."

Large salary or freedom

What is it like to work in academia in a field that has become so commercialised? After all, isn't there an awful lot of money to be made by whoever wins the tech race?

"Yes, our students and postdocs are offered much higher salaries by companies than by us which makes them hard to hang on to. Businesses also enjoy other advantages such as more access to data and more computing power. At conferences, it has become common for companies to present their research and publish scientific papers… But they are also more sensitive to the economic cycle. In Sweden, academia is in a relatively good position thanks to Wallenberg initiatives and government funding programmes such as ELLIIT. I personally prefer to work in academia, as there is a completely different freedom to explore new issues, which I find stimulating."