Making Augmented Reality Suitable For Society

Christian Holz, what is actually meant by augmented reality?

The vision of augmented reality is older than you might think. Ivan Sutherland, the pioneer of computer graphics, published his Ultimate Display concept - a computer display that serves all the human senses - 60 years ago. Nowadays, it's generally about opening up new dimensions of reality, for example, with smart and interactive glasses that can expand and enhance people's perceptions.

Is the impression misleading or is augmented reality still in its early stages?

In a sense we're already living in an augmented reality through our smartphones because we use their services every day. But the technology and the interfaces between the real and the virtual world are continuing to evolve - for example, we use interactive smart watches to check messages or make calls.

What topics will you be researching in the new hub?

The interfaces between the real and the virtual work smoothly with conventional devices. So if I press a key on my laptop keyboard, the corresponding letter appears in the virtual document on my screen. In augmented reality, on the other hand, the virtual content is incorporated directly in my physical environment - the keyboard appears on the table in front of me, and the document floats in the air next to it. But how can the system recognise and understand the world around me? How does it know where my hand is and whether I'm typing or merely gesticulating? And how can we embed whole virtual humans and 3D objects in the world so that they look deceptively real and we can plausibly interact with them? These are key research questions that we'll be studying in the hub.

What are the prerequisites for this to succeed?

Nowadays, we expect technology to be usable anywhere, to be completely mobile and ideally to be light and suitable for everyday use. One big difficulty for future platforms will remain the miniaturisation of hardware and the extension of runtime capability. Another problem is graphic generation. One of the things we're researching in the hub is therefore how scenes and objects can be recorded, reconstructed and correctly depicted in real time. Human-object interaction and activity detection are also major challenges.

Why are these human-device interfaces so important?

The AR device itself, but also how it is used, must be socially acceptable. Interactive interfaces not only influence an individual's experience but also impact the shared experience of a group. The famous American computer scientist Mark Weiser once said: "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it." The use of technologies must therefore be aligned with our everyday behaviour. Intelligent systems must also be able to distinguish and interpret our actions. This is precisely what my team is studying. For this purpose, we are also developing AI-based methods that can do this much more quickly and efficiently.