VR + AR = XR

Ever since virtual reality is a common term and “immersive technologies” can be purchased by normal consumers the exact terms of the technologies are likely to be confused. It should be mentioned that the most common terms – Virtual Reality, Mixed Reality and Augmented Reality – all serve to describe gradations of reality:

Virtual Reality (VR) describes the fully immersive simulation of physical space, i.e. the complete immersion in a virtual environment. Due to constant technological development, future virtual realities may no longer be distinguishable from the real world. Examples of VR are interactive experiences in which users can immerse themselves with virtual reality glasses.

In the context of VR, the term immersion describes the dive into an artificial, digital environment. The degree of immersion is strongly determined by the quality of the experience and the willingness of the recipient to accept the artificiality of the virtual worlds as natural (see also suspense of disbelief). The higher the degree of immersion, the stronger the presence of the user in the virtual experience: the perception of one‘s own existence shifts from the physical environment to the simulated environment.

The process of immersion and the state of presence in an artificial world are the main characteristics of virtual reality and the basis of its special potential. In game design, four different forms of immersion are distinguished: spatial, emotional, cognitive and sensory-motoric immersion.

Spatial immersion arises from the visual quality of the experience and determines the recipient‘s willingness to accept the artificiality of the virtual world as natural. Emotional immersion becomes possible when a narrative content leads to emotional interest in the events and thus to absorption by the story – it is thus similar to the effect of an exciting read or a gripping play. Cognitive immersion is based on concentrated abstract and creative thinking and is usually achieved by solving complex or creative tasks. Sensory-motoric immersion is the result of feedback loops between the physical actions of the user and their impact on the (game) experience, whereby the perception of one‘s own physical existence can shift from the physical environment to the simulated environment.

Unlike virtual reality, augmented reality „only“ enriches physical reality with digital content. One way to to that is using smartphones: the phones camera captures images of the „real world“, then digital objects are added on the phones screen. Another way is using data glasses and AR headsets in which reality remains visible through transparent glasses and is overlaid with digital image content. One example of AR applications is in museums, where visitors can capture a painting on the wall using a smartphone or tablet camera and the representation on their device is supplemented by new digital layers, such as early sketches of the work or X-ray images.

Extended Reality (XR) is an umbrella term for the entire spectrum of virtuality. According to the virtuality continuum introduced by Paul Milgram, this ranges from completely real to completely virtual. This is why XR includes not only virtual and augmented reality but also mixed forms such as augmented virtuality: the extension of a VR production by physical objects and props.

Some of the solutions and prototypes, which were created during the course of the project, include the innovative WebXR interface. This makes it possible to develop web-based applications to display three-dimensional content on a range of different AR and VR end devices. We used this new (and not yet full functional) standard, because it allows for manufacturer- and device-independent and thus transferable software developments.

The WebXR interface makes it possible to develop web-based applications to display three-dimensional content „device-independently“ – i.e. on VR glasses as well as on AR-compatible devices.

The relatively young standard is currently being developed by a consortium of Google, Microsoft, Mozilla and other XR industry giants. The goal of the standard, which was jointly developed by hardware and software manufacturers, is to recognise existing VR and AR devices and their technical specifications independently of specific products and to display 3D content accordingly. Integration in modern browsers makes access to corresponding 3D apps easy: no special software needs to be installed – one click is enough to start them.

The biggest advantage this offers is the resulting reach while at the same time reducing the programming effort: Instead of using an application or app to access only one type of device or even only one specific product, a WebXR website can be used, for example, on the screen (with navigation via mouse and / or keyboard), a smartphone in AR mode or through high-end VR glasses. Because of these advantages, many experts see WebXR as a key element of future immersive technologies.

But the use of such a universal interface also comes with some disadvantages: Where game engines normally simplify the development of AR and VR applications, for WebXR most things must (still) be programmed „by hand“. At the same time, the desired high compatibility with a wide range of devices and different display performances means that compromises have to be made in terms of visual quality and complexity.

Authors: Pablo Dornhege, Franziska Ritter