When the audience is integrated into an artistic performance through the use of participatory strategies, the suggested cooperation at eye level often takes place only superficially. The participants are only given limited power to act and are usually involved in the supposedly collective creative process without authorship.
As an extension of participatory processes, co-creative processes are playing an increasingly important role, not only in the artistic field. Their aim is to create works in which all actors are directly involved in the creative process and simultaneously act as recipients, co-authors and editors of the work. Co-creation thus describes the method and the result of a joint creative process by heterogeneous groups of people or statuses. This kind of art process distributes authorship between artists and the audience, leads to a kind of de-hierarchisation and expands the role of the recipient in the „artist-artwork-viewer complex“.
In collaboration with students from various universities and with scenographers and musicians, we developed and tested different approaches for the use of virtual reality in co-creative scenarios in prototypical experimental arrangements. In addition to exploring the technical limits and possibilities, the focus was on the question of how collective creativity and scope for action can be used to produce a joint work with artistic expressiveness. We implemented the insights gained in the experiments in the VR project „Spatial Encounters“: This hybrid-real encounter space explores dialogue processes between music, people and space at the intersection of analogue and digital worlds.
The motive for this project is the dissolution of classical concert conventions and frontal transmitter-receiver situations, combined with the search for new bidirectional forms of dialogue between acoustic and visual spaces. In this transformative transitional space, we want to direct the attention of the participants on the one hand to the moment of „making music“, and on the other hand to the joint creation. This is connected with an invitation to „listen differently“.
The underlying software solution for the project is a web-based XR application developed by us and hosted on a local server: Mobile VR headsets (for example Meta Quest 2) can access a shared virtual space via the integrated browser. This is where the live performance takes place: One or more musicians, a visual jockey as „Master of Virtual Scenography“ and up to nine visitors meet in an equal dialogue.
In an open space of about 150 square metres, the audience is immersed in a virtual scenery, which is then filled, designed and experienced together for the next 20 minutes. They move freely in these digital landscapes and generate visual effects and sculptures through their encounters and spatial relationships. The resulting immaterial spatial bodies and virtual sceneries are interpreted live musically. At the same time, the musicians themselves give stimulating impulses and moods to the performative interplay. Through these diverse interactions, the shared experience in virtual space becomes a catalyst for a co-creative process of creation.
In these digital landscapes, the members of the audience move freely and generate visual effects and sculptures through their encounters and spatial relationships. The resulting immaterial spatial bodies and virtual sceneries are interpreted live musically. At the same time, the musicians themselves give stimulating impulses and moods to the performative interplay. Through these various interactions, the shared experience in the virtual space becomes a catalyst for a co-creative process of creation – an ephemeral collective artwork and a musical-visual resonance space emerges. Co-creative processes are characterised by the greatest possible scope for spontaneity and the unexpected. In order to unfold this potential, a high degree of flexibility and openness on the part of all participants and clearly formulated framework conditions, rules and boundaries are necessary. For „Spatial Encounters“, we have therefore conceived a predefined framework that consists of a code-based set of rules that determines the visual and of a dramaturgical framework that moderates the processes.
In „Spatial Encounters“, this dramaturgical structure corresponds to a tried and tested sequence: the actual performance is embedded between an initial welcome scene with technical onboarding, a free exploration phase and a final offboarding.
Scenographically-visually, we work in two realities: On the one hand, there are pre-produced virtual sceneries and landscapes made of simple basic elements (ground, horizon and sky) in different dimensions, colourfulness and texturing. For our premiere, we have put together a series of monochromatic colour spaces, futuristic-technoid cyberspaces and naturalistic landscapes (for example, gentle Thuringian rapeseed fields and shimmering-hot desert surfaces). The combinability of scenes and components results in formally abstract but atmospherically specfic moods that leave room for interpretation. On the other hand, the staging of physical space – as a playing surface – focuses the audience‘s attention and at the same time formulates a protective space: the more spatial „safety“ can be created for the performing group, the better the participants can engage with the common ground. The use of scenographic design elements, such as staged lighting, fog, floor surfaces (soft carpeting, dance floor, etc.) and the visual framing of the playing area determine the performance space and thus the artistic event.
The hybrid-real stage space of „Spatial Encounters“ is defined by the synchronicity of subtle anchor points, recognisable spatial edges and surfaces that appear equally in both worlds. Both spaces – the physical and the parallel virtual – influence each other in their entanglement. The congruent overlapping creates a third space: a hybrid-real in-between world as a transitional space. This third space is characterised by special qualities and exposes the viewer to new perceptual phenomena. (Blumenkranz 2010). It is clearly more multi-layered, more complex and determined by more variables than the physical or virtual stage space. Such a hybrid-real bridging space enables communication between the real and the virtual. It functions as a medial space in the original sense of the word: as a mediator, as an intermediary. „The connection of the real and the virtual constitutes the substance of hybrid space. This space […] cannot exist autonomously, however, since no independent additional space is created.“ (Blumenkranz 2010) In its fleeting presence, it is indispensably linked to the coexistence of both realities. The nature of this in-between world is determined by the relationship between real and virtual space and the synchronicity in terms of time and place: are the two realities on an equal footing or do they have different importances? Are all the actors in the shared space at the same time and do their actions seem to be in real time or time-delayed? Are they congruent, divergent or contradictory in their spatial position and scale? The variability of these properties and the resulting complexity is what makes im/material spaces so appealing.
The perception of the in-between space is determined by the temporary state of separation between physical existence in physical space and mental existence in the virtual environment. Through this conversion of the familiar mind-body relationship to a new mind-avatar relationship, the tension between material and immaterial space can be experienced in one‘s own body. The dialogue between the realities mediated by the third space only appears through the (inter)actions of the actors. The spatial dialogue is fed by the interaction between the real bodies and their digital avatars: In order to actually be able to act in the virtual world, the actors need a representative entity that transforms their physical characteristics into the virtual. The digital body is the interface that needs to be designed: the avatarised body determines our relationship to the virtual environment (Fetzer 2020).
In „Spatial Encounters“, the virtual avatars are embodied as a cone-shaped stack of floating rings whose abstract forms are reminiscent of Oskar Schlemmer‘s costume designs for the Triadic Ballet. The coloured rings not only define the visual appearance, but also describe a protective area around the bodies of the users. The kinetic physique of the ring avatar has a slight delay in movement (like a billowing ball gown) and encourages the user to playfully explore his or her own ability to act: swaying the body in place, dancing through the playing area, jumping in the air, crouching down, lying down and even merging with other avatars. These physical actions additionally trigger visual effects, such as colour changes, perspective changes or light explosions. One performance participant describes her VR experience like this: „I was curious to get close to the other colour cones, to touch them, to sway with them in dance. Again and again we were connected by ribbons in a triangular relationship. Rhythmically rising lines became sculptures, like modern skyscraper architectures. We could change their shapes again and again. And then we could climb into these flying sculptures or watch them fly away.“
The dynamic movement patterns of the users evoked in this way become a shared dance in the group and spontaneous choreographies emerge. Unconsciously, the joint exploration of closeness and distance becomes the defining theme of the performances. Through the sense of self-location, body ownership and agency created by these simple means (Kilteni, Groten, Slater 2012), a „sense of embodiment“ develops in the users. This is a prerequisite for the success of immersion and co-presence in hybrid-real space. It is not the dramaturgical framework or the predefined staged spaces that creates the artistic work. Rather, the work unfolds through the interaction of the dancing actors and their immaterial movement sculptures, the virtual environments and the improvised live music. The performance thus becomes an intersubjective experience between the participants that places the relationships between them at the centre of the art event. The result is a multimodal – ephemeral – collective artwork.
Autor:innen: Franziska Ritter und Pablo Dornhege
Further reading:
Siegmund, Gerald: Das Problem der Partizipation: https://www.goethe.de/de/kul/tut/gen/tan/20708712.html, 2016
Simón Lobos Hinojosa & Charlotte Rosengarth: https://www.uni-hildesheim.de/kulturpraxis/partizipative-kuenste-im-rahmen-kultureller-bildung
Blumenkranz, Anna: Reale und virtuelle Räume. Interaktivität in raumbezogener Kunst, S.32 und S.75. Bachelorarbeit an der Ludwig-Maximilians-Universität München. 2010.
Fetzer, Frank. Mixed Reality Is Already There! The Player‘s Body as Foundation of the Videogame Experience in Mixed Reality and Games, 2020, S.252f
Konstantina Kilteni, Raphaela Groten, Mel Slater: The Sense of Embodiment in Virtual Reality in Presence. by the Massachusetts Institute of Technology, Vol. 21 No. 4, 2012, S. 373–387