Digitization of Event Spaces
Simple ways to create virtual event locations based on real or imaginary spaces.
In contrast to pure 360° video recordings, 3D modeled virtual rooms allow the user self-controlled freedom of movement in the room and the feeling of one’s own embodiment in this room. These are prerequisites for a high level of immersion, a feeling being in the location. Event locations have a great influence on the atmosphere of an event and thus on the experience. But the creation of visually appealing virtual spaces is complex. Whether using photogrammetry, multi-camera computer vision, 3D scanning or modeling, we find ways to generate individual virtual event spaces as easily as possible. And this regardless of whether real locations are to be copied and rethought or entirely new imaginary spaces are to be created.
Textured 3D model of the Chemnitz Opera Hall with underlying polygon mesh based on photogrammetric virtualization
Stereoscopic camera system for real-time transmission of real stage performances into virtual environments.
Transmission of Stage Content
Real stage content transferred to virtual spaces in real time.
Real stage performances form the core of cultural events and must be able to be transferred to virtual spaces. There are two main ways to do this: virtualization as 3D content, for example with avatars, or the transmission of real video recordings. To enable an experience as real as possible, we are primarily experimenting with the integration of real footage. For the greatest possible immersion, we use multi-camera systems that enable stereoscopy. There are many technical problems to solve: for example, the parallax effect that occurs when perspectives change, the optical fusion of real stage content into virtual event locations, and the coordination of lighting effects between real content and virtual space. For practical use, recording, transmission, and integration must function as simply as possible and in real time.
Spatial Sound
Spatial 3D audio as the basis for immersion, the feeling of being on location, and social presence, the feeling of a real being with other people.
High-quality audio has a strong impact on the sense of presence and the illusion of being there. Spatial audio is particularly important, also in terms of social interaction with others in the virtual environment. When sounds are heard from the expected directions, the listener’s ears can be fully immersed in the illusion of actually being at the event and interacting with others in the audience. In the audio domain, Fraunhofer IIS leverages its experience as a technology innovator by contributing a library for spatial sound reproduction and connecting its solutions for spatial audio.
Spatial 3D audio based on the most advanced development work of Fraunhofer IIS. (© Fraunhofer IIS / Kurt Fuchs)
Part of the project team takes part in the action in the digitized event space
Multi User Capability
Connecting Users with each other as the basis of social interaction and the perception of a communal experience.
Creating real-world events requires transforming viewers from passive consumers at a distance, as in video streaming, to active participants in the event. To do this, the user must become part of the environment and be able to interact with others. SocialSTAGE-VR not only puts everyone in the front row, but also removes the limitation of audience size – even the smallest club concert can be made accessible to an almost unlimited number of viewers. Aside from making the experience more realistic, the technology is also designed to inspire artistic production and open up new ways of audience participation. To prevent disruptive influences from other users on one’s experience, we are developing asynchronous multi-user features that allow for intensive interaction in small groups, but across groups only allow for the transmission of contextually relevant information.
Avatar Design
Visually appealing and customizable avatars enhance the social experience of virtual stage events.
The own embodiment and the embodiment of other locally distant persons as avatars in the virtual environment is a basic requirement for a SocialVR application. For virtual stage events, this creates the possibility to generate shared and interactive experiences with friends and acquaintances. The goal here is to increase social presence. For this purpose, we work on the basis of existing avatar systems to create a visual representation that is as realistic as possible or familiar to people we know, as well as a real-time capable transmission of non-verbal communication, through gestures and facial expressions. This allows the user to participate in lifelike social interactions in the virtual environment that feel very natural and to sing and dance together with other users.
