Presse Release: Experience Stage Shows Live from Home Thanks to Virtual Reality
Digital transformation in cultural industry: TU Chemnitz and partners develop a social virtual reality application to experience concerts and theater performances together, away from TV and live streaming
A few weeks ago, work continued deep into the night at the Chemnitz Opera House to create a virtual twin of the opera hall and other parts of the building. For the project „SocialSTAGE-VR“ of the Chemnitz University of Technology (Professorship of Ergonomics and Innovation Management and Professorship Production Systems and Processes), the three-dimensional model shall exemplarily form the basis for further research work to transfer theater performances into virtual spaces in the future and make them accessible as a social event from anywhere in the world. To this end, in addition to the Chemnitz University of Technology, other research institutions and companies such as the Fraunhofer Institute for Integrated Circuits IIS from Erlangen, Die Etagen GmbH from Osnabrück, YOUSE GmbH from Berlin and point omega AG from Heidelberg are conducting research and development in the project, which is funded by the German Federal Ministry of Education and Research
Textured 3D model of the Chemnitz Opera Hall with underlying polygon mesh.
Spatial and social presence for unique experiences
Within the project SocialSTAGE-VR, virtual reality glasses will be used to create artificial, computer-generated environments that users will perceive as real. Through the use of avatars, i.e. computer-generated images of real people, these experiences can be shared with people at a distance. For this to be successful, 3D models of the surroundings of an event must be as realistic as possible. Among other things, the hall of the Chemnitz Opera House was virtualized as a test environment. The work primarily involved the entire auditorium because the stage is to be integrated into the overall scenario later as a live recording.
Virtual twin using Matterport with many artifacts; High quality seat modeled by hand
Comparison of different recording technologies
The work had to take place during a period in which the theater was available for virtualization for several days at a stretch. This was because different technologies were used, which were to be compared in terms of their effort and results. This should help to remove hurdles later on to commercialize the approach on a broad scale. The project partner Die Etagen GmbH used photogrammetry methods, in which a 3D model is created semi-automatically from a large number of photographs, which were also taken using a drone, as well as 360° recording technologies. The project team received support from the University of Osnabrück, which additionally carried out recordings with a terrestrial laser scanner and a mobile LiDAR system.
Future subject studies to investigate factors influencing the experience
In order to determine the best possible approach in terms of effort-benefit comparison, not only purely technical criteria are relevant. Rather, the VR experience from the user’s perspective is crucial. Thus, it is not absolutely necessary that every detail is realistically recreated from every perspective. Also, the overall experience will not only depend on the realism of the environment, but will be significantly influenced by the stage content and by the social component of a collaborative experience with other people. To investigate this interaction and identify influencing factors for optimal experiences, the Professorship of Ergonomics and Innovation Management at the Chemnitz University of Technology, coordinating the research network, is conducting various subject studies. In preparation for this, further work is also planned for the next step at the Chemnitz Opera House. For example, test recordings of new stage plays will be conducted in the fall and transferred to the virtual model created for study purposes.
Background: "SocialSTAGE-VR" project
The „SocialSTAGE-VR“ research and development project has been funded by the German Federal Ministry of Education and Research for three years since September 2021. In addition to theater performances, the project also has its sights set on other live entertainment formats in the area of stage performances and TV formats. The cooperation with the Chemnitz Theaters is primarily intended to test research and development work that will provide the basis for commercializable applications after the project period. To support this, other associated partners such as ZDF and the Berlin pop group MIA are involved in the project as part of a user-centered development process. In addition, there is a close exchange with actors from the European Capital of Culture Chemnitz 2025. Interested parties from culture and the media are welcome to contact the project coordinator and enter an associated partnership. Regular information events are planned for the duration of the project, at which project results will be presented and discussed with the cultural and media sector.