Big Societal Questions
Digital realities such as virtual humans, metaverse-like applications, and digital twins are hot and controversial topics. More and more news platforms use virtual news anchors, cities are desperately seeking an urban digital twin, and one of the most influential companies of the past two decades changed its name from Facebook to Meta, a nod to the metaverse, an all-encompassing digital reality. A place where we should be able to meet, connect, shop, and work, it would be a forum for democracy, and some see it as a 3D version of the internet. Investments in digital realities are substantial.
At the same time, the real world is facing enormous challenges in terms of climate change, biodiversity, democratic erosion, aging, population growth, and increasing inequality in wealth. It feels contradictory to invest billions in a digital reality while the real world is experiencing such urgent problems. But it is no longer a question of whether we want digital realities – With significant investments in digital realities and technological innovations, we can assume that digital realities will play a role in how we experience the world. Only what will that world look like? That remains unclear.
Furthermore, how we experience these digital realities is often still the big question. Some key questions include: What is the impact of digital realities on interaction, in a digital environment, or with a virtual human? What opportunities do digital realities offer for education? What is the future of entertainment and media? Can digital realities contribute to combating loneliness, or will they cause it? What will be the role of digital realities in areas such as participation and urban development? And perhaps the most important question: can virtual worlds help solve real-world problems?
The SIA Sprong group DIGIREAL (2022-2029) develops and investigates innovations with these digital realities as key technologies and methodologies for addressing societal issues and challenges in various fields such as education, journalism, healthcare, and spatial planning at sea or on land. The universities of applied sciences BUas and Fontys lead the project with MindLabs and dozens of partners.
The Challenge
The underlying problem lies in the increasing difficulty for academic and applied researchers, let alone for users and decision-makers, to keep up with global technological developments and applications of these key technologies. Digital realities are largely provided by a profit-driven ‘big tech’ industry and/or startups, which often lack ‘transparency’ in their development and business models and sometimes have limited attention to societal implications. Therefore, a multidisciplinary R&D agenda is needed that explores both the technological developments for societal purposes and maps out the risks.
The Results
An important result of DIGIREAL is a knowledge and innovation centre in the field of Digital Realities that is part of the two involved universities of applied sciences, BUas and Fontys, and the MindLabs ecosystem. It includes, among other things, a joint research program, a coherent portfolio of multiple externally funded R&D projects, and chain cooperation between R&D labs.
Virtual Humans – Real Emotions
DIGIREAL investigates and innovates, among other things, the technology and application of virtual humans. Virtual humans are software programs that give the illusion of a human through human appearance (body) and verbal (speech) and non-verbal (facial expressions of emotion, gaze, gestures, and posture) behaviors simulation (Burden & Savin-Baden, 2019). This application of virtual humans is applied in various domains, such as education, healthcare, security, journalism, and public decision-making. This is partly funded by DIGIREAL but has also led to a growing portfolio of research grants, including a recent one from Regiodeal Midden Brabant (2022-2026).
Digital Twin of the Ocean
DIGIREAL also applies innovations in the field of game-based digital twins for spatial planning, especially at sea. One of the innovations in the EU project ILIAD Digital Twin of the Ocean is a virtual and augmented reality platform that, with AI, automatically generates super-realistic 3D representations of the consequences of spatial decisions at sea. Think of new experimental technology for offshore wind, such as floating turbines or solar panels. This can visualize the impact that this technology will have on shipping or marine ecology in the future – for example, for locations in the North Sea.
DIGIREAL in Education
DIGIREAL involves dozens of students from knowledge institutions in education or in the Cradle R&D lab in various ways in virtual humans, metaverse applications, and digital twin projects. In addition, 4 PhD candidates and 1 PD researcher use digital reality innovations for their research.
Partners
- BUas (Breda University of Applied Sciences), with three professorships: Serious Games, Entertainment Games, and Immersive Media.
- Fontys University of Applied Sciences, with various professorships, including ICT and Journalism.
- SME partners in the AI, Games, and Immersive Media sectors: Enversed, Reblika, Argaleo, Flooid, de Effenaar, among others.
- Other partners: MindLabs (Tilburg), Tilburg University, Digishape, Logistics Community Brabant, Chronosphere, among others.
- Government: Regiodeal Midden Brabant (2024), Ministry of Infrastructure and Water Management / Rijkswaterstaat, ILIAD Digital Twin of the Ocean (EU Green Deal), and many other EU projects.
“We have developed and tested a range of virtual humans. An application example is training doctors to recognize possible medical emergencies from the mother’s facial expression during childbirth. We work on simulations of non-verbal emotions, such as various expressions of sadness, including crying. This is useful for psychological research into empathy or for therapy. For MindLabs, we have developed the virtual human MAI (MindLabs Artificial Intelligence) as a science communicator for our own innovation ecosystem and the DIGIREAL program. We investigate the possibilities of virtual humans to improve the quality of societal discussions and political decision-making, for example, through virtual journalists.”
“We strive to distinguish ourselves in the field of VH by taking an open, critical, and socially responsible approach. We do this, among other things, through ‘moral design’ and based on principles such as open source and pre-competitive R&D. We also make VH innovations available for scientific and action research. Furthermore, we work on the application and evaluation of VH innovations for major societal issues, such as health, sustainability, and broad prosperity.”
“I don’t want to make toys. The products we work on must have a societal application. Otherwise, I’m not interested. There is so much technology coming at us now, and an open and critical perspective is important to investigate what digital realities can contribute to, or tear down.”
“DIGIREAL is a driving force behind the developments of MindLabs. Various knowledge institutions are working concretely, together, and achieving results that create an energy around our ecosystem from which spin-offs continue to emerge.”