Dr. Marco Scirea
Associate Professor
Dr. Marco Scirea is an Associate Professor at the SDU Metaverse Lab and is revolutionizing how we experience music in virtual environments through the lens of artificial intelligence and procedural content generation.
Today, virtual reality forms the basis for entertainment, education, and therapy for billions of people globally. Music plays a central role in our engagement with virtual worlds, and the efficiency of educational or therapeutic virtual environments.
Given the worldwide lack of resources for education and therapy the use of VR becomes vital. Through AI-driven procedural content generation, virtual environments can adapt to individual users, ensuring that mental healthcare is available to everyone in the future.
AI is the way to human technology
As an award-winning scientist, Dr. Scirea’s research focuses on utilizing artificial intelligence to create music and audio that changes dynamically based on both the actions of the individual and their emotional state.
This work is a necessity beyond merely making virtual worlds more engaging. Dr. Scirea seeks to improve the performance of virtual environments by redefining how we use audio. Hearing is key in how we experience the world yet, despite this, the use of audio in our interaction with design is underdeveloped at best.
Dr. Scirea’s work is fundamentally pushing the boundaries of how we interact with technology: Imagine playing a video game where the music changes based on your actions, or undergoing therapy in a controlled, yet dynamic environment tailored to your emotional state.
The potential for education software is substantial, with audio and content that changes based on the engagement level of a student. Dr. Scirea’s work creates the foundations for these future innovations.
Mental health care in VR
An example of the possibilities inherent in procedural content generation is VRaid. This multi-site research project has redefined the treatment of social anxiety, a very common disorder globally.
Today, the preferred therapy is exposure therapy, but this requires both patient and therapist to be in real-world situations that are uncontrollable and potentially dangerous. These setups are also not adaptable to the individual patients’ emotional landscape.
VRaid solves these problems by moving the therapy session into an individually tailored virtual reality. The data-driven model of anxiety, that Dr. Scirea has developed, additionally gives back control to the therapists by providing crucial insight into how each patient is feeling during remote therapy sessions.
AI-powered adaptive music
Another example is MetaCompose, which solves the problem of repetition in the music of virtual environments, such as video games. MetaCompose uses AI to generate adaptive and unique music that not only responds to the narrative in-game, but also to the player’s emotions and actions.
Dr. Scirea uses his expertise in procedural content generation and AI to pursue his goal of making technology more human. His work is a stepping stone towards a world where technology adapts to its users, not the other way around.
Dr. Scirea hails from Italy, moving do Denmark for his PhD. He teaches AI and game programming at the Game Development and Learning Technologies programme with the goal of inspiring a new generation of programmers to take virtual worlds to the next level.
Partnerships and collaborations
Publications
Current projects
VRaid
What does sweat, and virtual reality have to do with treating social anxiety? VRaid is an ambitious collaboration between clinical and technical researcher that seeks to develop the VR8 solution, an innovative and personalized treatment for social anxiety in adults.
Through VR patients will be exposed to anxiety-inducing situations controlled by their therapist, such as public speaking. In real-time biometric data, such as heart rate, sweat and respiration levels, informs the VR system that then adapts to the individual needs of the patient.
Through artificial intelligence the biometric data is compiled into an ‘anxiety model’. Over time the system creates an anxiety model for each patient, and this knowledge is used by the therapist to estimate how challenging the anxiety-inducing situation should be. For instance, if simulating public speaking should the audience be more attentive to what the patient is saying, or would it be beneficial for the treatment to introduce an indifferent audience?
Marco Scirea uses his extensive knowledge of sensors to collect the biometric data, which is then used in combination with state-of-the-art artificial intelligence methods to develop the ‘anxiety model’.
Follow the development of VRaid
Camouflage generation
Real-life camouflage patterns are restrictive. Not only are they copyright protected, making real-life patterns unusable in game development, but existing patterns only work in realistic environments. With this project the aim is to provide a method for game developers to create effective and copyright-free camouflage patterns that can work in any environment—from forests to alien planets.
In collaboration with Adjunct Professor Joseph Alexander Brown, this project presents a procedural method for the production of camouflage. Using the U.S. Woodland Battle Dress Uniform or M81 as an inspiration for the pattern, a genetic algorithm that uses image processing was created. This method can generate texture that allows the soldier model to blend with a woodland environment.
On-going research will pursuit developing camouflage for every environment imaginable, making copyright-free camouflage for fantastical and futuristic settings a reality.
Research Interests
- Affective computing
- Algorithmic composition
- Artificial Intelligence
- Computational Creativity
- Evolutionary Algorithms
- Game AI
- Procedural Content Generation
- Computational Intelligence
Past Projects
Have you ever gotten sick of the music in a video game? Not because it’s bad, but because you have heard the same ‘you’re fighting an enemy’-track for the past 70+ hours of gameplay. Metacompose aims to fix this!
Metacompose is an affective expressive real-time music generator that is designed to be used in video games. Now what does that mean? It means that Metacompose can be used to produce music in real-time, absolutely from scratch, while you’re playing a video game.
Not only that, but the music composed adapts to your actions. Do you enter a dark cave and it’s scary? It’s reflected in the music. Have you suddenly triumphed? The music will adapt appropriately to enhance the surge of ‘feelgood’-emotions beating a difficult boss merit.
But why? To enhance the player experience. Music communicates through emotions, and if video game music can express specific moods, it can reflect how the player’s emotional state is. With real-time adaptability the music can reinforce or manipulate how the player is supposed to feel.
Metacompose is a powerful tool in the toolbox of game developers, allowing in-game music to reach the responsive heights the interactive medium of video games deserve.
Read the award-winning paper
“MetaCompose: A Compositional Evolutionary Music Composer” here.
Teaching Philosophy
When I teach my main goal is to have students surprise me. When a student presents an idea that I would not have thought of, it means that the student has gained a strong grasp on the field and is able to critically look at the current research and see flaws or gaps.
I find that teaching centred around research produces the best results. By having the students work on innovative and personal ideas, they have the chance of truly becoming experts in their field. This way students can approach real-life problems from a creative and critical perspective.
I believe that there is a strong connection between being critical and creative. Both vital aspects to cultivate in students. A person that just accepts information without question will not see the nuances and possibilities allowed by an imperfect theory or approach.
In the current age of “post-truth”, which shuns expertise, I believe our role as teachers is critical to our society: we are the ones that must show students how important it is to exercise critical thinking.
Courses
Game Programming 1 teaches students how to program their own video games and how to solve game-related problems using algorithms and complex game engines such as Unity.
Game Programming 2 builds on the previous course by covering advanced topics, such as implementing online multiplayer, saving data using databases, and simple artificial intelligence techniques.
In Advanced AI for Games students learn a range of artificial intelligence (AI) techniques and how these can be used in the creation and playing of video games. Moreover, the course takes a research-based approach, where the students work on an AI-powered solution to a research problem.
Supervision Topics
- Adaptive content
- Generative systems
