SoundScape VR – Turning music into procedurally generated interactive worlds

As a musician and geek, I’ve always been passionate about both Music and Virtual Reality – so during the HackNY 2016 Hackathon, I decided to combine the two to create an unique immersive experience. My initial goal was to create a procedurally generated world out of an user selected song that pulsates and reacts to its ebbs and flows, bringing music into a new physical dimension, creating an immersive and interactive experience out of it.

SoundScape is an immersive virtual reality app that builds an entire virtual, unique and interactive world using an user picked song as a reference. As the song plays, various elements in the world come alive reacting to the song’s beat, tempo, frequency, volume and its other aspects. The user can move their hands freely around the game space with LeapMotion hand tracking, interact with randomly generated objects and even draw unique figures with rays of light that pulsate to a song’s BPM.

We built this using the Unity3D Game Engine and Leap Motion. We developed algorithms to map beats and frequencies of user-selected songs to generate unique environments that respond to the music as well as the user’s actions within its world. SoundScape was one of the winners of HackNY VR and will be shown live on stage at the NYC Tech Meetup in November.

Leave a Comment

Start typing and press Enter to search