One thing that has always bothered me with technology today is how it doesn’t understand user’s emotions, much less uses those emotions to craft unique experiences.
With Moodia I decided to do exactly what – by using a smartphone’s front camera to capture the user’s face, I set out to build an environmental puzzle game that’s governed by nothing more than an user’s expressed emotional state. By using facial recognition software I was able to correctly interpret changes in expression (such as eye brow movement and muscle contraction) into 6 easy-to-detect emotions: anger, sadness, joy, fear, surprise and disgust, which can then be used to trigger changes to the environment, like weather. The result is a 3D world truly responsive to your emotions, where a player must “act” through distinct scenarios to progress.
In the tech demo I created (which I’ll be publicly posting soon), the user must consciously switch his facial expression between joyful and sad to help a flower grow – a face full of joy brings out the sun, while a face full of sadness brings out the rain. Simplistic, but illustrates the concept. To finish the stage, the player must switch between the two effectively to nurture the flower and move to the next level.
The smartphone’s camera records the user’s faces in 320 x 240, which was the perfect balance between performance and reliability of results. Other physic-based puzzles were also created to demonstrate the concept, with emotions still manipulating natural elements like weather and temperature. The greatest thing about this project so far have been people’s amazing reactions when they used it – I’m not sure what Moodia is yet, but it seems to be something truly special.