On a May evening in Iaşi, Romania, the grand halls of the Palace of Culture came alive with snaking queues of eager visitors. They gathered to experience the newest interactive exhibit unveiled for the Night of Museums: an emotion detection device created by the technology innovators at Levi9 in collaboration with a local museum. Using Artificial Intelligence for facial analysis, the installation could detect and display a user’s prevailing mood in real-time.
Thousands of visitors interacted with this cutting-edge AI-enabled emotion recognition system, which was able to detect emotions such as happiness, curiosity, frustration, wonder, sadness, and many more. As for the emotions of the Levi9 team in front of their project’s success? “Pure joy,” says Engineering Lead Codrin Băleanu, who coordinated this project.
“Joy,” he emphasizes, “is the key to technological innovation.”
Step into the future
On Museum Night, visitors in Iași expected to step into the past with long walks along historical and cultural displays. Instead, they stepped into the future. Entering what seemed like a photo booth, they had an AI-powered immersive experience. A facial expression reader built by Levi9 volunteers used real-time facial analysis to identify key expressions like smiles, frowns, and surprise. This allowed the application to estimate the prevailing mood of users as they interacted with it. The interface then reflected the captured emotion back to the visitor, making the experience highly accessible and engaging for all audiences. The installation was such a popular exhibit that hundreds of people patiently waited in line to interact with the emotional scanning technology, only to be amazed and dazed when their turn finally arrived.
Decoding human emotion
Several years prior, attendees at a developer conference had a similar WOW experience. Codrin Băleanu, Levi9 Engineering Lead, was on stage, talking about topics such as computer vision, face detection, and machine learning. At a precise moment during the presentation, he turned a live webcam towards the audience, allowing the participants to see themselves through the eyes of Artificial Intelligence: each of them identified and framed by a rectangle, their public social media profiles displayed. At that time, Băleanu was one of the few computer vision aficionados who had the foresight to see what that spectacular technology would become.
With ChatGPT and image-generating AI like DALL-E and Stable Diffusion capturing the imaginations of millions of people, Codrin Băleanu saw an opportunity to ride the AI craze and engage the local community with technology. Of course, he already knew the answer to the question on everyone’s lips: Can artificial intelligence read human emotion?
“The human face is relatively easy to identify and recognize in the sense that there are some features that are exactly the same and exactly in the same places, regardless of the shape of the face or the general appearance. There are software solutions where the moment a picture is introduced as test data,” Băleanu explains. “In our case, to identify the face and then to determine the emotions themselves, we created an algorithm that was simple enough by Levi9 standards but efficient enough.”
The Levi9 AI facial expression reader scans for key facial landmarks like whether the mouth is open and smiling or the eyes are wide in surprise. By measuring the distance between these points and comparing them to defined intervals, the application categorizes the strongest emotion detected.
The interactive exhibit was built using the popular Python programming language along with the OpenCV and MediaPipe libraries for computer vision capabilities. OpenCV enabled vital image processing like capturing webcam input, framing the user’s face, and rendering the dynamic effects. MediaPipe’s pre-trained machine learning model detected the subject’s face in real-time and extracted 468 facial landmarks.
These facial points were key for mapping out the grid overlayed on the user’s face and classifying their emotion based on key expressions like smiles or raised eyebrows. The one exception was detecting a “happy” mood, which was inferred from the curvature and width of the mouth relative to the overall face width. Specifically, smiling was identified by measuring how open and upturned the corners of the mouth were in the facial landmark data.
While 468 data points seem the Matrix-level technology, by machine learning standards, this is not highly performant. But the data richness and complexity were just right enough to allow for relatively cheap computing power and an engaging experience for the public.
Technology and culture can empower each other
The Palace of Culture provided an ideal stage for making emerging technologies relatable through an immersive cultural experience. “The reaction of the participants was very good,” says Coralia Costas, PR, Marketing, Logistics, Projects, and Programs coordinator of the host.
Costas says that fusing technology and culture benefited all audiences. “For the young ones, who were born with technology all around them, the experience was native and it reduced the chronological distance they might feel from many exhibits. For seniors, it was an invitation to voluntarily and non-invasively test, along with their grandchildren, for example, the way in which technology is our life partner today.” One highly appreciated point of the interactive display was that it did not require any particular technological skills.
This is precisely what the Levi9 team has aimed for since the inception of the project. Codrin Băleanu knew the application needed to be visual, interactive, and playful for maximum impact. Just like making complex ideas accessible, wrapping advanced applications in cultural experiences broadens their appeal.
The wow factor of the emotional biometrics installation even worked on Codrin’s own peers. As Levi9 employees were passing by in the hallways, they stopped to inquire about the project and the technology behind it. They wanted to play with it, too!
The power of play
“For me, play is the key. It is the most important thing in this profession,” reveals Codrin. “I believe it helps us move forward, to continue to innovate and to be able to be in step with and even ahead of market trends.”
This quote encapsulates the ethos that made the emotion scanner possible—one valuing space for playful exploration above all else. Codrin had palpable excitement when the concept first formed, before final approvals were even completed. He dove right in, pulling together an agile team of four people united by their shared enthusiasm.
“I was so excited about this idea that I simply started working on it,” Baleanu recalled. “When you have enthusiasm, things happen almost without any kind of friction. It goes by itself, and development began to take place at a much faster rate than expected.”
This freedom to organically build and experiment is oxygen for innovation in fast-changing technology fields. It provides room for programmers to pick up emerging skills like computer vision outside of their daily responsibilities. Regularly indulging passions through side projects can combat the inevitable sense of routine that sets in.
New skills such as creating emotional analysis algorithms might prove invaluable in the future. These systems have diverse real-world applications beyond museum exhibits. Sophisticated implementations analyze sentiment across crowds in public spaces like metro stations, airports, and train platforms. Individual facial tracking helps identify VIP customers, as is the case in certain Asian banks where managers receive notifications when high-value clients approach so they can greet them personally.
Some travel involved
The runaway success of the emotion scanner proved museums can provide fertile sandboxes for the public to explore emerging technologies hands-on. It forged connections between groups that might not otherwise interact. The Levi9 team hopes to be able to bring this immersive experience to communities all around Romania.
Moreover, the collaboration between Levi9 and the Palace of Culture Museum was a prime opportunity to let the Levi9 team joyfully tinker with new applications of facial analysis and AI. Their creation delivered proof that dedicating time for play unlocks creativity and helps organizations respond nimbly to evolving technologies.
