On a May evening in Iaşi, Romania, the grand halls of the Palace of Culture came alive with snaking queues of eager visitors. They gathered to experience the newest interactive exhibit unveiled for the Night of Museums: an emotion detection device created by the technology innovators at Levi9 in collaboration with a local museum. Using Artificial Intelligence for facial analysis, the installation could detect and display a user’s prevailing mood in real-time.

Thousands of visitors interacted with this cutting-edge AI-enabled emotion recognition system, which was able to detect emotions such as happiness, curiosity, frustration, wonder, sadness, and many more. As for the emotions of the Levi9 team in front of their project’s success? “Pure joy”, says Engineering Lead Codrin Băleanu, who coordinated this project.

“Joy”, he emphasizes, “is the key to technological innovation.”

Step into the Future

On Museum Night, visitors in Iași expected to step into the past with long walks along historical and cultural displays. Instead, they stepped into the future. Entering what seemed like a photo booth, they had an AI-powered immersive experience. A facial expression reader built by Levi9 volunteers used real-time facial analysis to identify key expressions like smiles, frowns, and surprise. This allowed the application to estimate the prevailing mood of users as they interacted with it. The interface then reflected the captured emotion back to the visitor, making the experience highly accessible and engaging for all audiences. The installation was such a popular exhibit that hundreds of people patiently waited in line to interact with the emotional scanning technology, only to be amazed and dazed when their turn finally arrived.

Decoding Human Emotion

Several years prior, attendees at a developer conference had a similar WOW experience. Codrin Băleanu, Levi9 Engineering Lead, was on stage, talking about topics such as computer vision, face detection, and machine learning. At a precise moment during the presentation, he turned a live webcam towards the audience, allowing the participants to see themselves through the eyes of Artificial Intelligence: each of them identified and framed by a rectangle, their public social media profiles displayed. At that time, Băleanu was one of the few computer vision aficionados who had the foresight to see what that spectacular technology would become.

With ChatGPT and image-generating AI like DALL-E and Stable Diffusion capturing the imaginations of millions of people, Codrin Băleanu saw an opportunity to ride the AI craze and engage the local community with technology. Of course, he already knew the answer to the question on everyone’s lips: Can artificial intelligence read human emotion?

“The human face is relatively easy to identify and recognize in the sense that there are some features that are exactly the same and exactly in the same places, regardless of the shape of the face or the general appearance. There are software solutions where the moment a picture is introduced as test data,” Băleanu explains. “In our case, to identify the face and then to determine the emotions themselves, we created an algorithm that was simple enough by Levi9 standards but efficient enough.”

The Levi9 AI facial expression reader scans for key facial landmarks like whether the mouth is open and smiling or the eyes are wide in surprise. By measuring the distance between these points and comparing them to defined intervals, the application categorizes the strongest emotion detected.

The interactive exhibit was built using the popular Python programming language along with the OpenCV and MediaPipe libraries for computer vision capabilities. OpenCV enabled vital image processing like capturing webcam input, framing the user’s face, and rendering the dynamic effects. MediaPipe’s pre-trained machine learning model detected the subject’s face in real-time and extracted 468 facial landmarks.

These facial points were key for mapping out the grid overlayed on the user’s face and classifying their emotion based on key expressions like smiles or raised eyebrows. The one exception was detecting a “happy” mood, which was inferred from the curvature and width of the mouth relative to the overall face width. Specifically, smiling was identified by measuring how open and upturned the corners of the mouth were in the facial landmark data.

While 468 data points seem Matrix-level technology, by machine learning standards, this is not highly performant. But the data richness and complexity were just right enough to allow for relatively cheap computing power and an engaging experience for the public.

Technology and Culture can Empower Each Other

The Palace of Culture provided an ideal stage for making emerging technologies relatable through an immersive cultural experience. “The reaction of the participants was very good”, says Coralia Costas, PR, Marketing, Logistics, Projects, and Programs coordinator of the host.

Costas says that fusing technology and culture benefited all audiences. “For the young ones, who were born with technology all around them, the experience was native and it reduced the chronological distance they might feel from many exhibits. For seniors, it was an invitation to voluntarily and non-invasively test, along with their grandchildren, for example, the way in which technology is our life partner today.” One highly appreciated point of the interactive display was that it did not require any particular technological skills.

This is precisely what the Levi9 team has aimed for since the inception of the project. Codrin Băleanu knew the application needed to be visual, interactive, and playful for maximum impact. Just like making complex ideas accessible, wrapping advanced applications in cultural experiences broadens their appeal.

The wow factor of the emotional biometrics installation even worked on Codrin’s own peers. As Levi9 employees were passing by in the hallways, they stopped to inquire about the project and the technology behind it. They wanted to play with it, too!

The Power of Play

“For me, play is the key. It is the most important thing in this profession”, reveals Codrin. “I believe it helps us move forward, to continue to innovate and to be able to be in step with and even ahead of market trends.”

This quote encapsulates the ethos that made the emotion scanner possible—one valuing space for playful exploration above all else. Codrin had palpable excitement when the concept first formed, before final approvals were even completed. He dove right in, pulling together an agile team of four people united by their shared enthusiasm.

“I was so excited about this idea that I simply started working on it,” Baleanu recalled. “When you have enthusiasm, things happen almost without any kind of friction. It goes by itself, and development began to take place at a much faster rate than expected.”

This freedom to organically build and experiment is oxygen for innovation in fast-changing technology fields. It provides room for programmers to pick up emerging skills like computer vision outside of their daily responsibilities. Regularly indulging passions through side projects can combat the inevitable sense of routine that sets in.

New skills such as creating emotional analysis algorithms might prove invaluable in the future. These systems have diverse real-world applications beyond museum exhibits. Sophisticated implementations analyze sentiment across crowds in public spaces like metro stations, airports, and train platforms. Individual facial tracking helps identify VIP customers, as  is the case in certain Asian banks where managers receive notifications when high-value clients approach so they can greet them personally.

Some Travel Involved

The runaway success of the emotion scanner proved museums can provide fertile sandboxes for the public to explore emerging technologies hands-on. It forged connections between groups that might not otherwise interact. The Levi9 team hopes to be able to bring this immersive experience to communities all around Romania.

Moreover, the collaboration between Levi9 and the Palace of Culture Museum was a prime opportunity to let the Levi9 team joyfully tinker with new applications of facial analysis and AI. Their creation delivered proof that dedicating time for play unlocks creativity and helps organizations respond nimbly to evolving technologies.

Charging your electric vehicle is meant to be quick and hassle-free – just pull up to a station, plug in, and be on your way in minutes. But behind this seamless customer experience lies a complex orchestration of payment processing, security, and invoicing. Levi9’s .NET Tech Lead Mihaela Rădulescu and her team recently brought this frictionless process to life for a startup electrical car charging platform. The solution demanded the agility of a startup build, the security and reliability required when handling payments, and the automation needed to minimize operational costs. Stripe was the answer. It handles the payments and invoicing for the platform, and was chosen for its rapid integration, robust fraud protection, and automatic billing capabilities.

The Levi9’s customer’s platform connects Electric Vehicle (EV) charging stations with Electric Vehicle drivers. Charging station owners can manage and monitor their stations through the platform. Meanwhile, electric car drivers can find stations, track charging sessions, and more. At the end of each month, the platform needs to bill station owners, as well as EV drivers for usage.

With chargers having their own dynamic per kilowatt-hour rates that change frequently, and rates that vary based on time of day, Stripe was the ideal solution. “Nothing fits your use case 100%”, says Mihaela Rădulescu, who led the integration between the platform and payment processor Stripe. But Stripe came pretty close. “We needed a payment processor that could handle variable rates and automatically charge both end-users and charging station managers at the end of each month,” explains Mihaela.

Here’s an inside look at how Stripe works and how our client, an EV charging management company, integrated it into their platform.

Stripe acts as the payment gateway and processor, taking card data and routing it to the appropriate bank. A key advantage to Stripe is that it can be integrated with a customer’s platform, while allowing card data to go directly from the client to Stripe. “This means card data never reaches our servers, which saves us from all the time and effort needed to ensure PCI (Payment Card Industry Data Security Standard) compliance requirements are met and then upheld” revealed the Levi9 .NET Tech Lead.

To process payments, Stripe offers both its own check out page, as well as a library of pre-built components called Elements that integrate directly with Stripe.js. Elements allow website designers to better integrate the look and feel of their service during the payment process, while routing all sensitive data to Stripe directly.

When electric car drivers sign up for the charging platform, the app uses the Stripe Elements to securely collect their card details and create a Customer and Payment Method in Stripe. “This happens directly from our front end to Stripe without card data ever reaching our servers,” emphasizes Mihaela.

As drivers use charging stations, the platform tracks their usage in real-time. The platform receives “usage events” that contain station identification, start time, end time, and kilowatt-hours. Rates depend on the charging station location (city centers cost more than outskirts). Rates also vary based on time of day (peak hours vs. night). The platform matches each usage session with the correct rate.

One particular challenge during this process was finding a way to bill fractionary charging, as Stripe does not allow “product quantities” to be fractionary themselves. Some ingenuity was required from the development team at this point and they came up with the solution of billing watt-hours instead of kilowatt-hours.

At the end of the month, the EV charging platform totals each user’s power usage and generates invoices through Stripe’s API. The invoices contain line items with the Wh quantity and unit price. Stripe calculates total costs from there and handles tax calculation.

“We just provide the usage data and Stripe takes care of invoicing and payment,” explains Mihaela.

Stripe also manages a second type of invoice. The EV platform is also billing charging station owners, who use the platform to get an up-to-date overview of their stations, to manage them and associate different rates according to their business strategy.

Payments are automatically processed by Stripe within a maximum of one hour after invoicing. The payment process can be triggered by making an explicit call to the Stripe API, but the Levi9 development team preferred to allow Stripe to automatically withdraw the money from the payment card on the file. Failed payments, which sometimes happen, are automatically retried by Stripe.

For failed payments, Stripe also issues a notification, through a webhook, towards the charging platform, keeping them in the loop about the status of payment. These real time webhook updates are not limited to errors, but are sent throughout the entire process, for each event, such as successful charges and payment intents. These notifications allow the platform to always keep client data in sync.

While Stripe solved critical needs for all stakeholders, the process was not without challenges for the development team. A key lesson was around Stripe’s many options for billing. For example, subscription versus usage-based billing. Subscriptions provide ongoing charges based on fixed plans. But with dynamic per Wh pricing, subscriptions were not practical. “We had to get creative in using usage-based billing along with invoice line items and webhooks to implement a variable pricing model.”

For any company dealing with online payments, especially variable payments on-the-go, Stripe provides great advantages. By allowing the Levi9 team to successfully integrate a payment provider that lifts off the burden of financial regulations and automates invoicing and withdrawals, the start-up EV platform was able to focus on bringing the best experience to its customers and making a more sustainable, environment-friendly future possible.

If you were ever curious how many words there are in each sentence from Wikipedia’s entry on the legendary band Queen, you are about to find out. Even better, you’re about to find out how to find this out for yourself, using the power of automation and the Amazon Step Functions. Andrei Elefterescu, Levi9 JavaScript Tech Lead, has 10 years experience in AWS and worked with the Amazon Step Functions even before their launch, during the Beta program. Andrei will be your guide to the melodic orchestration of triggers, functions, validations, transition states, conditional routes, and callbacks. It sounds complicated, but trust us, it’s a kind of magic.

To find out how many words are in each sentence of Queen’s Wikipedia biography, you’ll need to do some common-sense actions: read the text, split it into sentences, count the words, then put back together all the information about each sentence in one single text.

Surely you don’t need automation for that. It’s a straight-forward, simple task. But what if you needed to count all the words in the Wikipedia entries of the Top 100 British bands of all time?

This is where Andrei Elefterescu steps in to explain what Amazon Step functions are.

Amazon Step Functions is a serverless orchestration service that helps us integrate multiple AWS services such as Lambda, S3, SNS, and so on to develop various applications. Going back to the Queen example, those steps define a “Step function”.

While it’s easier to visualize as a diagram, Step Functions are written in an AWS proprietary language, which is JSON-based.

Of course, Amazon Step functions are much more useful than counting words on a Wikipedia page. The service can be used to process images in S3, ETL (Extract – Transform – Load) processes, machine learning, microservice orchestration, IT and security automation, as well as Continuous Integration and Continuous Deployment (CICD), a process that automates the integration and deployment of code.

“Step Function has around 250 integrations with other AWS services and around 11,000 API calls that can be called from it. “In 2016, when they launched, they only had an integration with S3 and 3–4 other services.”

Before we delve deeper into how Step functions work, here are some key concepts, as explained by Andrei:

Workflow is the breakdown of what needs to be done and in what order. It can be built visually, like a diagram, in the very intuitive editor of the Amazon Steps Function. In our example, it’s the mental plan of how we are going to grab the information about the Queen Wikipedia blog article. A good plan needs to include some steps where you validate the inputs and a way to handle errors.

Triggers are what start the workflow. You can call the API or you can use other methods, such as starting it directly from the AWS console, triggering an S3 event through the SDK, or using the event bridge to start a Lambda function. For our planned mini-Queen automation, we’d just run this through the interface.

States are the different sequences in the workflow. Step Functions have several types of states, such as task states where a resource can be assigned, choice states where a decision can be made, and check states where a variable can be checked and make the automation behave differently depending on whether it’s true or not. In our mini-automation, the states include grabbing, parsing, and joining the information, but also a validation filter and a choice related to what happens when the automation throws an error.

Tasks are the actual actions and steps that are performed in each state. A task could be, for example, to split a text into sentences, and another task would be to count the words in each sentence.

Transitions are links between two steps, and each step is recorded as a transition. For each execution of a lambda, there are three entries: the transition before, the step itself, and the transition after. For the passing of information from one step to another, Amazon has a limit of 250 KB. For our particular case, the information “There are 10 words in the second sentence” must be carried from one step to the next.

Item execution is the transition from one state to another. The limit per workflow is 25.000 executions. For example, to find out how many words there are in each sentence, there are three executions: one is grabbing the actual sentence, two is getting the numbers, and three is sending those numbers to the following step.

Express vs Standard Step Functions are two types of workflows. The Standard has a year limit on being active, while the Express has 15 minutes and 100,000 parallel executions. Express workflows are used in web development, while standard Standard workflows are used for larger volumes of data.

The history of the execution is in the log, which you can easily check in the Step Function dashboard. There is a limit of 100,000 logs per workflow.

But is this melodic, yet complicated, orchestration needed? Couldn’t a Lambda function do the job instead?

For one thing, Amazon Steps are faster, highlights Andrei. This is due to their ability to support parallel runs, such as a simple map mode with 40 concurrent executions or a dynamic map mode with up to 10,000 parallel executions. On top of that, “Step Functions is a serverless service that allows for stateless states without having to have a separate database or cache, decoupling the logical side from the business side.” In other words, there is no need to worry about sizing the resources to fit the process. This is done automatically. Some other advantages that Andrei appreciates are the fact that it’s easy to see the state of your workflows and that it integrates retry mechanisms and exponential back-off.

This “Plan B” for errors plays a crucial role in workflows. For example, if we sent a blank Wikipedia text instead of the real Queen biography to the step Functions, an error mechanism would be triggered. Without it, the function would have stopped working. And errors can happen during the processing of large amounts of data, and some of them can stop the process completely. This is one of the several drawbacks that Andrei mentions.

There is a hard limit of 25.000 executions per workflow. While analyzing a Wikipedia article doesn’t seem like much, when you need to count the words in 10.000 sentences, things change. Remember, each step normally means three executions: the state before the step, the step, and the state after the step. “So, with 10.000 sentences, it’s pretty easy to get there”, emphasizes Andrei. “And if you go over 25.000, the workflow will stop, and there is nothing that you can do about it.”

Also, as step functions decouple the business logic from the logic part of the workflow, the resulting code is more complex and harder to understand. A relatively simple image processing workflow can have clear inputs and outputs, but the connection between the various microservices is challenging to understand. Another disadvantage is limited mobility. The state machine is written in the Amazon State Language, which is a proprietary language of AWS. This means that if you want to move to Azure, for example, you need to start from scratch.

Another disadvantage is the 250 KB limit that can be passed between states in the step function and between transitions. Plus, the execution history is kept for only 90 days, and the maximum execution of a workflow is one year.

Wait, a workflow that takes one year? “It’s possible”, says Andrei. “The Step Functions have callbacks, which allow the workflow to pause and wait until a human provides some input we are waiting for. Then, the callback is invoked, and the workflow resumes.”

Most of these drawbacks can be overcome by best practices. For example, AWS’s 256 Kb interstate is limiting, but AWS itself came up with a solution. You can write the payload to a PS3 file and process it with a Lambda function.

The native retry mechanism is also very useful. Step Functions provides a retry and exponential back-off mechanism to catch Lambda exceptions and throw specific errors. It also allows for the ability to make a specific decision if a Lambda fails with a throw error, such as not found.

Additionally, there is also a way to avoid the limit of having only 100.000 objects to process. “Yes, we did get there at some point, and it was not pleasant”. The solution might be to keep an eye on the process and trigger an alert when you are close to the limit.

Even so, the costs offset all the small inconveniences of the Step Functions. In Levi9’s experience, even complex processes for large clients do not go over $5 per month.