Step-by-step: how to access bank data with an API

Step-by-step: how to access bank data with an API

Open banking is a financial revolution at our fingertips. For customers, it gives them unprecedented freedom and ownership of their bank information. At a macro level, it allows for a standardized interface between various banks and fintech providers. For us, developers, it’s a playground for innovation.

Before diving into this quite technical guide for accessing bank data via an API provider, you might want to read our previous article on the history of Open Banking and some use cases.

What follows is a step-by-step guide on how to access bank information using an API. You might want to use your own account, or you might need to do this for testing or development purposes. This demo uses my own bank account. Feel free to use yours when you follow the instructions.

Accessing information about somebody’s account and transactions is more complicated than simply calling an API. Even though banks do have APIs, banking data is strictly regulated, so third parties that wish to call the API directly need to be registered and certified with a national authority. Registering with the bank’s API and buying a certificate costs around 1000 euros every two years. It might not be worth it if you just want to experiment and play around to see what’s possible. However, there are some limited options available.

Here are the options I considered:

Tink is a robust open banking platform, but it did not offer access to Romanian banks, which was needed for the purpose of this demo;

Smart Fintech is a Romanian third-party bank data integrator, but the free trial covered only the Payments solution and not the Accounts information solution I was interested in;

Nordigen, a company from Latvia, has a generous free tier and offers access to all relevant information about the account via an API.

For the purpose of this demonstration, I will use Nordigen.

Step 1: Get access token from Nordigen

Create an account with Nordigen, which is free to use for limited purposes. Find the API documentation at this link to be able to follow along with the demonstration.

Get your user account secret keys and import them into Postman service collections. See a guide here. You can use the secret keys to generate a token, as demonstrated below.

Get access token from Nordigen

Copy the access token from the API response and add it to the environment variables.

Get access token from Nordigen

The access token is good for a few hours and can be renewed if necessary. With the access token, you can then use the other endpoints from Nordigen to access your bank information, allowing you to make full use of the API.

Step 2: Get a list of banks in your country

Before you can access banking data, you have to call an API endpoint, which gives you a list of banks in that country, and you need to choose which bank you want to access.

call an API endpoint

Aside from a list of bank entities, the API response includes information on how many days back you can access transaction records. For example, ING, which I will continue to use in this guide, offers transactions for two years. Some banks might decide to offer less historical data. You’ll also see a logo for the bank, so you can easily identify it.

Step 3: Create an end-user agreement

Using the bank ID from the previous response, you can choose to create an end-user agreement if desired. In Postman, make sure to list the scopes for which you want to get permission. In this example, account balance access, account details, and transactions are all used.

This will allow Nordigen (or any other third party you use) to access your banking information for the next three months. By default, access is granted for three months and includes historical data. You will see this agreement both at the bank site and within your implementation. If you wish to grant access for longer than three months, you must explicitly change the values.

Create an end-user agreement

You will need the generated ID value for the next request.

Step 4: Build a redirect link

The next step is to build a redirect link. Normally, this link should contain unique identifiers such as the chosen bank and a way to identify the client ID. When the authentication process is done, the person is sent to the link you put in the redirect field. If this were an app and not just you experimenting in Postman, this would work like your “landing screen.” And once the customer has returned to your app, the app would have access to the customer’s accounts.

For the purpose of the guide, go ahead and just type

Build a redirect link

Step 5: Sign the agreement

The API response from the previous call will contain a link. When you click it, it will take you to an agreement page. You will be asked whether you agree to share the various types of bank data information you mentioned in the Postman scope variables, and you will clearly see how far back in time the API can grab information.

Postman scope variables

If you use your own account while following this demo, you will be taken through your usual bank authentication process. If you have multiple accounts, you can allow access to all, one, or several of them.

API response

Following the agreement, you’ll be taken back to the redirect link you defined in Step 4, which in the case of this demo is simply Google. In a real-life scenario, a customer would normally be taken to the landing page of the app.

Step 6: Check whether the account was linked successfully

Just in case something went wrong in the previous step, you can check whether the service was successfully linked to the account through the Nordigen API. Go back to Postman and copy the requisition ID from the previous call. You can use it to call the requisition endpoint and check the status.

Nordigen API

If the operation is successful, the status will display LN, which stands for “linked.”

Step 7: Get information about the accounts

The requisition response from Step 5 will also contain the account ID to which the API was granted access. Using it, you can get more details about the account using the endpoint that exposes the details of that particular account.

API was granted access

As a response to this call, you will receive more information about the account, such as the IBAN number, when it was first created, and when it was last accessed.

Nordigen also has an endpoint for account details. This endpoint gives access to information about the account’s currency, the owner’s name, and the type of account.

access bank data with an API

You can also see the balance of the account using a dedicated endpoint.

dedicated endpoint

Finally, you can use the Nordigen API to access the transaction history for that particular account. You can set the date parameters so that the response only shows transactions that happened within a certain time frame.

how to access bank data with an API

Step 8: [Optional] Rescind agreement

End users have control over their bank data, which means they can change their minds at any time. This is why Nordigen also provides a way to remove access to the accounts. To do this, you have an API endpoint that exposes all requisitions. By calling it, you can find the ID of the requisition you want to be deleted and then use a dedicated endpoint to delete the requisition using that ID.

rescind agreement

After you cancel the agreement, the API won’t be able to access your bank information anymore.

If you are curious about which banks offer open access, you can use this tracker. You’ll be able to see whether the bank has set up a developer portal and what authority you need to register with to have access to that portal as a developer. As an alternative, you can use the solution from this demo and choose an API aggregator instead. You’ll be able to see a list of integrations for each bank, so you can easily choose a service that works with the ones you are interested in.

We’re supporting your race at your own pace. Choose yours!

The Open Banking Revolution

The Open Banking Revolution

In 2007, Valentin Dominte was in high school, and he certainly did not follow news about how bureaucrats in the European Union were voting. Unbeknown to him, a quiet financial revolution started in Brussels that year, one that would later be significant for his software developer career, for fintechs all over the continent, and for every EU citizen’s money: open banking.

In simple terms, open banking is a way for people to take back control of their financial information. Like Valentin, you might have two or three bank accounts, each with its own app and its particularities for making transfers, checking the balance, or granting a loan. If you wanted to have an accurate overview of your finances, you’d need to log in to each of those bank apps, extract the information and do all the calculations yourself. Open banking breaks down the walls between all of these different apps, making it possible for apps to pull information from the accounts you choose and give you real-time information about your finances that is gathered from all of them.

Technology is at the heart of open banking. At Levi9, Valentin Dominte is one of our most experienced developers working in open banking since 2018, and we’ve asked him to give us his insights into this topic.

“The official definition of open banking is the process of enabling third-party payment service and financial service providers to access consumer banking information such as transactions and payment history through APIs”, says Valentin. Some key expressions he highlights are “third-party payment,” “consumer banking information,” and “APIs”.

consumer banking information

A third party is a service that aggregates that data“, explains Valentin. “It can be an application from one of your banks, but it can also be completely independent, and you can have different third-party providers for different use cases.” The main benefit for the consumer is that they can get information in a way that is easier to use.

Some of the consumer banking information that can be accessed through open banking includes the account holder’s name, the account type (current, savings, etc.), and information about transactions (amounts, merchants, etc.).

APIs are at the heart of open banking, serving as a bridge between multiple financial services. Through APIs, different systems can talk to each other in a standardized way, meaning that developers can use them to build new features or services on top of existing systems. One important feature of APIs, especially in open banking, is that information is shared in a standardized and secure manner.

2. The EU regulations on open banking

Perhaps Valetin did not pay attention to EU open banking regulations in high school, but looking back he says that the concept of open banking in Europe is tightly linked with those regulations known as PSD (Payments Services Directives).

The EU regulations on open banking

The first PSD was released in 2007, with the EU Commission seeking to stimulate competition in the financial industry, enhance the quality of services provided, and protect the end user.

A second version of the PSD was released in 2015, introducing the concept of consumer protection against bank or third-party providers. “The focus now was on the end-user experience and privacy.” Two main concepts were enforced by this PSD2 directive: the first — strong customer authentication. “Basically that means as a bank you shouldn’t allow people to connect to your API without multi-factor authentication, let’s say. And the end user should have the same way of authenticating directly to the bank or through a third party. There should be no difference.” A second concept was related to the fact that third parties should connect to banks in a standardized manner. Third parties are also obliged to register with an authority, adding another level of security.

Valentin says he is now keeping an eye on discussions related to a third directive. While following EU legislation might not be typical everyday work for a developer, Valentin builds a strong case for remaining one step ahead and analyzing the impact of legislation on technology.

EU FinTech cheatsheet

3. How screen scraping became obsolete

To prove this point, Valentin reminisces about one of his first projects in open banking. Before APIs became the norm and before strict European regulations, developers were still looking for ways to let users access their financial data in a more friendly manner. “Because developers are creative and can find workarounds, there is an alternative to APIs: screen scraping”.

Screen scraping imitates what a person does on a portal, doing everything automatically that a person can do by hand. “It meant impersonating the client in the bank portal to extract data or perform action.” Screen scraping solves the issue of missing APIs, but it introduces several other problems”.

“With screen scraping, the third-party provider controls how the consumer’s credentials are stored and secured,” warns Valentin. Moreover, the clients don’t get to choose what information they share but rather have to give full access to the third-party provider. On top of that, screen scraping cannot get around multifactor authentication and could trigger a possible violation of terms and conditions. Developers avoid screen scrapers not just because of security concerns but also because “this kind of integration is quite fragile.” What if the UI of the internet banking system changes for some banks? The third-party has to adjust to those changes each time.

Coupled with EU rules, the technical setbacks were the main reason that screen scraping became an obsolete practice.

4. How open banking breeds innovation

Open banking is a breeding ground for new ideas, and it encourages innovation by chipping away at large bank monopolies. “Third parties can provide a better user experience and steal the show, which should result in lower costs and, hopefully, a better experience for the end user,” says Valentin Dominte.

Saving time for customers

One way that open banking is different is by making it easier for customers to get loans. “For one of Levi9’s customers, we developed a system that saved the bank and its clients a significant amount of time. When applying for credit, clients had two options: one was to manually upload proof of their financial situation, such as salary slips, bank statements, rent agreements, or mortgage contracts. The second one was to log into the bank account, and choose which transactions represent income or housing costs.”

One immediate result was an improved customer experience. “The customers didn’t need to look for salary slips or dig around for their mortgage contract.” At first, about 40% of customers were unsure about sharing their information automatically with the bank. However, over the course of three years, the number of customers using the faster way to log in to the bank increased by a factor of ten.

Instant credit limit

In a second open banking Levi9 project, Valentin and his team replaced cumbersome manual steps and questionnaire filling with instant credit limit calculation. “We had the old system and the new, automated system run side by side. When clients applied for credit, they were randomly assigned to one of the two systems. Some were going the old road of filling out a questionnaire, providing proof of income and expenses, and getting their answers manually assessed by a bank employee. But other customers had a much more straightforward experience, thanks to the Levi9 project: they simply logged into their bank account, their transactions were automatically analyzed, and they were able to receive their credit limit on the spot.

open banking

With standardized communication between services through APIs and clear regulations, open banking is the perfect playground for technological innovation.

We’re supporting your race at your own pace. Choose yours!

Make your job harder and 10 other ways to adopt a total ownership mindset

Make your job harder and 10 other ways to adopt a total ownership mindset

When Codrin Băleanu was a junior software developer he used to print out his code on paper. He would select a particular intricate piece of code, send it to the printer, take the papers with him, and read them quietly. He would read until he saw the workflow in front of his eyes, until he could visualize the data flowing as smooth as rivers.

Now an Engineering Lead at Levi9, Codrin describes himself as simply a person who gets paid to do what he likes. And he credits most of his career advancement to that attitude that made him read code on paper until he understood it completely: Total Ownership Mindset.

Own the project

“Total ownership” is a concept that gets thrown around a lot during agile meetings. It might sound a bit intimidating, as it sounds like people are expected to do much more than their fair share and to place work, the customer, or the project above everything else, including personal life. But Codrin says the concept is completely misunderstood.

I think of it like a car I just bought”, says Codrin. “It is mine, I take care of it. I treat it with care, I don’t want it to get scratched, I don’t want to smash it into walls.” A car owner might seek to always improve his car, buying accessories, equipping it with new gadgets, and finding ways to make it run better. “In the same manner, if I own my work — be it a customer, a product, a task — I take care of it. I want it to work better, faster, and to be more interesting.

In other words, total ownership does not mean that your work never stops, but rather that you treat it as if it’s your job to make it better.

Here are 10 pieces of advice from Codrin about how to approach and boost your ownership mindset.

1. The customer business is your business

The first rule of the ownership mindset is to understand the business of your customer and understand how that business creates money, part of which will end up in your pocket. If your customer has an issue, you’ll be able to move mountains and do anything that needs to be done to solve that problem. You might end up solving problems that are not part of your expertise or technology, but that will help you grow. This is the root of total ownership.

Part of owning a project means understanding that you and the customer are fighting for the same goal. “Listen to the customer when he talks about business”, advises Codrin. “Your mind might be tempted to wonder, but if you understand the business, you’ll be able to open conversations, reframe your proposals from a business point of view and get your point of view across. “

The customer business is your business

Own your code

When you hear “be the owner of the code” you might be tempted to think “of course I am, my code is my baby”. But that’s the opposite of what it means! If your code is your “child” and you get defensive about it being cut, changed, transformed, you are harming the product and business. Ownership means always looking for ways to make it better, at the cost of your own ego sometimes.

2. Pick up the trash

When you walk on the street and see a piece of trash, you will probably take it and throw it in the bin. You can do the same in a project: if there’s a part of work that nobody wants to touch — a procedure, a database — own it. Make it your goal to fix it, repair it, solve it.

Refactoring is part of the same mentality of picking up the trash. For example, if you have a 2000-line Javascript program, don’t be the one that adds another 100. Refactor. Clean up after yourself, don’t postpone this for a future date, because you’ll never get to it.

Refactoring might not be part of the job description or existing inside a story point, so you have to convince the customer that the process is essential. However, try and explain it not from your point of view (“this code is messy”), but from the point of view of the customer. Focus on the value that refactoring will bring, such as: the code working faster, it’s easier to extend or it’s easier to maintain and repair if there are any bugs found. No product manager or architect or customer would refuse the cost. The condition is to bring value.

“Here is my rule”, clarifies Codrin. “If I repeat a line of code a second time, I consider a “yellow” warning. If I have to repeat the same line of code for a third time, I stop and I refactor. I never broke that rule.”

Pick up the trash

3. Wreck something

Once you have an ownership mentality, you will understand accountability. Once more, the concept of accountability sounds scary, because it’s often associated with blaming. Codrin Băleanu sees this differently: “accountability is seeing the bigger picture and asking yourself: Is there something that could be broken if I change this one line of code? Don’t be afraid of failure. Unless you experience it, you’ll never be a good engineer. Wreck something.

After a bit, this attitude gives you more time to innovate, learn or research. And this — as you’ll come to see — is the only way forward.

Wreck something

Own your time

One advice of Codrin for those who want to adopt a total ownership mindset might be summarized as “Don’t be the British Empire!” Sounds easy enough, right? But here is what it means.

“When the British Empire was at its peak, one of the reasons for its success was its ability to take people who were completely unprepared, place them in a factory and have them produce luxury goods, without any training. They had reduced manufacturing to such a degree, that any person was expandable, a cog in the mechanism.” While an admirer of British Empire history, Codrin warns that “If you repeat everything ad infinitum and do everything the same for years and years, you become expendable. The industry will disappear.”

A developer will never feel motivated and engaged in a British-Empire-like process. Simply repeating other bits of code does not leave you content. Being a developer means having the space to be creative and innovative and that also means pushing against being busy all the time. Developers are creative beings.

4. If you are 100% busy, you have no time to think

“If at this moment, you already know what you’ll do in the next 3 or 6 months, then that’s a problem. This is Agile done badly”, says Codrin. Cramming the schedule with tight-fit plans leaves no space for innovation. You cannot bring anything creative into something that has been planned for the next 6 months.

When we are blocked by work, we don’t have time to think. Always push against this. And you do this by continuously improving processes, so they are better and allow you time.”

If you are 100% busy, you have no time to think

5. Innovate. Innovate. Innovate

Monoliths will always fall, just like all the previous monoliths fell when Netflix appeared. In old companies, processes are what they are, people are working and business is going just fine. But all the while, someone from outside is looking at those processes, analyzing them, and seeing spots that can be done better. This is why process innovation is key to staying relevant.

Innovate. Innovate. Innovate

6. Change your way to work

Things tend to quickly get into a routine, but routine is the death of innovation and creativity. You always need to change something — sometimes as simple as changing your way to work. Another example is how you approach a story point, change a technology or change your entire playing field. In time, this will help you to not be scared by anything new, because change will be ingrained in you. You will stay relevant to the market.

Change your way to work

7. Be lazy

One of Codrin’s favorite advice to young developers is to “be lazy”. By that, he means to be very critical with the time they designate for writing code. “Sitting in front of a computer for 8 hours does not make you a software developer.” You need to always have the mindset of “what else can I do? ”. Or, on the contrary, the work might be boring —then find a way to make it interesting. “For example, if you just type data, write a script that automates the process. Make the machine work for you. Be lazy.

Be lazy

Own your progress

As a junior, Codrin used to look for the hardest, most scary thing to do. “I was scared by many things: Linux, databases, VIM editors, the cloud. As I felt overwhelmed by the new, the only solution to that was to learn. I would use a book, a tutorial, or a video.”

If your job is too easy, make it harder.” This attitude sums up Codrin’s approach to how the ownership mindset, together with continuous innovation ties up to professional progress.

8. Identify the people from your future

In the end, this ownership attitude is something that benefits not just the customer and the company, but the developer himself. “Recognized seniority comes with hard work and involvement. Seniority is something you gain. It is not given to you”, he says.

As a practical advice on how to push yourself on the road to a higher rank, Codrin says to look for the people coming from the future: your future. Identify the people you want to be like 5-10 years from now, as they represent your future. Learn from them to increase your chances to end up like them.

Identify the people from your future

9. Own your job to own your purpose

The road to seniority is peppered with “why”. “Why is this needed? Why does the customer want this? Why do we do things a certain way?” Having the answers to why gives you not only ownership, but a sense of professional purpose. When projects are too complex and opaque to understand, Codrin encourages team members to look for the right person to ask questions, until they gain a deeper understanding of its purpose. “If you don’t understand why you do something, and what is the purpose to which you are contributing, you’ll never like what you do.”

Own your job to own your purpose

10. In a changing world, your mindset is the constant

IT is an industry of permanent changes. One company rises, and three others fall. As systems get more and more complex, ownership gets distributed and it gets more and more difficult to understand who is in charge of what. The only way to navigate the continuously transforming landscape is to have this one constant mindset: total ownership.

In a changing world, your mindset is the constant

We’re supporting your race at your own pace. Choose yours!

How to organize a hackathon to awe your customers and boost team morale

How to organize a hackathon to awe your customers and boost team morale

Hackathons can be fun and engaging activities for your team, as well as incredibly useful for the business of your customers. However, organizing a hackathon is not so straightforward. Organize it at the wrong moment, in the wrong way, or for the wrong reasons and you might find yourself with a demotivated team and no valuable business results.

We compiled this easy hackathon organizing guide, taking you step by step through the decisions you face and the best strategy for making sure that both your team and the customer will have a smile on their faces at the end of it all. This guide is based on some of our recent customer hackathon experiences and it includes real-life examples.

Start with the why

“Hackathon” is a buzzword. It sounds good on a company website, it looks impressive on a customer report. But this is not reason enough to have one.

Hackathons are a great way to boost morale, engage the team in an interesting initiative and let the creativity flow. From the point of view of the customer, this activity might encourage creative development and bring value-adding features to its product or service. It’s a win-win!

Beware, though. If you just do it for the sake of the team, then your customers might not be on board and might not use any of the proof of concepts you showed them. This can prove extremely demotivating for the team and they might not be so excited about another hackathon the second time around. On the contrary, if it is only your customer who pushes for the hackathon, but the team is not enthusiastic, then organizing it will just create added pressure and stress.

How to organize a hackathon

Earlier this year, we organized a hackathon for a world-renowned transportation company, right after the first release of the driver app about which you can read more here. It was our customer who proposed that we organize it, with the dual purpose of relaxing the team, after a stressful period and encouraging creative development.

Time it correctly

Great! You have decided that a hackathon is just what you and your customer need… so when should you schedule it? Let’s start with when NOT to schedule it: before a release! As a rule of thumb, it’s best to avoid all the stressful periods during a project timeline. Otherwise, topping tight deadlines and delivery pressure with the obligation to think creatively for two days simply adds stress and is completely unproductive.

You should schedule the hackathon at a time when the team is not in “survival mode”, putting out fires left and right. It may be after a big release, or it may be during a period when you anticipate an ease in the issues that arise. Check the calendars, to make sure nobody is on holiday during that period. Make sure to agree on the timeline with the customer.

You should also leave a significant period of time to pass between the announcement and when the event actually takes place: aim for at least 3-4 months. This in-between time allows new ideas to rise to the surface, issues to brew, and new features to pop up in discussions between teammates or with the customer.

What about the duration? 1 or 2 days is the norm. Peak work and creativity cannot be sustained for long periods of time. Alternatively, if you just allocate a couple of hours for this fun activity, it’s not enough to come up with a useful result.

How to organize a hackathon

Collect ideas

If the hackathon is not very rushed, then your team has plenty of time to come up with ideas about what they want to do: maybe new features? A new UI? Implement a new technology? Don’t say no to anything just yet. You’ll do that at a later stage.

Both seniors and juniors can propose ideas. If you have a junior team that has never gone through a hackathon before, they might be uncomfortable proposing new and daring ideas. Don’t force them. The first hackathon will be a learning and calibrating experience, teaching them what to expect on similar occasions, what is feasible, how high can they dream, and how limited they are in implementation. The more senior members can set a good example and come up with ideas to be later presented to the whole team.

The customer might also have some ideas of their own. Maybe they’d like your team to tinkle with a new feature or to creatively solve an issue they noticed.

At this stage, all ideas are welcomed.

To keep track of them, you can do a brainstorming meeting or keep an open list.

Select the ideas

Take a look at that idea list. You will not be able to do all of them. But how to select what to work on?

First, use senior members to refine the list. As they have better exposure, they know what can be done. They have interacted with several frameworks and technologies and they already know if ideas such as a watch app, machine learning, or Siri are something that makes sense to be implemented for that particular project. They should also be more aware of the business value of each idea, keeping in mind that the end result is to add value to the customer’s business or service.

Second, validate the list with the product owner: does the customer have something they really want the team to be working on? Is there something they feel would be a complete waste of time? Third, take a team vote: what do they want to work on?

During one of our hackathons, we let our team members vote and decide who wants to do what. They were over-the-hills excited! Our final work order was a list of 2-3 topics per platform and everyone got their pick. No member of the team was forced to work on a topic they did not want.

How to organize a hackathon

Who pays for the hackathon?

Your team will work for several days straight on these ideas, so make sure you clearly know who is financially supporting this. Customer hackathons are usually part of the project management timeline and the project management budget. However, make sure this is very clear for both sides. As the hackathon might cost the customer some money, you might find yourself in the position of having to “sell” this idea to your customer. In this case, your clear “whys” from the first step will be very helpful.

How to organize a hackathon

The day of the hackathon

If your list was correctly put together, then everyone will eagerly await the hackathon. According to the list and the team members’ choice, they will normally be split into groups of two or one. A hackathon day is pretty much like an ordinary day, without all the meetings. Just make sure there’s plenty of food and coffee to go around.

End with a demo

This here is one of the hottest tips we can give you about organizing a hackathon: demo, demo, demo. End this intense, fun effort with a presentation. Each topic on your work list should have resulted in a tangible result that could be presented to the whole team and the customer. Have each group and/or person present his or her ideas in an engaging, visual manner for maximum impact on the customer: from new widgets to Siri voice activation — ideas only come to life if they are properly packaged.

Demos are more than simply impressing a customer. They are great tools for stepping into the shoes of the customer and understanding its needs and the business value of that particular feature or idea. More often than not, developers do not think about these criteria when they are working on something or making project decisions. Presenting something to a customer puts things in perspective and allows for a better understanding of the business expectations, needs and issues.

And one more thing: if the team is aware that they will be expected to deliver a demo at the end of the two days, they will go the extra mile.

Sometimes, the customer will ask you to do another demo, at a higher level. The more people in the audience, the better it is for the team.

Such was the case of our most recent hackathon. It ended with not one demo, but with several demos. As the first demo, inside the group, went very well, the Levi9 team was asked to make a demo at KY level.

How to organize a hackathon

How to measure success

How do you know if your hackathon was successful? Both parties of the hackathon should be quite pleased. Teams should feel more content with their work, after experiencing the immediate impact and reaction they got from the customer during and after the demo. The customers should be enthusiastic about some of the features that were presented and ask for implementation. Keep in mind that even if you fell in love with one of the new features, the customer still might not choose it: the business value will prevail in the final decision.

More often than not, you will know that your effort was successful once you hear this question from both the members of your team and the customer: “So when will we have another hackathon?”

How to organize a hackathon

We’re supporting your race at your own pace. Choose yours!

Bringing the tachograph’s simplicity back to drivers’ fingers with a transportation app

Bringing the tachograph’s simplicity back to drivers’ fingers with a transportation app

From the tachograph for trains in 1844 to fleet management solutions, the transportation industry was among the first to adopt modern safety measures for its beneficiaries. Our customer is definitely there to drive the shift, not only from a safety point of view but also from the technological point of view.

They approached us with the project to develop a tailored drivers’ app, with a friendly UX that gives the driver the features of a modern tachograph at the tips of the finger, for both users of Android and iOS.

Their old solution was a complex tool that suited everyone’s needs — administrative staff and drivers would access it alike. As such, a driver would stumble upon needless and distracting information. What they needed was a straightforward app, flexible enough to respond to drivers’ varied levels of tech-savviness and device types.

The Levi9 Approach

Levi9 saw the potential in this project and drew inspiration from the origin of all these safety systems, the tachograph. This instrument’s simplicity revolutionized the safety of transportation and inspired lawmakers to integrate it as a mandatory component, to prevent accidents. According to a large-scale international survey cited by the European Commission’s Road Safety Thematic Report – Fatigue ”between 20 and 25% of the car drivers indicated that, during the last month, they had driven at least once while they were so sleepy that they had trouble keeping their eyes open”. Components such as these ensure the long-time safety of drivers and the general public.

Tiberiu Grădinariu, who is a driver himself, took the wheel for this project. “I usually drive with my children in the car, so I try to be as careful as possible. However, my instinct is to drive fast and hurry, so my driving style is a mixture. Carefully in a hurry.

Tibi’s driving style reflected in the way he approached this project: a heightened attention for safety in the transportation app, as well as a drive to solve problems fast and in the most efficient way. This is what led him to a brave solution.

The multiplatform solution

Levi9 proposed a cross platform approach that optimizes the resources required to maintain the app long-term, both for Android and iOS. Before we tell you what we chose, here are the options we considered.

Levi9 The multiplatform solution

We chose the “new kid on the block”: the Kotlin Multiplatform Mobile. While quite fresh on the market, it brings advantages such as code sharing platform, access to all platform APIs, sharing of what makes sense, and a single logic codebase. When you add the simple maintenance and the ability to revert to native, you get a clean architecture and the ability to share as much common code as possible. According to KMM, over 50% of the participants in a survey share the following between platforms: networking, data serialization, data storage, internal utilities, algorithms and computations, data synchronization logic, and state management. The key benefit of KMM is having consistent logic between iOS and Android apps.

All good developers know that continuous learning and adapting is an important part of the job and this project helped us embrace a new technology that uses Kotlin and Swift. The iOS developers appreciated the challenges of becoming comfortable with Kotlin and Gradle and the KMM paradigm for accessing platform APIs and the Ktor and Koin libraries.

Most of the encountered technical challenges were on iOS and consisted of linking the share code library with the iOS project, Kotlin/Native concurrency, accessing local files in share code, debugging, and Kotlin concepts not available in Swift.

Result: A Cross-Platform Transportation App

Levi9 cross platform transportation app

Levi9 delivered a cross-platform app that has a high performance, a beautiful UX, and great functionality for the end user. The approach of the modern tachograph brings new functionality such as the day’s driving performance, distance and average speed, remaining driving time and when to stop driving, rest times, vehicle information such as fuel, reductant, and oil levels, the ability to create defect reports, and more.

To be continued: A hackathon for brave ideas

After the pilot release of the app, the team organized a hackathon. The context was suited for more experimentation and more courageous ideas.

Tiberiu Gradinariu Mobile Tech Lead Levi9

Thanks to our client’s enthusiasm and openness to new ideas, the transportation app will soon have some features developed during the hackathon.

Or as ideas for a second hackathon.

If you are curious how you can organize a hackathon for a client, with spectacular results, stay tuned — we’ll soon publish a guide based on our experience.

We’re supporting your race at your own pace. Choose yours!

10 ways Java is getting better

10 ways Java is getting better

You might think of Java as outdated. It is, indeed, 27 years old, which is the equivalent of one century old in “IT years”. Its younger, fresher, hipper siblings — such as Scala, Kotlin, Closure, or Groovy — might be more dynamic and easier to use. But don’t discard Java just yet!

Andrei Micu, Senior Scala Developer at Levi9, has noticed that Java is evolving and using its younger siblings for inspiration and improvement. Andrei talks here about two projects that intend to evolve the JDK, with their associated JEPs (JDK Enhancement Proposals): Project Amber and Project Valhalla.

Let’s see some instances where Java is a bit behind its siblings and how syntax sugar and standardization simplifies and makes the code more efficient.

1. Using records like in Scala or Kotlin

POJOS (Plain Old Java Objects) make data encapsulation a bit difficult. They needed a lot of writing but other languages such as Scala and Kotlin came up with features that standardized and simplified the writing. The same output could be achieved with a case class for Scala or data class for Kotlin. And this is what Project Amber tries to change.

Project Amber started with JDK15 and did some syntax sugar by adding the concept of records that brought in the same feature from Scala or Kotlin.

2. Better Pattern Matching

Another interesting change brought on by Project Amber is pattern matching. Old Java is not very flexible with switches, but Scala is way ahead. For example, in the second Scala example below — you can switch in the same object for lists, for components of lists and you can extract from them.

3. Pattern Matching for instanceof

pattern matching for classes for instanceof

The concept of pattern matching for instanceof was added in the first release of JDK14. Previously, when you checked if an object was an instance of a class, an extra line was necessary for the class to be cast to its checked type. But since JDK14 you can simply write the name of the variable after the class and you don’t need that extra line. This is a small, neat improvement.

4. Pattern Matching for classes in switch

Pattern matching classes in switch

JDK17 brought on pattern matching in a switch. If you want to check an object for its class type, now it’s possible in switch statements too.

5. Increased Expressiveness

Increased Expressiveness JDK18

JDK18 came along with increased expressiveness. Not only can you now check classes in switch statements, but you can also check for particular values and classes in the same switch. Guards and nulls can also be added.

6. Introducing Record Pattern (Preview)

Introducing Record Pattern (Preview)

In the future, we will also have record patterns. Previously, if we got a record at this point you had to write several lines of code to access its members, but now you can do an instanceof point with the name of the internal fields and it’s going to extract them for convenient use, afterwards.

In switches you can do the same for cases
. This sort of expressiveness is going to deconstruct that point, which is something pretty new in Java, so that’s why it probably takes more time to cook.

7. Classes for the Basic Primitives (Preview)

We are now in the territory of primitive types. This is the focus of Project Valhalla. In Java, primitive types are enough to model almost anything. But anything that’s not on the primitive type list is derived from Object, and this is quite an issue. Scala and Kotlin have a different approach to this. Both languages use just one class and the compiler does the improvements for you.

However, Java does not have a class for primitive types, making it difficult to work with them seamlessly.

But this may change and primitive types might get a class. This class will be marked with a `Q` prefix in the JVM internals to signal it’s a primitive type.

8. Declaring Objects that don’t have identity (Preview)

Java wants to offer the possibility of making objects as primitive types. The first step is to give the possibility of declaring objects that don’t have identity.

But what does “having identity” mean? Long story short, when an object has identity, two instances of the same value are different. This is not the case for primitive types and this is why we say that primitive types do not have identity.

Furthermore, objects that don’t have identity are stored in the stack, while objects with identity are stored in the heap.

Value objects (or classes) don’t have identity. Let’s look at how they are declared.

Declaring Objects that don’t have identity

While the outline of the definition is still work in progress, there are some definition rules that have been already established. One is that the class should be implicitly final. The fields of the class are implicitly final and no constructor can call super. Also, value objects can have reference cycles, which means they can have references to another object of the same type.

What do we get in return? The objects are put on the stack and the equality will hold between two references to the same value..

9. Primitive Classes (Preview)

Primitive Classes

A primitive class is like a value class with the additional restriction that no field can be of the declaring class, so we don’t have cycles like we have for value objects.

What do we get in return? We get almost the same performance as primitive-types. They do not allow nulls. Some might say this is not a benefit, but those who had null-pointer exceptions might think otherwise.

10. Reified Generics (maybe)

While not yet official, Project Valhalla does drop a hint about tackling reified generics, in which they may also take a hint from Scala and Kotlin.

What is the issue in Java right now?
Well, when generic type objects are passed around, the information about the inner types is lost at runtime. This is called `type erasure` and it is a drawback of the JVM.

There is a workaround in Java, that requires you to send the class as a separate parameter. Definitely not elegant! But Kotlin and Scala have a better way of dealing with this.

Kotlin deals with this with a neat feature called “reified generics”. If you declare the function inline — meaning that the function content is copied at the call site — then you can have access to the inner types of the generics. When you write reified, it adds in the back an extra parameter which is the class of the generic type parameter. This way, you don’t have to write extra parameters.

Scala works around this in a different way. It makes use of implicit parameters to leverage the compiler’s ability to pass the class information.

Reified Generics

What do you think?

Do these Java improvements seem significant to you? Andrei Micu stresses that standardization is not easy. Java’s little siblings might be a bit lighter on standardization, which allows them to develop features faster. Java, however, is so widely used, that keeping backward and forward compatibility is crucial. Sometimes, this means holding back on innovation, in order to keep all of its users happy.

Let’s keep in mind Java is running a marathon, says Andrei Micu. Its younger sibling languages, with more spring in their step, will also benefit from JVM improvements and new features. It’s a win-win.

We’re supporting your race at your own pace. Choose yours!

Levi9 IT Services is upgrading the infrastructure for bicycles

Levi9 IT Services is upgrading the infrastructure for bicycles of two major high schools in Iasi

On World Bicycle Day, we empower kids’ rides! And we have chosen the bicycle as the main means of transport and upgraded the infrastructure of two of the largest high schools in Iasi.

In partnership with the organization Iași Orașul Bicicletelor, Levi9 installs modular bicycle stations at the “Costache Negruzzi” National College and the “Gr. C. Moisil” High School of Informatics, the first two high schools to have entered this modernization program. Starting June 3rd, the students of the two high schools will be able to use the stations that will protect their bicycles during classes.

We like to pedal and move freely! Both my colleagues and I, as well as the parents’ and teachers’ associations of both high schools who have supported us in finding a sustainable solution for the active transportation of their students. Thus, together with our community partner, Iași Orașul Bicicletelor, we developed a project that would exactly meet the urban mobility needs of the high schools involved in the project, ” said Anca Gafițeanu, Delivery Center Director of Levi9 IT Services Iași.

The whole Levi9 community embraced the commitment to sustainability, from global to local and individual levels. Our “One common goal” value is reflected not only in the work environment. Every one of us finds a way to follow the green initiatives according to our personal needs and passions.

In recent years, Levi9 Iași has supported several urban mobility projects necessary for our community. In the next period, other school institutions from Iași will be included in the modernization program.

We support cycling because it helps young people move freely and independently, challenging them to discover Iași. In the meantime, it allows them to observe the beauty and the architecture of the city they live in while choosing the shortest way home or to school,” said Costi Andriescu from Iași Orașul Bicicletelor.

The civic initiative belongs to the Iași Orașul Bicicletelor community, and the implementation was possible due to the investment made by Levi9 IT Services. Every year, the company is involved in transforming students’ social habits so that more and more bicycles are properly parked in the high schools of Iași. We also thank the representatives of two Parents-Teachers Associations of the “Grigore C. Moisil” Theoretical High School of Informatics and “Costache Negruzzi” National College Iași, with whose support the Green Zone for bicycles came to life.

Although our color is blue, we go green. Of course, one step at a time. Hopefully, by putting all the steps together, we will discover how far we’ve got down the sustainability road.

Azure Skies and Smooth Cloud Sailing

Azure Skies and Smooth Cloud Sailing


130 years ago, when our client first started, a cloud may have been a bad omen. It could have meant the impending approach of a storm preventing its ships from reaching their destination. Today? The cloud is what helps our clients make the best decisions, based on hard, consistent, relevant data. One of these vessels shipping BI services through the cloud is called Levi9, with a team helmed by Carmen Girigan, Senior Data Engineer, and Eliza Enache, Medior Data Engineer.

From Data to Information

If a business founded in 1890 is still thriving today, it must mean that it has adapted over and over again. More recently, this meant adopting a cloud-based data solution with the help of Levi9 and other cloud providers. Vroon, an international company with a vast tradition in naval transport, was looking for ways to optimize their data solution, in order to make better decisions. We picked up a legacy on-premises solution and transformed it into a personalized, highly-scalable Single Source of Truth in the cloud.

The main challenge was picking up a complex solution using external data sources, requiring maintenance and development on multiple modules, and offering limited aggregated results. To learn how the business was doing, Vroon business users had to look into multiple sources.

However, data is mostly useless unless it becomes information. But what does this mean?

“You might look at Data as the input that your brain perceives through the senses”, says Eliza Enache, one of the Levi9 “sailors”. “The brain then creates connections between these data points by applying certain logical processes (filtering, algorithms, etc.). The connections between data points (and connections of connections) might be called Information.

Raw data is useful only when it is organized and interpreted in such a way that it helps in decision making, becoming information” completes Carmen Girigan, also a member of the Levi9 team.

After working with Levi9, Vroon users interrogate one service and get the answers they need.

Eliza Enache Carmen Girigan

From SQL Server to Azure cloud solutions

The Vroon business had been getting information using on-site Microsoft technologies, relying heavily on SQL Servers. Data was coming in from various siloed sources such as accounting, procurement, invoicing, customers, vessel activities, incident reporting, and many others.

Data were extracted, transformed, and loaded into the warehouse, via Microsoft SQL Server Integration Services, as well as custom applications and SQL Server stored procedures. On top of the data marts, there were 5 analytical databases with 11 multidimensional OLAP cubes, developed with Microsoft SQL Server Analysis Services. These cubes would then be accessed via the reporting tool, for reports, dashboards, and analysis.

The cloud conversion started in 2020. The new solution was based on Microsoft Azure technologies, in order to streamline extract-load-transform (ELT) processes and maintenance, by using a gradual approach that would allow evaluation and constant adjustment to new situations, as well as high availability for the users’ new requirements.

Now, data is extracted from its sources via Azure Data Factory and loaded into an Azure Data Lake container. It then goes through a transformation process using Azure Databricks or Azure Synapse Analytics. Afterward, it is loaded into an Azure Synapse Analytics data warehouse, based on which a tabular model is developed with Visual Studio and later deployed to Azure Analysis Services. Power BI is used for data presentation and user self-service report creation.

Levi9 Cloud Technology Stack

The result was the transformation into a modern, cloud-based, and highly-scalable solution, that provides information in a unified way: a Single Source of Truth for multiple business areas.

Having a single source of truth is essential for enabling good business decisions. “Everybody sees the same figures, at the same time, and it’s more reliable” Carmen explains. “Long story short, you avoid conflicting ‘versions of truth’ when different users analyze data exported at different points in time”, adds Eliza.

Cloud sailors tip: Clean your data like a ship’s floor

Whenever thinking of a ship, one image inevitably comes to mind: sailors cleaning the deck, almost to the point of obsession. In the 18th century, this was done to keep the crew healthy.

Today’s Levi9 data engineers have a similar attitude towards data: it should be maintained clean, to avoid the risk of infecting the budget, reports, and any meaningful analysis.

In the cloud, resources are much easier to maintain and scale — but we do have to pay attention to costs. There is no point in having duplicated or unutilized data, or transformations, in the cloud”, explains Carmen. Eliza adds that there is much more than that. “During the testing phase, we noticed that redundant or obsolete data led to improper analysis and estimation, and diversion from the ‘main track’ — which in turn led to an increase in time and costs, while also blocking further developments. Some of these redundancies made us go back through the whole logic.”

Those who don’t empathize with nautical experience might compare data cleansing with their wardrobes: “Just think of that closet full of clothes you don’t like to wear, things you’ve kept for years only cluttering your space and making for a hefty baggage” says Eliza.

An ongoing adventure

Part of Vroon’s legacy system is still running and is in the process of being transformed. Eliza explains why sometimes this is a better solution: “We soon realized that converting and rewriting the whole logic, from old technologies into Azure, would take a lot of time – thus stalling any further development initiative. As such, we devised a temporary workaround: part of the data is directly brought to and processed in Azure, while another part is brought from the on-premises data warehouse, and further integrated into the unified model.”

Starting with a pilot

But we won’t do everything at once. First, we’ll do a pilot project”, adds Carmen. Just like sending a sentinel ship before moving the entire fleet, a pilot provides insight into what the job entails.

Migrating from on-premises ETL to Azure ELT

Fair winds and following seas

All in all, the project continues to be a learning opportunity for the team. All the Levi9 Vroon sailors have at least one Azure certification, as the department and company strategy aligned with the work. Carmen Girigan and Eliza Enache stress that they owe much of their learning to the other half of the initial engineering team that had established the foundations on which they continued to build in the cloud.

With the new pilot project, the adventure continues. May the team have fair winds and following seas!

We’re supporting your race at your own pace. Choose yours!

Rebuild this castle - a data migration story

Imagine yourself in a run-down house. Make it a big one. Huge. Some walls are still standing, but barely. Some rooms are just a pile of bricks lying in the middle of the floor. There are secret passageways connecting random chambers, almost like a labyrinth. Did you make the house gigantic in your mind? Go on, make it even bigger. A quirky, dysfunctional, enormous castle.

Here is your challenge: you need to rebuild the castle on another land. In its new version, the building should regain its splendor, while guests and inhabitants alike should be able to find their way seamlessly. Oh, and if you fail, even slightly, even by a fraction, all the inhabitants of the palace lose their livelihood.


This is the magnitude of the challenge that software architect Sebastian Gavril found himself in, when he was approached to migrate a fintech’s core databases to the cloud.

Tinka, a Dutch fintech company, already had some of its services in the cloud, such as microservices, authentication systems, message brokers, CI/CD, and monitoring tools. But the core services, related to the actual financial processes, remained on-premise, part of the so-called legacy system: the virtual machines, databases, the Enterprise Service Bus, and workflows.

The job entailed moving from on-premise into the cloud 30+Tb of data, split in over 30 Oracle database instances across multiple Exadata machines — mini data centers that can be rented out — and 7 environments.

In other words, the core services were still in the “old castle” and needed to be moved. All those discarded bricks, secret passageways — they were actually scattered databases on old servers, connected among each other in a complex network, being used by various applications, microservices, or monitoring tools.

So how does this story unfold?

The heroes

This job needed an architect. A software architect. “Normally, an architect first needs to understand his customers, their needs and desires, constraints and opportunities. Then he translates that into a formal language that can be used by engineers to construct the building his customers desire. And during the construction phase, the architect is always present to make sure the plan is followed and change requests from clients are included with minimal impact on the integrity of the building.”

This is what Sebastian did. He needed to have an ensemble view of the challenge, decide what’s needed, and get the team together. They were: Simona Gavrilescu (Delivery Manager), Ancuta Costan (Test Lead), and Monica Coros (DevOps Engineer), as well as employees of Tinka.

At Levi9, the team is defined together with the customer, with the aim of enhancing collaboration and working together as one team.

The castle

Moving core databases is an extremely complex project. When you move databases, you don’t just move the data, but you need to make sure that all the critical systems will function correctly.

Plus, the old castle — the legacy systems — were responsible for a large part of Tinka’s income. This is true for most companies.

While looking to achieve a positive impact for the clients, one of Levi9’s goals of this project was to have no negative effect on the Tinka customers. Customers were not supposed to notice anything. However, if things did go wrong, the company could have lost or corrupted customer data.

The new kingdom

So where was this new land where the castle was being built? The cloud.

Until recently, “the cloud” actually meant just a couple of key players, such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform. Nowadays, any large, established company starts to develop its own cloud: Oracle Cloud, IBM Cloud, Digital Ocean, Heroku, Alibaba. To attract customers, many offers features such as niche-specific services, better integration between cloud and legacy systems, and migration tools.

The chosen solution was the RDS – Relational Database Services, the database service offered by AWS, mainly because part of Tinka’s infrastructure was already in AWS. After migration, AWS would be the one keeping up with maintenance and updates and many more as part of the package.

The challenge

“It was imperative to understand the complexity, which I saw on two levels”, says Sebastian.

The first level of complexity is that of the database. In this case, the team used a tool provided by the AWS Data Migration Service (DMS) of AWS called the Schema conversion tool. “When you move a database from one place to another, this tool generates a scheme that shows the objects, triggers, procedures, and packages. It also shows which ones can be converted automatically and where manual changes need to be made, through simple, medium, or complex actions. This helps us understand what database-specific effort we need” explains software architect Sebastian Gavril.

The second level of complexity is that of the “landscape”. That means everything that calls on the database. The landscape was very rich. From old, legacy servers – that nobody really understood, to modern microservices, with everything in between: ESB, custom jobs, etc.

It helps us understand where the problems are. What we need to change: connection strings, drivers, maybe even code that used specific functionality in Oracle Enterprise Edition (an edition we wanted to move away from).

The map

So what does moving the castle entail? Well, when planned systematically, things look pretty straightforward: you need to move the bricks and you need to redesign the rooms. In data transfer language this means that you need to take into consideration two aspects: the applications and the databases.

On the applications side, there are some conversions to be made: for example update libraries, change versions, rewrite some code, and make sure the configurations are in order.

The databases also need to be converted, either by using tools such as the schema conversion tool or manually. For some, you can find a procedure. For others, they need to be built from the ground up.

For migrating the actual data, just like renting a moving truck, AWS has a Database Migration Service offer. You can rent a machine, point to the “departure” address, set an “arrival” address, and up you go. It can take minutes, hours, days. Or weeks, in one of our cases.

It all comes crumbling down - Part One

Every project starts with “it’s not much”. “It’s practically done”. Sebastian Gavril knows to cringe at this one. The first migration project was started 1 year before and this was how it was presented to the team. The first go-live attempt was possible about 2 months after these famous phrases, after a lot of tweaking and testing on the initial solution.

And this was just one room.

In such a complex data transfer project, issues keep popping up at every step, sometimes baffling entire teams. “The funniest thing was when we migrated a database and ran a simple smoke test to validate that everything is working. The test failed repeatedly so we spent the whole day trying to fix the issue, only to roll back in the end. The second day we realized that we were simply testing the wrong way. We were probably a bit too nervous.”

Slaying the budget monster

Every castle needs its own monster and this was no exception. As one of the key issues of the project was to lower costs, the budget was the monster that needed slaying and slaying. “Traditionally, software engineers didn’t have to think about costs too much”. Having his own very healthy financial mindset for expenses at home, Sebastian Gavril explains why he took care to chip away costs at every opportunity. “With the rise of cloud providers, engineers are using cloud services more and more and taking most of the decisions in regards to the configuration of those resources. For example, the biggest databases you can order at the click of a button in AWS are ~80 times more expensive than what we chose for one of our medium workloads.”

For example, by keeping an eye on resource metrics for a database, the Levi9 team was able to cut costs by about 75% from initial estimates.

Time to say good bye

Some rooms of the castle simply got left behind and it was for the better.

One of the biggest surprises of the Levi9 team was when it realized it could take some smart decisions. The goal was to stop using our data center, not necessarily to migrate everything.

“We found out that the largest database, a 15 TB behemoth, was actually what we call “highly available”. That means, there are 2 databases instead of 1. The second one is there only if the first one crashes for various reasons. But this was not a business-critical database, so we did not need to have the db up and running 24/7. So we decided we can just drop one of the databases and rely on backups to restore the primary database in case of failure.”

It was only because of the Levi9 team’s culture of focusing on one common goal that enabled it to know the business and the client so well, that they were able to take such surprisingly easy, but important steps.

It all comes crumbling down - Part Two

And then there was the huge data latency problem.

Levi9 practically moved the databases from the premises in Zwolle (Netherlands) to Dublin (Ireland). Some apps were left in Zwolle (Netherlands), to be migrated later, but they were calling on the database in Dublin.

“Data transfer is pretty fast nowadays, so normally this extra distance is not noticed. But due to the way one of the applications was written, the data was going to and from the database much more than needed. It became unusable, as some common operations were now taking tens of seconds to complete”

Because of the architecture of this application, it was like going to the castle for a lamp, then for the bed, then for the cover. The team was on the brink of giving up because it was not able to change the application. In the end, they managed to tweak the infrastructure so that it would allow for much more traffic than usual. It was like sending 3 people to the castle each for one thing only – not ideal, but it worked.

The welcome party

For a while, you will live between castles. You will still have the old one, but also the new one, perfectly in sync. And then?

And then you just need to make up your mind and throw a welcome party!

In data migration, the welcome party is extremely tricky. The inhabitants must not notice any difference, almost like they would be teleported. The service cannot be down for more than 15 minutes. The data about each Tinka client must be precisely the same. And, God forbid, DO NOT, under any circumstances, mess up their bank account. Good luck!

When it was time for the last final move, Sebastian Gavril did what he always does. He planned for it. Sebastian Gavril and his team had a step-by-step, second-by-second to-do list.

How detailed should this plan be? “Just to give you an idea”, says Sebastian Gavril, “we prepared so thoroughly that one of the go-live plans had 66 tasks, each representing an action.”

There are hundreds of small decisions that need to be taken before the key moment: for example at what time do we go live? And there are numerous connections that should be made between databases and services, all with the purpose of making a seamless transition, with as little downtime as possible. Maybe most importantly, what to do not if, rather when things go wrong.

Sebastian Gavril and his team split the very detailed action plan into three: before go-live (checking everything is in order at the new site), during go-live (making sure that each team member knows what to do at each moment) and after go-live (monitoring the new setup and being ready to take action if needed).

The happy ever after

It all ends well, as it should.

For a result-driven, impact-delivering, change-making company like Levi9, failure was not an option at any point. No data was lost. Customers of Tinka might have noticed a two to a three-minute website glitch, but nothing more.

“Having managed to deliver this feat was a great self-accomplishment. It’s also the kind of project that challenges and empowers a senior engineer to grow, because it involves many aspects that are not part of the day-to-day life: coding, database administration, database development, cloud, networking, DevOps, architecture, third-party management, finance. A very good opportunity to develop as a well-versed engineer.”

To move critical systems, to make everything run better, to significantly increase budget savings, all the while protecting the client from any negative consequence — this could all be summarized in what Levi9 wants to achieve: impact.

We’re supporting your race at your own pace. Choose yours!

CodePipeline for Serverless Applications With CloudFormation Templates

CodePipeline for Serverless Applications With CloudFormation Templates

The CI/CD process of an application is crucial to its success and having an easy way to maintain said process can lead to an even healthier lifecycle for the application. Enter infrastructure as code: a reliable method of managing and provisioning pretty much anything you would need in a CI/CD pipeline to get it going, through the use of templates and definitions. Ever needed to keep the configurations of an AWS pipeline somewhere so you don’t need to remember the clicks from the Management Console by heart? Or maybe you wanted to give a working example to a colleague, so they can build their own pipeline. These problems and many more can be solved through infrastructure as code and CloudFormation if we’re talking AWS.

In the following lines, we’ll go through everything you need to create your own pipeline, connect it with other pipelines and maintain them only by running a not that complicated bash script. And by the end, you’ll probably come to realize how awesome infrastructure as code is (no need to thank us).

CodePipeline using Cloudformation

Building a pipeline in AWS’s CodePipeline is pretty simple and conceptually similar to other CI\CD tools out there, but it’s also quite verbose, requiring a considerable amount of code, especially if you want to build your pipeline in an infrastructure-as-code approach.

From a high level there are 5 main types of resources we need in order for us to put together an AWS Codepipeline:

1. An S3 bucket resource — where our code artifact will be stored
2. A CodePipeline resource— this will model what steps and actions our CodePipeline will include and execute
3. IAM Roles — one role that CodePipeline will assume during its execution, in order to create/update/deploy the resources in our codebase. A second role is used by a CodeCommit webhook(see #5).
4. CodeBuild Project(s) — are used by the CodePipeline resource to execute the actual commands we want in our pipeline.
5. An Event Rule — an AWS Event Rule that will act as a webhook triggering the pipeline on each master branch change(this is only required when working with CodeCommit. If you use Github or other supported repo providers there are build-in webhooks)

Now we’ll go over each of the resource types and then we’ll put it all together in a complete CodePipeline.yml definition for a serverless application built on AWS Lambdas using the Serverless Framework(but not limited to these).

S3 bucket definition
You can consider this the code artifact store for our pipeline. This will be referenced and used later on by the pipeline itself to first upload the artifact and then use it to deploy the resources.

The definition is pretty straightforward:

CodePipeline definition

This will be the bulk of our pipeline definition code that will define what are the stages of our pipeline.
The top-level properties of CodePipeline resource definitions are:
1. ArtifactStore — this is where we’ll reference the S3 bucket that we’ve created earlier. We will do this by using AWS’s !Ref intrinsic function
2. RoleArn — this is where we’ll reference the Role that the CodePipeline will assume during its run. We’ll do this by using the !GetAtt intrinsic function
3. Stages — a list of actions that our pipeline will do

A high-level view over our CodePipeline definition will look like this:

Notice that, for the sake of simplicity, we’ve left out the definition of the actual stages, which we’ll cover later on. But as you can see, in our case, for a serverless application, the stages of our pipeline will be:

Source — retrieving the sources from a supported git repo(CodeCommit, GitHub, Bitbucket) or an S3 bucket(this is useful if you’re not working with one of the supported git repos — but this implies having to upload your code to the bucket by your own means). We’ll stick to using a git repo since this is the most common scenario.
Staging — reference to the Deploy-to-Staging CodeBuild project which will contain the actual deploy commands.
Promote to Production manual approval gate — this will prevent an automatic deploy to Production each time the pipeline runs.
Production Deploy to Production — reference to the Deploy-to-Production CodeBuild project which will contain the actual deploy commands.

CodeBuild project

CodeBuild is the equivalent of build runners from other CI\CD tools. In our case, we’ll use their ability to run CLI commands in order to deploy our application. The definition of a CodeBuild project that deploys our serverless + node.js app to the staging environment/stack looks like this:

The most important part of a CodeBuild project is its buildspec.yml file which defines the actual CLI commands that the project will execute. You can see it being referenced in the Source property. The buildspec.yml file(located in the root of the project) looks like this:

Putting all the pieces together here’s what the definition of the stages inside our CodePipeline definition will look like:

IAM Roles

As mentioned earlier, we’ll create 2 IAM roles. One to be used by CodePipeline itself and one used by the CodeCommit webhook that we’ll create in a future step.


An IAM role is just a collection of policies or access rights that a certain resource is allowed to have. In our case, this is the place where we define the access limits of the CodePipeline. This is important because, according to the AWS Well-Architected framework, a resource should be limited to accessing only the resources it’s supposed to access, as per the principle of least privilege. In other words, we should avoid defining IAM policies that give unnecessary privileges, like the one below:

and instead strive to limit the access boundaries through policies that look like this:

There is no silver bullet when creating a CodePipeline IAM role because the policies(access rights) of the role will differ based on the actual resources that you use in your project. There are some policies that will be required regardless of your setup as you can see below:

You can use the above definition as a starting point. Run your pipeline and add more permissions to the role based on the error messages received from AWS. Does your setup contain lamdas? Then probably you should add some lambda permissions. Dynamo db? Then add the necessary dynamo db permission. It’s a bit of a tedious process but it will add to the security of your environment.

CodeCommit webhook(optional)

At this point, our CodePipeline is ready. The only thing missing is its ability to run on every change of the master branch. For this, we need just two things: an Event Rule and a Role. The “webhook” we’re creating is actually an Event rule that listens for CloudWatch events emitted by CodeCommit and triggers our pipeline whenever there’s a change in the master branch(or any branch, for that matter). Fortunately, these are less verbose and look like this:

Deploying the pipeline

At this point, you should have everything ready and we can deploy the pipeline. This is pretty straightforward and can be done by running the following command:

$ aws cloudformation deploy –template-file .yaml –stack-name –capabilities CAPABILITY_NAMED_IAM –region

Note: the — capabilities CAPABILITY_NAMED_IAM is just an acknowledgment that you are aware that the template file will create named IAM roles.

The Serverless Framework catch

If you’re using The Serverless Framework, there are some changes that you have to do in the serverless.yml file. Normally, your serverless.yml file looks something like this:

Notice the profile:default property — this dictates which user/profile will be used by The Serverless Framework. The default profile is usually taken from the .aws/credentials file or environment variables. The default profile is usually an admin user with full privileges.

But once the pipeline tries to deploy the serverless framework stuff, the default profile means nothing, because there are no credentials config files in our pipeline.

So we have to make use of the cfnRole property offered by The Serverless Framework. This property accepts an IAM role ARN as value and uses it when deploying the AWS resources. So we just have to put the ARN of the role we’ve created earlier in the cfnRole property, remove the profile property and we should be set. (This means, that we’ll need to deploy our pipeline template in order to create the role, find its ARN, and update the serverless.yml file)

See below the cnfRole property working along with the profile property by using the serverless-plugin-ifelse. This makes it work on staging/production environments/stacks(when CodePipeline does the deployment) as well as development stacks(when you want to deploy your stack from your development machine).

Multiple CodePipelines using AWS Templates and Nested Stacks

We have seen how we can declare various resources and tie them together into a fully functional, independent pipeline. But what if we want to build a much larger template by combining multiple smaller ones? Or do we want to group similar resources, such as roles, lambda configurations, or policies? AWS’s answer to these questions is simple: Nested Stacks.

Nested Stacks are stack resources part of bigger stacks. They are treated as any other AWS resource, thus helping us avoid reaching the 200 resources limit of a stack. Also, they offer functionalities such as input and output parameters, through which the stacks communicate between themselves. In addition, the use of Nested Stacks is considered a best practice, as it facilitates reusability of common template patterns and scales well with larger architectures (

So how can we make use of these fancy, well-regarded, Nested Stacks? Simple! Treat each of the 3 pipelines as an independent nested stack. Since they don’t contain that many resources (now or shortly, for that matter), segregating them by type doesn’t offer that big of an advantage and thus, the nested stacks won’t even require the use of input and output parameters, since they contain everything they need inside their definition. The 3 nested stacks would reside under a parent one, used as a single point of access whenever changes are made to any of them.

What needs to be done, then? There are 3 steps we need to follow to build our template with the use of nested stacks:

Define a root template

This file will reference all the other templates using the physical path:

This file is a simple collection of AWS::CloudFormation::Stack resources, referencing each template through its physical path. Here, you can also define the input/output parameters mentioned earlier, which are values passed between stacks when they need to communicate, but for us, it’s not the case.

Package the template

Once we’ve finished configuring our Nested Stacks setup, we can start combining them. Currently, Amazon only supports combining templates into larger ones using S3 buckets. The local files are uploaded to an existing bucket (passed as a parameter) and a new file is generated, this time containing not the physical path, but the S3 location URL

Running this command:

$ aws cloudformation package –template-file /root-template-file.yml –s3-bucket bucket-name –output-template-file my-packaged-template.yml

  • — template-file — the root template file, containing the physical paths
  • — output-template — the name and location of the newly generated template, containing the S3 paths
  • -s3-bucket — the name of the bucket used for packaging the files
    will result in something like this:

Note: The S3 bucket used for holding these files needs to already be created, before the execution of the “package” command. Also, one needs to be careful not to include the definition of the deployment bucket inside any of the templates, since this would lead to a circular reference.

Deploy the template

Once the output template is generated, we can safely deploy it. CloudFormation will look for the specified files in the S3 bucket and create/update the root stack and, implicitly, the nested stacks. If the stacks already exist, they are evaluated based on changesets and if any differences are found, CloudFormation updates only the ones that were modified.

The deploy command goes like this:

$ aws cloudformation deploy –template-file /my-packaged-template.yml –stack-name my-stack-name –capabilities CAPABILITY_NAMED_IAM –region my-region

And that’s about it. You should now have 3 different pipelines created by your template. Not the smoothest process, but pretty straightforward, nonetheless. A possible solution could be automating this whole endeavor through a script. In the following section, we will see exactly how we can achieve this.

Automating the package/deploy process

As we’ve seen earlier, there are a couple of steps that we need to do to get our nested stack template packaged and deployed. Unfortunately, we have to go through all of the above processes each time we modify our pipeline.

Besides being cumbersome, doing all of these steps manually is not recommended as it is prone to errors. After all, you’re creating an automated CI/CD pipeline to reduce the amount of work you have to do, not add to it. If you’ve reached this point and you’re asking yourself “do I have to do all of that every time I want to deploy my pipeline?”, then don’t worry because the answer is no. But how can we avoid this hustle and automate the entire process? The solution? Bash scripts to the rescue!

Using a bash script, we can achieve the same result as manually deploying the pipeline(s), without giving ourselves a headache. Take a look below at an example of a simple script that does everything we need:

While this would probably work just fine (assuming the bucket exists and the template is valid), it’s a good idea to follow certain conventions regardless of the programming language you use. Let’s take a look at how we can improve our script a bit:

The above bash script does a couple of things:

Make sure the bucket exists
Because of the way nested stacks work, we need to have an S3 deployment bucket where our templates will be stored to be later used by the root stack in the deployment process. Therefore the first thing we need to do is to ensure that the bucket exists. The head-bucket command (aws s3api head-bucket –bucket $bucketName ) is perfect for this because it determines whether the bucket exists and if we have permission to access it.

Validate the template
The next step is to make sure that the template that we’re going to deploy is valid. To do this we can use the validate-templatecommand (aws cloudformation validate-template — template-body $pathToTemplate) which, if the template is not valid, will return a error message detailing what is wrong with it. Once we confirmed that the template is good, we can move forward and deploy it.

Package the template
The aws cloudformation package — template-file $pathToRootTemplate — output-template $pathToOutputTemplate — s3-bucket $bucketName command returns a copy of the root template, replacing the references to the local template files with the S3 location where the command uploaded the artifacts. So basically, it sets us up for the next step, the actual deployment.

Deploy the pipeline(s)
After all this setup, we can finally deploy our pipeline(s). We do this with the deploycommand (aws cloudformation deploy — template-file $pathToOutputTemplate — stack-name $stackName — capabilities CAPABILITY_NAMED_IAM — region $region), which uses the template that was generated by the package command in the previous step to create our (pipeline) resources. If this step succeeds, the pipeline(s) resources will be created in the specified stack.

And that’s it. We now have a script that does all the heavy lifting for us. All that’s left is to add the script to your package.json’s scripts section and you’re all set.


Quite a ride, wasn’t it? We’ve seen how to write the definition of an AWS pipeline and all its components, we’ve rigged a bunch of them together using CloudFormation and Nested Stacks and finally, we’ve automated the whole process through the use of a bash script. Hopefully, all of this came in handy and helped you avoid too many of those pesky configuration item changes on AWS when building your pipeline (guess what gave us an unusually hefty bill at the end of the month).

If you have any feedback for the article or the code presented in it, please send us an email. Every thought and idea is appreciated.

Thanks for reading and happy coding,

Andrei Arhip
Tudor Ioneasa
Andrei Diaconu, .NET Developer @Levi9

We’re supporting your race at your own pace. Choose yours!