17 - Microservices explained

Well, here I am again setting myself and my blog another mission - writing the easiest to understand blog post on Microservices out there. Why? Because back in 2017 when I first got my proper Software Engineer job, Microservices were really picking up both speed and hype, but not many people were actually able to tell you why.

It’s like now when people try to tell you why React is good. One thing that has always bothered me about our industry is the fact that we are very prone to blindly jump on the hype train about a new technology or architecture without fully understanding it, or even without partially understanding it, and I feel that Microservices kinda fall into this category.

Yeah, everyone and their dog is talking about Microservice architecture, and how cool they are, but do we really now if they’re cool? What if they’re not? What if they are actually really bad? And if they are bad what makes them bad? What makes them good? Let’s try to understand together.

The good old Monolith

Chances are that most of us have worked in a company that had a Monolith. And now with the ever evolving hype around microservices, monoliths are getting such a bad rep, but they do have their place and time when they are a much better choice over microservices. Let’s be a bit of a negative Nancy and talk about the painpoints of a monolithic application first.

Problems with the Monolith

Now, as I said before, there’s a pretty huge chance most of us have worked with a monolith and have seen the pain points it causes. In my opinion, these are the biggest ones:

It can get messy

One of the worst things I have personally experienced working with monoliths that the code can get so messy and spread out between files and folders that sometimes my VSCode and Google Chrome have a mexican standoff about who has more tabs open.

Because the initial intention behind a Monolith is to peform a few sets of functionality, like let’s say it has a user registration system and some CMS, it starts nice and well. You have a folder with your user related functionality, one for the CMS, with sub-folders inside for your services, controllers, repositories, test etc. But as time goes on, the business grows more and we add more and more features, folders, tests, configs for different things, db migrations, that at one point your sidebar with the files structure becomes an absolute nightmare to even look at, let alone maintain.

The amounts of times that in my efforts to trace a couple of method calls I’ve ended up opening up an accordion on the side is more than I’d like to remember. So this is a big drawback of the Monoliths, as they grow, and the team grows it becomes very hard to maintain a tidy code structure. Not to mention debugging code and trying to find out what went wrong and where.

Deployments are slow

As our monolith grows, so does its deployment time. I have worked with monoliths with a full deployment taking well over an hour nad half (talking about deploying to staging and to production). Hotfixing took around 40 minutes, because the entire application had to be deployed. One time we were travelling to a conference and we found out that a CSS bug made the logged in section of our web app completely unusable.

The bug was pretty easy to fix, it was literally just updating a <div> with a class, but that one change took 40 minutes. I think you can agree that 40 minutes for a production hotfix that is just adding class name to an HTML element is a bit ridiculous, but this is what you get most of the times when you have a monolith - your code grows, you add more test that your deployment pipeline goes through, you have checks for migrations, etc - and the more your monolith grows, the more the time for deployment grows too. It’s just how things are.

Single point of failure

It goes without saying that when our entire application is contained inside a monolith, if something goes wrong anywhere in the monolith, then the entire application goes down. And that doesn’t only constrain to something going wrong in the code, but also something going wrong with a deployment, or a server going down. It is mildly infuariating when the entire application is unusable and we have to crank up the adrenaline to 10 and try to find out what went wrong and then wait 40 minutes for our hotfix to get deployed so we gain back usability.

Scaling is challenging

First, let’s talk real quick about scaling in terms of infrastructure. Let’s say that we know there’s a huge event coming up for our application and we are expecting a lot of traffic, or maybe we have some rules set up for autoscaling based on requests / CPU volume, etc, and we need to spin up more instances of our application.

As we know, there are 2 types of scaling - vertical and horizontal, vertical meaning that we beef up a single instance with more CPU, RAM, etc, and horizontal meaning that we spin up more instances to handle the load. Scaling can get very expensive and if we want our company to actually make profit and not spend all of our money on our AWS bill, we need to be careful around that. With a monolith scaling gets expensive. Sure, we are expecting a lot of new sign ups this weekend, and we spin up 30 more instances of our monolith to ensure we handle the load correctly, but while our focus is to improve the load on our sign up flow, which might be a combination of adding a new user to a database, sending an email and taking a payment, we can’t scale just those things in a monolithic environment.

We scale everything up - we might not need to scale other parts of the system for our event like a chat system or a music player, etc, but we do end up scaling them, so the result is beefer and more expensive instances to pay for in the end of the month. It’s like you expecting guests to come over at your house and you go to clean up your living room, but you also end up going on the roof to clean that too.

Technology limitation

While there might be some flexibility, if we are taking a monolithic apporach for our app, we need to be careful with our technology choices and think about scaling right off the bat. What I mean here is let’s say we decide to write our application in Python. While this is fine, this also means that we’d be limited to only using Python in our application, meaning that we need to look for developer that know Python (yeah I know it’s a pretty popular language but we’re in theory world right now).

If with time our app evolves and grows and we feel like a different language will do a better job even for one part of our app, let’s say our app has an embedded video editor, if we want to change that, we can’t just stick a C++ file in our Django or Flask web application and expect it work. We either have to rewrite the entire app in C++, or, you guessed it - extract the video editing in it’s own separate service our monolith will call. Do you see where I’m going with this?

So should we all hate on the monolith now?

Absolutely not. Like I said before, the monolith is a great pattern when applied in the correct circumstances. We had a geeky conversation at work the other day revolving around technology choices and one think really resonated with me - start with the domain. Before deciding if a monolithic approach is the correct choice for your application, get familiar with your domain, and your domain will give you the answers you need.

Are you developing a simple application that needs a single database and it has a couple of pages you need to maintain? Monolith is a great choice. Working on an API that again uses a single database and exposes like 5-6 endpoints that revolve around the same domain entities?

Monolith is great. That being said, even if you have an extremely simple application and you don’t anticipate change at the beginning, you should structure your code in a modular way and make everything inside your monolith as decoupled as you can, because that not only makes your application much easier to develop and maintain, but it also puts in a state that is ready to be split into microservices should you decide to along the way.

What exactly are microservices?

Ok, let’s put it in the simplest way possible to understand:

Microservices means splitting our application into small services, that are decoupled from each other, but communicate with eachother in order to achieve the overall application functionality. Usually a microservice will revolve around doing one thing and it should be able to be deployed inependently from the the other services.

Let’s make up a quick and simple scenario so we can understand it better:

A case study I made up

The year is 2005 and we are still working at MySpace as a senior engineer. We currently have a monolith app that holds all of MySpace’s functionality, but the engineering team has been good and the rest of the business has given us the time to develop our monolith in a nice, modular way, modelled after our domain. As MySpace grows, so do our features and our modules. We now have a payment service, an email service and an image service that is used to crop and compress user’s profile pictures.

As our code grows, so do our tests and deployments time. With so many users our application gets a hefty load of requests and we want to keep the business growing, so we keep developing new features, but it doesn’t take long for us to realise that our initial approach that we took is starting to slow us down.

Image processing starts to take long, because we decided to develop our application in PHP at first and php is not that good at anything image manipulation.

We had a small bug that was overlooked by tests in our email service and that brought the entire app down, we had to issue a hotfix, but the deployment time for that hotfix took 40 minutes, 40 minutes of our site being down.

Our CTO calls the engineering department into a meeting and we all agree that if we continue with our PHP monolithic architecture our app is doomed. Our bill in AWS has quadrupled with our growth because we spin up entire instances of our application when we have high traffic that mostly affects the payment service.

Clearly we need to do better. After a brief discussion, someone goes “Hey. Why don’t we take out the payment related code and make it in a separate service that the monolith will call. That way we can scale it as much as we want to and cause it will be smaller, we’ll deploy and iterate faster!”

The room is in awe. Did that guy just came up with microservice architecture?

And they all lived happyly until the end of time. Or until the next issue that came.

Microservices in a nutshell

So as we mentioned above, microservices is the concept of havimg multiple, smaller services, that in the best case scenario would do one thing well. Thy are modelled around our domain (A.K.A the Payment Service is modelled around payments in our domain), and they communicate to eachother via different protocols.

The benefits

Ok, so if you remember about the drawbacks of the monolith me mentioned above, the benefits of microservices are more or less the exact opposites of the drawbacks, but let’s talk about them anyway briefly to avoid cluttering our brains with information:

They are easier to maintain

This goes without saying, if a service does one thing, that means less code, which means less of a chance to dive into a rabbit hole of files to look for errors. Debugging is also easier, since well if let’s say we have a service that is responsible for sending emails, and there is a bug in the text formatting, like we’ve all seen - Hello, {{firstName!}, how are you!, then locating and fixing this bug will be much easier in a service responsible solely for sending emails.

Modular

If developed correctly, microservices have a great feature to them - being modular. Meaning that we can plug them in different systems with little configuration.

Here’s an example - if we have a payment service that is decoupled and developed in way that it takes some information about an item and then calls a payment provider API, making a payment for it and keeping a record in a database, if our company decides to release another product, let’s say the main product is a streaming service, but now we want to release an online shop where you can buy t-shirts with the most popular shows from the service, then we can re-use the payment service and deploy it with our new shop, not worrying about making payments from scratch.

Wider technology choice

Like I’ve mentioned before when we were looking at the example about image manipulation becoming slow with PHP, this is a great benefit of having microservices - in stead of being faced with the choice of re-writing our entire application in a different language in order to improve the functionality of a critical part of it, we can simply re-write a single service.

We have a video processing service that we initially wrote in JavaScript and now we figured out that it’d be faster in C++? Much easier to re-write a smaller service. Or maybe we realised that NoSQL is a better choice for storing certain data? Since most of the time each microservice has its own database its much easier to switch.

The beauty of having smaller, decoupled services is that we can mix and match multiple technology choices that are the best for our functionality across our entire application. We can have a C++ service, a python service that does statistics reports, a C# backend server, a Node server that is used for the frontend, anything that we want!

More efficient for scaling

If you remember, when it comes to scaling a monolith app, that we end up scaling everything. We can’t just scale up the customer service module, we need to scale the entire app, even the aprts of it that we don’t need for a high traffic event. With microservices that’s much easier to control.

If we know what endpoints are going to be hit the most during a traffic spike, we can scale just the services that power those endpoints. Maybe we own a big online betting website and there’s a huge boxing match coming up? Spin up more instances of the payment service. Gary Vee holding an online conference? DDoS our own website so nobody is exposed to his bullshit.

They help with your team structure and domain

We’ve said before that domain is King, and knowing your domain will give you the extreme benefit of knowing what your architecture and technology is going to look like. With microservices one great benefit is that they are modelled based on the domain and they can improve the team structure.

You can have hybrid teams that own different services, like you can have a payments team that is responsible for developing and testing payment related features, a team that’s responsible for booking functionality, and taking this approach, where we have small teams owning a set of microservices promotes autonomy - the teams can make their own choices about their services, as long as they meet the SLO’s and maybe the API contract set up.

This is great, because teams can choose the language, framework, storage solution and even their way of working around the services - you can have a classic agile team developing a reporting service with Python, you can have a Kanban style oriented team that works on a booking back end with Node, etc. Adopting a microservice oriented architecture can enable you to create teams that have complete freedom over how they develop their services.

So wait, are microservices the absolute best approach for writing software?

Absolutely not. Microservices is an amazing architecture, but it comes with its own pitfalls, and most importantly if it’s not developed correctly it can cause you much greater agony than a messy monolith. Let’s talk about what can go wrong with microservices:

Using microservices when you don’t need to

I’ve said it before, but I’ll say it again just to really hit on how important this is - KNOW YOUR DOMAIN! Your domain will tell you if you need microservices or not. If we have a very simple app, like maybe an online percentage converter, do you need to split this into a service that will calculate percentages, another one that will be a server and a third one that will track how many calculations you’ve done for the day?

Proably not, and in cases where your functionality does not require microservices, you are much better off with a good old monolith. Microservices are all about reducing and managing complexity, not about using them just cause they’re cool.

Splitting a monolith we don’t fully understand into microservices

I have had conversations with friends working in software about the company going “Yeah we need to split the monolith into microservices”, but then splitting it up turns out horrible, It ends up with weird stitched up parts of the monolith into very specific services, still using the same databasae that the monolith uses, one service can not function without another one, a third can’t work if the monolith’s broken, and so on, and so on.

In order for a monolith to be split into microservices, the work needs to start on the monolith. If you have a messy codebase with logic scattered across various services, a helpers file that ended up powering 40% of your business logic, then splitting the code into smaller services should be done with extra caution.

Not understanding the purpose of microservices but going for it anyway

This is a common pitfall, where an organisation’s goal is to just go for microservices. Remember that migrating towards a microservice oriented approach is not the goal, the goal is to reduce complexity, increase resilience and stability of your system, microservices are just a tool to achieve that, and if we become so focused on just implementing the tool without actually achieving the goal, we are setting ourselves up for failure.

Microservices best practices

One of the best things about microservices is the freedom they provide. There isn’t a pattern set in stone that has to be followed religiously, we are free to make a lot of our own choices that will fit around our use case as long as we understand the domain and the trade-offs. There are some best practices that can make your architecture much more resilient though, so let’s talk about them.

Splitting a monolith

First off, before we sit down to break up a monolith, we need to think about the reasons for the split. I think there are 3 clear indicators that make for a good reasoning ground to split up a monorepo.

Trying different technologies

Like we discussed before, genuinely identifying the need for a different technology choice for certain problems is a great cause to try out microservices. Maybe our initial technical choices were great for when the application was small with just a few features and a few request, but we’ve identified that with growth there might be a different language or a framework that willdo the job better than our current choice. Microservices architecture is a great way to try and measure different solutions.

Improving a team structure

We’ve all worked in or at least heard of the classic back end / front end separated teams, but now we’re seeing this structure slowly fading away and moving towards a structure of smaller autonomous teams focused on specific areas of the business.

I’ve heard these teams being called squads, crews, focus teams, but the idea is that you’d have a few full stack devs, some of them with a greater strength towards either the back end or the front end, a Product Manager and a Team Lead that will all work together in small team on a few services. Microservices really support this type of structure, and allows for smaller teams to rotate people between each other, thus having an organised, but flexible team structure.

Pace of change

If we are finding ourselves to do a lot of changes to our system, (maybe we’re in an experimental phase where we ship a lot of A/B tests), or we’re anticipating that there would be the need to be doing changes on our system, then going for am microservices approach makes perfect sense, as rapid changes are much easier to be done when your services are smaller and take less time to deploy.

In my opinion, identifying any of these 3 “core” signs is a good reason to go for microservice architecture. Now let’s talk about some tips when we’ve identified the need to split. How do we go for it?

Start small and increment

One huge error I’ve seen is the immediate ambition to just get in there and start extracting everything out. This never ends up nice. Remember how we spoke that microservices are a tool and not a goal? Cool, let’s still keep that in mind. We don’t need to initially go crazy and start spinning up new services and databases. Start small.

  • Extract one service first and let it use the main database.
  • Keep the old logic in the monolith and set up a feature switch, so you can control if we’re using the monolith code or the separate service
  • Take an A/B approach. Set in a small percentage of your users, maybe 10% to end up on the new flow and observe the load on your system and the conversion for your users. Are they taking less time to complete actions? Are they taking more?
  • Once we’ve gained enough understanding then we can go for a full migration, give the service its own database and remove the logic from the monolith.

Architecture notes

Communication

This is entirely dependant on your organisational needs. Microservices communicate with eachother in a variety of protocols, including REST, RPC, Event Driven Architecture, etc. The scope of this blog does not cover going over these protocols in detail, but depending on your needs, you’d pick the one that fits your needs the most. Recently I’ve really been interested in Event Driven Architecture, so maybe expect a post on that soon (tm).

Logging

Just splitting our application into microservices without having observability around them is not enough. We need to ensure that we know what’s happening in our services, identify key metrics and have a single source of truth place where we can track them. Luckily, there are a lot of great logging services, like Datadog, which allows us to collect logs and metrics from multiple services and display them on customizable dashboards.

One serice per host

This I think goes without saying, but it is recommended to have one service per host.

You don’t want to have something like payment.services.myspace.com and email.services.myspace.com, well because that defeats the purpose of having separate entities, decoupled from each other. Always have a service per host.

Automate your deployments and your scaling

You don’t want to manually provision deployments and these days you don’t have to. There are alot of great CI/CD tools that allow us to take advantage of automated deployments, and a lot PaaS (Platform as a Service, such as AWS, Azure, etc) that allow us to automate our deployments and our scaling. Set up clear rules for when we want our services to scale up and down.

Be prepared for failure

I have yet to experience any form of software working perfectly. I don’t think we ever will. You can have all of the tests in the world, a hardcore automated and manual QA process, but your services will fail from time to time. Sorry kiddo, that’s just how the world works. One very revelating moment I had recently was that we need to be shifting our focus not to be narrowed down to preventing failures, but to expecting them and handling them.

  • What happens if a service call fails?
  • How does the rest of the system get affected?
  • Which are the critical services that we need to be ready to handle when they fail?
  • Do we halt the entire system, do we use a retry mechanism?
  • If a single services fails, what is the fallback?
  • What if there are problems with the database a service is using?

These questions will change across organisations, but it is important to ask them, and in stead of us getting obsessed with metrics such as how many tests we have to prevent failure (tests are still very important, don’t get me wrong), we should be asking ourselves “How do we recover from failure and when failure happens how do we gracefully handle it?“.

Conlusion

This was kind of a big boi post, I tried keeping it somewhat compact, but I hope this can give you clarity on what microservices are, when to use them and how to approach making the switch to them. I think if there could be a couple main takeaways from this post, they would be:

  • Domain is king and understanding your domain will mean understanding if you need microservices
  • Microservices are a tool to reduce complexity and improve system stability, not a goal to strive for.

Microservices are a great architecture, but they need to be approached carefully and incrementally. Don’t fall into the hype traps of thinking one approach is king and don’t hate on something like the monolith, just because you saw a post on Medium about it. In our industry, everything has its own place and time, and a good engineer doesn’t go for the latest trend, but they identify the need for change and the nature of the change.

Until next time!


Written by Emil Mladenov - a slavic software developer who decided to use a blog as a digital rubber duck

I also have a podcast