Useful Tools for Managing Complexity of Microservice Architecture

At the 2016 OSCON and Software Architectures conferences, 1/3 of the public reports were about microservices. I heard complaints that managing a bunch of small services has become a nightmare. In this article you’ll find a list of useful tools for managing microservices effectively.

We almost always want to split an awkward monolith enterprise application into small pieces because business desires to reduce time to market, do fast experiments and deliver features continuously when they are ready with no delays. It is easy to achieve these requirements with architecture that consists of a bunch of self-described business-functions coupled with really small functional pieces, i.e. Microservices.

The key idea in microservice architecture is to create small (really small) services, develop and deploy them independently. Each microservice uses its own database and could be written in any programming language.

If you are new to microservices, a great starting point is Microservices Resource Guide.

A few years ago this great idea, about a microservice heaven, had some obstacles which killed all the advantages:

• If you split a system into 300 microservices, how would you connect all of them? Would you use API, a shared database, a message bus or a mix of integration approaches?
• If you choose one integration approach or use different integrations, depends on the requirements of the particular microservice
• How to manage all connections between hundreds of microservices?
• How to manage infrastructure for hundreds web services, API services and databases?
• Ultimately, how not to lose control of a system with microservice architecture?

My team has worked hard with microservice infrastructure for three years and has collected a list of useful tools. Here is a short of tools that we recommend to use for microservice architecture:

1. Containers, Clustering and Orchestration, IaC
2. Cloud Infrastructure, Serverless
3. API Gateway
4. Enterprise Service Bus
5. Service Discovery

In the rest of this article I’ll explain how these tools work and give you links.

Containers

Containers are a natural fit for microservices .You have to use containerization for microservice architecture, there are no options.

Containers give you the ability to create a full automation delivery pipeline — build and test a release in the specific environments, deploy a new release by creating a container instance to replace an old one. It is necessary for rapid deployment and for reducing time to market.

Learn more about this topic in these articles Why is Docker so Popular- Good and Bad of Docker and Using Containers to Build a Microservices Architecture

Infrastructure as Code Conception

It’s not just light-weight containers that make microservices architecture possible. Infrastructure as Code (IaC) stands behind containers and makes them so useful. A container description is a text file. This feature brings a list of new interesting things to the old development process. My top list of new things:

1. Put a container description to Git or Hg and commit any changes to DVCS.
2. Look at change history of project’s environment.
3. Compare environments, for example, compare staging and production environments.
4. Easy switch between environments on the local machine.
5. Share containers (environments) in the team.
6. Run the same multi-tiered application on a developer’s laptop, a QA server, or a production cluster of cloud instances, and it behaves exactly the same way.

Therefore, IaC will help you to manage microservice infrastructure.

Containers Orchestration

Containerization is an indispensable tool for scaling up a service by using scripts automatically. If you have a service under a high load, you have to dynamically increase or decrease the number of containers that run this service, which depends on the load level. Managing containers is a non trivial task so I recommend using orchestration.

For instance, Docker has had a built-in orchestration mechanism since the summer of 2016. There are a number of cloud and on-premise solutions for container clustering and orchestration. Check out 8 Container Orchestration Tools to Know.

Cloud Infrastructure

Microservices demand reliable and scalable infrastructure. Our clients chose clouds most of the time. We usually deploy microservice infrastructure on Amazon EC2 or Azure Service Fabric. All popular cloud platforms have API for Docker and other containers.

Serverless

I have to highlight Serverless because it has changed the rules. I recommend Gojko Adzic’s article Serverless architectures: game-changer or a recycled fad? where you’ll find why serverless is something new.

We use Azure Function or AWS Lambda for triggering infrequent operations or events. These tools are helpful for integrating pipelines and message processing. For example, when you process documents, you could send a piece of text to the Function to extract tags and get the result back.

API Gateway

If you connect a hundred microservices directly to each other, you’ll find how hard it is to control security, analyze traffic, create reliable channels and scale in/out an infrastructure for high-load API calls.

You can use API Gateway to avoid these problems and centralize API management. Azure API Management and Amazon API Gateway offer a cloud gateway for creating, publishing, maintaining, monitoring, and securing APIs at any scale. Also, there is a list of on-premise solutions, for example, Mulesoft API Gateway.

Your microservices should know only about API Gateway and connect to each other only through this gateway, therefore you’ll get full control under API calls.

Refactoring Legacy Systems

Specifically, API Gateway is helpful for replacing legacy systems with a new one or changing architecture to microservices. The first step is to change any direct API calls to go through API Gateway. As a result, every system will know only about API Gateway but not about other subsystems. Then you can easily split and change legacy systems one by one.

Enterprise Service Bus

As API Gateway centralizes synchronous calls, an Enterprise Service Bus (ESB) centralizes asynchronous messaging for inter-service communication.

Sometimes it is really hard to choose between ESB and API Gateway in any particular case. The reason for this obstacle is that ESB can be used for a synchronous call simulated by implementing two queues, one for an outgoing request and another for an incoming response. On the other hand, API Gateway can send an asynchronous call by creating a waiting thread on both sides. Start exploring this topic in the article API platforms are different from “just another ESB”.

Any mature cloud platform offers a reliable and scalable message bus. For example, AWS has Simple Queue Service (SQS) and Azure has Service Bus. My favorite on-premise solutions are Apache Kafka and RabbitMQ.

API Gateway and ESB

As a result, your architecture will look like some sort of combination of API Gateway and ESB:

Service Discovery

In a distributed system, services may move from one machine to another over time. This means service endpoint addresses change as the service moves to nodes with different IP addresses, and may open on different ports if the service uses a dynamically selected port.

When running containers at scale on an infrastructure made of immutable servers, how does an application identify where to connect to in order to find the service it requires? In this case you need a Service Discovery solution.

Service Discovery offers the ability to:

• Register and unregister a service
• Check the health of a particular instance of the service
• Balance the load among the “discovered” services.

They involve creating a directory of services, registering services in that directory, and then being able to look up and connect to services in that directory.

When we want to use Service Discovery, we have three options:

1. Build service discovery solutions by yourself. Check architecture that demonstrates a DNS- and load balancer-based solution on Amazon EC2 Container Service.
2. Use on-premise solutions such as Consul or ZooKeeper.
3. User cloud solutions such as Azure Naming Service or AWS Application Discovery Service

Anything else?

I explained the most common tools that have proven their worth with microservice architecture. If you have something to add to my list, feel free to suggest it in the comments.