We were told a monolith is evil and microservices are the answer. What nobody told us is that microservices come with many pain points deriving from its distributed nature.

In the past, we built an application connected to one database where normalized data was queried using “joins”. Then came: big data, big traffic and with that: big latency. We needed to solve query latency where no cache would help us, the data was too big.

Microservices came to the rescue, we needed to break our big application into small pieces. Scaling our servers only in the required areas, allowing teams to work in different parts of a system without deploying the whole application, enabling autonomy.

We all jumped and started writing microservices, not always understanding the implications. We read blog posts on big companies and their hundreds of microservices; how they scaled their architecture, seeing what they built, but sometimes without understanding the “how”.

“People try to copy Netflix, but they can only copy what they see. They copy the results, not the process” — Adrian Cockcroft, former Netflix Chief Cloud Architect

To build microservices we needed to remove dependencies across domains and we found that this was easier said than done. The basic idea was to build services that were responsible for their own logic and wrote to their own DB.

Distributed services dictated distributed data and this came with a price, we gained autonomy and lost ACID transactions, a single source of truth and “joins” between entities. In order to reconcile these we needed to find a way to communicate across services.

Microservices communication patterns

• Service-to-service communication: each service implements the logic it needs, gathering data from multiple services.

Pros: each service gathers all the data it needs, so the business logic is developed only in the corresponding service.

Cons: creates dependencies between services, coupling them for deployment and maintenance. This results in a “distributed monolith”. Each service will need multiple hops to get all the data it needs and then will need to “join” it using business logic instead of queries in the DB.

• API gateway: a single service that acts as the backend entry point.

Pros: Clients don’t need to understand the backend architecture. Reduces the number of requests from the clients.

Cons: Every API needs to be developed twice, once in the gateway and once in the service itself. We still don’t have joins in the DB, we do it programmatically in the gateway. The gateway service gets all the load. We get latency since the gateway hops to many services and “joins” the data.

• Aggregate data at the client side: clients perform different requests to all services in the backend and then aggregates the data.

Pros: services are autonomous and can be deployed separately without inter-dependencies at the backend.

Cons: clients need to store all the gathered data and “join” the sources, sometimes requiring business logic. In mobile apps this approach is not acceptable, since it will require the app to demand updates for every change in business logic.

• Events bus: each service sends events for its changes and the “interested” parties listen and save the info as needed.

Pros: Separation of concerns, low latency.

Cons: Data duplication that carries storage costs, the need to write failover procedures, the risk of data inconsistency across services.

Choosing each one of these communications patterns demands: monitoring, alerts, data-modeling for consistency, transactional updates across domains and rollback capabilities.

You need to understand that writing distributed systems is hard (microservices are after all distributed systems).

In my case, I am a fan of the “lean and mean” approach and architecture evolution.

In every new project, I start with the simplest approach that can be built with a small team and as the application and the organization scales, the architecture evolves. Remember:

• Never compromise quality, test everything. Integration and end-to-end tests are your best friends, then you can refactor as you go without risking instability and regressions.
• Build for the actual requirements of scale you have at the moment. Measure everything to know when to refactor and improve. Over-architecture is your enemy. Complicated technologies will stall your team’s progress.
• Use decoupled architecture and define APIs for future refactoring, defining clear domains and responsibilities (for example: separate APIs for users, orders, stock, etc. that can be separated in the future)
• Build everything using backwards compatibility where there are no breaking changes, neither in APIs nor in the DB. DB tables/columns are always added, never removed or changed.

Keep in mind that “There is no such a thing as a free lunch”.