Chamber 🏰 of Tech Secrets #9

The revenge of the monolith

The Chamber 🏰 of Tech Secrets is open. This week we’ll jump into the revitalized debate about microservices architectures, what they are, what they are good for, and if their days of utility are over. 🤷‍♂️

Thanks to everyone who has subscribed and is taking time to read and engage via comments and LinkedIn! Feedback and questions are always welcomed. Simply reply to this email (if you’re subscribed) or post in the comments below. 🙏

---

Is the microservices era over?

The Amazon Prime team created a stir this week when they released a blog post about their move from a microservices architecture using AWS Serverless technologies such as Lambda and Step Functions to a monolithic service in a single AWS ECS task.

Moving our service to a monolith reduced our infrastructure cost by over 90%. It also increased our scaling capabilities.”

I had been thinking about writing about this topic for some time and now my hand(s) are forced (to type). Let’s dig in!

What is a microservice? What is a monolith?

Let’s make sure that we have a common definition of these terms before we go too far.

A Microservices Architecture is a software architecture design pattern in which an application is composed of small, independently deployable services that communicate with each other using lightweight protocols, such as HTTP or messaging queues. Each microservice is responsible for a specific piece of functionality, encapsulating the code and data needed to perform that function.

• Single Responsibility Principle: Each microservice should have a single, well-defined responsibility or functionality. This principle helps ensure that the services are focused, maintainable, and easy to understand.

• Loose Coupling and High Cohesion: Services should be loosely coupled, meaning that they should have minimal dependencies on one another and communicate via simple, well-defined interfaces. High cohesion refers to the practice of keeping related functionalities together within a single microservice. At its best, this yields flexibility, maintainability, and the ability to evolve services independently.

• Autonomy and Isolation: Microservices should be autonomous and isolated, meaning they can be developed, deployed, and scaled independently without impacting other services. This principle allows teams to work on different services concurrently and reduces the risk of failure propagation.

• Fault Tolerance and Resilience: Microservices architecture should be designed to gracefully handle failures, ensuring that the entire system remains functional even if one or more services fail. This can be achieved through implementing patterns such as circuit breakers, fallbacks, retries, and timeouts.

A Monolithic Architecture is a large, single-tiered application where all the components, such as user interface, business logic, and data storage, are bundled together into a single unit. Monolithic applications are often characterized by a tightly-coupled structure, meaning that the different components depend heavily on one another and may be challenging to modify or scale independently.

Monoliths are usually more tightly coupled, use a single code base, share resources, are deployed “all or nothing”, and depend on internal API calls as opposed to external networking call (HTTP).

There are good monoliths (single stacks that encapsulate a particular domain’s processes) and bad monoliths (single stacks that contain multiple discreet business processes).

Credit: https://makeameme.org/meme/we-replaced-our

How I think about building services

I like building composable architectures using services. Services can make great building blocks (like legos) that help enable new and unexpected business outcomes with less duplication and less refactoring.

Here are some thoughts I have as I reflect on ~8 years of “microservices architecture” in a medium-sized software engineering organization.

• Microservices were often just a buzzy word for SOA: In our organization, most teams did not deploy microservices architectures and really did SOA, which was the right move. The more teams involved, the more decomposition required.

• Monoliths have at least L-shaped utility: Borrowing from Matt Rickards Monorepos have U-Shaped Utility, I believe monoliths have at least an L-shaped utility. They are far simpler to reason about and operate in the early stages of a product. Utility diminishes in correlation to the growth of the number of software teams involved in the domain. At some point carving off services along domain/sub-domain lines with network boundaries (HTTP calls, etc) will begin to be useful if not critical. I say “at least L-shaped” because I have never worked in an organization that has used a monolithic approach at very large scale (Google, Meta). If you have worked in those organizations, drop a comment with your thoughts.

• Software architecture is an art, not a science: Every single business is different. It’s composed of different people with different skills and different experiences. It has different leadership and different business objectives. Much of this is art, not science. How the organization is structured and what skills it possesses will naturally dictate what sorts of architecture make sense. The problem space, system usage, and other variables will also determine the “right” architecture. Soap box: I think we like these types of debates because we can argue about whats “right”… microservices or monoliths? The truth is neither is “right”… it truly depends on the situation, and its not black/white… there is a middle ground.

• Create well-defined and bounded contexts: Bounded context is a concept that defines the scope and boundaries of a specific domain. The goal is to encapsulate the domain’s logic, data, and functionality to ensure a clean separation of concerns between teams and prevent the complexity that can arise from mixing different domain concepts. Team A should not have to understand Team B’s implementation and vice versa.

• Default to avoiding service decomposition within the same bounded context: There is less marginal utility in this type of decomposition and a huge cost in complexity of reasoning about, operating, and developing new features in the system. If there are service components that have unique needs (scale, performance, etc) they can always be refactored into their own service later.

• Reverse-Conway: Conway’s Law says that organization’s tend to produce architectures that reflect their team structure. In many cases this will lead to sub-optimal decomposition (too many microservices). Instead, design the organization’s structure based on something like business capabilities (shoutout to the Enterprise Business Architecture world), which provide a head start on creating bounded contexts.

• Avoid stacking multiple un-related business capabilities into a single “application”: This is the bad kind of monolith. Teams should retain control over their own code base, environment, pipelines, etc. They should have as few outside dependencies as possible. When outside dependencies exist they should be kept to platforms and foundational services and other business services with clear, well-defined interfaces. Team A should never need to understand the implementation choices of Team B or have them in their critical path for delivery.

• Build evolution-friendly architectures: Expect your software to change and be re-architected over time. Instead of anticipating all future possibilities and trying to get everything “right” build a foundation that allows for incremental evolution and that anticipates significant future change. How easily can a component that has experienced unexpected scale be factored out into its own service? How easily can you shift from synchronous to asynchronous processing if needed? How good are your tests — can you confidently make significant changes to the architecture and know that they are “good”?

• Treat services as products: Products have to be explained and commercial products have to be marketed. Keep your API surface clean, simple, and easy to understand. If there are microservices, “hide” them as much as possible to create a clear surface for consumers. If you can’t explain how your architecture works and what value it adds in a few sentences, this might be a sign of unnecessary complexity.

• Use Serverless for lower-volume, scale-to-zero scenarios: I am a big fan of serverless technologies for “glue” functions such as reacting to object store puts or processing data streams with predictable throughput. I am less a fan for long-running applications for a host of reason that deserve (and will get) their own post in the future.

If I was starting a new project today, I would start with a single repository and build as a monolith until there were multiple teams colliding with each other, or components that were failing to meet customer expectations because of the architecture. At that point, I would selectively break out components that had special requirements. If there were multiple teams, I would follow the suggestions around carving off domains and letting each team decide what architecture makes sense for them within that domain. Keep scaling this approach as the system grows, but never decompose more than necessary.

Four thoughts from four smart people

I’ll leave you with three thoughts on the topic of microservices vs. monoliths from three smart people.

1. DHH (37 Signals):

Consolidate critical, dependent paths first. The worst form of microservice madness is when you splinter a single, coherent flow across multiple systems. Maybe this is signup, maybe this is checkout, maybe this is visiting a single piece of content. That's where microservices cause the most harm by making it cumbersome and error prone to update the entire flow. Making changes means coordinating across multiple systems, dealing with synchronization issues, and worse. So your consolation of microservices into macroservices on the way back to the monolith should start here.

2. Werner Vogels (CTO, Amazon):

And, if we had realized this at the start, and we chose an evolvable architecture, we could change components without impacting the customer experience. My rule of thumb has been that with every order of magnitude of growth you should revisit your architecture, and determine whether it can still support the next order level of growth.

I always urge builders to consider the evolution of their systems over time and make sure the foundation is such that you can change and expand them with the minimum number of dependencies. Event-driven architectures (EDA) and microservices are a good match for that. However, if there are a set of services that always contribute to the response, have the exact same scaling and performance requirements, same security vectors, and most importantly, are managed by a single team, it is a worthwhile effort to see if combining them simplifies your architecture.

However, I want to reiterate, that there is not one architectural pattern to rule them all. How you choose to develop, deploy, and manage services will always be driven by the product you’re designing, the skillset of the team building it, and the experience you want to deliver to customers (and of course things like cost, speed, and resiliency).

3. Darren Shepherd (Chief Architect, Acorn Labs)

3. Kelsey Hightower (Google):

Well done, Kelsey. Well done. 👏

---

The Chamber 🏰 of Tech Secrets is closed. Thanks to everyone for reading! 🙏 Go forth and create microservices, megaservices, monoliths, or whatever makes sense for your organization and your application.