Scalability With Asynchronous Messaging

Introduction

Many software systems are built using a distributed architecture. That is, a system composed of several independent services (also referred to as microservices) each performing a unique function.

For example, an e-commerce system might consist of the following:

Order Service
Inventory Service
Notification Service
Shipping Service
...

While a distributed architecture holds many benefits, one common challenge is the communication between microservices.

There are various protocols and methods of communication that can be used between microservices. One popular approach is REST APIs, which use HTTP methods for resource manipulation.

REST APIs follow a synchronous request-reply pattern, making it a suitable in scenarios where an immediate response is expected.

REST APIs follow a synchronous request-reply pattern.

While REST APIs are a viable option for communication between microservices, this blog article will focus on an asynchronous approach.

Asynchronous Messaging

The definition of asynchronous messaging can be explained quite succinctly by one of Microsoft's articles:

When using messaging, processes communicate by exchanging messages asynchronously. A client makes a command or a request to a service by sending it a message. If the service needs to reply, it sends a different message back to the client.

Publisher & Subscriber

Let's look into this pattern a bit deeper. Asynchronous messaging typically uses a model known as publisher & subscriber. Below is an image outlining this messaging model.

Topic: A subset or category of messages.
Publisher: Creates and sends message to a given topic.
Subscriber: Receives and consumes messages from a given topic.
Message Broker: Intermediary for transmitting messages.

E-Commerce Example: Creating an Order

Let's apply this architecture to the e-commerce scenario we mentioned earlier. Suppose we want to create a new order. After that, we need to update the inventory and send a confirmation email to the user.

That might look like the following:

Diagram depicting an e-commerce example (creating a new order) — E-Commerce Example (Creating a New Order)

The Order Service submits a message to the Fulfillment topic. Both the Inventory Service and Notification Service are subscribed to this topic. When they receive this message, the services can execute their respective logic.

Message Brokers

There are several enterprise grade message brokers that offer this capability. Some examples listed below:

Benefits of Asynchronous Messaging

Asynchronous messaging offers many benefits. Some notable ones are outlined below.

Decoupling & Scalability

As mentioned previously, an approach such as a REST API follows a synchronous request-reply pattern. This means that if the service receiving the request is down, the client service that sent the request will be blocked!

On the other hand, in asynchronous communication, services can operate independently from one another. For instance, the sender can continue with its work even if the recipients are unhealthy.

Fault Tolerance

In the case that a service becomes unhealthy, message brokers can store the message until the service becomes available to process the message. This prevents data loss and improves reliability of the system.

Improved Troubleshooting

Many message brokers offer what is called a dead-letter queue (DLQ). That is, a special type of queue that temporarily stores messages that were unable to be processed due to errors.

Dead-Letter Queue

A special type of queue that temporarily stores messages that were unable to be processed due to errors.

As AWS describes it:

This lets your developers focus on identifying the causes of the errors. They can investigate why the receiver couldn't process the messages, apply the fixes, and perform new attempts to deliver the messages.

Challenges of Asynchronous Messaging

While asynchronous messaging provides numerous benefits, it can introduce some challenges that are worth discussing.

Eventual consistency

It is common that subscribers update their database entities upon receiving a message. However, there is transient period of time (between the message dispatch and processing) where the service will not reflect the most recent data. Although the latency can be minimal, it is still important to be aware of.

Overhead and Complexity

Introducing asynchronous messaging via a message broker requires additional infrastructure, configuration, and maintenance.

Message Ordering

Message brokers typically do not guarantee that messages will be processed in a specific order (by default). If that is important for your given business domain, your design must take this into account. For example, an OrderCanceled message should only be processed after an OrderCreated message.

Note that message brokers such as Google Cloud do offer message ordering capabilities, but it needs to be configured.

Conclusion

If you are working on a distributed system where scalability and fault-tolerance are paramount, you should consider using asynchronous messaging.
It is important to be aware of other challenges this may introduce, such as eventual consistency.