Using Pub-Sub Messaging for Microservice Communication

What is the Pub-Sub Messaging Model?

Pub-Sub messaging is an asynchronous service-to-service communication pattern heavily used in microservices and serverless architectures. Services that adopt this communication model aim to push messages onto a topic, which is then received by all subscribers who subscribe to the topic.

This lets you seamlessly adopt an event-driven architecture while decoupling your services to help improve performance, reliability, and scalability.

Why should you consider using Pub-Sub architectures?

Consider the following scenario:

**Figure: Processing an order**

As shown above, when an order is made, three actions occur:

1. The customer is notified that an order was made.
2. The inventory is notified that an order needs to be prepared.
3. The invoice will be emailed to the customer.

An application that models this specific would greatly benefit from the event-driven pub-sub-based communication model rather than the traditional RESTful-based communication flow.

But, you may wonder - "Why is this necessary? Won't REST get the job done?"

Well, you're right. It does get the job done but with some tradeoffs. To understand this better, let's look at how this order fulfilment workflow would work in a RESTful environment:

Figure: A RESTful implementation of order fulfillment

The architecture proposed above poses two key issues:

• Linear scaling: Since each service is linearly invoked one after the other, all of these services will be scaled at the same rate.
• Error chaining: Due to this linear invocation manner, an error in one place would cause the entire workflow to be disrupted. For example, if "Notify inventory" runs into an error, both "Notify customer" and "Email bill" will not get invoked, thus disrupting the business workflow. This is true even if the error happens midway. For instance, if "Notify Customer" runs into an error, "Notify Inventory" will fail.

However, both of these issues can be resolved, and additional benefits can be obtained by moving this to a Pub-Sub messaging model. For example, consider the updated scenario:

Figure: Migrating architecture toward a Pub-Sub messaging model

The diagram above adopts the Pub-Sub messaging model. It has introduced a Topic - "OnOrderMade" that has three subscriptions:

1. Email Bill
2. Notify Customer
3. Notify Inventory

This means that whenever an order is made, it will:

1. Publish an event onto the "OnOrderMade" topic.
2. The subscribers who subscribe to the "OnOrderMade" topic will receive and process the message independently.

This completely decouples each service. They are no longer aware of each other and will only consider the event it receives. This lets these services scale independently while avoiding a linear error chain, as these services are not invoked one after the other but at their own pace.

How to implement a Pub-Sub messaging model?

At its heart, the Pub-Sub messaging model is heavily driven by event-driven architecture. So, the first step in implementing a Pub-Sub communication model is to refactor your existing codebase to use events.

An event is a state change or an update that has happened in your application.

In our case, an event could be an order being made.

An event can:

1. Carry the state of the object, like price, quantity, etc.
2. Include just an identifier of the object, like an order ID.

What are the critical components of an event-driven architecture?

After you have refactored your codebase to utilize events, you can proceed to the next step and start building your event-driven architecture. This involved setting up three critical components:

1. Event Producers: These services publish messages to the message router (topic).
2. Event Routers: This service filter and push messages to consumers. These event routers utilize standardized communication protocols to ensure that any service can communicate with one another through an event router. You can use a service like Kafka or AWS SNS to create and manage your topics.
3. Event Consumers: These services consume the events delivered via an event router.

By adopting these three components, you ensure that your producers and services are 100% decoupled from each other, thus letting your services be scaled, deployed, and managed independently.

Building the Pub-Sub model using Amazon SNS

Next, once you've established your event router, publishers, and subscribers, you can start building your Pub-Sub messaging model.

Your solution could look something like the diagram depicted below:

Figure: A solution for an event-driven pub-sub messaging model

As shown above, our initial business problem has now been refactored to use the event-driven pub-sub architecture. Our application now does the following:

1. Emits an event called - "Order Made" along with the Order payload.
2. The event gets pushed to an SNS Topic - "OnOrderMade"
3. The topic then immediately pushes the message onto its subscribers - "NotifyInventoryLambda", "NotifyCustomerLambda" and "EmailBillLambda" to execute their operations independently.

This ensures your application can avoid linear scaling and better scalability and overall performance.

However, when building pub-sub communication models, it's essential to consider the following:

Fault Tolerance

Fault tolerance is a crucial aspect developers must consider when building microservices-based applications. Since you're working in a distributed environment, there is room for errors. For instance, network delays, unavailability, or even high loads can disrupt your service-to-service communication.

Hence, you must ensure that your application has been architected to handle such cases.

Did you know that most messaging brokers (topics) try to guarantee a quality of service of level 1 (at least once)?

In a QOS Level 1, the subscriber will acknowledge the recipient of a message to ensure the message is not delivered again. Therefore, in cases where the subscriber runs into an error, the topic will continue pushing the message onto the subscriber until it gets an acknowledgment.

However, if there is an error, there is a chance that this might cause a recursive invocation loop. Therefore, you could introduce a Dead Letter Queue in which the Messaging Topic will automatically push the message to the DLQ after a determined set of retries so that it could be processed through a DLQ Redrive after the error has been resolved.

Figure: Using a DLQ Fault Tolerant Mechanism

As shown above, the DLQ will receive messages after a subscriber fails to process a message three times. The DLQ would have a high message retention period (unlike a topic, which has no retention period) which significantly helps manage errors.

For example, if the "EmailBillLambda" was not working due to a logical or network error, the DLQ Redrive could be initiated after the error was resolved to ensure that no messages were lost. Additionally, since the DLQ belongs to the Queue data structure, the order of messages is preserved.

Distributed Transactions

By using the pub-sub communication model to communicate with microservices, you're implementing a design pattern called "Fanning out".

Fanning out occurs when a single publisher publishes messages to a set of different subscribers and spreads the message across many services from one point.

But, by doing so, you have to ensure that your transactions across these services remain ACID-compliant.

For instance, in our order processing example, the operation can only be considered completed if the inventory is updated, the customer is notified, and the bill is emailed. If any operations fail, the entire process should be rolled back.

Luckily, this can be resolved by adopting the Saga Design Pattern. The Saga Design Pattern aims to improve data consistency across microservices in a distributed transaction. Essentially, the Saga Pattern creates a set of transactions that update microservices sequentially and publish events that trigger the following transaction for the subsequent microservice.

Idempotent Actions

When using a message broker that guarantees at-least once delivery, we need to consider that sometimes a subscriber could receive the same message more than once. Therefore, it's essential to implement protective measures to prevent the same message from getting processed twice by your subscriber.

If idempotentcy is not something that you can implement in your situation, you could consider implementing message deduplicating that prevents the message with the same deduplication ID from being processed over a given time period.

One way to do that is to track the messages processed in a database. The consumer can detect and discard messages when needed by recording the ID of every processed message in a database.

How Amplication Fits in

Adopting the pub-sub communication model can improve your microservices architecture's scalability, reliability, and performance. But building a scalable microservice architecture requires a lot of planning and boilerplate, scaffolding, and repetitive coding.

Amplication is a code generator for backend services that generates all the repetitive parts of microservices architecture, including communication between services using message brokers with all the best practices and industry standards.

You can build the foundation of your backend services with Amplication in minutes and focus on the business value of your product.