Make Sagas Consistent with Debezium Saga Orchestration for Microservices Using

Author : usitvhd
Publish Date : 2021-03-14 10:27:42

 Make Sagas Consistent with Debezium Saga Orchestration for Microservices Using



Saga Orchestration for Microservices Using the Outbox Pattern

Key Takeaways

    Sagas allow for the implementation of long-running, distributed business transactions, executing a set of operations across multiple microservices, applying consistent all-or-nothing semantics.
    For the sake of decoupling, communication between microservices should preferably happen asynchronously, for instance using distributed commit logs like Apache Kafka.
    The outbox pattern provides a solution for service authors to perform writes to their local database and send messages via Apache Kafka, without relying on unsafe “dual writes.”
    Debezium, a distributed open-source change data capture platform, provides a robust and flexible foundation for orchestrating Saga flows using the outbox pattern.

When moving to microservices, one of the first things to realize is that individual services don’t exist in isolation. While the goal is to create loosely coupled, independent services with as little interaction as possible, chances are high that one service needs a particular data set owned by another service, or that multiple services need to act in concert to achieve a consistent outcome of an operation in the domain of our business.

The outbox pattern, implemented via change data capture, is a proven approach for addressing the concern of data exchange between microservices; avoiding any unsafe “dual writes” to multiple resources, e.g., a database and a message broker, the outbox pattern achieves eventually consistent data exchange, without depending on the synchronous availability of all participants, and not requiring complex protocols such as XA (a widely used standard for distributed transaction processing defined by The Open Group) either.

In this article, I’d like to explore how to take the outbox pattern to the next level and use it for implementing Sagas—potentially long-running business transactions that span across multiple microservices. One common example is that of booking a trip comprising multiple parts: either all flight legs and accommodation should be booked together or none of them. Sagas split up one such overarching business transaction into a series of multiple local database transactions, that are executed by the participating services

A Primer on Sagas

    Related Sponsored Content
    Dataiku for Architects
    Getting Started with Progressive Web Apps (PWAs)
    GigaOm Report: High Performance API Management Testing
    Kubernetes Best Practices [2 Chapters]: Application Distribution and Staging, Managing State (By O'Reilly)
    [eBook] Microservices Reference Architecture

Related Sponsor

NGINX products give you much more than just support. Try NGINX Plus and NGINX App Protect free for 30 days.

For “rolling back” the overarching business transaction in case of a failure, Sagas rely on the notion of compensating transactions: each previously applied local transaction must be able to be “undone” by running another transaction which applies the inversion of the formerly done changes.

Sagas are by no means a new concept—they have first been discussed by Hector Garcia-Molina and Kenneth Salem in their SIGMOD ’87 Sagas paper. But in the context of the continuing move towards microservices architectures, Sagas backed by local transactions within the participating services are seeing growing popularity these days, as indicated by the MicroProfile specification for Long Running Actions, which is currently under active development. Sagas lend themselves in particular to the implementation of transactional flows taking a longer period of time, which typically can’t be addressed with ACID semantics.

To make things tangible, let’s consider the example of an e-commerce business with three services: order, customer, and payment. When a new purchase order is submitted to the order service, the following flow should be executed—including the other two services:


Figure 1. Order state transitions

First, we need to check with the customer service whether the incoming order fits into the customer’s credit limit (since we don’t want to have pending orders of one customer to pass a certain threshold). If the customer has a credit limit of $500, and a new order with a value of $300 comes in, then this order would fit into the current limit with a remaining limit of $200. A subsequent order with a value of $250 would be rejected accordingly, as it would exceed the customer’s then-current open credit limit.

If the credit limit check succeeds, the payment for the order needs to be requested via the payment service. If both the credit limit check and the payment request succeed, the order transitions into the Accepted state, so fulfillment for it could begin (which isn’t part of the process discussed here).

If the credit limit check fails, however, the order will go to the Rejected state right away. If that step succeeds but the subsequent payment request fails, the previously allocated credit limit needs to be released again, before transitioning to the Rejected state.
Implementation Choices

There are two general ways for implementing distributed Sagas—choreography and orchestration. In the choreography approach, one participating service sends a message to the next after it has executed its local transaction. With orchestration, on the other hand, there’s one coordinating service that invokes one participant after the other.

Both approaches have their pros and cons, (e.g., see this post by Chris Richardson and this one by Yves do Régo for a more detailed discussion). Personally, I prefer the orchestration approach, as it defines one central place that can be queried to obtain the current status of a particular Saga (the orchestrator, or “Saga execution coordinator,” SEC for short). Since it avoids point-to-point communication between participants, (other than the orchestrator), it also allows for the addition of further intermediary steps within the flow, without the need to adjust each participant.

Before diving into the implementation of such Saga flow, it’s worth spending some time to think about the transactional semantics that Sagas provide. Let’s examine how Sagas satisfy the four classic ACID properties of transactions, as defined by Theo Härder and Andreas Reuter (based on earlier work by Jim Gray) in their fundamental paper Principles of Transaction-Oriented Database Recovery:

    Atomicity: ✅ —The pattern ensures that either all services apply the local transactions or, in case of a failure, all already executed local transactions are compensated so that no data change is applied effectively.
    Consistency: ✅ —All local constraints are guaranteed to be satisfied after successful execution of all the transactions making up the Saga, transitioning the overall system from one consistent state to another.
    Isolation: ❌—As local transactions are committed while the Saga is running, their changes are already visible to other concurrent transactions, despite the possibility that the Saga will fail eventually, causing all previously applied transactions to be compensated. I.e., from the perspective of the overall Saga, the isolation level is comparable to “read uncommitted.”
    Durability: ✅ —Once the local transactions of the Saga have been committed, their changes are persisted and durable, e.g., after a service failure and restart.

From a service consumer point of view—e.g., a user placing a purchase order with the order service—the system is eventually consistent; i.e., it will take some time until the purchase order is in its correct state, as p


Category : general

Craft — a plugin for Sketch that allowed the InVision integration to work was often quite slow when creating those blue

Craft — a plugin for Sketch that allowed the InVision integration to work was often quite slow when creating those blue

- It wouldn’t be surprising to see many retail businesses downsize their operations by selling stores and laying off workers. If you actively invest in the market and your portfolio consists of re

SPLK-1003 Real Exam Questions And Answers PDF dump Exam

SPLK-1003 Real Exam Questions And Answers PDF dump Exam

- Authentic SPLK-1003 exam dumps will be the primary way for the preparation of SPLK-1003 questions. Splunk SPLK-1003 PDF Dumps. cover every single subject on the genuine Splunk Enterprise Certified Ad

Biden sets to work on reversing Trump policies with executive orders

Biden sets to work on reversing Trump policies with executive orders

- US President Joe Biden has begun to undo some of Donald Trumps key policies, hours after being sworn in.There is no time to waste when it comes to tackling

Easy Way to Clear MB2-716 Exam ~ Get MB2-716 Dumps PDF

Easy Way to Clear MB2-716 Exam ~ Get MB2-716 Dumps PDF

- Get latest and updated exam material from mockdumps with passing guarantee in first try. We provide 24/7 customer support to our honorable students