Consistency in Microservices: Transactional Outbox Pattern

6 min readDec 2, 2023

In a microservices architecture, different services often handle different aspects of business logic. When an operation requires multiple services, ensuring data consistency becomes challenging. For example, consider an Order Management System where placing an order involves updating inventory, sending notifications, and recording the order details. If any of these steps fails, it may lead to inconsistencies across services. In this article, we will explore the Transactional Outbox pattern in the context of an Order Management System and provide a detailed example to illustrate its implementation.

How the Transactional Outbox Pattern Solves the Problem:

Ensuring Atomicity and Reliability: The Transactional Outbox pattern ensures atomicity by persisting events within the same transaction as the business logic. It leverages the concept of an outbox, acting as a temporary storage location for events. This guarantees that either all steps within a transaction (including event generation) succeed, or none of them do.

In the context of an Order Management System, the Transactional Outbox pattern ensures that all relevant services are informed of changes that occur within the order workflow. Rather than immediately sending events to external systems, the events are persisted within the outbox as part of the same database transaction. This approach guarantees atomicity and ensures that events are not lost or duplicated, even in the face of failures.

Example Scenario:

Consider an Order Management System with three microservices:

Order Service: Responsible for order processing.
Inventory Service: Manages product inventory.
Notification Service: Sends notifications to customers.

Without Transactional Outbox:

A customer places an order.
The Order Service updates the order status but encounters an error when notifying the Inventory Service.
The order status is updated, but inventory is not adjusted, leading to inconsistency.

With Transactional Outbox:

A customer places an order.
The Order Service updates the order status and generates an “OrderCreated” event.
The “OrderCreated” event is appended to the outbox table within the same transaction.
An outbox processor asynchronously publishes the event to the Inventory Service and Notification Service.
If the Inventory Service encounters an error, the outbox processor retries, ensuring eventual delivery.

Key Components

Outbox Entity: Represents the structure of the outbox table where events are stored.

Event ID: Unique identifier for the event.
Event Type: Describes the type of event (e.g., “OrderCreated”).
Event Payload: Contains the data associated with the event.
Metadata: Additional information such as timestamps or routing details.

Transactional Logic: The business logic that triggers the generation of domain events within a transactional boundary.

Generate domain events based on the business operation.
Update the state of the application within the same transaction.

Outbox Persistence Mechanism: The mechanism to persist events within the outbox table.

Utilizes a relational database to store events within the same transaction.
Common fields in the outbox table: Event ID, Event Type, Event Payload, Metadata.

Outbox Processor: A background process responsible for processing events from the outbox.

Periodically scan the outbox table for new events.
Publish events to a message broker or event store.
Handle retries for failed event publications.

Message Broker or Event Store: A system for broadcasting events to downstream services.

Examples include Apache Kafka, RabbitMQ, or AWS SNS.
Ensures reliable event delivery and decouples event producers from consumers.

Event Consumers: Services that subscribe to and process events from the message broker.

Subscribe to specific topics or queues.
Handle incoming events and update their state accordingly.

Retry Mechanism: Ensure reliable delivery of events even in the face of transient failures.

Implement retry mechanisms in the outbox processor for failed event publications.
Exponential backoff, circuit breaker patterns, or custom retry logic can be applied.

Error Handling: Manage errors that may occur during event generation, persistence, or publication.

Implement error logging and monitoring.
Define strategies for handling errors during event processing.

Coding Implementation: Outbox Pattern

Step 1:Define the Outbox Entity:

@Entity
@Table(name = "outbox")
public class OutboxEvent {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "event_type")
    private String eventType;

    @Column(name = "event_payload")
    private String eventPayload;

    // Other fields and getters/setters
}

Step 2. Order Service:

@Service
public class OrderService {
    @Autowired
    private OutboxEventRepository outboxEventRepository;

    @Transactional
    public void placeOrder(Order order) {
        // Business logic to process the order and update the order status

        // Generate OrderCreated event
        String eventPayload = buildOrderCreatedEventPayload(order);

        // Save the event to the outbox within the same transaction
        OutboxEvent outboxEvent = new OutboxEvent("OrderCreated", eventPayload);
        outboxEventRepository.save(outboxEvent);
    }

    private String buildOrderCreatedEventPayload(Order order) {
        // Logic to construct the event payload
        // Include relevant information like order ID, customer details, etc.
        return "OrderCreatedPayload";
    }
}

Step 3. Outbox Processor:

@Component
public class OutboxProcessor {
    @Autowired
    private OutboxEventRepository outboxEventRepository;

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @Scheduled(fixedRate = 5000) // Run every 5 seconds
    public void processOutboxEvents() {
        List<OutboxEvent> outboxEvents = outboxEventRepository.findAll();

        for (OutboxEvent outboxEvent : outboxEvents) {
            // Publish events to Kafka
            kafkaTemplate.send("order-events", outboxEvent.getEventType(), outboxEvent.getEventPayload());

            // Remove the event from the outbox
            outboxEventRepository.delete(outboxEvent);
        }
    }
}

Step 4. Inventory Service and Notification Service:

Both services would have Kafka consumers to subscribe to the “order-events” topic and process the “OrderCreated” events accordingly.

@Service
public class InventoryService {
    @KafkaListener(topics = "order-events", groupId = "inventory-group")
    public void handleOrderCreatedEvent(@Payload String eventPayload) {
        // Logic to update inventory based on the order information
    }
}

@Service
public class NotificationService {
    @KafkaListener(topics = "order-events", groupId = "notification-group")
    public void handleOrderCreatedEvent(@Payload String eventPayload) {
        // Logic to send notifications to customers based on the order information
    }
}

Saga Pattern vs Outbox Pattern Use Case:

SAGA Pattern:

Saga is used for long-lived distributed transactions across services. Each service has local ACID transactions on its local database. However, in some cases, you need to combine the multiple local ACID transactions from different services (Order Create/ Update Order Status in order-service, complete payment in payment-service, order place in restaurant-service, update order state on each service’s response). For this, the first service should fire an event after completing the local ACID transaction. And the second service should read this event and start its own transaction. Thus, a distributed transaction basically requires a publish/subscribe mechanism to handle events across services.

SAGA Pattern

Above you see an example case, where 3 services communicate with events in the choreography approach using Kafka as the event store. Here, multiple Saga steps complete a Saga flow that changes the order state from Pending to Paid and finally to Approved.

First, payment service is called to set the order status as Paid
Then the restaurant service is called to complete an Order.

However, an important case in Saga is handling the rollback operation. For this, you need to create compensating transactions. If a Saga step fails, the previous step must undo its changes by applying the compensating transaction. For example, in above case, if the restaurant approval fails, the payment operation must be rollbacked using a compensating transaction.

Outbox Pattern:

While applying the Saga pattern, you will have two operations at each step. The local ACID transaction for business logic, and the event publishing. These two operations (Create order in local DB of order-service & publish event to Apache Kafka to process-payment) cannot be in the same single unit of work as they target separate data sources in different microservices. One is the local database, and the other is the event store.

To perform these operations (Saving order-service database & publishing event) consistently, you can apply the Outbox pattern. The Outbox pattern relies on having a local outbox table to hold events in the same database where you run the local transactions for business logic.

Then you can use these two database tables in the same transaction to perform local ACID transaction for business logic create order and event publishing. You can then read the events from the outbox table and publish them asynchronously.

Outbox Pattern

In summary, the Transactional Outbox pattern is suitable for scenarios where maintaining consistency within a microservice’s transactional boundary is crucial, while the Saga pattern is designed for managing distributed transactions and orchestrating long-running business processes across multiple microservices