Distributed Systems Essential

Saga Pattern

Managing long-running multi-service business transactions using a sequence of local transactions linked by events or a central orchestrator, with compensating transactions for failure recovery.

⏱ 12 min read

What it is

A saga is a sequence of local transactions where each step publishes events or messages to trigger the next step. If any step fails, the saga executes a series of compensating transactions to undo the work of all previously completed steps. There is no global lock — each local transaction commits independently to its own database and immediately becomes visible, which means sagas provide eventual consistency rather than strict ACID isolation. Two coordination styles exist: choreography (event-driven, decentralised) and orchestration (central coordinator drives the flow).

Why it exists

Microservices each own their own database. Achieving atomicity across service boundaries without a distributed transaction (which is expensive, coupled, and often unavailable) requires an alternative. The saga pattern, introduced by Garcia-Molina and Salem in 1987 for long-lived database transactions, was adapted for microservices to provide a way to manage business consistency across services without holding distributed locks. It trades strict isolation (no global snapshot) for loose coupling and high availability.

When to use

  • Business operations span multiple microservices with separate data stores.
  • Long-running workflows where holding locks across service calls would be impractical (seconds to minutes).
  • The business domain can tolerate intermediate inconsistent states between saga steps.
  • You need clear visibility into the progress and state of a multi-step operation.
  • Compensating transactions can be defined for every saga step.

When not to use

  • Operations requiring true isolation — sagas expose intermediate state to concurrent requests ("dirty reads" at the business level).
  • Steps that cannot be compensated (e.g., sending an irreversible physical shipment).
  • Very simple two-step operations where the added complexity outweighs the benefit.
  • Teams without the operational maturity to observe and debug asynchronous compensation flows.

Typical architecture

The orchestrated saga uses a central saga orchestrator (often a state machine) that sends commands to participants and reacts to their replies:

Orchestrated Saga — Happy Path & Compensation
═══════════════════════════════════════════════

Orchestrator        Order Svc      Payment Svc     Inventory Svc
     │                  │               │                │
     │─── CreateOrder ─►│               │                │
     │◄── OrderCreated ─│               │                │
     │                  │               │                │
     │─── ChargePayment ────────────────►               │
     │◄── PaymentCharged ────────────────               │
     │                  │               │                │
     │─── ReserveStock ──────────────────────────────────►
     │◄── StockFailed ───────────────────────────────────
     │                  │               │                │
     │  [FAILURE — begin compensation]   │                │
     │                  │               │                │
     │─── RefundPayment ────────────────►               │
     │◄── PaymentRefunded ───────────────               │
     │                  │               │                │
     │─── CancelOrder ──►│               │                │
     │◄── OrderCancelled │               │                │
     │                  │               │                │
     │  [Saga complete — compensated]    │                │

In choreography-based sagas, there is no orchestrator. Each service listens for events and reacts by performing its local transaction and publishing a new event. The overall flow emerges from the event chain, but understanding it requires reading all participating services.

Pros and cons

Pros

  • No distributed locks — each service commits locally and releases resources immediately.
  • High availability — failure in one service does not block others indefinitely.
  • Orchestration provides a single place to understand and observe the entire business flow.
  • Works across heterogeneous databases and messaging systems.
  • Naturally resilient — steps can be retried independently.

Cons

  • No isolation — intermediate states are visible to concurrent requests between saga steps.
  • Compensating transactions must be explicitly designed and tested for every step.
  • Choreography sagas are hard to understand and debug — flow is implicit in event chains.
  • Orchestrators can become logic dumping grounds, concentrating business rules in infrastructure code.
  • Idempotency of all saga steps must be carefully ensured to handle message redelivery.

Implementation notes

Saga state management is critical: the orchestrator must persist its current step durably (typically in a relational database or event store) so it can resume correctly after a crash. Each participant must implement both a forward operation and a compensating operation, and both must be idempotent (safe to replay on duplicate messages). Use the transactional outbox pattern to atomically write to the local database and publish an event in a single local transaction, avoiding the dual-write problem. Frameworks like Eventuate Tram (Java), MassTransit (C#), and Axon Server provide saga support with durable state machines.

Consider saga isolation anomalies carefully: lost updates (two sagas update the same record), dirty reads (saga reads uncommitted data from another in-flight saga), and fuzzy reads (data changes between saga steps) all require application-level countermeasures such as semantic locks, commutative updates, or version checks.

Common failure modes

  • Compensation failure: a compensating transaction itself fails, leaving the saga partially compensated with no further automatic recovery.
  • Non-idempotent steps: message redelivery causes a step to execute twice, producing duplicate side effects.
  • Concurrent saga interference: two sagas modify the same resource simultaneously, causing lost updates.
  • Orchestrator state loss: orchestrator crashes without durable state, unable to determine which steps have been executed.
  • Pivot transaction ambiguity: unclear which step is the "point of no return" where compensation becomes impossible.

Decision checklist

  • Can every saga step be compensated? Have you identified any non-compensatable operations?
  • Are all forward and compensating operations idempotent?
  • Is the saga orchestrator state persisted durably and recoverable after crash?
  • Have you addressed the dual-write problem (transactional outbox or similar)?
  • Is the team comfortable with eventual consistency and the saga isolation anomalies?
  • Do you have observability tooling to trace saga progress and diagnose stuck sagas?

Example use cases

  • E-commerce order placement: coordinating order creation, payment, inventory reservation, and shipping across four separate microservices.
  • Travel booking: reserving flight, hotel, and car simultaneously with compensation if any leg cannot be confirmed.
  • Financial transfers between accounts in separate banking systems.
  • Multi-step onboarding workflows that provision resources across independent internal services.

Further reading