Data Architecture Advanced

Event Store

An append-only log as the authoritative source of truth — persisting every domain event that changes an aggregate's state, enabling event replay, temporal queries, multiple read projections, and complete audit history.

⏱ 12 min read

What it is

An event store is a database optimized for storing an immutable, append-only sequence of domain events. Instead of storing the current state of an entity (e.g., the current balance of a bank account), the event store records every event that changed that entity's state (AccountOpened, MoneyDeposited(100), MoneyWithdrawn(30), MoneyDeposited(50)). Current state is derived by replaying events from the beginning (or from a snapshot). This is the storage mechanism that implements the Event Sourcing pattern.

Dedicated event store databases include EventStoreDB (Greg Young's purpose-built event database), which organizes events into named streams (one stream per aggregate instance, e.g., Order-42). Event stores can also be implemented on top of PostgreSQL (append-only table with stream ID and sequence number), Apache Kafka (partitioned log with event retention), or DynamoDB (partition key = stream ID, sort key = sequence number).

Key concepts: Stream — a sequence of events for one aggregate instance; Event — an immutable fact that something happened, with a type, payload, and metadata; Projection — a read model built by replaying events and applying them to an in-memory or persisted state; Snapshot — a point-in-time capture of aggregate state to avoid replaying all events from the beginning.

Why it exists

Traditional CRUD databases discard history — an UPDATE overwrites the previous value; a DELETE removes the record. Event sourcing preserves the complete history of every change, which is fundamentally valuable for: financial audit trails (regulatory compliance requires every change to account balances to be permanently recorded), debugging production issues (replay events to reproduce the exact state at the time of a bug), temporal queries ("what was the state of this order at 3pm yesterday?"), and building new read models from history (replay all events from the past 5 years to populate a new analytics view). Event stores also decouple the write model (events) from read models (projections), making it straightforward to build multiple different views of the same data.

When to use

When audit trails are a regulatory or business requirement — financial services, healthcare, and compliance-heavy domains require complete, tamper-evident change history.
When you need temporal queries — "show me the state of this entity at any point in its history" is trivially answered by replaying to that point.
When the domain is complex and event-rich — domains like order management, trading, workflow orchestration, and collaborative editing naturally model as sequences of events.
When you want to build new read models from historical data — replay all past events against a new projection to bootstrap a new view without ETL pipelines.
When you're implementing Event Sourcing with CQRS — the event store is the canonical write-side storage for event-sourced aggregates.

When not to use

Don't use event sourcing (and thus an event store) for simple CRUD domains — user profile management, product listings, and configuration storage have no meaningful event history; the added complexity is unjustified.
Don't use it if the team lacks familiarity with the event sourcing pattern — event stores have a steep learning curve; incorrect aggregate design (too-large aggregates, wrong event granularity) is hard to fix after events have been written.
Don't expect an event store to replace a query-optimized database — querying an event store for business data requires projections; direct event queries are not how the pattern is meant to be used.
Don't use it if GDPR right-to-erasure is a hard requirement without a plan — immutable event logs are at odds with the right to be forgotten; address this with event encryption-at-rest and key deletion, or crypto-shredding patterns.

Typical architecture


  Command side (Event Sourcing + Event Store):

  Client ──▶ CommandHandler ──▶ Load stream from EventStore
                                 Apply events to rebuild aggregate state
                                 Execute business logic + validation
                                 Emit new domain events
                              ──▶ Append events to EventStore (optimistic concurrency)

  EventStoreDB streams:
  ┌─────────────────────────────────────────────────────┐
  │ Stream: Order-42                                    │
  │  #0  OrderCreated    {customer: "alice", items: []} │
  │  #1  ItemAdded       {product: "SKU-1", qty: 2}     │
  │  #2  OrderConfirmed  {at: "2024-01-15T10:00Z"}      │
  │  #3  PaymentReceived {amount: 49.98, method: "card"}│
  │  #4  OrderShipped    {tracking: "TRK-999"}          │
  └─────────────────────────────────────────────────────┘

  Query side (Projections):
  EventStore subscription (catch-up or live)
     │
     ├──▶ OrderSummaryProjection ──▶ PostgreSQL (current order state)
     ├──▶ RevenueProjection      ──▶ ClickHouse (analytics)
     └──▶ FraudDetectionProjection ▶ Redis (real-time scoring)

Pros and cons

Pros

Complete audit trail: every state change is permanently recorded with who made it, when, and what the change was — regulatory compliance without additional instrumentation.
Temporal queries: derive the state of any aggregate at any point in its history by replaying events to that timestamp.
Multiple projections: the same event stream can be replayed against any number of read models — add a new read model at any time by replaying historical events.
Debugging: reproduce production bugs exactly by replaying the event stream that preceded the failure in a test environment.
Natural event publishing: events stored in the event store are also the domain events that drive integration — no dual-write problem between the write model and event bus.

Cons

High complexity: event sourcing requires deep understanding of aggregates, projections, eventual consistency, and snapshot strategies — not suitable for teams new to the pattern.
Query patterns limited: current state queries require reading projections (read models), not the event store itself — adds operational overhead of maintaining projections in sync.
Event versioning complexity: when events must change (schema evolution), upcasting strategies are required to handle old events read back from the store.
Snapshot management: aggregates with thousands of events require periodic snapshots to avoid full stream replay on every command — snapshot lifecycle must be managed.
GDPR/right-to-erasure conflict: immutable event logs cannot be easily erased; crypto-shredding (encrypting PII in events, then deleting the key) adds complexity.

Implementation notes

Optimistic concurrency control is fundamental to event stores. When appending events, specify the expected stream version (the version number of the last event read when loading the aggregate). If the stream has been written to by another process since loading, the append fails with a concurrency conflict, and the command must be retried by reloading the aggregate and re-executing the business logic. EventStoreDB's append API accepts an expectedRevision parameter for this. This prevents the lost-update problem without pessimistic locking.

Event versioning and upcasting: domain events are serialized to JSON/Avro/Protobuf and stored. When the event schema must evolve (adding a field, renaming a field, splitting an event into two), old events in the store still use the old schema. An upcaster transforms old event versions into the current version during deserialization, before the aggregate processes them. Store the event type name and version (OrderCreated_v2) in the event metadata to route to the correct upcaster. Design events for immutability: instead of changing events, create new event types when business semantics change significantly.

Common failure modes

Aggregate too large / stream too long: An aggregate that processes thousands of events per day accumulates millions of events; without snapshots, loading the aggregate requires replaying all events — latency grows unboundedly over time.
Projection lag causing stale reads: A projection is built asynchronously from the event store; a client writes a command and immediately queries the read model; the projection hasn't caught up yet; the client sees stale state and reports a bug that isn't a bug.
Upcaster not implemented for schema change: An event type is changed in a new deployment; old events in the store are deserialized with the new schema; missing fields cause null pointer exceptions or incorrect behavior during aggregate replay.
Forgotten snapshot cleanup: Snapshots accumulate indefinitely; the snapshot store grows to hundreds of gigabytes; old snapshots are never cleaned up because there's no snapshot lifecycle management policy.
Accidental event store mutation: A developer corrects a production bug by directly modifying event data in the event store; the immutability guarantee is broken; audit trail integrity is compromised.

Decision checklist

Does the domain have genuine audit, temporal query, or event history requirements that justify the added complexity over CRUD with a change log?
Is the team experienced with event sourcing, or is there a training plan before adopting an event store in production?
Is an optimistic concurrency control strategy defined for all aggregate write operations?
Is an event versioning and upcasting strategy defined before the first event schema changes occur in production?
Is there a snapshot strategy (snapshot every N events, or on idle) for aggregates that accumulate many events?
Is there a GDPR/right-to-erasure compliance strategy (crypto-shredding for PII in events) if required by regulation?

Example use cases

Banking core ledger: Every debit and credit to an account is stored as an immutable event. Account balance is derived by summing all events. Complete transaction history is available by definition. Regulatory audits can replay any account's full history.
Order management system: OrderCreated, ItemAdded, CouponApplied, PaymentReceived, Shipped, Returned events for each order. New "order analytics" read model built by replaying all historical order events against a new projection. Customer service can see exactly what happened to any order.
Collaborative document editing: Every text insertion, deletion, and formatting change is an event. The document at any revision is produced by replaying events. Collaborative conflict resolution applies operational transforms over the event stream.

Event Sourcing — The architectural pattern that an event store implements as its storage layer.
CQRS — Always paired with event sourcing: event store as write side, projections as read side.
Event Streaming — Event store events can be streamed to Kafka for integration with other services.