Message Routing

What it is

Message routing is the set of patterns and mechanisms that determine which consumer or queue receives each message. In a simple system, every consumer subscribes to every topic and filters what it processes. At scale, this is wasteful and brittle. Routing patterns push filtering and dispatch decisions to the infrastructure layer — the broker, an intermediary router service, or the message envelope itself — ensuring messages arrive only at handlers that need them, with less compute wasted and clearer data contracts.

The foundational routing strategies are: content-based routing (route based on payload or header values), topic-per-event-type routing (each event type has a dedicated topic), routing slip (the message carries a list of destinations to visit in order), and process manager (a stateful coordinator decides the next destination based on accumulated state). These are not mutually exclusive; production systems often combine them.

Why it exists

Without explicit routing, the alternatives are: (a) broadcast everything to all consumers and let them filter locally — this causes O(consumers × messages) unnecessary work; or (b) embed routing logic in the producer — this couples the producer to every consumer's existence and address. Routing patterns decouple producers from consumers by moving dispatch decisions to a neutral intermediary layer, enabling producers and consumers to evolve independently.

When to use

• Multiple event types on a shared bus — an order service publishes order.created, order.confirmed, order.cancelled; different consumers need different subsets.
• Multi-step workflows — a document processes through normalisation → validation → enrichment → archival, each as a separate consumer; the routing slip pattern encodes the pipeline in the message envelope.
• Regional or tenant routing — messages must be directed to the correct regional deployment or tenant-specific handler based on a header value.
• Dead-letter routing — failed messages must be routed to a dedicated dead-letter queue (DLQ) rather than silently dropped or blocking the main queue.

When not to use

• When all consumers need all messages — broadcast via pub/sub is the right tool; routing adds complexity with no benefit.
• Complex conditional routing as a substitute for good domain modeling — if your router has 50 rules, the domain model is likely wrong; consider splitting the service or redesigning the event schema.
• When routing logic changes frequently — dynamic routing tables are powerful but become operational risk if modified continuously; prefer convention (topic-per-event-type) for stable systems.

Typical architecture

Content-based routing inspects message headers or payload and directs the message to one of N queues. Topic-per-event-type routing relies on the broker's native subscription model (SNS filter policies, Kafka topic names, AMQP routing keys). The routing slip pattern embeds a list of queue destinations in the message envelope; each consumer reads from its slot, processes, and forwards to the next slot. The process manager pattern is a stateful service that queries its database to determine the next routing decision.

Content-Based Router
  Incoming message: { "type": "order", "region": "EU", "priority": "high" }
  ├── if region == "EU"   → queue: order-eu-high
  ├── if region == "US"   → queue: order-us-standard
  └── (no match)          → queue: order-dlq

Routing Slip Pattern
  Message envelope:
  {
    "routingSlip": ["normalise-queue", "validate-queue", "enrich-queue", "archive-queue"],
    "currentStep": 0,
    "payload": { ... }
  }
  Normalise consumer:
    1. Process step 0
    2. currentStep++ → forward to "validate-queue"
  Validate consumer:
    1. Process step 1
    2. currentStep++ → forward to "enrich-queue"
  ...

Pros and cons

Implementation notes

Broker-native routing (prefer this): Use the broker's built-in routing capabilities before adding custom routing services. SNS supports attribute-based filter policies that route to different SQS queues at zero extra latency. Kafka routes by topic name and partition key. RabbitMQ uses exchange types (direct, topic, headers, fanout) and routing keys. These are battle-tested, scalable, and require no extra infrastructure.

Content-based routing in code: When broker-native routing is insufficient (complex conditions, multi-field logic), implement a lightweight router consumer: subscribe to an incoming topic, inspect the message, publish to the appropriate downstream topic, ACK the original. Keep routing logic stateless and purely functional (no side effects) so the router can be scaled horizontally without coordination.

Dead-letter routing: Every queue or topic should have an associated dead-letter queue (DLQ) that captures messages that fail to process after N retries. DLQs should be monitored with alerts, and a runbook should document the remediation steps (replay, manual intervention, discard). Never silently discard messages; route to DLQ instead.

Routing in Apache Camel / Spring Integration: These frameworks provide a Content-Based Router EIP implementation: <choice><when><simple>${header.type} == 'order'</simple><to uri="activemq:order-queue"/></when><otherwise><to uri="activemq:unknown-dlq"/></otherwise></choice>. The otherwise clause is critical — always handle unmatched messages.

Common failure modes

• Missing otherwise/default route: A content-based router with no default sends unmatched messages to a black hole. Always define a catch-all route to a DLQ.
• Routing key typos: A producer publishes to order.creatd instead of order.created; no consumer is subscribed; messages are silently dropped. Validate topic/routing key names at startup with schema registry checks.
• Routing slip infinite loop: A step re-inserts its own queue into the routing slip; the message circulates forever. Validate routing slip on ingress and cap the maximum step count.
• Router bottleneck: A centralised routing service processes all messages sequentially; under high load it becomes the bottleneck for the entire system. Scale routing logic horizontally or push it into the broker.

Decision checklist

• Can broker-native routing (SNS filter policies, Kafka topics, RabbitMQ routing keys) serve the need without custom code?
• Does every routing path have an explicit default/otherwise route to a DLQ?
• Are routing key / topic names validated at startup against the schema registry?
• Is the routing logic stateless so it can scale horizontally?
• If using a routing slip, is the maximum step count validated and capped?
• Is dead-letter routing monitored with alerts and a documented runbook?

Example use cases

• Multi-region payment processing: Payments are routed to regional processors based on the region header; high-value payments additionally routed to a fraud review queue via a content-based router.
• Document normalisation pipeline: Incoming documents route to format-specific normalizers (PDF handler, Word handler, HTML handler) via a content-based router on the contentType header.
• Notification dispatch: A single notification.requested event routes to email handler, SMS handler, or push notification handler based on the user's notification preferences encoded in the message header.

Related patterns

• Publish/Subscribe — the underlying mechanism for topic-based routing.
• Message Queues — the destination queues routing directs messages to.
• Correlation IDs — essential for tracing a message's path through routing hops.

Further reading

• Content-Based Router (EIP) — canonical definition and patterns.
• Routing Slip Pattern (EIP) — dynamic sequential routing in message envelopes.
• Apache Camel Choice EIP — content-based routing in practice.