Communication PatternsMessagingScalability

Fan-Out / Fan-In

The scatter-gather pattern for parallel work distribution and result aggregation — fan out to N workers simultaneously, then fan in to combine results within a budget, handling partial results and timeouts.

⏱ 11 min read

What it is

Fan-out / fan-in (also called scatter-gather) is a pattern where a coordinator fans out a request or task to N independent workers simultaneously, waits for all (or enough) of them to complete, then fans in the results by aggregating them into a single response. The key benefit is latency: instead of calling N services serially (total latency = sum of individual latencies), you call them in parallel (total latency ≈ slowest individual latency). This is how API gateways assemble composite pages, how search engines rank across multiple indices in parallel, and how insurance quote engines collect bids from multiple carriers.

Fan-out can be implemented synchronously (fire N async calls and await all) or asynchronously (publish N messages to a queue, collect results on a reply topic). Synchronous fan-out is simpler and appropriate when all workers need to respond before the caller can proceed. Asynchronous fan-out via a message queue decouples the coordinator from the workers and is appropriate for long-running tasks or when the caller does not wait for a combined response.

Why it exists

Many real-world operations require input from multiple independent sources: compare prices from 5 carriers, enrich a user profile from 3 data providers, validate an order against 4 rules engines. Doing these serially creates compound latency and means one slow dependency blocks everything. Fan-out/fan-in collapses serial chains into parallel branches, dramatically reducing perceived latency and improving throughput. AWS Step Functions' Parallel state, Kubernetes Jobs with parallelism, and map-reduce all implement variations of this pattern at different scales.

When to use

Multi-vendor comparison — requesting price quotes, availability, or risk assessments from multiple independent providers simultaneously.
Parallel enrichment — augmenting an entity with data from multiple independent sources (geolocation, credit score, fraud score, user preferences) before returning a composite response.
Composite page assembly — a Backend for Frontend calls product, inventory, pricing, and recommendation services in parallel to assemble a product detail page.
Parallel data processing — splitting a large dataset into shards, processing each shard independently, then combining results (map-reduce pattern).

When not to use

When steps are sequential and dependent — if step B requires the output of step A, fan-out does not help; use a pipeline instead.
Very large fan-out (hundreds of workers) — creating hundreds of goroutines/threads or HTTP connections simultaneously can exhaust resources; use a bounded worker pool with a queue instead.
When partial results are unacceptable — if all N responses are mandatory (e.g., financial settlement across all counterparties), a single worker failure propagates to the entire operation; model this carefully with compensating actions.

Typical architecture

The coordinator launches N parallel tasks. Each result is placed in a channel or reply topic. A fan-in aggregator collects results as they arrive. A deadline timer fires after the configured timeout; at that point the aggregator processes all results received so far, treating missing results as timeouts/partial responses. Results are merged into a composite response returned to the caller.

Coordinator
  │
  ├──► Worker A (carrier Alfa)   ─┐
  ├──► Worker B (carrier Beta)   ─┤  Deadline: 500ms
  ├──► Worker C (carrier Gamma)  ─┤
  └──► Worker D (carrier Delta)  ─┘
                                   │
                              Fan-In / Aggregator
                              ├── A: $120  (arrived 120ms)
                              ├── B: $118  (arrived 200ms)
                              ├── C: timeout (500ms elapsed)
                              └── D: $125  (arrived 350ms)
                                   │
                              Merge: return [A, B, D] sorted by price
                              Mark C as "unavailable"

Pros and cons

Pros

Latency reduction — P99 latency approaches the slowest single worker instead of the sum of all workers.
Resilience — one slow or failed worker does not block the entire operation when partial results are acceptable.
Scales naturally — add more worker types to include more data sources without changing latency.
Clean separation — each worker is independently deployed, scaled, and owned.

Cons

Complexity — managing parallel execution, correlation, timeouts, and partial results is significantly more complex than serial calls.
Resource amplification — N parallel calls means N times the connection, memory, and CPU usage at peak.
Hardest worker determines overall latency — the aggregator waits until the deadline, so slow or stuck workers consume the full budget.
Error correlation — matching response fragments back to the original request requires correlation IDs tracked in the aggregator state.

Implementation notes

Deadline, not per-call timeout: Set a single deadline for the entire fan-out operation (e.g., 500 ms), not a timeout per worker. Workers that have not responded by the deadline are treated as partial/unavailable. Propagate the deadline to workers so they can abort early if they know the result won't arrive in time (context deadline in Go, gRPC deadlines).

Partial result handling: Define the minimum required responses before the aggregator can return a useful result. For multi-vendor pricing: any non-empty subset is useful. For security validation across 4 rule engines: all 4 must pass. Encode this policy explicitly rather than hard-coding the expected count.

Async fan-out via message queue: Publish N messages to a work queue with a shared correlationId. Workers publish results to a reply topic keyed by correlationId. The coordinator subscribes to the reply topic and aggregates results until all are received or the deadline fires. This approach supports long-running fan-out (minutes, not milliseconds) and does not require the coordinator to hold open N connections.

AWS Step Functions Parallel state: The Parallel state starts N branches simultaneously; all branches must complete (or one must fail to abort) before the state machine advances. Each branch can be a long-running workflow. Use Map state instead when the set of workers is dynamic (N items from an array), which is map-reduce at the workflow level.

Common failure modes

Aggregator holds state forever: If a worker never responds and the deadline is not enforced, the aggregator leaks memory for each in-flight correlation.
Unbounded fan-out: Coordinator fans out to all items in an unbounded list, creating thousands of goroutines and exhausting resources.
Ignoring partial results incorrectly: A required worker fails silently; the aggregator returns a "complete" result missing critical data.
Result mix-up: Workers publish results without the original correlation ID; the aggregator assigns results to the wrong operation.

Decision checklist

Are all N tasks truly independent (no data dependencies between them)? If not, use a pipeline.
Is a deadline set for the entire fan-out operation, not per-worker timeouts?
Is the partial result policy explicitly defined (minimum required responses, handling of unavailable workers)?
Is fan-out bounded (fixed N workers, not dynamic list growth)?
Does each worker's response include the correlation ID to allow correct fan-in aggregation?
Is aggregator state cleaned up (TTL or explicit delete) after the operation completes or times out?

Example use cases

Insurance quote aggregator: Fan out to 6 carrier pricing APIs simultaneously; fan in the top 3 results received within 400 ms; return partial results if fewer respond.
Search results assembly: Fan out to product index, seller index, and sponsored content simultaneously; fan in and merge ranked results; guarantee 200 ms deadline.
Parallel image processing: Fan out an uploaded image to thumbnail generator, virus scanner, EXIF extractor, and CDN uploader in parallel; fan in status of each operation before marking upload complete.

Competing Consumers — fan-out to a pool of identical workers from a shared queue.
Request-Response — the synchronous building block for each individual fan-out call.
Correlation IDs — essential for matching fan-in responses back to the originating fan-out.