API Design Essential

API Gateway

A single entry point for all API traffic that handles cross-cutting concerns — authentication, authorisation, rate limiting, request transformation, routing, and SSL termination — abstracting the internal service topology from external consumers.

⏱ 13 min read

What it is

An API gateway is a server that acts as the single entry point into a system from the outside world. It sits between clients and backend services, intercepting all API requests and applying a pipeline of cross-cutting concerns before forwarding the request to the appropriate upstream service and returning the response.

Core responsibilities of an API gateway include: routing requests to the correct backend, enforcing authentication and authorisation, rate limiting and throttling, SSL/TLS termination, request and response transformation, protocol translation (e.g., REST to gRPC), logging, and observability injection (trace IDs). Popular implementations include Kong, AWS API Gateway, Azure API Management, Nginx, Traefik, and Envoy-based solutions (such as Gloo Edge and Ambassador).

Why it exists

Without an API gateway, every microservice must independently implement authentication, rate limiting, TLS, and logging. This creates duplication, inconsistency, and a long tail of security vulnerabilities when teams implement these differently. The gateway centralises these concerns at the edge, allowing service teams to focus on business logic rather than infrastructure plumbing.

From a client perspective, the gateway provides a stable external surface: internal service URLs, ports, and topology can change without affecting consumers. The gateway also solves the protocol mismatch problem — external clients expect REST over HTTPS while internal services may communicate over gRPC or message queues.

When to use

Microservices or SOA architectures where multiple backend services must be exposed as a unified API surface.
Any system requiring centralised authentication, rate limiting, or WAF (Web Application Firewall) enforcement.
When external clients (third-party developers, mobile apps, partners) need a stable, versioned API surface.
Protocol translation: exposing gRPC or WebSocket backends to HTTP/REST clients.
Canary deployments or A/B testing by routing traffic percentages to different service versions.

When not to use

Pure internal service mesh communication — east-west traffic between services is better handled by a service mesh (Istio, Linkerd); the gateway is for north-south (external) traffic.
Single monolithic application — a gateway adds latency and operational complexity without providing benefits when there is only one backend.
Simple prototypes — the setup overhead of a gateway is not justified until you have multiple services or security requirements.

Typical architecture

Internet
   │
   ▼
[API Gateway]
  ┌────────────────────────────────────────────┐
  │ 1. SSL Termination (HTTPS → HTTP internal) │
  │ 2. Auth: validate JWT / API key            │
  │ 3. Rate limiting (per key/IP/endpoint)     │
  │ 4. Request transformation (headers, body)  │
  │ 5. Routing based on path/method            │
  │ 6. Load balancing to upstream instances    │
  │ 7. Response transformation                 │
  │ 8. Logging / metrics / trace injection     │
  └────────────────────────────────────────────┘
   │              │               │
   ▼              ▼               ▼
[Users Svc]  [Orders Svc]  [Products Svc]
(gRPC)       (REST)        (REST)

Service Mesh vs API Gateway:
  API Gateway    → north-south traffic (client ↔ cluster boundary)
  Service Mesh   → east-west traffic  (service ↔ service inside cluster)
  Both           → can coexist; Istio ingress gateway blurs the boundary

BFF + Gateway layering:
  Mobile Client → API Gateway → BFF (mobile) → [Services...]
  Web Client    → API Gateway → BFF (web)    → [Services...]
  Gateway handles: auth, rate limit, TLS
  BFF handles:    aggregation, transformation, client-specific logic

Pros and cons

Pros

Centralised enforcement of auth, rate limiting, and security policies — implemented once, applied uniformly.
Hides internal service topology from external consumers; backend can be refactored freely.
Single point for TLS termination, certificate management, and header injection.
Enables traffic routing for canary deployments, blue/green switches, and A/B tests.
Reduces duplication of cross-cutting concerns across service teams.

Cons

Single point of failure — gateway outage takes down all APIs; requires HA deployment (multiple instances + health checks).
Added latency per request from the additional network hop and processing pipeline.
Can become a bottleneck or a "smart pipe" anti-pattern if too much business logic migrates into it.
Configuration management is complex at scale; declarative config (Kong declarative, AWS CDK) helps but adds tooling overhead.
Vendor lock-in with managed cloud gateways (AWS API Gateway, Azure APIM) — migrating later is costly.

Implementation notes

Authentication at the gateway

The gateway validates incoming credentials and forwards a trusted identity to backend services. For JWT-based auth: the gateway verifies the signature and expiry, then either forwards the original token or exchanges it for a shorter-lived internal token (token exchange / token relay). For API keys: the gateway maps the key to a consumer identity and attaches metadata (consumer ID, plan limits). Backend services should trust the gateway's verified identity header and not repeat JWT validation.

Rate limiting strategies

Apply rate limits at multiple granularities: per API key, per IP, per endpoint, and per consumer plan tier. Use a distributed counter (Redis) for multi-instance gateways to ensure limits are enforced globally. Return 429 Too Many Requests with Retry-After and X-RateLimit-* headers. Consider sliding window algorithms over fixed windows to prevent burst traffic at window boundaries.

Request and response transformation

Gateways can transform requests (add/remove/rename headers, rewrite paths, inject correlation IDs) and responses (strip internal headers, reshape payloads, aggregate from multiple upstreams). Limit transformation complexity at the gateway — complex transformation logic belongs in a BFF. Keep gateway plugins thin; fat plugins become an unmaintainable hidden service.

Service mesh vs API gateway boundary

A service mesh (Istio, Linkerd) manages mTLS, observability, and traffic policies for east-west (internal) traffic. An API gateway manages north-south (external-to-internal) traffic. The two complement each other. In Kubernetes, an Istio ingress gateway or Gateway API resource can handle both, but many organisations run a dedicated API gateway at the edge (Kong, AWS API Gateway) and Istio internally. The key rule: don't put business logic in either; keep them as infrastructure concerns.

Kong, AWS API Gateway, and Azure APIM compared

Kong (open source) — plugin-based, Lua/Wasm/Go plugins, runs on Nginx; deployable on any infrastructure; self-managed HA requires effort.
AWS API Gateway — serverless, no capacity management, tight AWS integration (Lambda, Cognito, IAM); limited transformation capabilities without Lambda authorisers.
Azure API Management — full lifecycle management platform with developer portal, policy engine (XML-based), built-in analytics; more opinionated than Kong.
Nginx + Lua / Traefik — lightweight, config-as-code, good for container environments; requires more custom plugin development for advanced auth flows.

Common failure modes

Single-instance gateway — no redundancy; gateway restart causes full API outage.
Business logic creep — teams add validation, enrichment, and orchestration to gateway plugins; it becomes an untestable service without a deployment pipeline.
Misconfigured rate limits — limits set too high provide no protection; limits set too low block legitimate traffic spikes.
Token forwarding without validation — gateway passes the incoming JWT header to backends without verifying it; an attacker can forge claims.
Config drift — gateway configuration is managed manually through a UI, diverging from source control; reproductions fail, rollbacks are guesswork.

Decision checklist

Is the gateway deployed in HA mode (multiple instances + health-check-based load balancing)?
Is gateway configuration stored as code (declarative YAML, Terraform, CDK) and reviewed in version control?
Are rate limits configured per consumer tier, not just globally?
Does the gateway verify JWT signatures before forwarding requests to backends?
Are correlation IDs injected for every request, enabling end-to-end distributed tracing?
Is there a circuit breaker or timeout configured for each upstream service route?
Is business logic absent from gateway plugins?
Is SSL/TLS termination happening at the gateway (not at individual services)?

Example use cases

E-commerce platform API — AWS API Gateway routes /v1/products/* to ECS tasks, enforces Cognito JWT auth, applies per-API-key rate limits, and triggers CloudWatch alarms on 5xx rates above 1%.
Internal developer platform — Kong deployed on Kubernetes manages all internal service APIs; plugins enforce OAuth2 client credentials for service-to-service, route canary traffic (10%) to new service versions, and inject Datadog trace headers.
Financial services partner API — Azure APIM provides a developer portal for bank partners, enforces mutual TLS on partner certificates, translates REST requests to internal SOAP/WSDL services, and enforces SLA-based throttling per partner tier.

Backend for Frontend (BFF) — BFF sits behind the gateway and handles client-specific aggregation.
API Authentication — OAuth2 and JWT enforcement is a core gateway function.
API Rate Limiting — rate limiting patterns enforced at the gateway layer.
API Versioning — gateways route version prefixes to different backend versions.