API Gateway Pattern

The Front Door of Microservices

In a monolithic architecture, client requests go directly to the application. In a Microservices Architecture, your system might be split into 50+ different services. You cannot expect a mobile app to know the IP address of every service, handle authentication for each, and aggregate data from three different places.

The API Gateway is a server that acts as a single entry point into the system. It encapsulates the internal system architecture and provides an API that is tailored to each client.

1. Core Responsibilities

An API Gateway is more than just a reverse proxy. It handles cross-cutting concerns:

1. Routing: The most basic function. GET /users -> User Service. GET /orders -> Order Service.
2. Aggregation: A client needs user details AND their recent orders. The Gateway calls both services and combines the JSON into one response, saving the client from making 2 network calls (which is expensive on mobile).
3. Protocol Translation: The outside world speaks HTTP/JSON. Internal microservices might use gRPC (binary, fast) or AMQP (messaging). The Gateway translates between them.
4. Offloading: Moving functionality from individual services to the gateway:
   • SSL Termination
   • Authentication / Authorization
   • Rate Limiting
   • Caching

2. The BFF Pattern (Backend for Frontend)

A single API Gateway can become a bottleneck or a "God Class" that knows too much. The BFF Pattern suggests creating a separate gateway for each client interface.

• Mobile API Gateway: Small payloads, excludes heavy data to save battery.
• Web API Gateway: Rich payloads for desktop screens.
• Public Partner API: Strict rate limits and OAuth scope enforcement.

3. Rate Limiting Algorithms

Protecting your services from overuse or DDoS is critical. A gateway can use local counters or a shared store. A distributed limit needs an explicit consistency and failure policy.

A. Token Bucket

• Imagine a bucket that holds $N$ tokens.
• Tokens stream in at rate $r$ per second.
• To make a request, a client must take a token.
• Pros: Allows bursts of traffic (up to bucket size).
• Cons: slightly complex to implement.

B. Leaky Bucket

• Requests enter a bounded queue and are processed at a configured rate while work is available.
• If the queue is full, new requests are dropped.
• Pros: Smooths admitted bursts, subject to scheduling and processing capacity.
• Cons: Data burstiness is lost; valid traffic might be dropped during short bursts.

C. Fixed Window

• "100 requests per minute".
• Reset counter at 12:00, 12:01, etc.
• Problem: A client can send 100 requests at 11:59:59 and another 100 at 12:00:01. The server receives 200 requests in 2 seconds.

D. Sliding Window Log

• Keep a timestamp log of every request.
• Memory usage grows with retained requests. Exact enforcement requires atomic admission and a consistent time/window policy across participating workers.

4. Authentication & Security

The API Gateway is the Policy Enforcement Point.

1. Authentication: For a JWT, validate the signature and required claims under the configured issuer and algorithm policy. An opaque API key instead needs a lookup or another verification method defined by the key scheme.
2. Authorization: The Gateway checks scopes (Can this key access /admin?).
3. Forwarding: If valid, the Gateway passes the request to the microservice. It often injects headers:
   • X-User-Id: 12345
   • X-Role: Admin Strip client-supplied identity headers and authenticate the gateway-to-service path. Services must still enforce resource-level authorization; a role header alone is not a trust boundary.

5. Potential Drawbacks

Where they are useful, API gateways also introduce risks:

• Single Point of Failure: If the Gateway acts up, the entire system is down. It requires high-availability setup (load balancers in front of redundant gateways).
• Latency: It adds an extra hop. However, 1 hop on a high-speed internal network is usually cheaper than multiple round-trips from the client to separate services.
• Complexity: Configuring routes and policies for hundreds of microservices is a management burden.

6. Popular Technologies

• Nginx / HAProxy: The raw, high-performance base layers. Hard to configure for complex logic.
• Kong: Built on Nginx, adds plugins for Auth, Rate Limiting, Logging. Very popular open-source choice.
• Amazon API Gateway: A managed service with account, regional and API quotas; verify limits and throttling before sizing.
• Netflix Zuul: The original Java-based gateway from Netflix.
• GraphQL Servers (Apollo): Can act as an API Gateway, specifically excellent for Data Aggregation (Federation).

Summary

The API Gateway is the receptionist of your distributed system.

Worked boundary: a slow orders service

Assume a mobile screen aggregates a profile and recent orders. Its illustrative response deadline is 800 ms. Reserve 100 ms for gateway work and transit, and give each parallel downstream request a deadline below the remaining 700 ms. If orders times out, return an explicit unavailable-orders state only if the product allows a partial response. Do not return an empty order history that looks authoritative.

Retries consume the same end-to-end deadline and capacity budget. A request that already ran for 650 ms has little room for another attempt. For state-changing calls, use idempotency and resolve unknown outcomes before retrying. Test the gateway with an unavailable downstream and verify that its connection pool does not starve unrelated routes.

The AWS API Gateway quotas document bounded service capacity. The JWT best-current-practice standard describes validation requirements; signature checking alone is insufficient without issuer, audience and algorithm constraints.