MediumInfrastructure40 min

Design a Rate Limiter

Design a distributed rate limiter that can throttle requests based on various criteria like user ID, IP address, or API endpoint.

StripeCloudflareAmazonGoogle

Functional Requirements

Limit requests per user/IP/API key within a time window
Support different rate limits for different APIs
Distributed: work across multiple servers
Low latency - should not significantly slow down requests
Inform users when they are throttled (429 status, retry-after header)

Steps (0/6)

Step 1: Clarify Requirements

Key questions: Client-side or server-side? What identifies a user (IP, user ID, API key)? Hard vs soft limits? What happens when throttled? Do we need different limits per API? Typical: Server-side, identify by API key, hard limit, return 429 with retry-after.

Key Points

Server-side rate limiting is more reliable
Multiple identification strategies may be needed
Hard limit: reject immediately. Soft limit: may allow burst
Must work in distributed environment (multiple servers)

1 / 6