Control API traffic with algorithms that balance precision, memory, and burst tolerance.
Outcomes
Each algorithm has distinct pros and cons. Token bucket is simple and allows bursts. Leaking bucket processes at fixed rate. Fixed window is memory efficient but has edge spikes. Sliding window log is accurate but memory-heavy. Sliding window counter is a hybrid approximation.
| Algorithm | Pros | Cons |
|---|---|---|
| Token bucket | Easy to implement, memory efficient, allows bursts | Tuning bucket size and refill rate is challenging |
| Leaking bucket | Memory efficient, fixed outflow rate | Bursts fill queue, recent requests may be limited; tuning parameters |
| Fixed window counter | Memory efficient, easy to understand | Spikes at window edges can exceed quota |
| Sliding window log | Very accurate, no exceeding in rolling window | High memory usage, stores timestamps even for rejected requests |
| Sliding window counter | Smooths spikes, memory efficient | Approximation, assumes even distribution in previous window |
How to read: Follow arrows: client sends request to middleware, which checks Redis and either forwards to API or returns 429.
Redis is chosen for its speed and support for INCR and EXPIRE commands. INCR increments the counter; EXPIRE sets a timeout to automatically delete the counter.
Client sends request to rate limiting middleware.
Middleware fetches counter from Redis and checks if limit is reached.
If limit reached, request is rejected with 429.
If not, request is forwarded to API servers and counter is incremented in Redis.
In distributed environments, race conditions and synchronization issues arise. Race conditions occur when concurrent requests read and write counters without atomicity. Synchronization is needed when multiple rate limiter servers handle the same client.
Performance optimization includes multi-data center setup with edge servers to reduce latency, and eventual consistency for data synchronization. Monitoring ensures algorithms and rules are effective; adjust rules or switch to token bucket for burst traffic.