Rate limiting sounds simple until you try to do it across multiple Redis nodes, dozens of microservices, and tenants on different tiers. Then it becomes a coordination problem. The basic algorithm is about 5% of the work. The rest is making it correct under concurrency, cheap to run, and fair.
I've built a few of these. The biggest lesson is always the same: the textbook algorithms are a starting point, not a solution.
Let's start with what actually matters: picking the right algorithm for your use case, then implementing it in Redis without shooting yourself in the foot.
Token Bucket vs. Sliding Window - Pick Based on Traffic Shape
The token bucket algorithm gives each tenant a bucket that fills at a fixed rate up to a maximum capacity. Each request removes a token. No tokens means the request gets rejected (or queued, depending on your design). It's great when you want to allow short bursts but enforce a long-term average rate.
A tenant on the "Pro" plan might get a bucket capacity of 200 tokens, refilling at 20 tokens per second. They can burst 200 requests instantly, then they're throttled to 20/s. That burst tolerance is exactly what API consumers expect.
The sliding window log keeps a sorted set of timestamps for each request. To check the current rate, you count how many timestamps fall within the window. It's more precise, but it stores every single request timestamp in memory. At high throughput, that adds up fast.
There's a middle ground: the sliding window counter. You keep counters for the current window and the previous window, then interpolate. If your window is 60 seconds and you're 20 seconds into the current window, you estimate: previous_count * (40/60) + current_count. It's approximate, but the memory footprint is tiny compared to the log approach.
My recommendation: use token bucket for most API rate limiting. It handles bursty traffic gracefully and maps well to Redis operations. Use sliding window counter when you need strict "N requests per minute" semantics (billing systems, for example). Skip the sliding window log unless your throughput is low enough that memory doesn't matter.
Redis Implementation - Atomic Operations Are Non-Negotiable
Here's where most tutorials get lazy. They show you a basic GET → check → SET pattern and call it done. That pattern has a race condition so obvious it hurts.
# WRONG - race condition between GET and SET
current = redis.get(f"rate:{tenant_id}")
if int(current or 0) < limit:
redis.incr(f"rate:{tenant_id}"))Two concurrent requests both read current = 99, both see 99 < 100, and both increment. You just allowed 101 requests through. In a distributed system with multiple application servers hitting the same Redis instance, this happens constantly under load. If you've ever had to design API idempotency keys for distributed systems, you know how tricky these race conditions can be.
The fix is Lua scripting. Redis executes Lua scripts atomically - no other command runs while the script is executing. Here's a token bucket implementation that actually works:
- KEYS[1] = rate:{tenant_id}
- ARGV[1] = capacity, ARGV[2] = refill_rate, ARGV[3] = now_ms, ARGV[4] = requested
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
local data = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens = tonumber(data[1]) or capacity
local last_refill = tonumber(data[2]) or now
local elapsed = now - last_refill
local refill = elapsed * refill_rate / 1000
tokens = math.min(capacity, tokens + refill)
local allowed = 0
if tokens >= requested then
tokens = tokens - requested
allowed = 1
end
redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) * 2)
return { allowed, math.floor(tokens) }One round trip. One atomic operation. No race conditions. The bucket state (tokens remaining and last refill timestamp) lives in a single hash, and the script reads, computes, and writes in one shot.
For sliding window counter, the Lua script is even simpler:
local current_key = KEYS[1]
local previous_key = KEYS[2]
local limit = tonumber(ARGV[1])
local weight = tonumber(ARGV[2])
local current = tonumber(redis.call('GET', current_key) or 0)
local previous = tonumber(redis.call('GET', previous_key) or 0)
local estimated = previous * weight + current
if estimated < limit then
redis.call('INCR', current_key)
return { 1, math.floor(limit - estimated - 1) }
end
return { 0, 0 }The weight parameter tells the script how much of the previous window to count. Your application calculates this based on where you are in the current window.
Multi-Tenant Isolation - Don't Let Noisy Neighbors Win
Here's the thing about shared infrastructure: one tenant hammering your rate limiter shouldn't degrade another tenant's experience. But it does if you're not careful.
Key design matters. Always namespace your rate limit keys by tenant:
rate:{plan}:{tenant_id}:{endpoint}
Not just rate:{tenant_id}. Different endpoints might have different costs. A POST /search with a complex query costs way more CPU than GET /health. Your rate limiter should reflect that.
Cost-based limiting is the move here. Instead of counting each request as 1, assign weights:
endpoint_costs = {
"/api/search": 5,
"/api/users": 1,
"/api/bulk-import": 50,
}
cost = endpoint_costs.get(request.path, 1)
# pass cost as the 'requested' argument to the Lua scriptA tenant doing 20 search queries burns through their budget 5x faster than someone doing simple reads. This is fair, and it protects your infrastructure from expensive operations that happen to be "just one request."
Per-tenant configuration should live in a fast cache (Redis hash or even a local in-memory cache with TTL). Hitting a database on every rate limit check adds 2-5ms of latency per request, which adds up. Store tenant limits as a hash:
HSET tenant:limits:{tenant_id} capacity 200 refill_rate 20
Your Lua script can pull these values in the same execution, or you pass them from the application layer if you want the script to stay simple.
Avoiding the CPU Trap - Lua Is Fast, But Respect It
Lua scripts in Redis block the entire server during execution. A script that takes 50ms means every other client waits 50ms. For a rate limiter script, that shouldn't happen - these are simple arithmetic operations. But I've seen people try to do logging, analytics, and cleanup inside their rate limit scripts. Don't.
Keep the Lua script under 1ms of execution time. If you need to log rate limit events (for dashboards, alerting, or billing), do it asynchronously after the script returns. Publish a message to a Redis stream or channel:
result = redis.evalsha(script_sha, 2, current_key, previous_key, limit, weight)
if not result[0]:
redis.xadd("rate_limit_events", {
"tenant": tenant_id,
"endpoint": path,
"timestamp": time.time()
})The event logging doesn't block the rate limit decision. You process the stream with a separate consumer that writes to your analytics store.
Connection pooling is another CPU-adjacent concern. If each request opens a new Redis connection, the overhead of TCP handshakes and AUTH commands eats into your latency budget. Use a connection pool. In Python with redis-py, the ConnectionPool handles this. In Go, go-redis does it by default. In Node, ioredis has built-in pooling.
Handling Redis Failures - Graceful Degradation
Your Redis instance will go down at some point. When it does, you have two choices: fail open (allow all traffic) or fail closed (reject all traffic). (When these outages inevitably happen, having a post-mortem process like an incident review template helps your team prevent them from happening again). For a SaaS platform, failing open is almost always the right call. You'd rather allow some extra requests than take down your entire API.
try:
result = rate_limit_check(tenant_id)
except RedisConnectionError:
logger.warning(f"Rate limiter unavailable, failing open for {tenant_id}")
result = {"allowed": True, "remaining": -1}Log the event, alert on it, but keep the API running. You can also run a Redis replica and fail over to it automatically. Redis Sentinel or Cluster handles this, but even a simple primary-replica setup with manual failover is better than a single point of failure.
For multi-region deployments, use local Redis instances per region with eventual consistency rather than a single global Redis. The latency of cross-region Redis calls (50-200ms) makes them unusable for rate limiting. Accept that rate limits might be slightly off between regions for a few seconds. In practice, this is fine - nobody's quota enforcement needs to be globally consistent to the millisecond.
What I Got Wrong Early On
The first rate limiter I built used per-second windows. Ten requests per second, checked with a simple counter and EXPIRE. It worked in testing. In production, tenants complained about intermittent 429s even when they were well under their limit. The problem was window boundaries. A tenant sending 8 requests at 11:00:59 and 8 at 11:01:01 was fine on average, but each individual window showed 8/10. With a burst at exactly the boundary, they'd hit 10/10 in one window and get rejected.
Switching to token bucket with a capacity that was 2-3x the per-second rate solved this completely. The burst tolerance absorbed boundary effects, and the refill rate enforced the long-term average. The tenants stopped complaining, and the limit actually felt fair.
Rate limiting in distributed systems is one of those problems where the gap between "it works on my laptop" and "it works in production at scale" is enormous. It's why when building critical infrastructure, developers choose tools they trust rather than the newest trend. But the building blocks are solid: Lua scripts for atomicity, token bucket for burst tolerance, cost-based weights for fairness, and graceful degradation for resilience. Get those four things right and you've got a rate limiter that can grow with your platform.
