n8n Rate Limiting: Stop API 429s from Breaking Your Workflows

The problem: every workflow thinks it owns the API

You have three workflows hitting the same third-party API — maybe it's an enrichment service, a CRM, or a payment provider. Each workflow has its own retry logic. Each one sees a 429, waits a second, and retries. The problem: they all retry at the same time. The provider sees a spike, extends the ban, and your whole automation stack goes dark.

This is the thundering herd. It happens because n8n workflows are isolated by design — they share no memory, no state, and no awareness of each other's API calls. The fix is not "add more retries." The fix is a shared rate limiter that sits in front of every outbound call to that API.

What a rate limiter actually does

A rate limiter answers one question: "Can I make this call right now?" If yes, it decrements the budget and lets the request through. If no, it calculates the wait time, pauses the execution, and then lets it through. That is it. The complexity is in making that answer consistent across all workflows hitting the same API.

You need three pieces:

• A shared counter — lives outside any single workflow (Redis, a Google Sheet, Airtable, n8n Data Table, or a tiny SQLite file on the VPS).
• A token-budget algorithm — token bucket or leaky bucket. Token bucket is simpler: you have N tokens, each call costs one, tokens refill at R per second.
• A gate node — every workflow calls this before the HTTP Request. If the gate says wait, the workflow waits. If the gate says go, the workflow goes.

Pattern 1: Token bucket in a Google Sheet (zero infra, works everywhere)

If you self-host n8n on a VPS but don't run Redis, a Google Sheet is the fastest shared store to spin up. One sheet, two columns: tokens (number) and last_refill (ISO timestamp).

The gate is a small sub-workflow (call it Rate Limit Gate) that every workflow invokes via Execute Workflow before the HTTP Request:

// Rate Limit Gate — Run Once for All Items
// Input: { api: "apollo", cost: 1 }
// Output: { allowed: true, wait_ms: 0 } or { allowed: false, wait_ms: 1250 }

const sheet = getSharedSheet(); // your helper to read/write the sheet
const now = Date.now();
const refillRate = 10; // tokens per second (adjust to your API's limit)
const maxTokens = 50;  // burst allowance

const row = sheet.getRowByApiName($json.api);
const elapsed = (now - new Date(row.last_refill).getTime()) / 1000;
const currentTokens = Math.min(maxTokens, row.tokens + elapsed * refillRate);

if (currentTokens >= $json.cost) {
  row.tokens = currentTokens - $json.cost;
  row.last_refill = new Date(now).toISOString();
  sheet.updateRow(row);
  return [{ json: { allowed: true, wait_ms: 0 } }];
} else {
  const needed = $json.cost - currentTokens;
  const waitMs = Math.ceil((needed / refillRate) * 1000);
  // do NOT update the sheet — the caller will wait and re-check
  return [{ json: { allowed: false, wait_ms, retry_at: now + waitMs } }];
}

The calling workflow adds an If node after the gate: if allowed === false, a Wait node pauses for {{ $json.wait_ms }} milliseconds, then loops back to the gate. If allowed === true, it proceeds to the HTTP Request.

Pattern 2: Redis token bucket (production grade, sub-millisecond)

If you already run Redis (common for queue-mode n8n), use it. The Lua script runs atomically — no race conditions, no double-counting.

-- KEYS[1] = "ratelimit:apollo"
-- ARGV[1] = max_tokens (50)
-- ARGV[2] = refill_per_sec (10)
-- ARGV[3] = cost (1)
-- ARGV[4] = now_ms (timestamp)

local key = KEYS[1]
local max = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local cost = tonumber(ARGV[3])
local now = tonumber(ARGV[4])

local bucket = redis.call('HMGET', key, 'tokens', 'last')
local tokens = tonumber(bucket[1]) or max
local last = tonumber(bucket[2]) or now

local elapsed = (now - last) / 1000
tokens = math.min(max, tokens + elapsed * rate)

if tokens >= cost then
  tokens = tokens - cost
  redis.call('HMSET', key, 'tokens', tokens, 'last', now)
  redis.call('EXPIRE', key, 86400)
  return {1, 0, tokens} -- allowed, wait_ms, remaining
else
  local needed = cost - tokens
  local wait = math.ceil((needed / rate) * 1000)
  return {0, wait, tokens} -- denied, wait_ms, remaining
end

Call it from n8n with an HTTP Request to a tiny Node/Go/Python shim that wraps EVALSHA, or use the Redis community node if you have it. The gate returns allowed and wait_ms — same contract as the Sheet version.

Pattern 3: n8n Data Table (no external deps, native)

Since n8n 1.0, Data Tables give you a built-in KV store. Create a table rate_limits with columns api_key, tokens, last_refill. The gate workflow uses two Data Table nodes: Select to read, Upsert to write. Same token-bucket math, zero external services.

Trade-off: Data Tables live in your n8n database. Under heavy load (hundreds of calls/sec) they add write contention. For a solopreneur stack (<50 calls/min across all workflows) they are perfectly fine and keep the architecture simple.

Respect Retry-After — it is not optional

When the API does return 429, it often includes a Retry-After header (seconds or an HTTP-date). Your HTTP Request node's Response tab should capture headers. Add a Code node after the request:

// After HTTP Request — handle 429 with server's guidance
if ($json.statusCode === 429) {
  const retryAfter = $json.headers['retry-after'];
  let waitMs;
  if (!isNaN(retryAfter)) {
    waitMs = parseInt(retryAfter) * 1000; // seconds
  } else {
    // HTTP-date format
    waitMs = new Date(retryAfter).getTime() - Date.now();
  }
  // also decrement our local bucket so we don't hammer it again
  await decrementLocalBucket($json.api, 1);
  return [{ json: { retry: true, wait_ms: Math.max(waitMs, 1000) } }];
}
return [{ json: { retry: false } }];

Then an If node: if retry === true, Wait for wait_ms, then loop back to the HTTP Request. This respects the server's explicit instruction instead of guessing.

Only retry what deserves a retry

The single biggest waste is retrying 400, 401, 403, 404. Those will fail identically on every attempt. Branch before the retry logic:

// Classify the response
const code = $json.statusCode;
if (code === 429 || code >= 500) return [{ json: { retryable: true } }];
return [{ json: { retryable: false } }];

Only 429 and 5xx enter the retry loop. Everything else fails fast, surfaces the real problem (bad credential, malformed payload, missing resource), and stops burning your retry budget.

Add jitter so retries don't synchronize

If ten workflows all wait exactly 1000ms and retry together, you just recreated the thundering herd. Add ±25% jitter:

const base = $json.wait_ms;
const jitter = base * 0.25 * (Math.random() * 2 - 1); // -25% to +25%
const finalWait = Math.max(100, Math.floor(base + jitter));
return [{ json: { wait_ms: finalWait } }];

This spreads the retry wave and lets the token bucket refill naturally.

Circuit breaker: stop calling a dead API

If an API returns 5xx for N consecutive calls (e.g., 5 in a row), it is likely down. Continuing to call it wastes tokens, fills logs, and delays other workflows. A circuit breaker trips after the threshold and returns a fast failure for a cooldown period (e.g., 60 seconds).

Implementation: a shared counter consecutive_failures in your rate-limit store. On success, reset to 0. On 5xx, increment. If >= 5, set circuit_open_until = now + 60000. The gate checks this first — if the circuit is open, return allowed: false, wait_ms: remaining_cooldown without even touching the token bucket.

Where this fits alongside idempotency

Rate limiting and idempotency solve different failures:

• Rate limiting prevents you from causing the 429 by spacing calls.
• Idempotency (see n8n Webhook Idempotency) protects you when the provider retries despite your 200.

You need both. The rate limiter sits at the outbound edge (your HTTP Request). The idempotency gate sits at the inbound edge (your Webhook trigger). Together they bound the failure surface from both directions.

A checklist you can paste into your project

• Every outbound HTTP Request to a rate-limited API goes through a shared gate first.
• The gate uses a token bucket (Sheet / Redis / Data Table) with the API's real limits.
• On 429, read Retry-After, wait that long, then re-check the gate.
• Only retry 429 and 5xx — fail fast on 4xx.
• Add ±25% jitter to every wait.
• Circuit breaker trips after 5 consecutive 5xx, cooldown 60s.
• Log every gate decision (allowed/denied/wait) to a sheet — volume is your health metric.

Start with the Sheet gate

It takes 15 minutes to build, requires zero infra, and survives a VPS reboot. When you hit the contention ceiling (you'll know — the gate becomes the bottleneck), swap the Sheet for Redis. The calling workflows do not change — they still call Rate Limit Gate and read allowed / wait_ms.

If you want a runnable starting point, the Cold Outreach That Converts template already has a per-workflow throttle. Extract that logic into the shared gate and you have the pattern in production today.