Custom Software & Cloud Infrastructure

Designing Resilient GraphQL & REST APIs: Rate Limiting, Caching, and Token Authentication Strategies

Production patterns for API resilience: cost-based rate limiting, cache key design, token rotation, idempotency and graceful degradation.

Thushara SanjeewaLead Systems Architect & Media Buying Director8 min read

API resilience is not an availability number; it is the set of behaviours your service exhibits when clients misbehave. A single partner running an unbounded retry loop, a mobile release that polls every second, or one GraphQL query with nested pagination can consume the capacity of an otherwise healthy platform. Resilience engineering means bounding the damage any single caller can do while keeping honest traffic fast.

The objective for lead engineers is a contract that is explicit about cost, cacheability and identity: what a caller may request, how much of it, how long the answer stays valid, and what credential proves who they are.

Rate Limiting by Cost, Not Request Count

Counting requests undercharges expensive operations. For REST, weight endpoints; for GraphQL, compute query cost statically before execution.

  • Assign cost points per field, multiplied by pagination arguments, and reject queries above a ceiling with a clear error.
  • Enforce query depth and complexity limits, disable introspection in production, and prefer a persisted-operation allowlist for first-party clients.
  • Implement limits with a sliding window or token bucket in a shared store keyed by API key, then user, then IP — in that order of precision.
  • Always return RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset and Retry-After; unlabelled 429s guarantee retry storms.

Cache Key Design

Caching failures are almost always key design failures. Rules that hold in production:

  1. Vary on everything that changes the response: path, sorted query string, Accept-Language, and tenant or role when applicable.
  2. Never share a cache entry across authenticated users. Mark personalised responses private, no-store and cache the shared shell separately.
  3. Use ETag plus If-None-Match for cheap revalidation, and stale-while-revalidate to absorb origin latency spikes.
  4. Prefer entity-tag invalidation over time-based expiry so writes refresh exactly the affected resources.
  5. For GraphQL, cache at the resolver or data-source layer — HTTP caching of a single POST endpoint is largely ineffective.

Token Authentication

  • Short-lived access tokens (5–15 minutes) with rotating refresh tokens, bound to a device or session and revocable server-side.
  • Asymmetric signatures with published JWKS and key rotation, verifying issuer, audience and expiry on every request.
  • Scope minimally and authorise per resource in the data layer, not only at the gateway. Gateway checks answer "who"; the data layer must answer "may they touch this row".
  • Never store tokens in `localStorage` for browser clients; use HttpOnly, Secure, SameSite cookies with CSRF protection.
  • Machine clients use client credentials with mutual TLS or signed assertions, plus per-key quotas.

Protocol Tradeoffs

ConcernRESTGraphQL
HTTP cachingNative, per-URLWeak; needs persisted queries or GET
Over-fetchingCommon without field filtersSolved by field selection
Rate limitingPer-endpoint weightsRequires cost analysis
VersioningURL or header versionsSchema evolution plus deprecation
Error semanticsStatus codes200 with errors array
Client autonomyLowerHigh
Abuse surfacePredictableDepth and complexity attacks
Best fitPublic and partner APIsRich first-party clients

Blueprint: Idempotency and Cost-Aware Limiting

ts
type LimitResult = { allowed: boolean; remaining: number; resetSeconds: number };

async function consume(key: string, cost: number, limit: number, windowSec: number): Promise<LimitResult> {
  const bucket = `rl:${key}:${Math.floor(Date.now() / 1000 / windowSec)}`;
  const used = await redis.incrby(bucket, cost);
  if (used === cost) await redis.expire(bucket, windowSec);
  return { allowed: used <= limit, remaining: Math.max(0, limit - used), resetSeconds: windowSec };
}

export async function POST(request: Request) {
  const apiKey = request.headers.get("x-api-key") ?? "anonymous";
  const idempotencyKey = request.headers.get("idempotency-key");
  const gate = await consume(apiKey, 5, 600, 60);

  const headers = {
    "RateLimit-Limit": "600",
    "RateLimit-Remaining": String(gate.remaining),
    "RateLimit-Reset": String(gate.resetSeconds),
  };

  if (!gate.allowed) {
    return new Response(JSON.stringify({ error: "rate_limited" }), {
      status: 429,
      headers: { ...headers, "Retry-After": String(gate.resetSeconds) },
    });
  }

  if (!idempotencyKey) {
    return new Response(JSON.stringify({ error: "idempotency_key_required" }), { status: 400, headers });
  }

  const cached = await store.get(`idem:${apiKey}:${idempotencyKey}`);
  if (cached) return new Response(cached, { status: 200, headers });

  const result = await createPayment(await request.json());
  const body = JSON.stringify(result);
  await store.set(`idem:${apiKey}:${idempotencyKey}`, body, { ex: 86_400 });
  return new Response(body, { status: 201, headers });
}

Strategic Action Plan: 90 Days

  1. Days 1–12 — Measure per-caller cost. Attribute latency, error rate and database time by API key and endpoint; identify the top ten expensive operations.
  2. Days 13–30 — Ship cost-aware limits. Weight endpoints, add GraphQL depth and complexity analysis, and return standard rate-limit headers everywhere.
  3. Days 31–50 — Rebuild caching. Define explicit cache keys and Cache-Control per resource class, add ETag revalidation and entity-tag invalidation on writes.
  4. Days 51–70 — Tighten identity. Move to short-lived access tokens with rotating refresh, JWKS rotation, minimal scopes, and row-level authorisation in the data layer.
  5. Days 71–90 — Add resilience and proof. Introduce timeouts, circuit breakers, idempotency keys on mutations, contract tests in CI, and a load test that verifies limits under abuse.

Frequently Asked Questions

How should GraphQL APIs be rate limited?

Compute a static cost for each query before execution using per-field weights multiplied by pagination arguments, then debit that cost from a token bucket keyed by API key or user. Reject queries exceeding depth or complexity ceilings with an explicit error. Request counting is inadequate because one query can be thousands of times more expensive than another.

What is the safest place to store access tokens in a browser?

An HttpOnly, Secure, SameSite=Lax cookie combined with CSRF protection, because JavaScript cannot read it and XSS cannot exfiltrate it directly. Tokens in localStorage are readable by any script on the origin, including compromised third-party dependencies. Keep access-token lifetimes short and support server-side revocation regardless of storage.

When are idempotency keys required?

On every non-idempotent mutation that a client may retry: payments, order creation, message dispatch and provisioning. Store the first response keyed by caller plus idempotency key for at least 24 hours and return it on replay. Without this, network timeouts and retry logic produce duplicate charges and duplicate records.

How do we version an API without breaking clients?

For REST, publish explicit versions and support the previous one for a documented deprecation window, using response headers to warn on sunset dates. For GraphQL, evolve additively: add fields, never repurpose existing ones, mark removals with @deprecated, and instrument field-level usage so you can prove no client depends on a field before deleting it.