How to Rate Limit OpenAI API Routes in Next.js

Most vibe-coded apps have at least one route that calls an LLM: a chat endpoint, a "generate" button, a summarizer. Each call costs money. If that route is public and unlimited, anyone can write a loop that calls it thousands of times, and you pay for every request.

This guide shows how to put sensible limits on LLM routes in a Next.js App Router app. The same ideas work for Anthropic, Gemini, Groq or any paid API.

The layers of protection

No single fix covers everything. Stack these:

1. Keep the API key on the server.
2. Require a logged-in user.
3. Rate limit per user (and per IP for anything public).
4. Cap input and output size.
5. Set a hard spending limit at the provider.

Layer 1: Never call the LLM from the browser

If your frontend calls api.openai.com directly, your key is in your JavaScript bundle and anyone can copy it. AI tools sometimes do this with a NEXT_PUBLIC_OPENAI_API_KEY or VITE_OPENAI_API_KEY. Move the call to a route handler and use a server-only env var:

// app/api/generate/route.ts
import OpenAI from "openai";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

If the key was ever public, rotate it. See exposed keys in NEXT_PUBLIC_ and VITE_ and how to rotate leaked keys.

Layer 2: Require auth

Anonymous LLM endpoints are the easiest to abuse. Unless you have a deliberate free demo, require a session:

import { NextResponse } from "next/server";
import { requireUser } from "@/lib/auth";

export async function POST(req: Request) {
  const { user } = await requireUser();
  if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
  // ...
}

See securing Next.js API routes with auth for the requireUser helper.

Auth alone isn't enough when signup is free, since an attacker can just create an account. That's what rate limiting is for.

Layer 3: Rate limit with Upstash

In-memory counters don't work on serverless hosts, because each request may hit a different instance. You need a shared store. Upstash Ratelimit uses Redis over HTTP and works on Vercel, Cloudflare and other edge runtimes.

npm install @upstash/ratelimit @upstash/redis

// lib/ratelimit.ts
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv(); // reads UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN

export const perMinute = new Ratelimit({
  redis,
  limiter: Ratelimit.slidingWindow(10, "60 s"),
  prefix: "rl:gen:min",
});

export const perDay = new Ratelimit({
  redis,
  limiter: Ratelimit.fixedWindow(200, "1 d"),
  prefix: "rl:gen:day",
});

Then use it in the route:

// app/api/generate/route.ts
import { NextResponse } from "next/server";
import OpenAI from "openai";
import { z } from "zod";
import { requireUser } from "@/lib/auth";
import { perMinute, perDay } from "@/lib/ratelimit";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const Body = z.object({ prompt: z.string().min(1).max(4000) });

export async function POST(req: Request) {
  const { user } = await requireUser();
  if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });

  const [minute, day] = await Promise.all([
    perMinute.limit(user.id),
    perDay.limit(user.id),
  ]);
  if (!minute.success || !day.success) {
    const reset = Math.min(minute.success ? Infinity : minute.reset, day.success ? Infinity : day.reset);
    return NextResponse.json(
      { error: "Too many requests" },
      { status: 429, headers: { "Retry-After": String(Math.max(1, Math.ceil((reset - Date.now()) / 1000))) } }
    );
  }

  const parsed = Body.safeParse(await req.json().catch(() => null));
  if (!parsed.success) return NextResponse.json({ error: "Invalid input" }, { status: 400 });

  const completion = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    max_tokens: 500,
    messages: [{ role: "user", content: parsed.data.prompt }],
  });

  return NextResponse.json({ text: completion.choices[0]?.message?.content ?? "" });
}

Pick model and limits that fit your product. A short per-minute window stops bursts, and a daily cap stops slow, steady abuse.

Public endpoints: limit by IP too

For a logged-out demo, key the limiter on IP. How you get it depends on your host:

const ip =
  req.headers.get("cf-connecting-ip") ??              // Cloudflare
  req.headers.get("x-forwarded-for")?.split(",")[0]?.trim() ?? // Vercel and most proxies
  "unknown";
const { success } = await perMinute.limit(`ip:${ip}`);

IP limits are rough (shared networks, VPNs), so keep anonymous limits tight and add a CAPTCHA such as Cloudflare Turnstile if abuse continues.

Layer 4: Cap input and output

Cost scales with tokens, so control both sides:

• Input: limit prompt length with your schema, as above. For chat, cap the number of past messages you send back to the model.
• Output: set max_tokens (or max_completion_tokens on newer models) on every call.
• Model: don't let the client choose the model name. Hardcode it on the server, or accept a small allowlist.

const ALLOWED = new Set(["gpt-4o-mini"]);
const model = ALLOWED.has(body.model) ? body.model : "gpt-4o-mini";

Layer 5: Usage-based credits for paid plans

If your app sells credits or tiers, enforce them on the server in the same request. Read the user's balance from your database, refuse if it's empty, and decrement it in the same transaction or with an atomic update. Don't trust a credit count sent from the client, and don't let users edit their own credits column. The Supabase RLS checklist shows how to keep privileged columns out of user-writable rows.

Layer 6: Set a hard limit at the provider

Code has bugs. Your last line of defense is the provider's billing settings:

• In the OpenAI platform, set a monthly budget for the organization or project, and turn on usage alerts.
• Use a separate project key for each app, so one app's leak or bug doesn't drain everything.
• Check the usage page after launch, and again after any spike in signups.

Other providers have similar controls. Set them before you launch, not after the first surprise invoice.

Also protect Server Actions

If your LLM call lives in a Server Action instead of a route handler, it's still a public endpoint and needs the same auth and rate limiting. See Next.js Server Actions security.

Quick checklist

• No LLM keys in client code or public env vars
• LLM routes require auth (or have strict IP limits if public)
• Per-user rate limit with a shared store
• Prompt length and max_tokens capped
• Model chosen on the server
• Credits enforced server-side
• Provider budget and alerts set

Find the routes you forgot

AI tools add new endpoints quickly, and it's easy to lose track of which ones call a paid API. PreflightPR flags exposed keys and API routes without auth checks in your repo. Run a free scan on a public repo to see what it finds. It's an automated best-effort review, so keep the provider spending limit in place no matter what.