HTTP headers and their three audiences

A header is metadata addressed to a specific audience: the browser, an infrastructure layer between client and server, or the application itself. That audience reads the header and acts on it before anything touches the body. Pick the wrong header or value and the audience does something you didn’t intend: a CDN caches a private response and replays it to the next user, a browser refuses your cookie, or a rate-limit gateway lets a forged client through.

This lesson sorts every header by its audience, across the families a web app sets: content negotiation, caching, conditional requests, authorization, rate-limit signaling, the security baseline, request context, and custom headers.

Headers are addressed to one of three audiences

An HTTP message is a method or status line, a block of headers, and a body. The body carries the resource; the headers carry metadata. Before you write a header value, ask the question that determines everything else: who is going to read this?

Every header has one of three audiences:

• The browser, which enforces the header. It sets a cookie, applies a Content Security Policy, refuses an insecure transport, or denies camera access to a third-party iframe.
• The infrastructure layer between client and server (CDN, reverse proxy, load balancer), which acts on the header. It caches a response, picks a compression codec, or throttles a noisy client.
• The application, which reads the header to make a decision: which user is signing this request, what content type to return, or whether this is a retry of an operation it has already done.

The audience is the durable property of a header. Once you know who reads it, you know where in your stack to set it and what breaks when you get it wrong. Meet a new header later and you place it by reasoning about who reads it.

A few headers blur the boundary. Cache-Control is mostly infrastructure, since CDNs and shared caches are the design audience, but the browser caches and reads it too; Authorization is application-audience, yet the infrastructure layer may still log it. In practice the mapping is one-to-one for every header you’ll touch, and the boundary cases are called out where they come up.

Browser The browser enforces this header.

• Set-Cookie
• Strict-Transport-Security
• Content-Security-Policy
• Permissions-Policy

Infrastructure CDNs and proxies act on this header.

• Cache-Control
• Vary
• Content-Encoding
• Retry-After

Application Your application code reads this header.

• Authorization
• Idempotency-Key
• If-None-Match
• Content-Type

The three audiences a header can be addressed to.

Content negotiation: what the body is, what the client wants

Content negotiation is how the client and server agree on the body’s format. A handful of headers do the work, each naming a format on the request side (“what I’m sending, and what I want back”) or the response side (“what you’re getting”).

Content-Type describes what the current body is. On a request, Content-Type: application/json says “the bytes after the headers are JSON.” On a response, Content-Type: application/problem+json; charset=utf-8 says “the body is RFC 9457 Problem Details, encoded as UTF-8.” The request and response Content-Type name different bodies and are independent.

Accept is the request-side header for what the client wants back. A web app’s API client sends Accept: application/json; a browser navigating to a page sends a longer list, Accept: text/html,application/xhtml+xml,..., naming the formats it can render. Content-Type and Accept are different axes, so a POST can carry both: Content-Type: application/json with Accept: application/problem+json means “I’m sending JSON; if you reject me, send the error as Problem Details.”

Content-Encoding declares how the body was compressed. The client advertises the codecs it supports via Accept-Encoding, the server picks one, and the response names it as Content-Encoding. Default to zstd for new traffic: every current browser supports it, the major CDNs negotiate it by default, and it compresses more per unit of CPU than Brotli at the same quality. Keep Brotli where you must support older clients, with gzip as the fallback. You rarely set this by hand, since the platform negotiates the codec and stamps the header. Set it manually without compressing the body and the declared encoding won’t match the bytes, breaking downstream tooling.

Vary extends the cache key. A shared cache keys responses by URL alone; Vary: Accept-Encoding, Cookie tells it the response also depends on those request headers, so it folds them into the key. Without Vary: Cookie on a per-user response, the CDN serves user A’s response to user B; without Vary: Accept-Encoding, a Brotli body reaches a client that asked only for gzip and can’t decompress it. Any response that differs by a request header needs a Vary naming that header. The next section’s default for authenticated HTML, private, no-store, makes Vary: Cookie moot because the response isn’t cached at all, but on cacheable per-user surfaces it’s mandatory.

Two more round out the set. Content-Length declares the body’s size in bytes, which the infrastructure layer reads to frame the message and tell when the body has finished arriving; the platform sets it on serialization, so it rarely appears in your code. Accept-Language carries the client’s preferred locales, which Unit 17’s next-intl reads for locale negotiation.

On the wire, the request states what it wants and the response answers, picking zstd and varying by encoding and cookie so the CDN keys correctly.

GET /api/invoices HTTP/3
Accept: application/json
Accept-Encoding: zstd, br, gzip
Cookie: session=abc...

HTTP/3 200 OK
Content-Type: application/json
Content-Encoding: zstd
Vary: Accept-Encoding, Cookie

Cache-Control and the common defaults

Cache-Control is the header an infrastructure layer reads first. Its value is a comma-separated list of directives, each a knob on what caches may do with the response. Seven directives are worth knowing:

• private vs. public: private means only the client’s own cache (the browser) may store the response; public lets shared caches (CDNs, corporate proxies) store it too. Authenticated responses are private; anonymous responses can be public.
• max-age=N: how long, in seconds, the response stays fresh and may be served without revalidation.
• s-maxage=N: same as max-age but only for shared caches. Use it when the CDN should hold the response longer than the browser does.
• no-store vs. no-cache: not synonyms. no-store forbids any cache from storing the response. no-cache allows storage but forces revalidation (a conditional request, covered shortly) on every read. Confusing these two is the most common cache bug in production.
• must-revalidate: once stale, the response must be revalidated, and intermediaries may not serve it stale as a fallback.
• stale-while-revalidate=N: once stale, the cache may serve the stale response and fetch a fresh one asynchronously. Keeps p99 latency low at the cost of one window of stale content.
• immutable: the response will never change at this URL, so browsers may skip revalidation entirely, even on reload. The marker for hashed static assets.

Four defaults cover almost every response, each named by its trigger:

• Authenticated HTML (any page that varies by signed-in user): private, no-store. No shared cache may touch it, and the browser must not store it either, so the back button doesn’t flash signed-in content after sign-out.
• Hashed static assets (filename includes a content hash, like /_next/static/chunks/main.a1b2c3.js): public, max-age=31536000, immutable. That max-age is one year. New deploys ship new hashes, so the URL is the version and the old one never needs to change.
• Cacheable HTML on the CDN edge (marketing pages, blog posts, anything anonymous): s-maxage=300, stale-while-revalidate=86400. Fresh on the CDN for five minutes, with a day of stale-while-revalidate behind that. The user always gets a fast response while the CDN refreshes in the background.
• API responses that change rapidly on a logged-in surface: private, no-store. Same as authenticated HTML, because the server can’t guarantee the response won’t change between two reads.

Next.js 16 wraps these strings in the 'use cache' directive with cacheLife(...) and cacheTag(...), so you rarely write Cache-Control by hand; Unit 4 covers it. For now, recognize the wire shape so you can read it in DevTools.

Caution

Ship public, max-age=... on a response that varies by signed-in user and a shared cache (Vercel’s edge, Cloudflare, a corporate proxy) can store one user’s response and serve it to another, leaking their data into another browser. Use private, no-store for authenticated HTML, and reach for public only when you can prove the response is identical for every user, signed in or out.

no-store and no-cache cause the most confusion. Here’s the difference on the wire.

no-store (forbid storage)

HTTP/3 200 OK
Cache-Control: private, no-store

Nothing stores this response: not the browser, not the CDN, not a corporate proxy. The next request re-runs the server work. This is the default for anything user-specific.

no-cache (allow storage, force revalidation)

HTTP/3 200 OK
Cache-Control: no-cache
ETag: "v1"

The cache may store the response but must revalidate before serving it, using a conditional request with If-None-Match: "v1". The next section covers the revalidation flow.

Now apply the defaults. Only one candidate is right; the other three each leak something different.

A Server Component renders an invoice list for the signed-in user at /invoices. The page is server-rendered, varies by the user’s organization, and the client navigates here from a sidebar link. What value should Cache-Control have on the response?

public, max-age=300

private, max-age=300

private, no-store

no-cache

Conditional requests: ETag and If-None-Match

A cached copy raises one question: the client already holds a version of this resource, but has it changed? Conditional requests answer it.

The server stamps the response with ETag: "v1", an opaque token identifying this version of the resource. On the next read, the client sends the token back as If-None-Match: "v1". If nothing has changed, the server replies 304 Not Modified with no body, and the client renders from its cached copy. You save the body, not the round-trip: the request still goes, the payload doesn’t come back.

Reach for this on read-mostly resources the client revisits: list endpoints, large JSON payloads, image metadata, anything that doesn’t change often enough to justify re-shipping the body. It pairs naturally with Cache-Control: no-cache, where every read is conditional by design.

If-Match is the write-side version, implementing optimistic concurrency on a mutation: “apply this only if the resource is still at version v1; otherwise return 412 Precondition Failed and let me reconcile.” Useful for collaborative-edit surfaces, but not an everyday reach. Last-Modified and If-Modified-Since express the same pattern with a timestamp; prefer ETag, since timestamps have only second precision and can’t tell apart two writes in the same second, while an ETag is opaque and the server can derive it however it likes (a row version, a content hash, a counter).

On the wire:

GET /api/invoices HTTP/3
If-None-Match: "v1"

HTTP/3 304 Not Modified
ETag: "v1"

Watch video

Software Developer Diaries walks every Cache-Control directive and the ETag/If-None-Match revalidation flow with diagrams (21 min).

Authorization: cookies for browsers, Bearer for machines

The first thing an application decides on every request is “who is this?”, and the answer rides on either a cookie or an Authorization header. Which one depends on who’s calling.

For first-party browser traffic, the answer is cookies. The browser sets a cookie at sign-in via Set-Cookie:, returns it via Cookie: on every same-origin request, and clears it at sign-out. It also enforces the protections cookies carry: HttpOnly keeps JavaScript from reading the value, Secure requires HTTPS, SameSite=Lax defangs CSRF on top-level navigations, and the __Host- prefix locks the cookie to one origin. Those mitigations are why a browser session belongs in a cookie, not a header your JavaScript writes by hand.

For programmatic clients (a mobile app, a server-to-server SDK, a public API consumer), the answer is Authorization: Bearer <token>. A bearer token is a raw credential sent in plaintext on every request, with none of a cookie’s browser-managed mitigations. That makes it right for a client that manages its own token lifecycle, stores credentials in a vault or keychain, and has no cookie store at all.

In this course the split is concrete: Better Auth ships the project’s session as a __Host--prefixed HttpOnly; Secure; SameSite=Lax; Path=/ cookie, while public route handlers called by non-browser clients read Authorization: Bearer <token>. Both coexist inside one web app.

Two cousins round this out. Authorization: Basic is Base64-encoded username:password, safe only over HTTPS and used only for internal tooling, admin scripts, and CI/CD jobs, never anything user-facing. WWW-Authenticate is its companion, the header a 401 sends to tell the client which scheme the server expects (WWW-Authenticate: Bearer).

Caution

The Authorization header carries a long-lived credential, and most observability tools log it by default. Configure your logger’s redaction list to strip Authorization and Cookie before any request reaches a log line.

The full Set-Cookie attribute surface, including Partitioned, gets its own deep dive later. Here are the two channels side by side.

Browser session (cookie)

GET /invoices HTTP/3
Cookie: __Host-session=abc...

Set once at sign-in by Set-Cookie, then sent automatically on every same-origin request. The browser manages the lifecycle.

Machine API (Bearer)

GET /api/v1/invoices HTTP/3
Authorization: Bearer sk_live_abc...

The SDK or HTTP client adds the header on every call. The token lives in the client’s config or vault, not a browser cookie store.

Rate-limit signaling and Retry-After

When a server tells the client to slow down, it sends 429 Too Many Requests and two headers that spell out how much: RateLimit (the current state) and RateLimit-Policy (the policy that produced it), standardized by the IETF as draft-ietf-httpapi-ratelimit-headers. Their values are structured fields per RFC 9651, lists of items with parameters parsed by a defined grammar, replacing the loose X-RateLimit-* headers. Upstash, Cloudflare, and most gateways emit this shape.

Retry-After is the back-off signal the client must honor. Its value is a number of seconds or an HTTP-date. It rides on a 429 from a rate limiter and on a 503 Service Unavailable from an overloaded server telling you to come back later.

On a 429, the client reads Retry-After and schedules a retry. That retry must be idempotency-aware: retrying a POST without an Idempotency-Key is the bug Methods and retry safety closed. On a 503, read Retry-After and either retry transparently or surface a “service unavailable” message, as the UX needs. Either way, the server picks the wait, not you.

A 429 with all three headers:

HTTP/3 429 Too Many Requests
RateLimit: "auth-sign-in";r=0;t=27
RateLimit-Policy: "auth-sign-in";q=10;w=60
Retry-After: 27
Content-Type: application/problem+json

Reading the structured fields: the auth-sign-in policy allows 10 requests (q=10) per 60-second window (w=60), with zero remaining (r=0) and 27 seconds to reset (t=27). Retry-After: 27 says the same thing in the form every HTTP client library already reads. The application/problem+json body explains why in human terms, using the RFC 9457 Problem Details from Status codes and Problem Details.

The security baseline: the irreducible six

Six headers form the security baseline every web app sends. All are browser-audience: the browser enforces them, the infrastructure layer ignores them, and the application produces them at response time. The aim here is recognition, so you can spot one in DevTools and know its job; Chapter 81 handles production wiring, including the per-request CSP nonce.

• Strict-Transport-Security (HSTS): “always use HTTPS for this host, for the next N seconds, including subdomains.” Closes the downgrade-to-HTTP window an attacker uses to strip TLS in a man-in-the-middle attack.
• Content-Security-Policy (CSP): “only execute scripts from these sources.” The recommended shape is nonce-based with 'strict-dynamic': every request gets a fresh nonce , the inline <script> tags carry it, and other scripts inherit trust from nonced scripts. Closes XSS as a class.
• X-Content-Type-Options: nosniff: “do not sniff the body to guess a Content-Type; trust the header.” Closes the MIME-confusion attack where a .txt upload is sniffed as text/html and executed in a browser context.
• Referrer-Policy: “what to put in Referer on outgoing navigations.” Use strict-origin-when-cross-origin: full URL on same-origin, origin only when cross-origin, nothing on HTTPS-to-HTTP. Closes the credential-in-URL leak through referer headers.
• Permissions-Policy: “which browser features (camera, microphone, geolocation, USB, payment) this origin and its iframes may use.” Deny by default, granting only the features the app actually needs.
• frame-ancestors: a CSP directive, not a separate header, answering “who may embed this page in an iframe.” It replaces the old X-Frame-Options header, since the directive composes with the rest of CSP.

Alongside the six sits Reporting-Endpoints, naming URLs the browser can POST CSP and Permissions-Policy violation reports to.

These headers are nearly static across requests, so most live in next.config.ts, the build-time setter. CSP is the exception: its nonce must be fresh on every request.

Request-context headers and the trust-the-edge rule

Every request carries a few headers the application reads to learn its context: who’s calling, from where, and through what. Five worth naming:

• Cookie:: the value the browser sends, carrying the session. The application reads it; the infrastructure layer doesn’t.
• User-Agent:: which client is calling. Treat it as a forensic and analytics signal only, never as an authorization input, since the client controls the value. In logs it helps segment a bot from a mobile browser from a server-to-server SDK.
• Referer:: the previous URL the user came from, used for analytics. (The header name has been misspelled since 1996 and that spelling is now canonical; only the related Referrer-Policy header spells it correctly.)
• Origin:: the load-bearing header for CSRF defense on Server Actions. On a cross-origin request with credentials, the browser sets Origin: to the calling origin, and a Server Action rejects any value that isn’t the app’s own origin.
• Host: (the :authority pseudo-header under HTTP/2 and HTTP/3): which hostname the client meant. Both layers read it: the infrastructure routes on it, and the application uses it for multi-tenant subdomain routing (org A on acme.app.com, org B on globex.app.com).

That’s the easy half. The hard half is recovering the real client IP through a chain of proxies without trusting a value the client could have forged.

A request usually passes through several hops before it reaches you: CDN, then load balancer, then application. Your application sees only the load balancer as the source IP. To recover the original client IP, each proxy appends the IP it received the request from to X-Forwarded-For (or to the structured Forwarded header from RFC 7239), and the application walks the chain.

The catch is which entries you can believe. Only the rightmost, appended by the edge your application talks to directly, is trustworthy. Every entry to its left could have been forged by the original client, which is free to send a fake X-Forwarded-For. Rate-limit by the leftmost entry and any attacker can spoof an IP to slip past your limiter. The fix: count proxies from the right and stop at the first untrusted hop. On Vercel, read request.headers.get('x-vercel-forwarded-for'), the value the edge controls; on a custom edge, configure the equivalent.

Forwarded (RFC 7239) is the standardized replacement for the older X-Forwarded-* set, folding X-Forwarded-For, -Proto, and -Host into one header: Forwarded: for=192.0.2.60;proto=https;by=203.0.113.43. Most upstream tooling still reads the legacy set, so read whichever header your edge actually sets and configure the edge to set the modern one.

Caution

Reading X-Forwarded-For without verifying which hop appended each entry is the most common spoofable-header bug in production rate limiters. Prefer the value your trusted edge appends under a vendor-specific name: x-vercel-forwarded-for on Vercel, similar on Cloudflare and Fastly.

Naming custom headers without the X- prefix

When you invent a header for a webhook signature, a request ID, or an internal contract, skip the X- prefix. RFC 6648 deprecated it in 2012: once a prefixed header became a standard, dropping the prefix broke every consumer that had hard-coded the old name.

Name a new header by the strongest convention that fits: an IETF draft shape where one exists (Idempotency-Key, RateLimit), a vendor token for third-party headers (Stripe-Signature, or Svix’s Svix-Id / Svix-Timestamp / Svix-Signature), and a bare name for your own (Request-Id).

Where headers are set in a Next.js app

The three audiences map onto the three places a Next.js app sets headers.

• next.config.ts headers(): for headers static across requests, or static per route prefix, configured once at build time. HSTS, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, frame-ancestors-via-CSP (when no nonce is needed), and Cache-Control on /_next/static.
• proxy.ts (the Next.js 16 rename of middleware.ts): for headers that need per-request data. The flagship case is the CSP nonce, injected into both the Content-Security-Policy header and the <script> tags that carry it. The request-id header, the rate-limit headers, and per-request auth headers live here too.
• The response itself, meaning the route handler (route.ts), the Server Action’s response, or the page’s metadata export: for headers that are response-specific. Cache-Control per route, the ETag per resource, WWW-Authenticate: Bearer on the 401 from authedRoute, and the Problem Details Content-Type: application/problem+json on a 422.

next.config.ts headers() Static across requests.

• Strict-Transport-Security
• X-Content-Type-Options
• Referrer-Policy
• Permissions-Policy

proxy.ts Per-request data (CSP nonce, request-id).

• Content-Security-Policy
• X-Request-Id
• RateLimit

route.ts / Server Action Per-response (per-route cache, per-resource ETag).

• Cache-Control
• ETag
• WWW-Authenticate
• Content-Type

The audience model decides which file owns each header.

Three read-only snippets anchor the diagram, one per setter. You won’t write these files until later units; for now, note the shape: which file, which export, and where the header strings live.

The build-time setter, next.config.ts: the headers() async function returns an array of rules, each pairing a source path matcher with a list of (key, value) pairs.

next.config.ts

const nextConfig = {
  async headers() {
    return [
      {
        source: '/(.*)',
        headers: [
          { key: 'Strict-Transport-Security', value: 'max-age=63072000; includeSubDomains; preload' },
          { key: 'X-Content-Type-Options', value: 'nosniff' },
          { key: 'Referrer-Policy', value: 'strict-origin-when-cross-origin' },
        ],
      },
    ];
  },
};

export default nextConfig;

The per-request setter, proxy.ts at the project root: the proxy function runs on the Node.js runtime before the route resolves. Here it generates a per-request nonce and sets it on the response’s Content-Security-Policy and X-Request-Id headers.

proxy.ts

import { NextResponse, type NextRequest } from 'next/server';

export function proxy(request: NextRequest) {
  const nonce = crypto.randomUUID();
  const response = NextResponse.next();
  response.headers.set(
    'Content-Security-Policy',
    `script-src 'self' 'nonce-${nonce}' 'strict-dynamic'`,
  );
  response.headers.set('X-Request-Id', nonce);
  return response;
}

The snippet uses crypto.randomUUID() for parity with the Idempotency-Key in Methods and retry safety; the canonical CSP nonce is instead a fresh random value encoded as base64.

The per-response setter, the route handler at app/api/<route>/route.ts: each method is a named export (GET, POST, PATCH, and so on). Headers go on the Response object, so they ride with the body the handler returns.

app/api/invoices/route.ts

export async function GET() {
  return new Response(JSON.stringify({ invoices: [] }), {
    headers: {
      'Content-Type': 'application/json',
      'Cache-Control': 'private, no-store',
      ETag: '"v1"',
    },
  });
}

Check your understanding

Mark each claim true or false.

Each claim is about a header the application sends or reads. Mark each statement True or False.

Cache-Control: no-cache forbids the cache from storing the response, so it’s the default for authenticated HTML.

no-cache allows storage but forces revalidation on every read. The directive that forbids storage entirely is no-store. The default for authenticated HTML is private, no-store.

A first-party browser session uses cookies; a server-to-server API call uses Authorization: Bearer.

Browser sessions live in cookies because the browser enforces HttpOnly, Secure, SameSite, and the __Host- prefix, mitigations a bearer token doesn’t have. Programmatic clients use bearer because they manage their own credential lifecycle and shouldn’t store credentials in a cookie store.

Rate-limiting by the leftmost IP in X-Forwarded-For is safe because the proxies append in order.

The only hop you can trust is the immediate edge. Every value to the left of that hop could have been forged by the original client. Count from the right and stop at the first untrusted hop, or read the value your trusted edge appended (e.g. Vercel’s x-vercel-forwarded-for).

Vary: Cookie on a per-user response tells a shared cache to key the cache by the request’s Cookie value, so user A’s response doesn’t get served to user B.

Without Vary: Cookie, the cache keys by URL only and serves the first response it cached to every user that hits that URL. The auth default private, no-store makes this moot, since the response isn’t cached at all, but on cacheable per-user surfaces Vary is mandatory.

Custom application headers should be prefixed with X- to mark them as non-standard.

RFC 6648 deprecated the X- prefix in 2012. Use no prefix (e.g. Request-Id, Idempotency-Key) or a vendor token (Stripe-Signature, Svix-Id).

The CSP nonce is set in proxy.ts rather than next.config.ts because each request needs a fresh nonce.

next.config.ts headers() is for static-across-requests headers. CSP with a per-request nonce needs request-time data, which means the proxy. The same rule applies to any request-id, rate-limit, or per-tenant header.

False True

External resources

MDN — HTTP headers

developer.mozilla.org

The canonical reference for every HTTP header — syntax, semantics, browser support.

MDN — HTTP caching guide

developer.mozilla.org

Long-form guide to Cache-Control, ETag/If-None-Match, Vary, and the private/shared cache split.

OWASP Secure Headers Project

owasp.org

Canonical checklist of recommended response headers — HSTS, CSP, Referrer-Policy, Permissions-Policy, and more.

IETF httpapi — RateLimit headers draft

datatracker.ietf.org

Standards-track draft for the structured RateLimit and RateLimit-Policy response headers.