Skip to content
LEARNREACTSAAS
Chapter 34Lesson 7

Robots, sitemaps, icons, viewport

Build a SaaS app's site-level SEO and platform bundle with Next.js file conventions.

The Acme app renders. Every page carries its title, description, and social card, all wired up in the last lesson, Metadata and OG cards. So you ship it to a staging URL, run a pre-launch SEO audit, and the report comes back with gaps that belong to no single page:

  • No robots.txt, so crawlers have no instructions.
  • No sitemap.xml, so nothing tells crawlers which URLs matter or when they last changed.
  • The browser tab shows the default Vercel favicon, not the Acme mark.
  • Adding the app to an iPhone home screen yields a blurry screenshot, not an icon.
  • The mobile address bar is plain grey instead of the brand color.
  • The dangerous one: the staging deploy is fully indexable, so Google is about to put your unfinished app in its index, competing with production for the same keywords.

These are site-level artifacts: the standards-mandated files a crawler, operating system, or mobile browser expects to find at well-known paths near the root of your site, regardless of which page someone is on. The old web answered each by hand, dropping a robots.txt in /public, generating a sitemap.xml with a script, exporting a favicon at ten sizes, pasting ten <link> tags into <head>, all of it remembered and updated by hand.

Next.js replaces the lot with file conventions: typed functions you write near the root of app/. Each file shares the same four properties. It is discovered by filename, so a reserved name like robots.ts owns its output with no registration. It returns a typed value, a Robots or Sitemap object that is autocompleted and type-checked, so a typo is a build error. It is dynamic, free to read the environment or query the database. And it is cached by default, run once at build and served from the edge, unless you reach for a request-time API, which flips it to per-request, the same Cache Components rule you have followed all chapter. The only thing that changes from file to file is which artifact it owns.

What each filename emits

Section titled “What each filename emits”

On the left are the files as they sit in app/; on the right is what the platform emits for each.

  • Directoryapp/
    • layout.tsx export const metadata + export const viewport
    • robots.ts
    • sitemap.ts
    • favicon.ico
    • icon.png
    • apple-icon.png
    • manifest.ts
    • opengraph-image.png
File in app/
What the platform emits
robots.ts
served at/robots.txt
sitemap.ts
served at/sitemap.xml
favicon.ico
<link rel="icon" href="/favicon.ico" sizes="any">
icon.png
<link rel="icon" href="/icon?…">— size + type inferred
apple-icon.png
<link rel="apple-touch-icon" href="/apple-icon?…">
manifest.ts
<link rel="manifest">+ served at/manifest.webmanifest
opengraph-image.png
og:image recap
export const viewport
<meta name="viewport">+<meta name="theme-color">

The platform discovers each file, runs it at build, caches the result, and either serves it at a standard path or injects the head tag. No hand-edited <head>, no files dropped in /public.

We walk it top to bottom: the two crawl-control files, the icons, the viewport export, then the manifest. A short section names the property they all share, cached by default, and a final one assembles everything into the real app/ directory.

robots.ts: keep previews out of the index

Section titled “robots.ts: keep previews out of the index”

A crawler arriving at app.acme.com looks first for /robots.txt: a plain-text file listing which paths it may fetch and where to find your sitemap. It follows the Robots Exclusion Standard , and in Next.js you don’t hand-write it. You write app/robots.ts, a default-exported function returning a typed Robots object, and the platform serves the rendered text at /robots.txt.

The minimal shape declares which user-agents may crawl what, then points at your sitemap and canonical host:

app/robots.ts
import type { MetadataRoute } from 'next';


export default function robots(): MetadataRoute.Robots {
  return {
    rules: { userAgent: '*', allow: '/', disallow: '/api' },
    sitemap: 'https://app.acme.com/sitemap.xml',
    host: 'https://app.acme.com',
  };
}

userAgent: '*' matches every crawler, allow: '/' opens the whole site, disallow: '/api' keeps bots out of your route handlers, and sitemap and host hand over your map and canonical origin. Because the return type is MetadataRoute.Robots, a misspelled dissallow is a red squiggle rather than a line your text file silently ignores.

Note the export default. The course uses named exports everywhere else, but these file conventions are the carve-out: the framework finds them by filename and expects a default export, exactly like layout.tsx, page.tsx, and last lesson’s opengraph-image.tsx. Every file in this lesson follows that same framework-dictated shape.

The shape is mechanical; the one decision worth your attention is the branch, because it causes a real production incident. Every push to a feature branch gets its own Vercel preview URL, a full production build at a public address. If robots.ts returns allow: '/' unconditionally, that preview is crawlable: Google indexes your half-finished pages, and those preview URLs then compete with your real domain for the same keywords. That is duplicate content , and the staging copy can outrank production.

To prevent it, robots.ts reads which environment it is running in and branches:

import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function robots(): MetadataRoute.Robots {
  if (process.env.VERCEL_ENV !== 'production') {
    return { rules: { userAgent: '*', disallow: '/' } };
  }


  return {
    rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/dashboard/'] },
    sitemap: `${ORIGIN}/sitemap.xml`,
    host: ORIGIN,
  };
}

The whole file turns on this line. Vercel sets process.env.VERCEL_ENV to 'production', 'preview', or 'development', and every value but 'production' takes the early return.

import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function robots(): MetadataRoute.Robots {
  if (process.env.VERCEL_ENV !== 'production') {
    return { rules: { userAgent: '*', disallow: '/' } };
  }


  return {
    rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/dashboard/'] },
    sitemap: `${ORIGIN}/sitemap.xml`,
    host: ORIGIN,
  };
}

For any non-production build, return one rule: disallow: '/' blocks every crawler from every path, with no sitemap and no host. Previews stay invisible.

import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function robots(): MetadataRoute.Robots {
  if (process.env.VERCEL_ENV !== 'production') {
    return { rules: { userAgent: '*', disallow: '/' } };
  }


  return {
    rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/dashboard/'] },
    sitemap: `${ORIGIN}/sitemap.xml`,
    host: ORIGIN,
  };
}

Only production reaches the real rules. allow: '/' opens the public surface; disallow carves out the paths bots should never touch: the API and the signed-in dashboard.

import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function robots(): MetadataRoute.Robots {
  if (process.env.VERCEL_ENV !== 'production') {
    return { rules: { userAgent: '*', disallow: '/' } };
  }


  return {
    rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/dashboard/'] },
    sitemap: `${ORIGIN}/sitemap.xml`,
    host: ORIGIN,
  };
}

sitemap and host point at the absolute production origin. Hard-code it rather than deriving it from the request host: on a preview the request host is the throwaway preview URL, which you never want to hand a crawler as canonical.

1 / 1

The discriminator on line 6 is where people slip. The obvious instinct is process.env.NODE_ENV === 'production', but on Vercel a preview is a production build, so NODE_ENV is 'production' on previews too, leaving every preview indexable. Only VERCEL_ENV separates the three environments. This raw process.env read is stringly-typed and easy to mistype, so a later chapter moves it into a validated, typed env object.

Get the branch wrong the other way and the stakes are just as high: ship disallow: '/' to production and you tell Google to delist your entire site. Nothing errors, the build is green, the page renders. Weeks later organic traffic quietly craters, and that combination of invisible and slow-to-surface makes it one of the hardest failures to catch.

sitemap.ts: list the URLs you want crawled

Section titled “sitemap.ts: list the URLs you want crawled”

Where robots.txt tells crawlers where they may go, the sitemap tells them where to look. You write app/sitemap.ts, return an array of entries, and Next.js emits valid sitemap XML at /sitemap.xml.

The shape is an array, one entry per URL:

app/sitemap.ts
import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function sitemap(): MetadataRoute.Sitemap {
  return [
    { url: `${ORIGIN}/`, changeFrequency: 'weekly', priority: 1 },
    { url: `${ORIGIN}/pricing`, changeFrequency: 'monthly', priority: 0.8 },
    { url: `${ORIGIN}/blog`, changeFrequency: 'daily', priority: 0.6 },
    { url: `${ORIGIN}/help`, changeFrequency: 'weekly', priority: 0.5 },
  ];
}

Each url must be absolute; relative paths are invalid in a sitemap. changeFrequency and priority are hints crawlers weigh loosely. lastModified is the field that earns its keep, and it is omitted here because static marketing pages have nothing meaningful to put in it.

A real web app’s public surface is more than four hand-listed routes: every published blog post, every public help article, every public profile, and the set grows on every publish. Hard-code those URLs and the sitemap goes stale the moment new content ships. So the function goes async and queries the indexable rows instead:

app/sitemap.ts
export default function sitemap(): MetadataRoute.Sitemap {
  return MARKETING_ROUTES.map((path) => ({
    url: `${ORIGIN}${path}`,
    changeFrequency: 'weekly',
  }));
}

Fine until content ships. Four routes, known at build, never stale because they never change. A blog or help center breaks this the first time someone publishes.

Two things about the database version matter beyond the syntax. First, listPublicHelpArticles() is a read helper from your data layer, not an inline query: the course keeps every database read behind a verb-named function in db/queries/, so the call site stays clean and the query lives in one place.

Second, the rule that keeps a sitemap correct: only publicly indexable rows belong in it. No authenticated data, no tenant-scoped data, nothing behind a login. A private invoice URL leaks the fact that it exists and is useless anyway, since a crawler can’t fetch a page it isn’t allowed to reach.

Like the other files in this lesson, sitemap.ts is static by default: the build renders it once and the CDN serves that artifact to every crawler. A later section covers this property in full.

One scale note for recognition. Google rejects a single sitemap over 50,000 URLs or 50 MB; past that, generateSitemaps splits the output into numbered files plus an index. Most web apps never come close.

Icons: favicon, icon, and apple-icon

Section titled “Icons: favicon, icon, and apple-icon”

A favicon is the small mark in a browser tab, and the same idea now stretches across devices: a crisp tab icon, a larger one for Android and PWA installs, and a dedicated tile for the iOS home screen. Next.js handles all of it through three reserved filenames. Drop the image files in app/, and the platform fingerprints them, caches them, and injects the right <link> tags.

Filename
Formats
Emits
favicon.ico
.ico only
<link rel="icon" href="/favicon.ico" sizes="any">
icon.*
.ico.jpg.jpeg.png.svg
<link rel="icon"> type + sizes inferred from the file
apple-icon.*
.jpg.jpeg.png (raster only)
<link rel="apple-touch-icon">

favicon.ico lives only at the root of app/; icon and apple-icon can sit in any route segment to override per-section. Apple touch icons must be raster: no SVG, no .ico.

For almost every app, a static image is the whole job: drop it in app/, and the platform hashes its bytes for cache-busting, infers type and sizes from the file itself, and emits the <link> tag with no stale icon after a deploy.

Real devices want more than one size, though, since a 32-pixel tab icon and a 512-pixel install icon are different files. Add a numeric suffix to set several: icon.png, icon1.png, icon2.png, sorted lexically. The names carry no dimensions, so the platform reads each file’s real pixel size and infers sizes from the bytes.

  • Directoryapp/
    • favicon.ico legacy single-file favicon, broadest browser support
    • icon.png high-res mark, 512×512, crisp tabs and PWA splash
    • icon1.png mid-size mark, 192×192, Android / PWA installs
    • apple-icon.png iOS home-screen tile, raster only

One constraint comes from iOS, not Next.js: the home screen scales your apple-icon up to fill its tile, so an undersized source renders blurry. Next.js will not reject a small file, so the only symptom is a fuzzy icon on the phone. Ship a comfortably large raster apple touch icon , around 180×180 or larger, in png or jpg.

You can also generate an icon: name it icon.tsx or apple-icon.tsx and default-export a function returning an ImageResponse, the generated-image API from the last lesson. It fits narrow cases like per-organization branded favicons; reach for the static file otherwise.

The social cards from the last lesson follow the same pattern. A root-level opengraph-image.png, paired with opengraph-image.alt.txt, is the brand-wide fallback every route inherits unless it overrides, and it belongs in this site-level bundle. Keep it root-only: drop opengraph-image files into several nested layouts and they collide with no error telling you which won.

The PWA install manifest, defined in the next section, points at exactly the icon set you just declared.

The viewport export: the field that isn’t metadata

Section titled “The viewport export: the field that isn’t metadata”

One field here trips up nearly everyone: it looks like metadata but isn’t. It controls how your page renders on mobile and what color tints the browser chrome, and putting it where it seems to belong, inside metadata, triggers a build warning. So here is the rule first:

Viewport-affecting fields go in a separate export const viewport, never inside metadata.

Those fields are width, initialScale, themeColor, and colorScheme. They don’t describe your page to crawlers the way metadata does; they drive a different family of <meta> tags, viewport, theme-color, and color-scheme, that control how the browser physically renders the page on a device. Different job, so Next.js gives them their own typed export. Here is the canonical shape:

app/layout.tsx
import type { Viewport } from 'next';


export const viewport: Viewport = {
  width: 'device-width',
  initialScale: 1,
  themeColor: '#0f172a',
  colorScheme: 'light dark',
};

Each field does one job. width: 'device-width' with initialScale: 1 is the viewport meta default; the scaffold ships it, and you should not fight it. colorScheme: 'light dark' declares that the app supports both color schemes. And themeColor is the theme-color , the gap the audit flagged.

themeColor carries the one real decision here. A single color string tints the chrome the same way regardless of theme. But a web app with both a light and a dark theme wants the address bar to match the active theme: dark chrome in dark mode, light chrome in light mode. For that, themeColor takes an array of { media, color } entries, each keyed on a color-scheme media query:

app/layout.tsx
export const viewport: Viewport = {
  width: 'device-width',
  initialScale: 1,
  colorScheme: 'light dark',
  themeColor: [
    { media: '(prefers-color-scheme: light)', color: '#ffffff' },
    { media: '(prefers-color-scheme: dark)', color: '#0f172a' },
  ],
};

The single string is the simple default; the array is the better pick for a theme-switching app, so the chrome never clashes with the surface beneath it.

The mistake is easy to make and easy to miss. Putting a viewport field inside metadata doesn’t error: the build succeeds and the tag may even still emit. It only warns, and the warning is easy to scroll past. But it’s the deprecated path, flagged since Next 14 through 16, and it will eventually break. Seeing the two shapes side by side makes the fix clear:

app/layout.tsx
export const metadata: Metadata = {
  title: 'Acme',
  description: 'Invoicing for small teams',
  themeColor: '#0f172a',
  viewport: { width: 'device-width', initialScale: 1 },
};

Deprecated, silently. themeColor and viewport inside metadata warn at build, not error. The page still builds, so it’s easy to miss. This shape is on the way out.

There is a generateViewport function, the dynamic counterpart to generateMetadata, for the rare case of a route-dependent viewport such as a per-locale colorScheme. It takes the same Promise-shaped params. But viewport affects the initial paint and cannot be streamed, so a generateViewport that reads runtime data blocks the entire route. Keep viewport a static object; for the common case it’s the only right answer.

The web manifest: add to home screen

Section titled “The web manifest: add to home screen”

The last gap is the home-screen experience. When someone taps “Add to Home Screen” on the Acme app, the operating system needs a name, an icon, and a color to build the launcher tile and splash screen. That information lives in a web app manifest , and like everything else in this lesson it follows a file convention: app/manifest.ts, a default-exported function that returns a typed manifest object. The platform serves it at /manifest.webmanifest and injects <link rel="manifest">.

A minimal manifest reuses the brand strings and icon set you already declared:

app/manifest.ts
import type { MetadataRoute } from 'next';


export default function manifest(): MetadataRoute.Manifest {
  return {
    name: 'Acme — Invoicing for small teams',
    short_name: 'Acme',
    description: 'Send and track invoices.',
    start_url: '/',
    display: 'standalone',
    background_color: '#0f172a',
    theme_color: '#0f172a',
    icons: [
      { src: '/icon.png', sizes: '512x512', type: 'image/png' },
      { src: '/icon1.png', sizes: '192x192', type: 'image/png' },
    ],
  };
}

This costs almost nothing and unlocks real value: the app becomes installable, with a home-screen icon, a standalone display that drops the browser chrome, and splash colors that match your brand. The icons array points at the 192 and 512 marks you already shipped, the two sizes installers reach for.

Mind the boundary: a manifest is install metadata, not a full PWA. It gives you “Add to Home Screen” and a branded launcher; it does not make your app work offline. That cheap win is the whole point here.

The alternative is a static public/manifest.webmanifest, but the .ts convention wins for the same reason every other file in this lesson does: it is typed, and it pulls brand strings and the icon list from shared config instead of duplicating them in hand-maintained JSON.

These are cached route handlers: keep them pure

Section titled “These are cached route handlers: keep them pure”

Every file you just met, robots.ts, sitemap.ts, icon.tsx, apple-icon.tsx, manifest.ts, and opengraph-image.tsx, is a special route handler. Under Cache Components, all of them are statically generated at build and served from the CDN by default, with zero per-request work. A crawler hitting /sitemap.xml, a browser fetching /icon.png, a phone reading /manifest.webmanifest: each is a cache hit on a file the build produced once.

They flip to dynamic only if you make them, through the same trigger as the rest of the chapter: read a request-time API (cookies(), headers(), an uncached external fetch) or set a dynamic route-segment config, and the file re-renders on every request because its output now depends on the request. A sitemap.ts that calls cookies() runs its database query on every crawl. Viewport is the one place this bites harder, not softer.

It bites harder because viewport can’t be streamed: it gates the initial paint, so a runtime-reading generateViewport blocks the whole route with no static shell. A dynamic sitemap costs one invocation per crawl, but a dynamic viewport costs your page its first paint. If you truly need external (not request-time) data here, cache that read inside the function; almost always, the right answer is to keep viewport a static object.

Sort each special file by whether the platform caches it at build, or re-runs it on every request. Drag each item into the bucket it belongs to, then press Check.

Stays static Built once, served from the CDN
Flips dynamic Re-runs on every request
sitemap.ts returning a hardcoded array of routes
robots.ts branching on process.env.VERCEL_ENV
A static apple-icon.png file
sitemap.ts that calls cookies()
icon.tsx that reads headers()
opengraph-image.tsx that fetches uncached data per request

That is the line to carry out of this lesson: reading the environment happens at build time and stays static, which is why the env-aware robots.ts is still cached, while reading the request, cookies, headers, or live data, opts you out.

One corollary applies to the generated-image files. A dynamically generated OG card or icon rendered through ImageResponse is cold on its first request, taking a few hundred milliseconds before the result is cached, so a bot that scrapes the URL before anyone warms it waits. A post-deploy hook that pings your key OG URLs warms the CDN before traffic arrives. Static image files never need this; they are already bytes on disk.

Assembling the root SEO bundle

Section titled “Assembling the root SEO bundle”

Here is the complete root bundle for Acme as one concrete app/ directory, the map from the start of the lesson with every row filled in:

  • Directoryapp/
    • layout.tsx export const metadata (last lesson) + export const viewport (this lesson)
    • robots.ts env-aware crawl control, blocks indexing off production
    • sitemap.ts marketing routes + public help articles
    • favicon.ico legacy favicon, broadest browser support
    • icon.png high-res mark, 512×512
    • icon1.png mid mark, 192×192, Android / PWA installs
    • apple-icon.png iOS home-screen tile, raster only
    • manifest.ts install metadata, reuses the brand strings + icons
    • opengraph-image.png brand social-card fallback (last lesson)
    • opengraph-image.alt.txt alt text for the fallback card

One layout.tsx carries both the metadata and viewport exports; every other file owns a single site-level artifact. All ten are typed, code-generated, cached by default, and discovered by filename, with no hand-edited <head> and nothing dropped in /public.

The files that carry logic, assembled in one place, with the single decision each one embodies called out below it:

app/robots.ts
import type { MetadataRoute } from 'next';


const ORIGIN = 'https://app.acme.com';


export default function robots(): MetadataRoute.Robots {
  if (process.env.VERCEL_ENV !== 'production') {
    return { rules: { userAgent: '*', disallow: '/' } };
  }


  return {
    rules: { userAgent: '*', allow: '/', disallow: ['/api/', '/dashboard/'] },
    sitemap: `${ORIGIN}/sitemap.xml`,
    host: ORIGIN,
  };
}

The one decision: gate indexing on VERCEL_ENV so previews never leak into the index.

Every file in that directory is typed, code-generated, cached by default, and discovered by filename. Pair this site-level map with the per-page metadata from the last lesson and you have the complete SEO surface for a new SaaS.

External resources

Section titled “External resources”

This is a reference-heavy surface, full of exact filenames, supported formats, and object field names, the kind of thing you look up rather than memorize. These are the canonical Next.js pages to return to.

Metadata Files — file conventions reference
nextjs.org

The index of every metadata file convention — the authoritative map for this whole lesson.
sitemap.xml
nextjs.org

The Sitemap object shape, generateSitemaps, and the 50,000-URL limit.
robots.txt
nextjs.org

The Robots object — rules, sitemap, host.
App Icons — favicon, icon, apple-icon
nextjs.org

Supported formats, the numeric-suffix convention for multiple sizes, and dynamic generation.
generateViewport
nextjs.org

The viewport export and its dynamic counterpart, including the no-streaming constraint.
robots.txt — Introduction and guide
developers.google.com

Google's own take on the standard your robots.ts emits — including why a disallow is a request, not a lock.
Web app manifest — MDN reference
developer.mozilla.org

The platform-neutral spec behind manifest.ts — every member, display modes, and theme_color.