You’ve built it. You’ve deployed it. Now, how do you get people to find it? The answer lies in Search Engine Optimization (SEO), and the first step is telling Google exactly what pages exist on your site.

The Indexing Flow

Here is how the discovery process works:

Step 1: The Map (Sitemap)

Astro has an official integration that makes generating a sitemap automatic.

First, install the integration:

|

Then, open your astro.config.mjs. It is crucial that you define your site URL here, otherwise the sitemap cannot be generated.

// astro.config.mjs
import { defineConfig } from 'astro/config';
import sitemap from '@astrojs/sitemap';

export default defineConfig({
  site: 'https://your-awesome-site.com', // <--- IMPORTANT!
  integrations: [sitemap()],
});

Step 2: The Signpost (robots.txt)

Now we need a signpost to tell search bots where the map is. Create a file named robots.txt in your public/ folder.

User-agent: *
Allow: /

sitemap: https://your-awesome-site.com/sitemap-index.xml

This tells all bots (User-agent: *) that they are allowed to crawl the site, and provides the direct link to your sitemap.

Step 3: Verification

Run a build of your site to verify everything is working.

|

Check your dist/ folder. You should see a sitemap-index.xml and sitemap-0.xml.

Step 4: Submission (Google Search Console)

Finally, we need to hand-deliver this map to Google.

1. Go to Google Search Console.
2. Add your property (domain).
3. In the sidebar, click on Sitemaps.
4. Enter sitemap-index.xml and click Submit.