# Sitemap pipeline

## Shape

`/sitemap.xml` is a sitemap **index**, not one enormous file. Children are per content type and paged:

```
/sitemaps/questions-1.xml
/sitemaps/topics-1.xml
/sitemaps/categories.xml
/sitemaps/tags.xml
/sitemaps/profiles-1.xml
```

A child is listed in the index only when it would contain at least one URL.

## Why an index

A single file mixing content types starves whatever comes last: at 45,000 questions, a flat sitemap has no room left for topics, taxonomy or profiles. Splitting by type means each type is discovered independently, and question growth cannot push discussions out of the index.

## Paging

Page size is 20,000 URLs, comfortably under the 50,000 protocol limit.

Questions, topics and profiles are queried with `LIMIT`/`OFFSET` per page. Categories and tags are capped at the same page size; when a cap is reached the operator is warned through the file logger with the limit named, rather than URLs disappearing silently.

## Inclusion rules

| Content | Included when |
|---|---|
| Question | Published, has a slug, not `noindex`, canonical is same-site or absent |
| Topic | Published, has a slug |
| Category | Not `noindex` **and** has ≥1 published question **or** ≥1 published topic |
| Tag | Not `noindex` and has ≥1 published question |
| Profile | Active real member and meets the profile index threshold |

The category rule matters: after topics were introduced, a category holding only discussions is a real page. Requiring a question would have hidden it.

## `lastmod`

Reports meaningful change:

| Content | Source |
|---|---|
| Question | Publication or update time |
| Topic | `last_activity_at`, falling back to update then creation |
| Category, tag | Update time |
| Profile | Update time |

## Never included

Admin, authentication, account, notification, search and API URLs. Non-canonical variants. `noindex` records. Records with an off-site canonical override.

## Base path and origin

Every URL is built through the canonical URL builder, so a subdirectory install emits `/community/q/...` and not `/q/...`.

## Extending

`sitemap.entries` filters a generated entry list. A plugin can add its own public URLs or remove entries it knows are not indexable. Entries must stay well-formed; the renderer does not repair them.
