fix(seo): make the sitemap canonical-only, healthy, and testable

[KevinVandy]

Problem

The production sitemap currently contains 3,245 URLs. A live audit found several categories of non-canonical or unhealthy entries:

• 189 Charts catalog URLs use trailing slashes and immediately redirect to the no-trailing-slash canonical.
• /showcase redirects to a default query-string URL before canonicalizing back to /showcase.
• Four /drafts/ URLs and two README URLs are included.
• Draft and README examples currently return 200 pages headed Content temporarily unavailable.
• A stratified check of 160 sitemap entries found 23 redirects and one 404, including stale DB reference paths that redirect to a parent reference page or no longer resolve.
• Only 77 entries currently contain lastmod values; docs and most other content do not.
• Approved showcase detail pages are not included even though later showcase pagination is currently noindex/nofollow.

Google says sitemap entries should be the canonical URLs the site wants indexed: https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap

Relevant generation code: src/utils/sitemap.ts

Required changes

• Emit only URLs that return 200, are indexable, and self-canonicalize.
• Normalize Charts catalog entries to the production no-trailing-slash form.
• Exclude drafts, *.draft paths, README files, temporary-unavailable placeholders, hidden libraries, and other non-public docs artifacts.
• Apply docs redirect resolution before adding manifest paths to the sitemap.
• Remove stale or missing generated-reference paths.
• Reconcile /showcase so the sitemap entry does not require an avoidable redirect.
• Add approved showcase detail pages if they are intended for organic discovery.
• Keep query-string URLs only when they represent an intentionally indexable canonical page, such as the previous-partners directory.
• Add accurate lastmod values where the source provides a real meaningful modification date; do not synthesize timestamps on every request.

Automated validation

Add a sitemap contract checker that can validate generated entries and optionally check a deployed base URL.

For every sitemap URL, verify:

• Final status is 200 without following a redirect.
• Exactly one canonical link exists.
• Canonical equals the sitemap URL after consistent URL normalization.
• No robots noindex directive is present.
• The page is not a temporary-unavailable or error placeholder.
• The URL has no disallowed draft, README, embed, account, or admin pattern.

Acceptance criteria

• Production sitemap contains zero redirecting URLs.
• Production sitemap contains zero 4xx or 5xx URLs.
• Production sitemap contains zero noindex URLs.
• Every entry is self-canonical.
• Drafts and repository-maintenance documents are absent.
• Charts catalog URLs use the same trailing-slash policy as their canonical pages.
• Unit tests cover path filtering and canonical normalization.
• A CI or release-time check prevents regressions in the sitemap contract.