Exclude /registry/ placeholder from www sitemap (fixes cross-sitemap lastmod duplicate)

[workprentice]

What

content/registry.md is excluded from the www sitemap via a new sitemap_exclude front-matter flag, wired into layouts/_default/sitemap.xml. Documented in CONTRIBUTING.md.

Why

https://www.pulumi.com/registry/ is the only URL on pulumi.com that shows up in two different submitted sitemaps at once, with lastmod values 861+ days apart:

Source	lastmod
www sitemap (Hugo, this repo, content/registry.md)	2023-12-11T12:53:57-08:00
Registry app's own /registry/sitemap.xml	2026-04-22T16:45:33.000Z
Live origin Last-Modified header on https://www.pulumi.com/registry/	rebuilds within hours (observed same-day)
Search Console last crawl of the URL	within the last day

The 2023-12-11 value is also the single oldest lastmod in the entire 2,738-URL www sitemap, next oldest being 831 days.

content/registry.md is not a real content page. Its own front matter says so: it exists only so S3 has an object to hand to CloudFront at registry/index.html, so CloudFront can then forward the actual request on to the Registry app, which is what really serves /registry/. Google never sees the HTML Hugo renders here. Confirmed the git history is genuinely frozen at that 2023 commit (2c85e02c, "Replace infinite redirect with placeholder page (#10300)") via an unshallowed clone, so this is not a rollup-fix gap like the one PR #21036 closed for /blog/ -- this page has no descendants to roll up from, and its content is not the descendant's content either.

The follow-up card that flagged this asked whether to fix it with an explicit lastmod front-matter override refreshed by a script, or accept the git date as inherently uninformative. Neither is right: a script-maintained date on a deliberately empty file still leaves two sitemaps disagreeing about one canonical URL, and accepting the status quo leaves the property's only cross-sitemap duplicate and its single oldest lastmod in place. The Registry app's sitemap already carries this URL with an accurate value, so the correct fix is to stop claiming it a second time here.

Why not reuse private or block_external_search_index

Both of those existing front-matter flags also add a noindex meta tag (see layouts/partials/head.html). /registry/ is correctly indexed today (Search Console: Submitted and indexed, crawled within the last day) and should stay that way -- only the sitemap entry is wrong, not the page's indexability. sitemap_exclude is a new, narrowly-scoped flag that touches only layouts/_default/sitemap.xml's inclusion check.

Impact, honestly sized

This is a lastmod-trust fix, not a traffic play. Search Console already shows Google crawling and ranking /registry/ and the wider registry section fine (90d: landing page 382 clicks / 9.8K impressions / avg pos 13.0; the whole /registry/* section 29.9K clicks / 4.95M impressions / avg pos 7.2, driven by the long tail of individual provider pages). Google's own guidance is that it uses <lastmod> "if it's consistently and verifiably... accurate" (Build and Submit a Sitemap); a URL that contradicts itself across two sitemaps on the same property is exactly the kind of signal that erodes that trust property-wide, including across the 59,633 URLs in the registry sitemap where lastmod-driven crawl scheduling is doing real work.

Verification

Full local hugo --gc --minify -e production build, before and after this change, diffing the generated sitemap.xml URL sets:

$ diff sitemap_before.txt sitemap_after.txt
1973d1972
< https://www.pulumi.com/registry/

Exactly one URL removed, nothing else changed (2,162 -> 2,161 URLs). public/registry/index.html still renders (92,876 bytes) after the change, so the CloudFront placeholder trick this file exists for is unaffected.

Out of scope (follow-up cards filed on the Marketing Content Calendar board)

• The Registry app's own landing-page lastmod (2026-04-22) is itself git-date-based on a rarely-touched file and doesn't track the page's actual live content -- lower priority since it's not wrong, just not a great freshness signal.
• A duplicated <title>Pulumi Registry | Pulumi Registry</title> tag on the live registry landing page, traced to themes/default/layouts/partials/head.html in the pulumi/registry repo.
• The nested sitemap index at /registry/sitemap.xml (a <sitemapindex> pointed to by /sitemap-index.xml, itself pointing to two more sitemaps) -- Google has stated (via John Mueller) that nested sitemap indexes are not supported.

---

🧠 This PR was created by workprentice.

[github-actions]

Pre-merge Review — Last updated 2026-08-26T13:05:46Z

Tip

Summary: This is a small SEO/infrastructure change: it adds a sitemap_exclude front-matter flag to layouts/_default/sitemap.xml, sets it on the content/registry.md build-time placeholder (so the www sitemap stops emitting a stale 2023 lastmod for a URL the Registry app's own sitemap already covers), and documents the new field in CONTRIBUTING.md. The wrongness that would matter here is a template edit that drops the wrong pages from the sitemap, or contributor documentation that describes the flag inaccurately — I read the full sitemap template (the new condition sits in the same or chain as private / block_external_search_index / redirect_to, and the inner RegularPagesRecursive loop only computes a max lastmod, so nothing else changes), confirmed this is the site's only sitemap template, and verified the placeholder's frozen lastmod and its sitemap_exclude: true value against the repo and git history. One garbled sentence in the new CONTRIBUTING.md entry is the only outstanding item.

Review confidence:

Dimension	Level	Notes
mechanics	HIGH	Template, front matter, and contributor-doc changes all read end to end.
facts	HIGH

Investigation log

• Cross-sibling reads: not run (not in a templated section)
• External claim verification: 2 of 7 claims verified (0 unverifiable, 0 contradicted) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 6 Pass 1, 0 Pass 2, 1 Pass 3 (verified 0, contradicted 0, unverifiable 1).
• Cited-claim spot-checks: not run (no cited claims)
• Frontmatter sweep: ran on body
• Temporal-trigger sweep: ran (recency words present in diff; spot-check in-review)
• Code execution: not run (no static/programs/ change)
• Code-examples checks: not run (no fenced code blocks in content files)
• Editorial-balance pass: not run (not under content/blog/)

🚨 Outstanding	⚠️ Low-confidence	💡 Pre-existing	✅ Resolved
1	0	0	0

🔍 Verification trail

7 claims extracted · 2 verified · 0 unverifiable · 0 contradicted

• L130 in CONTRIBUTING.md "- sitemap_exclude: Set to true to omit the page from the generated sitemap.xml without affecting crawling or indexing (unlike…" → ➖ not-a-claim (evidence: (escalated from pass3) CONTRIBUTING.md documents the PR author's own new front-matter convention (sitemap_exclude). The current layouts/_default/sitemap.xml on master excludes pages via .Params.private…; source: gh api repos/pulumi/docs/contents/layouts/_default/sitemap.xml (master branch, base64-decoded))
• L3 in content/registry.md "# This file is a build-time placeholder only (see the readme note below); the" → ➖ not-a-claim (evidence: This is a code/build comment describing the file's own placeholder role, not a falsifiable factual assertion about a third party.; source: content/registry.md)
• L3-5 in content/registry.md "The CloudFront/S3 trick used for this placeholder page means Google never actually sees the HTML Hugo renders from it." → ➖ not-a-claim (evidence: This is a code-comment explaining the PR author's own CloudFront/S3 routing design for this placeholder file (sitemap_exclude, lastmod behavior), not a third-party-attributed factual assertion. The same file's readme block corroborates…; source: repo:content/registry.md)
• L3-10 in content/registry.md "Without the sitemap_exclude frontmatter setting, this placeholder file's git-derived lastmod date competes with the Registry sitemap's entry for the same…" → ➖ not-a-claim (evidence: The claim is a paraphrase of the file's own explanatory comment ("Without sitemap_exclude, this placeholder's own git-derived lastmod ... competes with the Registry sitemap's entry for the same canonical URL"), describing the author's…; source: repo:content/registry.md)
• L8 in content/registry.md "This placeholder file's git-derived lastmod has been frozen since the commit that introduced it on 2023-12-11." → ✅ verified (evidence: git history for content/registry.md in pulumi/docs shows commits only on 2023-11-23T02:10:59Z and 2023-12-11T20:53:57Z, with 2023-12-11 being the most recent — confirming lastmod has been frozen since that commit with no subsequent…; source: gh api repos/pulumi/docs/commits?path=content/registry.md)
• L12 in content/registry.md "The frontmatter field sitemap_exclude is set to true for this page." → ✅ verified (evidence: The file's frontmatter contains the line sitemap_exclude: true, matching the claim exactly.; source: repo:content/registry.md)
• L13-15 in content/registry.md "This registry.md page is intentionally left blank because of how the CloudFront CDN is configured to serve https://www.pulumi.com/registry/." → ➖ not-a-claim (evidence: The claim text is a paraphrase of the file's own readme frontmatter field, which explains this repo's own CloudFront/S3 configuration rationale for why registry.md is intentionally blank -- it's a faithful description of the PR…; source: repo:content/registry.md)

🚨 Outstanding in this PR

These must be resolved or refuted before merging.

• [L130] CONTRIBUTING.md — the new sitemap_exclude entry contains a garbled clause: "in a different sitemap that Google is submitted". Suggested rewrite of the trailing sentence:

  Use this only for pages whose canonical URL is already declared, with an accurate `lastmod`, in a different sitemap that is submitted to Google (for example, a page that is a build-time placeholder for a URL actually served and indexed from a different origin).
  

⚠️ Low-confidence

No low-confidence findings.

💡 Pre-existing issues in touched files (optional)

No pre-existing issues in touched files.

✅ Resolved since last review

No items resolved since the last review.

📜 Review history

• 2026-08-26T13:05:46Z — Sitemap template and placeholder front matter check out; one garbled sentence in the new CONTRIBUTING.md field description (d8cdc48)

---

• Refresh this review — comment @claude #update-review. Say what you fixed, or which finding you dispute and why; both work in the same mention.
• Ask for anything else — comment @claude with no hashtag (questions, one-off fixes). Leaves this review untouched.

Important

Please don't hide, resolve, or delete this comment! It breaks things!

📖 How pre-merge review works — the full lifecycle, short-circuits, and escape hatches.

[pulumi-bot]

Your site preview for commit d8cdc48 is ready! 🎉

http://www-testing-pulumi-docs-origin-pr-21139-d8cdc48d.s3-website.us-west-2.amazonaws.com

Changed pages:

• /registry/

[pulumi-bot]

Lighthouse Performance Report

Commit: d8cdc48 | Metric definitions

Page	Device	Score	FCP	LCP	TBT	CLS	SI
Homepage	Mobile	🔴 43	3.0s	5.7s	1180ms	0.022	7.3s
Homepage	Desktop	🟢 93	0.8s	1.0s	188ms	0.006	0.9s
Install Pulumi	Mobile	🟡 57	7.5s	16.1s	75ms	0.000	7.5s
Install Pulumi	Desktop	🟢 92	1.2s	1.5s	0ms	0.022	1.2s
AWS Get Started	Mobile	🟡 60	6.1s	15.6s	50ms	0.000	6.1s
AWS Get Started	Desktop	🟢 93	1.1s	1.4s	0ms	0.045	1.1s