SEO and Crawler Hygiene
Executive Summary
Only the production custom domain should be indexable, and the EDS origin hostnames should never appear in anything a crawler can reach. This is a containment concern rather than an access control: it does not stop anyone reaching an origin, it stops search engines and casual readers discovering one.
Where Environment Access Control covers the layers that decide who may reach an environment, this page covers which URLs the outside world is told about.
Canonical Domain
Only rhea.pdcoe.dev should appear in search engine indexes. All SEO signals must point to this domain:
| Signal | Must Point To |
|---|---|
| Canonical tags | https://rhea.pdcoe.dev/{path} |
| Sitemap entries | https://rhea.pdcoe.dev/... |
| Internal links | Relative paths (resolve to current domain) |
| Redirect targets | https://rhea.pdcoe.dev/... |
| robots.txt | Configured per site via config service |
| hreflang | https://rhea.pdcoe.dev/... |
robots.txt Per Environment
Configured via the Admin API config service per site.
Production (rhea)
User-agent: *
Allow: /
Sitemap: https://rhea.pdcoe.dev/sitemap.xml
Stage and Dev
User-agent: *
Disallow: /
What Must Be Avoided
To prevent crawlers and users from discovering .aem.live / .aem.page:
- No browser-side fetch() calls to .aem.live or .aem.page
- No canonical tags pointing to EDS origins
- No sitemap entries using Adobe default hostnames
- No redirects that expose .aem.live or .aem.page
- No JavaScript, CSS, image, or JSON URLs referencing EDS origins
- No preview links shared publicly
This is containment, not enforcement
None of the above is enforced by the access-control layers. A leaked origin URL stays reachable wherever access.site is not yet enabled — verified 2026-09-17: raw origins return 401 on rhea-stage but 200 on prod and dev. So these rules are the only thing keeping those hostnames out of an index today. See Environment Access Control for the lock-down rollout.
Verification
| Check | How |
|---|---|
| No origin hostnames in shipped code | Grep the built output for aem.live / aem.page |
| Canonical points at the custom domain | View source on a production page |
| Stage and dev are disallowed | Fetch /robots.txt on each environment |
| Nothing unexpected is indexed | site: search for the origin hostnames |
Related Documents
- Environment Access Control — the layers that gate access to each environment, and the canonical hostname-to-origin mapping.
- Security Model — CSP, sanitization and the wider defense-in-depth picture.
- EDS Architecture Overview — environment topology.
On this page
h3
true