Dokki gives each published site explicit controls for whether search engines and AI crawlers should discover it.
Listed sites
When a site is listed, Dokki serves crawl-friendly public pages with normal links between published resources. The site can expose sitemap.xml and llms.txt so crawlers can discover the public tree.
Unlisted sites
When a site is unlisted, pages remain reachable by direct link but are marked noindex and nofollow. Dokki also blocks robots.txt access for the site path and returns no sitemap.xml or llms.txt for that site.
Use unlisted for drafts, internal previews, customer-specific docs, or anything you want shareable by link but not indexed.
Sitemap
sitemap.xml includes the published site home and published resources. Commit changes after editing a published resource so the frozen content and timestamps reflect the public version you want crawled.
llms.txt
llms.txt summarizes the published site for AI crawlers and assistants. It includes the site name, summary, and public page list when the site is listed.
Custom domains and subpaths
Custom domains and subdirectory proxies should keep public links, canonical URLs, robots.txt, sitemap.xml, and page routes aligned. If you change domain routing, verify the live public URL directly.