https://www.xbcatm.com/tags/discovery/Illustrated field notes on HTTP, DNS, TLS, Caddy, SSH, releases, and recovery.Hugoen-ushttps://www.xbcatm.com/notes/robots.txt-and-sitemaps-answer-different-questions/Sat, 22 Aug 2026 00:00:00 +0000https://www.xbcatm.com/notes/robots.txt-and-sitemaps-answer-different-questions/Keep crawler preferences, URL discovery, and access control separate.