A preload announces a resource before its normal consumer discovers it. This can help when a needed image is created by JavaScript or a font is discovered only after loading and parsing a stylesheet. The benefit is an earlier opportunity to fetch. It does not follow that every hinted asset becomes the browser’s highest priority or that every page becomes faster. ...
A robots.txt file communicates crawling rules to automated clients that implement the Robots Exclusion Protocol. It can reduce unwanted traversal of expensive or unhelpful paths, but the server does not automatically enforce its contents against incoming requests. RFC 9309 explicitly separates these rules from access authorization. Treat robots.txt as a public interface for cooperating crawlers, while authentication and authorization decide who may actually retrieve protected resources. ...
Cross-Origin Resource Sharing lets a server opt into exposing a response to scripts from another origin. An origin consists of scheme, host, and port; two services on different ports can therefore be cross-origin even on the same machine. The browser evaluates the CORS response headers for the requesting page. This mechanism does not authenticate that page’s user or turn the endpoint into a private network service. Same-site cookie rules use a different boundary from same-origin response access, so similar-looking hostnames are insufficient to reason about both. ...
Content Security Policy tells a supporting browser which actions a document permits. Fetch directives govern categories such as scripts, styles, and images; other directives cover different boundaries. The policy is supplied with the document, commonly through a response header. It adds constraints to browser behavior, but does not repair an injection bug, authenticate application users, or promise that every possible form of script execution has been eliminated. ...
robots.txt is a crawler-facing preference; a sitemap is a URL inventory; neither is an authorization layer.