# Internet Archiver Wayback Machine User-agent: ia_archiver Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / User-agent: archive.org_bot Disallow: / # Everyone else (including AhrefsBot) User-agent: * Disallow: /cdn-cgi/