# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / User-agent: * Disallow: User-agent: SemrushBot User-agent: SemrushBot-SA User-agent: SiteAuditBot User-agent: SemrushBot-BA User-agent: SemrushBot-SI User-agent: SemrushBot-SWA User-agent: SemrushBot-CT User-agent: SplitSignalBot User-agent: SemrushBot-COUB Disallow: /