The AhrefsBot bot crawls and analyzes your content. There are several ways to block the bot. You can add the user agent to your robots.txt, use a block list in your .htaccess file or block the whole ip range.
Full User Agent
Mozilla/5.0 (compatible; AhrefsBot/5.1; +http://ahrefs.com/robot/)
Statistics
More information and statistics about the crawler.
| Respects robots.txt: | Yes |
| Last seen: | 2016-03-27 12:00:00 |
| URL: | http://ahrefs.com/robot/ |
| IP list (examples): |
|
| Versions: |
|
How to block AhrefsBot
There are several methods to control the access that the AhrefsBot has to your website. Good bots obey the robots.txt in your root directory. Just add the user agent to your robots.txt and the crawler won’t crawl any page anymore.
Robots.txt
Don’t allow the bot to crawler any part of your site:
User-agent: AhrefsBot Disallow: /
Dont allow the bot to crawler a specific folder and it’s contents of your site:
User-agent: AhrefsBot Disallow: /uploads/
Code for your .htaccess
If the crawler ignores the robots.txt, you can block the bot using the htaccess file. Of course you can add more bot user-agents next to the AhrefsBot. Here is a simple example.
SetEnvIfNoCase User-Agent "AhrefsBot" badbots SetEnvIfNoCase User-Agent "Another user agent" badbots <Limit GET POST HEAD> Order Allow,Deny Allow from all Deny from env=badbots </Limit>