EN
BosanskiDeutschEnglish
Log in Download free

The bee on your site

Seeing seo-pauk in your server logs? That is SEO Bee, a desktop tool for technical SEO audits. Here is what it does and how to stop it.

Who sends these requests

SEO Bee is not a search engine and has no server of its own crawling the web. The program is installed on a user's computer and crawls the site that user enters. That is why the requests come from the user's IP address, not ours. Usually it is the site owner or an agency doing their SEO.

The full default user agent:

Mozilla/5.0 (compatible; seo-pauk/0.1; +https://seo-bee.de/bot/)

What the bee respects

  • robots.txt, by default. Rules for seo-pauk take priority; if there are none, the rules for * apply.
  • 429 and 503 responses and the Retry-After header: the bee slows down or stops.
  • Bot protection: when the bee runs into it, it stops. It does not solve CAPTCHAs, switch IP addresses or pose as a browser to get through.

How to stop it

Add this to robots.txt:

User-agent: seo-pauk
Disallow: /

Or for just part of the site:

User-agent: seo-pauk
Disallow: /internal/

What we cannot guarantee

Users can change the user agent in the settings, for example to see what a site sends to Googlebot. The rules above will then not recognise it by name. A user can also switch off robots.txt, but only after confirming that the site is theirs or that they have the owner's permission. If the bee is putting load on your server or crawling something it should not, email abuse@seo-bee.de. Send the IP address, the time and a few lines from the log.