PleskenBot

The crawler of Plesken, a free search engine of boats for sale.

What it does

PleskenBot reads the boats for sale that boat dealers, brokers and shipyards publish on their own websites, so that buyers can find them in one search. Every boat on Plesken links to the seller's own page: Plesken does not sell boats, does not act as a middleman and does not charge sellers to appear.

It reads public pages only: it never logs in, fills in forms or contacts anyone.

How it behaves

• It follows robots.txt rules for PleskenBot (or * if there is no specific group), including Crawl-delay.
• It makes at most one request every 1.5 seconds to the same website, and fewer if Crawl-delay asks for it.
• It backs off when a site answers 429 or 503 and honours Retry-After.
• It never tries to get around a CAPTCHA, a login or any other access control.

How to recognise it

User-Agent:

Mozilla/5.0 (compatible; PleskenBot/1.0; +https://plesken.com/bot)

Its requests are signed with HTTP Message Signatures (Web Bot Auth). The public key directory is at /.well-known/http-message-signatures-directory, and every request carries Signature-Agent: "https://plesken.com".

How to block it or slow it down

Add this to your robots.txt to stop it completely:

User-agent: PleskenBot
Disallow: /

Or ask it to go slower, for example one request every 10 seconds:

User-agent: PleskenBot
Crawl-delay: 10

Contact

Questions or problems with PleskenBot, or if you want your boats removed from Plesken: contact form.