GrandTotalBot

You have probably arrived here from a line in your access log. Here is who we are, what we fetch, how often — and how to switch us off.

GrandTotalBot/0.1 (+https://grand-total.corepurpose.tech/bot)

Who

Grand Total compares the total delivered price of a shopping basket across shops — goods, shipping, fees and duty. We fetch product pages so we can show your price correctly, and we link customers on to you.

What we fetch, and how often

• We follow robots.txt. Both Disallow and Crawl-delay. If we cannot fetch your robots.txt, we do not fetch the rest.
• At most one request per second to the same host, and slower if your Crawl-delay says so. We do not run parallel requests against you.
• We do not guess at addresses. We fetch what your own sitemap states, and product pages from there. We do not try our luck with addresses you have not mentioned.
• We store the page before we read it, so a bug in our own code does not cost you another fetch.
• We fetch only HTML and sitemaps. No images, no order flow, no logins.

How to block us

Put this in your robots.txt. We respect it at the next fetch:

User-agent: GrandTotalBot
  Disallow: /

If you would rather have us go slower than go away, set a Crawl-delay — it is read and followed.

In English

Grand Total compares the total delivered price of a basket across shops: goods, carriage, fees and duty. We fetch product pages so your prices are shown correctly, and we send customers to you.

We honour robots.txt, including Crawl-delay. We make at most one request per second to a host and never run parallel requests against the same one. We fetch only what your own sitemap lists — we do not guess at URLs — and we request only HTML and sitemaps. To block us entirely, add the robots.txt block above; to slow us down instead, set a Crawl-delay.