About the PriceTap crawler
PriceTap collects and compares the public prices of online shops in Azerbaijan. This page states plainly what our crawler does.
How the crawler identifies itself
PriceTapBot/1.0 (+https://pricetap.az/bot)
If you see this line in your logs, it is us. We do not use another name and we do not disguise ourselves as a browser.
Rules we follow
- We fully respect your robots.txt and your Crawl-delay. If a delay is declared, we never open more than one channel to your host.
- Default load does not exceed 0.5 requests per second.
- We read public data only: product name, price, availability, URL, SKU. We do not copy your descriptions and we do not hotlink your images.
- We do not impersonate a browser.
- We do not attempt to bypass bot protection. If your site is protected we simply stop and contact you instead.
Opting out
If you do not want your products listed on PriceTap, one message is enough — we remove them within 48 hours and we do not argue.
If you prefer to block us yourself
Adding these two lines to your robots.txt is enough:
User-agent: PriceTapBot Disallow: /