| User Agent | Number of hits | Data Transferred (Kb) |
| The Knowledge AI | 500 | 190 |
| Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloal | 79 | 1,554 |
| facebookexternalhit/1.1 ( http://www.facebook.com/externalhit_uatext.php) | 63 | 1,776 |
| MauiBot (crawler.feedback wc@gmail.com) | 44 | 55 |
| Go-http-client/1.1 | 43 | 210 |
| HubSpot Connect 2.0 (http://dev.hubspot.com/) - BizOpsCompanies-Tq2-BizCoDomainValidationAudit | 42 | 12 |
| DF Bot 1.0 | 32 | 282 |
| ltx71 - (http://ltx71.com/) | 26 | 53 |
| CCBot/2.0 (https://commoncrawl.org/faq/) | 15 | 72 |
| panscient.com | 14 | 68 |
| clark-crawler2/Nutch-1.19-SNAPSHOT | 9 | 13 |
| python-requests/2.25.1 | 7 | 22 |
| jambot/jambot-1.2 (For the people.; https://jambot.com/webmaster; crawler at jambot dot com) | 5 | 171 |
| PostmanRuntime/7.26.8 | 4 | 21 |
| Scrapy/2.2.1 ( https://scrapy.org) | 3 | 11 |
| curl/7.78.0 | 3 | 35 |
| My User Agent 1.0 | 3 | 11 |
| colly - https://github.com/gocolly/colly/v2 | 2 | 10 |
| curl/7.68.0 | 2 | 1 |
| httpx - Open-source project (github.com/projectdiscovery/httpx) | 2 | 10 |
| NetSurf/1.2 (NetBSD; amd64) | 2 | 10 |
| python-requests/2.22.0 | 2 | 10 |
| PageThing http://pagething.com | 2 | 1 |
| rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler aardwolf-production-crawler-53@moz.com) | 2 | 1 |
| okhttp/3.6.0 | 2 | 10 |
| python-requests/2.26.0 | 2 | 10 |
| Screaming Frog SEO Spider/7.2 | 1 | 0 |
| Drupal/8.9.15 ( https://www.drupal.org/) GuzzleHttp/6.5.5 curl/7.29.0 PHP/7.3.28 | 1 | 0 |
| Screaming Frog SEO Spider/15.2 | 1 | 0 |
| webprosbot/2.0 (abuse-6337@webpros.com) | 1 | 10 |
| curl/7.29.0 | 1 | 0 |
| (compatible;PetalBot; https://aspiegel.com/petalbot) | 1 | 0 |
| Python/3.9 aiohttp/3.7.4.post0 | 1 | 0 |
| python-requests/2.24.0 | 1 | 0 |
| BaiduSpider | 1 | 0 |
| DemandbasePublisherAnalyzer/0.1 (http://www.demandbase.com; info@demandbase.com) | 1 | 34 |
| Total | 920 | 4,663 |
