Agent readiness · Browser and scraping infrastructure
scrapingbee.com
6 of 14 measurable points under formula 9.2, measured on 2026-08-12. 15 checks exist; each is one HTTP request with a published rule, so every sentence below can be rerun and argued with. 2 of them could not be measured from where we ask, and those are left out of the denominator rather than counted as failures.
A · Discovery
- PASSAnswers an agent user-agentAnswered 200 to LetAgentsIn/1.0 (+https://letagentsin.com/methodology)
- PASSllms.txt publishedllms.txt present at https://www.scrapingbee.com/llms.txt, and the 12 links we sampled across the file all answer
- FAILDocs readable without JavaScriptOnly 126 characters render without JS
- FAILOn-demand agents not blockedrobots.txt permits them, but your edge answered ChatGPT-User 403 and Claude-User 403 at https://docs.scrapingbee.com, which a browser is served
- N/APaths robots.txt points at answerrobots.txt names no concrete path, only patterns or nothing, so there is no claim to check
- PASSNo punishing crawl delayNo Crawl-delay applies to the agents we check
B · Agent entry
- FAILAgent entry pointNone of the 9 known agent entry paths returns a file rather than your page shell
- FAILOAuth dynamic client registrationNo OAuth metadata on any of the 8 hosts probed, including the usual auth and api subdomains
- PASSMCP surfaceLive MCP endpoint at https://mcp.scrapingbee.com/mcp, answers JSON
C · Registration
- UNMEASUREDNo CAPTCHA in the signup HTMLUnmeasurable: we found no link to an account signup on the pages we read, while you publish prices at https://scrapingbee.com/pricing, so this is a gap in our reading rather than a finding about youLink signup from your home page or your pricing page and this becomes measurable.
- UNMEASUREDSignup reachable without a browserUnmeasurable: we found no link to an account signup on the pages we read, while you publish prices at https://scrapingbee.com/pricing, so this is a gap in our reading rather than a finding about youLink signup from your home page or your pricing page and this becomes measurable.
D · Provisioning
- FAILProgrammatic key provisioningNone of the 7 provisioning phrases appears in the 2 documents we read
- PASSFree tier or no-card trial stated in textFree tier or no-card signals at https://scrapingbee.com/pricing: "No credit card"
E · Integration
- PASSTyped SDK on the registryscrapingbee@1.8.2 ships types, matched from the registry by who publishes it rather than by a link on your site
- FAILMachine-readable API descriptionNo OpenAPI spec at the 5 usual paths, none declared by https://docs.scrapingbee.com, and no markdown negotiationPoint at your spec from your docs page with rel="service-desc" and an agent finds it without guessing.
Tell me when this changes
The failures here are the kind nobody notices. An edge rule that starts refusing agents changes nothing a person sees in a browser, so the first sign is usually an integration that quietly stopped working. We rescan weekly and write only when a verdict moves.
This page is the newest scan we hold for scrapingbee.com and changes when we rescan. It is not a judgement of the product: we measure whether an unattended run can get through, not whether the thing is any good. The whole corpus is published as JSON and CSV.