Measured 8 September 2026. We will re-run this monthly and update the table.
Every proxy provider tells you that you need their product. None of them show you what actually happens if you don’t buy it. So we measured it.
We took a plain datacenter server, sent one ordinary-looking request to each of nine popular scraping targets, and recorded what came back. Not a hundred requests, not a crawl. One request, the way a script does it the very first time. If a site blocks that, the IP address is the problem, not your rate.
Setup
- Source: a virtual server at Hetzner (AS24940), Helsinki. A typical cloud IP, the kind every scraper starts on.
- Client:
curlwith a current Chrome user agent and normalAccept/Accept-Languageheaders, following redirects, 30-second timeout. - One request per site. No retries, no cookies, no JavaScript.
- Success means the real page arrived: we check for a marker that only exists on the genuine result page (for example Amazon’s
s-search-resultblocks). A200 OKalone does not count, because three of these sites serve their block page with status 200.
The full script, target list and raw JSON are open: this is the same tool we use for our proxy comparisons.
Results
| Target | What we asked for | Result | Time | What came back |
|---|---|---|---|---|
| api.ipify.org | our IP | ✅ page | 0.2 s | control, always works |
| Amazon | search results for “airpods” | ✅ page | 0.8 s | full results (172 KB). On one of four attempts across the day: a 2.6 KB “Robot Check” page instead |
| search results | ⚠️ empty shell | 0.1 s | 200 OK, 46 KB, zero results: a “please enable JavaScript” wrapper. Looks like success in your logs, contains nothing | |
| Zillow | listings page | ❌ blocked | 0.2 s | 403, PerimeterX “Access to this page has been denied” |
| Walmart | search results | ❌ blocked | 0.5 s | redirected to /blocked?url=…, “Robot or human?” captcha with status 200 |
| Best Buy | homepage | ❌ blocked | 0.2 s | connection reset mid-stream (HTTP/2 INTERNAL_ERROR). No page, no status code |
| public company page | ❌ blocked | 0.2 s | status 999, LinkedIn’s private code for “we don’t like you” | |
| Indeed | job search | ❌ blocked | 0.0 s | 403, “Security Check” page from Cloudflare |
| Nike | men’s shoes category | ✅ page | 0.5 s | full page, 1.2 MB, 685 product cards |
Score: 3 of 9 real pages, 5 hard blocks, 1 empty success.
What the failures look like from inside your script
This is the part that costs people money. Only two of the six failures are obvious.
Obvious: Zillow and Indeed return 403. Your HTTP library raises an error, you notice.
Not obvious:
- Google returned
200 OKwith 46 KB of HTML. A naive scraper stores it as a successful result. There is not one search result in it. If you are building a rank tracker on datacenter IPs, your database fills with empty pages that look complete. - Walmart returned
200 OKtoo, from a different URL: it silently redirected to/blocked. Unless you compare the final URL with the one you requested, you log a success. - Best Buy never sent a status code at all. The connection was accepted and then reset by Akamai after it decided we were not a browser. Depending on your library this surfaces as a retry-able network error, so a script retries it forever.
- LinkedIn’s
999is not a standard code. Some libraries treat anything ≥ 400 as an error, some do not. - Amazon let us in, three times out of four. The fourth was a “Robot Check” page. At one request per hour you would get 75 % clean data and 25 % garbage, and nothing in the status code tells you which is which.
The lesson generalises: detect blocks by content, not by status code. Our test script checks every response for a marker that only the real page contains. That is the single most useful line of code in a scraper.
Why a datacenter IP fails on arrival
None of these sites saw us do anything wrong. One request, correct headers, nothing aggressive. They blocked the address.
Every IP address belongs to an autonomous system (AS), and the AS is public information. Hetzner, AWS, DigitalOcean, OVH and the rest are labelled as hosting providers in every commercial IP database. A human sitting in Helsinki does not browse Walmart from a Hetzner server, so the request is scored as automated before the first byte of HTML is decided.
That is the whole business case for residential proxies: an address that belongs to Comcast or Vodafone scores as a person. Whether that is worth $5–15 per gigabyte is a separate question, and the answer depends on the target. Nike let a datacenter IP straight in. Zillow did not.
What we test next
This page is the baseline. The next runs put the same nine targets behind residential proxies from Oxylabs, Bright Data and a third provider (NetNut was on the list until the FBI seized its domains in July 2026; see our note on that), from the same server, with the same one-request rule, and record the same table. Success rate, block reasons, time to first byte, and whether the “residential” address actually belongs to a consumer ISP or to another datacenter.
If you want the numbers before you spend money, the comparison is on the residential proxies page. It is updated with every run and the date is at the top.
Method notes: single request per target, sequential, 8 September 2026 15:06 and 15:09 UTC. Results vary between runs, especially on Amazon; we report the run and note the variance rather than average it away. Raw data and the test script are in the openwebdata repository.