Which proxies are best for web scraping?

On 10 Sep 2026, measured against 9 real sites with 45 requests per provider, Oxylabs (20/45) passed the most requests, followed by Decodo (19/45), SOAX (19/45). The differences are small and come almost entirely from Zillow. Pick by the target matrix for the sites you scrape, then by price per gigabyte.

Why is "no proxy" in the table?

Because it is the baseline. If a provider does not beat a plain datacenter address, it is not worth paying for. In this run a direct request from our server passed 15 of 45; residential proxies added Zillow. Amazon and Nike let the datacenter address in anyway.

Why do all providers score so close to each other?

Because on a single plain HTTP request the target decides by address type and by whether a real browser is present, and all tested providers deliver real consumer addresses (Comcast and Charter on this run). The sites that blocked everything (Google, Walmart, Best Buy, LinkedIn, Indeed) block the missing browser, not the IP. The browser run on this page shows what changed when the same proxies were driven by a real Chromium: some of those sites opened, and one that plain requests passed closed.

How much does a residential proxy cost for scraping?

List prices of the providers on this page run from $2 to $8.4 per gigabyte. Traffic is what you pay for: this test moved about 3.0 MB per 45 requests, roughly 67 MB per 1,000 page requests, which is $0.13 to $0.56 per 1,000 requests at list price. Pages with images or JavaScript rendering cost several times more.

Can residential proxies be detected?

Yes, and this test shows how. Walmart, Best Buy, LinkedIn and Indeed blocked every request from every provider even though each came from a real Comcast or Charter address. Modern bot defences look at the TLS fingerprint, headers and JavaScript execution before they look at the IP. A residential address removes one signal; it does not make a script look like a browser.

Residential or datacenter proxies for web scraping?

Datacenter addresses are faster (median 0.3 s here versus about 2 s residential) and far cheaper, and they were enough for Amazon search and the Nike category page. Residential addresses were the only way in to Zillow. Start with datacenter, move a target to residential only when the datacenter address is blocked.

Is web scraping illegal?

Not in itself. Collecting publicly available data is lawful in most jurisdictions, and US courts have held that accessing public pages is not "unauthorised access" under the CFAA. What can be unlawful is what you collect and what you do with it: personal data, copyrighted content, bypassing a login, or breaching a contract you agreed to. This page is not legal advice; check the target site's terms and your local law.

Do providers pay for placement?

No. We earn a commission on purchases through the pricing links, and that is disclosed above the results. Providers do not see results before publication and cannot buy a position. The test script, target list and raw results are published.

How often is this page updated?

The test is rerun on the same day each month from the same server, and old results stay published with their date. The date at the top of the results table is the date of the numbers you are reading. Prices are checked on the same day.

Amazon · measured · dated

Scraping Amazon search: you probably do not need a residential proxy

Amazon is the most scraped site on the internet and the one most often used to sell residential proxies. In our test a plain request from an ordinary cloud server returned real search results five times out of five. The residential proxies did the same, one of them missing a single attempt. Here is the measurement, and here is where the money actually goes.

See the results When you do need one

Latest run

Tested: 10 Sep 2026
Targets: 9real sites
Providers: 3+ datacenter baseline
Baseline pass rate: 15 / 45no proxy

Quick answer

Everything we tried worked, including no proxy at all

Five requests per source per method to an Amazon search results page. A pass requires the real product grid in the response, not a status 200 with an empty shell.

Last tested 10 Sep 2026 · next run Oct 2026

Source	Plain requests (curl)	Headless Chromium	From
No proxy (datacenter)	5 / 5	5 / 5	$0
	5 / 5	5 / 5	$6/GB	See Oxylabs pricing
	not tested	not tested	$8.4/GB	See Bright Data pricing
	5 / 5	5 / 5	$2/GB	See Decodo pricing
	4 / 5	5 / 5	$5/GB	See SOAX pricing

Five requests per source per method against Amazon, measured 10 Sep 2026 (plain) and 10 Sep 2026 (browser). The reason shown is what the site returned.

Reading this honestly

What the result does and does not mean

One request is not a crawl

We sent five requests from each source, spread over a run, with no repetition of the same address. That is the situation of a script starting out. It is not the situation of a job pulling ten thousand product pages an hour from one address, which is where Amazon’s rate limiting begins and where a proxy earns its price.

Search pages, not every page type

The target is a search results page. Product detail pages, review pages and seller pages have their own defences and their own thresholds, and a result on one does not transfer to another. We test search because it is the entry point for price monitoring.

Amazon’s defence is behavioural

The robot check most scrapers meet is triggered by pattern, not by address type: too many requests, too regular, no cookies, the same path over and over. A residential address changes who you appear to be, not how you behave. Fix the behaviour first; it is free.

Where a proxy does pay

Volume, geography and continuity. Many requests without one address absorbing the whole pattern, prices as they appear in a specific country, and a run that keeps working when a single address gets throttled. Those are real reasons and they are about scale, not about opening the door.

If you are building this

What we would do to collect Amazon search pages

Start without a proxy and measure. A cloud server got the real results in our test. Run your own five requests before you buy anything; the answer for your target and your volume may be the same.
Slow down before you scale out. A request every few seconds with varied paths survives far longer than a burst through a hundred addresses. Rate limiting is the wall you will hit, and proxies move the wall rather than removing it.
Add datacenter proxies before residential ones. They are a fraction of the price and they passed this target in our test. Move to residential when a datacenter address stops working on the specific page you need, not before.
Do not add a headless browser for this. It passed here too, but it moved roughly ten times the traffic for the same page. That is a real bill on any per-gigabyte plan, and it bought nothing on this target.
Check the page type you actually need. If your job is product detail or reviews rather than search, test that page. We publish the raw results and the script so you can point it at your own URL.

Method

How this was measured

The same script and server as every other number on this site.

All nine targets

5 requestsper target per provider, each from a different exit IP. Pass rate is how many of the 5 brought back the real page.

9 targetsthe sites people actually scrape: retail, real estate, jobs, search. Picked for their bot defences.

Content, not statusa 200 with no results is a failure. Every response is checked for a marker only the real page has.

Monthlyrerun on the same day each month from the same server. Old results stay published, dated.

The target is an Amazon search results page, requested five times per source. Plain requests carry the headers of a current desktop Chrome and no JavaScript engine. Browser requests use a headless Chromium through the same proxy, each in a fresh context. A pass requires the search result grid markup in the response; the robot check page, a redirect and a 200 with an empty container all count as failures. Amazon returns different defences to different countries and at different times of day, so this is one measurement on one date, repeated monthly.

Providers

Prices, if your volume needs them

The same four providers and their list prices. Where they actually made a difference is on Zillow, not here.

Pool	Locations	Targeting	Sessions	Trial	Entry price
100M+	195 countries	City · ASN	Rotating and sticky	None on self-service	$6/GBStarter 5 GB, $30/mo, cancel any time	See Oxylabs pricing
72M+	195 countries	City · ASN · ZIP	Rotating and sticky	Free trial for verified businesses	$8.4/GBPay as you go, no commitment	See Bright Data pricing
115M+	195+ locations	City · ZIP · ASN	Rotating and sticky (minutes to days)	3-day trial, 100 MB	$2/GBPay as you go $4/GB, plans from $2/GB	See Decodo pricing
155M+	195+ locations	City · ISP	Rotating and sticky, custom length	3-day trial, 400 MB, $1.99	$5/GBPay as you go $5/GB, $3/GB on $200/mo plan	See SOAX pricing

We earn a commission if you buy through the pricing links. It does not change the numbers: the test script, the target list and the raw results are public, and providers do not see results before you do.

Best in our testOxylabs · from $6/GBSee pricing