Which proxies are best for web scraping?

On 10 Sep 2026, measured against 9 real sites with 45 requests per provider, Oxylabs (20/45) passed the most requests, followed by Decodo (19/45), SOAX (19/45). The differences are small and come almost entirely from Zillow. Pick by the target matrix for the sites you scrape, then by price per gigabyte.

Why is "no proxy" in the table?

Because it is the baseline. If a provider does not beat a plain datacenter address, it is not worth paying for. In this run a direct request from our server passed 15 of 45; residential proxies added Zillow. Amazon and Nike let the datacenter address in anyway.

Why do all providers score so close to each other?

Because on a single plain HTTP request the target decides by address type and by whether a real browser is present, and all tested providers deliver real consumer addresses (Comcast and Charter on this run). The sites that blocked everything (Google, Walmart, Best Buy, LinkedIn, Indeed) block the missing browser, not the IP. The browser run on this page shows what changed when the same proxies were driven by a real Chromium: some of those sites opened, and one that plain requests passed closed.

How much does a residential proxy cost for scraping?

List prices of the providers on this page run from $2 to $8.4 per gigabyte. Traffic is what you pay for: this test moved about 3.0 MB per 45 requests, roughly 67 MB per 1,000 page requests, which is $0.13 to $0.56 per 1,000 requests at list price. Pages with images or JavaScript rendering cost several times more.

Can residential proxies be detected?

Yes, and this test shows how. Walmart, Best Buy, LinkedIn and Indeed blocked every request from every provider even though each came from a real Comcast or Charter address. Modern bot defences look at the TLS fingerprint, headers and JavaScript execution before they look at the IP. A residential address removes one signal; it does not make a script look like a browser.

Residential or datacenter proxies for web scraping?

Datacenter addresses are faster (median 0.3 s here versus about 2 s residential) and far cheaper, and they were enough for Amazon search and the Nike category page. Residential addresses were the only way in to Zillow. Start with datacenter, move a target to residential only when the datacenter address is blocked.

Is web scraping illegal?

Not in itself. Collecting publicly available data is lawful in most jurisdictions, and US courts have held that accessing public pages is not "unauthorised access" under the CFAA. What can be unlawful is what you collect and what you do with it: personal data, copyrighted content, bypassing a login, or breaching a contract you agreed to. This page is not legal advice; check the target site's terms and your local law.

Do providers pay for placement?

No. We earn a commission on purchases through the pricing links, and that is disclosed above the results. Providers do not see results before publication and cannot buy a position. The test script, target list and raw results are published.

How often is this page updated?

The test is rerun on the same day each month from the same server, and old results stay published with their date. The date at the top of the results table is the date of the numbers you are reading. Prices are checked on the same day.

Indeed · measured · dated

Scraping Indeed: only one combination worked

Indeed sits behind Cloudflare. A plain request was refused from every source we tried, including three residential proxy networks. A headless browser from a datacenter address was refused too. The job cards arrived only when a real browser ran through a residential address, and that is the whole finding.

See the results Why the pair works

Latest run

Tested: 10 Sep 2026
Targets: 9real sites
Providers: 3+ datacenter baseline
Baseline pass rate: 15 / 45no proxy

Quick answer

Browser plus residential address, or nothing

Five requests per source per method to an Indeed job search page. A pass requires the real job cards in the response.

Last tested 10 Sep 2026 · next run Oct 2026

Source	Plain requests (curl)	Headless Chromium	From
No proxy (datacenter)	0 / 5HTTP 403	0 / 5HTTP 403	$0
	0 / 5HTTP 403	4 / 5	$6/GB	See Oxylabs pricing
	not tested	not tested	$8.4/GB	See Bright Data pricing
	0 / 5HTTP 403	4 / 5	$2/GB	See Decodo pricing
	0 / 5HTTP 403	5 / 5	$5/GB	See SOAX pricing

Five requests per source per method against Indeed, measured 10 Sep 2026 (plain) and 10 Sep 2026 (browser). The reason shown is what the site returned.

What happened

Neither half works alone

Plain requests: refused everywhere

Status 403 from the datacenter address and from all three residential networks. Cloudflare’s challenge is served as a page, and a client with no JavaScript engine can never complete it. The address type made no difference at this stage, which is why a proxy alone does not solve this target.

Browser from a datacenter address: refused

The same headless Chromium that succeeded through a residential proxy was refused five times out of five when it came from our hosting-company address. The challenge was offered but the result was not accepted.

Browser through a residential address: it opened

The challenge resolved on its own within a few seconds and the job cards arrived. One provider passed all five attempts and two passed four of five. No CAPTCHA solving service, no stealth plugin, no fingerprint patching.

Why the pair matters

Cloudflare weighs several signals together. A trusted address without a JavaScript engine cannot pass the test; an untrusted address that passes the test is still refused. Only both together clear the bar, and that is the clearest argument in our whole test for paying for a residential proxy on top of running a browser.

If you are building this

What we would do to collect Indeed pages

Budget for both, or do not start. A residential proxy alone will not work and a browser alone will not work. The cost is the proxy traffic plus the CPU to run browsers, and on this target that is the price of entry.
Count the traffic, not the requests. A browser moved roughly ten times the bytes of a plain request in our runs. On per-gigabyte pricing that is the number that decides your bill, so block images, fonts and media as we did.
Expect misses and retry through a new address. Two of the three providers passed four attempts out of five. A failed challenge is normal; treat it as a retry with a fresh exit address rather than an error.
Give the challenge time. Our runs waited up to fifteen seconds after load for either the job cards or the challenge to resolve. A scraper that gives up after two seconds will record a block that was about to become a page.
Watch for the silent failure. Cloudflare can return a full-size page with no job cards in it. Check for the content you need, never for status 200; that rule is why our numbers differ from comparisons that count responses.

Method

How this was measured

The same script and server as every other number on this site.

Raw results

5 requestsper target per provider, each from a different exit IP. Pass rate is how many of the 5 brought back the real page.

9 targetsthe sites people actually scrape: retail, real estate, jobs, search. Picked for their bot defences.

Content, not statusa 200 with no results is a failure. Every response is checked for a marker only the real page has.

Monthlyrerun on the same day each month from the same server. Old results stay published, dated.

The target is an Indeed job search page, requested five times per source. Plain requests carry the headers of a current desktop Chrome and no JavaScript engine. Browser requests use a headless Chromium through the same proxy, each in a fresh browser context so no cookie or challenge result carries over, waiting up to fifteen seconds for job cards or for the challenge to resolve. No CAPTCHA solving service and no anti-detection plugins were used. The two runs were made on different days, which is stated in the table.

Providers

What the working combination costs

List prices for the providers whose addresses cleared the challenge. Full results across all nine targets are on the front page.

Pool	Locations	Targeting	Sessions	Trial	Entry price
100M+	195 countries	City · ASN	Rotating and sticky	None on self-service	$6/GBStarter 5 GB, $30/mo, cancel any time	See Oxylabs pricing
72M+	195 countries	City · ASN · ZIP	Rotating and sticky	Free trial for verified businesses	$8.4/GBPay as you go, no commitment	See Bright Data pricing
115M+	195+ locations	City · ZIP · ASN	Rotating and sticky (minutes to days)	3-day trial, 100 MB	$2/GBPay as you go $4/GB, plans from $2/GB	See Decodo pricing
155M+	195+ locations	City · ISP	Rotating and sticky, custom length	3-day trial, 400 MB, $1.99	$5/GBPay as you go $5/GB, $3/GB on $200/mo plan	See SOAX pricing

We earn a commission if you buy through the pricing links. It does not change the numbers: the test script, the target list and the raw results are public, and providers do not see results before you do.

Best in our testOxylabs · from $6/GBSee pricing