Proxy Cost per Successful Request: How to Measure Real Scraping Cost

A proxy advertised at $1 per GB is not necessarily cheaper for a scraping workload than one advertised at $2 per GB. The lower-priced route can still cost more if it produces more blocked pages, incorrect locations, empty responses, retries, or records that fail validation.

The useful comparison is cost per successful request, or more precisely, cost per valid result. That calculation combines what the proxy billed with the number of responses your application could actually use.

This guide explains the formula, shows how to define success before testing, and provides a Python script for collecting the application-side evidence. It also separates proxy network cost from the full cost of running a scraper.

Quick Answer

Calculate proxy cost per successful request by dividing the proxy cost attributable to a controlled workload by the number of results that passed a predefined content check:

For a pay-per-GB proxy:

Do not count every HTTP 200 response as successful. A valid result should contain the expected content, come from the required location, meet freshness and completeness rules, and avoid known block or challenge pages.

Use the provider's recorded billable usage for the cost calculation. Locally observed response-body bytes are useful diagnostics, but they may omit headers, uploads, redirects, tunnel overhead, compressed transfer differences, browser assets, or other traffic included in the provider's meter.

Key Takeaways

• Price per GB measures an input. Cost per valid result measures usable output.
• Define a successful result before running the test. Otherwise, a block page or empty shell with a 200 status can inflate the result.
• Keep network cost separate from total workflow cost. Compute, browsers, scraping APIs, storage, validation, and engineering can cost more than the proxy traffic.
• Count failed attempts, retries, redirects, and fallback routes in the numerator. Excluding them makes an unreliable route appear cheaper than it was.
• Compare providers with the same targets, locations, client, concurrency, session policy, timeout, retry policy, and validation rules.
• Reconcile local logs with the provider dashboard. Application logs explain whether the result was useful; provider records explain what was billed.
• A small controlled test is evidence for that workload and time window, not proof of universal provider performance.

What Does Cost per Successful Request Mean?

Cost per successful request is the amount spent to obtain one response that meets the workload's acceptance criteria.

The phrase is sometimes used loosely. Three related metrics should remain separate:

These numbers can differ substantially.

Suppose a provider requires a $25 purchase that includes 25GB. A test consumes 5GB and produces 5,000 valid records.

If the traffic is valued at $1 per consumed GB:

But the immediate cash calculation is:

The remaining 20GB should not automatically be described as wasted. If it rolls over and the team later uses it, the longer-term effective cost can still be $1 per GB. The cash metric and consumed-traffic metric answer different questions.

Use the Right Formula for the Billing Model

Pay per GB

For bandwidth-priced access:

Use the provider's billed-usage difference for the controlled test. Do not substitute the size of the CSV or JSON file you saved.

Pay per proxy

For a monthly per-proxy plan:

The attribution rule must be consistent. If one proxy is dedicated to the workload for the whole month, use the full monthly fee. If several workloads share it, define a reasonable allocation based on time, traffic, or another recorded unit.

Do not assume that a plan labelled unlimited has no relevant usage conditions. Check fair-use rules, prioritized traffic, speed changes, rotation controls, concurrent connections, and other documented limits.

Per request, credit, or successful result

Managed scraping products may bill by request, credit, rendered page, browser time, or successful response. The formula is still:

The provider's billing definition of success may not match the buyer's acceptance rule. An API can deliver a response it considers successful even when the requested product, price, local result, or other required field is absent. Validate the returned data independently.

For the broader architecture decision, see Raw Proxies vs Scraping APIs.

A Successful HTTP Request Is Not Always a Valid Result

An HTTP status code describes the protocol response. It does not prove the scraper received the right business data.

A response can return 200 OK and still contain:

• a CAPTCHA or challenge page;
• a consent, login, or access-denied screen;
• an empty client-side shell that requires JavaScript;
• the wrong country, language, currency, or inventory;
• stale content;
• a soft-error page;
• a page with missing required fields;
• a duplicate of a record already collected;
• a truncated or malformed response.

Define success at two levels.

Retrieval success

The request completed without a transport error and returned an acceptable protocol response.

Example rules:

• no DNS, connection, TLS, proxy-authentication, or timeout error;
• final status is within the permitted range;
• redirect count is within the expected limit;
• content type matches the intended response.

Business-valid result

The response contains the exact evidence required by the workflow.

For a product-price collector, the rule might require:

• the expected product identifier;
• a parseable price and currency;
• an availability field;
• the requested country or market;
• a collection timestamp;
• no known block-page markers.

For a SERP observation, it might require the query, location, language, device, result list, and collection time. For an ad-verification task, it might require the expected creative and destination link.

Write these rules before the provider test. Changing the validator after seeing the responses makes the comparison difficult to defend.

Which Data Should You Collect?

Retain enough evidence to reproduce both the result and the cost calculation.

Use a dedicated credential, access point, or proxy where possible. It is much harder to reconcile usage when unrelated workers share the same endpoint during the test window.

Worked Example: Why the Lower Price per GB Can Cost More

The following is a hypothetical calculation, not a provider benchmark.

Two routes process the same 10,000 attempts. Each route is billed for 5GB. Route A costs $1 per GB but returns 4,000 valid results. Route B costs $2 per GB but returns 9,000 valid results.

Route A has half the unit price, but Route B has the lower network cost per valid result.

That does not mean Route B is universally better. A different target, page weight, location, client, session mode, or time window could reverse the result. The calculation only shows why advertised unit price cannot answer the workload question on its own.

Add the rest of the workflow

Assume Route B's test also incurred:

The total workflow cost becomes:

Use the network-only number to evaluate routing cost. Use the full-workflow number when deciding between your own collector, a managed scraping product, or a hybrid architecture.

How to Run a Controlled Proxy Cost Test

Use a public page you are allowed to access, an owned staging page, or another authorized target. The goal is to compare routes under the same conditions, not to stress a third-party service.

1. Freeze the workload

Keep these settings identical:

• target URLs and their proportions;
• country, city, ISP, ASN, or carrier requirement;
• HTTP client or browser version;
• headers and cookies;
• rotating or sticky-session behavior;
• connection reuse;
• timeout and retry policy;
• concurrency;
• request pacing;
• JavaScript and asset-loading behavior;
• validation rules.

Changing several variables at once prevents a clean comparison.

2. Record the provider usage before the run

Choose a short, exact time window and record the starting balance or usage. Isolate the test with a dedicated access point or credential if the provider supports it.

Do not record a password in the experiment sheet. Record the credential or access-point identifier instead.

3. Run the same sample through each route

A representative sample matters more than an arbitrary request count. It should include the normal target mix, locations, response sizes, and session behavior. Start with a bounded pilot, then repeat across different times if the workload is sensitive to inventory or network conditions.

Do not add hidden retries to only one route. If the production system retries, record each attempt and include its cost.

4. Record the ending provider usage

Calculate the test's billed usage from the same dashboard scope:

If the dashboard updates asynchronously, wait for the documented reporting delay before recording the final value. Keep the raw before-and-after evidence.

5. Reconcile application and provider records

The application knows whether the response passed the content rule. The provider knows what crossed its network and what it billed. Compare timestamps, access point, request count, traffic, and errors.

Investigate material differences instead of forcing the figures to match. The cause may be uploads, headers, redirects, browser subrequests, decompression, reused tunnels, a billing multiplier, unrelated workers, or differences between local and provider metering.

The high proxy data-usage guide provides a deeper traffic-reconciliation procedure.

Python Script: Measure Valid Results and Calculate Their Cost

The script below has two commands:

1. run sends a controlled sample and writes one CSV row per attempt.
2. calculate combines that CSV with the billed-GB difference from the provider dashboard.

It intentionally does not estimate billed traffic from len(response.content). Requests exposes the decoded response body to the application, while a provider may meter additional or different bytes. The local body size remains useful for diagnosing unusually heavy responses.

The example uses an HTTP proxy connection. Copy the complete generated proxy URL from your provider rather than guessing its credential format. If PROXY_URL is absent, the script prompts for it without echoing the value.

Install and run it

Use Python 3.10 or newer. The current Requests documentation lists Python 3.10+ support. Create an isolated environment:

On Windows PowerShell:

Run a controlled proxy sample:

The script securely prompts for:

The input is not echoed or written to the CSV. For an automated environment, provide the complete URL through a protected PROXY_URL environment variable or secret manager. Do not commit it to source control.

The script uses explicit connect and read timeouts. Requests does not time out by default, so production clients should set them deliberately; the official timeout documentation explains the distinction.

After the provider dashboard has recorded the test, calculate the cost. Replace 0.012345 with the observed billed-GB difference and 1.00 with the applicable unit rate:

--other-cost is optional. Use it only for measured costs attributable to this sample, such as incremental browser compute or a paid fallback API. Do not enter an unsupported estimate and present it as an observed result.

Adapt the validator to the task

The example marks a response valid when it has a 2xx status and contains one expected text string. That is enough to demonstrate the measurement flow, but it is not a universal scraper validator.

Replace validate_response() with rules for the intended output. A product collector could parse a product ID, price, currency, and stock state. A JSON API could validate required keys and types. A localized test could also verify country, currency, and language.

Do not add broad anti-bot or ban-evasion behavior to the validator. The purpose is to identify whether an authorized collection task returned the expected data.

How to Compare Two Providers or Routes Fairly

Use one row per provider, product, configuration, target group, and test window.

Do not combine several products into one provider-wide score. A provider's residential pool, mobile pool, datacenter network, and managed scraping API can have different pricing, locations, session controls, and billing rules.

Also avoid testing one provider with the fastest location and another with a distant mandatory location. Either hold the location constant or clearly explain why it could not be matched.

Report uncertainty and scope

A publishable test should state:

• exact test date and UTC window;
• sample size;
• target categories or named controlled targets;
• client and dependency versions;
• source region;
• requested proxy locations;
• session and rotation behavior;
• concurrency and pacing;
• timeout and retry rules;
• content-validation rules;
• pricing and billing assumptions;
• failures and exclusions;
• whether results were measured by the publisher or supplied by a provider.

Do not turn a small sample into a claim that one provider is universally fastest, cleanest, or most successful.

Why Cost per Successful Request Changes

Response weight

A lightweight API response and a rendered browser page can differ by orders of magnitude. Images, scripts, fonts, video, analytics, advertisements, and background requests can dominate browser traffic.

The saved record may contain only a few fields while the browser transfers several megabytes to create it.

Failure timing

An immediate connection failure may transfer little data. A content-validation failure can occur only after the full response has been downloaded. Both are failed results, but their bandwidth cost is different.

Retries

A retry is another attempt. If the first response transferred most of the page before failing validation, the retry can nearly double the traffic required for one valid record.

Track original attempts and retries separately. A bounded retry policy can improve final completion, but it does not make the first failure free.

Browser and connection behavior

Connection pooling, caching, service workers, prefetching, long-lived connections, and browser context reuse can affect both traffic and result quality. Test the same behavior you plan to operate.

If one comparison run uses a warm cache and another uses a fresh context, the traffic figures do not describe the same workload.

Targeting and billing rules

Some providers charge different rates or apply traffic multipliers for advanced geographic or network targeting. Record the exact configuration and use the resulting billed units rather than assuming every gigabyte has the same cost.

Unused traffic and minimum purchases

An underused package raises the immediate cash cost per result. Whether it raises the long-term economic cost depends on expiration, rollover, cancellation, and future use.

The Residential Proxy Pricing comparison separates purchase amount, traffic received, unused balance, promotions, and expiry rules.

Wrong but plausible data

The most expensive failure may not be an obvious error. Incorrect prices, locations, currencies, duplicate records, or stale pages can pass a superficial validator and corrupt downstream decisions.

Measure accuracy and completeness where the business risk justifies it. Cost per accepted record is meaningful only when the acceptance rule reflects the actual task.

How to Reduce Cost per Valid Result

Reducing unit traffic is useful only if the output remains valid.

Match the client to the page

Use a direct HTTP client when the required content is present in the response and does not require browser execution. Use a browser when JavaScript, interaction, or browser state is necessary.

Do not use a full browser only because it seems more realistic. Do not use a lightweight client when it returns an incomplete shell.

Block unnecessary assets carefully

Images, media, fonts, advertisements, and analytics may be unnecessary for structured text extraction. Test the output with and without each category.

Keep routing conditions consistent across comparison runs. In Playwright, enabling request interception can alter caching behavior, so a controlled comparison should route both the baseline and optimized runs and change only which resources are allowed.

Retry only classified transient failures

Do not retry every invalid result automatically. A missing selector caused by a site change will not be fixed by repeating the same request hundreds of times.

Use bounded retries for specific transient failures, apply backoff where appropriate, and preserve the original failure reason.

Align rotation with task boundaries

Independent requests may use rotating sessions. Multi-step flows often require a sticky session. Changing the route during a dependent sequence can create extra authentication, localization, or state failures.

Separate workloads

Use different access points or credentials for projects, clients, environments, and experiments. Attribution is easier when a high-bandwidth browser job does not share the same endpoint as lightweight API requests.

Validate before scaling

Run a bounded sample before purchasing a large plan or increasing concurrency. Confirm the target, location, content, traffic, latency, and billing behavior first.

For a broader testing framework, use How to Test Proxies.

Raw Proxies vs Scraping APIs: Compare the Whole Job

A raw-proxy price and a scraping-API price use different billing units and include different responsibilities.

Raw proxies can make sense when the team already has a reliable collector and wants control over requests, sessions, locations, parsing, and data storage. A managed API can make sense when faster implementation and outsourced retrieval work justify the service price.

A hybrid system can keep stable, high-volume targets on a self-managed collector and send a bounded set of difficult cases to a managed fallback. Track each route separately so the fallback does not become an invisible default.

How Proxidize Supports Cost Measurement

Proxidize supplies the proxy network layer; it does not decide whether the returned page satisfies your business rule. That separation is useful for measurement.

The current Proxidize Residential Proxy pricing starts at $25 per month for 25GB, or $1 per GB on the entry plan. The page states that unused residential bandwidth rolls over and remains available after cancellation. Use the current plan or contract price in the formula rather than copying this figure into a long-lived spreadsheet without a checked date.

The Proxy Analytics documentation describes two complementary views:

• Data Usage shows consumption trends and domains by volume.
• Requests shows successful and failed network requests, average success rate, and domains by request count.

Connection History can also expose traffic up and down, error type, duration, source IP, protocol, location, session key, access point, and credential identifier. These fields help reconcile a controlled run with provider-side records.

The dashboard's network-level status still does not replace application validation. A network request can complete while the response lacks the product, result, language, or other evidence the workload needs.

Proxidize residential proxies fit teams operating their own scrapers, browser automation, rank trackers, price-monitoring systems, or AI data pipelines. Teams that want the provider to return ready-made structured records should compare the complete cost and scope of managed scraping APIs as a separate product model.

Proxy Cost Test Checklist

Before accepting the result, confirm that you recorded:

• One exact target set and time window.
• The required location and observed location.
• The proxy product, plan, rate, and billing unit.
• A dedicated access point or credential identifier.
• The starting and ending provider usage.
• Total attempts and explicit retries.
• A predefined content-validity rule.
• Valid results and failure reasons.
• Median and high-percentile latency.
• Client, dependency, and runtime versions.
• Network-only cost per valid result.
• Total workflow cost per valid result, when comparing architectures.
• Limitations and any excluded observations.

Conclusion

Price per GB, proxy, request, or credit is only the starting point. A useful proxy-cost comparison measures what the workload actually produced.

Define a valid result before testing, include failed attempts and retries, reconcile application logs with provider-billed usage, and calculate both network cost and complete workflow cost. Then compare routes under the same target, location, session, client, and validation conditions.

That process will not produce one universally best provider. It will show which route delivered the required result at an acceptable cost for the workload you actually operate.