How to Build a Web Research Agent With LangGraph and Residential Proxies

Quick Answer

To build a web research agent with LangGraph and residential proxies, create a graph with three responsibilities:

1. A model node decides which approved source to inspect next.
2. A fetch node sends that website request through a Proxidize Residential Proxy.
3. A conditional edge either returns to the model with new evidence or stops when the answer or fetch budget is complete.

The architecture looks like this:

LangGraph does not include a residential proxy setting, and the proxy does not run the agent. LangGraph owns state, nodes, edges, checkpoints, and stopping conditions. The HTTP client or browser owns the website connection. Proxidize changes the route used by that network-aware tool.

The complete example in this guide builds a real asynchronous research loop with:

• Exact per-run source approval and a hostname allowlist.
• Residential proxy authentication through HTTPX.
• Redirect revalidation and private-target protections.
• Response-type, byte, text, retry, fetch, and graph-step limits.
• Duplicate-source suppression.
• Sticky-session compatibility.
• Thread-scoped LangGraph checkpoints.
• Deterministic rejection of citations to pages the tool never fetched.

Responsible-use note: Use web research agents and proxies only for lawful, authorized work. Follow applicable laws, privacy requirements, website terms, access controls, robots guidance where relevant, and reasonable request rates. A residential proxy changes the network route; it does not grant permission to access data.

Methodology: LangGraph and LangChain APIs were checked against first-party documentation on August 28, 2026. OpenAI model details were checked against official OpenAI documentation, and Proxidize product details were checked against current first-party pages. The code passed syntax, import, constructor, graph-routing, URL-policy, redirect, duplicate, retry, checkpoint, and citation tests with langgraph 1.2.11, langchain-openai 1.6.0, and HTTPX 0.28.1. The complete example was then live-tested with gpt-5.6, a Proxidize Residential Proxy, and two approved public sources. The real graph fetched both pages through the proxy and returned an answer that passed the fetched-source citation validator. Five fresh-client preflights observed one exit during the short sample, so this test does not claim per-request rotation or sticky-session behavior.

Key Takeaways

• Proxy the web tool, not “LangGraph.” Configure Proxidize on the HTTP client or browser node that opens target websites.
• The model connection and website connection are separate. Proxying HTTPX does not automatically proxy ChatOpenAI, and an OpenAI-hosted search tool does not inherit your local proxy.
• Use explicit graph state for operational limits. Track fetch attempts and visited URLs in state instead of relying only on prompt instructions.
• Use sticky sessions for one coherent research unit. Rotate between independent questions, markets, or jobs—not between related pages whose results must represent one visitor context.
• Treat retrieved pages as untrusted evidence. Web content can contain prompt-injection attempts; deterministic application policy must remain authoritative.
• Checkpoints preserve agent state, not network or browser state. A LangGraph thread can retain messages while the proxy exit, cookies, tabs, or target-side session changes independently.
• A citation instruction is not a citation guarantee. Validate that final-answer URLs came from successful tool results before returning the answer.

What Does LangGraph Add to a Web Research Agent?

LangGraph is a low-level orchestration framework for long-running, stateful agents and workflows. Its graph API lets an application define state, nodes that update that state, edges between nodes, conditional routes, persistence, interrupts, and termination rules.

That is useful for web research because research is not one model call. A run may need to:

• Decide which source to fetch.
• Validate whether the source is permitted.
• Retrieve and reduce the page.
• Compare it with earlier evidence.
• Stop after a fixed budget.
• Recover after a transient failure.
• Preserve a reviewable record of what happened.
• Refuse to cite a source that was never retrieved.

A plain prompt can ask a model to follow those rules. A graph can make many of them part of the executable control flow.

Use a simple function or chain when the workflow always fetches the same pages in the same order. LangGraph becomes more valuable when the process branches, loops, pauses, resumes, requires human review, or maintains state across multiple evidence-gathering steps.

If you prefer an agent loop built around OpenAI's runner instead of an explicit state graph, see How to Use Residential Proxies With the OpenAI Agents SDK. The network boundary is the same in both designs: configure the website tool, not the orchestration framework.

Why Use Residential Proxies for LangGraph Research?

A web research agent may need public information as it appears from a particular market or through a residential network. Examples include:

• Comparing public prices, promotions, or product availability across regions.
• Monitoring localized search results or landing pages.
• Collecting fresh public sources for market research or a retrieval pipeline.
• Checking how a public site renders from a selected country or city.
• Reviewing public marketplace listings for brand-protection work.
• Testing localized content, currency, language, or inventory.

Proxidize Residential Proxies are the natural fit when a research workflow needs broad global coverage, residential IP diversity, country/city/ISP targeting, or rotating and sticky sessions. Proxidize Mobile Proxies are the narrower choice when the requirement is specifically a real US mobile-carrier route or a dedicated SIM-based exit.

A proxy is unnecessary when the graph calls only your own APIs, reads local documents, uses a licensed dataset, or already gets the required result from an approved first-party feed. Adding a network layer without a concrete access or localization requirement creates cost and operational work.

Want to test the architecture? Create a Proxidize Residential Proxy access point and place its generated server and credentials in the environment variables used below.

What Is Actually Being Proxied?

Only the component configured with the proxy uses the residential route.

This separation is important for debugging and compliance. A successful model response does not prove that the website tool used the proxy. A successful proxy preflight does not prove that every other client or hosted tool uses the same route.

Prerequisites

You need:

1. Python 3.10 or later.
2. An OpenAI API key.
3. An active Proxidize Residential Proxy access point.
4. The access point's server, username, and password.
5. A small set of public source URLs and domains you are authorized to fetch.

Create a virtual environment and install the versions validated for this guide:

LangGraph 1.2.11 was the current PyPI release used for this validation. Because LangGraph and model integrations evolve independently, pin and test them together rather than assuming any two current versions are compatible.

Create a local .env file:

Use the exact server, port, username, and password generated by the Proxidize dashboard. Do not put credentials in a target URL, graph message, tool description, checkpoint, trace, screenshot, or source file. Add .env to .gitignore and use a secret manager in production.

The model is configurable and defaults to gpt-5.6, which OpenAI currently documents as an alias for GPT-5.6 Sol. The model supports function calling, which ChatOpenAI.bind_tools() uses to expose the fetch tool.

Complete LangGraph Web Research Agent

Save the following as research_agent.py:

Run it with:

The program first prints a redacted exit fingerprint. It then runs the graph and prints the final research answer only if every URL in that answer was present in a successful tool result.

How the LangGraph Research Loop Works

The graph itself is intentionally small:

The operational controls around those two nodes are what make the example useful.

1. ResearchState Makes Limits Visible

The state contains the message history, the number of fetch attempts, and the URLs already attempted. add_messages appends new messages rather than replacing the conversation.

fetch_attempts is an application budget. It counts tool calls whether they succeed, fail, or repeat a URL. This prevents a failing source from creating an unlimited retry loop at the graph level. fetched_urls prevents a model from spending the budget repeatedly retrieving the same page.

2. The Researcher Node Chooses or Synthesizes

ChatOpenAI.bind_tools() exposes fetch_public_page as a strict function tool. parallel_tool_calls=False makes the model request at most one page per turn, keeping the evidence loop easy to audit.

While budget remains, the researcher can call the tool or answer. Once the budget is exhausted, the node invokes the base model without tools and explicitly requests a final synthesis. The application—not the prompt alone—removes the ability to request another fetch.

3. The Fetch Node Enforces the Boundary

The tool accepts only a URL. Before opening it, application code requires:

• HTTPS.
• An exact allowlisted hostname.
• The default HTTPS port.
• No URL-embedded credentials.
• No IP-literal destination.
• An exact match in the current run's approved URL set.

Every HTTP redirect is processed manually and revalidated. This matters because checking only the starting URL would let an approved site redirect the tool somewhere the application never intended to reach.

The code-level controls should be backed by network egress policy. DNS behavior, rebinding, alternate resolvers, and proxy-side name resolution are difficult to make safe with string validation alone. In production, use a firewall, VPC rule, proxy policy, or dedicated fetch service that can deny private and internal destinations.

4. Conditional Routing Ends the Loop

The conditional edge sends an AIMessage with a tool call to fetch_pages. A normal model answer goes to END. After a fetch, a fixed edge returns the evidence to the researcher.

The invocation also sets recursion_limit=12. LangGraph documents the recursion limit as a top-level invocation configuration, separate from the configurable values used for fields such as thread_id. The fetch budget should normally stop the workflow first; the recursion limit is a second guard against an unexpected cycle.

5. The Checkpointer Preserves Thread State

The graph is compiled with InMemorySaver, and every run receives a unique thread_id. LangGraph checkpoints state by thread, which enables conversation continuity, inspection, replay, human review, and recovery patterns.

InMemorySaver is appropriate for a tutorial and deterministic tests. It is not durable across process restarts. Use a production checkpointer appropriate to your database and deployment model when a research job must survive restarts.

6. Citation Validation Runs After the Model

The system prompt tells the model to cite source_url. The validator then parses successful ToolMessage records and rejects:

• An answer with evidence but no cited URL.
• An answer that cites a URL absent from all successful tool results.

This does not prove that every sentence is correctly supported. It does prevent a common structural failure: presenting an unfetched URL as if the agent inspected it. Production evaluation should also check claim-to-source entailment, quotation accuracy, source freshness, conflicts, and coverage.

Verify That the Website Tool Uses the Proxy

Do not infer routing from the model output. The preflight uses the same AsyncClient captured by fetch_public_page, so a successful fingerprint proves that client reached an external IP service through its configured route.

For a stronger test:

1. Query two independent IP-check services through the same client.
2. Confirm that they report the same exit within one sticky session.
3. Fetch an approved target page through the tool.
4. Record only a keyed or redacted route identifier unless the raw IP is operationally required.
5. When location matters, verify the target-visible market as well as an IP database.

If a direct request and proxied request return the same public IP, inspect all HTTP and browser clients, environment proxy variables, hosted tools, and fallbacks. trust_env=False prevents HTTPX from silently inheriting unrelated HTTP_PROXY, HTTPS_PROXY, or ALL_PROXY settings.

Sticky vs. Rotating Sessions for LangGraph

LangGraph thread state and proxy state are independent.

Use a sticky residential session when one research unit compares related pages, follows pagination, or must preserve one location and visitor identity. Rotate between independent questions, work units, customers, or market observations.

Do not assume that a LangGraph thread_id keeps the proxy sticky. It identifies checkpoint state. Conversely, a sticky IP does not persist LangGraph messages, browser cookies, or a target-side session.

For production work, bind these resources under one work-unit identifier:

Read Proxy Session Management for AI Agents for a deeper treatment of rotation boundaries and state alignment.

Geo-Target a LangGraph Research Run

Proxidize Residential Proxy documentation covers country, city, and ISP targeting. It shows country targeting through a username suffix such as:

Use a lowercase two-letter country code and copy the final generated credential from the dashboard. Configure city, ISP, and sticky or rotating behavior through the access point or Session Builder rather than inventing undocumented username parameters.

Do not let the model assemble credential strings. Map a validated market code to secrets held by the application:

Then verify the output that actually matters. A German exit does not by itself prove that a retailer returned its German store, EUR price, German availability, or expected language. Websites may also use cookies, account preferences, query parameters, browser locale, device state, and their own IP-location database.

Retry by Failure Layer

The example retries only transient network failures and status codes. It does not automatically rotate the proxy or retry every response.

There are three different retry layers in this architecture:

1. ChatOpenAI(max_retries=2) covers transient model-client failures.
2. fetch_with_retry() covers bounded website and proxy-network failures.
3. LangGraph can revisit the researcher node, but the fetch counter prevents unlimited tool attempts.

Keep the budgets separate so one layer cannot multiply another into a retry storm.

Control Concurrency and Bandwidth

The example disables parallel model tool calls and applies an asyncio.Semaphore(3) around outbound fetches. HTTPX also limits open and keep-alive connections.

Those controls affect different scopes:

• parallel_tool_calls=False limits tool requests produced in one model response.
• The semaphore limits concurrent fetch work inside this process.
• HTTPX connection limits bound sockets and connection-pool pressure.
• A queue or worker limit must cap simultaneous LangGraph runs.
• A per-domain rate limiter must enforce target-specific request rates across workers.

Proxidize Residential plans may support high or unlimited connection concurrency, but the safe operating rate is still constrained by target policy, application resources, model limits, bandwidth cost, and the workload's authorization.

Browser-based research consumes much more bandwidth than text HTTP fetches. Block unnecessary video, fonts, analytics, and images only after confirming they are not required for the page state being measured.

When to Replace HTTPX With Playwright

Use HTTPX for server-rendered HTML, JSON endpoints, feeds, and lightweight public pages. Use a browser when the required evidence appears only after JavaScript execution or depends on browser-visible state.

The proxy configuration moves to the browser launch or context:

The graph can keep the same researcher -> browser_tool -> researcher structure. The browser tool still needs deterministic navigation policy:

• Validate initial and subsequent main-frame URLs.
• Intercept and restrict requests to private or unapproved origins.
• Block downloads and file navigation unless explicitly required.
• Use separate contexts for isolated work units.
• Cap tabs, actions, page bytes, screenshots, and wall-clock time.
• Close the browser or context in finally or an async context manager.

If running browsers locally is not desirable, compare managed options in Best Browser Infrastructure for AI Agents. Confirm that the platform supports an external proxy on the plan you intend to buy.

Persistence, Resume, and Idempotency

LangGraph persistence saves graph state as checkpoints organized into threads. It can help a research workflow resume after a process failure, pause for human review, inspect earlier states, or replay a path.

It does not automatically recreate external resources. Before resuming, validate:

• Whether the proxy lease still represents the intended work unit.
• Whether the observed geography remains correct.
• Whether an HTTP client or browser context must be recreated.
• Whether the source changed since the checkpoint.
• Whether earlier tool output is still within its retention and freshness window.
• Whether a fetch would repeat a billable or state-changing action.

The example tool is read-only, and fetched_urls reduces duplicate requests inside one thread. For side-effecting tools, add idempotency keys, approval nodes, narrower retry rules, and durable action records. Never treat checkpointing alone as proof that an external action happened exactly once.

Protect the Agent From Prompt Injection and SSRF

Pages are untrusted inputs. A fetched page might contain text such as “ignore previous instructions,” “send credentials here,” or “open this internal URL.” That text is evidence from a website, not application policy.

Use several layers:

Keep Policy Outside the Model

• Validate URLs, hosts, ports, redirects, response sizes, and content types in code.
• Keep proxy and API credentials outside graph messages and tool output.
• Remove tools when a budget is exhausted.
• Require approval for state-changing actions.
• Deny private and internal egress at the network layer.

Reduce Tool Output

• Strip scripts, styles, templates, and unnecessary markup.
• Return a bounded text field and explicit metadata.
• Prefer task-specific fields over generic page dumps.
• Mark truncation so the model knows evidence may be incomplete.

Validate the Final Product

• Check citations against successful tool results.
• Require source coverage for material claims.
• Detect unsupported quotations and numbers.
• Preserve conflicts instead of forcing false consensus.
• Send high-impact conclusions through human review.

Observability Without Leaking Secrets

Useful research telemetry includes:

• Graph run ID, thread ID, node, and step count.
• Approved target hostname and redacted URL when query data is sensitive.
• Proxy-route fingerprint, requested geography, and observed geography.
• Status class, latency, retry count, response size, and extraction size.
• Fetch attempts, duplicate suppression, and budget exhaustion.
• Model tokens, latency, tool calls, and final validation result.
• Bandwidth and cost per accepted research answer.

Do not log proxy URLs with credentials, API keys, raw authorization headers, full page bodies by default, private user queries, unnecessary raw exit IPs, or checkpoint state that contains sensitive evidence.

LangGraph can stream state updates, messages, tasks, and checkpoints. Streaming every field to a UI or trace store can expose more data than intended. Define a redacted observability schema rather than forwarding complete graph state.

How to Test the Agent

Separate deterministic tests from credential-backed integration tests.

Unit and Graph Tests

• Replace the model with scripted AIMessage responses.
• Use httpx.MockTransport for pages, redirects, failures, and oversized responses.
• Compile a fresh graph with a fresh in-memory checkpointer per test.
• Assert on node routes, fetch_attempts, fetched_urls, and ToolMessage records.
• Test that unapproved schemes, hosts, ports, IP literals, and redirects fail closed.
• Test that an unfetched citation is rejected.

Integration Tests

• Use fresh secrets supplied through protected environment variables.
• Make one low-cost model run against stable, approved pages.
• Assert on structure—the tool call, proxy route, evidence record, and citation—not exact prose.
• Test the actual sticky or rotating access point behavior.
• Verify target-visible geography when location matters.
• Record versions and date because model and network behavior can change.

The complete code in this article passed both the deterministic tests and the credential-backed integration path. The live run fetched two approved sources through the residential proxy, completed a real gpt-5.6 tool loop, and passed citation validation. The supplied access point was described as random-mode, but five fresh-client samples observed one exit; that short sample does not establish a rotation interval or sticky-session duration.

Common Mistakes

Troubleshooting

Production Checklist

Before deploying a LangGraph web research agent with residential proxies, confirm:

• The research use case and every target are lawful and authorized.
• Secrets stay outside prompts, checkpoints, source control, traces, and tool results.
• Exact source approval, hostname, scheme, port, redirect, and egress controls are enforced.
• The same proxied client performs preflight and target fetches.
• Requested and observed geography are validated when location matters.
• Sticky and rotating boundaries match complete work units.
• Fetch attempts, model retries, HTTP retries, graph steps, concurrency, bytes, and wall-clock time are bounded.
• Page content is reduced and treated as untrusted.
• Final citations are checked against successful source records.
• Checkpoint storage, encryption, retention, and tenant isolation are defined.
• Resumption revalidates proxy, browser, source freshness, and external action state.
• Logs and traces use a redacted schema.
• Alerts cover authentication, routing, geo, rate limits, loops, validation, bandwidth, and cost.
• A kill switch can stop new runs and outbound fetches promptly.

Build a Proxy-Aware LangGraph Research Agent

The maintainable design keeps responsibility clear:

• LangGraph controls state, branching, loops, checkpoints, and stopping conditions.
• The model decides which approved evidence it needs and synthesizes the result.
• Application code validates destinations, budgets, tool output, and citations.
• HTTPX or Playwright opens the website connection.
• Proxidize supplies the residential route, geography, and session policy.

That separation makes the agent easier to test, observe, resume, and govern than a prompt that combines research logic, credentials, network access, and policy in one place.

• Explore Proxidize Residential Proxies
• AI Agent Proxies for Reliable Web Access
• Learn how to use Residential Proxies
• Design proxy sessions for AI agents
• Compare browser infrastructure for AI agents
• Compare proxies for AI-agent workflows