How to Scrape YouTube Videos With Python: Metadata, Comments, Channels, and Search

To scrape YouTube videos with Python, use the official YouTube Data API when you need structured video metadata, comments, channel details, or search results. It is more stable than parsing YouTube's rendered pages and gives each field a documented meaning. Use a page extractor or a third-party tool such as yt-dlp only for a narrower requirement after confirming that YouTube has authorized that form of automated access.

In this guide, scraping YouTube videos means collecting public data about videos. “Scrape” reflects the common search term; the primary implementation uses YouTube's documented API and does not scrape YouTube application pages. It also does not copy the video or audio file. Download or reuse media only when you own it, have the necessary permission or license, and YouTube's terms allow the workflow.

The tutorial below builds a Python YouTube scraper that can:

• retrieve a video's title, description, channel, publication time, duration, and available statistics;
• search for public videos and enrich the results with full video metadata;
• paginate through top-level comments;
• resolve a channel handle and list its public uploads; and
• return metadata from one video URL with yt-dlp without downloading the media.

Quick Answer

For most projects, create a Google Cloud project, enable YouTube Data API v3, generate a restricted API key, and call videos.list, search.list, commentThreads.list, channels.list, and playlistItems.list from Python. Follow each response's nextPageToken, store raw identifiers and collection timestamps, and treat counts as time-specific observations.

Do not start with Beautiful Soup selectors copied from the YouTube website. Most useful content is rendered or embedded dynamically, the markup changes, and direct automated access may be restricted. A proxy changes the network route; it does not grant permission, stabilize selectors, or increase an API key's quota.

Key Takeaways

• The official YouTube Data API is the best default for structured, documented data.
• A YouTube video scraper normally collects metadata, comments, channel data, and search results—not the underlying media file.
• Public read requests can use an API key. Private account data and actions require OAuth 2.0 authorization.
• Search, comments, and channel-upload lists are paginated. Use returned page tokens instead of inventing page numbers.
• View, like, comment, video, and subscriber counts are snapshots. Some fields can be hidden, rounded, disabled, deleted, or unavailable.
• YouTube's API policies govern how long API data may be stored and whether derived metrics are allowed; public availability does not create unrestricted reuse rights.
• yt-dlp can extract metadata without downloading a file, but it is an unofficial third-party tool and can break when YouTube changes.
• Proxies are usually unnecessary for the official API. For a separately authorized location-aware collection workflow, residential proxies are the practical default; mobile proxies are for genuinely mobile-specific tests.
• Always review the current API policies, YouTube Terms of Service, copyright permissions, privacy obligations, and applicable law before collecting or reusing data.

What You Can Collect From YouTube

The YouTube Data API separates resources by type. Choosing the right resource avoids unnecessary calls and makes the output easier to validate.

This distinction matters. If you already know a video ID, use videos.list rather than performing a search. If you need every public upload from a known channel, resolve its uploads playlist and paginate through playlistItems.list instead of repeatedly searching by channel name.

Choose the Right YouTube Scraping Method

There is no single best method for every YouTube data task.

Use the least fragile source that answers the question. For public metadata, that usually means the Data API. For analytics on your own channel, use the authenticated Analytics API rather than estimating performance from public page elements.

Set Up the YouTube Data API

You need Python 3.11 or newer, a Google account, and a Google Cloud project. Google's YouTube Data API getting-started guide documents the current setup flow.

1. Create or select a project in Google Cloud Console.
2. Enable YouTube Data API v3 for that project.
3. Create an API key for public read requests.
4. Restrict the key to the YouTube Data API and, where practical, to the applications or source addresses that use it.
5. Store the key outside the source code.

Create a virtual environment and install the two packages used by the complete example:

On Windows PowerShell, activate the environment with:

Create a .env file that is excluded from version control:

Do not expose the key in a public repository, browser bundle, screenshot, proxy URL, or log. If a key leaks, revoke or rotate it in Google Cloud Console.

Complete Python YouTube Scraper

The following command-line script supports four jobs:

Save it as youtube_data.py:

The script uses safe field access because public resources do not always contain every optional field. It also deduplicates IDs and stops at the requested limit or when the API stops returning a continuation token.

Get metadata for one YouTube video

A YouTube URL such as https://www.youtube.com/watch?v=VIDEO_ID contains the video ID after v=. Pass that ID to the script:

The script requests snippet, contentDetails, and statistics. The response can include:

• title, description, tags, category, channel, and publication time;
• ISO 8601 duration and video definition;
• whether captions are available; and
• current view, like, and comment counts when exposed.

Google's videos.list reference documents the available parts and fields. Treat the result as an observation at collection time. A count retrieved today is not a historical count for the video's publication date.

Scrape YouTube search results

Pass a search query and the maximum number of videos to return:

The script first calls search.list to collect video IDs, then calls videos.list to retrieve complete, consistently shaped records. That second step is important: a search-result snippet is not a full video resource and does not include all video statistics or content details.

The search.list documentation allows up to 50 results per response. The script follows nextPageToken until it reaches the requested limit or no next page exists. Do not estimate the number of available pages from pageInfo.totalResults; Google explicitly warns that this value is approximate.

You can add documented filters such as publishedAfter, publishedBefore, regionCode, relevanceLanguage, safeSearch, or videoDuration to the request. Record every filter with the output so a future run is reproducible.

Scrape YouTube comments

Collect up to 250 top-level comments from a video:

The script requests plain-text comment content and follows each returned page token. The commentThreads.list reference permits up to 100 items in one response.

This example intentionally returns top-level comments only. A comment-thread resource may contain some replies, but Google notes that it does not necessarily contain all of them. If every reply is required, read each top-level comment's ID and paginate through comments.list with parentId.

Comments can be disabled or unavailable. The API may return 403 with a specific reason such as commentsDisabled; inspect the structured error reason before deciding what happened. Do not treat every 403 as an instruction to rotate an IP and retry.

Comment text and author information can be personal data depending on the project and jurisdiction. Collect only what the project needs, secure it, define a retention period, and avoid using public comments to infer sensitive traits about individuals.

Scrape a YouTube channel's public videos

Use the channel handle with or without its leading @:

The script resolves the handle with channels.list, reads the channel's uploads playlist ID, paginates through that playlist with playlistItems.list, and enriches the resulting IDs with videos.list.

This is more precise than searching for a channel name. Search results can contain similarly named channels and are ranked rather than designed as a complete upload inventory. Google's channels.list documentation supports channel handles directly, while the playlistItems.list reference documents pagination through the uploads playlist.

Public subscriber counts are rounded under YouTube's policy. Do not present the returned number as an exact internal subscriber total.

How YouTube API Quotas Work

Quota rules have changed over time, so check Google's current quota calculator before estimating capacity.

As documented in October 2026:

• search.list uses a separate Search Queries quota bucket;
• the default Search Queries allowance is 100 calls per day;
• every next page of search results is another search call;
• methods outside that bucket share a default allowance of 10,000 units per day; and
• every API request consumes at least one unit, including an invalid request.

The exact budget for a run depends on methods and pages, not only the number of records. For example, collecting 120 search results at 50 results per response needs at least three search calls, followed by up to three batched videos.list calls to enrich those IDs. Retrieving all comments and replies can require many more calls because each page is a request.

Use these practices to control quota use:

1. Call videos.list directly when you already have video IDs.
2. Request only the resource parts the workflow needs.
3. Use the maximum sensible page size.
4. Cache immutable identifiers and slowly changing metadata.
5. Store the last successful page token or collection watermark for recoverable jobs.
6. Stop a job when the API reports a quota or permission error; repeated invalid calls also consume quota.
7. Request a higher quota through Google's documented process only when the project has a valid need and can meet the applicable policies.

Daily quotas reset according to the schedule stated in the current Google documentation. Do not hard-code an assumed timezone without checking it.

Metadata-Only Extraction With yt-dlp

yt-dlp is an unofficial, open-source media downloader that can also extract metadata. It can be useful for a known URL when the official API does not expose the needed field in the required form and YouTube has authorized direct automated access for the workflow.

It is not a drop-in replacement for the Data API. Extraction can fail when YouTube changes, returned fields may be absent, and using it does not remove copyright, contract, privacy, or access obligations. YouTube's API Developer Policies also prohibit an API client from scraping YouTube applications or obtaining scraped YouTube data, so do not quietly combine this fallback with an API client. Use it only for a separately authorized workflow whose terms and rights you have verified.

Install it in an isolated environment:

This example sets download=False and skip_download=True, sanitizes the returned object, and prints a small metadata subset. It does not download the media:

Set the URL at runtime instead of hard-coding it:

Do not assume every key exists. The yt-dlp project explicitly treats many metadata fields as optional. Preserve the extractor version and collection timestamp so future failures can be reproduced.

Why Beautiful Soup Alone Is Usually the Wrong Tool

Beautiful Soup parses HTML that the HTTP client already received. It does not run JavaScript, call hidden application endpoints, accept consent flows, or turn unstable rendered markup into a supported data contract.

A short script built around copied class names may appear to work once and then return empty or misleading data when:

• YouTube changes its markup or experiments with another layout;
• the response differs by locale, consent state, device, or authentication;
• the required data is loaded after the initial document;
• a challenge or error page still returns HTTP 200; or
• a field has moved into an embedded JSON object.

Browser automation can render the experience, but it does not make undocumented selectors stable. If a permitted workflow truly requires a rendered observation, validate page identity and expected content, version the parser, retain allowed evidence, and fail closed when the schema changes. Do not silently convert an empty parser result into a valid zero.

Build a Reliable YouTube Data Pipeline

A tutorial script is only the collection layer. A reliable pipeline separates retrieval, validation, permitted storage, normalization, and approved analysis:

Preserve identifiers and provenance within the permitted retention period

Use YouTube's video, channel, playlist, comment, and parent-comment IDs as stable keys. Record the collection time, requested method, query and filters, page token where appropriate, API or extractor version, and a job identifier with each batch. Then apply the storage, refresh, and deletion rules that govern that data.

Separate entities

Do not place every field in one flat table. Videos, channels, comments, searches, and collection runs have different lifecycles. A basic model might contain:

This model illustrates how to keep mutable counters separate from entity metadata when retaining historical observations is permitted. It is not permission to keep public API data indefinitely.

YouTube's current Developer Policies generally require non-authorized API data to be deleted or refreshed within 30 calendar days. They also restrict aggregation and the creation of derived data or metrics unless the applicable requirements and approvals are satisfied. Review those rules before implementing historical snapshots, sentiment scores, rankings, or cross-channel comparisons. For owned channels or another authorized use, document the authorization and still apply the policy's refresh and deletion requirements.

Validate before accepting a batch

An HTTP success does not prove that the data is complete. Verify that:

• the response belongs to the requested resource;
• required IDs are present and unique;
• pagination advances instead of repeating a token;
• timestamps parse and use a defined timezone;
• integer counts are non-negative when present;
• the number of stored records matches the number accepted by validation; and
• an unavailable optional field remains null rather than becoming a fabricated zero.

Retry only transient failures

Use bounded exponential backoff with jitter for transient server or network errors. Respect Retry-After when a response provides it. Do not blindly retry invalid parameters, disabled comments, forbidden resources, exhausted quota, or missing permissions. Those require a code, scope, pacing, or access decision.

Minimize what you collect and retain

Collect only fields needed for the stated purpose. Comments and channel details can contain personal information even when publicly visible. Set access controls and deletion rules, and do not retain raw text or API data beyond the period allowed by the applicable policy, authorization, and law. Do not assume that replacing raw comments with a derived score automatically makes the use permissible.

For broader engineering patterns, see the Python web-scraping libraries guide and the pagination in web scraping guide.

Do You Need Proxies to Scrape YouTube?

Usually not when you use the official YouTube Data API. An API key, OAuth scope where required, good quota management, and correct pagination matter more. Routing an API request through new IPs does not create more quota because the quota belongs to the Google Cloud project and credentials.

A proxy can be relevant to a separately authorized workflow when the project needs:

• a controlled network route from a specific country or city;
• repeatable location-aware QA of a public experience;
• isolation between approved collection jobs; or
• resilient egress for infrastructure that cannot expose its origin address directly.

For permitted global public-data work, residential proxies are the practical starting point because they offer broad geographic routing. Use mobile proxies only when the test genuinely requires a mobile carrier identity or mobile-network context. A mobile proxy is not automatically better for ordinary metadata collection.

Proxidize Residential Proxies provide real residential IPs across 195+ countries, with country, city, and ISP targeting, rotating or sticky sessions, and standard HTTP, HTTPS, and SOCKS5 support. The current product page is the source of truth for availability and plan details.

When adding a proxy to an authorized extractor:

• keep credentials in environment variables or a secret manager;
• use a sticky session for a multi-request stateful flow;
• set connect and read timeouts;
• log only a redacted endpoint or internal route ID;
• test the exit location separately from the production target; and
• stop when the target denies access instead of treating rotation as permission to continue.

A proxy provides routing. It does not solve API quota, render JavaScript, repair a parser, accept terms, obtain a license, or guarantee access.

YouTube Terms, Copyright, and Privacy

Technical access and permission are different questions. Before collecting YouTube data:

1. Review the current YouTube Terms of Service, YouTube API Services Terms of Service, and YouTube API Services Developer Policies.
2. Use the official API where it covers the requirement.
3. Obtain authorization for account, private, restricted, or non-public data.
4. Download, reproduce, or redistribute video and audio only when the required rights and permissions are in place.
5. Apply the API policies' storage, refresh, deletion, aggregation, and derived-metric requirements to the data you collect.
6. Respect technical limits and stop conditions instead of trying to defeat them.
7. Get qualified legal advice for high-risk, large-scale, regulated, or cross-border projects.

YouTube's current terms restrict automated access except in stated circumstances, including public search-engine access in accordance with robots.txt or access with YouTube's prior written permission. Its API Services Developer Policies also prohibit API clients from scraping YouTube applications or obtaining scraped YouTube data. The existence of a public page, browser automation library, extractor, or proxy does not by itself authorize collection or reuse.

Common YouTube Scraper Errors

Conclusion

The most maintainable way to scrape YouTube video data is to start with the official YouTube Data API. Use stable IDs, documented resource methods, returned page tokens, quota-aware batching, validation, and timestamped snapshots. Add an unofficial extractor or rendered-page collector only when a specific, authorized requirement cannot be met by the API.

If an approved workflow also needs controlled geographic routing, explore Proxidize Residential Proxies or compare residential and mobile proxies. Use proxies as infrastructure for routing and location—not as a substitute for permission, quota, reliable parsing, or responsible data handling.