TL;DR
Google Shopping loads its product data dynamically, so a plain GET request returns an empty HTML skeleton with no listings in it. The old tbm=shop URL parameter no longer works either, and the current Shopping tab lives behind udm=28. To scrape it reliably, you need a headless browser like Playwright to render the page, rotating residential proxies to avoid IP bans, and realistic headers to pass Google's bot checks. Parse the rendered product cards, extract titles, prices, and sellers, then export to CSV or JSON.
Google Shopping aggregates product listings, prices, and seller information from thousands of online stores, which makes it one of the most valuable targets for price monitoring and e-commerce research. The problem is that Google actively blocks automated traffic, and most tutorials online still rely on methods that stopped working years ago. In this article, we'll explore how to scrape Google Shopping results in 2026, from building the correct URL to rendering the page, avoiding blocks, and extracting clean product data.
Why Plain Requests Don't Work Anymore
The first thing to understand is that Google Shopping is not a static page. When you send a normal GET request using a library like Python's requests, you get back an HTML skeleton with no product data. The actual listings load through asynchronous calls that only fire after the page renders in a real browser.
The second problem is the URL itself. Many older guides still use the tbm=shop parameter, but it's no longer supported. Google replaced it, and the Shopping tab now sits behind udm=28. So the correct search URL in 2026 is the standard Google search endpoint with your query plus the udm=28 parameter appended at the end.
If a tutorial tells you to combine requests with BeautifulSoup and a tbm=shop URL, it was written for a version of Google that no longer exists. You will get either an empty page or a redirect.
What You Need to Scrape Google Shopping
Three things make this work: a headless browser, rotating proxies, and realistic request behavior.
The headless browser handles rendering. Playwright is the best starting point because it renders JavaScript, waits for network activity to settle, and gives you the post-render DOM to parse. The official Playwright docs cover installation in one command.
The proxies handle detection. Google tracks request volume per IP, and repeated queries from the same address trigger CAPTCHAs fast. Rotating residential proxies solve this because every request exits through a different real-user IP, which is exactly the traffic Google expects to see. Datacenter proxies are cheaper and faster, but Google flags their IP ranges more aggressively, so save them for lighter workloads.
The request behavior handles everything else. Set a real browser User-Agent string, disable Playwright's automation flags, and add random delays between actions. Google checks more than your IP, and a perfect proxy with default bot headers still gets caught.
Also Read: How to Set Up Rotating Proxies for Web Scraping
Building the Scraper in Python
Here is a minimal Playwright setup that routes through a rotating proxy and pulls the rendered Shopping page:
import asyncio
from playwright.async_api import async_playwright
async def scrape_shopping(query):
async with async_playwright() as p:
browser = await p.chromium.launch(
headless=True,
proxy={
"server": "http://gate.proxyon.io:port",
"username": "user",
"password": "pass",
},
)
page = await browser.new_page(
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36"
)
await page.goto(f"https://www.google.com/search?q={query}&udm=28")
await page.wait_for_load_state("networkidle")
html = await page.content()
await browser.close()
return html
html = asyncio.run(scrape_shopping("mechanical keyboard"))Once you have the rendered HTML, parse the product cards with BeautifulSoup and extract the title, price, seller, and rating from each one. One thing worth knowing: Google changes its class names constantly, so target structural attributes and container patterns rather than exact class strings, and keep a copy of the raw HTML from failed runs so you can debug selector breakage later.
Also Read: How to Do Web Scraping Without Getting Blocked
Scaling Without Getting Blocked
Get one page working reliably before you touch concurrency. Once it does, the scale rules are simple. Pace your requests with random delays, rotate your IP on every request, and retry failed pages through a fresh proxy instead of hammering the same one. Handle the consent and regional prompts that Google shows in some countries, because a scraper that ignores them parses an empty page and reports success.
Geo-targeting matters more than it sounds. Google Shopping shows different products, prices, and currencies depending on where the request comes from. If you are tracking prices in a specific market, route your traffic through proxies located in that country, otherwise, your dataset mixes regional results and becomes useless for comparison.
FAQ Section
Is it legal to scrape Google Shopping?
Scraping publicly visible data is generally considered legal in most jurisdictions, but Google's terms of service prohibit automated access. Stick to public listings, keep your request volume reasonable, and consult a lawyer if you plan to build a commercial product on top of the data.
Why am I getting an empty page even with a proxy?
Either you are using a plain HTTP request instead of a rendered browser, or Google served you a consent prompt your scraper did not dismiss. Check the raw HTML of the response before blaming the proxy.
Can I use the old tbm=shop URL?
No. Google deprecated it, and the Shopping tab now uses the udm=28 parameter. Update any scraper still built on the old URL format.
Do I need residential proxies, or will datacenter proxies work?
Datacenter proxies work for small, slow-paced jobs. For anything at volume, residential proxies last much longer because Google treats their IPs as regular user traffic.
How often does Google change its HTML structure?
Frequently, and without warning. Class names on product cards can change every few weeks, so build your parser around container patterns instead of exact class strings and monitor your success rate to catch breakage early.
Can I scrape prices from a specific country?
Yes. Use proxies located in the target country and set the matching language and region parameters in the URL. Google serves different products, prices, and currencies based on where the request comes from.
Final Thoughts
Scraping Google Shopping in 2026 comes down to three things: render the page with a headless browser, use the current udm=28 URL format, and route your traffic through rotating residential IPs. Skip any of the three, and you end up with empty pages or CAPTCHAs.
