When your Python script starts hitting rate limits or getting blocked mid-scrape, the fix is usually a proxy. The Python Requests library makes it straightforward to route your traffic through one, and once you understand the setup, adding proxy support to any script takes just a few lines of code.
In this article, we'll explore how to configure proxies with Python Requests, from basic HTTP proxies to rotating residential ones.
How to Set Up a Proxy in Python Requests
The Requests library handles proxies through a simple dictionary you pass into your request:
import requests
proxies = {
"http": "http://username:password@proxy_ip:port",
"https": "http://username:password@proxy_ip:port"
\}
response = requests.get("https://example.com", proxies=proxies)
print(response.text)If your proxy doesn't require authentication, remove the username and password from the URL. To apply the proxy across every request without passing it manually each time, use a session:
session = requests.Session()
session.proxies.update(proxies)
response = session.get("https://example.com")
print(response.text)Sessions are the cleaner approach when making multiple requests since you set the proxy once and forget about it.
Also Read: How to Rotate Proxies in Python Requests (2026)
How to Use Rotating Proxies With Python Requests
Rotating proxies cycle through a pool of IPs automatically so the target website never sees the same source twice. Most services give you a single endpoint that handles rotation on their end:
import requests
proxies = {
"http": "http://username:password@rotating.proxyon.io:port",
"https": "http://username:password@rotating.proxyon.io:port"
\}
for i in range(10):
response = requests.get("https://example.com", proxies=proxies)
print(response.text)Every iteration hits a different IP without any extra work on your end. If you prefer managing your own list:
import requests
import random
proxy_list = [
"http://username:password@ip1:port",
"http://username:password@ip2:port",
"http://username:password@ip3:port"
]
for i in range(10):
proxy = random.choice(proxy_list)
proxies = {"http": proxy, "https": proxy\}
response = requests.get("https://example.com", proxies=proxies)
print(response.text)$For most use cases, a managed residential rotating proxy is the better option since you're not dealing with dead IPs or maintaining the list yourself.
How to Test If Your Proxy Is Working
Before running your full script, it's worth confirming that your proxy is actually routing traffic correctly. The easiest way is to send a request to an IP-checking endpoint and see what comes back:
import requests
proxies = {
"http": "http://username:password@proxy_ip:port",
"https": "http://username:password@proxy_ip:port"
\}
response = requests.get("https://api.ipify.org?format=json", proxies=proxies)
print(response.json())$If the IP returned matches your proxy and not your real machine, the proxy is working. If it returns your actual IP, something in your configuration is wrong.
To make this cleaner, you can wrap it in a quick function you run before anything else:
def test_proxy(proxies):
try:
response = requests.get("https://api.ipify.org?format=json", proxies=proxies, timeout=5)
print("Proxy working. IP:", response.json()["ip"])
except requests.exceptions.RequestException as e:
print("Proxy failed:", e)
test_proxy(proxies)$If the proxy fails, you'll get a clear error instead of your script running silently on your real IP. This is especially useful with rotating proxies since you can run it a few times to confirm the IP is actually changing between requests.
Also Read: Python Web Scraping Tutorial (2026)
Final Words
Setting up proxies with Python Requests is straightforward once you have the basics down. For simple tasks, a static proxy gets the job done, but for scraping at scale, rotating residential proxies are the way to go. Proxyon's residential proxies start at $1.75/GB with no subscription required, deposit $5, and start scraping in minutes at Proxyon.
