Python Using Proxies: Requests Configuration Tutorial

Hands-On Proxy Hanging on Python Requests

The old iron engaged in crawling know, do not take the agent to play crawler is like running naked on the battlefield. Today we will use the vernacular nagging how to Python's requests library to install a proxy armor, focusing on recommending our domestic goodies!ipipgoof agency services.

Why do I have to use a proxy?

For example, you visit a website 100 times in a row, the site is not a fool, immediately give you IP off the small black house. At this time, if you use a proxy IP wheel access, like playing face like, the site simply can not catch you. Especially withipipgoThis high stash of proxies can't even touch the hair of your real IP.

Basic Configuration Three-Piece Suit

import requests

proxies = {
    'http': 'http://用户名:密码@ipipgo proxies:port',
    'https': 'http://用户名:密码@ipipgo proxy:port'
}

response = requests.get('destination URL', proxies=proxies)

take note ofUser name and passwordDon't make a mistake, ipipgo can be found in the background. If you use the whitelist authentication is more simple, the proxy address directly fill in the IP:Port they give on the line.

Advanced players play like this

1. Automatic IP change: with ipipgo'sdynamic agent poolThe IP address is randomly selected for each request.

from random import choice

ip_list = ['ip1:port', 'ip2:port', 'ip3:port'] Fill in the ip pool provided by ipipgo.
proxy = {'http': f'http://{choice(ip_list)}'}

requests.get(url, proxies=proxy)

2. Timeout setting: to insure the request and change the proxy if it exceeds 5 seconds

try.
    requests.get(url, proxies=proxies, timeout=5)
except: requests.get(url, proxies=proxies, timeout=5)
    print("This IP is not working, switch to the next one now!")

How to choose an ipipgo proxy package

A guide to common pitfalls

Q: What should I do if the agent suddenly fails to connect?
A: First check if the IP is expired, ipipgo has real-time availability monitoring in the background. If frequent dropouts, it is recommended to switch to theirSLA GuaranteedPackages.

Q: What is the meaning of returning 407 error code?
A: This is an authentication failure, check three points: 1. whether the account is in arrears 2. whether the whitelist is bound 3. whether the password has special characters that require URL encoding.

Q: How do I test if the proxy is working?
A: Test with this command first:

print(requests.get('http://httpbin.org/ip', proxies=proxies).text)

If the returned IP is the same as the one given by ipipgo, it means the proxy is working.

Say something from the heart.

Having used seven or eight agency services, the thing that saves me the most with ipipgo is thatresponsiveness. Last time I did real-time price monitoring, the millisecond switching of their dynamic pool was really powerful. Newbies are advised to use theirtrial packageIf you're looking for a good deal, you'll want to check your local network compatibility. By the way, if you're short on cash, you can squat on official events, which often have buy-and-give offers.

Lastly, I would like to remind you that proxies are not a panacea, and random UA and request frequency control are the way to go. Encounter technical problems directly find ipipgo's 24-hour customer service, than their own blind thinking much stronger.