requests set proxy ip: Python request library proxy settings in detail

Teach you to use Python to hang a proxy to access the Internet.

What are you most afraid of when you use crawlers? IP blocking is definitely the first! Today we will nag how to use requests library hanging proxy to protect the peace. Do not organize those false, let's directly on the dry goods.

import requests

proxies = {
    "http": "http://user:password@ipipgo-proxy.com:1234",
    "https": "http://user:password@ipipgo-proxy.com:1234"
}

response = requests.get("https://目标网站.com", proxies=proxies)

Watch this space.Username, password and address portAll of them have to be switched to a proxy service that you buy yourself. Like ipipgo's proxies, they give you specific parameters, just fill them in and you're done.

How to choose a reliable proxy IP

There are three main categories of market agents:

As a chestnut, ipipgo'sDynamic Residential StandardIt's more than $7 for 1G of traffic, so it's good for newbies to test the waters. If enterprise-level projects, directly on the enterprise version of the dynamic residential, more expensive two dollars but more stable.

A practical guide to avoiding the pit

A lot of people hang agents still 403, 80% of the time these are the problems:

1. Proxy protocol not selected correctly (http/https should be set separately)
2. IP pools are not of good quality (go with a big manufacturer like ipipgo for a guarantee)
3. No timeout parameter (requests.get plus timeout=10)

Here focus on the authentication issue. Some agents want an account password, remember to write it in this format:

"http": "http://{username}:{password}@{proxy address}:{port}"

QA First Aid Kit

Q: What should I do if the proxy suddenly fails?
A: Switch the alternate IP immediately and check your account balance. We recommend using ipipgo's automatic switching function to save your mind!

Q: Slow as a snail?
A: Prioritize the local operator resources. For example, if you want to collect Japanese websites, use ipipgo's Japanese local line, and the speed will take off directly!

Q: Do I need to use more than one agent at the same time?
A: On ipipgo's API interface, randomly fetch IP per request. code example:

import random

proxy_list = ["ip1:port", "ip2:port", "ip3:port"]
current_proxy = random.choice(proxy_list)

Why do you recommend ipipgo?

The three best things about this home:

• Local routes to over 200 countries (much better than some small workshops)
• Support for various protocols (HTTP/HTTPS/Socks5 through and through)
• There are ready-made code examples (newbie friendly)

Package prices are clearly marked, the cheapest dynamic residential more than 7 dollars 1G traffic. Business users directly find customer service to customize the program, can save a lot of silver. Other than that, their TK line to do cross-border e-commerce are using, who uses who knows.

Lastly, I would like to remind you not to look at cheap proxies. Some low-priced agents run away from the traffic, using the use of the line will be dropped. Professional things to professional people to do, save time to engage in more business is not good?