Proxy.py Web Proxy: Python Lightweight Proxy Tool

What does this Proxy.py thing actually do?

Recently found a very interesting Python library called Proxy.py, do not look at its small size (installation package is less than 2MB), in the field of proxy services is a Swiss army knife. We do web development often encounter the need to debug the request, such as checking the structure of the data returned by an API, or test the loading speed of the site, this time the local start a proxy service is particularly convenient.

To give a real scenario: last week there was a friend doing e-commerce, they need to collect the price data of a platform. If you request directly with a script, the IP will be blocked in a few minutes. At this time, if you use Proxy.py with theDynamic IP Pooling with ipipgoIf you can't get the IP address, you can automatically switch the exit IP address, and the problem is solved.

from proxy import Proxy

 Three simple lines of code to start the proxy
handler = Proxy()
handler.start()

Hands-on with a smart agent service

Installing Proxy.py is as easy as installing a normal Python package, but there are a few configuration items that are particularly useful. The focus is on how to use it in conjunction with ipipgo's proxy IP:

pip install proxy.py
 Specify the upstream proxy at startup (use the ipipgo API interface here)
proxy --hostname 0.0.0.0 --port 8899
      --pac-file https://ipipgo.com/api/get_proxy_list?format=pac

Here's a little trick: mount the PAC file address provided by ipipgo directly on theIntelligent Routing. Requests that need to go proxy are automatically assigned new IPs, and requests that are directly connected stay as they are, much more flexible than global proxies.

Why do I need to pair it with a professional agency service?

It's not impossible to build your own proxy server, but the maintenance cost is too high. Especially when a large number of IPs are needed, it is neither cost-effective nor stable to get your own server. ipipgo has several advantages that are particularly obvious:

Especially for those who do data collection, the actual test with ipipgo'spay-per-use packageThe cost is at least 60% lower than a self-built server room, and you don't have to worry about the IP being blocked.

Frequently Asked Questions QA

Q: Can't I just use a free agent? Why should I buy the service?
A: The availability rate of free proxy is less than 30%, and often encounters the situation of not being able to connect and slow speed. ipipgo's proxy availability rate is guaranteed to be more than 99%, and it also comes with an automatic retry mechanism.

Q: Is there a Python version requirement?
A: Proxy.py supports Python3.6+, it is recommended to use the latest version. If you encounter installation problems, you can upgrade pip first

Q: How to switch proxy IP in real time?
A: Just call ipipgo's refresh interface in the code, and remember to set the interval more than 5 seconds:
requests.get('https://ipipgo.com/api/refresh_ip?key=你的密钥')

Guide to avoiding the pit

Recently, I encountered a typical case: a user set up a proxy but did not take effect, and I found that the system proxy settings were not turned off. Here is a reminder:

1. Windows users remember to turn off the "auto-detect settings".
2. Mac systems to clear the proxy configuration in the network settings
3. It's better to specify the proxy explicitly in the code and not rely on system settings

To test if the proxy is working you can use this command:
curl --proxy http://localhost:8899 https://ipipgo.com/check_ip
Seeing a change in the returned IP indicates success

If you're still worried about getting your IP blocked, we recommend trying theipipgo's 24-hour trial package. They send 5GB of traffic for new users, which is enough to test business scenarios. The key is that the API documentation is exceptionally well written, and docking to Proxy.py is a matter of half an hour.