Proxy ip get api: proxy API get interface development practice

Hands-on teaching you to engage in proxy API interface development

Recently, a lot of reptile development old iron asked, how to get a stable proxy IP access interface. Today we will use the vernacular, take ipipgo family service to give a chestnut, teach you to start from scratch the whole job.

API docking is not as hard as you think

First of all, there has to be a reliable proxy provider. Like ipipgo, which gives ready-made API documentation. Their interface returns a format like this:

{
  "code": 0,
  "data": [
    "103.88.46.1:8000",
    "201.15.32.2:8080"
  ]
}

focus onresponse status codeand a list of IPs. In Python, the entire request code would be five or six lines:

import requests

api_url = "https://api.ipipgo.com/getips?key=你的密钥"
response = requests.get(api_url).json()

if response['code'] == 0.
    proxies = [f "http://{ip}" for ip in response['data']]

Avoid the potholes.

During the actual test, there were three places that were particularly easy to roll over:

How Real Business Scenarios Play Out

To give an example of a crawler project, this is how it needs to be configured:

def crawl_with_retry(url).
    for proxy in proxies.
        try.
            return requests.get(url, proxies={"http": proxy}, timeout=10)
        except: continue
            return None
    return None

Pay attention here.timeout settingDon't exceed 10 seconds, encounter jammed hurry to change the next one. ipipgo home dynamic residential IP, the actual test can change 20+ per second, completely enough.

Frequently Asked Questions QA

Q: It works fine in testing, but falls off when you use it officially?
A: 80% of the IP pool is too small, it is recommended to upgrade to the enterprise version of the dynamic residential package, more than 9 yuan 1G traffic enough to build most of the day.

Q: Some websites always recognize that I am using a proxy?
A: Change their static residential IP, 35 bucks a month for that one, same as normal home network.

Q: What should I do if the API suddenly returns empty data?
A: First check if the key is expired, then check the account balance. They have real-time usage monitoring in the background, so don't wait until you run out before you think of charging.

Choose the right package to save big money

ipipgo These packages are the most practical:

• Fresh Start SelectionDynamic residential (standard)7 bucks more for 1G of traffic.
• For studio-scale useEnterprise Edition NewsSupport for high concurrency requests
• Doing account raising must be onStatic homesThe IP can be held for a long time.

Lastly, don't use free proxies to save money. I've seen people trying to save money and end up crawling all the wrong data. Regular channels to buy a proxy, data quality is guaranteed, not to mention that the problem can also find customer service to tear (although ipipgo technical support response is quite fast, basically can not use this trick).