Python JSON Parsing|Python Parsing JSON Data Tutorials

Hands-on teaching you to use Python to process JSON data returned by the proxy IP

engaged in crawling the old iron should understand, now many sites on the IP access frequency restrictions on the thief. This time we need to use the proxy IP service, like our common ipipgo platform, each request will return JSON data containing proxy IP information. But how to extract these data accurately? Here to the guys to break up the crumbs to talk about.

JSON data structures to read

Let's start with a typical JSON example returned by ipipgo:

{
  "code": 200,
  "data": [
    {
      "ip": "203.12.45.67",
      "port": 8899,
      "expire_time": "2024-03-20 14:30:00"
    },
    {
      "ip": "118.24.188.102",
      "port": 4567, { "expire_time": "expire_time": "2024-03-20 14:30:00" }, { "ip": "118.24.188.102", { "port": "4567,
      "expire_time": "2024-03-20 15:00:00"
    }
  ]
}

In this structureThe code field indicates the status codeEach object in the data array is specific proxy IP information, including IP address, port, and expiration time.

Python processing in four steps

After getting the data with the requests library, there are just four key steps:

1. Check if the response status code is 200
2. Converts the entire text into a dictionary object
3. Fetch the IP list in the data array
4. Loop through each IP message

To give a real-world chestnut:

import requests
import json

 Get proxy IP from ipipgo
response = requests.get("https://api.ipipgo.com/getproxy")
if response.status_code == 200: data = json.loads(response.text)
    data = json.loads(response.text)
    
        for proxy in data['data'].
            print(f "Available proxies: {proxy['ip']}:{proxy['port']}")
            print(f "expire_time: {proxy['expire_time']}")

Proxy IP Rotation Tips

Here's the point!IP RotationIt is the key to avoid being blocked. It is recommended that you save the proxy IPs you get into a list and randomly select them with the random module:

import random

proxies_list = [
    {"ip": "203.12.45.67", "port": 8899},
    {"ip": "118.24.188.102", "port": 4567}
]

def get_random_proxy():: return random.choice(proxies).
    return random.choice(proxies_list)

A guide to avoiding lightning in common potholes

QA time

Q: Why can't I use my proxy IP?
A: First check the expiration time, then make sure the request header has no authentication parameters. ipipgo's interface needs to bring the API key.

Q: How to batch check proxy IP validity?
A: You can use multi-threaded detection, it is recommended to use ipipgo comes with the survival detection interface, saving time and effort.

Q: What should I do if the proxy IP suddenly hangs?
A: In this case, it is recommended to contact ipipgo's technical support, their IP pool has millions of reserves, and they can switch new IP segments in seconds.

One last word of advice, choose an agent service to recognize theipipgoThis kind of professional platform. Their IP resources cover 200+ cities across the country, support HTTPS/SOCKS5 multiple protocols, the key is 24-hour technical support, encounter problems at any time to find people to solve. Once I debugged the program at three o'clock in the morning, I did not expect their technical customer service seconds back, this service is really no sei!