JSON Parsing Python: Dictionary Data Processing Methods

JSON as a courier package to open?Python dictionary so play it right!

Engage in data like receiving express delivery, proxy IP service providers often return JSON data wrapped tightly. Today we use a chopper to cut watermelon way, hand in hand to teach you how to use Python to dismantle this "data package", focusing on how to combine in practice ipipgo proxy service to use.

Guide to decrypting courier orders

Let's take a look at a typical scenario: when using ipipgo's API to get a proxy IP, the returned JSON looks like this:

{
    "code":200,
    "data":[
        { "ip": "1.1.1.1", "port":8000, "expire": "2024-03-01"}, { "ip": "2.2.2.2", "port":9000, "expire": "2024-03-01"}, }
        {"ip": "2.2.2.2", "port":9000, "expire": "2024-03-02"}
    ]
}

It's like the shelves at a delivery station.The shelf number data.Multiple packages (IP information) are placed underneath. The correct position to open the packages with Python is:

import requests
import json

resp = requests.get('https://api.ipipgo.com/getips')
ip_data = json.loads(resp.text)

 The point! First make sure the delivery is not wrong
if ip_data['code'] == 200.
    for package in ip_data['data'].
        print(f "Available IP:{package['ip']}:{package['port']}")

Common Rollover Scene Rescue

Newbies often get stuck in these potholes:

1. Wrong key.: clearly want to open the data door, the result is written as date. suggest first print(json.dumps(data,indent=2)) to print out the whole structure

2. Air Parcel Warning: some APIs return data may be an empty list, remember to add an if judgment: if ip_data.get('data')

How to break multiple layers of nesting

Come across this Russian nesting doll type data:

{
    "node1":{
        "node2":[
            { "detail":{ "ip": "3.3.3.3.3"}}
        ]
    }
}

IP-basedcorrect postureIs:

ip = data['node1']['node2'][0]['detail']['ip']

If you're afraid of making a mistake in writing, you can take it apart layer by layer like peeling an onion:

layer1 = data.get('node1',[{}])
layer2 = layer1.get('node2',[{}])
first_item = layer2[0] if layer2 else {}
detail = first_item.get('detail',{})
ip = detail.get('ip','not found')

Practical Tips and Tricks

When used in conjunction with the ipipgo agent, remember thisGolden Triple Strike::

proxies = {
    'http': 'http://user:pass@1.1.1.1:8000',
    'https': 'http://user:pass@1.1.1.1:8000'
}

resp = requests.get('https://需要访问的地址', proxies=proxies, timeout=5)
resp.raise_for_status() Important! Report an error if you find an exception
data = resp.json()

question-and-answer session

Q: Why does my JSON parsing keep reporting errors?
A: eighty percent of these three problems: 1) the network request itself failed 2) the return is not the standard JSON format 3) did not handle exceptions. Suggest using try...except to wrap the parsing process

Q: How can I quickly test the availability of ipipgo's proxy IP?
A: You can use this quick check code:

import requests
from concurrent.futures import ThreadPoolExecutor

def test_proxy(proxy)::
    try: resp = requests.get('')
        resp = requests.get('http://ip.ipipgo.com/checkip', proxies={'http':proxy}, timeout=3), timeout=3)
                          proxies={'http':proxy}, timeout=3)
        return "success" if resp.status_code == 200 else "failure"
    except.
        return "Timeout"

 Test multiple IPs at the same time with multiple threads
with ThreadPoolExecutor(10) as exe: results = exe.map(test_proxy, test_proxy, test_proxy, test_proxy)
    results = exe.map(test_proxy, ['1.1.1.1:8000','2.2.2.2:9000'])

Q: Why do you recommend using ipipgo's services?
A: Three hardcore reasons: 1) Exclusive IP pool without crashing 2) Support pay-per-volume without wasting 3) Have professional technical customer service on standby 7×24 hours. Especially when doing large-scale data collection, a stable proxy service is your digital bulletproof vest.