Python load JSON files: Python proxy JSON loading

Hands-on with Python to play with JSON and proxy IP!

Do people often encounter this situation? When writing a crawler to load the configuration file, docking API to deal with the return data. This time JSON format has become our good friend. However, when the amount of data or the need for multi-line operation, the light will read and write JSON can not be enough, but also with a proxy IP is stable enough.

JSON basic operation without rolling over

Let's start with the basics for our novice friends. Let's say we have a program calledip_config.jsonfile that holds proxy server information:

{
    "proxy": {
        "host": "123.45.67.89",
        "port": 8080,
        "auth": "user:pass"
    }
}

Loading this file with Python is a matter of three lines of code:

import json

with open('ip_config.json') as f.
    config = json.load(f)

print(config['proxy']['host']) output 123.45.67.89

Pay attention to the file path do not write the wrong, encountered Chinese garbled can add aencoding='utf-8′Parameters. It is recommended to use an editor like VSCode with a checksum function to avoid slipping your hand and writing the wrong symbols.

Proxy IP Practical Tips

After getting the configuration information, let's use the requests library to initiate requests with proxies. Let's take a real case, for example, we need to check IP availability in bulk:

import requests

proxies = {
    'http': f "http://{config['proxy']['auth']}@{config['proxy']['host']}:{config['proxy']['port']}",
    'https': f "http://{config['proxy']['auth']}@{config['proxy']['host']}:{config['proxy']['port']}"
}

try.
    resp = requests.get('https://httpbin.org/ip', proxies=proxies, timeout=10)
    print(f "Currently using IP: {resp.json()['origin']}")
except Exception as e.
    print(f "Connection Exception: {str(e)}")

Here's a pitfall to watch out for:The proxy format of the reqests library must be with a protocol header, such as http://或https://开头. Some brothers directly fill in the IP address, and the result is that they can't connect.

Exception Handling

In practice, these moths are often encountered:

Specialized tools for heart-saving solutions

Maintaining a proxy pool on your own is too much work, so we recommend using theipipgoof ready-made services. There are three packages available in their home:

• Dynamic residential (standard): suitable for individual developers, more than 7 dollars 1G traffic
• Dynamic Residential (Business): more cost effective for team use, $9+ 1G
• Static Residential: dedicated to fixed IP scenarios, $35 per month

It's super easy to get an IP with their API, and the code samples are all there for you:

import requests

 API address from ipipgo backend
api_url = "https://api.ipipgo.com/getip"

params = {
    'type': 'dynamic', dynamic home
    'count': 10, Get 10 IPs
    'format': 'json'
}

resp = requests.get(api_url, params=params)
ip_list = resp.json()['data']
print(f "Latest IP pool: {ip_list}")

Frequently Asked Questions QA

Q: Why is it still blocked after using a proxy?
A: It is recommended to switch IP types, static residential IPs survive longer. ipipgo's TK line is particularly suitable for scenarios requiring high anonymity

Q: What should I do if the JSON file is loaded with an encoding error?
A: Add encoding parameter to the open function, such asencoding='gbk'maybeencoding='utf-8(Note that the quotation marks are intentionally missing here, so don't miss the actual code)

Q: How do I choose a package for my enterprise level project?
A: Directly look for ipipgo customer service to ask for 1v1 customized program, they can do step quote according to the business volume, more cost-effective than the standard packages

One last rant about this proxy IP thing.Don't use free resources on the cheapThe first thing you need to do is to get your account blocked. I've had a brother who was trying to save a lot of trouble, and his account was blocked, not to mention that it delayed the progress of the project. The regular service providers like ipipgo have trial activities, first experience before ordering more secure.