Proxy IP to send cURL JSON POST: cURL proxy IP to send JSON data

Teach you to send JSON data with proxy IP by hand

Web developers should have encountered this situation: cURL send POST request is always blocked by the target site IP. this time we have to call out our savior - proxy IP. this time to ipipgo home services as an example, teach you how to use proxy IP to send JSON data safely.

Why do I have to use a proxy IP?

To give a real example, Zhang San does e-commerce data collection, 20 consecutive requests will be blocked IP. this time in the cURL set a proxy IP, equivalent to theWearing a mask to send a courier.The other site sees a different "courier" (proxy IP) in the delivery, naturally not easy to be recognized.

Here's a shout out to ipipgo'sDynamic IP Pool ServiceThe IP is automatically changed for each request, which is much more worrying than using a fixed proxy. The actual test with his family service to send 500 consecutive requests, the blocking rate can be reduced to 3% below.

Core configuration of cURL sending JSON

Remember these three key parameters:

-x or --proxy Specify the proxy address
-H Specify JSON format
-d Carries the contents of the data

Give a complete example with proxies:

curl -x http://user:pass@proxy.ipipgo.com:8080
-H "Content-Type: application/json"
-d '{"product": "cellphone", "price":3999}'
https://api.example.com/order

Here are the highlightsproxy format: Username, password, and proxy server address. If you use ipipgo, you can log in to the background to see your own exclusive proxy information, directly copy it over and use it.

A guide to avoiding the pit (real-world experience)

A real problem I recently encountered while helping a customer debug:

Advanced Tips: Automating IP Rotation

Use shell script combined with ipipgo's API to realize automatic IP change:

! /bin/bash
for i in {1..100}
do
   proxy=$(curl -s https://api.ipipgo.com/getproxy)
   curl -x $proxy -H "Content-Type: application/json" -d '{"page":'$i'}' https://example.com
   sleep 5
done

This script pulls new agents from ipipgo before each request, suitable for data collection tasks that need to run for a long time. Note the appropriate addition of SLEEP to avoid triggering frequency limitations.

QA time

Q: What should I do if my proxy IP is slow?
A: Preferred ipipgo'sBoutique RouteThe delay can be controlled within 200ms, although it is a bit more expensive. Also check if you are using HTTPS proxy, sometimes it will be faster if you switch to HTTP protocol.

Q: How do I test if the proxy is working?
A: First check the exit IP with this command:
curl -x proxy address https://api.ipipgo.com/checkip
If the returned IP is a proxy IP, the configuration is successful.

Q: What should I look for in an enterprise application?
A: It is recommended to buy ipipgo'sexclusive IP poolservice, to avoid sharing IP with other users, which may lead to banning. Concurrent requests are controlled within 50 requests/second, which is more stable with the retry mechanism.

One last thing: don't just look at the price of a proxy service, like ipipgo with a24-hour technical responseI'm not sure I can rely on them. Last week at two o'clock in the middle of the night, their engineers also helped me deal with the certificate problem, this kind of service dare to rest assured to use in the production environment.