First, what is the Curl POST request? First chatter to understand the basics
When you use Curl to send a POST request, it's like sending a courier to the server. Unlike a GET request, a POST request hides the data in a "package" and sends it quietly. For example, you want to use Curl to simulate login to the site, this time you have to pack the account password into the POST data.
curl -X POST -d "username=zhangsan&password=123456" http://example.com/login
here are-X POSTSpecify the request method.-dThe data to be transmitted will be picked up later. But there's a pitfall in playing it straight - your real IP will be memorized by the server in a small book, and it's time for the proxy IP to come on board.
Second, to Curl vest: the correct way to open the proxy IP
Want the server to not recognize you? Put a proxy vest on Curl and you're done. Highlights-xThis parameter, followed by the proxy server address:
curl -x http://ipipgo-proxy.com:8080 -X POST -d "data=test" http://target-site.com/api
Note that ipipgo's proxy server address is used here. One good thing about their proxies is that they supportDynamic IP PoolIt's much more stable than those fixed IP's, as it automatically changes IP's for each request.
Agent Type | dominance | Applicable Scenarios |
---|---|---|
Dynamic Residential Agents | Short IP survival time | High Frequency Data Acquisition |
Static Data Center | Fast connection speed | interface testing |
Third, the actual teaching: how to play with certified agents
Some proxy services want account password verification, like ipipgo's enterprise package. This time you have to use the-proxy-userParameters:
curl -x http://ipipgo-proxy.com:8888 --proxy-user vip_user:password123 -X POST -d @data.json http://api.example.com
Here the data is stored in the data.json file and submitted, which is safer than writing directly on the command line. ipipgo's proxy supportUsername+Password/IP WhitelistDual authentication, when engaged in batch operation especially save.
Fourth, avoid the pit guide: novice often step on the 3 minefields
1. timeout setting: Remember to add the -max-time parameter when the proxy network is unstable.
curl --max-time 30 -x http://ipipgo-proxy.com:8080 -X POST ...
2. HTTPS request: To check the validity of the certificate when going proxy, it is recommended to add -proxy-insecure
3. Chinese garbled code: Remember to add -H "Content-Type: application/x-www-form-urlencoded;charset=UTF-8" before the -d parameter.
V. QA time: what you might want to ask
Q: What should I do if I can't connect to the proxy IP all the time?
A: First, check the port of the proxy address is correct, ipipgo's residential proxy uses port 8080 by default. If it doesn't work, go to the background to refresh the IP pool.
Q: What should I do if I get stuck when POSTing a large file?
A: Instead of -d, use the -data-binary parameter, and ask the tech guy at ipipgo to open a dedicated proxy for you.
Q: How do I verify if the agent is in effect?
A: First without the proxy request http://httpbin.org/ip and then with the proxy request, compare the IP addresses returned twice.
VI. Why do you recommend ipipgo?
Having tested seven or eight proxy services, ipipgo has three major killers:
1. Dynamic IP Pool2 million + IP updates per day
2. Success Guarantee, failing to reach the 95% auto make-up time length
3. Dedicated customer serviceResponse time within 5 minutes
Now sign up to receive a 3-day trial, new users to send 10G traffic package. Do data collection of old iron, this wool not gripping white not gripping.
One last piece of cold knowledge: when using ipipgo's API to get a proxy, remember to add theX-Auth-Mode: streamingParameters, so that you can open the IP automatic rotation mode, anti-blocking effect directly pull full.