Curl POST Form: Curl Form Submission

Teach you how to use curl to play with form submission.

When you are doing data collection or automation, you often need to simulate the browser to submit the form. Today we will talk about how to use curl as a command line tool to deal with POST form submission, focusing on how to match the proxy IP to protect the real IP address.

Curl form submission basic posture

Let's look at the simplest example first. Suppose you want to submit a login form, containing a username and password:

curl -X POST
-d "username=myaccount&password=123456"
https://example.com/login

here are-X POSTSpecify the request method.-dThe back with the data to be submitted. But so naked can not, the site minutes to block your IP, this time the proxy IP on the field.

The right way to open a proxy IP

It's really easy to hook up a proxy to curl, add a-proxyParameters are fine. For example, use ipipgo's HTTP proxy:

curl -x http://user:pass@proxy.ipipgo.io:8080
-X POST
-d "username=test&password=abc123"
https://example.com/login

Here take care to replace it with your real account password obtained from ipipgo. ipipgo's proxy nodeSupports high concurrency(math.) genusAutomatic IP switching, which is particularly suitable for scenarios that require a large number of commits.

Pitfalls and countermeasures in practice

These problems are often encountered in actual use:

Essential Tips for Advanced Players

1. File uploads are so screwed up:

curl -x http://proxy.ipipgo.io:8080
-F "file=@/path/to/file.jpg"
-F "desc=description of file"
https://example.com/upload

2. Cookies are used to maintain the session:

curl -x http://proxy.ipipgo.io:8080
-c cookies.txt
-b cookies.txt
-d "data=value"
https://example.com/form

Frequently Asked Questions

Q: What should I do if the proxy keeps failing to connect?
A: First check whether the proxy address format is correct, recommended to use the ipipgo provideGenerate proxy address with one clickfunction to avoid hand output errors.

Q: What should I do if I submit garbled Chinese data?
A: In the request header add-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8"

Q: How do you handle the need for a lot of concurrency?
A: Use xargs with ipipgo'sMulti-line Polling Function, for example:

cat urls.txt | xargs -P 10 -I {} curl -x http://proxy.ipipgo.io:8080 {}

Why choose ipipgo agent

1. 独享高速通道，比公共代理低80%
2. Automatic CAPTCHA recognition function, save your energy and effort
3. Provision of completeRequest header camouflage programPerfect simulation of real-life operations
4. 7 × 24 hours technical support, response speed bar

Finally, remind the guys to do automated submissions to comply with the rules of the site. Reasonable use of proxy IP, both to protect privacy and improve the success rate, ipipgo new users now sign up for free!10G Traffic PackYou've got to try it out!