Teach you how to use Curl to hang a proxy!
Brothers engaged in network development should be familiar with Curl this command line tool, but many people do not know that it can also be used to hang proxy. Today, we will nag how to configure the proxy in Curl, especially with theipipgoof agency services to straighten out the job.
Why do I need a proxy for Curl?
For example, when you write a crawler script, your IP is often blocked, right? This time to hang a proxy IP will be able to change the armor to continue to work. Another example is that some websites restrict access to certain regions, so you can use a proxy IP to access them.ipipgoof the local agent nodes can be easily solved.
curl -x http://username:password@proxy.ipipgo.com:8000 https://target-site.com
Three positions to configure the agent
Position 1: Command Line Straight Plug Parameters
The least expensive way to do this is to just add the-x
parameter, note that theDon't misspell the password.::
curl -x http://账号:密码@proxy.ipipgo.com:8000 https://你要访问的网址
Position 2: Profile Daifa
Those who use proxies a lot can get a configuration file (~/.curlrc) and add the following line to it:
proxy = http://proxy.ipipgo.com:8000
proxy-user = "account:password"
Position 3: Temporary environment variables
Temporarily set it in a command line window (it expires when you close the window):
export http_proxy=http://账号:密码@proxy.ipipgo.com:8000
export https_proxy=http://账号:密码@proxy.ipipgo.com:8000
Guide to avoiding the pit
error message (computing) | method settle an issue |
---|---|
407 Proxy Authentication Required | Check if the account password has special symbols, it is recommended to use URL encoding process |
Connection timed out | changeipipgoserver address of the current node may be congested |
SSL handshake failed | At the end of the command, add-k Parameter Skip Certificate Validation |
practical skill
1. Test that the agent is in effect:
curl -x http://代理地址 -I https://httpbin.org/ip
2. When high-frequency IP switching is required, use theipipgoAPI to dynamically fetch new agents:
curl --proxy $(get_new_ipipgo_proxy) https://target.com
Frequently Asked Questions QA
Q: What about slow agents?
A: Tryipipgoof an exclusive high-speed line, or switch to a node closer to your physical location
Q: How do I set the proxy timeout period?
A: Add in the command-m 30
It's a 30-second timeout to prevent jamming
Q: Is it possible to use more than one agent at the same time?
A: Curl itself is not supported, but it can be used with theipipgoThe load balancing feature automatically distributes
Finally, remind the guys, the choice of agency services have to look at the stability and after-sales service.ipipgoThey provide 7 × 24 hours technical support, encounter problems at any time to find customer service girl nagging, more reliable than those who run away from the small workshop. If you don't know what you're talking about, just drop it in the comments section and we'll talk about it together.