Proxy IP Configuration cURL Proxy: cURL Command Line Proxy IP Settings

Teach you how to use cURL to hang proxy IP handily

Crawler brothers should understand, sometimes directly with their own network request data will trigger the anti-climbing mechanism. At this time you need a proxy IP to be a "stand-in", today focus on how to cURL this command line tool to hang proxy, especially with theipipgoof agency services.

Why do I need a proxy for cURL?

For example, if you continuously use the same IP to brush a website, the webmaster is not a fool, and will give you the IP to close the black room in a minute. Proxy IP is like playing "face", each request for a different identity, the success rate can be doubled. EspeciallyipipgoThis service that offers a massive IP pool can get thousands of IP addresses from different regions.

Three positions to set up the proxy

Here to teach you three common methods, remember to change the account password in the example to your own in theipipgo backstageGet the real credentials:

 Direct command line proxy hookup (good for temporary use)
curl -x http://user:pass@proxy.ipipgo.cn:9023 https://目标网站

 Setting environment variables (good for long term use)
export http_proxy=http://user:pass@proxy.ipipgo.cn:9023
export https_proxy=http://user:pass@proxy.ipipgo.cn:9023

 Write configuration file (suitable for older drivers)
vim ~/.curlrc
Add this line:
proxy = http://user:pass@proxy.ipipgo.cn:9023

Don't get the important parameters wrong.

Note the pitfall of looking at proxy types:

 HTTP proxy writing
curl -x http://代理地址 Destination URL

 SOCKS5 Proxy Write
curl --socks5 Proxy Address Destination URL

If you useipipgoThe service, their home supports both HTTP and SOCKS5 protocols, you can see the specific protocol type in the backend.

Detecting whether the agent works or not

After hanging the proxy, verify it with this command:

curl https://api.ip.ipipgo.cn/current-ip

If the returned IP is not the same as the local machine, it means the proxy is set successfully. This detection interface isipipgoSpecialized offers are more reliable than using third-party sites.

Common Rollover Scene QA

Q: What should I do if the proxy doesn't work after I set it up?
A: First check three elements: protocol type (http/socks5), port number, and account password. UseipipgoYou can ask their customer service for the test connection command

Q: How do I know if the proxy IP works?
A: Use this combo:
1. 先用ping命令测
2. Then use curl -I to measure the response header.
3. Final actual request data

Q: Do you support HTTPS sites?
A: As long as the proxy server supports tunneling mode (like theipipgo(which is supported by the full line of products), https sites can be accessed normally, the same way as normal http settings.

Hidden Tips

Sometimes it's necessary.Randomized switching of agents, you can write a shell script:

! /bin/bash
PROXY_LIST=(
  "http://user:pass@proxy1.ipipgo.cn:9023"
  "http://user:pass@proxy2.ipipgo.cn:9023"
  "http://user:pass@proxy3.ipipgo.cn:9023"
)
RANDOM_PROXY=${PROXY_LIST[$RANDOM % ${PROXY_LIST[@]}]}
curl -x $RANDOM_PROXY target site

This script works in conjunction with theipipgoThe multiple proxy channels can realize automatic IP rotation, and the anti-crawler effect is great.

The last point: the quality of the proxy IP directly determines the success rate. Like some free proxies often fail to connect, it is still recommended to use theipipgoThis kind of professional service providers, their home proxy channel with automatic verification and retry mechanism, save a lot of worry.