Curl proxy environment variables: Curl proxy settings

Hands on teaching you to use Curl with proxies

Played the command line partners know curl this tool, but many people stuck in the proxy settings this step. Today, we will use the most practical way to talk about how to set a proxy for curl through environment variables. Remember, the proxies we are talking about here are legal and compliant application scenarios, such as testing interfaces, collecting public data, and other serious uses.

environment variable setting method

Global settings at the system level are the least troublesome and are recommended to usehttp_proxyrespond in singinghttps_proxyThese two variables. Windows users should note that the variable names should be in all upper case, Linux/macOS upper and lower case are fine.

 The Linux/macOS terminal says this
export http_proxy="http://user:pass@proxy.ipipgo.com:9021"
export https_proxy="http://user:pass@proxy.ipipgo.com:9021"

 The Windows command line does this
set HTTP_PROXY=http://user:pass@proxy.ipipgo.com:9021
set HTTPS_PROXY=http://user:pass@proxy.ipipgo.com:9021

这里有个坑要注意：ipipgo的代理地址记得换成你自己买的套餐对应的地址。账号密码里有特殊符号的话，得用%加上ASCII码代替，比如@符号要写成%40。

A great way to use agents on a temporary basis

If you don't want to change the system settings, it's more flexible to add parameters directly to the curl command:

curl -x "http://user:pass@proxy.ipipgo.com:9021" https://target-site.com

This -x parameter (or -proxy) is best suited for situations where you need to switch proxies frequently. For example, if you want to test the effect of different regional IPs, you can write a script to call different proxies in a loop.

How to hide authentication information safely

It's never safe to write your account password in plain sight, try these two methods:

1. Saving authentication information with a .netrc file
2. Go IP whitelisting authentication (this depends on the proxy provider support)

For example, ipipgo's enterprise packages support binding server IPs so that you don't even need to bring the account password, just write it that way:

curl -x "http://proxy.ipipgo.com:9021" https://api.example.com

Operational demining guide

Don't panic if the proxy doesn't work, follow this order to troubleshoot:

question-and-answer session

Q：Why does curl still not listen to me after setting environment variables?
A: 80% of the variable name is not written correctly, Windows must use all-caps HTTP_PROXY, Linux/macOS is case-sensitive.

Q: How can I verify that the agent is indeed in effect?
A: Use this command to measure connectivity first:
curl -x http://proxy.ipipgo.com:9021 http://cip.cc
See if the returned IP is a proxy IP

Q: Who do I listen to when there are both environment variables and command line arguments?
A: Command line parameters have the highest priority, and the -x parameter overrides the environment variable setting

Q: Which is the right proxy package to choose for ipipgo?
A: personal development choose the volume billing package, enterprise use is recommended to buy a fixed IP package. Their dynamic residential agent to do data collection is particularly stable, there is a need to go to the official website to see the difference between different packages.

One last thing: If you use proxies well, you can double your efficiency. But always remember to comply with the site's terms of service, do not engage in violent requests that set. If you encounter technical problems, you can directly poke ipipgo's 7×24 technical support, those engineers respond quickly.