How to set cURL not to follow redirection? Scenario Case Analysis

Manual brake mode: cURL redirection control hands-on

Crawler friends should have encountered this situation: cURL request a certain URL, the results of the return is not the desired content. At this time, it is likely to be triggered by a web page redirection, like driving navigation suddenly give you a new route, but you may want to see the original road in the end through. This time you need to install amanual brake-- Disable auto-follow jumps.

Proxy IP test must off redirection

Let's take ipipgo's proxy IP test as an example. Let's say you want to verify that a proxy can access an e-commerce site properly, but the site has a feature that automatically redirects unlogged-in users to the login page. If you turn on auto-redirection, you can't tell whether the proxy itself fails to connect, or is jumped by the website policy.

This is the time to offer this order:

curl -x http://username:password@ipipgo-proxy-server:port -L 0 https://target-site.com

watch carefully-L 0This parameter combination, which is the key to turn off the automatic jump. ipipgo's proxy address remember to change to the actual assigned server address, with account password authentication is more secure.

Don't mix up the parameter comparison table

Special note: Some older tutorials will teach-location-trustedThis parameter is only used when you need to carry a cookie to jump, so don't add it to the normal proxy test, as it may cause security risks.

Practical guide to pothole draining

Last week a customer feedback, said with ipipgo's residential proxy test always return 301 status code. Later found that the target site is forced HTTPS jump, and his request command did not turn off redirection. Teach him to use this combo:

curl -x http://ipipgo动态代理 -I --connect-timeout 5 -m 8 -L 0 https://example.com

The -I parameter just needs the response header, -connect-timeout sets a 5-second connection timeout, and -m sets an 8-second maximum request time. This accurately determines whether the proxy isfirst jumpThe connection is successfully established.

Quick questions and answers to high-frequency questions

Q: Why does it still jump after setting -L 0?
A: 80% of the parameters are in the wrong order, cURL is sensitive to the order of parameters, make sure that the proxy setting parameters in the first, control parameters in the second

Q：How to determine whether the proxy is effective by returning 302?
A: Focus on the Location field in the response header to see if the domain name is the same as expected, with ipipgo's exclusive proxy can avoid shared IP contamination

Q: What if I need to handle multiple jumps?
A: instead of -max-redirs 3 this form, the number indicates the maximum number of jumps, with ipipgo's rotating agent pool can effectively avoid anti-climbing

The last nagging sentence: test the stability of the proxy remember to turn off the jump, just like the test brakes have to try on a straight road. ipipgo's proxy service comes with a redirection detection function, the background can directly see the real response chain of each request, than the manual debugging to save a lot of trouble. If you encounter any problems, you may want to try their technical customer service, the response speed is two steps faster than ordinary service providers.