
Proxy IP in the Java Web development of the wonderful use of scenarios
Partners who have engaged in network requests know that directly accessing external interfaces with their own server IPs in certain scenarios is like attending a formal meeting in slippers - both inappropriate and easy to reveal. For example, when collecting public data in bulk, the target website may intercept the request based on IP characteristics. At this time, relaying the request through the proxy IP is like putting a wig on the server, which can protect the real identity and avoid being targeted by the wind control.
Tomcat as an old Java Web container, itself does not have a built-in proxy function, but we can modify the JVM parameters or code configuration to achieve. Below are two practical programs to break down , remember to choose according to the project needs flexibility .
Option 1: JVM global proxy model
For cases where the entire application needs to go proxy, add these lines to the catalina startup script:
Java_OPTS="-Dhttp.proxyHost=123.45.67.89
-Dhttp.proxyPort=54321
-Dhttps.proxyHost=123.45.67.89
-Dhttps.proxyPort=54321"
Pay attention here.IP address and portTo replace it with the one provided by the actual proxy service provider, for example, with ipipgo's dynamic residential proxy, their ports are usually either 8080 or 8888. This way is simple and crude, but has a drawback - all outgoing requests go through the proxy, which may affect local debugging.
Option 2: Precise control of agency scope
You can use Apache HttpClient custom proxies if you want to fine-tune your management:
HttpHost proxy = new HttpHost("121.4.147.xxx", 8080);
RequestConfig config = RequestConfig.custom()
.setProxy(proxy)
.setProxy(proxy) .build();
CloseableHttpClient client = HttpClients.custom()
.setDefaultRequestConfig(config)
.setDefaultRequestConfig(config) .build();
This writing style is particularly suitable forhybrid scene——部分接口需要代理,部分。比如调用第三方支付接口走,采集公开数据走代理。用ipipgo的静态住宅IP时,建议每个线程绑定固定IP,避免频繁切换导致验证码弹窗。
Guide to Avoiding the Pit: Three Minefields of Proxy Configuration
| problematic phenomenon | check the direction of the investigation |
|---|---|
| Connection timed out | 检查防火代理设置,测试telnet代理端口 |
| 407 Proxy Authentication Required | Confirm that account password authentication is enabled |
| Slow and fast response time | Toggle proxy protocol type (HTTP/HTTPS) |
Recommendations for ipipgo selection
After testing multiple vendors, it was found thatDynamic Residential IPThe best price/performance ratio in data collection scenarios. Take ipipgo, their enterprise-level dynamic package is $9.47/GB and supports automatic IP pool switching. For crawling tasks that require a fixed IP, the $35/month static package is more appropriate.
Here's one.Hidden Tips: When getting a list of proxies through their API, remember to set theregion=target参数指定目标地区。比如采集美国网站就锁定美西节点,能降低40%左右。
Frequently Asked Questions
Q: The application can't be launched after the proxy is set?
A: Check that the JVM parameters are formatted correctly, especially if they contain special characters wrapped in quotation marks
Q: HTTPS request report certificate error?
A: Add SSLContext trust all certificates in the code (recommended for test environments only)
Q: How can I verify if the agent is in effect?
A: Add a test interface in the Controller and return therequest.getRemoteAddr()View actual egress IPs
As a final reminder, don't just look at price when choosing a proxy service provider. A provider like ipipgo supportsTK Line的,在处理东南亚业务时能控制在200ms以内,比普通线路快三倍不止。具体怎么选,还是得看业务场景——就像吃火锅选蘸料,麻酱香油各有所爱嘛。

