防止漏扫爬虫的IP段配置：防火代理屏蔽规则设置方法

Leak-scanning crawlers' routines and response logic

Brothers engaged in website operation and maintenance should have encountered this situation: server monitoring suddenly alarmed, a look at the log is full of some messy IP scanning directory, try password. These crawlers are just like the small advertisers in the neighborhood, and they will dislike your server when they catch an opportunity. The traditional approach is to block a single IP, but people change their armor and come back, it is impossible to prevent it.

This is the time to useBatch blocking of high-risk IP segmentsThe tough tricks. As if the neighborhood property directly prohibit all the flyers into the gate, rather than catching a rush one. In practice, we should pay attention to three points: 1. Identification of high-risk IP attribution 2. Determination of proxy IP characteristics 3. Dynamic update of the blocking list.

手把手配置防火代理规则

Let's take Linux as an example, and let's use iptables to practice. First of all, prepare a list of high-risk IP segments, and organize those ASN numbers that often cause trouble. For example, the IP segments of data centers in some Eastern European countries, you can check the whois information.

 创建自定义防火代理链
iptables -N BLOCK_SCANNERS
iptables -A INPUT -j BLOCK_SCANNERS

 批量添加屏蔽规则（示例IP段需替换实际数据）
iptables -A BLOCK_SCANNERS -s 192.168.34.0/24 -j DROP
iptables -A BLOCK_SCANNERS -s 10.88.152.0/22 -j DROP 

 保存规则避免重启失效
iptables-save > /etc/sysconfig/iptables

Here's the point! Remember to add a timed task auto-update rule:

 Update IP blacklist every morning
0 3 /usr/local/scripts/update_firewall_rules.sh

Bi-directional protection strategies for proxy IPs

It's not enough to block, you have to learn to counteract it with proxy IPs. Here to boastipipgo's dynamic residential packages, the $7.67/GB offer is a real bargain. The exact play is split in two ways:

The real test is to use their API to extract proxy IPs and build a protective layer in 5 minutes:

import requests

def get_proxy(): api_url =
    api_url = "https://api.ipipgo.com/get?format=json"
    res = requests.get(api_url).json()
    return f"{res['protocol']}://{res['ip']}:{res['port']}"

Guidelines on demining of common problems

Q: Will blocking IP segments hurt normal users?
A: focusing on sealing the data center IP segment, ordinary users basically use home broadband. ipipgo's static residential package is specifically for the need for fixed IP business, 35 a month is not expensive.

Q: Will a proxy IP slow down the website?
A：选对协议类型很重要。HTTP业务用SOCKS5协议，视频类大流量走TK专线，实测能控制在200ms内。

Q: How do I get enterprise-level protection?
A: directly find ipipgo's technical guy to customize the program, they that enterprise package support by volume billing, sudden traffic is not afraid of being ripped off.

Tell the truth.

防火代理规则不是一劳永逸的，得配合IP情报做动态调整。最近发现有些爬虫会用云函数当跳板，这时候就得靠代理池的IP质量了。用过五六家服务商，ipipgo的存活率确实能到90%以上，关键是技术支持响应快，上次凌晨三点提工单居然还有人回…

Lastly, I would like to remind newbies not to just block IPs, but to remember to put alimit_req_zoneDo the request frequency limit, two-pronged approach is stable. When you encounter problems that can't be solved, go directly to their official website to find online customer service, report my name...forget about it and no discount, their prices are already transparent.