ISP Reseller Sales: Broadband IP Provider Whitelisting

Figuring out the relationship between ISP proxies and broadband whitelisting

Many friends who do data collection have stepped into such a pit: obviously using a proxy IP, the target site is still sealed fast. There is actually aBroadband provider whitelistingThe doorway of the market, 90% of the agency service providers do not tell you understand.

The protection systems of legitimate websites are now upgraded and will focus on theServer Room IP SegmentThe company has a lot of information about the company's business and its products and services. For example, an e-commerce platform, as long as the IP is detected from the Ali cloud / Tencent cloud these data centers, direct flow restriction is not negotiable. At this time it is necessary toISP Level Residential Agents, that is, the civilian IPs taken directly from these broadband operators, Telecom/Unicom.

 Comparison of requests between traditional proxies and ISP proxies
import requests

 Ordinary data center proxy (easily blocked)
proxy1 = {"http": "122.152.3.4:8080"}

 ISP proxy (true broadband IP provided by ipipgo)
proxy2 = {"http": "user-ipipgo:password@isp.ipipgo.com:30001"}

 Test accessing the e-commerce site
print(requests.get('https://某电商平台/product/123', proxies=proxy1).status_code) return 403
print(requests.get('https://某电商平台/product/123', proxies=proxy2).status_code) returns 200

The Three Difficulties of Whitelist Certification

To get real broadband IP resources, you have to break through three hurdles with the carriers:

E-commerce scenarios real-world cases

Last year, a client doing price comparison software came to us, and they were blocked more than 300 times a day using traditional proxies. They switched to ipipgo.Dynamic Broadband IPAfterward, three changes are particularly noticeable:

1. Success rate of requests from 47% to92%
2. Extension of the lifetime of each IP from 2 hours to72 hours +
3. Data collection costs cut directly in half

The key operation is openIP geobindingfunction, so that each request carries real city attributes. For example, to collect the price of goods in Beijing, the export IP of Beijing Unicom is fixed, which completely simulates the browsing of real people.

A guide to avoiding the pitfalls of the white man

On the market claiming to be an ISP agent, eight out of ten are fake. Teach you three tricks to verify the authenticity:

①Check IP attributes
The whois information for the true broadband IP is displayed as"CHINA UNICOM"This type of operator name, not XX Technologies

② Measurement of survival time
Continuously pinging an IP for more than24 hoursTrue Broadband IPs change dynamically (home broadband generally has 48 hour mandatory redial)

③ Look at the address database
Require service providers to provideIP neighborhoodLevel of address repository (like ipipgo can be accurate down to the street office)

Frequently Asked Questions QA

Q: Why do I have to be certified as a business to use ipipgo's proxy?
A: Carriers have mandated that enterprise broadband mustreal name registration (e.g. of a person on the street)This is an important sign to recognize a real ISP agent from a fake one, and it's really not available to individual users.

Q: How to deal with IP blocking?
A: Turn it on in the ipipgo backendIntelligent switching modeThe system will automatically isolate the problem IP based on the historical success rate, which is 20 times more efficient than manually changing the IP.

Q: How is the IP accurately distributed in different cities?
A: In the API request addcity_codeparameters (specific city code table in the official website can be downloaded), for example, to Shenzhen Telecom IP to pass 755_CT parameters.

Q: What should I do if I always encounter connection timeout during peak hours?
A: Change the request timeout setting toStep Retry StrategyLike this:

 Best Practice Code Examples
from requests.adapters import HTTPAdapter
session = requests.Session()
session.mount('http://', HTTPAdapter(max_retries=3))
session.mount('https://', HTTPAdapter(max_retries=3))

 Set step timeout (in seconds)
timeout_config = (3, 7, 15) first 3 seconds, second 7 seconds, third 15 seconds