Collage Data Capture: B2B Contact Capture Solution

The rules of the game when it comes to Collab data capture

Brothers who do foreign trade know that there are big potential customers hidden on the Link. But manually pick data? That is really tired of individuals. This time we have to think of using technical means, but the anti-crawler mechanism of the Link is not a vegetarian ---Frequent operation of the same IP, a minute to give you blocked without negotiation!The

To give a real case: there is a mechanical export of the old man, he wrote a script to catch 200 pieces of data every day. As a result, on the third day, his account was restricted from logging in, and even his company's homepage was downgraded. Later found that he used his own office network, IP address has not changed.

Proxy IP is the key to breaking the mold

And here's where we come in with the killer--Dynamic Residential Proxy IP. Unlike server room IPs, these IPs come from the home networks of real users, and the stealth is pulled straight through. Tested with ipipgo's rotation strategy, no alerts were triggered for 8 hours of continuous collection.

Hands-on configuration

Here's one.It works.The configuration scheme of the

1. Choose the "Dynamic Residential" package in the ipipgo back office, and we recommend buying the Global Mixed Pool.
2. Set the frequency of automatic IP change (recommended 1 change per 50 requests)
3. Add the proxy authentication parameter to the crawler code, taking care to use theusername:passwordspecification

There's a pitfall to be warned about:Don't turn on the multi-threaded dash.! It is recommended to keep it to 1-2 requests per second, in conjunction with random clicks on page elements, to disguise it more like a real person.

Guidelines on demining of common problems

Q: Why is it still blocked after using a proxy?
A: Check two things: 1. IP purity (recommended business-class package with ipipgo) 2. whether the request frequency is too high

Q: What if there are duplicates in the collected data?
A: Add a de-duplication module in the code, use MD5 encrypted contact information to do the comparison, and then with ipipgo's IP geo-targeting function

Q: What should I do if I need to collect my company email address?
A: It can be combined with domain name guessing method, such as collecting thejohn.doe@company.comJust try it.johnd@company.comVarious combinations

These details make the difference.

1. The time zone has to be right.For example, if you want to catch American customers, you should use a US West IP and set the system time to the Pacific time zone at the same time.
2. Browser fingerprints to be randomized: Remember to change the User-Agent and screen resolution parameters every time you change your IP address.
3. Make good use of the follow function: Focus on the target user first, wait for the other party to return to the customs before collecting data, the success rate increased by more than 40%

Lastly, I'd like to introduce you to our own service:ipipgo's Collage Specialized PackagesThe company has been optimized especially for enterprise users. Not only provide API interface, but also according to the collection volume of intelligent adjustment of IP switching strategy, new users to send 5GB flow trial, enough to catch a small 1000 data.