ParseHub Text Validation: ParseHub Proxy Validation Scheme

ParseHub can't handle text validation? Proxy ip is the way to go!

Recently a lot of brothers do data capture with me to complain, ParseHub that text verification is more and more difficult to get. It's not easy to pop up the CAPTCHA, or it's just a direct IP blocking! Today we will break open the crumpled say, how to use proxy ip to cure this problem.

Authentication mechanism disassembled

ParseHub's verification system stares at three main metrics:Request frequency,IP Track,device fingerprint. Especially that IP track detection, as long as you find the same IP continuously sending requests, immediately give you on the verification code. What we have to do is to use proxy ip to hide the real IP.

real-world value-added scheme

Here we recommend the use of ipipgo's dynamic residential agent, their IP pool is updated quickly, measured verification breakthrough rate can reach 92%. the key is to match these parameters:

 Python Example
import requests

proxies = {
    'http': 'http://user:pass@gateway.ipipgo.net:9021',
    'https': 'http://user:pass@gateway.ipipgo.net:9021'
}

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
}

response = requests.get('https://www.parsehub.com', proxies=proxies, headers=headers, timeout=15)

Be careful to set thestochastic delay, which is recommended to fluctuate between 3-8 seconds. Don't try to go fast, ParseHub is particularly sensitive to sudden speed changes.

Common Pothole Detection

1. What should I do if I encounter 403? First check whether the proxy IP has been blacked out, it is recommended to change ipipgo's exclusive IP package.
2. Captcha appears repeatedly? Maybe the device fingerprint is exposed, remember to pair it with a browser automation tool!
3. Connection timeout problem? Adjust the timeout parameter to 20 seconds or so, some areas are really slow.

QA First Aid Kit

Q: Is it okay to use a free proxy?
A: Never! 9 out of 10 free proxies have been flagged, and it takes ipipgo's fresh IP pool to carry the verification

Q: How many IPs do I need to allocate to make it enough?
A: small and medium-sized projects recommended 50-100 IP rotation per day, large projects directly on ipipgo's automatic rotation packages

Q: What should I do if I am blocked?
A: Immediately deactivate the current IP segment, contact ipipgo customer service for a new IP pool, they have a blocked payout policy!

advanced skill

For complete invisibility, remember to pair these three pieces:
1. Proxy IP quality (emphasis! Recommend ipipgo's high stash of residential proxies)
2. Random generation of request headers
3. Mouse track simulation
Tested these three axes down, ParseHub's verification system is basically just a setup.

Lastly, don't gouge on the cost of proxy IPs. I have used seven or eight service providers, ipipgo IP survival time is really long, the average can use more than 12 hours. Those who use two or three hours to hang the proxy, purely for their own trouble.