Setting up proxy ip with selenium: automated browser IP binding tutorial

Hands-on with Selenium sets of proxy IPs

The old iron engaged in automation testing should have run into this situation: the site suddenly blocked your IP, the hard work of writing scripts directly scrapped. This time the proxy IP is a lifesaver. Let's use ipipgo's proxy service today to teach you how to give Selenium browser vest.

Don't be sloppy with your preparations.

Make sure you have all three on hand first:

1. Python environment (version 3.7+ recommended)
2. Selenium library (pip install selenium)
3. Reliable proxy IP (recommend ipipgo's dynamic residential package)

Focus on proxy IP selection. Dynamic residential IP looks cheap (more than 7 dollars 1GB), but the enterprise package, although more expensive (more than 9 dollars 1GB), much better stability. If you are doing a long-term project, it is recommended to go directly to the enterprise version, so as not to always have to toss.

Browser Vesting in Action

Here's a rant on two scenarios:

Scenario 1: Chrome Sleeve Proxy

from selenium import webdriver

proxy = "123.45.67.89:8080" Replace with your own ipipgo proxy.
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(f'--proxy-server=http://{proxy}')

driver = webdriver.Chrome(options=chrome_options)
driver.get("http://ipipgo.com/checkip") This page displays the current IP address.

Scenario 2: Firefox Proxy

profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", "123.45.67.89")
profile.set_preference("network.proxy.http_port", 8080)
profile.update_preferences()

driver = webdriver.Firefox(firefox_profile=profile)

Common Rollover Scene Handling

Q: What should I do if the proxy doesn't take effect after I set it up?
A: First check the IP format is correct, pay attention to the http and https protocols do not get confused. ipipgo's proxy supports dual protocols, but the port number to see clearly.

Q: How do I verify if the agent is in effect?
A: Visit the IP detection page of ipipgo official website, you can see the current export IP used. if the IP shown is not your proxy IP, hurry up to check the code.

Tips for using ipipgo

Their agents have a hidden feature--dynamic switchingThe following is a list of the most popular ways to prevent blocking. Add a loop in the code, each visit automatically change IP, anti-blocking effect. See this example:

import requests

 Get the API interface of the dynamic proxy
api_url = "https://ipipgo.com/api/getproxy"
proxies = {
    'http': requests.get(api_url).text,
    'https': requests.get(api_url).text
}

 Plug the fetched proxies to Selenium

Lastly, I would like to say a few words: the choice of a proxy service provider depends on the actual needs. Like doing data collection with dynamic packages, engage in account registration with static residential IP. ipipgo's enterprise version of the package to support the simultaneous use of 500 + IP, suitable for the studio team combat.