Selenium proxy IP + CSS selector: Selenium proxy CSS selector positioning

Selenium meets Proxy IP: The window is finally broken!

Brothers engaged in automation testing should understand that when using Selenium to simulate real user behavior, the most afraid of encountering IP blocked. If you don't have some proxy IP, the project will be yellow in a minute. Today, we will teach you how to proxy IP and CSS selector of these two magic tools into a piece of use.

Proxy IP's Eighteen Wonders

First of all, let's say a true story: last year there was an e-commerce price comparison team, did not hang the agent directly climb a platform data, the results of half an hour was blocked more than 200 accounts. Later changedDynamic Residential Proxy for ipipgo, in conjunction with the rotation strategy, hardened the data collection success rate from 231 TP3T to 981 TP3T.

The wonders of proxy IPs in automation scenarios:

• Fake real geolocation (e.g. catching regionally restricted items)
• Break through the single IP request frequency limit
• Prevent feature recognition (some sites take browser fingerprints)

CSS selectors: the embroidery needle for positioning elements

Why do old drivers love CSS selectors? For example, to find all the buttons on the page with the category "buy-now", you have to use XPath to write//[contains(@class,'buy-now')]and CSS directly.buy-nowIt's done. The difference is like the difference between using chopsticks to hold peanuts and scooping them with a spoon.

 Sample code
from selenium.webdriver import ChromeOptions

 Replace the proxy information with the one provided by ipipgo.
proxy = "123.123.123.123:8888"
options = ChromeOptions()
options.add_argument(f'--proxy-server={proxy}')

driver = webdriver.Chrome(options=options)
 Pinpoint the search box with CSS
search_box = driver.find_element_by_css_selector("inputkw.s_ipt")

Tips for working with gold partners

Name a few potholes that are easy to fall into:

1. When the proxy IP fails, remember to addRetesting mechanism(Recommended to use ipipgo's API to change IP automatically)
2. CSS positioning encounters dynamic class, try attribute selector[name^='dynamic_']
3. Verify that the proxy is in effect: visithttps://httpbin.org/ipLook at the return IP

QA Time: A Guide to Avoiding the Pit

Q: What should I do if my proxy IP suddenly fails?
A: ipipgo's recommendedDynamic Residential (Enterprise Edition)Package that comes with IP health check and automatically rejects failed nodes.

Q: CSS Positioning always fails to find an element?
A: 80% of the page is not yet loaded, add an explicit wait:
WebDriverWait(driver,10).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".list-item")))

the right tool saves effort and leads better results

Recently helped customers do cross-border e-commerce data collection with ipipgo'sStatic Residential AgentsWith CSS positioning, it grabbed 200,000 pieces of merchandise data in 3 days. Their TK line is really stable, and encountering CAPTCHA is half less than using ordinary proxies.

The last nagging sentence: do not gouge the cost on the proxy IP, a good proxy service can save at least 30% debugging time. Especially if you are doing a long-term project, directly on theipipgo's enterprise packageThat traffic package is cheaper than drinking milk tea when discounted.