Teach you to use Selenium to grab web pages, proxy IP so play to stabilize Recently, there are always brothers asked me to use Selenium to engage in data always be blocked IP how to do? Today we will nag about this matter. First of all, let's be honest, now the site is very fine, you take a local IP repeatedly brush, not seal you seal who? This time on ...

When the crawler meets the real estate: the pit of data collection Recently, I helped my friend to do the analysis of the price of second-hand housing, and I wrote a crawler script in Python. As a result, just two days after running, we found that the target website blocked our IP. At this time, I remembered to use a proxy IP, but the service providers on the market are either very expensive, or the IP pool...

Teach you how to use PowerShell to play with proxy servers Do you usually encounter slow loading web pages like a snail? Or you can't open some websites? This time we have to call out our network invisibility cloak - proxy servers. Today we do not whole those complex operations, with Windows comes with PowerShell, ...

The Hidden Trick of Playing Social Media: IP Splitting Old Zhang, an e-commerce company, recently had a headache when his 20 TikTok accounts were blocked in a row. It's not the content that's wrong, nor is it a violation of the law - the platform has directly labeled his device as a "machine number". This kind of bad thing happens every day, and...

Why is it always difficult to download stock data? Recently, Mr. Zhang wanted to analyze the historical data of US stocks, and found that it is harder than ever to download a csv file from many websites. The most important thing is that some of the platforms are blocking the domestic IP directly. At this time if you have a...

Peruvian agent for what has become the meat and potatoes of the South American e-commerce circle? In the last two years, many cross-border e-commerce iron found a strange phenomenon - obviously store decoration is quite beautiful, advertising costs also smashed a lot, but the actual turnover is not go. Later peeled open the data to see, found that the local shopping habits and we want to finish...

Don't be bypassed by the system proxy A lot of partners work with PowerShell, suddenly found that the network can not connect or the speed of the card into a dog. This time may need to configure the system-level proxy. However, Windows this goods it, the proxy settings hidden more than the ancestral secret recipe is deep, today we will break open the kneaded to say clearly ...

What's the hard part about Zillow data crawling? Anyone who has done real estate data crawling knows that Zillow's anti-climbing mechanism is stricter than property security. If you don't pay attention, your IP will be blocked, and the most pitiful thing is that sometimes even the CAPTCHA won't pop up, and it will directly give you a blank page. This site is mainly to prevent three kinds of operations: high-frequency access, IP...

Server strike?521 error what happened Recently there are old iron in the group complained, visit the site suddenly popped up a 521 error, anxious to scratch their heads. This thing is to say that you are looking for the web server is temporarily lying flat not work, common in server maintenance or sudden failure. This time the conventional operation is to wait ...