Windows manually set up a proxy full strategy Many people think that setting up a proxy is particularly complex, in fact, follow the steps to be three or five minutes of work. Let's first find the settings icon that looks like a gear, click on it and don't be intimidated by the screen full of options, run directly to the "Network and Internet" on the right. There is a small...

When the crawler meets the anti-climbing how to do? Try this combination of punches You do data capture of the old iron must have encountered this situation: just write a good crawler script, running suddenly run by the target site blocked IP. At this time, do not be in a hurry to smash the keyboard, we want to talk about today's proxy IP + HTML parsing combo punch, specializing in ...

Random IP in the end what is the use? After reading these scenarios you will understand The network of friends should understand that the IP address is like your network ID card. There are times when you need to change a "vest" to do things, such as do data collection brother, with a fixed IP is easy to be blocked; do test brother to simulate different...

When the crawler meets the anti-blocking mechanism how to do? Do data collection of the old iron know that the target site's anti-climbing mechanism is like the summer mosquitoes - indefensible. Yesterday, the page can be accessed normally, today suddenly give you a pop-up CAPTCHA, or directly blocked IP. this time you need to give the program to wear a vest, and instead of...

What the hell is a web proxy? To put it bluntly, it is to give web access to a vest, as if you go to the market to buy food wearing a mask, the stall owner can not recognize who you are. You don't need to download software to use online web proxy, you can use it by opening a web page and inputting a URL, especially suitable for the scenario that you need to hide your real IP temporarily. Lift...

The old driver of the data are so play product ID capture Doing e-commerce friends must have encountered this scenario: want to analyze the competitor's data, but the direct climb of other people's websites in a minute to be blocked IP. this time we have to rely on the proxy IP to play guerrilla warfare, especially like ipipipgo this professional player, can let you in the collection of product ID when...

What the heck is headless Chrome? It's a browser robot that works on its own. Normally, we can see a lot of colorful pages in the browser, but this headless mode is like putting a stealth cap on the browser, so you can perform various operations in the background without opening a window. For example, you want to batch capture ...

First, SERP in the end is what the hell? The old iron to do network projects, it is estimated that have heard the word SERPs. To be clear, it is the search engine results page, such as you search in Baidu, "which milk tea is good to drink", popping up those web links is the SERP. but many people do not know, this thing...

Engage in Node.js crawl must know the proxy pit Recently to help a friend get a price comparison site, with Node.js to capture data when the old ban IP. this matter is really not a technical problem, the key in the proxy IP play. For example, a certain continuous capture of an e-commerce platform, less than half an hour was blocked, and then changed the ip ipgo move...