Search Engine Results Tracking: How to Configure Search Results Tracking

Hands-On Search Data Monitoring

Brothers engaged in data analysis know that the search engine results rankings like the weather forecast - said change on change. Today, we teach the guys to use proxy ip to build a 24-hour monitoring system to ensure that the data to catch accurate and stable.

Why do I have to use a proxy ip?

For example, if you use your own broadband to check keyword rankings in Beijing, you will see the local results. But if you want to monitor the real rankings in different parts of the country, you have to use the ip of the local operator.Dynamic Residential AgentsIt comes in handy to simulate the network environment of real users.

This is a must.ipipgo's dynamic residential packagesThe traffic of more than 7 dollars 1G is enough and cost-effective. Their ip pool covers more than 380 cities across the country, so you can find the corresponding ip for whichever city you want to check the ranking.

Don't complicate the configuration tool

It is enough to prepare these things:

1. Python environment (version 3.6 or above)
2. requests library (installation command: pip install requests)
3. ipipgo API key (you can get it by registering on the official website)

Beginners are recommended to use their client, directly select the region - point to connect - open the monitoring of three consecutive operations, white users 5 minutes to get started.

Sample code

Use ipipgo's API to extract proxy ip, remember to replace it with your account password:

import requests

def get_proxy(): api_url =
    api_url = "https://api.ipipgo.com/getproxy"
    params = {
        "key": "Your API key",
        "protocol": "socks5",
        "location": "Shanghai"
    }
    response = requests.get(api_url, params=params).json()
    return f'socks5://{response["ip"]}:{response["port"]}'

proxy = get_proxy()
print(f "Currently using proxy: {proxy}")

There's something to be said for monitoring frequency.

Don't check every minute like a douchebag, it costs traffic and is easy to get blocked. Different industries suggest this:

Guidelines on demining of common problems

Q: What should I do if my proxy ip is slow?
A: Priority ElectionExclusive Static PackageOr contact customer service to open a TK line, the delay can be suppressed to less than 200ms.

Q: What's wrong with the configuration always failing?
A: First check the protocol type is right, http request don't use socks5 proxy. Then try theirClient Direct ModeIt's all basically solved.

Q: What happens when the data suddenly stops flowing?
A: dynamic ip default 15 minutes automatically switch, need to continue to monitor remember to add in the API parameters "lifetime=3600″ to extend the validity period

Tips for saving traffic

1. Set the request header with the compression parameter, can save 30% traffic
2. Use theirTraffic pack prepaid modelIt's half the cost of per-visit billing.
3. The frequency of monitoring in the middle of the night can be lowered, after all, no one changes the ranking in the middle of the night.

Finally, I'd like to tell you an industry secret: Static residential ip monitoring, data stability is more than 3 times higher than dynamic. Although 35 bucks an ip look expensive, but do long-term projects brother spread down actually more cost-effective. Recently, their anniversary celebration, new users can also play 20% off the first single, to get on the car tight.