Python Google Finance Interface: Stock Data API Solution

Why is it so hard to get stock data these days?

Recently, a number of quantitative trading friends and I complained that the use of Python directly to the Google Finance interface is always a problem. Either suddenly give you a429 Too Many RequestsThe data is not even visible. What's more, some areas of the network environment connecting port address are ping can not be, you say urgent people are not urgent people?

Older drivers are playing with proxy IPs like this

Have you seen the veterans of the reptile business? They've got one in their pocket.Proxy IP PoolThe server won't recognize you. For example, with ipipgo's rotating proxy, every time you request a new vest, the server can't even recognize who you are. This is like playing hide-and-seek, changing clothes every time, the security guards how to remember ah.

import requests
from itertools import cycle

 Proxy pool provided by ipipgo (example)
proxies = [
    "http://user:pass@gateway.ipipgo.com:30001",
    "http://user:pass@gateway.ipipgo.com:30002".
    "http://user:pass@gateway.ipipgo.com:30003"
]
proxy_pool = cycle(proxies)

def fetch_stock(symbol).
    current_proxy = next(proxy_pool)
    try.
        resp = requests.get(
            f "https://www.google.com/finance/quote/{symbol}",
            proxies={"http": current_proxy}, timeout=10
            timeout=10
        )
        return resp.text
    except Exception as e.
        print(f "Failed with {current_proxy}, move to the next one!")

How are the proxy parameters best tuned?

Don't think that just because you've hung up your agent, there's a lot more to it than that:

Focusing on ipipgo'sIntelligent RoutingFunction, it can automatically pick the fastest node. It's like a delivery boy who doesn't need you to direct him and knows which road is not blocked by traffic himself.

A practical guide to avoiding the pit

1. Encounter403 ForbiddenDon't panic, it's probably the request header. Remember to bring a real User-Agent, not Python's default one.

2. Data suddenly out of sync? Try adding a random hibernation to your code, learnHuman Operational Rhythm

3. ipipgo'sexclusive IP poolSuitable for high-frequency access scenarios, like chartered Internet cafes, not afraid of other people robbing the speed of the Internet

I'm sure you want to ask these.

Q: Is it okay to use a free proxy?
A: Brother, the free ones are the most expensive! Those public proxy pools were played out long ago, nine out of ten can't connect, and the remaining one is slower than a snail.

Q: Why do you recommend ipipgo?
A: His familyResidential IPThe degree of camouflage is so high that the actual test of 200 consecutive requests did not trigger the wind control. Unlike some server room IP, once used to be recognized.

Q: How is the frequency of data capture controlled?
A: It is recommended to check 1 stock every 30 seconds with ipipgo's 5 concurrent channels so that you can check 10 per minute, which is both efficient and safe.

Tips for getting on the road

One last trick to pass on: take ipipgo'sProxy ListStore it in Redis and randomly select it when you use it. Combined with an asynchronous request library, the speed can be more than tripled. But be careful not to write the API key and proxy configuration to death in the code, using environment variables is more secure.