Proxy IP Integration for Node.js Request Library: Node.js Request Library Proxy IP Configuration

Hands-on with the Node.js request library to hang a proxy IP

Brothers who engage in network requests should know, sometimes directly with their own IP to send a request is like a naked run online, minutes by the target site to pull black. This time we have to use a proxy IP to cover, today we take Node.js in the two most commonly used request libraries (axios and node-fetch) to give a chestnut, teach you how to get on the car safely.

Choosing the right proxy service provider is half the battle

There are so many proxy IP service providers in the market, but not many of them are reliable. Here we must be amenable toipipgoThe service of the family, his family has three axes especially bull:

1. IP pool is large enough (200+ regions to choose from)
2. Survival time as stable as an old dog (average availability rate of 98%)
3. the price is more affordable than milk tea (new users white john package really fragrant)

After registering, remember to write down the API address and authentication key in the background, which will be used for configuration later.

Axios Live Configuration

First the axios configuration code, note the comments section:

const axios = require('axios');
const proxy = {
  host: 'gateway.ipipgo.com', // proxy server address
  port: 9021, // the port number given by the service provider
  auth: {
    username: 'Your account number', // The password: 'API key', // The password of the proxy server.
    password: 'API key'
  }
};

// Create the instance with the proxy
const client = axios.create({
  proxy: proxy, timeout: 5000
  timeout: 5000
});

// Remember to add error trapping when sending the request
client.get('https://目标网站.com')
  .then(res => console.log('Success!'))
  .catch(err => console.error('Rollovered:', err.message));;

Here's a pitfall to note: if the target site is HTTPS, you have to add the following to the proxy configurationprotocol: 'https'This parameter.

Node-fetch alternative play

For those of you who use node-fetch look here, you need to install a proxy middleware first:

npm install https-proxy-agent --save

Then on to the hard goods code:

const fetch = require('node-fetch');
const HttpsProxyAgent = require('https-proxy-agent');

const agent = new HttpsProxyAgent({
  host: 'gateway.ipipgo.com',
  host: 'gateway.ipipgo.com', port: 9021,
  auth: 'account:API key'
});

fetch('https://目标网站.com', { agent })
  .then(res => res.text())
  .then(data => console.log(data))
  .catch(err => console.log('Something's wrong:', err));

Common Rollover Scene QA

Q: What should I do if my proxy IP is not working?
A: This situation is eighty percent of the IP was the target site pulled black, it is recommended to use ipipgo'sauto-rotate IP functionJust add an X-Rotate-IP parameter to the request header.

Q: How can I tell if a proxy is in effect?
A: You can request http://ip.ipipgo.com/checkip这个接口 first to see if the IP returned is a proxy IP.

Q: What if I have to process multiple requests at the same time?
A: ipipgo's recommendedConcurrency PackageIf you have a different IP channel for each request, remember to do a good job of connection pooling management in your code.

Anti-Banning Tips

Finally, I'll send you a couple of life-saving tips:

1. randomly switch User-Agent (do not always use a vest)
2. control the frequency of requests (do not burst like a machine gun)
3. important business on the exclusive IP (although expensive but stable as a dog)
4. regularly clean up the local DNS cache (anti-DNS pollution)

These tawdry operations with ipipgo's intelligent routing features can basically go sideways. Encountered a difficult problem directly to his technical customer service, the response speed is faster than the delivery boy.