Node.js Request Library: Axios Proxy Configuration

Hands-on teaching you to use Axios to hang proxies

Crawlers should understand that many sites are now added to the anti-climbing mechanism, directly with the local IP hard just minutes to be pulled black. This time we have to rely on proxy IP torotational identityRecently, a number of people asked me how to match the proxy in Axios in Node.js, today we take our own!ipipgoservice as an example to give the guys a whole lot of practical tutorials.

Pre-configuration three-piece suite

Make sure you have all three on your computer first:
1. Node.js environment (v14+ recommended)
2. Install the axios library (npm install axios)
3. ipipgo account (for new users)Free 200(Test IP)

Base Agent Configuration

Hooking up proxies in axios is actually massively simple, look at this code:

const axios = require('axios');
const proxy = {
  host: 'proxy.ipipgo.com', //proxy server address
  port: 8000, // port to see package type
  auth: {
    username: 'Your account', //s
    password: 'Password'
  }
};

axios.get('target url', { proxy })
  .then(response => console.log('Success!'))
  .catch(error => console.log('Rollover:', error));

take note ofDon't make a mistake with the port number.The ports of different packages of ipipgo are not the same, which can be found in the user's backend. It is recommended to write the proxy configuration as a separate module for easy maintenance later.

Dynamic Agent Pool Play

Single with a fixed IP is easy to be blocked, this time it is necessary to go on the ipipgodynamic agent pool. Use their API to get the available IP in real time:

You can switch dynamically in the code like this:

async function getProxy(){
  const {data} = await axios.get('https://api.ipipgo.com/getip');
  return `http://${data.ip}:${data.port}`;
}

// Update the proxy before each request
axios.interceptors.request.use(async config => {
  config.proxy = await getProxy();
  return config; }); }); // Update the proxy before each request.
});

Common Rollover Scene Handling

A few common pitfalls encountered by newbies:

1. Connection timeout

Check if the proxy IP is not working, use ipipgo'sSurvival Detection InterfaceVerify IP status first

2. Return 407 error

Wrong account password, special attentionEnterprise Edition usersTo whitelist IPs

3. Slow as a dog

Switching ipipgo'sBGP lineIt's more than three times faster than a regular line.

Practical QA session

Q: What should I do if my proxy IP is not working?
A: It is recommended to use ipipgo'sAutomatic package switchingIf you want to use this, configure the refresh interval in the backend, and add a fail-retry mechanism in the code.

Q: What if I need a high anonymous proxy?
A: ipipgo'sEnterprise PackageAnonymous authentication, no X-Forwarded-For field is exposed in the request header.

Q: How do I handle HTTPS requests at the same time?
A: Add in the proxy configurationprotocol: 'https'Note that the corresponding port should be changed to 443.

Performance Optimization Tips

1. UseConnection Pool ReuseReduced handshake time
2. Set a reasonable timeout (recommended 5s for connection timeout and 30s for response timeout)
3. In conjunction with ipipgo'sGeographic selectionfunction to select the node closest to the target server

Lastly, remember to add the ipipgo proxy to your code.User-Agentdisguise, the success rate can be improved by more than 60%. Their house is recently doing activities, report the secret code [AXIOS666] can be collected3-Day Enterprise TrialIf you need it, hurry up.