node-pre-gyp: Unable to use proxy authentication

Thanks for the great work guys! I really appreciate that. Just wanted to report back that our CI is happy again as well: It’s correctly downloading a pre-built bcrypt binary via our corporate proxy. 👍

i just pushed a version making your changes - i think it makes perfect sense to let node-fetch handle the parsing. url.parse is deprecated and parsing the url should be the responsibility of the api in most cases. i just tested locally and all seems to work, so i pushed the changes to the branch. if you want to try again, feel free to, but as they are your changes i think everything will work just fine.

Maybe related: The switch from needle to node-fetch seems to have broken proxy support for us as well (without proxy authentication). After upgrading bcrypt from 5.0.0 to 5.0.1 our builds are now failing because Python is needed to build from source:

> bcrypt@5.0.1 install node_modules/bcrypt
node-pre-gyp install --fallback-to-build
node-pre-gyp ERR! install request to https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-x64-musl.tar.gz failed, reason: connect ECONNREFUSED 140.82.121.3:443 
node-pre-gyp WARN Pre-built binaries not installable for bcrypt@5.0.1 and node@14.16.0 (node-v83 ABI, musl) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error request to https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-x64-musl.tar.gz failed, reason: connect ECONNREFUSED 140.82.121.3:443 

In my experience, only the now deprecated request library ever got proxy support right. Every other HTTP library I tried either didn’t support proxies at all or failed for edge-cases (e.g. redirects not going through the proxies). 😞

@bmacnaughton - It works!

i think it makes perfect sense to let node-fetch handle the parsing. url.parse is deprecated and parsing the url should be the responsibility of the api in most cases.

I agree with you, because node-fetch constructor parses a request url at first, so we don’t need to parse the url in most cases.

Thank you for your rapidly and great supports!

@bmacnaughton - Sorry for the late reply. I got an following error.

node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.1
node-pre-gyp info using node@14.15.5 | win32 | x64
node-pre-gyp info check checked for "C:\Program Files\Jenkins\workspace\MY_POJECT\node_modules\bcrypt\lib\binding\napi-v3\bcrypt_lib.node" (not found)
node-pre-gyp http GET https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-win32-x64-unknown.tar.gz
node-pre-gyp http download proxy agent configured using: "http://id:pass@myproxy.com:8080/"
node-pre-gyp ERR! install request to https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-win32-x64-unknown.tar.gz failed, reason: getaddrinfo ENOTFOUND github.com 

I found out the part(L70) that seems to be the cause. https://github.com/mapbox/node-pre-gyp/blob/a0100abca39b53184a28fd7c93789e66eb136c1d/lib/install.js#L67-L72

I changed to following and it works for me, but I don’t understand for detail yet, because I haven’t checked node-fetch args at well. https://github.com/feuxfollets1013/node-pre-gyp/blob/13ea032e189b3efa224923e6623805694ad76a4f/lib/install.js#L67-L70

Sorry for the late reply! This looks good to me:

> https_proxy=http://proxy.company.net:3128 http_proxy=http://proxy.company.net:3128 ../.bin/node-pre-gyp install --update-binary
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@1.0.1
node-pre-gyp info using node@14.13.1 | darwin | x64
node-pre-gyp http GET https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-darwin-x64-unknown.tar.gz
node-pre-gyp http download proxy agent configured using: "http://proxy.company.net:3128"
node-pre-gyp info install unpacking napi-v3/bcrypt_lib.node
node-pre-gyp info extracted file count: 1
[bcrypt] Success: "/Users/.../Desktop/node-pre-gyp-bug/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node" is installed via remote
node-pre-gyp info ok

Thanks for the work, @bmacnaughton!

I made a few changes to your code and tried to run it. In my environment, Non-SSL URL w/ httpsAgent status is 200.

EDIT: The redirect is also followed correctly when utilizing https-proxy-agent and I’m receiving the bcrypt binary:

I was able to download it successfully too.

const fetch = require('node-fetch');
const HTTPProxyAgent = require('http-proxy-agent');
const HTTPSProxyAgent = require('https-proxy-agent');

const proxyUrl = 'http://id:pass@myproxy.com:8080/';
const httpAgent = new HTTPProxyAgent(proxyUrl);
const httpsAgent = new HTTPSProxyAgent(proxyUrl);

const fetchBcrypt = async() => {
  const res = await fetch('https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-x64-musl.tar.gz', { agent: httpsAgent });
  console.log(await res.text());
}

(async() => {
  console.log(`Non-SSL URL w/ httpAgent  : ${(await fetch('http://example.com', { agent: httpAgent })).status}`);
  console.log(`Non-SSL URL w/ httpsAgent : ${(await fetch('http://example.com', { agent: httpsAgent })).status}`);
  console.log(`SSL URL     w/ httpAgent  : ${(await fetch('https://example.com', { agent: httpAgent })).status}`);
  console.log(`SSL URL     w/ httpsAgent : ${(await fetch('https://example.com', { agent: httpsAgent })).status}`);
  await fetchBcrypt();
})().catch(console.error);

Results

Non-SSL URL w/ httpAgent  : 200
Non-SSL URL w/ httpsAgent : 200
SSL URL     w/ httpAgent  : 503
SSL URL     w/ httpsAgent : 200

@soulchild - i will take a look at this sometime this week. thanks for your detailed thoughts.

there was some risk to replacing request (now unsupported and deprecated) with the modern node-fetch, but continuing to use an unsupported and deprecated package wasn’t an alternative.