nodejs.org: https://nodejs.org/dist/ Blocking JFrog Artifactory User Agent: Artifactory
Version
18.15.0
Platform
Darwin adamb-mac 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64
Subsystem
No response
What steps will reproduce the bug?
curl -H "Host: nodejs.org" -H "User-Agent: Artifactory/" https://nodejs.org/dist --head
HTTP/2 403
date: Sun, 30 Jul 2023 11:17:09 GMT
content-type: text/html; charset=UTF-8
cache-control: max-age=15
expires: Sun, 30 Jul 2023 11:17:24 GMT
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
server: cloudflare
cf-ray: 7eed318d7de809c3-HFA
How often does it reproduce? Is there a required condition?
always
What is the expected behavior? Why is that the expected behavior?
Allow access to https://nodejs.org/dist/ from User Agent: Artifactory
What do you see instead?
HTTP/2 403
date: Thu, 03 Aug 2023 15:50:03 GMT
content-type: text/html; charset=UTF-8
cache-control: max-age=15
expires: Thu, 03 Aug 2023 15:50:18 GMT
x-frame-options: SAMEORIGIN
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
server: cloudflare
cf-ray: 7f0fb6cf8a11416a-LHR
Additional information
Hello, My name is Adam and I’m a Product Manager @JFrog. We recently discovered that our mutual customers are being blocked from accessing the following URL when requesting data from user agent “Artifactory” https://nodejs.org/dist/
We would like to collaborate together to understand why this restriction was enabled and see how we can resolve any issues from the Jfrog Platform source.
We are looking forward to working together Thanks in advance, Adam
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 29 (16 by maintainers)
Dear @Zvikac @jensborrmann @crazed
We have conducted a thorough investigation of this issue together with our friends at nodejs.org and confirmed there is No disruption of service around the main ability to download binaries from https://nodejs.org/
However, we do see 2 issues exhibited specifically by the experience in JFrog/Artifactory that behave differently than what you are probably expecting, Namely: “TEST connection” and “Remote Repository Browsing”. Remote repository browsing is not supported by all/most registries/remote targets.
I invite you to contact your ESL or me directly to help with these issues.
Additionally, We have identified an article https://sionwilliams.com/posts/2020-12-09-node-n-npm-mirror/ that is showing an outdated practice. In response, we have reached out to the maintainer to update this article and in parallel JFrog will provide an official Knowledge base article on how to set this up correctly.
If you have stumbled upon this thread, please refrain from using https://nodejs.org/dist in your Generic Remote Repository URLs, the correct URL for nodejs binaries is https://nodejs.org/ (WITHOUT /dist)
From our perspective, this Issue closed We deeply appreciate all the help from our friends at Nodejs @ovflowd, @targos - Thank you 🙏
There is a big misunderstanding here. https://nodejs.org/dist/ is not the same as npmjs.org. It is not the npm registry, nor is it a mirror for it. While we have seen some requests coming from misconfigured maven repositories, the vast majority of requests come from misconfigured npm registry.
Hey @adam-browning, I appreciate your effort here and the communication. 🙇
Dear @Zvikac & @jensborrmann I apologize for not responding sooner and I would like to address your issues. Please reach out to me directly via email to adamb@jfrog.com and let’s connect to see how we can help you.
I was previously in contact with both @ovflowd & @targos and they have been extremely patient and attentive in truly helping with this issue. Unfortunately, I have not been available for some time and I intend to remedy that now.
@ovflowd & @targos I truly appreciate all you have done here to assist us. I hope we will be able to continue our collaboration and address your concerns to eliminate the invalid requests that were impacting your servers.
Thanks in advance, Adam
@ovflowd O.K. I got your 2nd comment.
@ovflowd I’m not really trying to open https://nodejs.org/ but it’s a part of the trial-error troubleshooting I’m going through, I want to focus on the root cause. and if I get 403 on the main link than obviously I get it down the road.
Understood, moving to OpenJS
There is a related discussion in - https://github.com/nodejs/build/issues/3223.
If the complaint is that requests for non Node.js downloads are being blocked, then that issues covers why the project sees those requests as a problem. If it is blocking requests for Node.js downloads that might not be intentional but a side effect of trying to block the
spamrequests.Probably better asked over at nodejs/nodejs.org. Maybe an admin can move the issue.