puppeteer: [Bug]: Unable to install Puppeteer on remote server, chrome-linux64 is blocked - Google-Edge-Cache: Forbidden 403
Steps to reproduce
I just updated puppeteer to version ^21.3.6 and it no longer installs. On my local development server, everything works correctly. However, on my production server which is a VPS hosted on OVH when I run npm install it fails to download this particular file:
https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip
Expected results
I expect the install to work correctly like it did before.
Actual results
Running npm i fails with this error:
1131 error path /home/user/apps/monorepo/node_modules/puppeteer
1132 error command failed
1133 error command sh -c node install.mjs
1134 error ERROR: Failed to set up Chrome r117.0.5938.92! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
1134 error Error: Download failed: server returned code 403. URL: https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip
1134 error at file:///home/user/apps/monorepo/node_modules/@puppeteer/browsers/lib/esm/httpUtil.js:74:31
1134 error at ClientRequest.requestCallback (file:///home/user/apps/monorepo/node_modules/@puppeteer/browsers/lib/esm/httpUtil.js:52:13)
1134 error at Object.onceWrapper (node:events:629:26)
1134 error at ClientRequest.emit (node:events:514:28)
1134 error at HTTPParser.parserOnIncomingClient (node:_http_client:693:27)
1134 error at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)
1134 error at TLSSocket.socketOnData (node:_http_client:535:22)
1134 error at TLSSocket.emit (node:events:514:28)
1134 error at addChunk (node:internal/streams/readable:343:12)
1134 error at readableAddChunk (node:internal/streams/readable:316:9)
1135 verbose exit 1
1136 timing npm Completed in 44921ms
1137 verbose unfinished npm timer reify 1696127353577
1138 verbose unfinished npm timer reify:build 1696127388958
1139 verbose unfinished npm timer build 1696127388977
1140 verbose unfinished npm timer build:deps 1696127388978
1141 verbose unfinished npm timer build:run:postinstall 1696127389197
1142 verbose unfinished npm timer build:run:postinstall:node_modules/puppeteer 1696127389484
1143 verbose code 1
I can’t even download this file manually with wget or curl. wget -d https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip
DEBUG output created by Wget 1.21.2 on linux-gnu.
Reading HSTS entries from /home/users/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'chrome-linux64.zip' (UTF-8) -> 'chrome-linux64.zip' (UTF-8)
--2023-10-01 02:32:10-- https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip
Resolving edgedl.me.gvt1.com (edgedl.me.gvt1.com)... 2600:1900:4110:86f::, 34.104.35.123
Caching edgedl.me.gvt1.com => 2600:1900:4110:86f:: 34.104.35.123
Connecting to edgedl.me.gvt1.com (edgedl.me.gvt1.com)|2600:1900:4110:86f::|:443... connected.
Created socket 3.
Releasing 0x0000562da5e82430 (new refcount 1).
Initiating SSL handshake.
Handshake successful; connected socket 3 to SSL handle 0x0000562da5e83af0
certificate:
subject: CN=edgedl.me.gvt1.com
issuer: CN=GTS CA 1D4,O=Google Trust Services LLC,C=US
X509 certificate successfully verified and matches host edgedl.me.gvt1.com
---request begin---
GET /edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip HTTP/1.1
Host: edgedl.me.gvt1.com
User-Agent: Wget/1.21.2
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 403 Forbidden
content-length: 37
content-type: text/plain
x-request-id: a1feba98-44f0-45d6-8e5d-8768bd302917
alt-svc: h3=":443"; ma=2592000, h3-29=":443"; ma=2592000
date: Sun, 01 Oct 2023 02:32:09 GMT
server: Google-Edge-Cache
---response end---
403 Forbidden
Registered socket 3 for persistent reuse.
Skipping 37 bytes of body: [Google-Edge-Cache: forbidden
Error: 7] done.
2023-10-01 02:32:10 ERROR 403: Forbidden.
Update:
Since there is no way to download the chrome-linux64.zip file from my remote server, I had to download it from my local machine, then rsync it to the remote server, and finally extract it’s contents to /home/YOUR_USERNAME/.cache/puppeteer/chrome/linux-117.0.5938.92.
Now everything appears to be working again.
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Reactions: 2
- Comments: 19
I have filed the issue for the CDN (internal) but I am not sure it’s the issue with the CDN or some of the cloud provider networks. So far the issue has been reported for AWS and OVH. In any case, it’s not actionable for Puppeteer. Please post a comment to https://github.com/GoogleChromeLabs/chrome-for-testing/issues/60 if it still affects you.
I got this working by forcing ipv4 like this
npm ci --dns-result-order=ipv4firstI am speaking from memory, but there are a couple of things you need to do. First and foremost, you need to extract the archive in a very specific location.
First, go to
/home/user/.cache/puppeteer/chrome, then create another directory that precisely matches the version of puppeteer you downloaded, it’s in the URL.Example: https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.92/linux64/chrome-linux64.zip
So that would be:
/home/user/.cache/puppeteer/chrome/linux-117.0.5938.92Place the
chrome-linuxarchive there and unzip it, so the final path to the executable becomes:/home/user/.cache/puppeteer/chrome/linux-117.0.5938.92/chrome-linux64/chrome < chrome here is the executable, not a directory.
Finally, you can set the
PUPPETEER_SKIP_DOWNLOADenv variable and run npm install:PUPPETEER_SKIP_DOWNLOAD=true npm install.It was either
PUPPETEER_SKIP_DOWNLOAD=trueorPUPPETEER_SKIP_DOWNLOAD=1, I forgot which one.That’s it.
Concourse CI build is failing
I’ve found out that the issue seems to be related to ipv4 vs ipv6. For example, this will fail:
but if you force ipv4 with the
--ipv4flag then it downloads successfully.Same story with wget.