puppeteer: Navigation failed because browser has disconnected error on AWS Lambda
I’m using puppeteer-core on AWS Lambda with chromium builds taken here.
I create a new browser instance per lambda invocation, like this:
const browser = await puppeteer.launch({
args: [
'--disable-gpu',
'--disable-dev-shm-usage',
'--disable-setuid-sandbox',
'--no-first-run',
'--no-sandbox',
'--no-zygote',
'--single-process',
],
executablePath: './bin/headless-chromium',
userDataDir: '/tmp/puppeteer_data'
});
Everything works fine except when I have many, many invocations of this lambda. It’s hard to tell how many but I’d say ~ 500 invocations near the same time.
error: Error: Navigation failed because browser has disconnected!
at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (/var/task/node_modules/puppeteer-core/lib/FrameManager.js:1181:107)
at emitNone (events.js:106:13)
at CDPSession.emit (events.js:208:7)
at CDPSession._onClosed (/var/task/node_modules/puppeteer-core/lib/Connection.js:231:10)
at Connection._onClose (/var/task/node_modules/puppeteer-core/lib/Connection.js:121:15)
at WebSocketTransport._ws.addEventListener.event (/var/task/node_modules/puppeteer-core/lib/WebSocketTransport.js:45:22)
at WebSocket.onClose (/var/task/node_modules/puppeteer-core/node_modules/ws/lib/event-target.js:124:16)
at emitTwo (events.js:126:13)
at WebSocket.emit (events.js:214:7)
at WebSocket.emitClose (/var/task/node_modules/puppeteer-core/node_modules/ws/lib/websocket.js:180:10)
-- ASYNC --
at Frame.<anonymous> (/var/task/node_modules/puppeteer-core/lib/helper.js:144:27)
at Page.goto (/var/task/node_modules/puppeteer-core/lib/Page.js:624:49)
at Page.<anonymous> (/var/task/node_modules/puppeteer-core/lib/helper.js:145:23)
at getHTML (/var/task/src/details/details.js:100:16)
at <anonymous>
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
This error keeps going on for each event that triggers the lambda, until the lambda container is gone.
- Puppeteer version: 1.11.0
- Platform / OS version: AWS Lambda
- URLs (if applicable):
- Node.js version: 8.10
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 9
- Comments: 26 (1 by maintainers)
Looks like a dupe of #2463
--single-processflag solved the issue for me.@jared-w-smith not sure if it will help but I solved my problem by changing the version of Chromium back to 79, which based on what module you are using, if you are just using Puppeteer, you can downgrade to
2.0.xMy solution with
chrome-aws-lamdais here just a reference for anyone experiencing the same problem.I find that clearing the
/tmpdirectory before and after launching chrome the number of invocations goes up to ~ 2000 before it stops working again. I think chrome leaves too much files behind that fill the/tmpdirectory of a lambda container.@userme99x Hi there! Yes, I just found a set that works! “@sparticuz/chromium”: “123.0.1” and “puppeteer-core”: “22.7.1” works as a charm in my env!
Single process Finally, for the purposes of comparison, Chromium supports a single process model that can be enabled using the --single-process command-line switch. In this model, both the browser and rendering engine are run within a single OS process.
The single process model provides a baseline for measuring any overhead that the multi-process architectures impose. It is not a safe or robust architecture, as any renderer crash will cause the loss of the entire browser process. It is designed for testing and development purposes, and it may contain bugs that are not present in the other architectures.
--single-processdid not fix the problem for me, I was using it from the startHad this issue with both 1.11 and 1.12 (chrome 72 and 73). I can confirm that one of flags at: https://github.com/alixaxel/chrome-aws-lambda/blob/master/source/index.js solved this issue for me. I’ve used them all so haven’t isolated which one.
Not sure if I should, but I tried to
try {} catch {}this error to always close the browser, and when I try to close it I get: