prerender: Unrecoverable error with headless chrome

prerender version: 5.0.1 chrome version: Started Chrome: HeadlessChrome/62.0.3202.94

2017-11-16T16:41:14.452Z getting http://peka2.tv/muzmurkin
2017-11-16T16:41:16.589Z unable to evaluate javascript on the page
2017-11-16T16:41:16.589Z Chrome connection closed during request
2017-11-16T16:41:16.590Z got 504 in 2138ms for http://peka2.tv/muzmurkin
2017-11-16T16:41:16.590Z error closing Chrome tab Error: not opened
    at WebSocket.send (/root/chat/node_modules/chrome-remote-interface/node_modules/ws/lib/WebSocket.js:355:18)
    at Chrome.enqueueCommand (/root/chat/node_modules/chrome-remote-interface/lib/chrome.js:124:16)
    at /root/chat/node_modules/chrome-remote-interface/lib/chrome.js:86:28
    at new Promise (<anonymous>)
    at Chrome.send (/root/chat/node_modules/chrome-remote-interface/lib/chrome.js:85:16)
    at Object.handler [as closeTarget] (/root/chat/node_modules/chrome-remote-interface/lib/api.js:32:23)
    at Promise (/root/chat/node_modules/prerender/lib/browsers/chrome.js:146:22)
    at new Promise (<anonymous>)
    at Object.chrome.closeTab (/root/chat/node_modules/prerender/lib/browsers/chrome.js:144:9)
    at Object.server.finish (/root/chat/node_modules/prerender/lib/server.js:239:16)
2017-11-16T16:41:16.686Z getting http://peka2.tv/muzmurkin
{ Error: connect ECONNREFUSED 127.0.0.1:9222
    at Object._errnoException (util.js:1031:13)
    at _exceptionWithHostPort (util.js:1052:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1195:14)
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 9222 }
2017-11-16T16:41:16.696Z got 504 in 10ms for http://peka2.tv/muzmurkin
2017-11-16T16:41:17.060Z getting http://peka2.tv/muzmurkin
{ Error: connect ECONNREFUSED 127.0.0.1:9222
    at Object._errnoException (util.js:1031:13)
    at _exceptionWithHostPort (util.js:1052:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1195:14)
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 9222 }
2017-11-16T16:41:17.065Z got 504 in 5ms for http://peka2.tv/muzmurkin

and everything further dies in 504 and ECONNREFUSED 127.0.0.1:9222 until manual service restart service code

var prerender = require('prerender');

var memoryCache = require('./cache');

var server = prerender({
  chromeFlags: [ '--no-sandbox', '--headless', '--disable-gpu', '--remote-debugging-port=9222', '--hide-scrollbars' ],
  workers: 4,
});

server.use(prerender.sendPrerenderHeader());
server.use(prerender.removeScriptTags());
// handle 404
server.use(prerender.httpHeaders());
server.use(memoryCache);

server.start();

no-sandbox used due to running via root inside docker container

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 42 (19 by maintainers)

Most upvoted comments

I just pushed v5.2.0 of the Prerender server to have Prerender restart Chrome every 10 minutes if there are no requests in flight and every hour if there are requests in flight for the whole hour.

At the hour mark, all new requests will wait until in-flight requests have finished and until the server is restarted before continuing. Again, that’s only if there’s a browser request in flight for the full hour. The waiting is to prevent a 504 for any URLs by letting them take a little longer to load by waiting.

This bandaid isn’t ideal but it should be a short term fix and shouldn’t cause any issues.

Please let me know if you try this out and if it improves the stability of the Prerender server.

Hm, one more issue

2017-11-26T18:14:17.743Z got 200 in 2054ms for http://peka2.tv/nikulin
2017-11-26T18:14:18.335Z getting http://peka2.tv/neosapient
2017-11-26T18:14:18.337Z got 200 in 2ms for http://peka2.tv/neosapient
2017-11-26T18:14:23.520Z getting http://peka2.tv/darsas2
2017-11-26T18:14:25.563Z got 200 in 2043ms for http://peka2.tv/darsas2
2017-11-26T18:16:38.204Z getting http://peka2.tv/beshaniyzayka
2017-11-26T18:16:58.239Z page timed out http://peka2.tv/beshaniyzayka
2017-11-26T18:17:03.243Z parse html timed out http://peka2.tv/beshaniyzayka
2017-11-26T18:17:03.243Z got 504 in 25039ms for http://peka2.tv/beshaniyzayka
2017-11-26T18:18:20.677Z getting http://peka2.tv/
2017-11-26T18:19:20.666Z getting http://peka2.tv/
2017-11-26T18:19:22.420Z getting http://peka2.tv/news/2017/11/23/konkurs-issledovanie-ostalos-li-na-youtube-chto-to-smeshnoe
-chetverg-20-00-po-moskve
2017-11-26T18:19:22.421Z got 200 in 1ms for http://peka2.tv/news/2017/11/23/konkurs-issledovanie-ostalos-li-na-youtube-chto-
to-smeshnoe-chetverg-20-00-po-moskve
2017-11-26T18:19:27.959Z getting http://peka2.tv/anyreu
2017-11-26T18:20:20.656Z getting http://peka2.tv/
2017-11-26T18:20:35.392Z getting http://peka2.tv/
2017-11-26T18:25:09.558Z getting http://peka2.tv/pawlow
2017-11-26T18:25:27.756Z getting http://peka2.tv/adolfra
2017-11-26T18:25:27.760Z got 200 in 4ms for http://peka2.tv/adolfra
2017-11-26T18:26:56.342Z getting http://peka2.tv/akudji
2017-11-26T18:26:56.345Z got 200 in 3ms for http://peka2.tv/akudji
2017-11-26T18:27:13.642Z getting http://peka2.tv/blogs/adolfra/2017/11/06/kak-razmnozhayutsya-chitery-v-pubg
2017-11-26T18:27:13.644Z got 200 in 2ms for http://peka2.tv/blogs/adolfra/2017/11/06/kak-razmnozhayutsya-chitery-v-pubg
2017-11-26T18:28:00.852Z getting http://peka2.tv/
2017-11-26T18:29:00.862Z getting http://peka2.tv/
2017-11-26T18:30:00.837Z getting http://peka2.tv/
2017-11-26T18:31:00.869Z getting http://peka2.tv/
2017-11-26T18:32:00.873Z getting http://peka2.tv/
2017-11-26T18:33:00.842Z getting http://peka2.tv/
2017-11-26T18:33:20.698Z getting http://peka2.tv/tapkina
2017-11-26T18:34:01.257Z getting http://peka2.tv/kngarthur
2017-11-26T18:34:18.627Z getting http://peka2.tv/kolesnik
2017-11-26T18:34:18.628Z got 200 in 1ms for http://peka2.tv/kolesnik
2017-11-26T18:35:01.331Z getting http://peka2.tv/kngarthur
2017-11-26T18:36:01.375Z getting http://peka2.tv/kngarthur
2017-11-26T18:36:29.715Z getting http://peka2.tv/krasnaisx7
2017-11-26T18:37:01.480Z getting http://peka2.tv/chirik2016
2017-11-26T18:38:01.530Z getting http://peka2.tv/chirik2016
2017-11-26T18:39:01.584Z getting http://peka2.tv/chirik2016
2017-11-26T18:39:42.930Z getting http://peka2.tv/annadanboo
2017-11-26T18:39:46.854Z getting http://peka2.tv/n1kname
2017-11-26T18:40:13.821Z getting http://peka2.tv/
2017-11-26T18:40:24.233Z getting http://peka2.tv/lp
2017-11-26T18:41:01.589Z getting http://peka2.tv/istigi
2017-11-26T18:43:15.292Z getting http://peka2.tv/
2017-11-26T18:45:06.508Z getting http://peka2.tv/
2017-11-26T18:45:43.378Z getting http://peka2.tv/
2017-11-26T18:46:06.584Z getting http://peka2.tv/
2017-11-26T18:46:25.482Z getting http://peka2.tv/oxygen2007rus
2017-11-26T18:46:25.483Z got 200 in 1ms for http://peka2.tv/oxygen2007rus
2017-11-26T18:46:28.773Z getting http://peka2.tv/nikadotova
2017-11-26T18:46:28.774Z got 200 in 1ms for http://peka2.tv/nikadotova
2017-11-26T18:46:45.829Z getting http://peka2.tv/ichirei
2017-11-26T18:46:45.830Z got 200 in 1ms for http://peka2.tv/ichirei
2017-11-26T18:47:06.611Z getting http://peka2.tv/hedopa3ymehue
2017-11-26T18:47:06.613Z got 200 in 2ms for http://peka2.tv/hedopa3ymehue
2017-11-26T18:47:07.589Z getting http://peka2.tv/
2017-11-26T18:48:07.588Z getting http://peka2.tv/zabanennbli
2017-11-26T18:48:07.588Z got 200 in 0ms for http://peka2.tv/zabanennbli
2017-11-26T18:48:08.054Z getting http://peka2.tv/blogs/cocacola
2017-11-26T18:48:14.937Z getting http://peka2.tv/
2017-11-26T18:48:26.291Z getting http://peka2.tv/

and so one. I dont know why, in container i see a lot of chrome processes

root@srv5:/# ps -aux                                                                                                       
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  20056  2808 ?        Ss   Nov25   0:00 /bin/bash /home/node/entrypoint.sh supervisord -n
root         7  0.0  0.0  56008 16228 ?        S    Nov25   0:16 /usr/bin/python /usr/bin/supervisord -n
node        10  0.0  0.0  20052  2760 ?        S    Nov25   0:00 /bin/bash -c sleep 10s; cd /home/node/chat; node modules/pr
node        12  0.5  0.9 1802008 635476 ?      Sl   Nov25   9:22 node modules/prerender/index.js
node      4491  1.7  0.5 863108 330576 ?       Sl   12:08   7:09 /usr/bin/google-chrome --headless --disable-gpu --remote-de
node      4497  0.0  0.0   4396   700 ?        S    12:08   0:00 cat
node      4498  0.0  0.0   4396   748 ?        S    12:08   0:00 cat
node      4500  0.0  0.0   6392   732 ?        S    12:08   0:00 /opt/google/chrome/chrome-sandbox /opt/google/chrome/chrome
node      4501  0.0  0.0 346424 37376 ?        S    12:08   0:00 /opt/google/chrome/chrome --type=zygote --headless --headle
node      4503  0.0  0.0 346424 10260 ?        S    12:08   0:01 /opt/google/chrome/chrome --type=zygote --headless --headle
node      4524  0.0  0.0 1020228 63944 ?       Sl   12:08   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node      4947  0.0  0.0 432124 48748 ?        Sl   12:12   0:05 /opt/google/chrome/chrome --type=gpu-process --headless --h
node      4948  0.0  0.0 352132 11688 ?        S    12:12   0:00 /opt/google/chrome/chrome --type=gpu-broker
node     11657  0.0  0.0 395600 16988 ?        Sl   18:18   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11666  0.0  0.0 395600 16988 ?        Sl   18:19   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11676  0.0  0.0 395600 16988 ?        Sl   18:19   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11685  0.0  0.0 395600 16988 ?        Sl   18:20   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11695  0.0  0.0 395600 16988 ?        Sl   18:20   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11705  0.0  0.0 395600 16988 ?        Sl   18:25   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11715  0.0  0.0 395600 16988 ?        Sl   18:28   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11725  0.0  0.0 395600 16988 ?        Sl   18:29   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11734  0.0  0.0 395600 16988 ?        Sl   18:30   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11744  0.0  0.0 395600 16988 ?        Sl   18:31   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11754  0.0  0.0 395600 16988 ?        Sl   18:32   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11763  0.0  0.0 395600 16988 ?        Sl   18:33   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11773  0.0  0.0 395600 16988 ?        Sl   18:33   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11782  0.0  0.0 395600 16988 ?        Sl   18:34   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11792  0.0  0.0 395600 16988 ?        Sl   18:35   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11802  0.0  0.0 395600 16988 ?        Sl   18:36   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11811  0.0  0.0 395600 16988 ?        Sl   18:36   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11821  0.0  0.0 395600 16988 ?        Sl   18:37   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11831  0.0  0.0 395600 16988 ?        Sl   18:38   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11841  0.0  0.0 395600 16988 ?        Sl   18:39   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11850  0.0  0.0 395600 16988 ?        Sl   18:39   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11859  0.0  0.0 395600 16988 ?        Sl   18:39   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11869  0.0  0.0 395600 16988 ?        Sl   18:40   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11878  0.0  0.0 395600 16988 ?        Sl   18:40   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11887  0.0  0.0 395600 16988 ?        Sl   18:41   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11897  0.0  0.0 395600 16988 ?        Sl   18:43   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11907  0.0  0.0 395600 16988 ?        Sl   18:45   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11916  0.0  0.0 395600 16988 ?        Sl   18:45   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11926  0.0  0.0 395600 16988 ?        Sl   18:46   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11936  0.0  0.0 395600 16988 ?        Sl   18:47   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11945  0.0  0.0 395600 16988 ?        Sl   18:48   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11955  0.0  0.0 395600 16988 ?        Sl   18:48   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11964  0.0  0.0 395600 16988 ?        Sl   18:48   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11980  0.0  0.0 395600 16988 ?        Sl   18:49   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11989  0.0  0.0 395600 16988 ?        Sl   18:49   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     11998  0.0  0.0 395600 16988 ?        Sl   18:49   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     12007  0.0  0.0 395600 16988 ?        Sl   18:49   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     12017  0.0  0.0 395600 16988 ?        Sl   18:50   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     12026  0.0  0.0 395600 16988 ?        Sl   18:50   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
node     12035  0.0  0.0 395600 16988 ?        Sl   18:50   0:00 /opt/google/chrome/chrome --type=renderer --use-gl=swiftsha
root     12044  0.0  0.0  20228  3036 ?        Ss   18:50   0:00 bash
root     12050  0.0  0.0  17504  2116 ?        R+   18:50   0:00 ps -aux

looks like they are hanged. Test request with ?_escaped_fragment_=1 returns 504. Everything is fine after container restart.

Aw bummer. We have some health checks on our infrastructure that detect any issues and pulls those servers from the load balancer. We also have other checks to restart Chrome every so often. You might want to try doing something like that too.

It’ll be hard to look into this without a reproducible test case, and from what you sent over it looks like Prerender is losing the connection to Chrome somehow. I’ll try to look into some ways to improve general reliability of detecting an issue with the connection.