sails: sails.js /socket.io/ URLs behind nginx leads to lots of 'upstream timed out'

I’m running several sails.js instances behind an nginx proxy with sticky sessions. And I keep seeing a lot of messages in my nginx error.log regarding sails.js /socket.io/ URLs timing out:

2016/01/04 20:55:15 [error] 12106#12106: *402088 upstream timed out (110: Connection timed out) while reading response header from upstream, client: x.x.x.x, server: example.com, request: "GET /socket.io/?__sails_io_sdk_version=0.11.0&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=polling&t=1451930055065-4&sid=jvekCYDAcFfu0PLdAAL6 HTTP/1.1", upstream: "http://127.0.0.1:3001/socket.io/?__sails_io_sdk_version=0.11.0&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=polling&t=1451930055065-4&sid=jvekCYDAcFfu0PLdAAL6", host: "example.com", referrer: "https://example.com/languageExchange/chat/63934"

2016/01/04 20:55:17 [error] 12105#12105: *402482 upstream prematurely closed connection while reading response header from upstream, client: y.y.y.y, server: example.com, request: "GET /socket.io/?__sails_io_sdk_version=0.11.0&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=websocket&sid=QnAe1jiKEHgj-zlKAAKu HTTP/1.1", upstream: "http://127.0.0.1:3001/socket.io/?__sails_io_sdk_version=0.11.0&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=websocket&sid=QnAe1jiKEHgj-zlKAAKu", host: "example.com"

It doesn’t happen for every client, but the number of such messages is significant. And sails.js does not show any relevant errors.

How should I investigate the nature of these issues?

Here’s what I’ve tried so far (and it didn’t help):

  • Upgrade socket.io client to the latest version so far (1.3.7)
  • Explicitly turn off caching for /socket.io/ requests in nginx

Here’s the relevant config files:

  • sails sockets.js:
adapter: 'socket.io-redis'
  • nginx:
location ^~ /socket.io/ {
        proxy_pass http://sails;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_no_cache true;
        proxy_cache_bypass true;
        proxy_redirect off;
        proxy_intercept_errors off;
}

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 17 (3 by maintainers)

Most upvoted comments

I read that this may be due to sticky sessions issue. Nginx (free) does not support sticky sessions, therefore long polling will result in a time out error. If you are using Nginx Plus then you can enable sticky sessions and the issue will not be apparent. As suggested by @mikermcneil I suspect dropping the polling and going for basic websocket support when using free nginx will alleviate the issue. I will make the changes to my server config tonight and report back to see if this alleviates the issue.

So the issue is a bad configuration on the NGINX sites-avalable conf file.

location /socket.io/ {
      proxy_pass http://sails/;
...
}

should be

location /socket.io/ {
      proxy_pass http://sails/socket.io/;
...
}

Pretty basic stuff: the “location” is not forwarded to the proxy_pass (why would it be, right?) – So you need to make sure the socket requests are redirected to the exact socket endpoint.