Got the same problem on our build servers. 1.7.0 is de facto unusable - changing system settings on the build infrastructure is IMHO not an option at all to “fix” a software issue. @pmuens This would be a strong reason to provide a 1.7.1 bugfix release outside of the standard release cycle.

I verified that 1.8.0 now works again without emitting the EMFILE error.

Ok. Finally found a way to reproduce it.

Here are the steps:

1. Use this serverless.yml and package.json file:

service: deployment

provider:
  name: aws

package:
  include:
    - node_modules/**

functions:
  foo:
    handler: handler.handler

{
  "dependencies": {
    "bluebird": "^3.4.7",
    "chai": "^3.5.0",
    "express": "^4.14.1",
    "jquery": "^3.1.1",
    "lodash": "^4.17.4",
    "mocha": "^3.2.0",
    "serverless-plugin-diff": "^1.3.0",
    "sinon": "^1.17.7",
    "underscore": "^1.8.3",
    "webpack": "^2.2.1"
  }
}

1. cd into the service and install the dependencies:

Note: Those are randomly picked packages

npm install

1. Spin up a container with the following command:

docker run --ulimit nofile=20:20 -v $PWD:/app -w /app -i -t node:4.3.2 bash

1. Package the service:

serverless package

I find it kinda frustrating that the when searching for an solution for this problem on a new setup(Node 14.17.1, Framework 2.72.2) i’m only finding “Closed”-issues? How can this be closed when it is still happens?

Edit: Downgrading from 2.72.2 to 2.72.1 solved the issue in my case, however, i feel that this is caused purely by luck and that the issue might resurface at any point… most likely when i need to fix something urgent in production 😦

2nd edit: Sure thing! I got it working for a while, but then it suddenly started with “Too many files” again…it seems random as i feared…

For those that are still running into this. Check out https://stackoverflow.com/questions/8965606/node-and-error-emfile-too-many-open-files

FROM node:8

RUN apt-get update -qq \
    && apt-get install -y curl zip python-pip python-dev build-essential \
    && pip install awscli

# Prevent `EMFILE: too many open files, scandir`
# https://stackoverflow.com/questions/8965606/node-and-error-emfile-too-many-open-files
RUN ulimit -Sn 65536
RUN echo "session required pam_limits.so" >> /etc/pam.d/common-session
RUN echo "root soft  nofile 40000" >> /etc/security/limits.conf
RUN echo "root hard  nofile 100000" >> /etc/security/limits.conf

WORKDIR /app

Thanks for the updates.

The other quick “solution” would be to go back to readFileSync (which is used e.g. in v1.5.1). However the downside is that large services will eat up a bunch of memory and so CI / CD systems might suffer (that’s why we’ve changed it to createReadStream in the first place).

Other suggestions how to solve this problem are highly welcomed!

Regarding my experiences with the webpack plugin: With the graceful-fs fix I experienced that the serverless process will just exit to the shell after printing “Packaging service…”. Not exactly sure what’s happening there exactly.

@ThomasEg if you want it to be solved from the source it is might be smarter to open another detailed issue

It is not a new issue, it’s been an issue for the last couple of years. I highly doubt it can/will be solved because i create a new issue. I mostly added my comment for “google history”. I’m not even sure it’s an issue with Serverless, NodeJS or my setup somehow, but it’s still broken and it seems nondeterministic somehow, which makes error reports like mine almost useless…

@scott-martin This is on the “urgent” list and will be fixed soon!

There’s already the graceful-fs PR out there (#3259) which resolves the issue. However it introduces other issues with e.g. the usage of the serverless webpack plugin (@HyperBrain discovered this).

Not sure if there’s another way right now to fix this in an elegant way.

This error has officially made serverless UNUSABLE. its been ongoing for the longest time and down grading is NOT an option. all solutions online are only posted for OSX and nothing is out there for windows. Unusable Framework

@fsobh i fixed it by exluding the heavier node_modules dependencies in

package:
  exlude:
      # list of biggest modules that are in devdepenedecies and similar
      - node_modules/aws-sdk/**
      - node_modules/serverless-domain-manager/**
      - node_modules/@serverless
      - node_modules/serverless
      - node_modules/java-invoke-local
      - node_modules/tabtab
      - node_modules/snappy

same for me. Node 7.5 failing with same error after running serverless deploy --noDeploy

events.js:161
      throw er; // Unhandled 'error' event
      ^
Error: EMFILE: too many open files, open .../node_modules/serverless/node_modules/lodash/sum.js'

This happens with Node v6.9.5 too. I can reproduce with the node:6.9.5 Docker image. Using --ulimit nofile=20:20 didn’t work for me though. NPM would fail installing the dependencies and Serverless itself. I launched the container without the option, cd /app && npm install && npm install -g serverless, then ulimit -n 1024, then sls deploy --noDeploy.

@stevecaldwell77 On Windows here so no such luck

FYI, I ran into this issue too and worked around it by upping my open file limit (I’m on OSX) to 65536 as described here:

http://blog.mact.me/2014/10/22/yosemite-upgrade-changes-open-file-limit