serverless-dynamodb-local: ECONNREFUSED for sls dynamodb start

Actual Behaviour

Trying to start the database with sls dynamodb start it hangs for a while then gives error:

connect ECONNREFUSED 127.0.0.1:8000

I was able to get it running earlier, but after restarting the computer it stopped working.

Restarting the computer, changing the port and clearing reinstalling node_modules doesn’t change anything.

There is nothing running on that port, and there are no java processes already running to suggest the database is already alive. http://localhost:8000/shell does not run and the serverless functions timeout trying to connect to the database.

Expected Behaviour

Run sls dynamodb start and it runs.

Steps to reproduce it

// package.json
{
  // ...
  "devDependencies": {
    // ...
    "dynamodb-local": "^0.0.24",
    // ...
    "serverless-dynamodb-local": "^0.2.35",
    "serverless-offline": "^3.31.3",
    // ...
  }
}
# serverless.yml
custom:
  # ...
  dynamodb:
    stages:
      - dev
    start:
      migrate: true
      inMemory: true
      noStart: true

plugins:
  - serverless-step-functions
  - serverless-plugin-typescript
  - serverless-dynamodb-local
  - serverless-step-functions-offline
  - serverless-offline

# ...

resources:
  Resources:
    ExampleTable:
      Type: AWS::DynamoDB::Table
      Properties:
        AttributeDefinitions:
          - AttributeName: foo
            AttributeType: S
        KeySchema:
          - AttributeName: foo
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMODB_TABLE}

LogCat for the issue

$ SLS_DEBUG=* yarn sls dynamodb start
yarn run v1.12.3
$ serverless dynamodb start
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command invoke
Serverless: Load command invoke:stepf
Serverless: Load command dynamodb
Serverless: Load command dynamodb:migrate
Serverless: Load command dynamodb:seed
Serverless: Load command dynamodb:start
Serverless: Load command dynamodb:noStart
Serverless: Load command dynamodb:remove
Serverless: Load command dynamodb:install
Serverless: Load command step-functions-offline
Serverless: Load command offline
Serverless: Load command offline:start
Serverless: Invoke dynamodb:start
Serverless: DynamoDB - Error -

  Networking Error ---------------------------------------

    connect ECONNREFUSED 127.0.0.1:8000

        For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

NetworkingError: connect ECONNREFUSED 127.0.0.1:8000
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1121:14)
From previous event:
    at ServerlessDynamodbLocal.createTable (/Users/foo/bar/node_modules/serverless-dynamodb-local/index.js:290:16)
    at BbPromise.each (/Users/foo/bar/node_modules/serverless-dynamodb-local/index.js:169:55)
From previous event:
    at ServerlessDynamodbLocal.migrateHandler (/Users/foo/bar/node_modules/serverless-dynamodb-local/index.js:169:26)
    at BbPromise.resolve.then (/Users/foo/bar/node_modules/serverless-dynamodb-local/index.js:219:45)
From previous event:
    at ServerlessDynamodbLocal.startHandler (/Users/foo/bar/node_modules/serverless-dynamodb-local/index.js:219:10)
    at BbPromise.reduce (/Users/foo/bar/node_modules/serverless/lib/classes/PluginManager.js:391:55)
From previous event:
    at PluginManager.invoke (/Users/foo/bar/node_modules/serverless/lib/classes/PluginManager.js:391:22)
    at PluginManager.run (/Users/foo/bar/node_modules/serverless/lib/classes/PluginManager.js:422:17)
    at variables.populateService.then.then (/Users/foo/bar/node_modules/serverless/lib/Serverless.js:157:33)
    at processImmediate (timers.js:632:19)
    at process.topLevelDomainCallback (domain.js:120:23)
From previous event:
    at Serverless.run (/Users/foo/bar/node_modules/serverless/lib/Serverless.js:144:8)
    at serverless.init.then (/Users/foo/bar/node_modules/serverless/bin/serverless:44:28)

  Get Support --------------------------------------------
    Docs:          docs.serverless.com
    Bugs:          github.com/serverless/serverless/issues
    Issues:        forum.serverless.com

  Your Environment Information -----------------------------
    OS:                     darwin
    Node Version:           11.3.0
    Serverless Version:     1.34.1

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Reactions: 4
  • Comments: 27

Most upvoted comments

Any news on this? I have similar issues whereby ^C doesn’t terminate the dynamodb-local process, although is said otherwise in the documentation. It only stops the serverless offline process.

However, once I do

 kill $(lsof -t -i :8000)

to kill all process runing on that port, and then run

sls offline start

for the next time around, the terminal hangs in this stage:

....
Dynamodb Local Started, Visit: http://localhost:8000/shell
....

and then it finally displays the error

Networking Error ---------------------------------------

  connect ECONNREFUSED 127.0.0.1:8000

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

NetworkingError: connect ECONNREFUSED 127.0.0.1:8000

before terminating the prompt.

However even after the prompt is terminated, i.e.

error Command failed with exit code 1.

http://localhost:8000/shell/ works, which is very very weird and sad.

Restarting the whole process would yield the exact same behavior 😦 And yes, even after running the command sls dynamodb remove, the exact same behavior is still happening.

Any help would be appreciated 😃

Ok. Figured out the fix. Had the .dynamodb directory lying around.

rm -r .dynamodb

And I was able to start up the database. Though is this expected behavior? Why did it cause this particular error and how do I stop it in the future without manually deleting things?

In my case was a bad indentation of noStart: true, which has to be at the same level as start

I run into this problem when I start tests which want to connect to the DynamoDB.

Setup: GitLab CI Custom docker image: Ubuntu with Node and Open JDK 11

The error does NOT occur during startup (all tables seem to be created successfully), but then connecting fails in the subsequent jest tests.

NetworkingError: connect ECONNREFUSED 127.0.0.1:8000Error: connect ECONNREFUSED 127.0.0.1:8000

Needless to say, on dev machines running Linux or Mac OS, this is all running fine.

This happens to me when I run these commands inside a container. It doesn’t happen on my local machine. I haven’t found out why 😕

@ahonarmand-xealth I’ve got same issue. It’s about container, actually JVM. JVM use C runtime internally such as glibc, or musl. DynamoDB offline is running on JVM with glibc, not musl. If you use an container based alpine, it uses musl, so dynamoDB is not working. You can use an docker image with glibc and JVM for dynamodb offline.

If you want to use dynamodb offline with alpine linux, check the links below.

  1. https://github.com/jetsbee/dockerInDocker
  2. https://blog.gilliard.lol/2018/11/05/alpine-jdk11-images.html

The way in second link isn’t working, because some JVM modules for running dynamodb-offline are not included. but you can get some related info in this link.

I’m on MacOS. This command kill the java process that is using port 8000: kill -9 $(lsof -i TCP:8000| grep LISTEN | awk '{print $2}')

Credit: https://medium.com/@valgaze/utility-post-whats-running-on-port-8000-and-how-to-stop-it-2ed771fbb422

If you’re running in a Windows 10 environment, then there is an issue with the dynamodb start command using child_process to spawn the java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb.

The solution I found was to modify node_modules/dynamodb-localhost/dynamodb/start.js and include the flag shell: true into the options at line 38.

In my case was a bad indentation of noStart: true, which has to be at the same level as start

Unbelievable solution. This fixed it for me @diosney . Thank you. Hopefully it will help someone else

Finally fix this issue (working with typescript and webpack from this code base)

  • Install: sls dynamodb install
  • Reset: sls dynamodb remove
  • Run locally: serverless offline start --stage test
# serverless.yml
plugins:
  - serverless-webpack
  - serverless-dynamodb-local
  - serverless-offline

custom:
  dynamodb:
    start:
      inMemory: true
      migrate: true
    stages:
      - test
// package.json
{
 "dependencies": {
    "aws-lambda": "^0.1.2",
    "aws-sdk": "^2.625.0",
    "node": "^12.3.1",
    "uuid": "^3.3.2"
  },
  "devDependencies": {
    "@types/aws-lambda": "8.10.1",
    "@types/node": "^8.0.57",
    "@types/uuid": "^3.4.4",
    "serverless-dynamodb-local": "^0.2.39",
    "serverless-offline": "^4.10.6",
    "serverless-webpack": "^5.1.1",
    "source-map-support": "^0.5.6",
    "ts-loader": "^4.2.0",
    "typescript": "^2.9.2",
    "webpack": "^4.5.0"
  }
}

This appears to happen whenever you define the port that serverless-offline is running on sls offline start --port 3008 fails whilst sls offline start works fine

I ended up just greping for the process that was running dynamo and killing it before starting again.

Then eventually moved the dynamodb into a docker container so its easy to kill now along withe everything else 😄

The problem should be the line noStart: true

From the documentation: # Uncomment only if you already have a DynamoDB running locally # noStart: true