aerospike-client-nodejs: Randomly getting AEROSPIKE_ERR_TIMEOUT

Unhandled rejection AerospikeError: AEROSPIKE_ERR_TIMEOUT
    at Function.AerospikeError.fromASError (/home/eduard/node_modules/aerospike/lib/aerospike_error.js:113:10)
    at Client.DefaultCallbackHandler [as callbackHandler] (/home/eduard/node_modules/aerospike/lib/client.js:71:72)
    at putCb (/home/eduard/node_modules/aerospike/lib/client.js:1247:10)
Unhandled rejection AerospikeError: AEROSPIKE_ERR_TIMEOUT
    at Function.AerospikeError.fromASError (/home/eduard/node_modules/aerospike/lib/aerospike_error.js:113:10)
    at Client.DefaultCallbackHandler [as callbackHandler] (/home/eduard/node_modules/aerospike/lib/client.js:71:72)
    at putCb (/home/eduard/node_modules/aerospike/lib/client.js:1247:10)
Unhandled rejection AerospikeError: AEROSPIKE_ERR_TIMEOUT
    at Function.AerospikeError.fromASError (/home/eduard/node_modules/aerospike/lib/aerospike_error.js:113:10)
    at Client.DefaultCallbackHandler [as callbackHandler] (/home/eduard/node_modules/aerospike/lib/client.js:71:72)
    at /home/eduard/node_modules/aerospike/lib/client.js:836:10

I get about 560 of the first error message at once, sometimes. Here’s my config:

Aerospike.client({
    hosts: [{
        addr: '127.0.0.1',
        port: 3000
    }],
    log: {
        level: Aerospike.log.WARN
    },
    policies: {
        timeout: 5000
    },
    maxConnsPerNode: 1024
})

namespace live {
    replication-factor 2
    memory-size 500M
    storage-engine device {
        file /home/eduard/db/db.dat
        filesize 2G
        data-in-memory true
        post-write-queue 1
    }
}

DB and client are on the same machine (server 3.9.0.3, node client 2.3.1). I can provide more information if needed. Is there anything I can do to fix these timeouts?

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 2
Comments: 22 (10 by maintainers)

Most upvoted comments

Closing this issue.

While not a direct fix for this issue, client release v3 has improved timeout & retry handling. The new BasePolicy class has two separate socketTimeout and totalTimeout policy values; while the former is the socket idle timeout value for the network connection to the server, the latter is the total timeout value for the entire database operation. If totalTimeout > socketTimeout the client will automatically retry the operation up to maxRetries times.

jhecking on Oct 10, 2017

The stack traces for errors raised in async operation callbacks are unfortunately not very useful. I’d like to add support for better stack traces at least in debug/development mode similar to how e.g. the redis client does it. But you can’t do that in production since it comes with a pretty significant performance penalty. What is clear from the stack traces is that the timeouts happen in both put and get operations.

jhecking on Aug 23, 2016