nats.js: Unexpected unhandled NatsError: BAD_AUTHENTICATION

Client version: 2.8.0
Node version: 16.14.2

Some servers, after loosing the connection to the NATS node receive following error:

NatsError: BAD_AUTHENTICATION
    at Function.errorForCode (/code/node_modules/nats/lib/nats-base-client/error.js:98:16)
    at ProtocolHandler.processInfo (/code/node_modules/nats/lib/nats-base-client/protocol.js:402:47)
    at ProtocolHandler.push (/code/node_modules/nats/lib/nats-base-client/protocol.js:435:22)
    at Parser.parse (/code/node_modules/nats/lib/nats-base-client/parser.js:460:45)
    at ProtocolHandler.<anonymous> (/code/node_modules/nats/lib/nats-base-client/protocol.js:189:45)
    at Generator.next (<anonymous>)
    at fulfilled (/code/node_modules/nats/lib/nats-base-client/protocol.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)

chainedError property of the error says that:

TypeError: Cannot read property 'write' of undefined
    at NodeTransport.send (/code/node_modules/nats/lib/src/node_transport.js:327:21)
    at ProtocolHandler.processInfo (/code/node_modules/nats/lib/nats-base-client/protocol.js:398:32)
    at ProtocolHandler.push (/code/node_modules/nats/lib/nats-base-client/protocol.js:435:22)
    at Parser.parse (/code/node_modules/nats/lib/nats-base-client/parser.js:460:45)
    at ProtocolHandler.<anonymous> (/code/node_modules/nats/lib/nats-base-client/protocol.js:189:45)
    at Generator.next (<anonymous>)
    at fulfilled (/code/node_modules/nats/lib/nats-base-client/protocol.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)

As seen from the chained error, it’s actually not an auth error but some race condition inside of the NATS base client, which encounters an error here.

Since that error forces connection close, it wrongly closes connection for good and NATS will no longer try to reconnect despite there’s actually no auth error.

This is an extremely rare error I saw only happening during reconnection on the server. One of the servers started to throw NatsError: DISCONNECTED, and after a few attempts, threw NatsError: BAD_AUTHENTICATION and closed the connection.

This is how statuses changed:

The issue is probably related to https://github.com/nats-io/nats.js/issues/523 because it happened simultaneously, though only on 1 machine.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 20 (11 by maintainers)

Commits related to this issue

[WIP] possible fix for #524 — committed to nats-io/nats.js by aricart 2 years ago

Most upvoted comments

@aricart Thank you for keeping an eye on it. Edge version was deployed yesterday. So far didn’t see it. Let’s see how it goes through the next week.

ArmorDarks on Nov 12, 2022

So the current node transport, nullifies the socket, and then calls _closed(), which actually would know and log the error. However because of the unexpected close event, it NPEs and that mechanism never reports on that. I am flipping the behaviour - the client will log the error (because currently there’s not much it can do with it)

@ArmorDarks npm install nats@next if you would like to try the fix, and see if you run into the issue.

aricart on Oct 25, 2022