iconv-lite: Stumped on error with requiring ../encodings

I’ve been stumped by this error a few times, and it’s really hard to reproduce because it only happens once on a new install. So in a desperate attempt to glean more information, I’m posting this here to see if someone with a bit more knowledge can help point me in the right direction.

This error seems to be something with iconv-lite failing to require ../encodings but it does exist. What’s even more strange is that if you restart the server it works fine thereafter.

Here is the stacktrace of the error:

15:40:48 err! [console] Error: Cannot find module '../encodings'
    at Function.Module._resolveFilename (module.js:326:15)
    at Function.Module._load (module.js:277:25)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.getCodec (/somepath/node_modules/body-parser/node_modules/iconv-lite/lib/index.js:61:27)
    at Object.getDecoder (/somepath/node_modules/body-parser/node_modules/iconv-lite/lib/index.js:118:23)
    at getDecoder (/somepath/node_modules/body-parser/node_modules/raw-body/index.js:44:18)
    at readStream (/somepath/node_modules/body-parser/node_modules/raw-body/index.js:218:15)
    at getRawBody (/somepath/node_modules/body-parser/node_modules/raw-body/index.js:106:12)
    at read (/somepath/node_modules/body-parser/lib/read.js:76:3)
    at jsonParser (/somepath/node_modules/body-parser/lib/types/json.js:121:5)
    at Layer.handle [as handle_request] (/somepath/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/somepath/node_modules/express/lib/router/index.js:312:13)
    at /somepath/node_modules/express/lib/router/index.js:280:7
    at Function.process_params (/somepath/node_modules/express/lib/router/index.js:330:12)
    at next (/somepath/node_modules/express/lib/router/index.js:271:10)
    at expressInit (/somepath/node_modules/express/lib/middleware/init.js:33:5)
    at Layer.handle [as handle_request] (/somepath/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/somepath/node_modules/express/lib/router/index.js:312:13)
    at /somepath/node_modules/express/lib/router/index.js:280:7
    at Function.process_params (/somepath/node_modules/express/lib/router/index.js:330:12)
    at next (/somepath/node_modules/express/lib/router/index.js:271:10)
    at query (/somepath/node_modules/express/lib/middleware/query.js:49:5)
    at Layer.handle [as handle_request] (/somepath/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/somepath/node_modules/express/lib/router/index.js:312:13)
    at /somepath/node_modules/express/lib/router/index.js:280:7
    at Function.process_params (/somepath/node_modules/express/lib/router/index.js:330:12)
    at next (/somepath/node_modules/express/lib/router/index.js:271:10)
    at Function.handle (/somepath/node_modules/express/lib/router/index.js:176:3)
    at EventEmitter.handle (/somepath/node_modules/express/lib/application.js:173:10)
    at Server.app (/somepath/node_modules/express/lib/express.js:38:9)
    at Server.<anonymous> (/somepath/node_modules/socket.io/node_modules/engine.io/lib/server.js:434:22)
    at Server.<anonymous> (/somepath/node_modules/socket.io/lib/index.js:260:16)
    at emitTwo (events.js:87:13)
    at Server.emit (events.js:172:7)
    at HTTPParser.parserOnIncoming [as onIncoming] (_http_server.js:528:12)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:88:23)
  EventEmitter.logerror [console] (application.js:629)
    at Console.error (/somepath/baymax.bundle.js:160:3265)
    at EventEmitter.logerror (/somepath/node_modules/express/lib/application.js:629:43)
    at Immediate.immediate._onImmediate (timers.js:445:18)
    at processImmediate [as _immediateCallback] (timers.js:383:17)

About this issue

Original URL
State: open
Created 8 years ago
Reactions: 10
Comments: 37 (11 by maintainers)

Most upvoted comments

I got the same problem as @Marak is describing, and it does not solve if I restart. His fix worked for me. And I agree with @Marak. Instead of excusing this behaviour, treat it as the bug it is and fix, please.

kleggas on Jul 26, 2017

Issue is due to changing the current working directory of node.js process after process starts and then you attempt to perform method using iconv-lite. The lazy require breaks since it’s no longer in the same relative path. Same if node_modules somehow disappears after process start.

I think I found the solution.

Try changing:

iconv.encodings = require("../encodings");

iconv.encodings = require("iconv-lite/encodings");

https://github.com/ashtuchkin/iconv-lite/blob/master/lib/index.js#L61

Marak on Feb 9, 2017

Still experiencing this problem. I am running inside a Docker environment.

FrenchMajesty on Nov 15, 2018

When you run app.js, first thing it does is it loads all the required modules in memory. For most apps it means that you can delete node_modules directory after your process has started and it will work just fine.

Iconv-lite is an outlier in this regard as it can take from 20 to 150ms to load all the character tables (especially chinese, japanese encodings). This translates directly to startup time of any app that uses e.g. express framework. As encoding conversion is not actually needed for 99.99% apps, it makes sense to lazy-load character tables so that you pay this latency penalty only if you actually need them. If at that point node_modules folder is not present, then that error happens.

ashtuchkin on Mar 16, 2021

Thanks for digging through it @MeanManMachineMan! It’s becoming much clearer now. Basically node_modules directory was being cleaned up by a different task while the test was running, right? That would definitely result in this error. Let me know if you need anything else.

ashtuchkin on Mar 16, 2021

@ashtuchkin -

I’ve been reviewing this thread again since it popped up in my feed.

Re-reading my old posts I can see the language and tone I was using wasn’t the best for collaboration.

If you’d like to get this issue closed I believe all the information needed is here posted above with my comments and other user’s comments.

I would consider removing the lazy loading entirely or make it a configurable option which is disabled by default.

Marak on Jul 15, 2020

The lazy loading also seems to be causing an issue in other libraries… https://github.com/facebook/jest/issues/2605

jcreamer898 on Mar 28, 2017

I just encountered this in an expressjs application after upgrading from node 4.2 to 6.6.0. After upgrading node, I removed my node_modules folder and ran a fresh ‘npm install’. First startup seemed healthy until posting a form to the server where I saw this issue. As mentioned by @leanderlee restarting the service seemed to fix it.

ZapRowsdower910 on Sep 22, 2016