jest: Cannot load native module when running under jest

We were having the same issues combining the SAP Hana client library with Jest.

Setting environment variables was no solution for us, since we are working on different systems (Windows, Linux, Mac). The following steps solved it for us:

1. set the global setup parameter in the jest config file:

globalSetup: "<rootDir>/sapHana.config.js",

1. create a config file (in this example: sapHana.config.js in the root folder) which sets the environment variable:

var path = require('path');

module.exports = async () => {
    var extensions = {
        'darwin': 'dylib',
        'linux': 'so',
        'win32': 'dll'
    };
    
    // Look for prebuilt binary and DBCAPI based on platform
    var pb_subdir = null;
    if (process.platform === 'linux') {
        if (process.arch === 'x64') {
            pb_subdir = 'linuxx86_64-gcc48';
        } else if (process.arch.toLowerCase().indexOf('ppc') != -1 && os.endianness() === 'LE') {
            pb_subdir = 'linuxppc64le-gcc48';
        } else {
            pb_subdir = 'linuxppc64-gcc48';
        }
    } else if (process.platform === 'win32') {
        pb_subdir = 'ntamd64-msvc2010';
    } else if (process.platform === 'darwin') {
        pb_subdir = 'darwinintel64-xcode7';
    }
    
    var modpath = path.dirname(require.resolve("@sap/hana-client/README.md"));
    var pb_path = path.join(modpath, 'prebuilt', pb_subdir);
    var dbcapi = process.env['DBCAPI_API_DLL'] || path.join(pb_path, 'libdbcapiHDB.' + extensions[process.platform]);
    
    process.env['DBCAPI_API_DLL'] = dbcapi;
  };

Please note that this was tested with version 2.4.126 of @sap/hana-client and things might change with newer versions.

Thanks for sharing, IMO it’s much easier though to just require("@sap/hana-client/build.js"); in your jest config.

Is there anything under the scenes that would explain why the native (nodejs) require for native modules would behave different via jest?

The problem here is with process.env.

Every test file is run inside VM module. But VM context does not have Node globals(require, process and etc), so Jest modifies these globals and then injects them in. VM and these modifications are needed to ensure all test files run in isolation. a.test.js should not affect the outcome of b.test.js.

process.env is one of the globals which is modified. Jest makes a new object from the prototype of process.env and reimplements the behaviour. Problem is that process.env is a bit special since Node stores env variables in C++ core and passes it down to child processes and native bindings if needed.

So if you do this in normal Node process

process.env['DBCAPI_API_DLL'] = "/path_to_my_project/node_modules/@sap/hana-client/prebuilt/darwinintel64-xcode7/libdbcapiHDB.dylib";

All the child processes and native bindings can see this value in their Environment Variables. This is how hana-client works. It read this value from Environment Variables and then tries to load the file.

The reason it does not work with Jest is because Jest’s process.env is just a Javascript object. When you set a value on it, this update only exists in Javascript’s context. Child processes and Native bindings cannot see it. Since hana-client cannot see this value it fails. But if you do it outside of test context, like in global setup it should work.

I don’t think this issue is solvable from Jest’s codebase, you will have to use the workarounds described above. Nodejs only has one environment per process. if you run your code in VM module, you get a new Javascript Context but you cannot create new environment without creating a new process.

It seems the issue still exists. It fails even when hard coding the DBCAPI_API_DLL and loading the hana-client like below.

process.env['DBCAPI_API_DLL'] = "/path_to_my_project/node_modules/@sap/hana-client/prebuilt/darwinintel64-xcode7/libdbcapiHDB.dylib";
require('/path_to_my_project/node_modules/@sap/hana-client/prebuilt/darwinintel64-xcode7/hana-client_v10.node');

seems that the native module loading done thru jest, behaves different than not via jest.

The stack trace is as below:

Error: Failed to load DBCAPI.
    at internal/modules/cjs/loader.js.Module._extensions..node (internal/modules/cjs/loader.js:807:18)
    at internal/modules/cjs/loader.js.Module.load (internal/modules/cjs/loader.js:653:32)
    at internal/modules/cjs/loader.js.tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at internal/modules/cjs/loader.js.Module._load (internal/modules/cjs/loader.js:585:3)
    at internal/modules/cjs/loader.js.Module.require (internal/modules/cjs/loader.js:692:17)
    at internal/modules/cjs/helpers.js.require (internal/modules/cjs/helpers.js:25:18)
    at /path_to_my_project/node_modules/jest-runtime/build/index.js._loadModule (/path_to_my_project/node_modules/jest-runtime/build/index.js:673:29)
    at /path_to_my_project/node_modules/jest-runtime/build/index.js.requireModule (/path_to_my_project/node_modules/jest-runtime/build/index.js:536:10)
    at /path_to_my_project/node_modules/jest-runtime/build/index.js.requireModuleOrMock (/path_to_my_project/node_modules/jest-runtime/build/index.js:699:21)
    at /path_to_my_project/node_modules/@sap/hana-client/lib/index.js.Object.<anonymous> (/path_to_my_project/node_modules/@sap/hana-client/lib/index.js:112:14)

Is there anything under the scenes that would explain why the native (nodejs) require for native modules would behave different via jest? Or there are more variables required to be set for loading that hana-client module.

Oh my bad.

https://github.com/facebook/jest/blob/master/packages/jest-util/src/createProcessObject.js#L19 I changed a return statement of this function with return process.env