ethers.js: React Native: Solutions for slow crypto-operations (`Wallet.createRandom()`)

I did it.

@zemse @mirceanis @ricmoo how do I swap the pbkdf2 function?

The one from this library takes 14 seconds:

ether's pbkdf2 took 14279.91887497902ms

and my pure C++ JSI implementation takes 0.001 second:

C++ pbkdf2 took 1.4742500185966492ms

…so it’s almost 10.000x faster.

I’d love to make that open source if there’s a way to swap the implementations.

Step 1: create a patch js file with the following content:

const crypto = require("react-native-quick-crypto")
exports.pbkdf2 = crypto.pbkdf2Sync

Step 2: update the module-resolver config in babel.config.js:

{
   alias: {
      // other alias ....
      '@ethersproject/pbkdf2': 'path/to/your/patch.js',
    }
}

Step 3: npm start --reset-cache

Hey - we’ve just published the fully native C++ library for Crypto. 🎉🥳

As I said before, doing crypto in JS simply won’t work for mobile apps (React Native) since JS is too slow in such cases. That’s why libraries like BN.js or crypto-browserify are a huge bottleneck in React Native applications. Instead, we at Margelo implemented those functions fully natively in C++ to offer huge speedups (in BN.js we measured up to 300x improvements and in QuickCrypto up to 58x improvements!)

As of today, I recommend to install both react-native-quick-crypto and react-native-bignumber in your projects and use them as drop-in replacements for free app speedups!

Install:

yarn add react-native-bignumber
yarn add react-native-quick-crypto

Then add them to your babel.config.js (see the READMEs for details).

This makes ethers.js run smoothly on React Native! Would be cool if we could add that to the README or docs of ethers.js? cc @ricmoo

Do you have an ETA for v6?

@ricmoo ethers.js currently doesn’t use crypto (nor react-native-quick-crypto) if it is available, not sure where the try/import/catch is happening (or if you guys are doing that), but just wanted to make sure that y’all are aware of this.

I had to manually patch node_modules/@ethersproject/pbkdf2/... to use our crypto.pbkdf2Sync function (which is implemented natively in react-native-quick-crypto) to improve the performance for wallet creation.

const start = performance.now()
const wallet = ethers.Wallet.createRandom()
const end = performance.now()
console.log(`Creating a Wallet took ${end - start} ms.`)

Before patching @ethersproject/pbkdf2 (with your JS-based implementation):

Creating a Wallet took 16862 ms

After patching @ethersproject/pbkdf2 (with our native C++/JSI react-native-quick-crypto implementation):

Creating a Wallet took 289 ms

Here’s the patch I used:

diff --git a/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js
index e211793..06a80af 100644
--- a/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js
+++ b/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js
@@ -1,44 +1,4 @@
 "use strict";
-import { arrayify, hexlify } from "@ethersproject/bytes";
-import { computeHmac } from "@ethersproject/sha2";
-export function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) {
-    password = arrayify(password);
-    salt = arrayify(salt);
-    let hLen;
-    let l = 1;
-    const DK = new Uint8Array(keylen);
-    const block1 = new Uint8Array(salt.length + 4);
-    block1.set(salt);
-    //salt.copy(block1, 0, 0, salt.length)
-    let r;
-    let T;
-    for (let i = 1; i <= l; i++) {
-        //block1.writeUInt32BE(i, salt.length)
-        block1[salt.length] = (i >> 24) & 0xff;
-        block1[salt.length + 1] = (i >> 16) & 0xff;
-        block1[salt.length + 2] = (i >> 8) & 0xff;
-        block1[salt.length + 3] = i & 0xff;
-        //let U = createHmac(password).update(block1).digest();
-        let U = arrayify(computeHmac(hashAlgorithm, password, block1));
-        if (!hLen) {
-            hLen = U.length;
-            T = new Uint8Array(hLen);
-            l = Math.ceil(keylen / hLen);
-            r = keylen - (l - 1) * hLen;
-        }
-        //U.copy(T, 0, 0, hLen)
-        T.set(U);
-        for (let j = 1; j < iterations; j++) {
-            //U = createHmac(password).update(U).digest();
-            U = arrayify(computeHmac(hashAlgorithm, password, U));
-            for (let k = 0; k < hLen; k++)
-                T[k] ^= U[k];
-        }
-        const destPos = (i - 1) * hLen;
-        const len = (i === l ? r : hLen);
-        //T.copy(DK, destPos, 0, len)
-        DK.set(arrayify(T).slice(0, len), destPos);
-    }
-    return hexlify(DK);
-}
+var crypto = require("crypto");
+exports.pbkdf2 = crypto.pbkdf2Sync;
 //# sourceMappingURL=pbkdf2.js.map
diff --git a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
index 45c6f03..f2c584d 100644
--- a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
+++ b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
@@ -1,47 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.pbkdf2 = void 0;
-var bytes_1 = require("@ethersproject/bytes");
-var sha2_1 = require("@ethersproject/sha2");
-function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) {
-    password = (0, bytes_1.arrayify)(password);
-    salt = (0, bytes_1.arrayify)(salt);
-    var hLen;
-    var l = 1;
-    var DK = new Uint8Array(keylen);
-    var block1 = new Uint8Array(salt.length + 4);
-    block1.set(salt);
-    //salt.copy(block1, 0, 0, salt.length)
-    var r;
-    var T;
-    for (var i = 1; i <= l; i++) {
-        //block1.writeUInt32BE(i, salt.length)
-        block1[salt.length] = (i >> 24) & 0xff;
-        block1[salt.length + 1] = (i >> 16) & 0xff;
-        block1[salt.length + 2] = (i >> 8) & 0xff;
-        block1[salt.length + 3] = i & 0xff;
-        //let U = createHmac(password).update(block1).digest();
-        var U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, block1));
-        if (!hLen) {
-            hLen = U.length;
-            T = new Uint8Array(hLen);
-            l = Math.ceil(keylen / hLen);
-            r = keylen - (l - 1) * hLen;
-        }
-        //U.copy(T, 0, 0, hLen)
-        T.set(U);
-        for (var j = 1; j < iterations; j++) {
-            //U = createHmac(password).update(U).digest();
-            U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, U));
-            for (var k = 0; k < hLen; k++)
-                T[k] ^= U[k];
-        }
-        var destPos = (i - 1) * hLen;
-        var len = (i === l ? r : hLen);
-        //T.copy(DK, destPos, 0, len)
-        DK.set((0, bytes_1.arrayify)(T).slice(0, len), destPos);
-    }
-    return (0, bytes_1.hexlify)(DK);
-}
-exports.pbkdf2 = pbkdf2;
+var crypto = require("crypto");
+exports.pbkdf2 = crypto.pbkdf2Sync;
 //# sourceMappingURL=browser-pbkdf2.js.map

You can DM me on Twitter if you have any questions or want to collab on getting this integrated with ethers.js. 😃

Just as an FYI, I’ve been adding an option to v6 to register custom implementations for certain crypto methods, so this should be easier to swap out in v6.

So I patched @ethersproject/pbkdf2/lib/browser-pbkdf2.js to use my custom C++ implementation instead of the JS one, and here’s the results:

Before

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 45114.42308330536ms!

After

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 1120.2051666378975ms!

…so I brought it down from 45 seconds to 1 second, a 45x performance improvement.

diff --git a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
index 45c6f03..41e4bf6 100644
--- a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
+++ b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js
@@ -1,47 +1,10 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.pbkdf2 = void 0;
-var bytes_1 = require("@ethersproject/bytes");
-var sha2_1 = require("@ethersproject/sha2");
+
 function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) {
-    password = (0, bytes_1.arrayify)(password);
-    salt = (0, bytes_1.arrayify)(salt);
-    var hLen;
-    var l = 1;
-    var DK = new Uint8Array(keylen);
-    var block1 = new Uint8Array(salt.length + 4);
-    block1.set(salt);
-    //salt.copy(block1, 0, 0, salt.length)
-    var r;
-    var T;
-    for (var i = 1; i <= l; i++) {
-        //block1.writeUInt32BE(i, salt.length)
-        block1[salt.length] = (i >> 24) & 0xff;
-        block1[salt.length + 1] = (i >> 16) & 0xff;
-        block1[salt.length + 2] = (i >> 8) & 0xff;
-        block1[salt.length + 3] = i & 0xff;
-        //let U = createHmac(password).update(block1).digest();
-        var U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, block1));
-        if (!hLen) {
-            hLen = U.length;
-            T = new Uint8Array(hLen);
-            l = Math.ceil(keylen / hLen);
-            r = keylen - (l - 1) * hLen;
-        }
-        //U.copy(T, 0, 0, hLen)
-        T.set(U);
-        for (var j = 1; j < iterations; j++) {
-            //U = createHmac(password).update(U).digest();
-            U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, U));
-            for (var k = 0; k < hLen; k++)
-                T[k] ^= U[k];
-        }
-        var destPos = (i - 1) * hLen;
-        var len = (i === l ? r : hLen);
-        //T.copy(DK, destPos, 0, len)
-        DK.set((0, bytes_1.arrayify)(T).slice(0, len), destPos);
-    }
-    return (0, bytes_1.hexlify)(DK);
+    // uses my C++ implementation.
+    return global.pbkdf2(password.buffer, salt.buffer, iterations, keylen, hashAlgorithm)
 }
 exports.pbkdf2 = pbkdf2;
 //# sourceMappingURL=browser-pbkdf2.js.map

I still wonder why it takes a full second to import the wallet from the passphrase, I am even using the native C++ atob, btoa and Buffer implementations, will debug further - but at least now the app is useable.

Thank you to everyone in this thread and particularly @ricmoo and @mrousavy for their work with these libraries.

Using the thread above I was able to get Ethers v6.6 and react-native-quick-crypto (RNQC) working with RN. Without using RNQC Ethers v6.6 was able to create a wallet in around 1300ms which was a big improvement from Ethers v5.7 (around 5500ms). However when using RNQC, Ethers V6.6 creates a new wallet in 300ms dropping to 50ms for any additional requests.

I will include my setup for anyone wishing to work with Ethers v6:

1. Follow the installation steps as mentioned in the RNQC repo including updating the babel.config.js file: https://github.com/margelo/react-native-quick-crypto
2. You no longer need to use “react-native-get-random-values” as mentioned in the Ethers v5 docs. Instead use getRandomValues from RNQC before importing the @ethersproject/shims. I created a file called ‘ethers-setup.ts’ with the following:

import { getRandomValues } from 'react-native-quick-crypto';
global.getRandomValues = getRandomValues;

export * from '@ethersproject/shims';

Then imported ‘ethers-setup’ into the App.tsx file.

1. For me, the patch mentioned by @margox only works with Ethers v5. For v6 I utilised the new approach outlined by @ricmoo to register pbkdf2. The code I have looks like this:

import * as ethers from 'ethers';
import { pbkdf2Sync } from 'react-native-quick-crypto';

ethers.pbkdf2.register(
  (
    password: Uint8Array,
    salt: Uint8Array,
    iterations: number,
    keylen: number,
    algo: 'sha256' | 'sha512'
    // eslint-disable-next-line max-params
  ) => {
    return ethers.hexlify(pbkdf2Sync(password, salt, iterations, keylen, algo));
  }
);

export * from 'ethers';

I now import this into my project instead of the usual import { ethers } from ‘ethers’;

This is available in v6, using the .register functions on each of the hash, hmac, etc. functions.

For example:

ethers.keccak256.register((data: Uint8Array) => {
  yourCustomLibHere.hash(data);
});

See the src.ts/crypto/ files for more details. I’ll be adding more docs for RN, Expo, etc. to help too.

Thanks! 😃

Yes, I did.

I’m working on releasing the package as open-source today or tomorrow, stay tuned. Follow me on Twitter for the announcement

@mrousavy Do you have the pure C++ JSI implementation mentioned in the reply above available for public? The example here is for swapping the pbkdf2 ethers would use.

He already said it’s closed source

Yeah great point @mrousavy. I am intrigued to see whether the encrypt wallet method could be improved as it takes 30 seconds+ during testing. It seems this process relies on scrypt but I struggled to get the RNQC library to work with ethers, using the ethers.scrypt.register approach.

I did have a question @mrousavy if you could confirm the security of the native implementations? Ethers uses noble-hashes which has been audited. Can you think of any security issues that could arise by replacing these functions with native implementations you have written?

@mrousavy amazing work! Improvement was from 4108ms down to 210ms replacing @ethersproject/pbkdf2/lib/browser-pbkdf2 with your implementation of pbkdf2Sync 🥳

@anarkrypto I was using an expo managed workflow and apply the @ethersproject/project/pbkdf2 patch applied above. However I found that it won’t work in the managed workflow, and you’ll have to eject to bare workflow if you want it to work due to the native code modification that will happen under the hood. It’s not enough to apply the changes in the babel.config.js you have to patch the pbkdf2 in order to ensure that ethers works performantly when creating new wallets. Hope this helps. I can confirm that the patch does work and lowers wait times from ~6-10 seconds [depending on the device, beforehand it was 6 seconds on new iPhones like the iPhone 13 pro, and 10-12 seconds on iPhone 7s, all the way to sub to single second waits for all phone’s on all builds for new wallet creation in production builds for iOS, I haven’t built it yet for android but I assume that it works as well, but maybe some build.gradle changes will have to be made.

Note: I got the error that it doesn’t work in Expo Go and that’s exactly the point remember this is drop in native code, you’re going to have to use less convenient local building tools like the react-native cli or expo dev-client builds from now on to build the app for emulators locally, and the nice things that come with testing expo go on a real phone on the same network also goes away, hope this helps 😄