Coming from https://github.com/Expensify/App/issues/21444

Threads with context: https://expensify.slack.com/archives/C03TQ48KC/p1687893191020179 https://expensify.slack.com/archives/C036S0BVAHH/p1687812903398059

Problem

Setting a lot of data in Onyx is super slow.

This started/finished is calls to onyx. You can see it takes 20 seconds to set the key 19 and 1 minute 8 seconds the second one.

19:58:45.761 web.development.js:1304 started 19 () => mergeCollection(key, value)
19:58:45.762 web.development.js:1304 started 20 () => mergeCollection(key, value)
19:59:06.247 web.development.js:1305 finished 19 () => mergeCollection(key, value)
19:59:53.190 web.development.js:1305 finished 20 () => mergeCollection(key, value)

I have a file with the data that’s failing, but I think it might contain sensitive information but I can say it’s 30MB uncompressed. The 20th call above is the biggest as one might expect.

In order to test this in dev (for internal engineers only) is to change the OpenApp code to (data.json is the file with all the data):

    } elseif ($command === 'OpenApp') {
        $response = json_decode(file_get_contents('data.json'), true);
//        $response['onyxData'] = AppInit::openApp($authToken, $_REQUEST['policyIDList'] ?? '');

I assume it can also be reproduced by joining 1000s of policies. This user’s data:

• 3216 policies
• 7k personal details (x 2)
• 9k reports

Solution

Either make onyx be faster or recognize we need to do pagination of policies and reports

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01a864f2f0848526ae
• Upwork Job ID: 1676300792032952320
• Last Price Increase: 2023-07-04