sharp: AWS Lambda running out of memory pretty quickly - then crash/timeout

I am using sharp in one of my application which runs on AWS lambda. I am not using any fancy operation with sharp - just resize & toFormat.

Now the problem is - Lambda “Max Memory Used:” is keep on increasing until it hits the roof and then crash. To illustrate the problem, i have written a small function below

"use strict";

process.env.VIPS_DISC_THRESHOLD = '20m';

const v8    = require("v8"),
    log     = console.log,
    Busboy  = require('busboy'),
    sharp   = require("sharp");

v8.setFlagsFromString('--max_old_space_size=50');  
//v8.setFlagsFromString('--max_new_space_size=50'); 
v8.setFlagsFromString('--trace_fragmentation');
//v8.setFlagsFromString('--trace_fragmentation_verbose');
v8.setFlagsFromString('--expose_gc');  

sharp.cache(false);
sharp.concurrency(1);

module.exports.handler = (event) => new Promise((resolve, reject) => {
    let buffer;
    const busboy = new Busboy({
        headers: {
            'content-type': event.headers['content-type'] || event.headers['Content-Type'],
        }
    });

    busboy
    .on('file', (fieldname, file, filename, encoding, mimetype) => {
        log(fieldname, mimetype, encoding, filename);
        file.on('data', (data) => buffer = data);
    })
    .on('error', (err) => reject(`error: ${err}`))
    .on('finish', () => {
        sharp(buffer)
        .sequentialRead(true)
        //.resize({width: 500, height: 500})   //this operation can be anything - toFormat(), webp() etc.
        .toFormat("webp", {force: true})       //this operation can be anything - resize(), webp() etc.
        .toBuffer()
        .then(() => {
            log("=> before gc memoryUsage", process.memoryUsage());
            global.gc();
            log("=> after gc memoryUsage ", process.memoryUsage());

            const headers = {
                'content-type': "text/plain; charset=us-ascii"
            };
            resolve({statusCode: 200, body: "done", isBase64Encoded: false, headers});
        })
        .catch((err) => {
            log(err);
            reject(`error: ${err}`);
        });
    });

    busboy.write(Buffer.from(event.body, event.isBase64Encoded ? 'base64' : 'utf8'));
    busboy.end();
});

Now, i could see these memory related question in the forum and i tried to incorporate all the suggestions i could find but without any luck

things tried

process.env.VIPS_DISC_THRESHOLD = '20m';
v8.setFlagsFromString('--max_old_space_size=50');
v8.setFlagsFromString('--max_new_space_size=50');  //this is not supported any more
sharp.cache(false);
sharp.concurrency(1);
sequentialRead(true)

also enabled gc - v8.setFlagsFromString(‘–expose_gc’); - and started running GC just before i return from the handler.

I am printing memoryUsage before and after the GC call on every function invokation. the heap usage is almost constant as expected - only RSS keep on increasing and finally “Max Memory Used” hits the root.

i tried with different “MemorySize” starting from 128M to 512M (even though my sample app is very small, i tried to increase the MemorySize to prove my point)

Few sample logs below in the start the Log is something like below - Max Memory Used: 162 MB

imageFile image/jpeg 7bit 00 (3rd copy).JPG before gc memoryUsage { rss: 82427904, heapTotal: 22519808, heapUsed: 13582184, external: 1467043 } compaction-selection: space=OLD_SPACE reduce_memory=0 pages=0 total_live_bytes=70 space=CODE_SPACE reduce_memory=0 pages=0 total_live_bytes=0 2 pages, 580344 (56.3%) free after gc memoryUsage { rss: 81100800, heapTotal: 20082688, heapUsed: 9323088, external: 1467043 } Duration: 3788.14 ms Billed Duration: 3800 ms Memory Size: 512 MB Max Memory Used: 162 MB

After few itertaion the Log is Max Memory Used: 369 MB

imageFile image/jpeg 7bit ali-al-mufti-365944-unsplash.jpg before gc memoryUsage { rss: 195997696, heapTotal: 16015360, heapUsed: 13140688, external: 3533290 } compaction-selection: space=OLD_SPACE reduce_memory=0 pages=0 total_live_bytes=21 compaction-selection: space=CODE_SPACE reduce_memory=0 pages=0 total_live_bytes=0 2 pages, 680840 (66.0%) free after gc memoryUsage { rss: 191299584, heapTotal: 11300864, heapUsed: 8301824, external: 3533258 } Billed Duration: 4100 ms Memory Size: 512 MB Max Memory Used: 369 MB

Towards the end before memory running out - Max Memory Used: 464 MB

compaction-selection: space=OLD_SPACE reduce_memory=0 pages=0 total_live_bytes=0 compaction-selection: space=CODE_SPACE reduce_memory=0 pages=0 total_live_bytes=0 [MAP_SPACE]: 2 pages, 692896 (67.2%) free imageFile image/jpeg 7bit florian-klauer-489-unsplash.jpg before gc memoryUsage { rss: 243482624, heapTotal: 23261184, heapUsed: 19224656, external: 5971426 } compaction-selection: space=OLD_SPACE reduce_memory=0 pages=0 total_live_bytes=0 compaction-selection: space=CODE_SPACE reduce_memory=0 pages=0 total_live_bytes=0 2 pages, 686384 (66.6%) free after gc memoryUsage { rss: 237264896, heapTotal: 15994880, heapUsed: 12989440, external: 4670102 } Billed Duration: 19900 ms Memory Size: 512 MB Max Memory Used: 464 MB

Thanks