playwright: [BUG] File uploads fail which are > 50 MB

I’m trying to prioritize this, could you share what kind of testing scenario requires uploading 100Mb+ files?

hey, any updates?

Seeing the same issue for files ranging from 90MB to 320MB. Any workarounds?

any updates?

Hello @pavelfeldman. We would like to use it for end to end testing of our in vitro diagnostic medical application. The crash appear during upload/download genomic and other medical files (170mo to 5go and bigger files).

Large file uploads also seem to be slow. (30 seconds for 70 megabytes). I think it might be caused by this byte-by-byte mapping of the content of the target file / buffer.

https://github.com/microsoft/playwright/blob/06ab3c0fda2223ef6fd813c48373208b1b2c01b7/packages/playwright-core/src/server/injected/injectedScript.ts#L660

In my case, I used playwright to upload video files to video hosting under test.

I’m trying to prioritize this, could you share what kind of testing scenario requires uploading 100Mb+ files?

The website (AI training purposes) allowed user to upload video files. In machine learning model, each video file minimally exceed 2GB.

This is what I get when running with DEBUG=pw:browser*:

pw:browser [err] [0914/103304.384699:ERROR:http_connection.cc(37)] Too large read data is pending: capacity=104857600, max_buffer_size=104857600, read=104857600 +2s
pw:browser [err] [0914/103304.384745:ERROR:devtools_pipe_handler.cc(278)] Connection closed, not enough capacity +0ms

Here is the Chromium max buffer size.

Hello! I am still unable to upload files (500 mb and 2gb files) as the test would hang. It only works for small files.

Context:

Playwright Version: 1.25.2 Operating System: Mac OS 12.5.1 Python Version: 3.7.9 Browser: Chromium

Hello @mxschmitt @pavelfeldman

Are there still blockers for this issue? Do you need additional information?

Cheers

@mxschmitt I see you assigned the issue to yourself and even added a label for a fix version, but removed both later on. Are there any plans to get this issue fixed in the near future?

@pavelfeldman In my case, I’m uploading some files which are 200 MB (firmware update and some patches inside). Currently this issue is blocking the Jenkins pipeline since all tests are failing. Any workaround?

In my case, I am testing a vmware-like web app, which requires user to upload OS .iso file / VM .ova file which is usually larger than 1GB.

Maybe a begin of a workaround … (@joel-s) Increasing cache memory size of the browser at launching with parameters regarding to @dgozman browser error. I saw here how to increase cache memory depending of browser used and playwrith api to set config at browser launching

Testing with Chrome It should be something like this :

browser = await chromium.launch({
    args: ['--disk-cache-size=2073741824'], // in bytes = 2Gb
    headless : false,
});

Chrome is nicely launched with this param and others which seems to be playwright’s defaults args : BUT Cache still little… 😭

Testing with Firefox It should be something like this :

  browser = await firefox.launch({
     headless: false,
     firefoxUserPrefs: {
       'browser.cache.disk.smart_size.enabled': false,
       'browser.cache.disk.capacity' : 2048576, //2gb
     }
   });

But still failing as shawn in my previous post whereas when adding a page.waitForTimeout(xxxx) I can manualy check the about:config and params are goodly set as I wanted… so maybe not a “cache-problem” with firefox…

I’m loosing faith… Have you got any idea ? Am I missing something ?

For me, uploading a 320MB file crashes on all 3 browsers. The browser disappears and the test hangs. The same thing works without problems on all 3 browsers with a smaller file.

Can anyone recommend a workaround for this (in any browser, but Chromium would be best), considering we were planning to use Playwright to test a Web UI where a typical upload file size is 2-3GB? The 320MB file was created as a way to speed up some tests.

/**
 * @fileOverview Sample to demonstrate crash when file size is large.
 */
const LARGE_TIMEOUT = 2 * 60 * 1000;

describe("Upload page", () => {
  it("should receive an upload", async () => {
    await page.goto("https://uppy.io/examples/xhrupload/");

    page.setDefaultTimeout(LARGE_TIMEOUT);

    await page.setInputFiles("input[type='file']", "/some/320mb/file", {
      timeout: LARGE_TIMEOUT,
    });
  });
});

This is on MacOS 10.14.6 Mojave, Playwright 1.9.2

Exactly same problem here. I had troubles with node memory when I uploaded multiples small files one after an other so I added export NODE_OPTIONS=--max_old_space_size=4096 to force memory of node to 4gb.

Then I realised that playwright didn’t throw unused memory in a scenario If you don’t manually reset the context of the browser. ( happened for me also by uploading many small files )

But now, I still have a problem when uploading a simple file of more than 300Mo… This is what I got as error (nothing in pw:browser) :

What I find curious is that I used Firefox as testing browser and it talks about chrome…

Ubuntu 20.04 LTS, playwright 1.7.0