onnxruntime: [Web] chrome V3 extension `TypeError: URL.createObjectURL is not a function`

Describe the issue

We are using onnxruntime-web in a Chrome extension.

We are in the process of migrating to Chrome V3 format (worked great in V2) and hit the error:

TypeError: URL.createObjectURL is not a function
    at Object.locateFile (packed.js:formatted:18165:126)
    at O (packed.js:formatted:345:61)
    at Object.ib (packed.js:formatted:718:49)
    at Object.mb (packed.js:formatted:722:73)
    at fe (packed.js:formatted:594:48)
    at Te (packed.js:formatted:807:42)
    at ort-wasm-simd-threaded.wasm:0x772d56
    at ort-wasm-simd-threaded.wasm:0xeb64e
    at ort-wasm-simd-threaded.wasm:0x1c4048
    at ort-wasm-simd-threaded.wasm:0xe5575

URL.createObjectURL does not appear to be available to Chrome V3 extensions.

Is there a different way for us to use onnxruntime and avoid this API? Thanks!

To reproduce

You could install our extension: release.zip

Editing a multi-line text box (like GitHub) hits the error, and you can view from the extensions page:

Source code here: https://github.com/jonathanpeppers/inclusive-code-reviews-browser/pull/90

Urgency

Medium. Chrome V2 extensions are being phased out this year, and this seems to be a blocker.

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.13.1

Execution Provider

WASM

About this issue

Original URL
State: open
Created a year ago
Reactions: 2
Comments: 16 (7 by maintainers)

Commits related to this issue

Set `ort.env.wasm.numThreads = 1` Context: https://github.com/microsoft/onnxruntime/issues/14445 When moving to Chrome V3, we hit the error: TypeError: URL.createObjectURL is not a function ... — committed to jonathanpeppers/inclusive-code-reviews-browser by jonathanpeppers a year ago
Set `ort.env.wasm.numThreads = 1` (#195) Context: https://github.com/microsoft/onnxruntime/issues/14445 When moving to Chrome V3, we hit the error: TypeError: URL.createObjectURL is not a f... — committed to jonathanpeppers/inclusive-code-reviews-browser by jonathanpeppers a year ago

Most upvoted comments

I can look at it - might be a little complicated to fix.

guschmue on Jul 14, 2023

Indeed this is blocking the efficient usage of the very popular transformers.js in the browser for local and private ML. @fs-eire @guschmue

MentalGear on Nov 2, 2023

The error does not happen in loading the model, it happens when initializing the web assembly.

The reason why ort.env.wasm.numThreads=1 works is because the function URL.createObjectURL() will only be called when initializing multi-threaded version web assembly. The function is used to create a blob for ‘ort-wasm-threaded.worker.js’. So setting to load the single-threaded version is a workaround.

https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/wasm-factory.ts#L124-L128

fs-eire on Feb 2, 2023

I would love to see the update as well. I wanted to work with transformers.js to run LLMs in the browser but this issue makes it very inefficient due to lack of multithreading support.

Dolidodzik on Nov 12, 2023

An update to allow multi-threading would be really beneficial !

MentalGear on Oct 13, 2023

Would love to get an update on this. At the moment the the node library is pretty much unusable for me due to lacking support for onnxruntime-extensions (probably should be a separate bug report). But with this fixed we can just use rely on the web library for both web and node usage (we can now already but it would be great if threading worked properly)

ianmarmour on Oct 29, 2023

thank you for the feedback. it looks like URL.createObjectURL() cannot be used in a service worker. we are trying to find a way to fix this.

fs-eire on Jan 27, 2023

@fs-eire @guschmue any updates on this? 😇 I’ve been working on improving my chrome extension demo for transformers.js, and being able to get this working would be great!

xenova on Jul 14, 2023

However, running single threaded slows everything down. I’d love to work on this if anyone points me in the right direction 😃

lizozom on Jul 9, 2023

I got this error, specifically working with @xenova/transformers in a chrome extension. I had this code throwing the same error:

import { env, pipeline } from '@xenova/transformers';
env.localModelPath = 'models/';
env.allowRemoteModels = false;
this.embedder = await pipeline("embeddings", EMBEDDING_MODEL_NAME);

I added env.backends.onnx.wasm.numThreads = 1; and it resolved the error.

lizozom on Jul 7, 2023

@fs-eire - got it. It is this code here: https://github.com/microsoft/onnxruntime/blob/main/js/web/lib/wasm/wasm-factory.ts#L124. Aand it sounds you cannot use URL.createObjectURL with manifest v3 (ie https://groups.google.com/a/chromium.org/g/chromium-extensions/c/u0NH7L3v9L4). If I understand the code correctly, we do this to avoid distributing the js files for simd. We might want to find a different way of doing this.

@jonathanpeppers, the disadvantage for single threaded is only performance. Your model looked not very compute intensive so you might not notice.

guschmue on Feb 3, 2023

Thanks! It appears I can workaround with ort.env.wasm.numThreads = 1;, but I’m not sure exactly what the setting does:

https://github.com/jonathanpeppers/inclusive-code-reviews-browser/pull/195

Is there a doc/readme that explains what the 4 different *.wasm files do?

jonathanpeppers on Jan 27, 2023