ort: Support for `wasm32-unknown-unknown`

What

ONNXRuntime supports wasm targets via it’s onnxruntime for web bindings.

According to this page you can build a static lib of onnxruntime for wasm targets which you can then bundle with your C++ WebAssembly. project.

From what I understand, the wasm lib provides the same C API that regular onnxruntime provide.

Why

Supporting wasm targets will make it easy to deploy models built with ort to web browsers and wasm runtimes such as WasmTime and Wasmer.

Since wasm-runtimes are cross-platform, users can bundle their models along with pre and post processing code in a single, universal, executable module that they can then run on browsers or on any wasm-runtime.

About this issue

Original URL
State: closed
Created a year ago
Reactions: 4
Comments: 22 (11 by maintainers)

Commits related to this issue

feat: support for WASM targets, closes #75 — committed to pykeio/ort by decahedron1 9 months ago

Most upvoted comments

wasm32-unknown-emscripten is now supported in v1.15.5.

decahedron1 on Sep 29, 2023

Seeing how requested WASM support is, I’ll keep working on solutions. It may be possible to shim the needed Emscripten APIs but that’s a lot of work and I can’t guarantee it’ll be successful.

decahedron1 on Apr 4, 2024

@k2m5t2 Got a repro, the issue is rustc stable. It compiles fine with nightly, still trying to figure out what commit fixed it.

decahedron1 on Apr 20, 2024

Please give it a try:

[dependencies]
ort = { git = "https://github.com/pykeio/ort.git", branch = "wasm32-unknown-unknown" }

A few notes:

You must call ort::wasm::initialize() at the very top of your program, before you use any ort APIs.
The filesystem is disabled, so no profiling, and you must load models from memory with SessionBuilder::commit_from_memory_directly.
You must use .ort format models - see here for how to convert. TL;DR python -m onnxruntime.tools.convert_onnx_models_to_ort ./my_model.onnx
Runtime optimization is disabled, so do not call SessionBuilder::with_optimization_level.

decahedron1 on Apr 13, 2024

I was actually in the middle of writing some docs about this very issue 😂

From what I can tell, compiling to wasm32-unknown-emscripten is very tricky. I haven’t been able to make it work either. I’d have to recommend alternative libraries that do support plain old wasm32-unknown-unknown like tract or wonnx for the time being.

decahedron1 on Apr 4, 2024