opendal: Tracking issues of OpenDAL wasm32-unknown-unknown target support

Can we be used on wasmtime?

Can we be used on wasmedge?

For now, we can’t use it on both wasm runtimes, because both of the runtimes is based on wasm32-wasi I have reminded it in the previous comment.

The WASI runtime is not support for the wasm-bindgen which is dependent by reqwest now.

If we want to support the wasi runtime, we may need the other HTTP client which is compatible with the WASI spec

@suyanhanx it seems surprising that ring would be failing in that way: the issue you linked seemed to say that it should be able to build for wasm32-unknown-unknown: briansmith/ring#657 (comment)

the author (@briansmith) does note that some functionality isn’t available yet, but is open to extending support. do we know what is being used that isn’t supported?

Thank you for pointing that out! I’ve tried to compile ring before but it kept failing. Finally, I realized that the llvm comes with macos doesn’t have a backend of wasm. I reinstalled llvm and it compiled successfully. Sorry for the confusion!

For now, we can’t use it on both wasm runtimes, because both of the runtimes is based on wasm32-wasi I have reminded it in the previous comment.

Thanks, let’s remove them. I just found that support wasm32-unknown-unkown is totally different from support wasm32-wasi. Lesson learnt.

web-sys

After a quick search, it seems that web-sys has a clear mapping to js web api. However I think the most dirty work comes to the data crossing boundaries: wasm (prepare the net calls) -> js (fetch hook) -> wasm (reading the result)

Let’s wait for the stable wasm releases first… LOL

Furthermore, my intention is to use opendal in my 320k downloaded project (remotely-save if you are interested). If you want to talk about the user case, you are welcome to DM me (username fyears) on discord .

Hi @fyears, thank you for your interest! As @Young-Flash mentioned, our WebAssembly support is still in the early stages and not yet ready for practical use.

• any doc to use the wasm version? by importing any library in js/typescript? any npm package?

Not yet. WASM support is still in active development, and we need to address existing bugs such as https://github.com/apache/incubator-opendal/issues/3810 before proceeding.

We haven’t released it yet. (However, you can create a wasm library by running cargo add opendal and building it.)

• would you like to consider adding hooks for js fetch function? For example, I plan to use opendal in a special js environment and I have to use a special fetch function to get around the annoying cors limitatioms. Moreover, users might need to intercept/monitor/debug some network connections .

We are using web-sys now, and I’m not sure if web-sys support this use case.

• would you like to consider adding hooks for js fs functiom? For example, in browser environment, file system might need some mocking by using indexeddb.

We will not support the fs service on wasm, which means there’s no need to provide hooks either. We have a plan to support opfs: https://github.com/apache/incubator-opendal/issues/2442. If users require IndexedDB, we can offer native support for it.

• in js world, there is a technique called “tree shaking” (by importing part of the package). For example developers only interested in s3 might only import opendal/s3 without importing others. It’s especially useful to support this because users have to download the assets in browser environment. Is it possible for opendal to support tree shaking?

Nice question. We want this as well, but I’m not sure how to implement it. We can re-visit this after we have build a usable opendal wasm lib.

it seems surprising that ring would be failing in that way

Thanks for raising this up! I re-visit the gcs support on wasm, and confirmed that gcs is able to compile on wasm32-unknown-unknown. I will start a PR to address this.

For the record,

wasm-pack test --chrome --headless and wasm-pack test --chrome is both working.

But need some special S3 settings like

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "POST",
            "PUT",
            "HEAD"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]