wasm-bindgen: Allow returning Vec

This issue is kind of a blocker for pretty much any sort of a bigger project using rustwasm.

For those looking for a workaround on this, if you can turn your data into a Vec<u8> or &[u8]

#[wasm_bindgen]
pub struct ByteStream {
    offset: *const u8,
    size: usize,
}

#[wasm_bindgen]
impl ByteStream {
    pub fn new(bytes: &[u8]) -> ByteStream {
        ByteStream {
            offset: bytes.as_ptr(),
            size: bytes.len(),
        }
    }

    pub fn offset(&self) -> *const u8 {
        self.offset
    }

    pub fn size(&self) -> usize {
        self.size
    }
}

A good example of how to use this is creating a texture in Rust to render in Javascript, so for example:

#[wasm_bindgen]
pub fn render() -> ByteStream {
  let texture = Vec::new();
  // ...
  ByteStream::new(&texture);
}

const texture = render();
const textureRaw = new Uint8ClampedArray(memory.buffer, texture.offset(), texture.size());
const image = new ImageData(textureRaw, width, height);

It would still be awesome to have builtin support for something like this which is quite normal case:

#[wasm_bindgen]
pub struct IntersectResult {
    pub hit: bool,
    pub triangle_index: u32,
    pub u: f32,
    pub v: f32,
    pub distance: f32,
}

#[wasm_bindgen]
pub struct IntersectResultArray {
    pub intersects: Vec<IntersectResult>,
}

@alexcrichton Exactly, that’s precisely my usecase.

It’s not a complete solution, but I created #1749 which adds in FromIterator for Array:

use js_sys::Array;

#[wasm_bindgen]
pub fn token_ranges(text: &str) -> Array {
    get_vec_somehow().into_iter().collect()
}

This means that now you can send Vec<T> to JS, you just have to return an Array and use .into_iter().collect() to convert the Vec<T> into an Array.

I’m with a similar issue here, but with a complex struct.

code:

#[derive(Clone)]
#[wasm_bindgen]
pub struct Coords {
    x: usize,
    y: usize
}

#[wasm_bindgen]
pub struct Cell {
    state: State,
    position: Coords,
    neighboors: Vec<Coords>,
    neighboors_alive: i32
}

#[wasm_bindgen]
impl Cell {
    pub fn new(state: State, position: Coords, neighboors: Vec<Coords>) -> Cell {
        Cell {
            state,
            position,
            neighboors,
            neighboors_alive: 0
        }
    }
}

error:

error[E0277]: the trait bound `std::boxed::Box<[Coords]>: wasm_bindgen::convert::FromWasmAbi` is not satisfied
  --> src\lib.rs:36:1
   |
36 | #[wasm_bindgen]
   | ^^^^^^^^^^^^^^^ the trait `wasm_bindgen::convert::FromWasmAbi` is not implemented for `std::boxed::Box<[Coords]>`
   |
   = help: the following implementations were found:
             <std::boxed::Box<[u16]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[wasm_bindgen::JsValue]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[u8]> as wasm_bindgen::convert::FromWasmAbi>
             <std::boxed::Box<[f32]> as wasm_bindgen::convert::FromWasmAbi>
           and 5 others
   = note: required because of the requirements on the impl of `wasm_bindgen::convert::FromWasmAbi` for `std::vec::Vec<Coords>`

@rookboom @Hywan do y’all basically need Vec<T> where T has #[wasm_bindgen] on it?

@dragly As explained in the PR, you need to use .map(JsValue::from), like this:

objects.into_iter().map(JsValue::from).collect()

This is because structs are a Rust data type, and so you have to manually use JsValue::from to convert them into a JS data type (the same is true for other Rust data types like &str, i32, etc.).

If anyone is still looking at this, I was able to work around this using Serde to serialize/deserialize the data. This was the guide I used: https://rustwasm.github.io/docs/wasm-bindgen/reference/arbitrary-data-with-serde.html

Edit: For those wanting to avoid JSON serialization, the guide above also includes a link to serde-wasm-bindgen which “leverages direct APIs for JavaScript value manipulation instead of passing data in a JSON format.”

My workaround is to pass JSON over the wasm boundary… Not ideal but works for now.

Hi guys, great to see a PR has been merged. Is there any ETA for this feature ? Or is it possible for you to trigger a new release, so we can have this feature using cargo add.

Thanks

The docs seem to incorrectly indicate that this is already supported: https://rustwasm.github.io/wasm-bindgen/reference/types/boxed-jsvalue-slice.html

Boxed slices of imported JS types and exported Rust types are also supported. Vec<T> is supported wherever Box<[T]> is.

@zimond Yes, sorry my bad.

@Pauan @alexcrichton Do you guys have a simple example which we could use as a guide to implement a function that returns Vec<Struct> ?

@Kinrany Yes, but it requires a bit of a hack:

#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen(typescript_type = "Array<string>")]
    type MyArray;
}

(The typescript_type attribute can specify any TypeScript type, even complex types like |)

Now you just do fn foo() -> MyArray and use .unchecked_into::<MyArray>() to cast the Array into MyArray.

Great work @Liamolucko! I look forward to using this once released.

@bushrat011899

I have had no response from @chinedufn @alexcrichton regarding the failing test. I spent a lot of time on this PR and this is discouraging to say the least. I wonder if they just never look at PRs unless they pass all tests. In which case, I could assume an interpretation of the comment in the test, and get it to pass. That way I’d get their attention, and if I assumed the wrong interpretation they will tell me to change it. However the PR is quite old now and probably has merge conflicts with master to sort out. If I’m going to do more work on this I want some assurance that I’m not wasting my time.

@bushrat011899

I have had no response from @chinedufn @alexcrichton regarding the failing test. I spent a lot of time on this PR and this is discouraging to say the least. I wonder if they just never look at PRs unless they pass all tests. In which case, I could assume an interpretation of the comment in the test, and get it to pass. That way I’d get their attention, and if I assumed the wrong interpretation they will tell me to change it. However the PR is quite old now and probably has merge conflicts with master to sort out. If I’m going to do more work on this I want some assurance that I’m not wasting my time.

That really is unfortunate, this seems like a really obvious feature to work on in my opinion. My first thought for something useful with WASM was to write a REST API client. That way, I could have type guarantees and unit tests that, for example, my Axum server was interacting with a web UI correctly. Kinda hard to do that if something simple like “give me a list of data” requires type erasure.

I was able to get this solution working for me, as a disclaimer, I just hacked it together and it could be greatly improved but I wanted to get something working before i attempted to clean it up into macros.

Cargo.toml

[dependencies]
serde = { version = "1", features = ["derive"] }
wasm-bindgen = {version= "0", features=["serde-serialize"]}
js-sys = "0.3.55"

lib.rs

// Struct Code
#[wasm_bindgen]
#[derive(Serialize, Deserialize)]
struct User {
    id: String,
    name: String,
    age: i64
}

impl From<std::option::Option<query_user::QueryUserQueryUser>> for User {
    fn from(user: Option<query_user::QueryUserQueryUser>) -> Self {
        let data = user.as_ref().expect("user");
        User {
            id: String::from(&data.id),
            name: String::from(data.name.as_ref().expect("name")),
            age: data.age.expect("age")
        }
    }
}

// Excluded the unrelated code needed here
#[wasm_bindgen]
pub async fn get_user() -> js_sys::Array {
  // Code that fires off a request and sets response
    log(&format!("response body\n\n{:?}", response));
    response.data
        .expect_throw("response data")
        // Vec<std::option::Option<query_user::QueryUserQueryUser>>
        .query_user
        .expect_throw("query_user")
        // Convert it into an iterator
        .into_iter()
        // Map it to the struct
        .map(User::from)
        // Map that struct to js value
        .map( { |user|
            JsValue::from_serde(&user).unwrap()
        })
        // Collect it into a js_sys::Array
        .collect::<js_sys::Array>()
}

I believe latest wasm-bindgen release already allows return Vec<T> where T is any type implementing WasmDescribe + JsCast. Currently available types includes primitives, and types from js-sys. So returning Vec<JsValue> is fine now.

Edit: #[wasm_bindgen] do not generate the trait impls. So there’s more work to be done

Unfortunately, solutions with js_sys have huge performance overheads. You can also return Box<[T]> with T being basic numeric type which is efficient on the JS side (it just slices relevant wasm memory). This however needs a copy on the Rust side. In general, it is faster than js_sys though.

What I come up with (and seems to be faster by a margin) is to have the following wrapper type:

pub struct MySlice<T> {
    phantom: std::marker::PhantomData::<T>,
    _ptr: u32,
    _len: u32,
}

impl<T: wasm_bindgen::describe::WasmDescribe> wasm_bindgen::describe::WasmDescribe for MySlice<T> {
    fn describe() {
        wasm_bindgen::describe::inform(wasm_bindgen::describe::REF);
        wasm_bindgen::describe::inform(wasm_bindgen::describe::SLICE);
        T::describe();
    }
}

impl<T: wasm_bindgen::describe::WasmDescribe> wasm_bindgen::convert::IntoWasmAbi for MySlice<T> {
    type Abi=wasm_bindgen::convert::WasmSlice;

    #[inline]
    fn into_abi(self) -> wasm_bindgen::convert::WasmSlice {
        wasm_bindgen::convert::WasmSlice {
            ptr: self._ptr,
            len: self._len,
        }
    }
}

impl<T> std::convert::From<&Vec<T>> for MySlice<T> {
    fn from(vec: &Vec<T>) -> Self {
        let _ptr = vec.as_ptr() as u32;
        let _len = vec.len() as u32;
        
        Self {
            phantom: std::marker::PhantomData,
            _ptr,
            _len
        }
    }
}

This is both zero-copy on the rust side and single copy between wasm memory and JS on the JS side. Basically, the type just emulates ref to a slice through standard wasm bindgen API (which means support for returning &[T] should be relatively easy to add if somebody knows internals of bindgen) A caveat is that you shouldn’t save MySlice, it should only be used to return immediate data to JS (because it detaches pointers, the code cannot guarantee lifetimes)

Usage is simple:

#[wasm_bindgen]
pub struct Container {
    data: Vec<f64>,
}

#[wasm_bindgen]
impl Container {
    pub fn read(&self) -> MySlice<f64> {
        (&self.data).into()
    }
}

with the JS side using

const c = Container.new() // somehow create
const values = c.read()

@9oelM That’s because you explicitly declared that decoded_bytes is &Vec<u8>. There’s another API Vec::as_slice which could give you a &[u8] from Vec<u8>. So you could use decoded_bytes.as_slice().into(), or Uint8Array::from(decoded_bytes.as_slice()).

More learning materials: Deref coercions, Type conversion , Lifetime ellision (which is why &'a [u8] and &[u8] are the same)

Rather than returning pointers and lengths manually, you can use this, which should be slightly less error prone: https://docs.rs/js-sys/0.3.9/js_sys/struct.Uint8Array.html#method.view

Gabriel and rook - have y’all found a workaround? Solving or working around this would add much flexibility to WASM in Rust.

Eventually, being able to use HashMaps, or structs from other packages (Like ndarrays) would be nice, but having some type of collection that maps to JS arrays would be wonderful; not sure if I can continue my project without this.

@9oelM there is From<&[u8]> implemented for Uint8Array, so basically you could just return decoded_bytes.into()

@i-schuetz : the workaround that I use is having a return type of Vec<JsValue> and converting your vector to it on return: myvec.iter().map(JsValue::from).collect().