EmbedAnything: Can't run on iOS with tauri - error: failed to run custom build command for `ort-sys v2.0.0-rc.9`

Describe the bug EmbedAnything runs on MacOS with tauri but doesn’t on iOS. Instead it throws an error: ort-sys v2.0.0-rc.9

To Reproduce See this example repo with all instructions and errors: https://github.com/do-me/tauri_test_embed_anything

Desktop (please complete the following information): Works on MacOS Sequoia.

Smartphone (please complete the following information): Does not work on iOS iPhone 15 Pro, 16.1 through xcode.

Seems like it might be related to a candle bug. See also this conversation on tauri channel: https://discord.com/channels/616186924390023171/1329833396758908928/1329833396758908928

As well as this issue for candle directly: https://github.com/huggingface/candle/issues/1841

Related: https://github.com/tauri-apps/tauri/discussions/12370

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 17

Most upvoted comments

Currently, there is not option to add path. But what you can do is set the following environment variable to where you want it to download

HF_HOME

https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables

Let me know if this solution works for the short term. I will have to figure out how to allow a path to be set through the builder.

I have allowed passing a token. To avoid passing too many arguments to the function for scalability in the future, I have moved the Embedder creation to a builder pattern.

You can now use it as follows:

let model = Arc::new(EmbedderBuilder::new()
        .model_architecture("jina")
        .model_id(Some("jinaai/jina-embeddings-v2-small-en"))
        .revision(None)
        .token(None)
        .from_pretrained_hf()
        .unwrap());

I hope this is more convenient. The changes are in the same make-features branch. Let me know how it goes.

No not at all. On MacOS it works perfectly fine as well, that’s the weird thing.

  let embedder = Arc::new(
      Embedder::from_pretrained_hf(
          "bert",
          "sentence-transformers/all-MiniLM-L6-v2",
          None,
      )
      .map_err(|e| e.to_string())?,
  );

Related: https://github.com/huggingface/hf-hub/issues/54

I guess I got this specific problem solved by just creating the token file before accessing the HF hub. However, running into a new problem now:

Error: 1/0 error Operation not permitted (os error 1)

Image

Not quite sure where this problem stems from. I have a feeling that it’s some nasty permission/security bug that has something to do with tauri/xcode but is not necessarily related to embedanything. Here is the test repo: https://github.com/do-me/tauri-embedanything-ios

Created an issue here: https://github.com/tauri-apps/tauri/issues/12497

@akshayballal95 bingo! No more compilation error, awesome! 🥳 I had to fix the gemm-f16 problem (since mobile devices apparently do not support it) with this workflow. Need to fix a few other things in my frontend setup (considering cargo tauri dev / ios dev setup) to actually confirm it’s working properly. Will report back soon.

Exactly! Just make sure that you specify the dependency in your Cargo.toml file as follows:

embed_anything = {git = "https://github.com/StarlightSearch/EmbedAnything.git", branch = "make-features", default-features=false}

If this works out for you, I can merge it to main and make a release

Also on a side note, with my experience in integrating embed_anything in tauri, make sure that you move the embedding methods to a separate thread by doing tauri::async_runtime::spawn(...) to avoid the UI from getting locked up.

Hey. Unfortunately I haven’t been able to test with iOS. But if the issue is related to ort, one thing I can do is to move ort to a feature flag. That way we can isolate the bug to either onnx or candle. Let me do this and get back here. You can test it out then and let me know how it goes.