tauri: [bug] Loading a video/audio as an assets does not work.

For the people who are on Linux, images can be loaded through asset protocol (https://github.com/tauri-apps/tauri/discussions/1438) but not for videos and audio due to webkit lib issue (see https://bugs.webkit.org/show_bug.cgi?id=146351, last email). Someone on tauri discord found a workaround for audio file (this maybe applicable to other type of file), you just have to load the file with fs and create a Blob which you create an DataURL with it

Example :

  import { readBinaryFile } from "@tauri-apps/api/fs"
  const [urlAudio, setUrlAudio] = useState<string>("")
  return (
    <>
      <Button
        variant="ghost"
        onClick={() => {
          const filePath = .....
          void readBinaryFile(filePath)
            .catch((err) => {
              console.error(err)
            })
            .then((res) => {
              const fileBlob = new Blob([res as ArrayBuffer], { type: "audio/mpeg" })
              const reader = new FileReader()
              reader.readAsDataURL(fileBlob)
              const url = URL.createObjectURL(fileBlob)
              setUrlAudio(url)
            })
        }}
        className="w-full justify-start font-normal"
      >
        ok
      </Button>
      <audio controls>
        <source src={urlAudio} type="audio/mpeg" />
      </audio>
    </>
  )

You just need to setup tauri.conf.json permission for letting fs reading files with "scope": ["**"]

EDIT : you could do even better with a simple fetch, you will have to add linux-protocol-headers to the features of tauri your cargo.toml file

...
tauri = { version = "1.4", features = [...., "linux-protocol-headers"] }
....

And the code simplifies to

          const filePath = "/home/yanovskyy/Musique/Shinigami.m4a"
          fetch(convertFileSrc(filePath))
            .then((res) => {
              // create blob url from response
              res
                .blob()
                .then((blob) => {
                  const url = URL.createObjectURL(blob)
                  setUrlAudio(url)
                })
                .catch((err) => {
                  console.error(err)
                })
            })
            .catch((err) => {
              console.error(err)
            })

This, I think, is the best workaround for small video/audio file this moment because readBinaryFile slow down the process and there is memory problem. If you have big files (up to GB), well rip …

There are 2 issues mixed up here:

1. videos not loading in general, including remote videos (for example from youtube) -> The fix is to install gstreamer
2. local videos not loading via convertFileSrc -> No fix available right now (gstreamer would still be necessary)

I can reproduce this issue, seems to be a webkit2gtk problem.

@Siirko yes. you can try to read the thread and find references from commits from other repos to this issue

This config works for me.

    "allowlist": {
      ...
      "protocol": {
        "asset": true,
        "assetScope": ["**"]
      }
    },
    "security": {
      "csp": "default-src 'self'; img-src 'self' asset: https://asset.localhost; media-src 'self' asset: https://asset.localhost"
    }

Edit:

Don’t do this. It’s dangerous: https://github.com/tauri-apps/tauri/issues/3735#issuecomment-1072978250

This is quite annoying I hope webkit lib solves it soon, although would still be problematic for users who haven’t updated the lib

webkitgtk still has issues to load video from custom URL scheme. I also have a reproducible example in pure C. https://github.com/wusyong/gtkbrowser/tree/video But I couldn’t track the upstream issue. I’ll find some time to open one again (or find the one I commented before.)

I got an issue where i need to load image and video inside tauri as an asset where the file is located in randomly path, but i solve that issue by copying the file inside tauri PATH, in my case i use appDataDir and it work just fine

My example app:

I use react-player in this example and work fine

My code

I have same problem but with images on MacOS This code fix this issue "protocol": { "asset": true }, "security": { "csp": "default-src: 'self'; media-src 'self' asset:" },

Commenting the function name so others can find this issue when they search for convertFileSrc.

That codebase is shared between apple, webkitgtk, and wpe (and more?). So having a gstreamer folder doesn’t mean much for the macos port, especially if we take a look at what other folders are in that mediastream folder.

I can load an image asset but not for audio/video asset even with correct configuration.

"protocol": {
        "all": false,
        "asset": true,
} // leave the assestScope here because I load audio file using tauri file picker

"security": {
      "csp": "default-src 'self'; media-src 'self' asset: https://asset.localhost"
},

Don’t mind our project boards yet, we’re just playing around with them, trying to find something that works for us. In the latest iteration they only contain PRs which is why this issue was removed.

Looks like the issue still remains and I tried my best to write a C example to report back: https://bugs.webkit.org/show_bug.cgi?id=146351#c5

Hi all, I don’t know if it helps, I have just tested a basic video sample using the asset protocol. With a regular up-to-date ubuntu release, using webkitgtk2.36.6. Videos are still not playing on my station. Hopefully the next webkitgtk releases will have it fixed 😃

Loading audio with the asset protocol is also broken

It’s weird 'cause I was sure it worked on previous webkit2gtk releases 😦

Is https://asset.localhost/<filepath> supposed to work also? I get “Connection refused” with that but nothing at all with asset://.

asset:// is Linux and macOS and https://asset.localhost/ is Windows.

Is there a workaround that we can use? Or we just wait for a proper fix?

Ah whoops, didn’t see that you were 2 different people, sorry.