tauri: [bug] Loading a video/audio as an assets does not work.

Describe the bug

Loading a video as an asset does not work.

I followed the guide here: https://tauri.studio/docs/api/js/modules/tauri

It works fine with image files (png/jpg etc.) but not with videos. I tried loading the video https://www.w3schools.com/html/mov_bbb.mp4 directly from the URL, and this worked without problems. But using the asset:///home/paul/Downloads/mov_bbb.mp4 URL this does not work Loading images from the same folder does work.

In the network panel you can see that the request failed but no further reason is specified image

image

The loading of assets this way is IMO not very good explained and when it fails, for some reason, it is very difficult to find out why.

Thank you for creating Tauri, it is awesome!

Reproduction

No response

Expected behavior

No response

Platform and versions

Operating System - Arch Linux, version Rolling Release X64

Node.js environment
  Node.js - 17.7.1
  @tauri-apps/cli - 1.0.0-rc.6
  @tauri-apps/api - 1.0.0-rc.2

Global packages
  npm - 8.5.4
  pnpm - Not installed
  yarn - 1.22.17

Rust environment
  rustup - 1.24.3
  rustc - 1.59.0
  cargo - 1.59.0
  toolchain - stable-x86_64-unknown-linux-gnu 

App directory structure
/dist
/node_modules
/src-tauri
/src
/.git
/public

App
  tauri - 1.0.0-rc.4
  tauri-build - 1.0.0-rc.4
  tao - 0.6.4
  wry - 0.13.3
  build-type - build
  CSP - default-src tauri: asset: https://asset.localhost
  distDir - ../dist
  devPath - http://localhost:3000/
  framework - React

Stack trace

No response

Additional context

No response

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 17
  • Comments: 69 (23 by maintainers)

Commits related to this issue

Most upvoted comments

For the people who are on Linux, images can be loaded through asset protocol (https://github.com/tauri-apps/tauri/discussions/1438) but not for videos and audio due to webkit lib issue (see https://bugs.webkit.org/show_bug.cgi?id=146351, last email). Someone on tauri discord found a workaround for audio file (this maybe applicable to other type of file), you just have to load the file with fs and create a Blob which you create an DataURL with it

Example :

  import { readBinaryFile } from "@tauri-apps/api/fs"
  const [urlAudio, setUrlAudio] = useState<string>("")
  return (
    <>
      <Button
        variant="ghost"
        onClick={() => {
          const filePath = .....
          void readBinaryFile(filePath)
            .catch((err) => {
              console.error(err)
            })
            .then((res) => {
              const fileBlob = new Blob([res as ArrayBuffer], { type: "audio/mpeg" })
              const reader = new FileReader()
              reader.readAsDataURL(fileBlob)
              const url = URL.createObjectURL(fileBlob)
              setUrlAudio(url)
            })
        }}
        className="w-full justify-start font-normal"
      >
        ok
      </Button>
      <audio controls>
        <source src={urlAudio} type="audio/mpeg" />
      </audio>
    </>
  )

You just need to setup tauri.conf.json permission for letting fs reading files with "scope": ["**"]

EDIT : you could do even better with a simple fetch, you will have to add linux-protocol-headers to the features of tauri your cargo.toml file

...
tauri = { version = "1.4", features = [...., "linux-protocol-headers"] }
....

And the code simplifies to

          const filePath = "/home/yanovskyy/Musique/Shinigami.m4a"
          fetch(convertFileSrc(filePath))
            .then((res) => {
              // create blob url from response
              res
                .blob()
                .then((blob) => {
                  const url = URL.createObjectURL(blob)
                  setUrlAudio(url)
                })
                .catch((err) => {
                  console.error(err)
                })
            })
            .catch((err) => {
              console.error(err)
            })

This, I think, is the best workaround for small video/audio file this moment because readBinaryFile slow down the process and there is memory problem. If you have big files (up to GB), well rip …

There are 2 issues mixed up here:

  1. videos not loading in general, including remote videos (for example from youtube) -> The fix is to install gstreamer
  2. local videos not loading via convertFileSrc -> No fix available right now (gstreamer would still be necessary)

I can reproduce this issue, seems to be a webkit2gtk problem.

@Siirko yes. you can try to read the thread and find references from commits from other repos to this issue

cursed way to fix this issue:

const blobUrl = URL.createObjectURL(await fetch(asset).then(res => res.blob()))
const sound = new Audio(blobUrl)

This is quite annoying I hope webkit lib solves it soon, although would still be problematic for users who haven’t updated the lib

I got an issue where i need to load image and video inside tauri as an asset where the file is located in randomly path, but i solve that issue by copying the file inside tauri PATH, in my case i use appDataDir and it work just fine

My example app:

Screenshot 2023-02-12 at 21 25 06

I use react-player in this example and work fine

My code

Screenshot 2023-02-12 at 21 30 07

Commenting the function name so others can find this issue when they search for convertFileSrc.

Is https://asset.localhost/<filepath> supposed to work also? I get “Connection refused” with that but nothing at all with asset://.

asset:// is Linux and macOS and https://asset.localhost/ is Windows.

This config works for me.

    "allowlist": {
      ...
      "protocol": {
        "asset": true,
        "assetScope": ["**"]
      }
    },
    "security": {
      "csp": "default-src 'self'; img-src 'self' asset: https://asset.localhost; media-src 'self' asset: https://asset.localhost"
    }


Edit:

Don’t do this. It’s dangerous: https://github.com/tauri-apps/tauri/issues/3735#issuecomment-1072978250

I’ve encountered this exact issue while building audio applications in Tauri. The asset:// protocol limitations on Linux (webkit2gtk) are particularly frustrating so I ended up creating a library specifically to solve this.

NPM: desktop-audio-proxy github: DAP

As mentioned in this issue, Tauri’s asset:// protocol has webkit2gtk limitations that prevent reliable audio/video streaming, especially on Linux. The CSP and protocol restrictions make it unnecessarily complex to handle media files.

How it’s used:

import { TauriAudioService } from 'desktop-audio-proxy/browser';

// Automatic setup - handles convertFileSrc internally
const audioService = new TauriAudioService({
  autoDetect: true
});

// Works with any local audio/video file
const streamUrl = await audioService.getStreamableUrl('/path/to/local/audio.mp3');
audioElement.src = streamUrl; // Now plays reliably across all platforms

How to Use DAP with Videos:


// Works with video URLs just like audio
const videoUrl = await client.getPlayableUrl('https://cors-restricted-site.com/video.mp4');
videoElement.src = videoUrl; // Now plays without CORS issues

Tauri Video Example:

import { TauriAudioService } from 'desktop-audio-proxy/browser';

const service = new TauriAudioService();

// Local video file
const localVideo = await service.getStreamableUrl('/path/to/local/video.mp4');
videoElement.src = localVideo;

// Remote video with CORS issues  
const remoteVideo = await service.getStreamableUrl('https://external-site.com/video.webm');
videoElement.src = remoteVideo;

Instead of fighting with protocol limitations, the library serves media files through a local HTTP proxy that webkit2gtk handles perfectly. It’s the same approach that works reliably in browsers, adapted for desktop applications. This has solved the asset loading issue for multiple of my Tauri projects dealing with audio/video streaming across different platforms

Still the same alternatives/workarounds as mentioned earlier in this issue:

  1. Load small files via fs functions as binary
  2. Large files require a http server on the rust side (or a sidecar in your language of choice i guess) so that the file will be loaded over http instead.

Because we do not want to add a http server into tauri itself (assuming nobody changed their minds), we’re stuck with this until we have an alternative for webkitgtk.

webkitgtk still has issues to load video from custom URL scheme. I also have a reproducible example in pure C. https://github.com/wusyong/gtkbrowser/tree/video But I couldn’t track the upstream issue. I’ll find some time to open one again (or find the one I commented before.)

I have same problem but with images on MacOS This code fix this issue "protocol": { "asset": true }, "security": { "csp": "default-src: 'self'; media-src 'self' asset:" },

Loading audio with the asset protocol is also broken

In the tauri docs is an example, how to load a video file via asset protocol. I tried this for audio without success.

Yeah it doesn’t because that’s exactly what this issue is about. And no, there were no updates and there very likely won’t be any until we drop webkitgtk.

I would have liked to see something similar to our existing localhost plugin that serves videos over http but that never happened and at this point probably never will.

Yeah that would be cool but only windows supports modifying arbitrary http requests/responses like that (and it’s also how the current custom protocol implementation works on windows). The current macOS and Linux webviews only support monitoring requests.

This is exactly how the default tauri and asset protocols work and the root cause of the issue here so yet another protocol with the same implementation would not help anyone.

Perhaps it could be worked around by Tauri by allowing to register custom response generator under a path of default scheme? Something like tauri::Builder::default().register_uri_path("/videos", |_app, request| { ... , where all requests going to /videos would be handled by custom code?

Could WebviewBuilder::on_web_resource_request be the solution?

Any updates on this? I’ve been holding off a feature in Linux that allows video play from local asset. Can anyone from the team please respond if there are any alternatives?

That codebase is shared between apple, webkitgtk, and wpe (and more?). So having a gstreamer folder doesn’t mean much for the macos port, especially if we take a look at what other folders are in that mediastream folder.

I’m not sure if this is the same bug, but I’m unable to play bundled audio files. I’m using SvelteKit and doing

<audio controls>
  <source src="/switch.ogg" type="audio/ogg" />
</audio>

and placed the corresponding sound file in src-svelte/static/switch.ogg. I can play this sound on yarn tauri dev but not the release build. Other assets such as font files or images in the static directory load just fine.

If instead I set the audio source to be some random file from https://en.wikipedia.org/wiki/Category:Wikipedia_Ogg_files, then that external sound file plays successfully even on the release build.

If instead I place the file in src-svelte/src/lib/sounds/switch.ogg and try to programmatically play the sound:

<script lang="ts">
  import clickSound from "$lib/sounds/switch.ogg";
  ...
  function playClick() {
    const audio = new Audio(clickSound);
    audio.play();
  }
</script>

<button
  ...
    on:click={playClick}
  >
    ...
</button>

I get the error

0:00:10.166424993 20275 0x559a88e1aa00 WARN            uridecodebin gsturidecodebin.c:1409:gen_source_element:<uridecodebin0> error: No URI handler implemented for "tauri".

where console.log(clickSound) gives me

tauri://localhost/_app/immutable/assets/switch.326f3a0f.ogg

convertFileSrc doesn’t appear to help either.

I can load an image asset but not for audio/video asset even with correct configuration.

image

"protocol": {
        "all": false,
        "asset": true,
} // leave the assestScope here because I load audio file using tauri file picker

"security": {
      "csp": "default-src 'self'; media-src 'self' asset: https://asset.localhost"
},

Don’t mind our project boards yet, we’re just playing around with them, trying to find something that works for us. In the latest iteration they only contain PRs which is why this issue was removed.

Looks like the issue still remains and I tried my best to write a C example to report back: https://bugs.webkit.org/show_bug.cgi?id=146351#c5

Hi all, I don’t know if it helps, I have just tested a basic video sample using the asset protocol. With a regular up-to-date ubuntu release, using webkitgtk2.36.6. Videos are still not playing on my station. Hopefully the next webkitgtk releases will have it fixed 😃

It’s weird 'cause I was sure it worked on previous webkit2gtk releases 😦

Is there a workaround that we can use? Or we just wait for a proper fix?

Ah whoops, didn’t see that you were 2 different people, sorry.