react-native-video: VTT Subtitles broken on iOS

Bug

I am trying to show captions on my videos using the textTracks and selectedTextTrack properties. I am using VTT as per guidance in the documentation as it is the only type supported by iOS and Android.

On Android, I have no issues and it works perfectly. On iOS, there are several serious issues if textTracks and selectedTextTrack are provided - they resolve if removed:

  • Videos freeze and have to be reloaded several times
  • When they do play, they don’t have any sound
  • Only the first entry of the VTT file shows. When its timeframe has ended, sometimes it disappears, sometimes it stays permanently.

The VTT file is being served from a signed S3 URL. By the fact that it works on Android and occasionally shows on iOS, I am assuming that the URL itself is not the problem.

Platform

  • iOS

Environment info

Library version: 6.0.0-beta.3 Device: iPhone 11, iOS 17.2

Steps To Reproduce

Watch streamed MP4 video with streamed VTT subtitles

Expected behaviour

  1. Video plays with sound
  2. Subtitles update at the correct timestamps

Reproducible sample code

        <Video
          style={videoStyle}
          ref={ref}
          source={{ uri: src }}
          resizeMode={resizeMode}
          posterResizeMode={posterResizeMode}
          automaticallyWaitsToMinimizeStalling
          disableFocus
          useTextureView={false}
          playWhenInactive
          paused={isPaused || isSeeking}
          ignoreSilentSwitch={IgnoreSilentSwitchType.IGNORE}
          controls={!!showControls}
          repeat={!!shouldRepeat}
          muted={!!isMuted}
          poster={loadingImage}
          onReadyForDisplay={onReady}
          onLoad={handleLoad}
          onBuffer={(ev) => setBuffering(ev.isBuffering)}
          onError={(err) => handleError(err.error)}
          onEnd={handleFinish}
          progressUpdateInterval={50}
          onProgress={handleProgress}
          selectedTextTrack={
            showCaptions && transcriptUri
              ? { type: SelectedTrackType.INDEX, value: 0 }
              : { type: SelectedTrackType.DISABLED }
          }
          textTracks={
            transcriptUri
              ? [
                  {
                    title: 'Closed captions',
                    uri: transcriptUri,
                    language: 'en',
                    type: TextTracksType.VTT,
                  },
                ]
              : undefined
          }
          subtitleStyle={subtitleStyle}
          bufferConfig={{
            minBufferMs: 2500,
            maxBufferMs: 60000,
            bufferForPlaybackMs: 2500,
            bufferForPlaybackAfterRebufferMs: 2500,
            maxHeapAllocationPercent: 0.5,
          }}
        />

I have tried with 2 versions of the same VTT data, one with optional style and index properties set and one without. Both result in the same issues:

With style and order properties set:

WEBVTT

1
00:00:00.000 --> 00:00:05.760 align:middle line:50% position:50% size:80%
This is a test recording that I'm going to make at least 10 to 15 seconds long because I want it to

2
00:00:05.760 --> 00:00:10.880 align:middle line:50% position:50% size:80%
pick up on my words and do a full transcription of everything that I'm saying. I want to see how

3
00:00:10.880 --> 00:00:17.520 align:middle line:50% position:50% size:80%
well that it works and to see that the subtitles are actually prepared to show at the right time.

Without:

WEBVTT

1
00:00:00.000 --> 00:00:05.760
This is a test recording that I'm going to make at least 10 to 15 seconds long because I want it to

2
00:00:05.760 --> 00:00:10.880
pick up on my words and do a full transcription of everything that I'm saying. I want to see how

3
00:00:10.880 --> 00:00:17.520
well that it works and to see that the subtitles are actually prepared to show at the right time.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 5
  • Comments: 29 (4 by maintainers)

Most upvoted comments

In case somebody wants an easy solution, I’m now using another package to handle showing subtitles.

Hey, I am facing a similar issue where the video doesn’t work at all with the following props:

 selectedTextTrack={{
          type: 'language',
          value: 'en',
        }}
        textTracks={[
          {
            title: 'English',
            language: 'en',
            type: TextTracksType.VTT,
            uri: 'https://bitdash-a.akamaihd.net/content/sintel/subtitles/subtitles_en.vtt',
          },
          {
            title: 'Deutsch',
            language: 'de',
            type: TextTracksType.VTT,
            uri: 'https://bitdash-a.akamaihd.net/content/sintel/subtitles/subtitles_en.vtt',
          },
        ]}

for me passing texttracks just breaks playback entirely

IOS

React native video - v 5.2

On Android everything works ok

<Video paused={paused} source={url} style={[styles.video, !ios && fullscreenContext.value && fullscreenStyle]} rate={velocities[currentRate]} resizeMode=“contain” fullscreen={fullscreenContext.value} onLoad={handleLoad} ignoreSilentSwitch=“ignore” volume={volume} onProgress={handleProgress} onEnd={handleEnd} ref={playerRef} thumbnail={thumbnail || null} onBuffer={onBuffer} onLoadStart={onLoadStart} playInBackground={backgroundAudio} selectedTextTrack={{type: ‘index’,value: selectLegendas, }} textTracks={[ { index: 0, title: ‘English CC’, language: ‘en’, type: TextTrackType.VTT, uri: legendaEn}, { index: 1, title: ‘Português PT’, language: ‘pt’, type: TextTrackType.VTT, uri: legendaPt}, { index: 2, title: ‘Spanish ES’, language: ‘es’, type: TextTrackType.VTT, uri: legendaEs} ]} />

I’m also having freezes when using a VTT.

In my case, the video uses a signed S3 URL but VTT is public. It seems that if both are public URLs, the video plays properly but subtitles are currently not appearing on v6.0.0-rc.0.

Cloud someone test this PR #3557 ?

Hi @brycnguyen, unfortunately not. It’s now playing without freezing, but the sound is still missing and the text tracks aren’t loading at all. This is the response body from onLoad:

{
  "textTracks": [],
  "canStepForward": true,
  "canPlayFastForward": true,
  "canPlaySlowReverse": true,
  "audioTracks": [],
  "currentTime": 0,
  "canPlayReverse": true,
  "duration": 17.753999710083008,
  "canPlaySlowForward": true,
  "target": 1719,
  "canStepBackward": true,
  "naturalSize": {
    "height": 1920,
    "width": 1080,
    "orientation": "landscape"
  }
}

I’ve confirmed in a separate console log that the transcriptionUri is definitely being supplied