shaka-player: BAD_ENCODING: 2004 error when switching text track stpp -> vtt

Have you read the FAQ and checked for duplicate open issues? Yes

What version of Shaka Player are you using? 2.5.5

Can you reproduce the issue with our latest release version? Yes

Can you reproduce the issue with the latest code from master? Yes

Are you using the demo app or your own custom app? Our custom app

If custom app, can you reproduce the issue using our demo app? Not tested

What browser and OS are you using? Chrome/Firefox/Edge MacOS/Win10

For embedded devices (smart TVs, etc.), what model and firmware version are you using?

What are the manifest and license server URIs? Internal URIs

What did you do? If you play a live stream which contains text tracks with 2 different mime types where one of them contains binary header in a segment (i.e. stpp with mime: application/mp4) and the second one is a pure text (i.e. vtt with mime: text/vtt) then if you switch from the stpp one to the vtt sometimes (can be very rare) you’ll get BAD_ENCODING: 2004 error.

What did you expect to happen? No errors should happen when switching stpp -> vtt.

What actually happened? The 2004 error has happened.

I’ve analyzed and debugged the shaka’s code and here what’s happening: This situation has roots in fetchAndAppend_ method in lib/media/streaming_engine.js and happens when we still have the last stpp segment pending but lib/media/media_source_engine.js has already reinitialized a text parser to use lib/text/vtt_text_parser.js (text track has been switched) and we finally got the last (before switching) stpp segment and passed it down to the vtt text parser then the last-mentioned doesn’t expect binary header in the segment data and fails when trying to treat is as a pure utf8 text. So I came up with the fix which successfully resolves this situation the way it drops the last segment of the previous text track if it has different full mime type:

// In fetchAndAppend_ method in lib/media/streaming_engine.js:
...
try {
      const results = await Promise.all([initSourceBuffer, fetchSegment]);
      this.destroyer_.ensureNotDestroyed();
      if (this.fatalError_) {
        return;
      }
      // FIX STARTS HERE
      let mediaSourceUpdateIsSafeOperation = true;

      // Check if a mime type has changed while fetching init segment
      // of a text track due to a switch. It's not safe to call appendBuffer
      // in that case (i.e. in case of switching stpp -> vtt) so
      // we just waste the buffer.

      if (mediaState.type === ContentType.TEXT) {
        const currentFullMimeType = shaka.util.MimeUtils.getFullType(
            mediaState.stream.mimeType, mediaState.stream.codecs);
        mediaSourceUpdateIsSafeOperation =
            startingFullMimeType === currentFullMimeType;
      }

      if (mediaSourceUpdateIsSafeOperation) {
        await this.append_(mediaState,
            presentationTime, currentPeriod, stream, reference, results[1]);
      } else {
        // eslint-disable-next-line max-len
        shaka.log.v2(logPrefix, 'cancelling appending the given segment due to switch');
      }
      // FIX ENDS HERE
      this.destroyer_.ensureNotDestroyed();
      if (this.fatalError_) {
        return;
      }
      ...

// In initSourceBuffer_ method in lib/media/streaming_engine.js:
...
if (!mediaState.needInitSegment) {
      return;
    }
    // FIX STARTS HERE
    const ContentType = shaka.util.ManifestParserUtils.ContentType;

    const startingFullMimeType = shaka.util.MimeUtils.getFullType(
        mediaState.stream.mimeType, mediaState.stream.codecs);
      // FIX ENDS HERE
     ...
      try {
        const initSegment = await fetchInit;
        this.destroyer_.ensureNotDestroyed();
        shaka.log.v1(logPrefix, 'appending init segment');
        // FIX STARTS HERE
        let mediaSourceUpdateIsSafeOperation = true;

        // Check if a mime type has changed while fetching init segment
        // of a text track due to a switch. It's not safe to call appendBuffer
        // in that case (i.e. in case of switching stpp -> vtt) so
        // we just waste the buffer.

        if (mediaState.type === ContentType.TEXT) {
          const currentFullMimeType = shaka.util.MimeUtils.getFullType(
              mediaState.stream.mimeType, mediaState.stream.codecs);
          mediaSourceUpdateIsSafeOperation =
              startingFullMimeType === currentFullMimeType;
        }

        if (mediaSourceUpdateIsSafeOperation) {
          const hasClosedCaptions = mediaState.stream.closedCaptions &&
              mediaState.stream.closedCaptions.size > 0;
          await this.playerInterface_.mediaSourceEngine.appendBuffer(
              mediaState.type, initSegment, null /* startTime */,
              null /* endTime */, hasClosedCaptions);
        } else {
          shaka.log.v2(logPrefix, 'cancelling appending the init segment ' +
              'due to switch');
        }
      // FIX ENDS HERE
      } catch (error) {
        mediaState.needInitSegment = true;
        mediaState.lastInitSegmentReference = null;
        throw error;
      }
      ...

I can create a PR if this approach seems correct for the shaka team.