video-quickstart-ios: Error when adding app audio track in ReplayKit SampleHandler

Description

I am trying to add app audio using ExampleCoreAudioDeviceRecordCallback(sampleBuffer) to the broadcast extension. However, when I use the following code to add it, I got a crash from the broadcast extension:

Code

    override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
        switch sampleBufferType {
        case RPSampleBufferType.video:
            videoSource?.processVideoSampleBuffer(sampleBuffer)
            break
        case RPSampleBufferType.audioApp:
            // trying to add app audio
            ExampleCoreAudioDeviceRecordCallback(sampleBuffer)
            break
        case RPSampleBufferType.audioMic:
            // I don't want mic audio
            break
        }
    }

Otherwise I use the code from your latest ReplayKit example. It is reproducible with ReplayKit example on 2.6.0-preview branch of video-quickstart-swift, you just need to move the ExampleCoreAudioDeviceRecordCallback(sampleBuffer) from RPSampleBufferType.audioMic to RPSampleBufferType.audioApp.

Expected Behavior

Broadcasting the screen capture along with the system audio (not mic)

Actual Behavior

The broadcast extension crashes

Logs

didConnectToRoom:  <<TVIRoom: 0x283e9ed80> name: 4D149C63-8BD7-4CCB-A37F-04DDDE6175AA, state: Connected, sid: RM8043cedde0c5210929792b120f3edf55, delegate: <squad_broadcast.SampleHandler: 0x283b950a0>>
2018-12-17 11:05:51.289171+0100 squad-broadcast[635:8535] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'The supplied sizeInBytes is invalid. The sizeInBytes must match with the size returned by TVIAudioFormat:bufferSize utility method.'
*** First throw call stack:
(0x225d0fea0 0x224ee1a40 0x225c2997c 0x1043d0ec4 0x1040333e0 0x104037150 0x1040378d4 0x242580fe4 0x2257496c8 0x22574a484 0x2256f1be0 0x2256f2728 0x2256faec8 0x22592c0dc 0x22592ecec)
libc++abi.dylib: terminating with uncaught exception of type NSException

Versions

Video iOS SDK

2.5.6

Xcode

10.1

iOS Version

12.1

iOS Device

iPhone 6s

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 15 (8 by maintainers)

Most upvoted comments

Hi,

Thanks for the info, regardless of the setup, I still have not seen a way to capture app audio, it would be really great to see in example working to make sure it even is compatible with twilio in the simplest case (Just from the broadcast extension and no mixing mic)

I agree, and we have cut ISDK-2334 for internal tracking on the issue. We need to get this working before moving onto the next step, and @piyushtank has got you covered on this one.

Thanks for your patience, Chris

ceaglest on Dec 17, 2018

@MilanNosal Thanks for filing the issue. Wanted to let you know that I have started working on the problem and I am able to reproduce it. I will keep you posted.

Piyush

piyushtank on Dec 20, 2018

@ceaglest Thanks for the info, regardless of the setup, I still have not seen a way to capture app audio, it would be really great to see in example working to make sure it even is compatible with twilio in the simplest case (Just from the broadcast extension and no mixing mic)

etown on Dec 17, 2018