cordova-plugin-iosrtc: Problem with change camera during video call (replaceTrack is not a function)
Expected behavior
Switching from front and retro camera during video call through the replaceTrack function, see code below
Observerd behavior
I get errors on “undefined is not an object (evaluating ‘sender.track.kind’)” and “replaceTrack is not a function”.
Steps to reproduce the problem
switch camera through the code below
Platform information
- Cordova version: 8.1.2
- Plugin version: 6.0.9 and 6.0.10
- iOS version: 12.4.5 Iphone 6
- Xcode version:11.4.1 Swift 4.2
Thanks, Regards Maurizio
WebRTCSession.prototype.switchMediaTracks = function(deviceIds, callback) {
if(!navigator.mediaDevices.getUserMedia) {
throw new Error('getUserMedia() is not supported in your browser');
}
var self = this,
localStream = this.localStream;
if (deviceIds && deviceIds.audio) {
self.mediaParams.audio.deviceId = deviceIds.audio;
}
if (deviceIds && deviceIds.video) {
self.mediaParams.video.deviceId = deviceIds.video;
}
localStream.getTracks().forEach(function(track) {
track.stop();
});
navigator.mediaDevices.getUserMedia({
audio: self.mediaParams.audio || false,
video: self.mediaParams.video || false
}).then(function(stream) {
self._replaceTracks(stream);
callback(null, stream);
}).catch(function(error) {
callback(error, null);
});
};
WebRTCSession.prototype._replaceTracks = function(stream) {
var peers = this.peerConnections,
localStream = this.localStream,
elemId = this.mediaParams.elemId,
ops = this.mediaParams.options,
newStreamTracks = stream.getTracks();
this.detachMediaStream(elemId);
newStreamTracks.forEach(function(track) {
localStream.addTrack(track);
});
this.attachMediaStream(elemId, stream, ops);
for (var userId in peers) {
_replaceTracksForPeer(peers[userId]);
}
function _replaceTracksForPeer(peer) {
peer.getSenders().map(function(sender) {
** error here sender.replaceTrack(newStreamTracks.find(function(track) {
return track.kind === sender.track.kind; ** error here
}));
});
}
};
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 26 (18 by maintainers)
@abardik I will see what I can do after #508 and #494 is released
@hthetiot
I can confirm that this feature will tremendously improve user experience, because
replaceTrack
allows to switch between front/rear cameras, screen capture (which you already implemented viagetDisplayMedia
) or any other potential video sources, as well as between audio sources (for example, microphone and system sound) with no renegotiation at all - the replaced track just appears on the other side immediately, so remote user should not wait for renegotiation (which usualy takes 2-5 seconds) every time you switch your camera/screen/etc.Speaking of compatibility,
replaceTrack
is already supported by all browsers, including Safari 13.1+: https://caniuse.com/#search=replacetrackAnd I can confirm - it really works like a charm, with no failure at all (for comparison, renegotiation used to fail often enough to start annoying users).
So if, in any case, it’s possible to implement
replaceTrack
without spending enormous amount of time, it would be great to have this feature on iOS devices.Thank you.
Hi @hthetiot,
I have a quick clarification regarding current ‘replaceTrack’ implementation
Is it intended to have it work via ‘negotiationneeded’?
I just checked the MDM documentation and it says the method works without additional negotiation - it just replaces a track and the receiver party started to get new track immediately, w/o additional logic
Could you please clarify it. Thanks
@Christina05
If you need a workaround for
replaceTrack
, you can use the following code (place it after cordova.plugins.iosrtc.registerGlobals). But it’s not a nativereplaceTrack
, so it will require a renegotiation. And it’s based on outdatedaddStream
/removeStream
, so please, use it on your own risk, because with ‘unified-plan’addStream
/removeStream
creates new tracks, but doesn’t really remove old tracks, just mutes and marks them disabled.Source: https://blog.mozilla.org/webrtc/warm-up-with-replacetrack/