Video Calling

Set up video calls with the Sinch Android Voice with Video SDK.

Setting Up a Video Call

Just like audio calls, video calls are placed through the CallController and events are received using the CallControllerListener (callback onIncomingCall()) and VideoCallListener which is a subclass of CallListener to include other callbacks like onCallEstablished(), onCallEnded() etc. The call controller is owned by the SinchClient and accessed using SinchClient.callControler.

Showing the Video Streams

Once you have created a VideoCallListener and added it to a call using Call.addCallListener method will be called.

override fun onVideoTrackAdded(call: Call) {
    // Get a reference to your SinchClient, in the samples this is done through the service interface.
    val videoController = sinchServiceInterface.videoController
    val localPreviewView = videoController.localView
    val remoteView = videoController.remoteView

    // Add the views to your view hierarchy
    ...
}

After the call has ended, don’t forget to remove the views from your view hierarchy again.

Pausing and Resuming a Video Stream

To pause or resume the local video stream, use the method Call.pauseVideo() or Call.resumeVideo()

The call listeners will be notified of pause and resume events via the callback methods VideoCallListener.onVideoTrackPaused(). Based on these events it's for example appropriate to update the UI with a pause indicator, and subsequently remove such pause indicator.

override fun onVideoTrackPaused(call: Call) {
     // Implement what to be done when remote user pause video stream.
}

override fun onVideoTrackResumed(call: Call) {
     // Implement what to be done when remote user resumes video stream.
}

Video Content Fitting and Aspect Ratio

Use VideoController.setResizeBehaviour(VideoScalingType) and VideoController.setLocalResizeBehaviour(VideoScalingType) methods to control various aspects of the local and remote video. Possible arguments are VideoScalingType.ASPECT_FIT, VideoScalingType.ASPECT_FILL and VideoScalingType.ASPECT_BALANCED. The local preview will always use VideoScalingType.ASPECT_FIT.

Switching Capturing Device

The capturing device can be switched using VideoController.setCaptureDevicePosition(Int) to alternate the two.

Accessing Video Frames of the Remote Streams

The Sinch SDK can provide access to raw video frames via a callback function. This callback can be used to achieve rich functionality such as applying filters, adding stickers to the video frames or saving the video frame as an image.

Your video frame handler needs to implement RemoteVideoFrameListener interface by implementing the onFrame() callback. Note that this method blocks rendering. In-place modification of provided I420 frame be rendered.

Example:

class YourVideoFrameHandler: RemoteVideoFrameListener {

    @Synchronized 
    override fun onFrame(callId: String, videoFrame: VideoFrame) {
    // Process videoFrame
    }
}

Use VideoController.setRemoteVideoFrameListener() to register your video frame handler as the callback to receive video frames.

Example:

val videoFrameHandler = new YourVideoFrameHandler()
sinchClient.videoController.setVideoFrameListener(videoFrameHandler)

Accessing Video Frames of the Local Streams

Similar to the accessing the remote stream video frames, use VideoController.setLocalVideoFrameListener() callback.

Converting video frame from I420 to NV21

The Sinch SDK provides a helper function to convert the default I420 frame to NV21 Frame, which is handy to work with when you need to save it as an image on Android. Use VideoUtils.I420toNV21Frame(VideoFrame) for the conversion. Note that this helper doesn't release the original I420 video frame.

Example:

val videoFrame = ... // Get the video frame from onFrame() callback
val nv21Frame = VideoUtils.I420toNV21Frame(videoFrame)

val image: YuvImage = YuvImage(nv21Frame.yuvPlanes()[0].array(),
                              ImageFormat.NV21,
                              nv21Frame.width(),
                              nv21Frame.height(),
                              nv21Frame.yuvStrides())