sendVideoFrame Does Not Work As Expected

In my app I have a custom camera implementation, from which I can intercept frames. my plan is to get frames from my camera and feed them into the Zoom Video SDK using ZoomVideoSDKVideoSender.sendVideoFrame. However, the frames I send using this function are not received anywhere.

Here are the relevant steps in the setup:

  1. I create a session as follow:
        let sessionContext = ZoomVideoSDKSessionContext()
        sessionContext.token = token
        sessionContext.sessionName = topic
        sessionContext.userName = name

        let vidOption = ZoomVideoSDKVideoOptions()
        vidOption.localVideoOn = false
        let audOptions = ZoomVideoSDKAudioOptions()
        audOptions.connect = true
        audOptions.mute = false
        sessionContext.videoOption = vidOption
        sessionContext.audioOption = audOptions

        sessionContext.externalVideoSourceDelegate = self

  1. As I set the external video source delegate to self, I extend ZoomVideoSDKVideoSource
extension ZoomViewController: ZoomVideoSDKVideoSource {
    func onInitialize(_ rawDataSender: ZoomVideoSDKVideoSender, supportCapabilityArray: [Any], suggest suggestCapability: ZoomVideoSDKVideoCapability) {
        self.frameSender = rawDataSender
  1. When a user joins the session, I find that user and subscribe to their video pipe
other.getVideoPipe()?.subscribe(with: self, resolution: ._360)
  1. and I extend with ZoomVideoSDKRawDataPipeDelegate as follow:
extension ZoomViewController: ZoomVideoSDKRawDataPipeDelegate {
    func onPixelBuffer(_ pixelBuffer: CVPixelBuffer?, rotation: ZoomVideoSDKVideoRawDataRotation) {
        print("Frame received")
    func onRawDataFrameReceived(_ rawData: ZoomVideoSDKVideoRawData?) {
        print("Received raw frame ")
  1. When I receive a frame from my camera implementation, I send it using the frame sender initialized earlier

Now the issue is that neither of onPixelBuffer and onRawDataFrameReceived (in step 4) is ever triggered when I send frames (step 5)

Is my use of sendVideoFrame correct?

Which iOS Video SDK version?
iOS 1.7.10

@wbaroudi ,

could you share the code on how you are using sendVideoFrame?

The understanding of your flow above is correct.

There is a possibility the issue on the sendVideoFrame side.

Hello @chunsiong.zoom ,
My bad, I missed that part. sendVideoFrame is used in the following process:

  1. I have a struct ZoomFrame which looks like this
struct ZoomFrame {
    let buffer: UnsafeMutablePointer<CChar>
    let width: UInt
    let height: UInt
    let dataLength: UInt
    let rotation: ZoomVideoSDKVideoRawDataRotation
  1. My camera implementation gives me frames as CMSampleBuffers, and I convert them to ZoomFrames as follows:
extension CMSampleBuffer {
    func toZoomFrame() -> ZoomFrame? {
        // Ensure buffer data is ready
        guard CMSampleBufferDataIsReady(self) else { return nil }

        // Try to get an image buffer
        guard let imageBuffer = CMSampleBufferGetImageBuffer(self) else { return nil }

        // Get the width and height of the image buffer
        let width = UInt(CVPixelBufferGetWidth(imageBuffer))
        let height = UInt(CVPixelBufferGetHeight(imageBuffer))

        // Lock the base address
        CVPixelBufferLockBaseAddress(imageBuffer, .readOnly)

        // Get the data pointer
        guard let dataPointer = CVPixelBufferGetBaseAddress(imageBuffer) else { return nil }

        // Get the data length
        let bytesPerRow = UInt(CVPixelBufferGetBytesPerRow(imageBuffer))
        let dataLength = bytesPerRow * height

        // Cast to UnsafeMutablePointer<CChar>
        let charPointer = dataPointer.assumingMemoryBound(to: CChar.self)

        // Unlock the base address
        CVPixelBufferUnlockBaseAddress(imageBuffer, .readOnly)

        return ZoomFrame(buffer: charPointer, width: width, height: height, dataLength: dataLength, rotation: .rotationNone)

  1. I feed the converted frames into the following function:
    func sendFrame(frame: ZoomFrame) {
        if let sender = frameSender {
            sender.sendVideoFrame(frame.buffer, width: frame.width, height: frame.height, dataLength: frame.dataLength, rotation: frame.rotation, format: .I420) // TODO: maybe change format?
            print("Frame sent")

And the print “Frame sent” is reached.
Hope this makes things clearer.



After converting CMSampleBuffer to ZoomFrame, is it possible to do either of these test?
I’m assuming this is in YUV420 format, but there is a possibility the format might not be valid.

Testing if conversion has a valid YUV420 format

  1. Instead of sending the converted YUV420 frames to sendFrame, save it locally to a recording.yuv file.
  2. Convert the recording.yuv file into a mp4 file using ffmpeg command line (might not be 100% accurate, you might need some adjustments for the parameters)
    ffmpeg -f rawvideo -pix_fmt yuv420p -framerate 25 -i recording.yuv -f mp4 recording.mp4
  3. Verify that the mp4 plays, and is in correct format without any distortion or color errors

Testing with a video file instead of camera buffer stream.

  1. Download a sample video. I’ve previously used big buck bunny 720p 10s 1MB
    MP4 ( H.264 ) | Test Videos
  2. Convert the video into YUV420 buffer
  3. Send the buffer into sendVideoFrame

@chunsiong.zoom ,

Apparently the CVPixelBuffers in my frames are in 32BGRA format, which may be the cause of the issue.
However, I have tried the video test, and, unfortunately, I still didn’t manage to receive a frame on the other side. Here are the steps in my test:

  1. Get the video and convert it to a YUV420 buffer
    func getSampleVideoBuffer(url: URL) -> CVPixelBuffer? {
        // Create AVAsset
        let asset = AVAsset(url: url)
        // Get video track
        guard let videoTrack = asset.tracks(withMediaType: .video).first else {
            print("Failed to get video track")
            return nil
        // Get video settings
        let videoSettings: [String: Any] = [
            kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
            kCVPixelBufferWidthKey as String: videoTrack.naturalSize.width,
            kCVPixelBufferHeightKey as String: videoTrack.naturalSize.height
        // Create AVAssetReader
        do {
            let reader = try AVAssetReader(asset: asset)
            // Create AVAssetReaderTrackOutput and assign to reader
            let readerOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: videoSettings)
            // Start reading
            // Create pixel buffer from sample buffer
            if let sampleBuffer = readerOutput.copyNextSampleBuffer(),
               let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
                // Release sample buffer
                // Return pixel buffer in YUV420 format
                return pixelBuffer
            } else {
                print("Failed to create sample buffer")
                return nil
        } catch {
            print("Failed to create AVAssetReader: \(error)")
            return nil
  1. Send the buffer:
if let zoomFrame = sample?.toZoomFrame() {
                sendFrame(frame: zoomFrame)

Note that I could view the resulting zoomFrame in the debugger and it looked fine.

@wbaroudi , if you are able to see the frame just fine, there is a high chance the format is not correct. YUV420 raw format looks something like this

Reference How to extract frames from a .yuv video file (YUV420) using python and openCV? - Stack Overflow

@chunsiong.zoom ,
I’m not sure I understand the issue. If the format is correct, the frame should look in grayscale (similar to the sample you showed), instead of its normal colors?
The format of the frame is kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange
is there a specific format that sendVideoFrame expects?

@wbaroudi ,

I don’t have swift code, but here’s how I’ve managed to do it on c++ & windows.
The process should be similar. This code reads from a mp4 file, converts it frame by frame to yuv420 format and sends it

int video_is_playing = 1;
char* frameBuffer;
// Calculate the size of the YUV420 buffer based on frame dimensions
int frameLen = height / 2 * 3 * width; // Height/2 for subsampled U and V planes
frameBuffer = (char*)malloc(frameLen);

// Execute in a thread.
while (video_is_playing > 0 && video_sender) {
    Mat frame;  // Represents a single video frame
    VideoCapture cap;  // Represents a video capture device;; //opens up a mp4 file from local storage path
    if (!cap.isOpened()) {
        cerr << "ERROR! Unable to open camera stream or video file\n";
        video_is_playing = 0;
    cout << "Start grabbing" << endl;
    while (
        // Wait for a new frame from the camera and store it into 'frame'
        // Check if we succeeded in reading a frame
        if (frame.empty()) {
            cerr << "ERROR! Blank frame grabbed\n";
        // Convert Mat to YUV buffer
        Mat yuv;
        cv::cvtColor(frame, yuv, COLOR_BGRA2YUV_I420);

        // Get the YUV buffer data
        char* frameBuffer = (char*);
        int width = yuv.cols;
        int height = yuv.rows;
        int frameLen = * yuv.elemSize(); // Calculate YUV420 buffer size

        // Send the YUV buffer to the video sender
        SDKError err = ((IZoomSDKVideoSender*)video_sender)->sendVideoFrame(frameBuffer, width, height, frameLen, 0);
        if (err != SDKERR_SUCCESS) {
            cout << "sendVideoFrame failed: Error " << err << endl;

video_is_playing = -1;

I see.
As far as I understand, all else being the same, if I use OpenCV to convert my camera frames from 32BGRA to YUV_I420, sendVideoFrame should work fine and I should receive frames on the other side which is subscribed to my videoPipe.
Anything else I should change?


@wbaroudi the frame length is also slightly different, do reference the code i’ve shared

Hello, @chunsiong.zoom
I was trying again with the video test that you mentioned, attempting to replicate your c++ conversion code in swift. However, in the process, I noticed that the frames coming from the video I downloaded are already in YUV420 bi-planar (full range) format. Is that the case or am I missing something? and if so, then there’s no need for the cv::cvtColor(frame, yuv, COLOR_BGRA2YUV_I420); step in the first place, correct?

@wbaroudi ,

You are trying to send video right?

The video which you downloaded, is this referring to your camera stream which has been converted to YUV420 format?

cv::cvtColor(frame, yuv, COLOR_BGRA2YUV_I420) is not necessary if it is already converted to YUV

yes, I’m trying to send send the big buck bunny 720p 10s 1MB video frame by frame.
Here’s the full code for it:

import AVFoundation
import ZoomVideoSDK

class VideoProcessor {
    var videoIsPlaying: Int = 1
    var assetReader: AVAssetReader?
    var asset: AVAsset?
    var videoTrackOutput: AVAssetReaderTrackOutput?
    private var frameSender: ZoomVideoSDKVideoSender?
    init?(videoURL: URL, sender: ZoomVideoSDKVideoSender) {
        frameSender = sender
        self.asset = AVAsset(url: videoURL)
        do {
            try self.assetReader = AVAssetReader(asset: self.asset!)
        } catch {
            print("ERROR! Unable to open video file")
            return nil
        guard let videoTrack = self.asset?.tracks(withMediaType: .video).first else {
            print("ERROR! Could not get video track")
            return nil
        let outputSettings: [String: Any] = [
            kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
        self.videoTrackOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: outputSettings)
    func startProcessing() {
        while self.videoIsPlaying > 0 {
            if let sampleBuffer = self.videoTrackOutput?.copyNextSampleBuffer() {
                // Get the YUV buffer data
                if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
                    let width = CVPixelBufferGetWidth(imageBuffer)
                    let height = CVPixelBufferGetHeight(imageBuffer)
                    let frameLen = CVPixelBufferGetDataSize(imageBuffer)
                    // Access the YUV data (ensure the memory block is locked for reading)
                    CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags.readOnly)
                    let rawPointer = CVPixelBufferGetBaseAddress(imageBuffer)
                    // Convert UnsafeMutableRawPointer to UnsafeMutablePointer<CChar>
                    let frameBuffer = rawPointer?.assumingMemoryBound(to: CChar.self)
                    // Send the YUV buffer to the video sender
                    sendVideoFrame(frameBuffer, width, height, frameLen, 0)
                    CVPixelBufferUnlockBaseAddress(imageBuffer, CVPixelBufferLockFlags.readOnly)
            } else {
                // End of file or an error occurred
                self.videoIsPlaying = -1

    func sendVideoFrame(_ frameBuffer: UnsafeMutablePointer<CChar>?, _ width: Int, _ height: Int, _ frameLen: Int, _ timestamp: Int64) {
        if let sender = frameSender, let buffer = frameBuffer {
            sender.sendVideoFrame(buffer, width: UInt(width), height: UInt(height), dataLength: UInt(frameLen), rotation: .rotationNone, format: .I420) // TODO: maybe change format?
            print("Frame sent")
        } else {
            print("Sender is not initialized")


Note that since the frames from this video are already in bi-planar yuv420, I skipped the conversion step.
The issue persists, and I don’t seem to receive any frames on the other side.

Hi @chunsiong.zoom
Would you happen to have a working sample where you make use of the sendVideoFrame function in iOS?

@wbaroudi sorry no I don’t.
Here’s a sample on windows nonetheless. I hope it helps in some way

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.