Meeting SDK Sending Audio raw data

Hello @chunsiong.zoom !

I’ve used your repositories as samples to write code for sending and receiving raw audio/video frames to the meeting room, however, the only part that is not working is the audio-sending part.

I’ve written the ZoomSDKVirtualAudioMicEvent.h header file:

#pragma once
#include <unistd.h>
#include <iostream>
#include <cstdint>
#include "rawdata/rawdata_audio_helper_interface.h"
#include "zoom_sdk.h"
#include "zoom_sdk_raw_data_def.h"

using namespace std;
using namespace ZOOMSDK;

class ZoomSDKVirtualAudioMicEvent :
	public IZoomSDKVirtualAudioMicEvent
{

private:
	IZoomSDKAudioRawDataSender* pSender_;

protected:

	/// \brief Callback for virtual audio mic to do some initialization.
/// \param pSender, You can send audio data based on this object, see \link IZoomSDKAudioRawDataSender \endlink.
	virtual void onMicInitialize(IZoomSDKAudioRawDataSender* pSender);

	/// \brief Callback for virtual audio mic can send raw data with 'pSender'.
	virtual void onMicStartSend();

	/// \brief Callback for virtual audio mic should stop send raw data.
	virtual void onMicStopSend();

	/// \brief Callback for virtual audio mic is uninitialized.
	virtual void onMicUninitialized();
};

the ZoomSDKVirtualAudioMicEvent.cpp: (Which I copy-pasted from your code)

#include <unistd.h>
#include <iostream>
#include <cstdint>
#include <fstream>
#include <cstring>
#include <cstdio>
#include <vector>

#include "rawdata/rawdata_audio_helper_interface.h"
#include "ZoomSDKVirtualAudioMicEvent.h"
#include "zoom_sdk_def.h" 

#include <thread>

using namespace std;
using namespace ZOOM_SDK_NAMESPACE;

int audio_play_flag = -1;

void PlayAudioFileToVirtualMic(IZoomSDKAudioRawDataSender* audio_sender, string audio_source)
{
	// execute in a thread.
	while (audio_play_flag > 0 && audio_sender) {

		// Check if the file exists
		ifstream file(audio_source, ios::binary | ios::ate);
		if (!file.is_open()) {
			std::cout << "Error: File not found. Tried to open " << audio_source << std::endl;
			return;
		}

		// Get the file size
		int file_size = file.tellg();

		// Read the file into a buffer
		vector<char> buffer(file_size);
		file.seekg(0, ios::beg);
		file.read(buffer.data(), file_size);

		// Send the audio data to the virtual camera
        std::cout << "************ BEFORE SEND IS CALLED!";
		SDKError err = audio_sender->send(buffer.data(), buffer.size(), 44100);
        std::cout << "***************************************************************";
		if (err != SDKERR_SUCCESS) {
			std::cout << "Error: Failed to send audio data to virtual camera. Error code: " << err << std::endl;
			return;
		}
		file.close();
		audio_play_flag = -1;
	}

}

/// \brief Callback for virtual audio mic to do some initialization.
/// \param pSender, You can send audio data based on this object, see \link IZoomSDKAudioRawDataSender \endlink.
void ZoomSDKVirtualAudioMicEvent::onMicInitialize(IZoomSDKAudioRawDataSender* pSender) {
	//pSender->send();
	pSender_ = pSender;
	printf("OnMicInitialize\n");
}

/// \brief Callback for virtual audio mic can send raw data with 'pSender'.
void ZoomSDKVirtualAudioMicEvent::onMicStartSend() {

	printf("onMicStartSend\n");


	std::cout << "onStartSend" << std::endl;
	if (pSender_ && audio_play_flag != 1) {
		while (audio_play_flag > -1) {}
		audio_play_flag = 1;
		std::string audio_source_ = "pcm1644m.wav";
		thread(PlayAudioFileToVirtualMic, pSender_, audio_source_).detach();
	}
}

/// \brief Callback for virtual audio mic should stop send raw data.
void ZoomSDKVirtualAudioMicEvent::onMicStopSend() {
	printf("onMicStopSend\n");

	audio_play_flag = 0;
}
/// \brief Callback for virtual audio mic is uninitialized.
void ZoomSDKVirtualAudioMicEvent::onMicUninitialized() {
	std::cout << "onUninitialized" << std::endl;
	pSender_ = nullptr;
}

and in the main.cpp file Besides all initialization I wrote the following:

void startSendRawAudio()
{
        ZoomSDKVirtualAudioMicEvent * send_audio_source = new ZoomSDKVirtualAudioMicEvent();
	IZoomSDKAudioRawDataHelper * send_audio_helper = GetAudioRawdataHelper();

	SDKError err = send_audio_helper->setExternalAudioSource(send_audio_source);
	
	if (err != SDKERR_SUCCESS)
	{
		std::cerr << "Error occured start send raw audio" << std::endl;
		return;
	}
	std::cout << "Success start send raw audio" << std::endl;
}
  • this is just a single function which I invoke in onInMeeting() function; Everything works fine except the audio sending part. I wanna know what I am missing here.

When I join the boot to a meeting the audio is not sent and it is just so silent. (The bot is unmuted)

Would love if you help me to overcome this issue.

Big thanks to you <3

@stutberidze there are 3 things to take note when sending raw audio.

  1. You might need to call the method below, which unmutes the microphone
void turnOnSendVideoAndAudio() {
	//testing WIP
	if (SendVideoRawData) {
		IMeetingVideoController* meetingVidController = m_pMeetingService->GetMeetingVideoController();
		meetingVidController->UnmuteVideo();
	}
	//testing WIP
	if (SendAudioRawData) {
		IMeetingAudioController* meetingAudController = m_pMeetingService->GetMeetingAudioController();
		meetingAudController->JoinVoip();
		printf("Is my audio muted: %d\n", getMyself()->IsAudioMuted());
		meetingAudController->UnMuteAudio(getMyself()->GetUserID());
	}
}
void turnOffSendVideoandAudio() {
	//testing WIP
	if (SendVideoRawData) {
		IMeetingVideoController* meetingVidController = m_pMeetingService->GetMeetingVideoController();
		meetingVidController->MuteVideo();
	}
	//testing WIP
	if (SendAudioRawData) {
		IMeetingAudioController* meetingAudController = m_pMeetingService->GetMeetingAudioController();
		meetingAudController->MuteAudio(getMyself()->GetUserID(), true);

	}
}

  1. There is noise suppression turned on for Meeting SDK, hence there is possibility some audio files might play “nothing” audible.

  2. For the code above, you would need a PCM file with 16 bit signed, mono channel and little endian order. The frequency can be set when sending. In this case it is 44100hz in the sample code.

Well I saw that in your code and imported and called that function to turn on the mic, however, the problem is the virtualmicevent class’s onMicStartSend function is not been called.

Update: now it works but when calling the send function it results in segmentation fault

Well Somehow I made it work and now it invokes the initialization and startSend functions, however, the problem now I am having is segmentation fault when calling the send function.

Here is the code below:

int audio_play_flag = -1;
constexpr double PI = 3.14159265358979323846;

void generateSineWave(std::vector<char>& audioData, double frequency, double duration, int sampleRate) {
    const int numSamples = static_cast<int>(duration * sampleRate);
    for (int i = 0; i < numSamples; ++i) {
        double sample = std::sin(2.0 * PI * frequency * i / sampleRate);
        int16_t intSample = static_cast<int16_t>(sample * 32767); // Assuming 16-bit signed PCM

        audioData.push_back(intSample & 0xFF);
        audioData.push_back((intSample >> 8) & 0xFF);
    }
}

void PlayAudioToVirtualMic(IZoomSDKAudioRawDataSender* audio_sender, string audio_source)
{
	printf("PLAYAUDIOFILETOVIRTUALMIC invoked!");
	
    std::vector<char> audioData;
    generateSineWave(audioData, 444.0, 5.0, 44100);

//    for (auto i : audioData)
//    {
//        std::cout << i << std::endl;
//    }

    audio_sender->send(audioData.data(), audioData.size(), 44100);
}

and this is where the thread is created to invoke the sending function:

void ZoomSDKVirtualAudioMicEvent::onMicStartSend() 
{
	printf("onMicStartSend\n");
	std::cout << "onAudioStartSend" << std::endl;
	// if (pSender_ && audio_play_flag != 1) {
	// 	// while (audio_play_flag > -1) {}
	// 	// audio_play_flag = 1;
    thread(PlayAudioToVirtualMic, pSender_, audio_source_).detach();

	// }
}

Fix

I just have fixed it.

Needed to set the audio sender:

void ZoomSDKVirtualAudioMicEvent::onMicInitialize(IZoomSDKAudioRawDataSender* pSender)
{
	printf("ZoomSDKVirtualAudioMicEvent OnMicInitialize, waiting for turnOn chat command\n");
    this->pSender_ = pSender; // Here this was missing
}
1 Like

Glad to hear that @stutberidze

1 Like