Inconsistent black screen video for participants on join

KieranAC · January 21, 2022, 5:37am

Description
While running our app and in a meeting, occasionally the normal video elements of joining participants will be permanently black. There is no avatar or user name. It seems to happen most often when multiple users are joining at once at the beginning of a meeting.

Which macOS Meeting SDK version?
v5.9.1

To Reproduce(If applicable)
Steps to reproduce the behavior:

Create a Zoom meeting and invite several users
Join the meeting with the custom app/SDK
Wait for other users to join
When other users join, sometimes their normal video element will be black.
Restart the app and rejoin the meeting
Users will (likely?) now have video

Screenshots

Device (please complete the following information):
Apple MacBook Pro (13-inch, M1)
macOS 12.0.1

Additional context

Only applies to ZoomSDKNormalVideoElement. Active video element is fine.
If displayUserNameOnVideo is set, the user name is usually(?) incorrect. It shows a different participant name.
App is not using the resize method for normal video elements. When should this be used?
App is not using the setResolution method for normal video elements. When should this be used?
I was able to reproduce the issue with logging in the ZoomSDKVideoContainerDelegate delegate methods. None of these were called when the issue occurred, including onCustomVideoSubscribeFail.
I was worried about some sort of retain issue (the normal video element getting deallocated) but it seems like the Zoom SDK holds a reference to the element until it is told to destroy it.

Normal Video Element creation code:

- (ZoomSDKNormalVideoElement *)createNormalVideoElementForUserID:(NSNumber *)userID
                                                      withBounds:(NSRect)bounds {
    
    ZoomSDKVideoContainer *videoContainer = [[[ZoomSDK sharedSDK] getMeetingService] getVideoContainer];
    
    ZoomSDKNormalVideoElement *normalVideoElement = [[ZoomSDKNormalVideoElement alloc] initWithFrame:bounds];
    
    ZoomSDKError zError = [videoContainer createVideoElement:&normalVideoElement];
    
    if (zError != ZoomSDKError_Success) {
        AC_LOG_ERROR_ZOOM(@"Failed to create normal video element");
    }
    
    [normalVideoElement setUserid:userID.unsignedIntValue];
    
    return normalVideoElement;
}

And the calling code: (not actual, this is summarized)

{
     ZoomSDKNormalVideoElement *normalVideoElement = [self createNormalVideoElementForUserID:userID withBounds:containingViewController.view.bounds];

    [normalVideoElement.videoView setAutoresizingMask:NSViewWidthSizable|NSViewHeightSizable];
                
    [containingViewController.view addSubview:[normalVideoElement getVideoView]];
}

jon.zoom · January 21, 2022, 10:36pm

Hi @KieranAC, thanks for the post.

Sorry to hear you’re running into issues with video subscription. We’ll need some more information in order to determine exactly what’s going on here.

Is there an average number of participants that must join a meeting before this issue is present?
Over what period of time must the participants join in order for this to be reproduced?
How frequently is this reproducible?
Are you able to reproduce with the default UI, or is it only present with a custom UI?

If displayUserNameOnVideo is set, the user name is usually(?) incorrect. It shows a different participant name.

This sounds like a different bug than the one initially described. Is it only present when the black video is present?

App is not using the resize method for normal video elements. When should this be used?

This should be called after resizing a video element within your meeting UI. Depending on the change in size, the SDK may also adjust the resolution of that video element.

App is not using the setResolution method for normal video elements. When should this be used?

This can be used if you want to manually adjust the resolution of a video element (e.g. you want to limit all video subscriptions to 90p).

I was able to reproduce the issue with logging in the ZoomSDKVideoContainerDelegate delegate methods. None of these were called when the issue occurred, including onCustomVideoSubscribeFail.

It sounds like something may be going on which is preventing the SDK from attempting to subscribe. What is the return value of the subscribeVideo method when you aren’t seeing any callbacks?

Thanks!

KieranAC · January 21, 2022, 10:57pm

Is there an average number of participants that must join a meeting before this issue is present?

Our internal meetings are usually 4-8 people and it’s reproducible with that number

Over what period of time must the participants join in order for this to be reproduced?

It typically occurs when the meeting is starting, and all the participants are joining within a minute of each other. Hard to say precisely. We aren’t doing anything special to reproduce, just organic meeting joining. (We are using our own client for internal meetings, for example) If the affected client leaves and rejoins the meeting, all the videos for the participants load at that point. The issue does seem to have a correlation with participants joining.

How frequently is this reproducible?

It’s been pretty difficult to reproduce while deliberately attempting to debug the issue. But it is reproducible almost daily in our internal meetings. It could be that the ‘rate of joining’ is higher in an organic meeting.

Are you able to reproduce with the default UI, or is it only present with a custom UI?

We only use custom UI

This sounds like a different bug than the one initially described. Is it only present when the black video is present?

I believe so yes.

This should be called after resizing a video element within your meeting UI. Depending on the change in size, the SDK may also adjust the resolution of that video element.

Right now I am simply using constraints and set the autoresizing attributes for the view (see sample code). Is this insufficient? Should I always call ‘resize’ whenever the view is changed from its original size?

This can be used if you want to manually adjust the resolution of a video element (e.g. you want to limit all video subscriptions to 90p).

What is the default if this isn’t used? Does the SDK adjust the resolution dynamically? What happens if bandwidth limits are hit? Is this relevant at all for this issue, or should I ignore this API for now? I imagine you might want to manually adjust the resolution via user input or in response to connectivity issues, neither of which are an issue here?

It sounds like something may be going on which is preventing the SDK from attempting to subscribe. What is the return value of the subscribeVideo method when you aren’t seeing any callbacks?

I am not using this method at all presently. I implemented the normal video elements using this as a reference:
https://marketplace.zoom.us/docs/sdk/native-sdks/macos/custom-ui-feature

which doesn’t mention ‘subscribeVideo’. I also don’t see this being used in the sample app.

It seems likely that this could be the root cause. Is ‘subscribeVideo’ necessary? What does this API do? Should it be called while creating the normal video element? If this is necessary, it is strange that our app works 95% of the time without it?

KieranAC · January 24, 2022, 7:09pm

I added

[normalVideoElement subscribeVideo:YES];

to my normal element creation method and was able to reproduce the same issue.

I have not been able to reproduce the issue using multiple devices on my local home network despite trying many times. It seems like it only happens with outside participants, which might suggest some sort of timing issue (with network latency playing a factor) It’s relatively easy to reproduce with out-of-network participants.

KieranAC · January 25, 2022, 6:23pm

I reproduced the issue and checked the return status of subscribeVideo this time. It did not return any error when the issue occurred

- (ZoomSDKNormalVideoElement *)createNormalVideoElementForUserID:(NSNumber *)userID
                                                      withBounds:(NSRect)bounds {
    
    ZoomSDKNormalVideoElement *normalVideoElement = [[ZoomSDKNormalVideoElement alloc] initWithFrame:bounds];
    
    ZoomSDKError zError = [self.videoContainer createVideoElement:&normalVideoElement];
    
    if (zError != ZoomSDKError_Success) {
        AC_LOG_ERROR_ZOOM(@"Failed to create normal video element");
    }
    
    [normalVideoElement setUserid:userID.unsignedIntValue];
    
    zError = [normalVideoElement subscribeVideo:YES];
    if (zError != ZoomSDKError_Success) {
        AC_LOG_ERROR_ZOOM(@"subscribe failed with error %@", @(zError));
    }
    
    return normalVideoElement;
}

jon.zoom · January 25, 2022, 11:31pm

Hi @KieranAC,

If the affected client leaves and rejoins the meeting, all the videos for the participants load at that point.

This probably rules out the possibility of exceeding the bandwidth, unless there is a possibility of your implementation’s behavior changing based on how many participants are in a meeting when you join. The biggest impact I would expect to see here is if the size of the video elements change based on how many people are present, as this can influence which resolution the SDK receives and whether or not you have exceeded the bandwidth limit. I don’t think this is the case though, since you mentioned not seeing the onSubscribeUserFail callback when the issue is present.

We only use custom UI

I understand that the default UI is not directly relevant to your implementation, but it can help us eliminate some variables. Can you please try using the SDK sample app with the default UI to see if it’s reproducible?

I believe so yes.

This is highly unusual. If you can reproduce this aspect with the default UI, this may give us some insight into what exactly is going wrong.

Right now I am simply using constraints and set the autoresizing attributes for the view (see sample code). Is this insufficient? Should I always call ‘resize’ whenever the view is changed from its original size?

If you aren’t seeing any issues with the resolution of video you are receiving after resizing elements, I wouldn’t worry too much about this.

What is the default if this isn’t used? Does the SDK adjust the resolution dynamically? What happens if bandwidth limits are hit? Is this relevant at all for this issue, or should I ignore this API for now? I imagine you might want to manually adjust the resolution via user input or in response to connectivity issues, neither of which are an issue here?

The resolution you receive is dynamic and based on many factors, so there isn’t a default value. As mentioned previously, reaching a bandwidth limit would be reflected in an error callback. You could try using this method to limit the bandwidth to 90p for testing this issue, but unless you’re seeing the failure callback when you try subscribing, I am doubtful that this will impact the issue you’re seeing. You could try to manually adjust the bandwidth based on network connectivity, but I would not recommend that. In 99.99% of cases, it is best to just allow our internal logic to decide what resolution to deliver.

I am not using this method at all presently. I implemented the normal video elements using this as a reference: which doesn’t mention ‘subscribeVideo’. I also don’t see this being used in the sample app.

It seems likely that this could be the root cause. Is ‘subscribeVideo’ necessary? What does this API do? Should it be called while creating the normal video element? If this is necessary, it is strange that our app works 95% of the time without it?

This is not mandatory to call, and was just meant for debugging purposes. Since you mentioned that it returns ZoomSDKError_Success when the issue is present, I don’t think this is worth looking at this part further.

If you were able to rule out bandwidth limits as the cause of this based on the information above, I think we’ll need to look at the encrypted SDK logs in order to determine what’s going on here. Can you please provide that from a session where the issue is present? If you are able to reproduce with the default UI in the sample app, please provide the logs from the sample app and your own application. Otherwise, just your own application’s logs are fine.

Thanks!

KieranAC · January 28, 2022, 9:59pm

@jon.zoom

I have reproduced the issue with Zoom SDK logging enabled. Here’s a download link:
https://drive.google.com/file/d/1hrjDEWK9Vb51yPQC1bRyqYFbjNhucroV/view?usp=sharing

Steps to reproduce:

Launch app
Authenticate and log in is successful
Join a meeting (not hosting) (userID 16778240)
Other user joined (userID 16779264)

Second user has black screen normal video element

Third user joined (user ID 16780288)

Third user video is working

Terminated app

One (new) observation:

We have noticed that the video fails most often for one person. This person happens to be the host of the meeting we use daily. I can see that the host changes right after the second user joins:

-[StatusManager onHostChange:] [Line: 251] User ID: 16779264

Other logs for this user:

onUserJoin:UserID: 16779264
createNormalVideoElementForUserID: 16779264
onHostChange: UserID: 16779264
onUserVideoStatusChange:UserID: Video status updated to 0 for user 16779264
onUserVideoStatusChange:UserID: Video status updated to 1 for user 16779264
onUserVideoStatusChange:UserID: Video status updated to 1 for user 16779264
onUserInfoUpdate: UserID: 16779264
onUserActiveAudioChange to UserID: 6779264

I’m not sure what else could be different between this user and the others, but I see the issue near 100% for this user in particular in this meeting.

I am still not seeing any errors from other SDK methods.

KieranAC · January 29, 2022, 12:34am

We had somewhat of a breakthrough today with the addition of a delay before creating the normal video element when a user joins

I was able to consistently reproduce the issue with one person in particular joining a meeting (just two people in the meeting) as I mentioned in the previous post.

I made a change to add a delay (dispatch_after) like the sample code below (heavily summarized code)

- (void)onUserJoin:(NSArray *)array {
    dispatch_after(dispatch_time(DISPATCH_TIME_NOW, 1 * NSEC_PER_SEC), dispatch_get_main_queue(), ^{

        self.userList = [[[[ZoomSDK sharedSDK] getMeetingService] getMeetingActionController] getParticipantsList];

        [self reloadData]

});
}

reloadData triggers an update to an NSCollectionView which I am using the present the video elements. During this process, I create video elements for each user that joined.

Without the delay, I can reproduce the issue 100%. With the delay added, the issue is 100% resolved after numerous attempts with single users and multiple users joining all at the same time over and over.

I suspect there is some issue with creating normal video elements too soon after a user joins.

We’re going use this as a workaround for now, but it doesn’t seem like a permanent (or good) fix. If you have any ideas about what might be going on, I’d still be very interested.

jon.zoom · January 31, 2022, 9:36pm

Hi @KieranAC,

Thanks for the additional information. Is the host by chance joining the meeting through the web instead of a native client? I recently saw another issue where chat messages could not be sent to users joining from the web for a short period and am wondering if this is another side effect of the same underlying cause.

Thanks!

KieranAC · February 1, 2022, 12:08am

@jon.zoom No we only use either the Zoom client or our custom app for the meeting participants and host during out testing. I haven’t tried the web client.

jon.zoom · February 1, 2022, 8:47pm

Hi @KieranAC,

Thanks for confirming that this is a different issue. We’ll need to take a look at the logs then to see what is causing this apparent race condition. I’ll keep you updated once we have any updates.

It sounds like your workaround should be fine for now, but let me know if you notice any changes in behavior.

Thanks!

jon.zoom · February 10, 2022, 6:22pm

Hi @KieranAC,

Instead of setting up the video subscription when the participant joins the meeting, can you try subscribing when you see the onUserVideoStatusChange callback and see if that helps?

Thanks!

KieranAC · February 10, 2022, 6:48pm

I made the change as so far it looks good, but it’s very intermittent so I’ll test this for a few days internally.

My previous ‘fix’ has been consistently working since I last posted, so I was wondering myself if onUserJoin was just too early to set up the video elements.

KieranAC · February 18, 2022, 8:31am

@jon.zoom I have found a new case where this is occurring consistently

Start meeting with custom SDK app
Start meeting as host
Enable waiting room
Join meeting with second app (Zoom client)
User will join the meeting and idle in waiting room
Admit the waiting room user
User will join the meeting. Video is very often (>50% rate) blackscreened.

Result: (Two top video elements joined previously, bottom is blackscreened. The bottom also has the name of the current client user which is also incorrect)

I am creating the video elements in onUserVideoStatusChange as discussed previously.

I was able to reproduce with the previous dispatch_async workaround as well.

jon.zoom · February 23, 2022, 8:47pm

Hi @KieranAC,

Have you changed the version being used, or is this still on 5.9.1?

Thanks!

KieranAC · February 23, 2022, 9:04pm

We upgraded to 5.9.3 when it was released

jon.zoom · February 24, 2022, 9:46pm

Hi @KieranAC,

Thanks for confirming that you’ve tested on the latest version. After several attempts, I was unable to reproduce this behavior. The video is rendered correctly 100% of the time after admitting the user from the waiting room. Are you able to provide a demo application in which this issue is reproducible?

Thanks!

KieranAC · March 17, 2022, 6:42pm

I have not had a chance to set up a demo app yet.

I have been receiving reports from users that the blackscreen issue is still occurring outside of the waiting room case, although I have not seen it personally since the last fix. Using the latest SDK in that build.

jon.zoom · March 18, 2022, 8:17pm

Hi @KieranAC,

Please let me know if you’re able to reproduce this issue locally and/or provide a demo so that we can look into this further. We haven’t been able to reproduce this yet, so we don’t have a lot of information upon which we can act.

Thanks!

KieranAC · March 18, 2022, 8:47pm

One of the symptoms in this black screen issue has been the video element user name string. When a video is black, the user name is visible, but set to the current user instead of the expected user.

It seems possible that there may be some link between this issue and the other two issues I have reported.

Missing microphone icon in text field
Sometimes missing user name but video works

Do you think it’s possible those two cases may be a ‘partial failure’ with the same root cause?

All of this seems like some sort of race condition in the normal video element creation which has varying levels of failure.

Topic		Replies	Views
ZoomSDKNormalVideoElement displays no video in some cases macOS	9	610	April 1, 2021
Inconsistent black video macOS	21	1432	March 23, 2021
[Mac SDK] Unable to see participant's video during video call macOS	3	895	August 24, 2021
User Video Not Rendering Properly Windows	1	636	October 24, 2020
User name captions on normal video elements missing sometimes macOS	2	363	April 18, 2022

Inconsistent black screen video for participants on join

Related Topics