Custom Video Element not respecting Group HD feature

Hello

Working with latest SDK for macOS

If I:

1/ start a meeting where the creator has group HD enabled
2/ join with one other client
3/ create a video element for the remote client and size it to 1080p

OUTCOME:
The remote client shows as sending 1080p video as expected, the SDK app shows as receiving 1080p video as expected

however if I then:
4/ add a third user to the meeting

OUTCOME
the remote client which was previously showing as sending 1080p now drops down to 360p and it is not possible to get the feed back up to 1080p

however if I then
5/ have the third user leave the meeting, dropping the count of users back to 2

OUTCOME
the remote client bumps back up to 1080

NOTE
that in the same call with >= 3 users present, if I full screen pin a user then the vanilla clients can send and receive 1080p, and it seems like the custom UI app can send 1080p, so the meeting settings are definitely correct

SUMMARY
this suggests that the custom UI app is treating the meeting like a normal zoom call without group HD enabled, even though vanilla clients in the same call are able to send and receive the 1080p feed.

Is there something I should be doing to force the custom video elements to send and receive 1080p? Or is this a bug in the SDK?

3 Likes

This one is going to be pretty critical for us to sort out quickly because it invalidates the custom UI platform for professional projects while this limit is in place, unless there is a patch or workaround.

1 Like

Hi @liminal_andy and @richard1, thanks for bringing this to our attention.

So it sounds like there are two areas of behavior here that are of concern:

  1. The full HD stream is no longer available after a third attendee joins the meeting
  2. Full HD capabilities do not return in a custom UI after going back down to 2 participants.

When the third participant joins, I would expect the remote usersā€™ streams to scale down to 720p rather than 360p, so it seems that there are actually two problems with #1.

The first thing worth checking is the height of the individual views. If the height goes below a certain amount, the capability will be limited. (Note on this, I am working on confirming the height values with the team since I noticed an inconsistency and will share those here after theyā€™ve been confirmed)

Next, if cleanVideoElement is not called, the SDK may still register as having that third participant and continue restricting the max resolution (I donā€™t think this is the issue if you are still stuck at 1080p with only one remote video subscription).

If you feel confident that these options are definitely not the cause of the behavior you are seeing, please send the logs over so that we can investigate further.

Additionally worth noting that there was a change in the latest SDK version released today (which includes M1 support! :tada: ) related to video resolution, so itā€™s possible that behavior around this could have changed.

Thanks!

Thanks Jon - thatā€™s massively helpful. Iā€™ll have a play with the new sdk and see what that does

Just to clarify one thing, we were getting a reversion to the HD video when the user count dropped back to two, apologies if I wasnā€™t clear!

Richard

Glad I could help!

Thank you for clarifying. If the information from my previous reply is not enough to resolve this issue, knowing that it is reverting back to 720p will be very useful information to have. :slightly_smiling_face:

OK So I think Iā€™ve worked out whatā€™s going on - the below is all with the newest version of the SDK which was just releasedā€¦

1/ To get the video to stream at the required resolution you have to resize the view to an odd size. I donā€™t know if the size is based on anything specific but I am finding, for example that I need a height of 899+pixels for 720p, 539+pixels for 360p and 350+pixels for 180p

2/ In a non-HD account call, when the call only has two users a ā€œnormal video elementā€ will give you a resolution up to 720p until a third user joins, once this third user is in you can no longer get above 360p on this view, even when the call drops back down to two users

3/ in an HD account call 2/ holds true but with the addition that I believe when the call drops back to two you regain the ability to get the view up to 720p resolution

4/ The only way to get a 1080p feed is to use an ā€œActive Video Elementā€ - this shows either the current active video user, or the most recently spotlit user (if there is one), you donā€™t appear to be able to pin to this or have any control over what it shows

Obviously this is a significant limitation of a custom UI client over the vanilla version. Ideally I would be able to create more than one custom view, and for these views to respect pin commands (so one view would be seen as on screen 1, the other on screen 2), as well as the ability to select active speaker if required

@jon.zoom so in reference to the tests above by Richard, what options do we have for moving specific people to surfaces where we can grab them at 1080p (without spotlighting the whole call)? On the vanilla client as well as in the Zoom UI version of the SDK, you can roll both surfaces to speaker view and then pin to either, and those surfaces can receive 1080p video each. Can the same be achieved by any means in the custom UI?

Hi @richard1 and @liminal_andy,

Thanks for the additional info. Since this does not appear to match the behavior of the client or default Zoom UI, it is possible that there is an issue within the SDK. The fact that regular HD is not allowed through the SDK when there are 3 total participants (self + two remote users) does not match our publicly documented behavior for Group HD, so we will need to further investigate this behavior and let you know as soon as we have any updates.

Thanks!

Ok @jon.lieblich , thanks for looking into that for us. Regarding the ability to pin to active speaker surfaces, do you know whatā€™s going on there? Is there a workaround for that? We could temporarily patch together the functionality we need if such a thing was possible.

Hi @liminal_andy,

I do not believe that the active speaker view is meant to be compatible with the spotlight feature in custom UI mode, but will let you know if I see anything indicating otherwise.

Thanks!

So we do have the spotlight currently taking over the active speaker view, but not the pin. So I guess my question is if the presence of the spotlit user on the speaker screen is a bug or if the lack of pin ability is the bug.

Hi @liminal_andy,

Sorry about that! I seem to have gotten my wires crossed when I responded yesterday. :sweat_smile:

We are looking into whether or not pinning is currently possible with custom UI. If it is intentionally not supported, would you find this valuable enough to submit a feature request, or are you just trying to use it temporarily as a workaround for the increased video quality?

Thanks!

Hey Jon, no worries. At the end of the day, the main thing I need is the ability to route a subset of users (ideally between 2-4) to a set of locations where I can grab each of their videos at full quality. Pinning seems like a natural method that reflects how the Client UI is built, but Iā€™m really open to any system that can achieve this functionality.

We are mainly concerned with finding ways to get high quality clean feeds of individual users.

If I understand the design intent of the sdk then I would think the way to do it would be to allow the creation of a limited number of ā€˜High quality video elementsā€™ to which users could be attached. Alternatively it could be a case of being able to create a number of active video views, with the added ability of being able to pin to them. This would allow for the features of the vanilla app to be replicated in custom UI.

Thank you @liminal_andy and @richard1 for the additional context.

I think the use of individual video elements for each attendee would probably be the best approach for your use case as it provides the most flexibility and control.

That being said, as we continue to investigate this I want to make sure that we are properly setting expectations. Constraints exist outside of the SDK which prevent it from receiving more than one 1080p stream or two 720p streams from other users in the meeting (not including your own local video). You also cannot display a combination of 1080p and 720p elements simultaneously. By the time you get to three remote users, you are capped at 360p.

Finally, just for comparisonā€™s sake, can you please provide the height of the following:

  • The active video element in which you are able to receive 1080p
  • The largest height value of the normal video element where you are only receiving 720p

Thanks!

Hi Jon, on the point about multiple 1080p feeds, Zoom Client SDK apps running with dual monitor mode enabled can pull two simultaneous 1080p feeds when a user is pinned to each display, as can the vanilla zoom client. Iā€™ll try to verify that again in the latest builds but that used to be what we were observing.

Hi @liminal_andy,

Thatā€™s an interesting point regarding dual monitor mode. Just to make sure, when you are seeing this behavior, can you please confirm that both of the pinned users are remote users and not the same user that you joined the meeting as?

Thanks!

Correct, the circumstances are:

-Hosted on a meeting that has been approved and set to 1080p
-5 contributors with 1080 capable networks and cameras
-ZoomSDK Client UI app in meeting, running in dual monitor mode.
-Pin set to two of these users, one to each display
-Each user sees their contributing resolution bumps to 1080p

Great, thanks for confirming. Iā€™ll let you know as soon as we have any updates on this.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.