Video freezing when setting virtual background

Description
Setting virtual background causes video to freeze, and background isn’t applied

Which macOS Meeting SDK version?
5.10.3 and 5.10.1 (tested both)

To Reproduce(If applicable)

This is some quick and dirty code I wrote to try out the feature by toggling between blur / not blur modes.

NSArray <ZoomSDKVirtualBGImageInfo *> *backgroundItemList = [virtualBackgroundSettings getBGItemList];
    
    zError = ZoomSDKError_Success;
    
    static bool flag = NO;
    if (flag) {
        // Should be blur. This freezes the video
        zError = [virtualBackgroundSettings useBGItem:[backgroundItemList objectAtIndex:1]];
    } else {
       // Should be None. This will un-freeze the video
        zError = [virtualBackgroundSettings useBGItem:[backgroundItemList objectAtIndex:0]];
    }
    flag = !flag;

Screenshots
If applicable, add screenshots to help explain your problem.

Device (please complete the following information):
Apple MacBook Pro (13-inch, M1) 12.2.1

Additional context

I also tried uploading an image using

ZoomSDKError zError = [virtualBackgroundSettings addBGImage:@"/Users/kieran/Desktop/toronto.jpeg"];

and then trying that with

zError = [virtualBackgroundSettings useBGItem:[backgroundItemList objectAtIndex:2]]

and this also freezes until ‘none’ is set or video is restarted.

Based on some other forum threads I also tried a few other things with no effect

  • Disabling hardened runtime
  • Setting team identifier before stating the SDK due to re-signed Zoom binaries

There are no errors from any of the SDK calls above

This post was flagged by the community and is temporarily hidden.

Hi @KieranAC,

It looks like you may be trying to access the virtual backgrounds too soon. Are you checking any of the callbacks in ZoomSDKVirtualBackgroundSettingDelegate before trying to set a virtual background?

Thanks!

Does that apply to the two “built-in” BG items? Referring to “NewUI_Settings_None” and “Blur”

image

I can see isSelected = true for the BG item with name ‘Blur’ but the video is frozen at this point. I can then select “NewUI_Settings_None” and the video is un-frozen.

Hi @KieranAC,

Looking into this further, the behavior around Blur may have changed, so this callback may not apply to the two items you’ve mentioned. That being said, there does seem to be some unexpected behavior related to the blur setting, even when trying to use it with the default meeting UI. We’ll need to investigate this and get back to you. As far as I can tell this issue is internal to the SDK, so there probably would not be a workaround available in the meantime unfortunately.

Thanks!

Thanks for looking into it.

I think virtual backgrounds (with an image) do work, it’s just a bit more work to hook it all up. I will finish that part up and look for an update in a future SDK release on the blur-screen part.

@jon.lieblich I discovered another issue which explains my confusion around the ‘addBGImage’ functionality with custom backgrounds.

I noticed the same freezing effect in my original post, but later saw this working (as I mentioned in my followup)

However, using a custom background image ONLY seems to work if the setUsingGreenScreen setting is enabled. When this is disabled, setting a background image has the same freezing effect. The Blur selection freezes regardless of this setting.

I implemented a checkbox for the greenscreen functionality and had it checked when I made my followup post above.

  • (ZoomSDKError)setUsingGreenScreen:( BOOL )bUse;

I can provide additional information if needed, but it’s pretty consistent behaviour on my end and should be reproducible similar to the blur functionality.

Hi @KieranAC,

Can you please confirm whether you are re-signing the SDK and providing a teamIdentifier when initializing the SDK?

Thanks!

I was not setting the teamIdentifier previously, but I was resigning the *.app files in the SDK bundle. I tried adding the teamIdentifier and re-built but I still get the same behaviour. I tried a debug build but also signed + notarized a release build with no change.

DEVELOPMENT_TEAM = "8********7"
let params = ZoomSDKInitParams()
params.needCustomizedUI = true
params.teamIdentifier = "8********7"

DIR=$1
SIGN=$2

echo $DIR
echo "Re-signing *.app files in directory \"$DIR\" with ID \"$SIGN\"..."

for file in $DIR/*; do
  if [ "${file: -4}" == ".app" ]; then
    echo "Signing file $file..."
    codesign --force --verify --timestamp --verbose --options=runtime --sign $SIGN $file
  fi
done

Hi @KieranAC,

Thanks for confirming that the issue is still present after setting the teamIdentifier. Can you please provide the SDK logs so we can get some additional insight into what’s going on?

Thanks!

@jon.lieblich

Steps:

  1. Launch app
  2. Wait for Zoom authentication / initialization
  3. Open settings
  4. Turn on blur (initial state at launch was ‘none’ and green screen is enabled)

Hi @KieranAC,

Looking into this further, it does appear to be an issue with signing. In order to resolve this, your app and all of the libraries must be re-signed. Can you please try that and see if it works?

Thanks!

The previous logs I sent were from a debug build. It turns out that this is actually working in a fully signed + notarized release build, but not in a debug build.

I am re-signing all files in the SDK bundle, including embedded frameworks within the *.app files.

There’s no difference in my build steps between an archive build and a debug build, so I’m a bit puzzled.

Do you have any idea why this might be the case for debug builds?

Also, if this has something to do with signing, why do virtual backgrounds work while green screen is enabled (in debug mode) but the video freezes when green screen is disabled?

I’m happy that it’s working in a release build at least, and that unblocks this feature for now, but it would be interesting to figure out the debug/release quirk.

Hi @KieranAC,

That’s great that it’s at least working in a release build! I’m not sure why the build type would matter if the signing is being done the exact same way. We’ll look into it and see if we can provide any additional guidance.

Also, if this has something to do with signing, why do virtual backgrounds work while green screen is enabled (in debug mode) but the video freezes when green screen is disabled?

I think this may fall into the category of undefined behavior, but since running without signing is not something we usually test, we likely would not be able to investigate that further unless there were a significant impact of this behavior.

Thanks!

Hi @KieranAC,

We were able to confirm that there will not be any difference in how the SDK verifies signing between release and debug variants. This means that there is likely something happening differently between the two in your environment.

Thanks!