Process hang when it is about to exit

Meeting SDK Type and Version

Windows x86 6.1.0.41187

Description
After we upgraded from version 5.16.10.26236 to 6.1.0.41187, the process randomly hang in our production environment.

Error?
In all the repro, the last few MEETING_STATUS transitions are always:

MEETING_STATUS_CONNECTING => MEETING_STATUS_WAITINGFORHOST => (a few minutes later) => MEETING_STATUS_CONNECTING

Troubleshooting Routes

How To Reproduce
I don’t know the exact steps to reproduce the issue. It happened randomly in our production environment. I managed to get memory dump from our production server. The process contains only one thread. That thread was stuck in a Meeting SDK routine in the context of RtlExitUserProcess with the following call stack. It looks like the SDK was trying to do some cleanup when the process was about to exit.

 # ChildEBP RetAddr      Args to Child
00 005981cc 6e9da169     00000324 00000000 00000000 ntdll_771a0000!NtWaitForSingleObject+0xc
01 00598240 6e9da0c2     00000324 ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x99
02 00598254 552670a3     00000324 ffffffff 0059829c KERNELBASE!WaitForSingleObject+0x12
WARNING: Stack unwind information not available. Following frames may be wrong.
03 00598264 55271ce7     ffffffff c3be4d39 0726a020 util!ssb::event_t::wait+0x13
04 0059829c 55279087     10e43050 00000000 ffffffff util!ssb::o2o_msg_queue_t::send_msg+0xf7
05 005982bc 55279022     10d9761c 10e43050 00000000 util!ssb::thread_wrapper_t::send_msg_i+0x57
06 005982e4 0b73b1bb     10e43050 00000000 ffffffff util!ssb::thread_wrapper_t::send_msg+0xb2
07 00598b4c 0b7bcf56     f99465a2 08e1f598 0b7bca70 ssb_sdk!TermModule+0xaadb
08 005993f0 0a86ced3     558a7804 00000002 00000000 ssb_sdk!TermModule+0x8c876
09 00599540 0a854902     00000004 00000000 00000000 zVideoApp!TermModule+0x1eee43
0a 00599654 0a688a90     00000000 9220803a 08e1f1f0 zVideoApp!TermModule+0x1d6872
0b 00599734 554b69d3     00000000 42af0741 02906b28 zVideoApp!TermModule+0xaa00
0c 005998dc 55a720e7     00000001 e7051723 55b83b58 zoom_meeting_bridge!GetGlobalBusinessHelper+0x5c9c3
0d 005999bc 55a3d376     00000000 55b83b18 55b83b14 sdk!Ordinal4+0x3bcc7
0e 005999d8 55a3b91e     e7051763 55b83ac0 028a9b64 sdk!Ordinal4+0x6f56
0f 005999fc 55ac4f12     e7051483 55b13dd0 028a9b64 sdk!Ordinal4+0x54fe
10 00599a1c 712cbdc2     57a018b9 00000001 00000001 sdk!Ordinal4+0x8eaf2
11 00599a58 712cbcce     57a0186d 00000001 00000001 ucrtbase!<lambda_f03950bc5685219e0bcd2087efbe011e>::operator()+0xc2
12 00599a8c 712cbc9a     00599aa0 00599aa4 00599aa8 ucrtbase!__crt_seh_guarded_call<int>::operator()<<lambda_69a2805e680e0e292e8ba93315fe43a8>,<lambda_f03950bc5685219e0bcd2087efbe011e> &,<lambda_03fcd07e894ec930e3f35da366ca99d6> >+0x30
13 00599aac 55afa115     55b76d30 55afa74c e7051473 ucrtbase!_execute_onexit_table+0x2a
14 00599aec 55afa5fe     00000001 00599b38 55afa86e sdk!RetrieveCustomizedResourceHelper+0x13da5
15 00599af8 55afa86e     55a00000 00000000 00000001 sdk!RetrieveCustomizedResourceHelper+0x1428e
16 00599b38 55afa90c     55a00000 00000000 00000001 sdk!RetrieveCustomizedResourceHelper+0x144fe
17 00599b4c 77211656     55a00000 00000000 00000001 sdk!RetrieveCustomizedResourceHelper+0x1459c
18 00599b6c 771d815c     55afa8f0 55a00000 00000000 ntdll_771a0000!LdrxCallInitRoutine+0x16
19 00599bb8 771e4565     00000000 00000001 b1a75b91 ntdll_771a0000!LdrpCallInitRoutine+0x55
1a 00599c58 77201ae5     00000000 00000003 00000000 ntdll_771a0000!LdrShutdownProcess+0x165
1b 00599d28 6f1155b2     00000003 77e8f3b0 ffffffff ntdll_771a0000!RtlExitUserProcess+0xb5
1c 00599d3c 712d7e67     00000003 00000003 712d7d50 kernel32!ExitProcessImplementation+0x12
1d 00599d48 712d7d50     00000003 00599d68 00599d78 ucrtbase!exit_or_terminate_process+0x20
1e 00599d70 71337962     00000000 00599d9c 71335b51 ucrtbase!common_exit+0x5c
1f 00599d7c 71335b51     00000003 54d27bd0 558db920 ucrtbase!_Exit+0x12
20 00599d9c 5582f0ef     550f3c42 fffffffe 00000000 ucrtbase!abort+0x61
21 0059f524 5582f503     00000000 00000113 00006076 Cmmlib!Cmm::CTimerProc::_OnTimer+0x9f
22 0059f54c 6e46639b     00000000 00000113 00006076 Cmmlib!Cmm::CTimerID::MyTimerProcCallback+0x53
23 0059f578 6e454d9c     5582f4b0 00000000 00000113 user32!_InternalCallWinProc+0x2b
24 0059f654 6e45604c     5582f4b0 00000000 00000113 user32!UserCallWinProc+0x25c
25 0059f6c0 6e455d40     00000113 0059f71c 009f0244 user32!DispatchMessageWorker+0x2fc
*** WARNING: Unable to verify checksum for zoom_bot.exe
26 0059f6cc 009f0244     0059f6e4 3a532dcc 0059f8c8 user32!DispatchMessageW+0x10
27 0059f71c 009cac56     00000000 3a532d1c 00000028 zoom_bot+0x1f0244
28 0059f7cc 009cbc24     0059f8c8 3a532174 0287a850 zoom_bot+0x1cac56
29 0059fba4 01548f54     00000017 02865e58 0287a850 zoom_bot+0x1cbc24
2a 0059fbec 6f1105c9     003f3000 6f1105b0 0059fc58 zoom_bot+0xd48f54
2b 0059fbfc 772078bd     003f3000 b1a73b91 00000000 kernel32!BaseThreadInitThunk+0x19
2c 0059fc58 7720788d     ffffffff 77226717 00000000 ntdll_771a0000!__RtlUserThreadStart+0x2f
2d 0059fc68 00000000     01548fdc 003f3000 00000000 ntdll_771a0000!_RtlUserThreadStart+0x1b

It looks like process hang is just the symptom. The bigger issue is why Cmmlib!Cmm::CTimerProc::_OnTimer called abort.

Could someone from Zoom help this issue? I found a consistent repro

Prerequisites

  1. the meeting host enables the waiting room
  2. the meeting host customize the waiting room with a video clip

Repro Steps

  1. start the meeting SDK bot in a Windows docker container and join the meeting
  2. meeting host starts the meeting
  3. the meeting SDK bot will hang when the host is prompted that the meeting SDK bot is in the waiting room