Crash on DispatchQueue.main.sync from isolated thread

I'm troubleshooting a crash I do not understand.

I have a queue called DataQueue which never has anything dispatched to it - it's the sample buffer delegate of an AVCaptureVideoDataOutput. It can call DispatchQueue.main.sync to do some work on the main thread.

It works fine no matter what we test, but has some crashes in the field that I need to fix. Here's it crashing:

          AppleCameraDataDelegate.dataQueue
0  libsystem_kernel.dylib         0x7bdc __ulock_wait + 8
1  libdispatch.dylib              0x4a80 _dlock_wait + 52
2  libdispatch.dylib              0x486c _dispatch_thread_event_wait_slow$VARIANT$mp + 52
3  libdispatch.dylib              0x113d8 __DISPATCH_WAIT_FOR_QUEUE__ + 332
4  libdispatch.dylib              0x10ff0 _dispatch_sync_f_slow + 140

The main thread isn't doing something I asked it to, but appears to be busy:

          Thread
0  libsystem_kernel.dylib         0x71a4 __psynch_cvwait + 8
1  libsystem_pthread.dylib        0x7fd8 _pthread_cond_wait$VARIANT$mp + 1232
2  grpc                           0x2cb670 gpr_cv_wait + 131 (sync.cc:131)
3  grpc                           0x119688 grpc_core::Executor::ThreadMain(void*) + 225 (executor.cc:225)
4  grpc                           0x2e023c grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::__invoke(void*) + 146 (thd.cc:146)
5  libsystem_pthread.dylib        0x482c _pthread_start + 104
6  libsystem_pthread.dylib        0xcd8 thread_start + 8

Can anyone help me understand why this is a crash?

Answered by DTS Engineer in 859859022

So, yeah, third-party crash reporters are a bad idea, especially when trying to deal with hard problems. I talk more about that in Implementing Your Own Crash Reporter.

I presume that you have some additional metadata that makes it clear that you’re crashing in __ulock_wait, because nothing in that crash report indicates that’s the case. Regardless, you’re not crashing there, at least not really. Consider this:

(lldb) disas -n __ulock_wait
libsystem_kernel.dylib`__ulock_wait:
    0x18bf739b0 <+0>:  mov    x16, #0x203               ; =515 
    0x18bf739b4 <+4>:  svc    #0x80
    0x18bf739b8 <+8>:  b.lo   0x18bf739d8               ; <+40>

Due to the way crash reports are generated, the __ulock_wait + 8 in your crash reports means that the actual PC was at <+4>, that is, the system call instruction (svc) itself. It’s unlikely that crashed your process. Rather, your process was blocked — possibly deadlocked — and something else crashed it.

What that was is anybody’s guess. My advice is that your remove your third-party crash report, wait until you get some Apple crash reports, and then resume debugging.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

It’s hard to offer insight here based on then snippets you’ve posted. Please post a full Apple crash report. See Posting a Crash Report for advice on how to do that.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Hi Quinn attached is what I have (from firebase crashlytics).

Please note that for some reason the thread numbers are stripped out: in the one i've uploaded the main thread (#1) is the one you'll see has libc++ involved, and the thread that has crashed on DispatchQueue.main.sync is AppleCameraDataDelegate.dataQueue.

Does this help? Would using a semaphore + dispatch async work better than the dispatch sync?

... @DTS Engineer it doesn't make sense to me why a semaphore+dispatch_async would be better but maybe we should try it? we can't reproduce this ourselves, it's our biggest crash in the wild though

So, yeah, third-party crash reporters are a bad idea, especially when trying to deal with hard problems. I talk more about that in Implementing Your Own Crash Reporter.

I presume that you have some additional metadata that makes it clear that you’re crashing in __ulock_wait, because nothing in that crash report indicates that’s the case. Regardless, you’re not crashing there, at least not really. Consider this:

(lldb) disas -n __ulock_wait
libsystem_kernel.dylib`__ulock_wait:
    0x18bf739b0 <+0>:  mov    x16, #0x203               ; =515 
    0x18bf739b4 <+4>:  svc    #0x80
    0x18bf739b8 <+8>:  b.lo   0x18bf739d8               ; <+40>

Due to the way crash reports are generated, the __ulock_wait + 8 in your crash reports means that the actual PC was at <+4>, that is, the system call instruction (svc) itself. It’s unlikely that crashed your process. Rather, your process was blocked — possibly deadlocked — and something else crashed it.

What that was is anybody’s guess. My advice is that your remove your third-party crash report, wait until you get some Apple crash reports, and then resume debugging.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Crash on DispatchQueue.main.sync from isolated thread
 
 
Q