Diagnosing MetricKit crash reports with EXC_CRASH / SIGKILL with no termination reason

I'm collecting and uploading MetricKit crash diagnostic reports from my app in the field, and the huge majority of the reports I get back have exception type EXC_CRASH, signal SIGKILL with no terminationReason. The report lists thread 0 as attributed, but it doesn't have any of my code active. Here's an example symbolicated trace of thread 0:

mach_msg2_trap (in libsystem_kernel.dylib) + 8
mach_msg2_internal (in libsystem_kernel.dylib) + 76
mach_msg_overwrite (in libsystem_kernel.dylib) + 424
mach_msg (in libsystem_kernel.dylib) + 24
__CFRunLoopServiceMachPort (in CoreFoundation) + 160
__CFRunLoopRun (in CoreFoundation) + 1188
_CFRunLoopRunSpecificWithOptions (in CoreFoundation) + 532
GSEventRunModal (in GraphicsServices) + 120
-[UIApplication _run] (in UIKitCore) + 796
UIApplicationMain (in UIKitCore) + 332
main (in BRFree) (main.m:0)
start (in dyld) + 6928

There is code from my app in other threads, but they're all waiting on NSConditions and not doing anything.

I thought they might be 0xdead10cc crashes, but there are other reports that have "terminationReason": "Namespace RUNNINGBOARD, Code 0xdead10cc".

Can anyone help me narrow down what could be causing these crashes?

I would file a DTS support ticket, but it won't let me without either a request from someone at Apple or a focused test project. If an engineer sees this thread and gives me a green light for a code-level support request, I can provide a sampling of the MetricKit reports.

Answered by DTS Engineer in 903762022
I’m still researching options and I’ll reply here when I know more.

So it’s hard to say how I came to this conclusion, but I suspect that the majority of your crashes will have a backtrace that leads into SQLite:

0  libsqlite3.dylib           … wherePathSolver + 952
1  libsqlite3.dylib           … sqlite3WhereBegin + 2988
2  libsqlite3.dylib           … sqlite3Select + 16536
3  libsqlite3.dylib           … yy_reduce + 5136
4  libsqlite3.dylib           … sqlite3Parser + 36
5  libsqlite3.dylib           … sqlite3RunParser + 796
6  libsqlite3.dylib           … sqlite3Prepare + 548
7  libsqlite3.dylib           … sqlite3LockAndPrepare + 224
8  BRFree                     …
…
29 BRFree                     …
30 libswift_Concurrency.dylib … completeTaskWithClosure(swift::AsyncContext*, swift::SwiftError*) + 1

This is from thread 11 of your first crash report, btw.

This suggests you’re hitting the 0xdead10cc termination, as explained in EXC_CRASH (SIGKILL).

It would be nice if MetricKit included that code in its MXCrashDiagnostic, and I encourage you to file an enhancement request requesting just that (if you do file this ER, please post your bug number, just for the record).

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Normally I’d ask you to post the full crash report, but that’s tricky here because MetricKit doesn’t give you crash reports in a standard format. How does your app capture these MetricKit reports? Do you just render the whole thing to JSON and upload that?

If so, can you post an example of that JSON? Put it in a text file and attach that to your reply. Posting a Crash Report explains that part of the process.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I'm rendering the whole thing to JSON. Here's an example.

That one actually has some of my app's symbols active on a background thread - here's an example of one that doesn't.

Thanks for those.

Both crashes show that the main thread crashed with a backtrace like this:

0  libsystem_kernel.dylib  0x24929dcd4 mach_msg2_trap + 8
1  libsystem_kernel.dylib  0x2492a130c mach_msg2_internal + 76
2  libsystem_kernel.dylib  0x2492a122c mach_msg2 + 4
3  libsystem_kernel.dylib  0x2492a122c mach_msg_overwrite + 424
4  libsystem_kernel.dylib  0x2492a1078 mach_msg + 24
5  CoreFoundation          0x1998cc344 __CFRunLoopServiceMachPort + 160
6  CoreFoundation          0x199896310 __CFRunLoopRun + 1188
7  CoreFoundation          0x19989554c _CFRunLoopRunSpecificWithOptions + 532
8  GraphicsServices        0x23f87d498 GSEventRunModal + 120
9  UIKitCore               0x19f5c9670 -[UIApplication _run] + 796
10 UIKitCore               0x19f534158 UIApplicationMain + 332
11 BRFree                  0x1028d5b78
12 dyld                    0x1964a1c1c start + 6928

This is pretty standard for crashes that aren’t really crashes, for example, when your app is terminated by the watchdog. In most such cases MetricKit captures relevant info in the terminationReason field, but that hasn’t happened here )-:

I’m still researching options and I’ll reply here when I know more.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Accepted Answer
I’m still researching options and I’ll reply here when I know more.

So it’s hard to say how I came to this conclusion, but I suspect that the majority of your crashes will have a backtrace that leads into SQLite:

0  libsqlite3.dylib           … wherePathSolver + 952
1  libsqlite3.dylib           … sqlite3WhereBegin + 2988
2  libsqlite3.dylib           … sqlite3Select + 16536
3  libsqlite3.dylib           … yy_reduce + 5136
4  libsqlite3.dylib           … sqlite3Parser + 36
5  libsqlite3.dylib           … sqlite3RunParser + 796
6  libsqlite3.dylib           … sqlite3Prepare + 548
7  libsqlite3.dylib           … sqlite3LockAndPrepare + 224
8  BRFree                     …
…
29 BRFree                     …
30 libswift_Concurrency.dylib … completeTaskWithClosure(swift::AsyncContext*, swift::SwiftError*) + 1

This is from thread 11 of your first crash report, btw.

This suggests you’re hitting the 0xdead10cc termination, as explained in EXC_CRASH (SIGKILL).

It would be nice if MetricKit included that code in its MXCrashDiagnostic, and I encourage you to file an enhancement request requesting just that (if you do file this ER, please post your bug number, just for the record).

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

FB24609180

Thanks.

Oh, and a quick look at that bug suggests that it might be worth checking again on iOS 27 beta, because 27.0b5 and later have improvements in this space. (I’d do it myself but I’m super busy right now.)

But for now I'll proceed under that assumption and see how far I get.

Cool.

Also, are you getting crash reports from Xcode organiser? If you are, those should confirm my theory.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

If you mean that MetricKit on iOS 27 is better about including terminationReason, I'll watch for that. I do also plan to support the new MetricKit.

As for the Xcode organizer, its top entry in Crashes under Reports does look like a 0xdead10cc termination - it carries that code if I go look at the raw report on disk (side note - the Organizer does not expose these terminations well - FB12292887). But on the Terminations screen under Metrics, it shows background terminations as being 98% System Pressure, and 1% File Lock and 1% Other. I don't believe 1% of terminations works out to enough to explain the numbers of reports I've been seeing, and certainly not to the count shown on that top entry in Crashes under Reports.

As another signal, the original impetus for me chasing this is that the Crashes metric in App Store Connect for my app shows a higher crash count per day than I expect, not matching any other reporting source I have. It's why I started uploading all my MetricKit reports from devices in the field.

I don't believe 1% of terminations works out to enough to explain the numbers of reports I've been seeing

Agreed. After posting my response yesterday I had a chat with DTS’s crash report experts [1] and they pointed me at this quote from the docs:

The following crash report types aren’t available through the Crashes organizer …

  • Watchdog events, such as those from slow app launch times
  • Invalid code-signature crashes
  • Thermal events, where a device overheats because an app uses too much CPU
  • Jetsam events, where an app has high memory use

While this list doesn’t specifically mention 0xdead10cc terminations, it seems that they fall into the same trap (r. 110550085)-:

If you mean that MetricKit on iOS 27 is better …

Right. This is what I see for a 0xdead10cc crash on iOS 27.0b8:

{
  …
  "crashDiagnostics": [
    {
      …
      "diagnosticMetaData": {
        …
        "terminationReason": "Namespace RUNNINGBOARD, Code 0xdead10cc",
        …
        "osVersion": "iPhone OS 27.0 (24A5430a)",
        …
      }
    }
  ]
}

Huzzah!

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Well, we’re all experts in diagnosing crashing from crash reports (-: but I’m talking about the experts in how crash reports are handled by App Store Connect.

Diagnosing MetricKit crash reports with EXC_CRASH / SIGKILL with no termination reason
 
 
Q