*.ips CrashReport not always available when dext crashes

Question

Created 4d

Replies 1

Boosts 3

Participants 2

While developing our driver, we've noticed that the *.ips report that contains the stacktrace of the crash is not always generated. I'm wondering why this report may not get generated, or if there's anything specific to do to guarantee it gets generated.

Answered by DTS Engineer in 861988022

While developing our driver, we've noticed that the *.ips report that contains the stack trace of the crash is not always generated.

To be clear, you're talking about DEXT level users space crashes, NOT kernel panics? I've assumed below that you're dealing with standard crashes, but please let me know if I've misunderstood.

I'm wondering why this report may not get generated.

So, the first step here is to figure out what actually happened, which means digging through the console log. In general, there are multiple components tracking (notably, launchd) any system component (including DEXTs) and they'll be notified of your process's death, including how/why it "died". If you're on development hardware then there are some tips in "Your Friend the System Log" that can be useful for quickly getting at this sort of data; however, I'd generally trigger a sysdiagnose and use that console archive. The big advantage of the sysdiagnose is that you capture "everything", which means you're not going to miss data if something unexpected is going on. Take a look at "Using a Sysdiagnose Log to Debug a Hard-to-Reproduce Problem" for more guidance on that process.

In any case, what you're going to find is ultimately one of two things:

A log wasn't generated because your app didn't crash, it exited normally by calling "exit()".
Something really weird/strange is going on. I'm not going to talk about this case, but if you find something strange in the console, then please post the details of that and I'll see what I can determine.

...with #1 being VASTLY more common case. Unfortunately, why exactly called exit is hard to guess at. Most of the cases I've seen were just misunderstandings of the normal DEXT lifecycle, and the solution for this is generally just better logging in your DEXT so that you can "see" what's happening. There are some cases where the DriverKit infrastructure "itself" will intentionally call exit, but those should either all be cases you can't trigger* or are actually part of the normal DEXT teardown cycle. It's possible there are some other load checks which can exit; however, by design, "exit" itself is not defined by the DriverKit SDK, so our family-level drivers should never call exit. Note that every early exit path should also log to the console, so if it came from DriverKit, there should be data in the console to tell you "why".

*For example, one of them is an argument count check at launch which doesn't matter since you don't control the arguments your DEXT is launched with.

or if there's anything specific to do to guarantee it gets generated.

Unfortunately, no, not currently, and if you have a moment, I'd appreciate you filing a bug on this issue and then posting the bug number back here. What I'd normally suggest here is adding an "atexit" handler which directly called "abort()", effectively converting every exit() into a crash. That's a very helpful tool for exactly this situation; however, atexit is not part of the DriverKit SDK, which means you can't call it.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Boost

Answer 1

DTS Engineer OP

Apple

1d

Recommended

While developing our driver, we've noticed that the *.ips report that contains the stack trace of the crash is not always generated.

To be clear, you're talking about DEXT level users space crashes, NOT kernel panics? I've assumed below that you're dealing with standard crashes, but please let me know if I've misunderstood.

I'm wondering why this report may not get generated.

So, the first step here is to figure out what actually happened, which means digging through the console log. In general, there are multiple components tracking (notably, launchd) any system component (including DEXTs) and they'll be notified of your process's death, including how/why it "died". If you're on development hardware then there are some tips in "Your Friend the System Log" that can be useful for quickly getting at this sort of data; however, I'd generally trigger a sysdiagnose and use that console archive. The big advantage of the sysdiagnose is that you capture "everything", which means you're not going to miss data if something unexpected is going on. Take a look at "Using a Sysdiagnose Log to Debug a Hard-to-Reproduce Problem" for more guidance on that process.

In any case, what you're going to find is ultimately one of two things:

A log wasn't generated because your app didn't crash, it exited normally by calling "exit()".
Something really weird/strange is going on. I'm not going to talk about this case, but if you find something strange in the console, then please post the details of that and I'll see what I can determine.

...with #1 being VASTLY more common case. Unfortunately, why exactly called exit is hard to guess at. Most of the cases I've seen were just misunderstandings of the normal DEXT lifecycle, and the solution for this is generally just better logging in your DEXT so that you can "see" what's happening. There are some cases where the DriverKit infrastructure "itself" will intentionally call exit, but those should either all be cases you can't trigger* or are actually part of the normal DEXT teardown cycle. It's possible there are some other load checks which can exit; however, by design, "exit" itself is not defined by the DriverKit SDK, so our family-level drivers should never call exit. Note that every early exit path should also log to the console, so if it came from DriverKit, there should be data in the console to tell you "why".

*For example, one of them is an argument count check at launch which doesn't matter since you don't control the arguments your DEXT is launched with.

or if there's anything specific to do to guarantee it gets generated.

Unfortunately, no, not currently, and if you have a moment, I'd appreciate you filing a bug on this issue and then posting the bug number back here. What I'd normally suggest here is adding an "atexit" handler which directly called "abort()", effectively converting every exit() into a crash. That's a very helpful tool for exactly this situation; however, atexit is not part of the DriverKit SDK, which means you can't call it.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

0