Xcode downloads client crash report with reason "index 0 beyond bounds for empty array" but the stacktraces don't contain any of my app's symbols

All the threads only contain system calls. The crashed thread only contains a single call to my app's code which is main.swift:13.

What could cause such a crash?

So, the direct issue is a combination of two issues:

• The crash itself was caused by an exception being thrown. Note that this sequence is the the standard "boilerplate" the system uses to proces exceptions and doesn't really tell you anything about why you crashed.

Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x0000000193eff720 __pthread_kill + 8 (:-1)
1   libsystem_pthread.dylib       	0x0000000193f37f70 pthread_kill + 288 (pthread.c:1721)
2   libsystem_c.dylib             	0x0000000193e44908 abort + 128 (abort.c:122)
3   libc++abi.dylib               	0x0000000193eee44c abort_message + 132 (abort_message.cpp:78)
4   libc++abi.dylib               	0x0000000193edca40 demangling_terminate_handler() + 348 (cxa_default_handlers.cpp:77)
5   libobjc.A.dylib               	0x0000000193b853e4 _objc_terminate() + 156 (objc-exception.mm:496)
6   libc++abi.dylib               	0x0000000193eed710 std::__terminate(void (*)()) + 16 (cxa_handlers.cpp:59)
7   libc++abi.dylib               	0x0000000193eed6b4 std::terminate() + 108 (cxa_handlers.cpp:88)
 

• The crash should have included an additional thread stack about thread 0 that was the backtrace of the exception that was thrown. The typical reason it would be absent is that this was a C++ exception (which we can't capture stack traces from), however, there are few other "oddities".

Here's what's "odd":

1)

This line typically comes from the ObjC exception object and is specifically describing an ObjC exception. That would imply that something weirder is going on.

Exception Reason:      *** -[__NSArray0 objectAtIndex:]: index 0 beyond bounds for empty array
 

2)

This is a "trip wire" we added relatively recently that intended to identify cases where one our background threads interfered with normal exception processing. It doesn't actually point to what went wrong, but it support the ideat that "something" weird was occurring.

Thread 6:
0   HIServices                    	0x000000019a9b0150 SOME_OTHER_THREAD_SWALLOWED_AT_LEAST_ONE_EXCEPTION + 0 (HIExceptions.m:11)
1   Foundation                    	0x00000001951e9444 __NSThread__start__ + 724 (NSThread.m:991)
2   libsystem_pthread.dylib       	0x0000000193f382e4 _pthread_start + 136 (pthread.c:931)
3   libsystem_pthread.dylib       	0x0000000193f330fc thread_start + 8 (:-1)

3)

Finally, there is this thread chunk:

Thread 2:
0   libobjc.A.dylib               	0x0000000193b66a20 getMethodNoSuper_nolock(objc_class*, objc_selector*) + 252 (objc-runtime-new.mm:7188)
1   libobjc.A.dylib               	0x0000000193b6a01c lookUpImpOrForward + 436 (objc-runtime-new.mm:7609)
2   libobjc.A.dylib               	0x0000000193b69b84 _objc_msgSend_uncached + 68
3   UserActivity                  	0x00000001a4266fb0 __39+[UAUserActivityManager defaultManager]_block_invoke_2 + 168 (UAUserActivityManager.m:201)
4   libsystem_trace.dylib         	0x0000000193c8f248 ___os_state_request_for_self_impl_block_invoke + 40 (state.c:222)
5   libdispatch.dylib             	0x0000000193d855b4 _dispatch_client_callout + 20 (object.m:576)
6   libdispatch.dylib             	0x0000000193d94e08 _dispatch_lane_barrier_sync_invoke_and_complete + 56 (queue.c:1104)
 

Two points here:

1. Specifically crashing inside the ObjC runtime like this isn't common. It typically indicates that this was what actually crashed, however, that doesn't match up with the array indexing issue referenced in #1. The other way it can happen is that there is some kind of "entanglement" with the underlying problem, which is then causing the crash to play out with very specific timing.

2. Looking at our code, the codes that's running here is specifically diagnostic data that's collected by the framework itself. That could indicate that the crash originated in something tied to the activity system, however, the code involved is also not something I can see really crashing.

With all of that context, a few followup questions:

• How often are you seeing this crash and how consistent is the log, particularly when it comes to #3 above. If you're seeing exactly the same pattern in a large number of crashes then it could be a key factor, otherwise it's probably noise.

• Do you have any other information about the crash? With difficult crashes, knowing the larger context is at least as important as the crash log itself.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for your insights. Unfortunately I don't have any more information about the crash: the crash report downloaded by Xcode is all I have. I think this is the first time I've seen this kind of crash. Whenever I see a crash report that gives me no clue about what the issue is, I have no choice but to assume that it's something unrelated to my own code, but if that's really the case, then in my opinion I shouldn't even get the crash report at all, since I cannot do anything about it. Looking into these reports only to realize that there's nothing I can do about it still takes quite some time when summing them all up. I thought I might still ask if there's anything I can do.

Thanks for your insights. Unfortunately I don't have any more information about the crash: the crash report downloaded by Xcode is all I have. I think this is the first time I've seen this kind of crash. Whenever I see a crash report that gives me no clue about what the issue is, I have no choice but to assume that it's something unrelated to my own code, but if that's really the case, then in my opinion I shouldn't even get the crash report at all, since I cannot do anything about it.

So, there are a few different things I want to say here:

• It's entirely possible for bugs in your app to cause crashes that don't contain any of your code. Simple crash ("your code crashed here") happen because your code did something wrong and immediately failed. Complex crashes (like this one) happen because your code did one or more things which created the circumstances which lead to the final failure. Both case are ultimately caused by "your code", the second case is just more complicated.

• The system have VERY little ability to determine whether or not a particular crash will be "meaningful" to you. That's partly because the basic analysis itself is hard (it's VERY close the solving the halting problem) but it's also because the system doesn't know what other information/knowledge you have.

This idea in particular:

then in my opinion I shouldn't even get the crash report at all, since I cannot do anything about it.

...is one I've very ACTIVELY argued against. At a minimum, giving you a crash log means you are at least aware that "something" is going on. I'll talk more about what your options are below, but the only thing worse than a crash you can't fix is a crash that you don't even know about.

Looking into these reports only to realize that there's nothing I can do about it still takes quite some time when summing them all up. I thought I might still ask if there's anything I can do.

First off, if you haven't already make sure you look at all of the "raw" log data, not sure Xcode's display of it. Crashpoint files are actually file packages, so you can access the raw crash logs directly using "Show Package Contents" in the Finder. While you're looking at that data, don't just look for issues in the stack data itself, but also path attention to things like the crash time or the app path (this works much better in iOS apps). One of the things you can catch this way is cases when a set of crashs logs are actually from a single/limited user and not necessarily a broad problem. As an example, I once looked at set of seemingly unrelated, very low level crashes which seemed concerning but were actually from a single user (based on the UUID in the install path) on a modified device (based on what was in the library list) which had all happened over ~3 hours (one you lined up the timestamps).

I'd also recommend looking at any other crash logs you've received and/or failure reports from users. It's not unusual for a single problem to result in multiple crash patterns (for example, based on the timing between events) and correlating those logs together may help you find the underlying problem.

Finally, think about how you get more/better information. There isn't any fixed approach for that, but it includes things like:

• Implementing app logging so you can determine when was happening in your app if/when your able to connect with a user who's experiencing the crash.

• Changing your apps implementation so that it includes clear indications of it's actual activity in the crash log.

• Making sure you've got a system in place where end users can contact you asking for help.

That last point is critical here. It's often the case that the key to fixing a critical issue isn't any particular code change, but is actually being able to connect with a user who is able to reproduce the problem and is wiling to work with you to try and fix it.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

It's entirely possible for bugs in your app to cause crashes that don't contain any of your code

Ok, but if I don't see any of my code in the stack trace, it can be quite difficult (if not infeasible) to find out what caused the crash.

Complex crashes (like this one) happen because your code did one or more things which created the circumstances which lead to the final failure

Since I don't have the means of understanding what the final failure was, I would expect that your internal code that caused the final failure would throw some meaningful error that would allow me to understand the issue, or that an error is thrown as soon as your internal code detects that the "circumstances" or preconditions are invalid.

Changing your apps implementation so that it includes clear indications of it's actual activity in the crash log.

You mean also in case that the user contacts me, right? Otherwise I wouldn't know how to make this visible in the crash report.

Making sure you've got a system in place where end users can contact you asking for help

How could I make sure of that, or how could I detect that a crash has happened or is about to happen if I have no idea what code causes it?

Ok, but if I don't see any of my code in the stack trace, it can be quite difficult (if not infeasible) to find out what caused the crash.

Yes. This kind of crash can be very difficult to find and fix.

Since I don't have the means of understanding what the final failure was, I would expect that your internal code that caused the final failure would throw some meaningful error that would allow me to understand the issue, or that an error is thrown as soon as your internal code detects that the "circumstances" or preconditions are invalid.

That's certainly what we try to do but, unfortunately, it's simply not possible for us to do that in the truly general case.

You mean also in case that the user contacts me, right? Otherwise I wouldn't know how to make this visible in the crash report.

A lot of this depends on exactly what your app does and how your app works. At the basic level, it's things like ensuring you've named very queue/thread to make sure you're pushing more data into the log. As a more complex solution you can actually use a thread to "publish" information about exactly that your app is going as it moves through it's normal operation. In concrete terms, you intentionally block a thread in a function like "DoingOperationX" and then allow that function to return once "operation X" is finished. It wastes a thread is not particularly elegant, but it can be helpful if you're able to narrow the range of possible failure points and are trying to pin things down more precisely.

It's also worth looking at this issue from the other direction. Thy crashing your app at other "points" in it's lifecycle and compare the state of the process at those time against your crash log. I don't know enough about your app to know if that will be useful or not, but I've worked with apps in that past where the crash location could be narrowed simply because the app didn't "look" like the log most of the time.

How could I make sure of that, or how could I detect that a crash has happened or is about to happen if I have no idea what code causes it?

You wouldn't. The idea here is to have a plan and investigative data in plave if/when a user does contact you. Case in point, having app logging in place so that your able to trace what your app was doing before it crashed.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for the suggestions. No user has contacted me about these kinds of issues yet, so logging wouldn't help until someone does. I don't think manipulating threads would help me, since even once I knew what thread causes the crash, I would still be looking for a needle in a haystack, without any information about the kind of function call that caused the crash. My app is an AppKit app which spawns some threads here and there with DispatchQueue.

The best (and only feasible) thing, in my opinion, would be for macOS to catch invalid state early and throw a meaningful exception.