Triage TestFlight crashes in Xcode Organizer

Hello. Welcome to Triaging TestFlight crashes with Xcode Organizer. My name is Matt, and I'm an Xcode engineer.

Today I'm going to show you some of the new tools we have built into the Xcode Organizer that will help you triage crashes across all of your app's targets and how to leverage feedback from your TestFlight users so you can get crash fixes into your builds faster.

Then I'm going to show you the new Terminations Organizer and advancements we've made in MetricKit.

If your app is utilizing TestFlight to test your beta versions, you'll know that crash feedback from your testers can be really valuable. When a tester writes some feedback after they have hit a crash, you can view that feedback in App Store Connect and download the crash log to start analyzing what happened. Later on, I'm going to show you the changes we made in Xcode that will supercharge this workflow.

First, I want to give a quick overview of the features available before Xcode 13.

In the Crashes Organizer, we support crashes for both apps on the App Store and TestFlight.

We show your app's crashes across all of our platforms, including crashes from your App Clip and app extensions.

The Organizer ranks your issues to show you the highest priority crashes and shows you how those crashes are distributed across different devices and OS versions. And once you've decided to start attacking that crash, you can open it directly in your Xcode project. All of these great features work out of the box by simply signing into Xcode with your developer account. The journey of making a great app can be long and bumpy at times, because once you ship it, customers always seem to break it in ways that you didn't even know were possible. In Xcode 13, we wanted to make the Crashes Organizer even better so you can continue to get closer to the goal of no crashes. This set us on a mission to make crash log delivery almost instant, provide richer data about your crashes, and give you more context to help you fix those hard-to-reproduce issues. I am thrilled to show you all of the incredible new features we've added to Xcode 13 in the Crashes Organizer. There are six major improvements that I'm going to share with you. First, speedy delivery of your TestFlight crashes.

Now when any of your testers crash a TestFlight build, we will deliver that crash to you in the Organizer just moments later. This quick crash delivery now allows you to view all of your TestFlight crashes from the last day, and you will now have an hourly graph telling you when the selected crash occurred. In Xcode 13, we're now making a year's worth of crash history available to you.

You can now filter crashes by time period to see all of the crashes that have occurred in the last year. Because we can now show you a year's worth of crashes, you will see a time distribution in the inspector's graph area showing you the prevalence of a crash on a month-to-month basis. We wanted to give you more tools to better understand the health across all parts of your app. So this year, we're adding more filters to the Crashes Organizer to allow you to focus on specific parts of your app.

The Organizer can now also filter the crashes list by all versions and builds or drill down to any individual past release. You can also filter by any of the extensions that ship with your app, like your watch app, App Clip or share extension. You can even check in how your iOS app is doing running on those phenomenal Apple silicon Macs. Part of triaging your app's issues is getting the full picture of who is affected by a particular crash and that crash is distributed across different categories.

So this year, we've added the ability to see how a unique crash is distributed across your app’s versions and across TestFlight and the App Store. You all have shipped some amazing apps as solo artists, in large teams, and everything in between. In Xcode 13, we are introducing a new way to collaborate with each other by adding the ability for you to share crash reports directly from the Organizer. Using the new Share button in the toolbar, you can share a crash directly with members of your team or just simply add it to your to-do list.

And when you click on one of these links, the Organizer downloads and focuses on this issue, allowing you to zero in on this crash and start investigating.

TestFlight is a valuable tool for developers to quickly get new features and fixes in the hands of their testers, who help find new crashes and bugs. Beta testing an app with TestFlight helps instill confidence in developers that they are going to be shipping a sweet update to the App Store. I am super excited to announce that in Xcode 13, we are bringing TestFlight Crash Feedback directly into the Crashes Organizer.

App Store Connect shows you all of your TestFlight feedback, including crash feedback. You can continue to download the raw feedback and crash log directly from App Store Connect or, new this year, you can open the associated crash right in the Organizer with the new Open in Xcode button.

Viewing your crash feedback is now easier than ever. There is a new inspector in the Crashes Organizer dedicated to TestFlight crash feedback. Opening this Inspector will show you all of the feedback that has been submitted by testers that have hit the crash you have selected. This allows you to see one unified feedback list alongside your crash report that you can quickly sift through to find that extra bit of information needed to help you fix a crash. We think this added context is invaluable when tracking down hard-to-reproduce issues. So a lot of cool new stuff. Let's dive in and start triaging some crashes in the new Crashes Organizer.

I've been working on my team's app, Fruta, and we've have recently shipped a new version to the App Store but are also continuously deploying new versions to TestFlight with Xcode Cloud. Today I want to see if there have been any recent crashes in our App Clip, so let's open up the Organizer and see what's going on. I can do this by selecting the Window menu bar item and selecting Organizer.

From the sidebar on the left, I can tell that Xcode has opened to the Crashes Organizer for my iOS app, Fruta. In the toolbar at the top of the window, you'll notice all of the new filter options that are available this year. Underneath the filter bar is the crashes list, which is showing us all of our crashes for the last two weeks across all versions, builds, and products. Quickly scanning this list, I can see a number of different kinds of issues. Crashes in the list display various badges indicating whether they came from our App Clip, our watch app, or an app extension.

This allows us to quickly identify which area of our app a crash is affecting.

To the right of the crashes list, I see the log view. This is showing me the friendly, simplified view of the crash I have selected in the crashes list. And finally, to the right of the log view, I see the inspector, which shows me all kinds of interesting information about the selected crash. Now that we're familiar with the Organizer, I want to take a look at all of our recent issues for our App Clip. I can open up the products filter in the toolbar which lists all of the products that ship with my app, like our watch app, App Clip, or app extensions. I'll go ahead and select App Clip here.

Selecting App Clip has filtered the crashes list to only display our App Clip crashes from the last two weeks. The first crash in the list here is our top crash. Underneath the title, the Organizer is letting us know that this crash has been affecting 21 devices in the last two weeks and 7 in the last day. Now, before I start digging into the code, I like to ask myself some questions that will help guide my investigation and lead me to some context clues. First, how long has this been an issue? If we head over to the inspector's graph area on the right side of the window, we can tell from the graph legend that this crash first appeared in version 2.0 and is affecting our latest TestFlight build, 2.0.1. Next question, is this affecting my production app or just TestFlight? I can change the filter to Release to find out that this is affecting both TestFlight and App Store builds. So this issue is pretty important, because our customers are experiencing it, and it looks like it may have regressed in our most recent App Store release.

Now let's focus on the log view. The log view is conveniently highlighting for me the exact frame that crashed. I also know that this happened on a background thread by looking at the thread number above the back trace, thread number 6.

I can see this is crashing while we're syncing favorites, but isn't an area of code that I work in too often, and so I don't quite know how to reproduce this. I can't really think of more questions to ask myself that are going to give me more insight into this crash, so it would be really helpful to get some more context. Fortunately, there's a feedback icon next to this crash in the list, so I know that one of our testers has experienced this crash and submitted feedback. This might just be the missing piece of context I need to start working on this crash. I can see their feedback by heading over to the inspector area and opening the new Feedback Inspector.

I see that several testers have hit this same crash, including one who saw it just a minute ago.

In the inspector, I can see a preview of each feedback report, or if I click on one, I can see more details in a popover. I'll click on this latest feedback, which opens up a popover and shows me all of the rich context for this tester's crash, like the exact version and build they had installed, their device model, battery level, and available disk space. These can all be helpful clues when diagnosing a crash. In this case, I can see the tester was on cellular, and their comment says, "I was going through a tunnel and I hit the favorite button. A few seconds later, it crashed." Hmm. if they were going through a tunnel, they might have had a poor network connection. Now we can combine all of our clues together to know that pressing the favorite button will trigger some background task that will eventually crash. I think we have enough to start diving into the code. I'll use the Open in Project button to open this crash directly in the Fruta Xcode project.

OK, so Xcode has opened and is showing the backtrace in the Debug Navigator on the left, and the editor with our source file and the exact line of crashing code highlighted. So why is this code crashing? Well, it seems like we're crashing on purpose with a fatal error, which is an interesting choice. Let's take a quick pass through our syncFavorites function to understand what it's trying to do.

First, we're jumping onto a background thread. Next, it appears like we're saving this smoothie as a favorite to disk. Then we're updating our back end, and we're using a dispatch group to sync these operations together.

We then wait on these operations with a 10-second timeout, and if it takes longer, we crash. This is pretty unfair. Remembering our tester's feedback, it said they were going through a tunnel when they favorited a smoothie. If they were going through a tunnel, then they probably had a poor quality network connection, which means that syncing with our backend could definitely take a lot longer than 10 seconds. This crash makes total sense now. Now that we know that our timeout is too short and we aren't handling it properly, let's fix this. I've been practicing my Combine skills recently, so I think I want to use Combine to fix this.

Awesome. So now we are merging the results of two publishers together, and when we timeout, which is now 30 seconds, we call our completion handler rather than crash. Problem solved. Let's head back to the Organizer and mark this issue as resolved. Oh, it looks like my boss just sent me a text.

Ooh, Craig hit a crash in his demo. Thankfully, my boss used the new sharing feature in the Organizer to send me the exact crash that Craig hit, so I don't have to go digging for it. When I click on this link, the Organizer creates this focused view of the single crash that was embedded in that link. I've actually seen this crash before. I just fixed this the other day and marked it as resolved. I also gave the crash a friendlier name so that I could remember the underlying issue. I'll go ahead and let my boss know that I've already fixed this.

So we've just looked at how to triage our App Clip crashes using the new filters in the Organizer, how to use the new distribution graphs in the inspector, and how to make use of our TestFlight feedback to fix an issue that would have been otherwise hard to reproduce. Now let's take a look at the new Terminations Organizer and improvements we've made to MetricKit. The new Terminations Organizer shows trends of app terminations that may be affecting your customers. The crashes you'll see in the Crashes Organizer are the result of programming failures, but there are other ways that your app's process can be terminated.

Terminations are categorized by reasons like "timing out on launch" or "hitting the system memory limit." You can compare these categories of termination events for your app against previous versions to find regressions.

Some of these terminations happen in the background, but in certain cases, they can happen while your app is onscreen. The Terminations Organizer can help you understand the difference so you can make a plan to fix them. If you'd like to dive into regressions, check out the Application performance survival guide session this year, and to learn more about terminations, I recommend checking out the session Why is my app getting killed? from 2020. If you're not familiar, you can use the Apple framework MetricKit to collect crash logs in your code. All you need to do is import MetricKit, begin listening to the MetricKit manager, and use the crashDiagnostics property on MetricKit diagnostic payloads when they are received.

New this year, crash diagnostics are no longer aggregated and delivered once daily but are now delivered on the next launch of your app so you gain access to crash logs faster. I'm also happy to announce that MetricKit now supports macOS. To learn more about using MetricKit, I recommend the session What's new in MetricKit from 2020. There are many other ways to track down and view crashes. You can access crashes on your connected device from the Devices window or by sharing a log directly from your device. XCTest will collect your crashes that occur when running your tests, and you can use the Console app to view crashes from your Mac and the simulator. For more on these additional tools, check out the session Understanding crashes and crash logs from 2018. Today we saw some great new features. We hope that the new speedy crash log delivery, TestFlight feedback integration, and distribution metrics will dramatically improve your experience fixes crashes and ultimately improve your app's health and your users experiences. Thank you for watching. And remember, the road to success is always under construction. [percussive music]