Identify trends with the Power and Performance API
Track your app's performance metrics in custom team dashboards, bug reporting systems, and other custom workflows with the Power and Performance Metrics and Diagnostics API. Explore how you can access the same data that drives the Power and Performance analysis tools in Xcode to quickly identify trends and regressions. Learn how to leverage diagnostic signatures and logs — including call stack trees — to prioritize and debug issues. And discover how you can integrate this API with your development team's existing tools to troubleshoot issues quickly, offering better overall performance for people who use your app.
Hello my name is Ashish and I'm a performance tools engineer at Apple.
Yeounoh will join me later and we are very excited to talk to you about the new App Store Connect API for obtaining power and performance data. The amazing work done by developers in building great applications is an essential part of our users overall software experience. And two of the most important aspects of ensuring a great user experience are optimizing the power and performance impact of your applications. At Apple, we are deeply committed towards providing you actionable tools to help you easily identify the areas to focus on. Last year WWDC 2019, we first introduced a new feature within Xcode 11 for out-of-the-box power and performance analytics.
The Xcode Organizer allows you to view aggregated power, performance, and I/O metrics to help identify areas for improvements in your applications. Since that release, we have received many requests to be able to access the same Xcode data programmatically. One of the common requests we heard was the ability to consume this data in your own analytics and monitoring pipelines. So we are very excited to announce the new Power and Performance API, which is part of the App Store Connect API being released later this year. We highly recommend you to check out the "Expanding Automation with App Store Connect API" video to learn more about App Store Connect.
The new API enables you to programmatically access the Xcode metrics and diagnostics data, allowing you to consume this data yourself. We have also provided a new built-in smart insights feature to identify your application's power and performance hotspots without any additional effort.
As people use your application, we collect the power and performance data from consented devices. This data is then aggregated on our servers and made available to you through the App Store Connect API. You can use data from this new API for building highly customized analytics pipelines, as well as automated monitoring of your app's performance through dashboards.
During the rest of this talk, we'll first go through a brief overview of this new API and the data types vended by it. Next we'll walk through a few examples of accessing the same aggregated data that drives the power and performance analytics in Xcode as well as the new smart insights. Finally, we’ll present a live demo to extract aggregated metrics, insights, and diagnostics for a sample application and wrap up with a brief summary.
Now let's walk through an overview of this API. There are four new REST API resources available this year. Let's take a look at each resource. The first API resource uses your applications ID to provide aggregated metrics and insights for your apps most recent versions. The smart insights help identify your applications hotspots. We'll go into more details on this data in the next section. The second API resource allows you to download metrics for a specific app version. To get this data, you'll need to provide the Build ID provided by App Store Connect API for a specific application version.
Next you can access top diagnostic signatures for a specific app version.
These signatures are used to group common problems. We also recommend you check out this year's talk on Xcode Organizer which goes into details about accessing the same diagnostics data through Xcode. Finally you can download logs corresponding to diagnostic signatures for root cause analysis.
This requires the diagnostic signature ID as part of the GET request.
To summarize the new API, you can incorporate these 4 endpoints into your analytics workflows to automatically track your applications footprint using aggregated metrics, smart insights, and diagnostics data.
Now let's walk through a few examples to get data and insights through these API endpoints starting with aggregated metrics data. The power and performance metrics data for your application is aggregated by each unique metric And device type. These metrics are the same as those available in Xcode Organizer today, such as battery drain, launch, and disk writes. The new API currently supports vending this data for all iPhone an iPad device groups. As discussed earlier we need to make the following GET request with your application ID to receive both aggregated metrics and insights for your recent app versions.
This returns a structured JSON string with two major sections.
Aggregated metrics which is represented as product data, and insights. Let's take a look at a sample Metrics JSON. For each metric, such as hang rate, the JSON structure contains some metadata, such as unit. And for each unique device and metric combination, the JSON contains the 50th and 90th percentile values. Finally the metric summary is generated for up to eight most recent versions of the application. As this example JSON indicates, you can use this data to build highly customized power and performance analytics across different metrics, devices, and percentiles. But we didn't stop here. Our team went one step further to also help you quickly identify your applications power and performance hotspots without the need for custom analytics.
We are really excited to bring you the new smart insights feature. These smart insights use the earlier metrics data to automatically flag major hotspots in your application. This is especially helpful if you want to track top application regressions in the field without any additional effort.
Currently these insights flag regressions and uptick in metric values. The insights JSON response contains three sections: the versions analyzed for the insight, a summary of the regression, and the list of impacted devices and percentiles. Let's walk through the details of a sample insight. The highlighted section indicates the metric type and summary of the identified regression.
You can use these summary strings to track top regressions for your application.
As shown here, the application's launch time Is regressed only for the 90th percentile iPhone users. Such insights can help you take more focused actions to improve your app's performance. The population section contains a detailed structured list of impacted percentiles and devices. And the third part contains some metadata, such as the latest application version, and the list of previous application versions used for comparison. In the final part of the section, let's take a look at diagnostics data for root cause analysis. Over to Yeounoh now to continue the rest of the talk.
Thank you Ashish. Hi. I'm Yeounoh, a performance tools engineer at Apple.
In addition to metrics and smart insights we also provide diagnostics data to help you deep dive into regressions. There are two parts to power and performance diagnostics. First, we use diagnostic signatures to group similar problems together. For instance, similar disk writes exceptions are aggregated to generate disk writes signatures. Using diagnostic signatures, we can do root cause analysis by problem group. The API returns most prevalent top signatures for your app version, and given a top signature and it's ID, you can get related diagnostic logs with more details. To receive top signatures for your app version, you need to make following GET request with build ID from App Store Connect API. This year, you can access disk writes diagnostic signatures from both Xcode and the API. The response contains a list of top signatures in data, and each signature contains a unique signature ID and attributes. From this attribute, you can tell that this is an example of a DISK_WRITES signature. And the signature string is the call frame that represent the callstack in Xcode UI. And weight is the normalized signature weight which indicates the relative importance of this signature. And it also comes with a URL to get diagnostic logs.
As you can see, this URL already contains the signature ID. The second part of power and performance diagnostics is diagnostic log. It contains anonymized diagnostic details from individual devices. And this includes metadata information such as platform, OS version, and device type. And also, the log contains function callstack trees, which can be extremely helpful to understand what is causing the power and performance hotspots. The diagnostic callstack JSON structure is also shared with MetricKit. So check out this year's MetricKit talk to learn more about on device power and performance diagnostics.
Getting diagnostic logs requires the following GET request with a signature ID.
This is the same kind of URL that we have seen from the diagnostic signature response.
The response contains a list of diagnostic logs and each log contains diagnostic metadata with information like device type, OS version, and build version. In this example, the log is generated for a disk writes exception event.
The log also comes with callstacks. Each callstack frame contains raw frame and parsed frame information and a list of sub frames. Following the sub frames from the root frame, you can traverse the entire callstack tree structure. Now let's jump into a demo. In this demo, you will see how to access the API yourself as well as using a simple script that will be released with the API later this year. For the purpose of this demo, let's assume that I'm a developer for an app called Meal Planner. This app helps me plan meals and save recipes. In the latest release, we introduced a new feature to save delicious photos with entries. And now I can use the power and performance API to make sure that app users don't experience any regression.
To access the API we first need to generate an API access token.
I will run a script to generate a token using an App Store Connect API key.
With the token, I can access perfPowerMetrics resource with Meal Planner app ID.
As you can see in this response, we get insights as well as product data in the same format that we've seen during the talk, and product data contains a lot of aggregated metrics information. So let's go look at smart insights to see if we have any regression.
Taking insight's regression and its summary string, we see that there is a disk writes regression for the latest app version for 50th percentile users on all iPhones. To further investigate this, let's get diagnostics logs and look at exception callstacks. First I will request top diagnostic signatures using the latest app version build ID. By taking the top disk writes signature and its related URL, I will request diagnostic logs for the signature.
The diagnostic log response contains callstack information. And this callstack structure is efficient for machine processing. And for us to look at the original callstack, I've written a sample code to parse and pretty-print this stack information. This simple script programmatically access the same diagnostic API resources. Hit enter, we get the original callstack from the log. Looking at the call function frames from the bottom, I see that the new feature to add photos with the entries is causing heavy disk writes I/O. By following the stack trace, I can go to the exact location in the source code to debug and optimize the implementation.
The sample codes to generate token as well as to get diagnostics data will be released with the API later this year, so please check out the sample codes for quick reference to API access. To summarize this demo, you can view and monitor aggregated metrics using the API and use smart insights to identify power and performance regression. And finally, use diagnostics data to the root cause analysis of the problem. Let's recap what we have discussed today. We introduced our new App Store Connect API for power and performance.
With this new API, you can programmatically access the same metrics and diagnostics data as Xcode Organizer and build customized data and analytics and monitoring systems around power and performance metrics. And we also introduced smart insights, that can help you identify key trends in the regression in metrics.
We are very excited about this new API for you to continue building amazing apps.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.