What’s new in Background Assets

♪ ♪ Jared: Hello, and welcome to "What's new in Background Assets." My name is Jared, and I'm a software engineer here at Apple. I'm really looking forward to sharing with you everything that's new this year with Background Assets. Now, our team has changed a few things over the last year. However, I'd encourage you to check out my prior session where Background Assets was introduced. There's a lot of detailed information in there that this new session builds upon. In this session, I'll start by giving you an overview of Background Assets and why you should adopt it into your app. Then, I'll discuss what's new this year in Background Assets, which includes an entirely new feature that our team has been really excited to share with you. I'll then walk you through a sample implementation of Background Assets that you can even download yourself, and I'll show you how adopting this technology can significantly improve the user experience of your app. I'll then give you some guidance on debugging your app extension and simulating its entry points. Okay! Let's go over a quick recap of Background Assets.

One of the primary goals behind Background Assets is to prevent waiting. The last thing a user wants to experience is to launch your app and have to wait for a large download to complete. Background Assets solves this problem through a combination of its framework and an associated app extension. This new technology was introduced in iOS 16.1 alongside the initial release of macOS Ventura. It supports the ability to download additional content for your app using your CDN provider or a server you manage. For instance, this content may be fetched during an initial app install, when the app updates, or periodically in the background when the user isn't using your app. Through the paired app extension, you are able to write code that runs when the app is not actively being used by the user. This technology is currently supported on macOS, iOS, and iPadOS, so it's already available on your favorite platforms. One of best parts about using Background Assets is that the extension has the ability to run before the user has launched your app. This provides a way to start fetching assets the moment your app is installed through the App Store. The extension may also be launched by the system periodically in the background. This is to ensure that any new or updated assets are present when your app is launched by the user. The extension is also used to service downloads when the app is not running. For instance, when a file finishes downloading, the extension will be launched so that it can move the file to its final destination. Something to keep in mind is that the extension's runtime is limited. This is to ensure that the user's device is optimized for power and performance. I'll go into more details about this shortly. It's also important to know that the app extension you develop to use with Background Assets is placed into a specialized sandbox. This is to ensure that the extension is only being used to manage content via Background Assets. If you find that a capability or API is not available within the sandbox, please reach out to us through Feedback Assistant. I mentioned earlier that your app extension is invoked during three system events: app install, app update, and periodically in the background. Let's take a look at how this lifecycle is managed.

The lifecycle of your extension begins when the App Store installs or updates your app on the device. The Background Assets system service is then notified and prevents the app from launching. The system then inspects your app bundle and reads its Info.plist for the BAManifestURL key. The system will begin downloading the manifest referenced by that key and report progress of the download back to the App Store. Once your manifest has been downloaded, the system then wakes your extension by issuing a content request for the given install or update event. The content request includes a path to the downloaded manifest. Your extension should use the manifest to determine the URLs, file sizes, and what assets to schedule for download. Then once your extension has determined what assets need to be downloaded, it will return those downloads as a set of BADownloads. The system then pauses your app extension, or sometimes terminates it, to save power and performance on the device. The downloads will then begin and your extension will be notified shortly after their completion.

The periodic content request is nearly identical to the app install event, with the only key difference being that your device determines when the event will occur. The device makes this decision based on how the user has been using their device. Key factors such as Low Power Mode, Background App Refresh, or how frequently your app has been launched are all considered. Now, let's dive a little deeper and look at the factors that contribute to when your extension runs periodically.

We care a lot about a device's overall performance and power usage, which is why Background Assets has limitations placed on the extension's runtime. This includes an enforcement on your extension's memory usage. If your extension exceeds a few megabytes of memory, it may be terminated by the system. So you may want to consider memory mapping any large files that your extension needs to read, as memory-mapped data backed by the device's storage does not count against this limitation.

When an app is initially installed, it is provided with a default allotment of a few minutes of runtime per day. While this may not sound like much, with a properly designed extension, this can go a long way. The runtime also changes based on app usage. If an app hasn't been launched in some time, the system may start throttling launches of the extension. For instance, a rarely used app may see its runtime more heavily restricted, whereas a commonly used app may be given additional runtime.

The BADownloaderExtension protocol defines functions that are used as entry points into your app extension. The runtime starts being counted when a function is invoked by the system and stops being counted when that function exits scope. Once your function exits scope or your extension's runtime has been exhausted, the system may suspend or terminate the extension. I'll provide an example in a moment.

However, there is one exception to function scope controlling the runtime of your extension. If an asynchronous exclusive control API is invoked, your extension will be kept running up until the point that its completion handler is invoked and returned. There are a couple of ways that the extension's runtime may be controlled by the user. For instance, if the device is in Low Power Mode or has Background App Refresh disabled, whether that's globally or for your specific app, then your extension will never run. Previously, I mentioned how extension runtime is determined based on function scope. Let's have a look at an example to better understand how that works.

This code represents the extension's interface that services your background downloads. The BADownloaderExtension protocol defines the functions that the system will invoke into your extension. Now let's add one of the required functions to conform to this protocol. The "downloads for request" function is one of the primary entry points into the extension. Its BAContentRequest defines if it's being invoked during an app install, app update, or as a part of a periodic check in the background. The manifestURL argument provides a path to a local file that was downloaded before the extension was invoked. The manifest file is commonly used to compare what is currently downloaded versus what downloads might be available on the server. The function definition's return type requires a set of download objects that conform to the BADownload type. This means that answering this function's request requires you to synchronously return any content needing to be downloaded before the function exits scope. However, in this contrived example, let's say that you invoke a function called parseManifest. This function reads the downloaded manifest and then returns the BADownload objects needing to be downloaded.

However, let's say that the parseManifest function is poorly implemented and takes 30 minutes to parse and construct the downloads. This will end up exceeding the extension's runtime significantly, and the extension will be terminated. It's important to remember that the extension's runtime is calculated from the moment the "downloads for request" function is called, up until the point it exits scope and returns. Let's take a look at another problematic example.

Whenever any of the BADownloaderExtension protocol functions exit scope, the extension may become suspended and then terminated. You'll notice that the protocol does not define any of its functions as mutating, and there's a good reason for this. When your extension is terminated, any instance variables or in-memory state will not be saved. If your extension needs to maintain any state, you should serialize that state to disk. Next, let's talk about the API you'll use from both the extension and app to manage downloads.

The download manager within the framework is the primary way to communicate with the Background Assets system service. The manager is a singleton object that can be used throughout your app. Using the manager, you can schedule downloads of your assets in the background and promote already scheduled downloads to the foreground. From the download manager, you can also manage downloads that are currently in-flight, which I mentioned earlier could have been scheduled by the extension before your app was ever launched. There's also a synchronization mechanism that you may use to ensure that both your app and its associated extension are not performing similar operations at the same time. The last thing I'd like to bring up about the download manager is that it has a delegate for receiving callbacks about downloads, similar to the BADownloaderExtension protocol. If you register a delegate on BADownloadManager, then it will receive those callbacks instead of your extension. This is useful as it provides your app a way to manage its downloads while it is running. Now that you've had a quick refresher about how to manage downloads, let's talk about how you should manage those files once they're already on the user's device.

Any files you download with Background Assets are marked purgeable, which means that the system may remove them under critical circumstances. Think a system security update or if a user needs to capture a video of their child's first step.

However, if you modify or expand a downloaded asset, then those files are not tracked by the system and therefore are not purgeable. You should think carefully about how you modify assets or extract data out of them. If you incorrectly manage your downloaded assets, you could increase the size of a user's backup of their device or prevent a critical security update from being downloaded. Therefore, you should try to store your downloaded assets in your caches directory. That way, the system knows that it can purge them when it is critical to do so. Now that you have a full recap of how all of this works, let's take a look at what's new this year with Background Assets.

Earlier this year, we introduced essential downloads, which provides a way to fetch content while your app is installing or updating. This means that your downloads are completely integrated into the iOS Home Screen, macOS Launchpad, and the App Store. To the end user, the download of your assets appear to them as if the app is currently still being downloaded from the App Store. This also means that while your essential downloads are in-flight, your app cannot be launched by the user. All the user can do is cancel or pause installation. Since pauses are supported, your server should support HTTP ranges so that resumes are possible. Since essential downloads occur during app install, they take priority over any non-essential downloads. Let's take a look. It all begins when your app is requested from the App Store or TestFlight. If the app's Info.plist contains essential asset keys, then progress is set up on the device and we go through this flow. Once your app has been downloaded and installed, the system wakes your extension by issuing a request for content, which includes whether the request is for an app install, app update, or for a periodic fetch. During this time, an authentication challenge may be sent in order for the manifest to be downloaded. Your extension will then vend back a combination of essential and non-essential downloads. As a side note, it's important that your extension vends the downloads back quickly, as your app's download progress will appear frozen to the user until this function returns.

The moment the extension provides the downloads, any downloads that were marked as essential will immediately begin. Your extension may also receive an additional authentication challenge during this time. Once all of your essential downloads have finished, the system will terminate your extension and the app will become launchable by the user. The extension will then receive a batch of successful and potentially failed downloads. If there are any failed downloads you can re-enqueue them as nonessential using BADownloadManager. As your extension is receiving completion messages for essential downloads, the system will immediately start downloading the nonessential assets. The nonessential downloads will then be sent to the extension as they finish downloading. Now let's take a look at how essential downloads integrate into the App Store installation progress on the iOS Home Screen. A percentage of the progress indicator is broken down into the time it takes to download your base app, plus the amount of time to do the install, followed by the amount of time necessary to download your essential assets. The new BAEssentialDownloadAllowance key defined in your app's Info.plist is used to set up the initial overall progress indicator. Then once contentForRequest is invoked into your extension and your extension returns downloads, the file size of each essential download is added together to determine how much is actually being downloaded. If the amount that you schedule for download is significantly less than the essential download allowance, then the progress indicator may move rather quickly. You should aim to get your essential download allowance close to what is actually being downloaded to ensure smooth progress for the user. It's important to keep in mind that everything we've discussed can be disabled by the user. In the App Store settings pane, there is a section for disabling in-app content. While this doesn't disable Background Assets in its entirety, it does prevent essential assets from downloading and the ability for the extension to run before the app has been launched by the user. So it's important to think of essential assets as just that: essential but not a requirement for your app to launch. Therefore, its important for your app to handle flows where essential assets are not already on the device when your app is launched. The ability to use essential assets was actually introduced earlier in the spring as part of iOS 16.4 and macOS Ventura 13.3. The new APIs are quite minimal and should be easily added into your existing extension.

The first API that was created to support essential downloads was actually a new initializer on BAURLDownload. There are two new arguments we've added specifically to support this feature. The essential argument, as its name implies, specifies if the download should be marked as essential, where essential means contributing to the app's overall download and installation progress. The file size argument is the size of the assets that will be downloaded. The file size must be accurate when creating essential downloads. The system needs this information so that the app install progress on the user's device is displayed properly. If the file that Background Assets downloads does not match the file size provided here, then the download will fail if the download is marked as essential. If your extension does not know the size of the file, then the file size should be included in the BAManifestURL that is provided to the extension before the extension is launched.

Another API that was introduced provides an easy one-liner for creating a nonessential representation of a download. Since essential downloads can only be enqueued in the contentForRequest function, this API can be useful in many cases. For example, let's say fetching an essential download failed, perhaps because of a networking issue or the file was simply temporarily unavailable. Well, in the background download failed function within your extension, you can easily create a nonessential representation of that download and re-enqueue it. The download will then begin in the background and your extension or app will be notified when it is completed. Now let's take a look at some of the required keys that need to be present in your app's Info.plist.

In last year's session, I went over each of these keys in detail. If you'd like a more in-depth explanation, I'd encourage you to check out that talk. It's important to keep in mind that these keys are required not only to use the Background Assets framework, but are also necessary in order to submit your app to the App Store. There are two new keys this year that are required to support essential assets: BAEssentialDownloadAllowance and BAEssentialMaxInstallSize. The essential download allowance is represented in bytes and defines an upper bound on how large the sum of all of your essential assets will take to download. It's important to try to get this number as close as possible to the size of the essential assets you enqueue so that download progress is smooth for the user when they install your app. The other new key, BAEssentialMaxInstallSize, represents the maximum size of those assets extracted onto the user's device. This number appears on the App Store as a way to tell users how much storage your app will use after the essential assets have been installed. That pretty much sums up the new APIs we've added for essential assets. As you just saw, adding essential assets support to your existing app can be done with minimal code changes. It's really just that easy. Now for the fun part. Let's take a look at how you can extend an existing app that uses URLSession into using Background Assets. The app I'll be showing you today downloads WWDC Sessions, just like this one, and stores them on your device for offline viewing. Currently, the app has to be launched before the videos will download. By adopting Background Assets, we can eliminate this wait time by having the videos already downloaded before the app is launched. Let's have a look. Here's the app we'll be building upon today.

You'll notice that the moment it is launched, sessions immediately begin downloading. The way this app currently works is that a manifest downloads from a server, which contains a list of WWDC sessions. After the manifest is fetched, the sessions start downloading and then become viewable once tapped. Let's take a look at what's necessary to adopt Background Assets into this project. Before you begin to use the Background Assets API, the first thing to add are the initial Info.plist keys that I discussed earlier. These key are required to be present in your app bundle's Info.plist file. The next thing you'll need to do is add a background download extension and embed it into your app. You'll want to make sure that your extension's bundle identifier is prefixed with your app's bundle identifier. You'll also want to ensure that both your app and its extension are in a common app group, as the app group is how your extension shares downloaded assets with your app. The last thing you'll want to ensure is that both your app and its extension are signed with your team identifier. With those steps out of the way, you can begin to adopt Background Assets.

Here in front of you is the Xcode project for the app you just saw. I've already went and created the download extension and embedded it into the app. I've also added the required Info.plist keys. With all of that out of the way, let's begin by navigating to the SessionManager.

The SessionManager in this project currently uses URLSession to fetch the latest downloads. URLSession is a fantastic API. We'll continue to use it within the app to fetch the manifest. However, we'll migrate to using Background Assets to fetch the actual sessions. This is so that the app can take advantage of promoting any assets scheduled in the background by the extension we're about to create to the foreground when the app is launched. To begin, we'll starting by importing the Background Assets framework module.

I'll then scroll down and remove the variables associated with URLSession as they are no longer needed. Now I'll navigate to the "start download" method. As you can see, the existing code was tracking sessions to download by the URLSession download task. This won't be necessary anymore, so let's remove it. Here's where things begin to get interesting. When you work with Background Assets, its important to think about your extension and app possibly running at the same time. To coordinate this in a near effortless way, you'll need to use withExclusiveControl to guarantee that any work that needs mutual exclusion with the other process can do so. Let's add that in now.

As you can tell, this API is asynchronous and escaping. Any work scheduled inside the closure is guaranteed to run independently from the extension if the extension also uses this API. We'll implement the extension in a moment, but for now, let's focus on the app. Since we know we're running in a mutually exclusive context, let's ask the download manager if there are currently any downloads in-flight.

There's no reason to reschedule a download if the extension has already scheduled it. However, one thing we can do is, if we find an existing download, we can promote it to the foreground.

Promoting a download to the foreground can significantly decrease the time it takes for a download to finish. Since the user is currently using the app, it's a great opportunity to fetch the download as quickly as possible. The user might want to view it. If the download does not already exist, we'll create it.

Then, regardless of if we just created the download or if the extension did, we'll go ahead and start it in the foreground. Promoting a download from the background to the foreground does not cause the download to restart, it is simply resumed from where it left off in that transition. The next thing on the list is to implement the BADownloadManagerDelegate, but before I do that, I'll delete the old URLSessionDelegate.

Now that the old delegate is gone, let's create the Background Assets delegate.

Since the session manager is now conforming to the delegate, it's important that it's wired up to actually receive those messages. So I'll go up to the initializer and wire that up now.

Since BADownloadManager connects directly to the system scheduler, it's a singleton object. Having this delegate attached to the download manager will cause your app to receive messages over the extension if your app is currently running. Now let's head back to the functions we need to implement.

For this app, there are three specific functions on the delegate protocol that we'll be implementing. The first is for progress handling. Let's implement that now.

Before we start blindly updating progress in the UI, we'll make sure that the download we're receiving progress messages for is something that the manifest is currently tracking. If a download is being tracked, we'll call updateDownloadProgress, which is a helper function within the app that sends the progress directly over to SwiftUI. Next, I'll implement what happens when a download finishes.

It starts pretty much the same way, which is to make sure it's handling only downloads that are expected. Then replaceItemAt is used to move the object from the temporary location that Background Assets has given us to its final location. It's important to use move operations here, as the system will track and purge the file if the device becomes low on space. So you should make sure that your app always checks to see if any files are missing and refetches it if it so needs to. The last thing that happens here is, a Task is spawned against the MainActor, the state is marked as downloaded, and the app begins fetching the session's thumbnail. Now, I'd like to say that all downloads will succeed. However, the unfortunate truth is that they can fail, whether that's because the server no longer has that resource or there's a network issue. Background Assets does retry and wait for network connectivity problems, but after a certain point in time, you have to know that the file is not on its way. Another thing to keep in mind is that downloads that are promoted to the foreground will fail almost instantly if there is a network connectivity issue. There's not much our app needs to do when a download fails. It could present UI or reschedule it, but for this example, let's log that there was a problem.

Since the delegate is now fully implemented, let's relaunch the app and see how it looks.

Well, to no surprise, it looks identical, and that's what we really wanted to see here. Adding Background Assets in place of URLSession is quite effortless. The next thing I'll show you is how to implement the app extension for handling background downloading. Adding this app extension is how you can leverage Background Assets to fetch your content before your app is installed or updated and is what provides the support for enqueuing essential assets. In essence, the extension is what is responsible for scheduling downloads while your app is not running. Let's have a look. Here we are in the background download handler, which receives messages within your extension related to Background Assets. From the extension, the first thing I'll do is create a logger so that we can see from Console.app when our extension is running.

Next, I'll implement the contentForRequest function that's part of the BADownloaderExtension protocol.

The first thing this extension will do is parse the manifest that was predownloaded before the extension was launched. If the manifest that was downloaded is somehow invalid, the extension will be configured to enqueue no downloads. Once the extension knows that the manifest is valid, it is atomically saved into the app group. This is so that the app and the extension have the latest version of the manifest locally that they can reference later. Since this save is done atomically, using withExclusiveControl is not necessary. The extension will then create a mutable set of download objects that the extension will return to the system to be scheduled. As discussed earlier, essential downloads are only supported during app installation or app update. I'll then iterate through the manifest for all sessions that are remote, which in this context means that they aren't downloaded. A BAURLDownload object is then created for every download that needs to be scheduled. The download is given a unique identifier, a URLRequest, an annotation for if the download should be fetched as essential, its file size, the app group the asset will be downloaded into, and a relative priority to control which downloads the scheduler should start first. One important thing to keep in mind is that any downloads marked as essential must have an accurate file size, or the download will marked as a failure. This is to support smooth progress on the iOS Home Screen, macOS Launchpad, and the App Store. Now that we have support for enqueuing downloads, let's take a look at handling successful downloads.

The extension processes downloads if the app is not running or if the app doesn't have a delegate assigned to BADownloadManager. The first thing I'll implement in the download finished handler will be asynchronously acquiring exclusive control. Since acquiring exclusive control is asynchronous, we need to hang on to the temporary file the extension just vended us. As you can see, I'm accomplishing this by moving the file into a temporary location that will outlive this function's scope. I'll also add a Swift defer to ensure that the ephemeral file that the extension downloads is always cleaned up. Although the system will delete the file for you, it's best practice to manage it yourself. I'll then load the manifest from the app group and verify its validity. The extension will then check to make sure the downloaded identifier being processed matches a session that is expected in the manifest. Then the extension will move that file from its ephemeral location to its final destination inside the app group. A LocalSession is then constructed to quickly validate that the session is downloaded into its appropriate location. The last thing to handle are failed downloads.

An important thing that is commonly forgotten is that if a BAManifestURL fails to download, the extension is actually notified about it. Its type inherits from BAURLDownload. However, that's not its exact internal type. Therefore, the extension filters to ensure its only dealing with BAURLDownload objects. Since essential assets download in the foreground, they only wait a few seconds for network connectivity since those downloads impact app installation progress. Therefore, it can be a good idea to re-enqueue essential downloads as nonessential if they happen to fail. One way you can easily convert an essential download into being nonessential is to use the new removingEssential() function. That function returns a nonessential copy. You then vend that copy to BADownloadManager's scheduleDownload function. The system will then fetch the download at an opportune time. Now that our app and its extension have fully adopted Background Assets, let's add some UI that indicates that a specific session was downloaded as essential.

Remember, essential means that app installation and app update prohibit app launch until the assets are fully downloaded. However, since the app being blocked from launching only occurs during an install from the App Store or TestFlight, having an indicator present is a good way to visibly recognize this. This indicator will be implemented in SwiftUI within the VideoSelector view.

If the session is marked as essential in the manifest, a green circle is drawn in the navigation view. And that's all there is to it. Now that you've seen just how easy it is to implement Background Assets, let's take a look at debugging the extension and simulating its entry points. As I discussed earlier, the extension launches during an app install, app update, or periodically in the background. Since app installation is controlled by the App Store and periodic events are controlled by the device, you'll need a way to force your extension to launch in order to debug it.

Fortunately, our team has created a tool that ships with Xcode, which can be used to launch the extension and simulate its entry points. The tool is accessible from Terminal and can be launched by entering xcrun backgroundassets-debug. There is also help information and a man page that goes along with it. When using backgroundassets-debug, you pass in a simulate argument that provides the ability to trigger different types of BAContentRequests. When any of these simulation messages are sent to the device, your extension's runtime is reset to help ease your debugging. backgroundassets-debug provides numerous features, including support for multiple devices. As long as the device is paired with your Mac, it can be used to simulate your extension. These commands may be sent over Bluetooth or Wi-Fi so that your device doesn't have to be tethered to your Mac via USB. Lastly, Developer Mode must be turned on in order for the device to honor simulation events triggered by backgroundassets-debug. This can be easily enabled from the Privacy & Security section within your device's Settings app. Now let's take a look at using the backgroundassets-debug tool with the demo app we just built. Here we are in Terminal. Since Xcode is installed on this Mac, I'll go ahead and list the connected devices. You'll see that there is currently only one connected device, and that happens to be my iPhone. You'll want to copy the device's UUID that you are choosing to simulate against. In the arguments provided, you see that the device's UUID is present along with the app's bundle identifier. I'll be invoking the tool using the simulate argument mixed with the app-install argument. This will cause the app extension to wake as if it's being awoken during an initial app-install event. Now that I've pressed enter, the tool tells you that the app-install event has been sent to the device. Let's launch the app and see what it looks like.

As you can see, all of the session videos have been enqueued for download. Some of them, as I'm speaking, have already finished downloading. And that's Background Assets in action. If you're like me, you've probably already started thinking about the many potential ways to reduce waiting in your apps. Now that I showed you how to debug your app extension, let's take a look at some important things to remember.

Essential assets download with the app and contribute to app installation progress. Therefore, these types of assets are only schedulable during a BAContentRequest install or update event within the extension. If any essential downloads that were enqueued by the extension fail, the app will eventually still become launchable. You must ensure that you re-enqueue your essential downloads as nonessential either within the extension or on first app launch if you require those assets. Any downloads not marked as essential are downloaded after the app is installed or updated and receive background download priority. It's also important to remember that the extension is limited by its memory usage. So make sure to debug your extension to ensure that the code you write is well below the memory threshold. If your extension uses excessive memory, you will notice that it will crash with a memory exception while debugging. If the extension crashes during an app install or app update content request, then the app will become immediately launchable. And finally, remember to make your app a good citizen by managing the assets it downloads appropriately. One way to manage those assets is by placing them into your app's caches directory. So that's what we have this year for Background Assets. If there's anything you should take away from this session, it's to eliminate waiting in your apps. One way you can eliminate waiting is by adopting Background Assets and using the new essential assets feature. Make sure to utilize TestFlight before submitting the app to the App Store. By using TestFlight, you'll be able to see what it looks like when your essential assets download with your app. If at any point you find yourself with questions, please check out the developer documentation and feel free to reach out on the Apple Developer Forums. I'd also like to encourage you to utilize Feedback Assistant. Our team is constantly improving Background Assets, and we're looking forward to having you adopt it into your app. Oh, and if you haven't already, I'd recommend checking out my prior session where Background Assets was introduced. There's a lot of additional information in there. And with that, thanks for joining me today. On behalf of everyone at Apple, thanks for watching.