Make blazing fast lists and collection views

♪ ♪ Hello, I'm Aditya Krishnadevan, an engineer on the UIKit team. At the core of many apps are lists or collection views. Having super-smooth scrolling is a big part of making those apps feel great. This video will set you up for success when making blazing-fast lists and collection views. We'll build this app here that uses collection views to display a list of image posts of some great travel destinations. It's fairly simple at first glance, with a photo of the destination and a couple of text labels. Throughout this video, we'll talk about how it's set up and how it achieves the performance that people expect. First, we'll learn how to start from a strong foundation when using APIs, like diffable data source and cell registrations. We'll refresh our understanding of the life cycle of a collection view cell. We'll then talk about why you might not see perfectly-smooth scrolling, and some advances in prefetching that can help. Finally, Patrick will explain how to correctly update cells when their content comes in asynchronously, and how to use new UIImage API to get the absolute best scrolling performance on all devices. Okay. Let's begin by talking about how the app structures its data. The sample app retrieves a list of posts for display, where each post is represented by this DestinationPost struct. DestinationPost conforms to identifiable, which means it has this ID property that stores its identifier. This is a unique identifier for each DestinationPost that remains stable even if other properties change. Diffable data source is built to store identifiers of items in your model, and not the model objects themselves. So, in the sample app, the diffable data source is populated using the ID property here, and not the DestinationPost itself. Here's the diffable data source used in the app. Like we just discussed, it uses the DestinationPost.ID type for its item identifiers. The Section type here is an enum with one case, as the app has only one section. To populate the data source, the app first creates an empty snapshot and appends the main section. Then, it fetches all the posts from its backing store and appends their identifiers. This way, if one of the other properties of a DestinationPost changes, its representation in diffable data source remains stable, as the identifier does not change. The final step is to apply the snapshot to the data source. Before iOS 15, applying a snapshot without animation would be translated to a reloadData internally. That wasn't great for performance, as the collection view had to discard and recreate all the cells on screen. From iOS 15 onwards, applying a snapshot without animation will only apply the differences and not perform any extra work. With iOS 15, diffable data source also gains a new reconfigureItems method that makes it very easy to update the contents of visible cells. We'll go through how it works later in this video. First, let's get the data from our data source into cells and onto the screen. Cell registrations are a great way to keep all the configuration for each type of cell in one place, and they give us convenient access to the identifiers from diffable data source. UICollectionView maintains a reuse queue for each of instance of a registration, so ensure that you create registrations only once for each type of cell. Here's a simplified registration for cells in the app. The postID that's passed in is used to retrieve a DestinationPost and an asset object containing the image. The properties from the DestinationPost are used to set the title and image in the cell. To use a registration, call dequeueConfiguredReusableCell inside the data source's cell provider. Note how the registration is created outside the cell provider and then used inside. This is important for performance, as creating a registration inside the provider would mean that the collection view would never reuse any of its cells. Now that we understand how to configure a cell, we'll move on to when a cell is configured and what its life cycle is like. The life of a cell is composed of two phases: preparation and display. The first step for preparation is fetching the cell to work on. Whenever UICollectionView needs a cell, it asks for one from its data source. If this is a diffable data source, it runs the cell provider and returns the result. When the cell provider runs, the collection view is asked to dequeue a new cell using a registration. If a cell exists in the reuse pool, UICollectionView will call prepareForReuse on it, and then dequeue the cell. If the reuse pool is empty, it will initialize a new cell. That cell is then passed in to the configuration handler from the registration. This is where apps set up the cell for display for a given item identifier and index path. The configured cell is returned to the collection view for the next step.

The collection view queries the cell for its preferred layout attributes and sizes the cell appropriately. At this point, the cell is fully prepared and ready for phase two: display. willDisplayCell is called on the delegate, and the cell is made visible inside the UICollectionView. The cell is now on screen. There are no more changes to its life cycle while it remains visible. When it is scrolled off screen, didEndDisplaying is called for the cell, and it ends up right back in the reuse pool. From the reuse pool, a cell can be dequeued again, repeating this process. Let's now check what the app feels like with these basics in place. The app is featuring Cusco in Peru, and Saint Lucia in the Caribbean. Let's scroll through the app and see some other destinations, but notice how it doesn't scroll smoothly.

These interruptions during scrolling are called "hitches." To understand what causes a hitch, let's first learn how an app updates the display. For each frame, events such as touches are delivered to an app. In response, it updates the properties of its views and layers. For example, a scroll view's contentOffset will change during a pan gesture, changing the on-screen location of all the views it contains. As a result of those changes, the app's views and layers perform layout. This process is called a "commit." Then, the layer tree is sent to the render server. Each frame has a commit deadline. This is the time by which all commits for that frame need to finish. The amount of time an app has to commit for each frame depends on the refresh rate of the display. For example, on an iPad Pro running at a higher refresh rate of 120 Hz, apps have less time to finish work for each frame compared to an iPhone running at 60 Hz. Here's a typical example of scrolling a list of cells in a collection or table view. When a new cell becomes visible, there's a longer commit, during which the new cell is configured and performs layout. Then, there's a couple frames where it's just the existing cells being moved around on screen. The commits for these frames are quick because no new cells are needed. Eventually, the scroll position changes enough to cause a new cell to become visible, and this pattern repeats. So what causes hitches like in the demo earlier? When the commit for a frame takes too long and misses the deadline, those updates don't get incorporated into the intended frame. The display keeps the previous frame on screen until the commit finishes, and this delayed frame can render. This is a commit hitch and is perceived as a momentary interruption when scrolling.

To learn more about this and other types of hitches, watch the "Explore UI animation hitches" video. To help avoid these hitches, UICollectionView and UITableView both have a brand-new cell prefetching mechanism in iOS 15.

We're back here to the example of an expensive cell causing a hitch during scrolling. A key takeaway from this is that you typically don't need a cell every frame. We have a couple of frames with very short commits doing minimal work. Cell prefetching in iOS 15 takes advantage of this spare time by preparing the next cell right after finishing a short commit.

Then, when the cell is eventually needed, it's just a matter of making it visible. That's why the commit for the frame where the prefetched cell becomes visible is very quick, because all of the work was done earlier. The amount of time spent prefetching the cell is the same as when it causing a hitch. But because we're able to get a head-start, we're able to avoid a hitch. Let's understand why this works by stepping through each commit. Before the prefetching happened, we performed the commit for this frame. Since no cells were needed, it was a quick commit, and it finished with lots of time left before the deadline. Instead of just waiting around until the next frame, in iOS 15, the system recognizes the situation and uses the spare time to start prefetching the next cell. Now, the following frame is where things get interesting. Because the cell being prefetched is expensive, it actually causes the commit for that frame to start later than normal. However, even though that commit starts late, it still finishes well before its deadline because it's quick. Compare this to the illustration we saw earlier without prefetching. Notice how there are no longer any commits missing deadlines, and so, there are no more hitches with cell prefetching. This means that your apps get up to twice the amount of time to prepare each cell, without causing any hitches. What's more, all you need to do to get this great new functionality is to build your app with the iOS 15 SDK. When I last ran the demo, the app was built with the iOS 14 SDK. Let's check out scrolling in the app when built with the iOS 15 SDK.

This is great! Looks like prefetching is doing exactly what we'd want. Scrolling is now perfectly smooth, and we didn't have to change a single line of code.

Remember, all you need to do is to build your app using the iOS 15 SDK. For UICollectionView, this new prefetching expands on what was introduced in iOS 10. Cell prefetching is now supported for lists, as well as all other compositional layouts. This great new prefetching is now even enabled in UITableView. Prefetching can improve scrolling performance by eliminating hitches, but it will also reduce power usage and increase battery life. If your cells are quick to prepare, the system can use the extra time to run in a more energy-efficient state and still avoid hitches. So, even if you don't notice any hitches, it's still very important to make your cell configuration and layout implementations as efficient as possible. Let's now talk about how prefetching affects cell life cycle. This is the life cycle we talked about earlier, without prefetching, with the two distinct phases. When a cell is prefetched, it is the preparation phase that is executed ahead of the cell being required on-screen. To take full advantage of prefetching, a cell must be fully configured in this phase. Don't wait until a cell is visible to perform any heavy work. When a cell returns to the collection view, it is sized to get its preferred layout attributes, also as part of the prefetch. After being prefetched, there is now this in-between state, where a cell is waiting to be displayed. Given this new phase, there are two important considerations for apps. It is possible for a prepared cell to never be displayed, which could happen if the user suddenly changed the scroll direction. Then, once a cell is displayed, it can go right back into the waiting state after it goes off screen. The same cell can be displayed more than once for the same index path. It's no longer the case that a cell will be immediately added to the reuse pool when it ends displaying. Prefetching helps us achieve smooth scrolling, but only because it gives us more time. On other devices with a higher frame rate, it's still possible that the app is going to have hitches during scrolling. Patrick will now give you more detail about how the app configures its cells, and also talk about strategies to reduce the amount of time per commit when displaying images. Thanks, Adi. Hi, I'm Patrick from the High-Level Performance team. Now, I'll guide us through updating existing cells in the sample app, and then how to display images with the best-possible performance, utilizing some new APIs in iOS 15. The sample app was built with local image files on disk. As we scroll the app, the cells are prepared off screen, and the images within them are loaded from the file system immediately. Now, we wan display images stored on a remote server. So, when cells scroll in, we may not have the image to show in the image view. When the image view is first visible, it will be blank and only filled in once the server request completes. Let's take a look at extending our registration's configuration handler to support this new approach. Here in the registration's configuration handler, we already fetched the asset from the asset store. The store will always return an image, but it may not be the full asset. It might need to be downloaded. The asset object indicates this with the isPlaceholder property. When this is true, we will ask the asset store to download the full image. When the load operation completes, it's time to update the cell's image view. Here, we take the existing cell object and set the asset on its image view. This is a mistake. Cells are reused for different destinations, and by the time the asset store loads the final asset, the cell object we have captured could be configured for a different post. Instead of updating the cell directly, we must inform the collection view's data source of the needed update.

iOS 15 introduces the reconfigureItems snapshot method. Calling reconfigureItems on a prepared cell will rerun its registration's configuration handler. Use this instead of reloadItems because it reuses the item's existing cell, rather than dequeuing and configuring a new cell. In our sample app, we'll declare a setPostNeedsUpdate method, which calls reconfigureItems on the ID passed in.

Now, back in our registration's configuration handler, when the image is a placeholder, we will download the full-size asset and call the new method. reconfigureItems will then call this handler again, but now, fetchByID will return the full asset and not the placeholder. This allows us to keep all our view-updating code in one place and asynchronously update our cells once we have data. To maximize prepare time, we can also use our downloadAsset method inside our prefetchingDataSource. Data-source prefetching is a great place to kick off network downloads for a collection-view item. It gives more time to download the asset and have it ready before the cell is visible, reducing the time users see placeholder content.

Let's take a look at how this looks in our app. It looks fine, but there are visible hitches while scrolling. They also appear to coincide with when new images are displayed. When a new cell is prepared, there is no hitching. It's only when an image is updated with the full-resolution image that we hitch. That's because all images take time to decode for display, and some images, like the larger non-placeholder assets, are too large to be decoded in time for display. When the cell registration's configuration handler is first called and the asset is a placeholder, the code begins an async request for the full-size image and completes its configuration. When the asset is finally downloaded later, the cell configuration handler is rerun with the final image. When an image view tries to commit a new image, it must first prepare the image for display on the main thread. This can take a long time, and there's a hitch when the app missed its commit deadline. Image preparation is a mandatory process that all images must undergo to be displayed.

The render server can only display images that are bitmaps, which means they are raw pixel data. Images come in many different formats, like PNG, HEIC, and JPEG, which are compressed and must be processed and unpacked to be displayed. Image views do this processing when it commits a new image, and it happens on the main thread. Ideally, we could prepare the image in advance and only update the UI when it's finally completed. That way, we never block the main thread and do not hitch. iOS 15 introduces the image preparation APIs, giving you control over where and when image preparation happens. These APIs produce a new UIImage, which only contains the pixel data that the renderer needs. There is no additional work needed once it's set on an image view. It comes in two forms: a synchronous one, which you can run on any thread, and asynchronous ones, which run on an internal UIKit serial queue.

To use it, we take a UIImage we've created and set a placeholder image on our image view. Then, calling the new API kicks off the preparation in the background on the larger image. When it completes, we can just set it on the image view. Prepared images solve a large problem in any image-heavy app, but they also come with some considerations. The prepared images contain the raw pixel data from the original image. It will remain free to display in an image view as long as it's retained in memory. But this also means it takes up a lot of memory, and they should be cached sparingly. Finally, because of their format, they are not ideal for disk storage. Instead, save the original asset to disk. One last consideration is how image preparation can utilize prefetching. Prefetching gives extra time for the image to be downloaded and prepared. Giving the process more time means the users will not see the placeholder for long, and probably not at all. In the sample app, we already have an asynchronous path for image retrieval. After the download completes, we can then prepare the asset before calling the completion handler. These assets are large, but also valuable, so once the image is prepared, we want to cache it. Our image cache uses the image's size to estimate the memory use of the prepared image. Now, when a cell asks for an asset, we check that cache before fetching it from our server. If we had smaller images, we would be able to cache more. Images can be large, and iOS 15 introduces a similar API for preparing thumbnails of images.

These can scale and prepare an image to a smaller size. It ensures that the image is read and processed with its destination size in mind, saving a lot of CPU time and memory.

You use it just like the Image Preparation APIs. First, take a UIImage and set a placeholder image on the image view. Then, call the new resizing API, with the view's size as the target size for the thumbnail.

When its prepared, just update the image view with the new thumbnail. Along with the Image Preparation APIs, it's much easier to accelerate images and avoid hitches in any app with iOS 15. When working with images, focus on having an asynchronous API that can update the UI when an image is ready. In the meantime, use a placeholder image, which is small or cheap enough to display synchronously. When used with prefetching and reconfigureItems, showing asynchronous content in collection and list views has never been easier or more performant.

To get started with fast collection and table views, first, build your app with the iOS 15 SDK to unlock many new optimizations. Particularly, ensure you validate the behavior of your collection and table views with the new prefetching. All the new APIs demonstrated here can be found in the sample code for this talk. Check it out and make sure to adopt the image preparation and resizing APIs across your app. This will ensure your collection and table views are blazing fast. Thanks for watching. [upbeat music]