Bring Core Data concurrency to Swift and SwiftUI

♪ ♪ Hello, everybody. I'm Michael LeHew from the Core Data team. And today, I'm really excited to talk to you about some of the enhancements the team has made to make working with Core Data and Swift a truly excellent experience. I'm going to start with a review of how Core Data is a great solution for your customers’ data persistence needs across all Apple platforms. I'll then follow up with a discussion on some of the ways that Core Data has adopted the new concurrency opportunities in the Swift runtime. Next, I'll cover some of the enhancements we've made to Core Data APIs to make your code more expressive in Swift. And then conclude the discussion with an exploration of the dynamic capabilities that Core Data is adding to our existing SwiftUI support that we introduced in 2020. Let's start at the beginning, though, where regardless of which Apple platform you are developing for, you simply have an application. At some point, your application is going to eventually encounter user data, and you'll likely need to store it somewhere. A great choice for that need is to use Core Data. Core Data is Apple's hallmark framework for application developers who desire to persist their users' data in a robust and feature-rich way. The framework takes care of many of the complexities of appropriately managing user data, from the way it was represented as an object graph in memory to how it's modeled in storage. The framework also goes through great efforts to manage important runtime considerations, like memory use and latency. The capabilities provided by the framework are scalable too. You can start with a simple, locally-persisted store and expand to use multiple execution contexts to improve performance, and even create powerful shared data experiences through CloudKit. Core Data also works across all Apple-supported platforms. And I'd really like to emphasize this last point. Once you start using Core Data, everything you learn will work on each of our platforms, from your Mac, to your iPhone, or even your Apple Watch.

And, of course, Core Data works great in Swift too. Over the past few years, we've been continuing to enhance Core Data API to be as expressive as possible in Swift. And this is a pretty exciting year for Swift with the introduction of the all-new concurrency features in the language and runtime. Since its inception, Core Data has always cared about running code concurrently, and there's a really good reason for this. Persisting data requires reading and writing to some external storage media, and this makes supporting the new concurrency model a natural fit. Let's explore how this works in the context of our Earthquakes sample application. This application reads a data feed from the US Geological Survey and uses Core Data to store information about recent earthquakes, such as their magnitude, location, and the date for which the event occurred. Architecturally, Earthquakes is a Swift application that has a view context to drive the UI and a background context to ingest the data provided by the USGS. Our sample has a local container for our application and gathers quake data from the USGS's JSON feed.

As we download data, we hand it off to our JSON parser and then import it into our background context to be turned into managed objects and saved to our local store. The view context then merges the changes to magically update our UI. In 2020, we focused on how to efficiently handle this data through the use of batch operations. Now, however, I'd like to focus on how we perform these operations concurrently. In particular, I'd like to focus on the three steps that we take to import data into the application. Step one is to download the raw data, and upon successful download, the application needs to convert it to our specific local representation. And finally, save the new objects into the persistent store. Let's go ahead and convert this to high-level code. I've abstracted each operation into its own function or closure. The app first retrieves the raw data from the server, processes it into a convenient local representation, and then imports the objects into Core Data by making a batch insert request on the background managed object context.

Written like this, it is a bit easier to visualize potential bottlenecks. Loading data over the network would be a great opportunity to consider doing our work asynchronously. Converting might also be a place to consider. Additionally, importing the data into our persistent store seems quite opportune.

Historically, though, in all of these cases, you would need to implement any such asynchronous mechanism yourself or lean heavily upon a framework-specific implementation. Let's talk about Core Data's specific abstraction.

In the case of Core Data, when you call performAndWait, the managed object context will execute the provided closure within its own protected execution environment. This can potentially tie up the calling thread until the work is complete.

If we were to visualize this, we can imagine three blocks of code which I've labeled BEFORE, DURING, and AFTER.

When our code runs, first the code labeled BEFORE gets a chance to execute on the originating thread. We then call performAndWait, and the calling thread will block until the work that happens DURING the closure completes. When that work is done, the code described AFTER will execute.

Of course, if you don't need to wait for the closure to finish, we've always offered a fully asynchronous variant. But new this year, Swift has gained powerful concurrency model with deep language integration which allows Core Data to more accurately describe the intention of our API. The syntax is a bit different. You ask to await the results of the perform, but the mental model for using this new API is exactly the same that managed object has always supported. The benefit, however, is that the concurrency is no longer a hidden implementation detail, but instead deeply integrated into the Swift language. Because of this, the compiler can automatically prevent many common concurrency bugs, such as data races and deadlocks, and even efficiently make use of resources when tasks are known to be awaiting results. Let's go back to the code and see what using this in practice is like.

As we saw, you ask to await an async-declared function. This has the potential to suspend the calling execution context until the async function yields control by returning.

It also seamlessly works with Swift's existing structured error handling by routing any thrown errors to the calling frame, just as you would expect. Now that we've seen an example of calling an asynchronous function, let's take a look at how they're declared by looking at Core Data's brand-new way to perform asynchronous work within a managed object context. There is quite a bit of functionality packed into this small stanza of Swift code, but I'm only gonna talk about a few important details, and then we'll jump into showing how you can use it in practice. Starting with the declaration of the new perform overload. You can see it's generic based on the kind of results that it can return and is decorated with the new async keyword, which opts this function into the new concurrency capabilities in Swift. Perhaps the most significant facet of this new API is that the provided closure now allows you to throw an error or return a value, saving you the effort of routing these back to the calling frame by hand. Let's see how cool this is by exploring a few different scenarios. Historically, since the concurrency was hiding inside of our implementation, one of the only ways to route errors outside of a performAndWait was to close over an optional and then check it afterwards. This could be more complicated if you were using the fully asynchronous versions of perform because you'd need to do a lot of plumbing by passing completion handlers around and making sure you used them consistently. With the new concurrency model in Swift, all of that plumbing is handled for you! Just try and await your asynchronous work, and if an error occurs, just throw it, and things will unwind to the calling frame naturally.

So now, we focused on errors, but what about results? Well, everything I've described works exactly the same. Let's look at a concrete example.

Let's sketch out what we wanna do before jumping into the code. For this example, I'd like to configure a fetch request to identify the number of earthquakes that have happened in the last five hours. As a sentence, this is a straightforward task to describe. But in code, we're going to need to reorder things a bit.

We'll first need to figure out when five hours ago was, for which we can use the Calendar APIs to calculate this in a robust way. We'll then configure a fetch request with a predicate in terms of that date and ask for a count result type. In code, it looks pretty much like our plan. We use Calendar's offset API to calculate five hours before now, and then configure a Quake FetchRequest to return a count result with a predicate that matches the dates that we care about. Historically, returning results followed a similar pattern to the way we captured errors. You would close over any state that you needed to mutate, perform your computations in the managed object context, and then later, use the result after regaining control.

Now, we can simply just try and await the result of our perform call and return the result for perform directly to our calling frame. The rest of the code is exactly the same. It's only the by-hand value routing that we avoid along with any potential bugs or nuances that that code may have had. This new code is quite succinct and expressive.

It is worth mentioning, however, that there are times where you should be cautious. Let's look at a different example to see why. This example attempts to return the most recent earthquake as a managed object. While the new API makes it really easy to return values, it is not safe to return managed objects that are already registered to a managed object context. It is only valid to refer to such registered objects within the closure of a call to perform. Instead, if you need to refer to a managed object between different execution contexts, either make use of the object ID and refetch as needed, or make use of the dictionary representation option of the fetch request. Now before we look at one more example, I'd like to cover a detail that I haven't talked about yet. And that detail is the ScheduledTaskType. So far, every async perform that we have seen has been in terms of the default value for this option: .immediate. There is a second option called .enqueued, and to understand the difference between these two scheduling approaches, it helps to think about what specifically happens inside a managed object context when you ask to schedule work. As we've seen, .immediate behaves a lot like a Swift-async-aware version of performAndWait. If you're running on a different execution context and ask to await work performed on the background context, you will wait until it is scheduled and completed.

If you're already on the same execution context, however, the work will be optimistically scheduled right away.

.enqueued, on the other hand, is a bit simpler. It simply always just appends the requested work to the end of the contexts' work set, regardless of the affinity of the originating call site. Let's go ahead and look at one more example. All of these async features can be adopted by you as well. Here, I factored the import logic that we have been talking about into a new importQuakes function, decorated with the new async keyword. This function is, in turn, implemented in terms of other async functionality.

Now anyone can await upon this new function to take advantage of the new concurrency features in Swift. Let's summarize what we've seen so far. Taken altogether, this new API brings the support for Swift's structured concurrency right into Core Data. The new variants of the perform API are just Swift concurrency-aware versions of the existing Core Data API you already know and love. We strongly encourage you to take advantage of this new API in your applications.

Further, NSManagedObjectContext is not the only type in Core Data that supports performing tasks within its protected concurrency domain. We're also adding similar API to both NSPersistentContainer and NSPersistentStoreCoordinator. The general shape and behavior of these APIs are quite similar to what I've already described. But with all that concurrent power, I would be remiss to not offer the advice of using existing debugging tools available at your disposal. Of course, the Xcode-provided address and thread sanitizers are incredibly helpful for catching bugs you might not even know existed. These can both be found in the Diagnostics pane of the scheme editor's Run settings. Each sanitizer detects different kinds of issues, including validating safe memory use assumptions and appropriate use of data from multiple threads. It's always a good idea to qualify your applications and their associated tests with both sanitizers before you release your software to your community of users.

And while the sanitizers are useful in all contexts, I also wanna highlight that Core Data provides a special runtime flag that you can enable to get more domain-specific help. By enabling this option, Core Data will turn on a number of useful assertions to validate internal locks and confirm appropriate use of various Core Data types.

Adopting Swift concurrency support is not the only change made to Core Data this year. Every new API that we are introducing, from CloudKit sharing to the new Spotlight integration, has been crafted with its presentation in Swift in mind. This year, we have a separate session for each of these topics, and I encourage you to check them out. We additionally made a pass throughout the entire framework to identify other places where we can make improvements in Swift, and I'd like to show you a few of these now, starting with the kinds of different persistent stores that we support. Recall, persistent stores describe how you physically want to store your customers' data. Core Data currently provides four such stores: XML, binary, in-memory, and SQLite. And you use these identifiers all the time. New this year, we've gone ahead and gave these more natural names in Swift. The existing names will continue to work, but the new API that consumes these will be a lot more ergonomic to use due to the shorter names and ability to autocomplete these symbols. Of course, persistent stores are not the only thing in Core Data that concerns itself with types. After all, the framework is all about storing typed data, and such types are described with attribute descriptions.

And this year, we're adding a new extensible enumeration to attribute description that provides a much more natural syntax for working with their types. Let's take a look at these in action by writing a unit test that can validate that our runtime model matches what we designed in the Xcode model builder.

For simplicity, we'll just try to validate a single runtime type defined by our earthquake object model, but you can imagine how this would scale. This might seem like a small test to write, but it's a good thing to validate as it could speed up more interesting diagnoses in the future. To write this test, we'll write a quick helper function in terms of the new attribute type. Let's go ahead and describe this function now. We'll start with the signature, which expects an attribute name, the entity description that we care about, and the type, described in terms of the new AttributeType enumeration.

The definition of this utility is fairly straightforward. We first validate that we have an attribute with the provided name, failing the test if we can't find it. And then we validate that the type of the attribute is as expected. And that's really all there is to it. We could repeat this for each entity and property and enjoy peace of mind that our runtime behavior matches the model we defined.

And this is just a sampling of some of the ergonomic improvements that we've made to Core Data enumerations in Swift this year.

Up till now, I've been focusing on a lot of lower-level framework interactions and how they manifest in Swift, but what about presenting data to your users? In 2020, we introduced a number of conveniences for working with Core Data in SwiftUI. And now, my colleague, Scott, has quite a bit to share with you about the new enhancements we are introducing this year. Scott? Thanks, Michael! There are a bunch of improvements to the experience of using Core Data with SwiftUI this year, starting with lazy entity resolution in fetch requests, which relaxes the requirement that apps have their Core Data stacks set up before they construct their views. Also this year, fetch requests pick up dynamic configuration for their sort descriptors and predicates.

And there's a new kind of fetch request in town that supports sectioned fetching. I'm going to walk through each of these using our Earthquakes sample app that Michael mentioned earlier in the talk, starting with lazy entity resolution.

Probably in your app, you have some code like this. This container property here isn't really necessary to support the code in this type, or even the broader app. All that stuff gets what it needs from the QuakesProvider type directly. No, this property exists to make sure the Core Data stack has been set up before any of the views in the environment try to refer to any entities before the model has been loaded. See here that the environment view modifier is called after ContentView has been initialized. This trick isn't necessary anymore when deploying against this year's SDKs. The FetchRequest property wrapper now looks up entities by name lazily at fetch-time, at which point the environment has guaranteed that the Core Data stack has been set up, so it's now safe to delete this property...

And just refer to...

the QuakesProvider shared container directly in the environment call. Moving on to some new APIs, FetchRequest now supports dynamic configuration. There are two new properties on the wrapped value for directly changing the request's predicate as well as its sort descriptors, which are expressed both with the NSSortDescriptors that you're used to as well as a new SortDescriptor value type that provides more convenience and safety when fetching entities with automatically generated managed object subclasses.

And finally, there's a configuration binding with the same set of properties as the wrapped value for easier integration with views. Before this new API, I would've had to design my views so the sort and predicate parameters were passed through a view's initializer, but that made it really tricky to support things like configuring my fetch request using controls in a toolbar. This friction is eliminated by these new dynamic configuration properties, and I'd love to show you how to use them by adding sorting and filtering to the Earthquakes sample app. Let's look at sort descriptors first. By default, the Earthquakes app sorts by recency, but I'd also like to order them by magnitude, so I'm going to add a menu that lets me control the results' order.

I'll start by adding a static array of tuples...

containing the sort descriptors I'd like to support as well as names for them. See here that they're also using the new SortDescriptor type.

I'll also want a bit of state to track which sort order I am currently using. I've already created a type for this, so I'll add it as a property of the content view. Now I'll add a toolbar menu to the list view...

That modifies the selected sort as well as an onChange modifier that updates the fetch request's sort descriptors.

Now in the preview, we can see the new menu, and I can use it to sort the earthquakes by magnitude. Great! Now to add filtering. I'd like to filter based on the earthquake's place. The first thing I need is some state for the search field's text. And I'll make a binding property...

for the search field that updates the fetch request.

With those in place, all I need is the UI. Conveniently...

searchable takes a binding to a string, so we can just plop that right in here.

Now in our preview, we can narrow down all the earthquakes near a place matching a sandwich by just typing "sandwich" in this new field here.

And that's dynamic configuration for FetchRequest. Another commonly requested piece of functionality is support for sectioned fetching, which arrives this year as a new property wrapper type called SectionedFetchRequest. This type supports the same new dynamic configuration properties as FetchRequest, but it gets initialized with an additional parameter, a key path to a property that identifies the section, a lot like NSFetchedResultsController.

But unlike the fetched results controller, the property that identifies the section can have any type you like, so long as it's hashable. This gets encoded in the type system using an additional generic parameter on SectionedFetchRequest. Finally, this new type wraps a two-dimensional result type. SectionedFetchResults is a collection of sections, each of which is itself a collection of results. Each section also has a property with the section identifier.

This is really easy to adopt, so I'm going to add sectioned fetching to the Earthquakes app. First, I update my FetchRequest declaration.

Quake already has a property for day, so I'm going to use that for the sectioning key path.

Next, I need to update the body property...

To match the new sectioned results type.

The outer loop here iterates over the sections, so I'm emitting a Section view here, and each section itself is a collection of Quakes, so this inner ForEach iterates over the section, just like I was iterating over the results before.

If we look over at the preview, I've now got earthquakes ordered by time and sectioned by day. And SwiftUI even gives me automatic support for collapsing sections.

This new SectionedFetchRequest type supports the same dynamic configuration properties as FetchRequest as well as an additional configuration property for the section identifier key path. This is super important because it's not actually safe for us to change the sorting anymore. It could cause the sections to be discontiguous because time and earthquake magnitude aren't perfectly correlated, which is probably for the best. To fix this, I need to update the sorts up top...

so each has a corresponding section identifier key path.

Next, down in the toolbar...

I need to update the section identifier key path each time I update the sort descriptors.

But here's the important part. Changes to the request are committed whenever the results getter is called, so to update both the sorting and the sectioning safely...

I need to update the configuration on a reference to the results that I've pulled into a local.

Now in the preview, we can see that changing the order also changes the sectioning. We can flip between earthquakes ordered by time, sectioned by day, and earthquakes ordered and sectioned by magnitude.

And there we have it: lazy stack initialization, dynamic configuration, and sectioned fetching, all easily applied to an existing app using iOS 15 and macOS Monterey.

So, to recap, Core Data is your one-stop shop for managing your app's data persistence needs across all of Apple's platforms. It harnesses the new concurrency features available in Swift through a new perform API, and still has powerful thread safety debugging built right in.

It's got new enumeration interfaces that make store and attribute types even more natural to use in Swift, plus CloudKit sharing and Spotlight integration. And it's easier than ever to connect your data to your views using SwiftUI with dynamic configuration and sectioned fetching.

There's lots more new stuff to learn related to these topics. We suggest checking out the collections "Simplify with SwiftUI and Meet Swift Concurrency." And that's it! I'm really looking forward to seeing what you all build with these new APIs. [upbeat music]