Dive into App Intents

♪ Mellow instrumental hip-hop music ♪ ♪ Hi, folks. My name is Michael Gorbach from Shortcuts Engineering. Thanks for tuning in for this deep dive into App Intents, our new framework for exposing your app's functionality to the system. Here's the plan for our dive. After a quick introduction, I'll talk about intents and their parameters, and how to define entities. I'll go over some powerful finding and filtering features you can build, and how your intents can interact with the user. Lastly, I'll cover App Intents architecture and lifecycle. Let's start at the beginning. In iOS 10, we introduced the SiriKit Intents framework, which lets you hook up your app's functionality to Siri domains like messaging, workouts, and payments. Now we're introducing a new framework called App Intents. It has three key components. Intents are actions built into your app that can be used throughout the system. Intents use entities to represent your app's concepts. App Shortcuts wrap your intents to make them automatic and discoverable. Let's talk about a couple of the ways that App Intents can make your app's functionality available in more places, and benefit your customers. With App Shortcuts, everyone can use your app's features with their voice through Siri, without needing to set anything up first. The same adoption also makes your intents appear in Spotlight when people search for your app and when your app's suggested. This will put your work front and center. Using App Intents, you can also build Focus Filters, letting customers customize your app for a specific Focus. For example, they might set up their Calendar app to only show their work calendar while they're actually at work. Check out this session to learn more about how to adopt Focus Filters. With App Shortcuts, your intents show up in the Shortcuts app automatically, without needing to be added manually. Integrating your actions into Shortcuts is incredibly valuable for customers because they can run shortcuts, and take advantage of your app's features, from so many places throughout the system. They can run shortcuts with a single tap on the Home Screen, from the menu bar on macOS, and in many other ways. They can even set shortcuts up to run automatically with automations. Supporting shortcuts multiplies the power and capability of your app by connecting it to the entire Shortcuts ecosystem, harnessing the power of an array of apps from Apple and other developers. That's because a shortcut can combine actions from multiple apps, letting users invent entirely new features and capabilities without you needing to do any work. If you want to learn how to make your actions work well with others and fit seamlessly into this ecosystem, check out our design talk. Our goal in building App Intents was to make it a joy to develop for. App Intents is concise. Writing a simple intent can take only a few lines of code, but the API also scales to deeper and more customizable actions. App Intents is modern. We've gone all in on Swift, leveraging result builders, property wrappers, protocol-oriented programming, and generics. These APIs just couldn't exist without cutting-edge language features. Adopting App Intents is also easy, because it doesn't require re-architecting your products and targets or creating a framework. It doesn't require an extension and can be adopted right in your app. And App Intents code is maintainable. Like SwiftUI, App Intents uses your code as the fundamental source of truth, avoiding the need for separate editors or definition files. This lets you rapidly build and iterate on your adoption, and simplifies maintenance because everything lives in one place. With that said, let's explore these new APIs, starting with the intent, the central building block of our new framework. An app intent -- or "intent" for short -- is a single, isolated unit of functionality that your app exposes to the system. For example, an intent could make a new calendar event, open a particular screen, or place an order. An intent can be run by the user on request -- like by running a shortcut or asking Siri -- or automatically -- like using Focus filters or a Shortcuts automation. When an intent is run, it will either return a result or throw an error. An intent includes three key pieces: metadata, or information about the intent, including a localized title; parameters, which are inputs that the intent can use when it's run; and a perform method, which does the actual work when the intent is executed. Our starting point today is this Library app. Since I'm a huge bookworm, it's all about tracking books I've read, want to read, or am currently reading. Each category is shown in separate tab of the app that I call a Shelf. My users visit the Currently Reading shelf all the time, so I'm going expose an app intent to make opening it quicker and more convenient. I'll create an OpenCurrentlyReading intent here by defining a Swift struct that conforms to the AppIntent protocol. I need to implement only one method, called perform. In my app, I've already got a navigator that can open tabs, so implementing the intent for me is only a few lines of code. I'll annotate the perform method with @MainActor, since my Navigator expects the main thread. My intent also needs a title. Like all the other strings I'll be showing you today, this will get localized automatically if I add the key to my strings files. This is all I need to do to get a basic app intent working. Now that it's defined in my code, it will automatically appear in the Shortcuts editor, where my user can add it to a shortcut. Just exposing this intent provides huge leverage, because once customers turn this intent into a shortcut, it can be used from a ton of places in the system, including all of these. To make my new intent easy to use and discover, I'll also add support for App Shortcuts. With a little bit of code, I can make my intent show up automatically in Spotlight and the Shortcuts app, and I can define a phrase that people can say to Siri to use this intent with their voice. Check out the "Implement App Shortcuts with App Intents" session to get all the details. So far, I've exposed an intent to open the Currently Reading shelf. Next, let's generalize it, adding a parameter so it can open any of the shelves. I have an enum that represents shelves. In order for it to be used as an intent parameter, I need to conform it to the AppEnum protocol. AppEnum requires a String raw value, so I'll add that first. It also requires that I provide localizable, human-readable titles for each of my enum cases. These must be provided as a dictionary literal, since the compiler will read this code at build time. Finally, I'll add a typeDisplayName: a user-visible, localizable name for this enum type as a whole. I'll use "Shelf." In an intent, each parameter is declared using an @Parameter property wrapper, which is initialized with information about the parameter, like the title. Here, I define a new shelf parameter, which I read in my perform method. Parameters support all of these types, including numbers, strings, files, and more, as well as entities and enums from your app. Here's how this intent looks in the Shortcuts editor. Note that the shelf parameter appears in a table row. I can make the UI more streamlined, and make it fit better into Shortcuts, by using the ParameterSummary API. The Parameter Summary is a sentence that represents your intent and its parameters in the editor, like "Open ." For best results in Shortcuts, you should always provide a Parameter Summary for every intent you create. You can also define which parameters show up below the fold and which are hidden. These APIs can do some pretty cool stuff, like varying the summary based on the actual values of any parameter of your intent, using the When and Otherwise APIs, or the Switch, Case, and Default APIs. To add a parameter summary, I implement this static property. Here I'll return the string "Open", and interpolate the shelf parameter. The last thing I need to do to get Open Shelf working is make sure that the intent opens the Library app when it's run, like this. Opening the app by is controlled by the static property, openAppWhenRun. It defaults to false, which is great for most intents. But for intents that open something in the UI like this one, I'll need to set it to true. I just created an intent to open shelves. This is super simple because the set of shelves is fixed. But what if I wanted to build an intent that opens Books, the set of which is dynamic, not fixed? For that, I'll need entities. An entity is a concept that your app exposes to App Intents. You should use an entity instead of an enum when the values are dynamic, or user-defined, like a note in Notes or a photo or album in Photos. To provide instances of entities, your app can implement queries, and return entities as results from intents. I'll start by making an intent to open a book in the app. In the Shortcuts editor, it should look like this. When people tap on the Book parameter, they'll get a picker to choose a book, including a set of suggested entities that my app has provided. They can also find any book in their library with this search field at the top of the picker. Before I build the intent itself, I'll need to create a book entity and the corresponding query. An entity contains at least three things: an identifier, a display representation, and an entity type name. To add an entity, start by conforming a struct to the AppEntity protocol. Here, I'll define a new struct for the BookEntity, but I could also conform an existing type from my model. You provide an identifier by conforming your entity to the Identifiable protocol. App Intents uses this identifier to refer to your entity as it's passed between your app and other parts of the system. The identifier should be stable and persistent, since it might be saved in a shortcut created by your customers. The display representation is used to show this entity to the user. This can be as simple as a string of text, like a book title. You could also provide a subtitle and an image. The typeDisplayName is a human-readable string representing the type of an entity. In this example, it's "Book." Now, to round out the book entity, I need to add a query. A query gives the system an interface for retrieving entities from your app. Queries can look up entities in a few ways. All queries need to be able to look up entities based on an identifier. String queries support search. And later, you'll run into property queries, which are more flexible. All queries can also provide suggested entities, which allow the users to pick from a list. Every entity should be associated with a query so the system can look up instances of that entity. You provide a query by making a Swift struct that conforms to the EntityQuery protocol. The basic query has only one required method, which you implement to resolve entities given an array of identifiers. I've implemented this by going to my model database and finding any books matching those identifiers. Now, I need to hook up the query to the entity. I do this by implementing the defaultQuery static property on the BookEntity type and returning an instance of my BookQuery. When the user picks a book, its identifier will be saved into the shortcut. When the shortcut is run, App Intents will pass the identifier to my query to retrieve the BookEntity instance. Now that the BookEntity type conforms to the AppEntity protocol, I can use it as a parameter in my OpenBook intent. The perform method uses my Navigator to navigate to the book.

In order to support the book picker, my query also needs to provide suggested results. To do that, I need to implement one more method on query, returning all the books added to my Library app. Shortcuts will fill the picker with these results. Notice that the Shortcuts UI has a search field on top. My app could have a lot of book entities, so I should really run the search in my app process, against my database directly. The StringQuery API lets me do that. Adopting the StringQuery subprotocol gives me one more method to implement, called entities (matching string:), to return results given a string. Here, I've implemented it as a simple case-insensitive match against the title of the book, but I could have done fancier stuff like searching through the author or series name, for example. If I have a huge list of books, and a smaller list of favorites, I could return just the favorites in suggestedEntities, and rely on entities (matching string:) to allow my users to search across the longer list. Now I've exposed a way to open books in my app, and built a book entity and book query in the process. I can use the same entity and query to create more intents. My next task is to build an intent to add books to the library. Customers can quickly add books while browsing online using a share sheet shortcut, or they can tell Siri on HomePod to add a book without even looking at a screen. Building intents like this that manipulate your model directly without showing your UI can really empower your users. Here's the implementation of my AddBook intent, taking as parameters the title of the book and an optional name of the author. It also includes an optional note to record which friend recommended the book. The perform method will add the book to the library by looking it up with an API call using async/await. It will throw an error if it can't find a match. To localize this error, I conform my error type to the CustomLocalizedString ResourceConvertible protocol. I'll return a localized string key from this property, and add the key to my strings files. This Add Book intent is incredibly useful as is, with Siri, widgets, and more. But it gets even more flexible if it can be combined with other intents. With a little bit of work, I can allow combining my Add Book intent with the Open Book intent I built earlier, passing the result from one to the other. To do so, I'll have the Add Book intent return a value as part of its result. Notice that my perform method's return type has picked up a new protocol to represent the value I'm returning. Now, users can connect the result value of this intent to other intents that take a book entity as a parameter. The Add Book intent and the Open Book intent pair quite naturally together, so you can make a shortcut that adds a book and then immediately opens it in the library. It's a common pattern to return a result from an intent and open it in the app. App intents have a built-in way to express this called the openIntent. If I add an openIntent, customers will get a new switch in Shortcuts called "Open When Run." If they turn the switch off, they'll be able to use this intent as part of a shortcut in the background without interruption. If they leave the switch on, the newly added book will be immediately opened in my Library app. Adopting openIntent is as easy as creating an instance of the Open Book intent and returning it as part of the result. When this intent is run, if the Open When Run switch is on, the Open Book intent will automatically be performed after the Add Book intent finishes. There's a lot more you can do with entities and queries. With the next set of APIs, AppIntents opens up some powerful abilities you never had before with the SiriKit Intents framework. Let's take a look at how you can expose more information from your entities, and allow customers to find and filter based on that. So far, I've added all the basic requirements to my book entity. But to let people integrate books more deeply into their shortcuts, I'm going to need to expose a bit more about my books. Entities support properties, which hold additional information on the entity that you want to expose to users. In this case, I'll add the book's author, publishing date, read date, and who recommended it, so that people can use those properties in their shortcuts. I add properties to my BookEntity using a property wrapper called @Property. Properties support all the same types that parameters do, and each one takes a localized title. With these new properties, my customers can now use magic variables in Shortcuts to pull out each new piece of information when working with a book entity When using the earlier Add Book intent, they can use the author or publishing date of a newly added book in their shortcuts.

When you combine properties with queries, your app automatically gets these incredibly powerful Find and Filter actions in Shortcuts, with this flexible predicate editor UI. Now, my customers will be able to find and filter books based on date read, title, author, and more. It's a piece of cake to find all the books by Delia Owens, for example. Using the Sort by and Limit options, you can support even more advanced queries, like find the three most recently published books by Delia Owens. A customer can use these building blocks to do some pretty cool stuff, like finding the three most common authors in their collection. To enable all this, I'll need to adopt another kind of query called a property query. Property queries find entities not based on a string, or an identifier, but on the properties within the entity. There are three steps to implementing property queries. First, you declare query properties, which specify how your entity can be searched using its properties. Then, you add sorting options, which define how query results can be sorted. And finally, you implement entities(matching:) to run the search. The query properties declare every way AppIntents can search on the entity associated with this query. Each one lists a property of my entity, and the comparison operators -- like contains, equal to, or less than -- that are available for it. Here, I list "less than" and "greater than" comparators for my date properties, and "contains" and "equal to" for my title property. Query properties map each combination of property and comparator into a type of your choice, called the comparator mapping type. Here, I am using CoreData, so I'll use an NSPredicate. If I was using a custom database or a REST API, I could design my own comparator type and use that instead. Here's the code to set up the query properties for my books. I conform BooksQuery to the EntityPropertyQuery protocol. Then I implement static var properties using the QueryProperties result builder. Each entry specifies a keyPath of a Property that can be queried, and within it, each comparator that is applicable to that property. For each comparator, I provide an NSPredicate, because I've chosen NSPredicate as my comparator mapping type. When the system asks my app to return results for the query, it will provide back the NSPredicates that I'm constructing here. There's a similar definition for sorting. This is a list of all the properties my model can sort books by. In this case, I allow sorting by title, date read, and date published. Finally, I implement entities(matching:), which queries my database and returns matching entities. This method takes an array of the comparator mapping type I used in the previously defined query parameters -- in this case, NSPredicate. These predicates are describing what criteria on the properties of my entity I want to query by. It also takes a mode, indicating whether to combine the predicates with "and" or with "or," key paths to sort by, and an optional limit for the number of results. My implementation uses these parameters to perform a query against my CoreData database. What can customers do with this property query? They can pick a random book from their library to read. They can find all their books published in the early part of the twentieth century. They can leverage the Shortcuts ecosystem and make my app more useful by connecting it to others. For example, they can use a spreadsheet app to export all the books they read this year to a CSV file. Or, they can use a graphing app to make a chart of how many books they've read every year over the last 10. And that's just the beginning. This kind of deep App Intents adoption really lets customers use your app to do what they need it to do, making it a critical part of their workflow. Each of these integrations -- like making graphs, for example -- is a feature you don't have to build. When your intents are performed, your app may need to interact with the user to show or speak a result, or to resolve ambiguity, whether it's a Siri request or a shortcut. App Intents supports a number of these interactions: dialog for giving text and voice feedback to your users when an intent has completed, and snippets for giving visual feedback. Request value and disambiguation for asking the user to clarify values for intent parameters, and confirmation for verifying parameter values or checking with the user on intents that are transactional or destructive. Dialog provides a spoken or textual response to the person running an intent. It's really important to provide dialog for intents to work well in a voice experience. In my Add Book intent from earlier, I'll add a needsValueDialog that's spoken when asking for a book title and a result dialog returned from my perform method. These will be read or shown by Shortcuts or Siri across our many platforms. You can think of snippets as the visual equivalent of dialog, letting you add a visual representation to the result of your intent. To use a snippet, just add the SwiftUI view of your choice as a trailing closure to your intent result. Like with a widget, your SwiftUI view will be archived and sent over to Shortcuts or Siri. App Intents also supports asking the user for a value by throwing requestValue. For example, this comes in handy when you need a value for a parameter that is sometimes optional. Here, requestValue helps me when my string search returns more than one book. In this case, I prompt and ask for an author to narrow the book search down. requestValue gives me an error I can throw, which will prompt the user, and rerun the action with the updated author name. Disambiguation, meanwhile, is great when you need the user to choose between a set of values for a parameter. This gives me an even better way to handle multiple possible results in my Add Book action. Here, I get a list of author names from the generated books, and request disambiguation with those possible values. The user will be asked to pick between them, and I'll get the result back. Lastly, App Intents supports two different kinds of confirmation. The first kind is confirmation of a parameter value. You might use this when you have a guess at what that value should be but you want to confirm, just to make sure. When adding a book, sometimes the web service I call to look up books by title returns a couple matches, but one of them is by far the more popular. In these cases, I'm going to assume that the user meant to add that popular book, but I'll add confirmation to make sure I got it right. To do that, I'll call requestConfirmation on the title parameter. The second kind is a confirmation of the result of an intent. This is great for placing orders, for example. If I wanted to monetize my Library app and add ordering through a bookstore, I'd want to make sure that I have the order right. To do this, I could call requestConfirmation on my intent, passing in the order to be placed. I'll specify a snippet here too, showing a preview of the order. I prefix the call with "try" because requestConfirmation will throw an error if the user cancels instead of confirming. Before I leave you, there are a couple aspects of the App Intents architecture I want to cover, which you should know as you adopt the framework. There are actually two ways to build your App Intents: within your app or in a separate extension. Of these, implementing intents directly in your app is the simplest. This is great because you don't need a framework or to duplicate your code, and you don't need to coordinate across processes. Using your app also gives some higher memory limits, and it gives you the ability to do some kinds of work that are harder from an extension, like playing audio. Your app can be run in the foreground if you implement openAppWhenRun on your intent to return true. Otherwise, it will be run in the background. When running in the background, your app will launch in a special mode without scenes being brought up to maximize performance. In fact, if you implement background app intents in your app, we strongly encourage you to also implement scene support. Or, you can build your app intents in an extension. This has a couple advantages. It's lighter weight, because the extension process only handles app intents and doesn't require spinning up your app. If you are handling Focus intents, using an extension also means that you'll immediately get intents performed on your extension when Focus changes, without the requirement that your app is running in the foreground first. An extension is a bit more work, since you'll need to add a new target, move some code into a framework, and handle coordination between your app and the extension. To create an App Intents extension, go to File > New Target in Xcode and choose App Intents Extension. With App Intents, your code is the only source of truth. App Intents achieves this elegant developer experience by statically extracting information about your intents, entities, queries, and parameters at build time. Xcode will generate a metadata file inside your app or extension bundle during your build process, containing information received from the Swift compiler as it runs on your code. To make sure all of this works, keep your App Intents types directly in the target or extension, not in a framework. Similarly, your localized strings should be found in a strings file within the same bundle where your App Intents types live. For those of you who have existing apps with SiriKit Intents that you want to upgrade, if you adopt intents to integrate with widgets, or domains like messaging or media, you should keep using the SiriKit Intents framework. But if you add custom intents for Siri and Shortcuts, you should go ahead and upgrade to App Intents. You can start the upgrade process by clicking the Convert to App Intent button in your SiriKit Intents definition file. Integrating your app into Shortcuts with App Intents is a great way to maximize your leverage as a developer, because by doing a small amount of work to adopt App Intents, you create a large amount of value for customers. Thank you for joining! I really hope that you'll try out App Intents today and give us your feedback. I'm excited about how this new framework can help you surprise, delight, and empower folks using your apps! Happy reading and hope your WWDC is epic! ♪