Explore enhancements to App Intents

♪ ♪ Roman Efimov: Hello, everyone. My name is Roman Efimov, and I’m an Engineer on the Shortcuts team. Today, I'll be covering some exciting new features and enhancements in App Intents, helping you create even better apps. First, I'm going to start with widgets. As you know, widgets have become an increasingly important part of the iOS user experience. And now, App Intents and widgets work seamlessly together to provide new experiences through interactivity and configuration. Next, I'll move on to the improvements in developer experience that we made this year. I will cover some quality improvements, such as framework support, as well as some recent enhancements to static extraction. And finally, I’ll dive into updates to the Shortcuts app integration with App Intents. We have a pretty packed agenda, so let’s get started with a couple of exciting updates to widgets. First, let's talk about widget configuration. When you create a configurable widget, you can specify the options you want the person to be able to select on the backside of the widget. These options are called parameters, and you can define them using the same system that you use to add support for Siri and Shortcuts to your app: Intents. The configuration UI of a widget displays an ordered list of parameters that are included in the corresponding Intent. Each parameter added to the Intent is presented as a row in the widget configuration interface. In the past, you had to declare your Intents in Xcode using an Intent Definition File. Now in iOS 17, we have made it even simpler to define the schema of your widget's configuration using App Intents right in your Widget extension code.

To do that, first, you will need to start using the AppIntentConfiguration WidgetConfiguration type instead of the IntentConfiguration that you may have used previously to configure your widget. Next, define a new type that conforms to the WidgetConfigurationIntent protocol. WidgetConfigurationIntent is a sub-protocol of App Intent, and you can conform to it directly in your Widget Extension code. I've been working on a widget for my bus schedule app that displays the time and route of the next scheduled bus for a particular stop. This would allow people to quickly check when the next bus is arriving without having to open the full app. I will be using App Intents to provide the configuration Intent for my widget. To let users configure my Next Bus widget, I'm going to start by defining a struct that conforms to the WidgetConfigurationIntent protocol and includes the following parameters: a Bus Stop, chosen from the list of saved stops or by searching for a new stop, a specific bus Route, and a Direction of travel for the selected route. Once I finish defining the parameters required to configure my widget, I'll need to provide dynamic options for each of the parameter types. In the past, providing dynamic options for a parameter required creating a separate Intents extension. With App Intents, I can implement queries and dynamic option providers directly within my widget extension, leading to a cleaner and more efficient project. To learn more about dynamic options providers and queries, I would suggest checking out the “Dive into App Intents” video. Now, let's talk about migrating your existing widgets from SiriKit to App Intents. Migrating your existing widget configuration to App Intents is easy. In fact, it can be done with a single click in Xcode. Migrating your widget lets you support both the latest OS version, and previous versions before you were able to convert your Widget to App Intents. Existing configured widgets can continue working. Once you no longer need to support previous OS versions, you can remove your SiriKit Intent definition file. To migrate, navigate to your SiriKit widget configuration Intent within your Intent definition file and click the Convert to App Intent button. Xcode will produce App Intents code that is equivalent to your old Intent definition. You will need to ensure that the schema stays the same, meaning that all App Intent parameter names and types should match what you had in your Intent definition. Feel free to add new parameters to your App Intent. You can add an optional parameter or even a required one that has a default value. Existing widgets, created before the parameter was added, will pick up an empty value for that parameter or a default value if you've provided one. If you do plan to support people on previous iOS versions and let them use that new parameter, then you will need to maintain your SiriKit Intent definition file and add that new parameter in there, as well. Whenever a customer updates your app, their widget will get automatically migrated. It is crucial to test that the migration goes smoothly, as your app will only have one opportunity to do so. To learn more about migration, I encourage you to watch the “Migrate custom Intents to App Intents” video. Moving on to interactivity in widgets. Widgets can now react to button taps and toggles, allowing people to adjust settings, play media, or access any other important functionality from your app right from their homescreen. In my Next Bus widget, I would like to make the time buttons tappable. When people tap one of these buttons, I want to set an alarm in my app, ensuring that they know exactly when to leave, so they don't miss their bus! How can I do that? SwiftUI Buttons and Toggles have been updated to support App Intents, making it easy to add interactivity to widgets. If you implemented an App Intent before, you should be instantly familiar with it. First, I would need to define a struct that conforms to the App Intent protocol. Then, annotate any key properties with the Parameter property wrapper to let the system know I need the associated information to perform the action. After that, I need to implement the perform method that would actually execute the action. Finally, in my Widget view, I simply associate my SetAlarm App Intent with a Button. SwiftUI integration with App Intents is not only available for interactive widgets but also for regular SwiftUI apps. By consolidating the code into App Intents, you can reduce redundancy and ensure consistent behavior across your app. Since App Intents serve both as a configuration and as providers of interactive actions, it is quite easy to reuse the Intent code for Shortcuts. For example, my ShowNextBus widget configuration Intent can be used as both a widget configuration and a Shortcuts action that can provide me with up-to-date information when I need it. In addition, the App Intent I used to add interactivity to my widget also serves as a great Shortcuts action, allowing people to set the alarm for their preferred bus arrival time. To learn more about Widget Interactivity, check out “Bring your widget to life”. Let’s move on to several advanced techniques that can enhance the functionality and design of your Widget configuration. First, let’s talk about enhancements to Dynamic options and queries. Dynamic options is an interface for providing the available values for a parameter of your App Intent, and it can be implemented by conforming to DynamicOptionsProvider or EntityQuery family of protocols. In some cases, you may want to show options that are only available when a certain condition based on the value of another parameter is met. For example, in my widget configuration, I want to display only the route options that are available based on the Bus Stop parameter. To do that, I can use a new API in iOS 17 called IntentParameterDependency. It is a property wrapper that lets you access parameters from your Intents, within a DynamicOptionsProvider or Query. You can read these parameters and use them to create more dynamic and context-aware options. In my example, I return the available bus routes filtered by the user's selected bus stop. IntentParameterDependency works in all environments, such as Widgets, Shortcuts, and Focus Filters. In my example, I have a struct called BusRouteQuery that conforms to the EntityQuery protocol. This struct has a property called ShowNextBus, which is wrapped with the IntentParameterDependency property wrapper. This means that the Bus Route query has a dependency on the showNextBus App Intent, specifically on the bus stop parameter. Notice the suggestedEntities method. It returns an array of suggested Route objects. It first checks if the showNextBus Intent property is non-nil. If so, it filters the available routes so that the person will only see routes that match their specified bus stop. IntentParameterDependency can also depend on multiple parameters. For example, in my direction query, I want to rely on both the bus stop and route parameters to provide the direction options. You can also depend on multiple App Intents within the same query or dynamic options provider. My direction query reads parameters from two Intents: ShowNextBus and ShowFavoriteRoute. The IntentParameterDependency property wrapper is used to specify the dependencies on the busStop and route parameters for ShowNextBus Intent, and the route parameter for ShowFavoriteRoute Intent. The route computed property returns the value from either showNextBus or showFavoriteRoute, depending on which one is available.

Widget Configurations often have array parameters. For instance, my Favorite Routes widget can display the bus schedules for a person's favorite routes. However, due to limited screen space, a person should be able to select only up to three routes. So how can I declare that? New in iOS 17, you can now declare the size when defining an Array parameter. The size here can also accept a mapping from the widget family to the array size since sometimes larger widgets can accommodate more items than smaller ones. Once I have defined my Widget Configuration App Intent and its parameters, I might want to define which of these parameters are shown to the user and when. ParameterSummary defines the visual representation of an App Intent’s parameters. It powers the appearance of your App Intent in the Shortcuts editor, Focus Filters, and now in Widget Configuration. You can use parameter summaries to define which parameters are shown and in what conditions. For Widgets, the UI will first show the parameters in the Summary sentence and then any additional parameters listed in the closure. Here, the sentence contains the routes parameter, and the closure has includeWeatherInfo, so they're displayed in that order in the configuration UI. New in iOS 17, you can now use the When statement with the widget family, allowing your widget configuration to change based on widget size. For example, I want to display the toggle that shows weather information only in large widgets, while other sizes will not have this capability. So I add the includeWeatherInfo parameter to the Parameter Summary, only for large widgets. Otherwise, for small widgets, I won't add it, so the parameter is hidden. Now that I've implemented configuration for my widget using App Intents, how do I determine what happens when the person taps on it? The person is taken to my app whenever they tap anywhere in my widget. I would like to take them directly to the screen that shows information about the specific route that they selected in their widget configuration. When a person taps your widget and your app is launched, you can get the associated configuration Intent by calling the widgetConfigurationIntent method on the user activity. Once you have the App Intent, you can use it to update your app's UI accordingly. Here, I extract the content from my Configuration Intent and use it to navigate my app to the specific bus stop view for the corresponding stop and route. When you build a widget, you'll want to make sure people see it at just the right time in their Smart Stacks. To do that, you can use the new RelevantContext APIs for Widget suggestions on iOS and watchOS. Drawing inspiration from the previous INInteraction, INDailyRoutine, and INRelevantShortcut APIs, we have designed the new RelevantIntentManager and RelevantIntent, to be more Swift-friendly and work seamlessly with App Intents. Imagine a sports app that wants to surface its widget during games. With the new RelevantContext API, you can specify this Intent and its relevant date range. By providing this relevant date information, the sports app widget will automatically be suggested within the Smart Stack, ensuring that people have easy access to the game information when it's most important.

Relevance APIs are also great for surfacing your watch complications. To learn more about the watchOS side of relevance, check out "Build widgets for the Smart Stack on Apple Watch." Now that we've covered Widgets, let’s dive into the developer experience improvements that we’ve made in iOS 17 and Xcode 15. We’ll start with Framework Support. If your app requires the ability to perform App Intents from both the main app and an App Intents extension, you currently need to compile your App Intent code into both targets. Unfortunately, this approach leads to code duplication, which can introduce maintenance issues and increase the likelihood of errors or inconsistencies. This also bloats the binary size, which can negatively impact the app's performance and download times for people. In iOS 17 and Xcode 15, frameworks can now expose App Intents directly, so there's no more need to compile your code twice. You can now use the AppIntentsPackage APIs to recursively import dependencies in your app. By conforming types to the AppIntentsPackage protocol, both your app and frameworks can re-export metadata from other frameworks. I'm going to use framework support to simplify the implementation of my Bus Schedule app. I have a framework called BusScheduleIntents that provides various App Intents for viewing bus schedules. It makes itself available for re-exporting without any dependencies. I have another framework called BusScheduleUI that provides custom interface elements for the Bus Schedule app. This framework depends on and re-exports the BusScheduleIntents framework. Finally, I import the BusScheduleUI framework from my Bus Schedule app. Since the AppIntentsPackage is a protocol, I can make my SwiftUI App struct conform to it. The Bus Schedule app only needs to mention its direct dependency on the BusScheduleUI framework. I can now create a SwiftUI button within my Bus Schedule app that displays my favorite bus route by performing the ShowSchedule App Intent. The same App Intent, ShowSchedule, is also available to Shortcuts users, which means that they can create custom Shortcuts to quickly access their favorite bus route schedules without even opening the app. Moving your App Intents into Frameworks helps make your codebase simpler and more streamlined. The new framework support is especially great when building Widgets with App Intents, since you might need to access the same Intents from both your app, and your Widget extension. One more tip on keeping your App Intents code more modular: you can now create an AppShortcutsProvider and define App Shortcuts in your App Intents extensions. Previously, you had to define your App Shortcuts entirely in your main app bundle. This would mean that your app is always launched in the background when an App Shortcut is run. Now you can define your App Shortcuts in an App Intents extension. This is great for performance because you can optimize your App Intents extension to come up faster than your entire main app and avoid bringing up UI, analytics, or other non-critical code. All these features rely on static metadata extraction enhancements that we've made in Xcode 15. So let's talk about how App Intents content is statically extracted while your code is built. The Swift compiler outputs information about the types available in your code, as well as type-level and some value-level information from your App Intents implementations. Another tool then parses this information to generate a Metadata.appIntents directory in your built product, which contains files describing your App Intents, parameters, entities, queries, and more. In Xcode 15, the static extraction process has been significantly improved. It is now faster, more reliable, and works in more cases than ever before. When building your apps with Xcode 15, if Xcode is unable to statically extract something it expects, you will now see error messages directly in the Xcode editor, along with line numbers, so you know where to go and fix the problem. Before we talk about Shortcuts integration, there are two more great abilities we've added to App Intents this year which are worth mentioning. First up, is the ability to continue the execution of an Intent in your app, even if that Intent was previously running in the background. We call this the ForegroundContinuableIntent protocol. For example, if my App Intent that fetches the next bus fails to retrieve the bus schedule due to invalid parameters or connectivity issues, I could ask the person to continue in the app to resolve the issue. To do that, first, I conform my App Intent to the ForegroundContinuableIntent protocol. The ForegroundContinuableIntent protocol is designed for Intents that initially start their work in the background but may need to request continuation in the foreground. Next, I call the needsToContinueInForegroundError method, which returns an error to me. When I throw that error, the system stops performing the App Intent and asks the user to continue execution in the foreground. I can also provide an optional continuation closure that will be executed on the main thread to update my app's state after it comes into the foreground. Here, I'm using this closure to navigate my app to an error screen. Use needsToContinueInForegroundError when you want to stop the Intent execution and require action to continue, like in the previous example. We have another API you can use if you want to continue executing the App Intent, instead of stopping it entirely. For that case, call the requestToContinueInForeground method. I might use this when the bus app detects that a bus route is having a maintenance issue, and I want to present a custom UI in my app to choose an alternate route. Once the person has chosen the route, I can return that updated route from my app and continue the App Intent execution. This time, instead of throwing an error, I'm simply calling a method with try and await. The closure passed in can return a value, which I can get back within my perform. That allows me to continue executing the App Intent after getting input from the user. Here, I take the alternate route the user has chosen and return a snippet showing the next bus for that route.

In summary, use the throwing method when you want to completely stop the execution of App Intent. Otherwise, if you want to get a result from the person and use it to complete the App Intent's perform, use requestToContinueInForeground and await its result.

This year, we've also added support for Apple Pay to App Intents. You can now initiate an Apple Pay transaction directly within your perform method. Using Apple Pay in your perform is simple. I'm going to create a PKPaymentRequest instance and configure it with the necessary information. Next, I use PKPaymentAuthorizationController to present the Apple Pay payment sheet and handle authorization. A guard statement checks if the controller is presented successfully. If not, I return a dialog with "Unable to process payment." Otherwise, the payment is processed successfully. Finally, let's dig into a couple of updates to integration between App Intents and the Shortcuts app. Let's start with all the different places in the system where App Intents is integrated. App Intents is a modern way to build Shortcuts actions, and App Shortcuts make it easier to discover and use your app's functionality with Siri and the Shortcuts app. There's also integration with Focus Filters and the Action button on Apple Watch Ultra. In iOS 17, App Intents have become even more widely accessible thanks to the integration with Interactive Live Activities, Widget Configuration and Interactivity, and SwiftUI. App Shortcuts have grown too, to include support for Spotlight Top Hits and Automations. All these integrations mean that the same App Intents code can be reused in lots of different ways. Since App Intents are now deeply integrated into key system components, it's very important to ensure that the App Intents you create are being good citizens. Providing a good parameter summary is crucial to ensure that your App Intents look great when they are surfaced across the system. Write your parameter summaries so that they read like a sentence, with optional parameters tucked beneath the fold. The system will then determine the optimal visual representation for your parameter summary based on the context. While it's desirable for your App Intents to work well everywhere, there might be cases where you need to make App Intents for use within your app or in an interactive widget, and you want to hide them from other parts of the system. For example, when an App Intent invokes a local function in your app that would not make a useful Shortcut action. In that case, you can set the isDiscoverable property on your App Intent to false. I am going to add a refresh button to my Next Bus widget that will retrieve the latest data from the server. While it serves a purpose in my widget, it doesn't make a useful Shortcuts action. Since I only want this App Intent to be used from my interactive widget, I will set isDiscoverable to false for it. Note that App Intents marked as undiscoverable cannot participate in App Shortcuts either. The Intents in my app perform pretty quickly, but not all Intents do. This year, we introduced a new way for you to provide progress for long-running Intents. To report progress, simply make your App Intents conform to the ProgressReportingIntent protocol. Inside the perform() method, you can access the provided progress object. Update the progress by setting the totalUnitCount and incrementing the completedUnitCount as your Intent execution advances. The Shortcuts app will now automatically display the progress of your App Intent execution. Implementing progress reporting is particularly important for long-running Intents. It is really valuable for people to have that feedback, so they know the perform of the Intent is moving forward, and when it might complete. This year, we've also improved how your app can integrate with Find actions. Shortcuts users love being able to find content within your app by specific criteria, with actions like Find Notes. The outputs of these actions can be sent on to other Shortcuts actions, like sending an email, enabling lots of powerful workflows. In iOS 16, you could automatically get a Find action for your app by implementing an EntityPropertyQuery, declaring the criteria you want users to be able to specify. Starting with iOS 17, you can now also use the EnumerableEntityQuery protocol instead. It's really simple and easy to adopt. Implementing EnumerableEntityQuery is as simple as returning all the possible values for your entity in the allEntities() method. Shortcuts and App Intents take it from there, automatically generating find actions. The difference between EnumerableEntityQuery and EntityPropertyQuery is that with EntityPropertyQuery, we send you, the developer, the criteria, and you run the search on behalf of the user. That means you will often return a limited set of results. With EnumerableEntityQuery, you give the framework all the possible entities, and Shortcuts does the filtering. Because it returns all the entities, EnumerableEntityQuery is really simple to use, but it's also optimized for a small number of entities. It works well for cases, such as Safari's Tab Groups, but is not suitable for a large number of entities, which would be typical for the Notes app. It is also not suitable for very large entities that take up a lot of memory. In this case, use EntityPropertyQuery so you can run the search on your end, instead of returning all the possible entities at once. Lastly, I want to tell you about some updates to IntentDescription. This is the type you use to fill out the Shortcuts UI people see when tapping the details button to get more information on your action. IntentDescription includes description text, category name, and search keywords. In iOS 17 the Intent Description type has been updated with a new property called resultValueName, so you can provide a more descriptive name for the output of your action. Here, "Add Reminder" provides a resultValueName of "New Reminder" for the reminder that it has created and returned. When the "Add Reminder" action here is connected to the Show Result Action, the parameter in the Show Result action shows that name: "New Reminder." To provide a resultValueName, just use the new initializer on IntentDescription. Starting with iOS 17, you can also include an Intent description for your Find actions that are generated using the EntityPropertyQuery or EnumerableEntityQuery protocols. To do this, simply adopt the findIntentDescription property within your query types. If you categorize your actions with categoryName, this will let you display the generated Find actions under your desired category in the list of actions supported by your app. To summarize, App Intents are a great way to expose functionality of your app to the system and to your users. To learn even more about how you can turn your App Intents into App Shortcuts so people can use them right away, I suggest you check out the "Spotlight your app with App Shortcuts" session. This year, App Intents enable you to build configurable, interactive widgets and live activities, and provide an even smoother developer experience, with deeper integration into the Shortcuts app. I'm truly excited to see how your apps can leverage the new App Intents technologies to surprise and delight. Thanks for joining me.