Spotlight your app with App Shortcuts

♪ ♪ Jerry: Hi, I'm Jerry, an Engineering Manager for Siri and App Shortcuts. Today, I'm gonna share with you how you can spotlight your app with App Shortcuts. I'll start by looking at why you should adopt App Shortcuts to help your app create quick, habitual actions that can be used from all over the operating system. Then I will go over the basics of implementing an App Shortcut. After that, I will go over how to make your App Shortcuts great for both visual and voice use, as well as new ways App Shortcuts have improved in iOS 17. And finally, I'll wrap by looking at new ways App Shortcuts can be extended across a number of Apple devices beyond iPhone. To start, App Shortcuts make it easy to discover and use your app's functionality. They are automatically available as soon as your app is installed and require zero user setup. Unlike in iOS 15 and before, there's no need to use the Shortcuts app or use the Add To Siri button to set up an App Shortcut before using it.

App Shortcuts can be run from Siri by speaking one of their trigger phrases. They're also featured prominently right in the search results when searching in Spotlight, and they're found in the Shortcuts app where they can be part of powerful user Shortcuts and Automations.

App Shortcuts can help raise the visibility of your app's features to your users throughout the system and be great launching points for both functionality that opens your app or simply do things in the background, so the user can maintain their focus. They are fast, memorable ways to lower the friction for using your app's core features, and also make them usable for more contexts, including hands-free with just your voice or in the home with HomePod. With memorable App Shortcuts targeting your app's most important features, you can help people discover, remember, and build a habit around using your app.

For example, throughout this session, I will be working with a hypothetical app called "Demo" for managing my to-do lists. I want to expose a new bit of functionality, summarizing my lists, to the system. This will let me get a quick idea of what's left to do either via Shortcuts or by asking Siri. This sort of quick and lightweight feature is perfect for App Shortcuts. They let people get what they need from my app seamlessly and be on their way.

I'll quickly walk through how you would implement an App Intent for summarizing a to-do list, and then implement an App Shortcut, so my users can summarize a list with a verbal command from Shortcuts and be highlighted in Spotlight.

App Shortcuts are built with the App Intents framework, our Swift only framework built from the ground up to make it faster and easier to build great intents right in your Swift source code. All App Shortcuts begin by defining an intent in your source code. Intents represent individual tasks that can be completed with your app, like creating a to-do list, summarizing its contents, or checking off an item. After you create an app intent, you can create an app shortcut with it, so it can be used from Spotlight or Siri. This associates your intent with Siri trigger phrases, titles, and symbols that are needed. Once you have both definitions, all you have to do is build and test your app to see it work.

I'm going to start by creating a very simple app shortcut. This app shortcut will let me create a new empty to-do list right from Spotlight or Siri by entering the name of the list. Implementing this will be a simple two-step process: define the app intent and define the app shortcut. I start by importing the App Intents framework. And then I implement a new struct for my intent, conforming to the AppIntent protocol. I include my intent's actual functionality in the perform method that will be called by the system.

And finally, I implement an AppShortcutsProvider struct. Each app can have at most one struct conforming to this protocol. In the AppShortcutsProvider, I can specify all the app shortcuts my app supports. Here I've added the Create List app shortcut. Each app shortcut has a few components: The intent that will be invoked when triggered. The phrases that Siri should recognize to perform this app shortcut. You can have multiple of these per App Shortcut. A short title and system image name, so the system knows how to represent this app shortcut visually.

That's all I need to implement for a simple app shortcut. By running my app I can immediately start creating to-do lists right from Siri or the Shortcuts app, all by only creating two structs in my code.

I want to try something a bit more advanced though, where I can get a quick summary of my tasks by providing a specific list to be summarized. This will touch on two key concepts in App Intents: Entities and queries. Let's try it out. First, I need to implement a SummarizeList App Intent. Like before, I need to create a struct that conforms to the AppIntent protocol. For the TodoList struct, I've left a stub implementation. I'll come back to that. I have a convenient function already in my app that returns the summary of a list, so I will just use that. I can return this summary as a custom dialog as part of the app intent's results. This will let Siri speak or show the result.

Every intent needs a title that the system can display.

Intents can take inputs by simply declaring variables and annotating them with the @Parameter property wrapper. In this case, I've referenced a ToDoList struct that, as mentioned, is just a stub for now.

Now I'd like to tell you about two concepts that are useful for many app intents: entities and queries. Entities are concepts relating to your app that your users will want to reference. In my app, it makes sense to have a to-do list entity, since that's a logical object that people will want to refer to when performing tasks in my app. In other apps, you might have entities for users, documents, orders, and so on. Entities from your app can be put to use in app intents when those intents use the entities as input parameters. In my app, my "SummarizeList" intent takes a to-do list entity as a parameter. In order to find entity instances that fit into the parameters of an app intent, the system relies on queries. At runtime, the system instantiates and calls query objects to find entities based on various search parameters. Queries return instances of relevant entities to the system. The system later uses these entities when running an intent. For my app I'll need to implement the to-do list entity, as well as a query to help the system search for relevant to-do lists. Back to the list summary app shortcut I was working on. I'll expand my stub implementation of ToDoList by conforming it to AppEntity. I'll implement the typeDisplayRepresentation property so that the system knows how to describe this entity type. Likewise, I'll implement a displayRepresentation, so the system knows how to describe this specific instance. Note that if I want the entity shown at the top level in Spotlight or Shortcuts, my entities need to have an image or symbol in the display representation. And finally, I declare a default query for this entity.

I've implemented a simple query for my to-do list entity. At a minimum, my query must be able to retrieve a to-do list by its identifier. I'll also want to implement suggestedEntities. This lets my app have control over what to-do lists I think are most relevant to my user. The system will use these suggested entities to populate my app shortcut automatically. With just the SummarizeList app intent and its associated entity and query structs, I can now use this function from inside the Shortcuts app and as part of Automations. I'm not done yet though. I want to make sure this functionality is accessible from Siri and is fully featured in Spotlight. To do this, I need to implement an app shortcut.

I will start by extending the AppShortcutsProvider I started earlier. I'll add a new App Shortcut to the existing one. I'll also include an array of spoken phrases that will invoke my App Shortcut from Siri. Notice that rather than using my application's name directly in the string, I've used a special token. This allows Siri to not only recognize my app's main name, but any app name synonyms I've defined. I've also added the intent's parameter, "list," into some of the phrases. App Shortcuts supports extending trigger phrases with pre-defined parameters so that users can provide a parameter immediately upon invoking Siri. In this case, the system will call the suggestedEntities method on my ToDoListQuery to get the array of possible values. Note that I've also provided a phrase that does not have the "list" parameter. This is a good idea in general so that if my App Shortcut is used without a list, Siri or Shortcuts can prompt the user to select one. App Shortcuts automatically prompts for a value prior to performing the intent if the parameter is not optional. For optional parameters, you can manually prompt for a value inside your perform method. As before, we need to include a short title and system image.

Picking a good short title and system image are important, since they're the first things my users will see when they search for my app in Spotlight.

App Shortcuts supports parameters that have a fixed set of well-known parameter values that you can specify to Siri ahead of time. They do not support open-ended values like "Search my app for X," where X can be any input from the user. You can use parameters that are App Enums, in which case, the values are known ahead of time, or you could use parameters that are App Entities to be fully dynamic. All you need to do is return a list of entities in your query via suggestedEntities, like in my app.

And as a final step, I need to call updateAppShortcutParameters on my AppShortcutsProvider object. Calling this method signals to the system that your App Shortcut parameters have changed and will cause the system to call suggestedEntities on any relevant queries to re-fetch them. I need to do this each time my to-do lists have changed. That includes additions and deletions, but also mutations such as renaming. For example, if any part of my entities' displayRepresentation has changed, I'll need to call this method.

It's important to remember to also call this upon your app's first launch. App Shortcut phrases referencing entity parameters won't work until the system has successfully fetched entities for the first time.

And that's everything I need to implement for my Summarize List App Shortcut. Now when using my app, I can say, "Hey, Siri. Summarize my groceries list with Demo." And immediately get a verbal and visual response. With just a small amount of code, I've now made it easier for people to get the information they need in a quick, seamless way.

When you've implemented a great full-fledged App Shortcut, it's important to help your users discover it. There are two powerful ways people can discover your App Shortcuts throughout the course of simply using their device: Spotlight and Siri Tips. Spotlight is one of the most popular ways people launch apps on iPhone and iPad. The Spotlight Top Hits section shows not only apps, but also their App Shortcuts. This is a great way to make your App Shortcuts discoverable. People can find your App Shortcuts right away when they search for your app, and your app is the top hit. Or they can find them by searching for the title of the App Shortcut itself, like "Summarize List" here.

In addition to Spotlight, the Siri Tip is a great way to let your users discover your App Shortcuts while they are still using your app. Here I've added one in the list view, since that feels like the place where people would find this tip useful. The Tip View is available in both SwiftUI and UIKit, and we've provided a number of styles, so they look great in any application. Siri Tips are best placed contextually so that they are relevant to the content onscreen. Between Spotlight and Siri Tips, there are ways for your users to find out about App Shortcuts both in and outside your app.

If you want to dig more deeply into both App Intents and App Shortcuts, check out two sessions from WWDC22. "Dive into App Intents" is great for an overview of the App Intents framework, and "Implement App Shortcuts with App Intents" explores a greater variety of user experiences available throughout App Shortcuts, such as disambiguating between multiple possible parameter values.

Now that I've implemented my first App Shortcut, I'd like to tell you about what's new this year to help you make the most out of App Shortcuts and all of the great new features of iOS 17.

In iOS 17, we've updated how App Intents and App Shortcuts are shown in the Shortcuts app. This new design lets people find your app's intents more easily, and gives one-tap access to the app's most important actions. App Shortcuts also now get prominent placement in Spotlight. Not only are frequently-used App Shortcuts shown here, but we've also added new APIs this year to make your App Shortcuts more visually distinctive, and also introduced the ability for Spotlight to highlight entities of interest.

The Shortcuts Automation setup has been redesigned to be easier to use, and your App Shortcuts are featured, so people can get started with them more simply.

To take advantage of all of these great new features and bring attention to your App Shortcuts, we've introduced new APIs around colors, entity thumbnails, and short titles with symbols. These new APIs are required for all App Shortcuts and will help your app really stand out.

I'll start with colors. You can now set up to two colors in your app's Info plist that the system will use when your app appears in Spotlight or Shortcuts. Consider adopting colors similar to your app's in-app style, so you can bring a little bit of your in-app experience into the rest of the system. Next, each entity instance in your app can now have an optional thumbnail image. This is an extension of the existing DisplayRepresentation API. Your thumbnail can be provided either as a URL where the image resides, a Data object representing the image, a named bundled image resource, or a system image name. Choosing a unique and distinctive image for your entities will help make your Spotlight appearance more engaging. And finally, every app shortcut now requires a short title and system image. When shown in Spotlight or Shortcuts, the short title and system image are used to style the action. Pick something that's distinctive and represents what your app shortcut does.

In addition to all of the great new ways App Shortcuts are visually featured in the system, we've also made big improvements this year to how App Shortcuts can be used with Siri. In iOS 16, Siri will only recognize your App Shortcut trigger phrases if they are spoken exactly as you've defined them in your source code and strings files. But people using your app may use different words and phrasings when trying to access your App Shortcut. For example, in my app, I provided, "Summarize my groceries list with Demo," as a phrase, so it triggers my App Shortcut as expected when spoken, but something quite similar and reasonable like, "Tell me the summary of my groceries list with Demo," would not work. In iOS 17, we're giving you the ability to speak more naturally to trigger your App Shortcuts. This takes advantage of on-device machine learning to allow phrases similar to the ones you've provided in your App Shortcut to also just work. This is powered by a new Semantic Similarity Index.

With this new flexible matching feature, people can use more natural ways of invoking app shortcuts, and you don't have to provide every possible different phrasing. And most importantly, there are no code changes required to get flexible matching. Simply rebuild your app with Xcode 15, and phrases similar to the ones you already have will just work. If you're not yet ready to adopt flexible matching, you can opt-out by disabling the "Enable App Shortcuts Flexible Matching" build setting, and revert to having exact-match only on your app shortcuts phrases.

To help further broaden the naturalness of App Shortcuts phrases, there is also a new synonyms API. This is a small addition to the DisplayRepresentation API so that you can define additional synonyms for AppEntities and AppEnum cases. This will further broaden the reach of your App Shortcuts by making it possible to speak more naturally to Siri.

For example, in my app, I can now refer to my lists by alternate names right in the trigger phrase.

The synonyms also work if Siri prompts me for a list, if I didn't specify one upfront. These synonyms are associated with each instance of an entity, and if they change, you should call updateAppShortcutParameters again.

In order to support Flexible Matching, there is also a new Negative Phrases API. You only need to use this if you find that phrases that do not belong to your App Shortcuts are nonetheless incorrectly invoking your app. In this case, "Send a Demo summary to my grocery store," might incorrectly trigger my App Shortcut, so I will include this in my negative phrases.

And finally, testing App Shortcuts with Siri can be slow-going, requiring you to build and launch your app and speak to Siri each time. This is especially time-consuming if you want to test in multiple locales. To help make authoring App Shortcuts faster and easier, we're introducing a powerful new tool in Xcode called App Shortcuts Preview. You should note that App Shortcuts Preview is only available when running Xcode 15 on macOS Sonoma. Let's take a look at a demo.

Here I have the project for my Demo app, where I've previously implemented App Shortcuts. I can test my App Shortcuts by launching my app and speaking to Siri, but I want to show you a faster way of quickly previewing the effects of phrases. This new feature can be found in Product, App Shortcuts Preview. It requires building my app first so that the Semantic Similarity Index has an opportunity to generate. After building, I can simply select my app on the left and immediately start entering phrases that I think people may speak to Siri.

Let's try a couple of phrases that I know will work, since it's the one I defined in my code. Summarize the shopping list with Demo. That correctly matches with my SummarizeList intent.

Let's try a phrase that's similar to the first, but not one I provided. Give me the summary of the shopping list with Demo.

Thanks to Flexible Matching, this phrase also matches with the same intent without me having to do any additional work.

Now let's try with a third phrase that might be a little bit too different to be recognized reliably. Give me the gist of the shopping list with Demo.

This has no match with any App Shortcuts in my app. I can very quickly add a new phrase to my App Shortcut, rebuild, and test the phrase again.

Give me the gist of the shopping list with Demo. And the new phrase matches, and I am able to verify that without running my app.

Now I want to test my app in Mandarin. Rather than switch my phone's language, run my app, and ask Siri, I can simply use the locale selector in the Preview. This lets me immediately test a new phrase. And there, the Mandarin phrase works as expected, all without me needing to use a development device.

App Shortcuts Preview is a faster alternative to manually testing how your App Shortcuts work with Siri, and it's gonna make developing your App Shortcuts easier.

In iOS 17, we're also improving how you can localize your App Shortcuts trigger phrases, along with the new String Catalog file format. Previously, you were limited in how many phrases you could have for each App Shortcut in each locale. If you define five phrases for an App Shortcut in your Swift code, for example, you would be limited to five phrases in each locale. In iOS 17, with the help of String Catalogs, we're lifting this limitation so that you can have as many phrases as is naturally needed for each and every locale. You should note that this feature is available only to apps targeting iOS 17 later. Taking advantage of this new feature is simple. If you have not adopted App Shortcuts in your app before this year, simply create a new String Catalog called "AppShortcuts." After rebuilding your app, you will see the phrases you defined in your AppShortcutsProvider populated automatically.

As you edit your Swift source code and rebuild, the new String Catalog format will keep up with any phrases you add or remove automatically.

Alternatively, if you've already adopted App Shortcuts prior to this year, simply migrate your AppShortcuts.strings file to String Catalog.

Right click on it in Xcode and follow the migration assistant. Upon completion and rebuilding, you will see your phrases populate automatically, preserving any localization you may have done already.

Once migrated to String Catalog, you can add additional phrases for each locale without limitation.

Flexible matching, the new App Shortcuts Preview, and String Catalogs will make App Shortcuts far better with Siri. Here are a few tips on making your trigger phrases great: It’s important to choose memorable phrases. Your app's users will have lots of other apps on their phones that support App Shortcuts, and in practice, they'll have a hard time remembering exactly how to phrase their shortcuts. Take advantage of app name synonyms. This will help further broaden how your users can invoke your App Shortcuts. Avoid using non-human-readable placeholder keys in your Swift source code. If you refer to any parameters in your phrases, they must be defined by the interpolated string placeholders I demonstrated earlier. This is necessary for the runtime to load and fill these placeholders correctly at runtime.

In addition, there are some limitations to App Shortcuts you should be aware of as you are adopting them in your app. Your app can have at most ten App Shortcuts. You may have at most 1,000 trigger phrases for your entire app. Note that this includes all combinations of parameters. So for example, a parameter with ten possible values would expand the number of effective phrases by a factor of ten. You can avoid hitting this limit by making sure any parameters in phrases don't have too many possible values. Testing with App Shortcuts Preview will also help ensure you don't define redundant phrases that might be covered by Flexible Matching already. All trigger phrases must contain your app name or an app name synonym. For a guide on how to define additional app name synonyms, refer to the link associated with this video.

App Shortcuts that open the app are now eligible to be shown in Spotlight. If you have an app intent that opens your app with an entity, you should conform that app intent to the OpenIntent protocol.

Now that I've gone over how to create a great App Shortcuts experience, let's talk about how you can bring this same great experience across a wider variety of devices beyond iPhone.

App Shortcuts now work on Apple Watch, but you should be aware of some limitations to get the best possible experience. App Shortcuts on Apple Watch must come from a watchOS app installed on the device itself. App Shortcuts from a paired iOS device cannot be run on the Watch. This means that if you have an iOS app installed on iPhone, but not the corresponding watchOS app, your App Shortcuts will not work from watchOS.

Flexible matching with Siri is not available on Apple Watch, so phrases must be spoken exactly. Apple Watch support was introduced in watchOS 9.2 and is available on all devices running that version or later.

App Shortcuts are also now discoverable on Apple Watch. Just like on iOS, the Shortcuts app on watchOS prominently features App Shortcuts available on the device.

Secondly, App Shortcuts are now supported on HomePod. This requires an App Shortcut enabled app to be installed on a companion iOS or iPadOS device. Unlike running on iOS or iPadOS, where App Shortcuts can result in launching the app, on HomePod, App Shortcuts may not launch the app at all. For example, by simply having my Demo app on my iPhone, I can say to my HomePod, "Hey, Siri. Summarize my groceries list with Demo." HomePod support for App Shortcuts is available from version 16.2 and newer.

When designing App Shortcuts for HomePod, consider that it is a voice-only device. Make sure you are returning clear and concise dialog to be spoken by Siri, since there won't be any corresponding visuals. The IntentDialog class has an initializer that takes both a "full" and "supporting" string. The HomePod will use the "full" dialog, while visual devices like iPhone will use the alternate "supporting" dialog. This allows you to be more brief when a user can reference a visual output, but remain fully descriptive if no screen is available.

For more guidance on how to design App Shortcuts, including voice-only use cases, check out the "Design App Shortcuts" session from WWDC22.

I want to wrap up by reiterating some key takeaways. App Shortcuts help raise the visibility of your app's features throughout the system, and lowers the friction to habitual usage for your app.

App Shortcuts are more discoverable than ever, with prominent placement right in Spotlight helping drive more usage of your app and more styling options to help your app stand out.

App Shortcuts with Siri are hugely improved with Flexible Matching, allowing users to trigger App Shortcuts with their voice more easily. And developing and testing App Shortcuts is faster and easier with the new App Shortcuts Preview tool in Xcode.

For deeper dives into App Intents and App Shortcuts, there are two more sessions from this year you should check out.

With all of these improvements, I think App Shortcuts will be a great fit for your app and your users. I'm looking forward to seeing all the great App Shortcuts you'll create. ♪ ♪