Platform and Version
iOS
Development Environment: Xcode 16.2.0, macOS 15.3
Run-time Configuration: iOS 18.3, 17.x
Description of Problem
We have started migrating some of the app’s core functionality over to App Intents.
Our first release of App Intent support focused on two settings a user can modify on their Bose products, Audio Modes and Immersive Audio, giving users the ability to modify these settings via Siri and shortcuts. The implementation uses two separate shortcuts for each setting type, with each shortcut supporting a single phrase for Siri each: “Change my Bose mode to <setting>” and “Change my Bose immersive audio to <setting>”. Each shortcut uses their own App Intent, and each App Intent has support for optionally providing both a product and a setting when performing the intent. Failing to provide a device, which happens when the intent is performed via Siri, simply auto selects a currently connected Bose product. Failing to provide a setting, like in cases where a user says “Change my Bose <setting type>” without providing a setting will simply have Siri confirm the setting the user wants to change before changing the setting. We are using AppEntity to identify a Bose product for both App Intents. Because the App Intent for the Audio Modes setting has a larger number of supported values (up to 15 maximum), we are also using AppEntity to identify these settings. We are using AppEnum to identify available settings for the Immersive Audio App Intent, as only 3 static values are supported.
Our original implementation of App Intent support had quite a few phrases supported for each shortcut. We had explicit support for direct synonyms of the verb “Change” in other phrases, supporting words like “Switch” and “Set”. We also had support for words that are like the word “Change”, but not directly related, like the word “Toggle” for instance. We also had support for phrases with or without the setting in each phrase. However, early on we had a lot of trouble with phrase detection with Siri. Siri had a hard time identifying what shortcut was being requested, as well as not being able to identify what settings the user was providing for the setting parameter of each App Intent. While researching potential fixes for this issue, we found a response to a thread in the Apple forums (https://developer.apple.com/forums/thread/759909) that seemed to indicate that Siri phrase recognition was very much an aggregate process. With the total number of phrases supported combined with the available settings for each phrase further compounding the total number of phrases Siri needs to learn to recognize for each shortcut. So, to hopefully improve Siri phrase detection, we added logic to limit the amount of Audio Mode settings supported based on what Audio Modes the user had setup on their Bose products. But, more importantly, we limited the number of explicit phrases supported for each shortcut to just a single phrase. In our testing, not only did this improve phrase recognition, but support for synonyms like “Set” or “Switch” seemed to implicitly still be recognized by Siri.
The issues we ran into with Siri phrase detection above has us a bit concerned about scaling App Intent support to other settings and features for our products in the future. Our app supports the ability to modify a large number of settings on their Bose products, with support constantly expanding to new products as they are released. Our roadmap for App Intent support was initially very ambitious, supporting much more than just the two settings mentioned above. But our initial experience with App Intents has us tapering our expectations a little bit as far as how much can be supported in total for App Intents.
One thing we also noticed is less than optimal display of default shortcuts in the Shortcuts app. The default shortcuts appeared like so, with shortcuts displayed based on available settings fro each shortcut:
However, we could not find a way to indicate to users that one particular section pertained specifically to the Audio Mode setting and the other to the Immersive Audio setting. The only information the user has to make this determination for themselves is the available settings (or shortcuts) for each. This may not be immediately clear to a new customer who might be using one of our products for the first time. This display of default shortcuts in the Shortcuts app has us wondering if our shortcuts implementation is what is intended as far as support for the Shortcuts app is concerned. We did survey default shortcuts displayed by other third-party applications and they mostly dealt with navigation with a single section containing default options clearly indicating where the user can navigate with a shortcut. We couldn’t find an example of an application supporting the ability to change different setting types, with each setting type having their own available values for each.
So, to summarize the questions we have concerning App Intent support:
- What can we do with our App Intents and Shortcuts implementation to guarantee optimal performance with Siri?
- What is an ideal number of phrases to support for each Shortcut.
- What limitations should we be placing as far as the total number of available settings for each Shortcut.
- Are there phrases that might work better than others for what we’re trying to achieve with App Intent support?
- i.e. Is “Change my Bose mode” or “Change my Bose immersive audio” a good phrase to use for this kind of functionality? Or should we be using different verbs or wording?
- Assuming optimal support of each Shortcut above. What is a reasonable expectation as far as how many different supported shortcuts we can scale to support at the same time.
- One issue we ran into early on was Siri confusing one shortcut with the other and triggering the wrong App Intent at times. While this was ultimately resolved, this outcome seems much more likely the greater the number of individual shortcuts supported.
- Are there any recommendations on how to display these App Intents to customers as far as default shortcuts in the Shortcuts app is concerned?
- Is what we currently display for default shortcuts in the Shortcuts app what was initially intended for third party support for App Intents?
- If what we are currently displaying is expected, would it be possible to support the ability to provide additional context to each section of default shortcuts displayed? We would like to indicate to the user that one set of shortcuts pertains to the Audio Modes settings, and the other to Immersive Audio. Something along the lines of a section header like some of the first-party apps use.
- Are there any recommendations or tips for supporting App Intents, particularly phrases for Siri, in other languages?