Challenge: Voice Control Synonyms
June 10, 2021
Challenge yourself to make your app accessible through Voice Control and provide support for voice-based interaction. Voice Control is a feature built into iOS, iPadOS, and macOS, and empowers those who can’t use traditional input devices to control their Mac, iPhone, and iPad entirely with their voices. For people with motor limitations, having full voice control of their devices is truly transformative. People can gesture with their voices to click, swipe, and tap anywhere — they can do everything someone could do with a mouse or with touch. On iOS and iPadOS, Voice Control has the additional option to show Item Names, which place a name next to each tappable item. In this challenge, we’ll be making the “Show Names” experience better.
Suppose that you create a button that looks like a paper airplane. What do you say to tap? “Tap send”? “Tap reply”? “Tap airplane”? In UIKit you can use the accessibilityUserInputLabels string array to respond to these prompts, while in SwiftUI you’d use the .accessibilityInputLabels modifier.
How to enable Voice Control
To use Voice Control, go to Settings > Accessibility > Voice Control. If it’s your first time enabling this setting, you'll be asked to Set Up Voice Control and download a short file.
Once Voice Control has been set up, you can enable it in a few different ways:
- You can ask Siri to turn Voice Control on or off for you at anytime.
- You can use the Accessibility Shortcut in Settings > Accessibility and set the shortcut to Voice Control. Now, when you triple click the side button (or Home button, depending on your device), you can quickly turn Voice Control on or off.
Use Voice Control to interact with iPhone
Begin the challenge
We’re challenging you to make your app’s UI as easy to navigate by voice as possible and improve the Voice Control experience in your app. Start by turning on Voice Control by visiting Settings > Accessibility > Voice Control, and enable Overlay > Show Names.
Next, take a screenshot of your interface with the “Show names” overlay displaying on top of it. Explore what it’s like to navigate your app by Voice Control alone. What experience are you giving people right now? Are you struggling with any common tasks? How could you make it better?
Once you’ve spent some time with your app in Voice Control mode, it’s time to make some improvements. Here are a few tweaks you can make to your code to make your experience better for everyone.
Explore accessibilityInputLabels
First, you can implement accessibilityInputLabels to create short, concise labels that someone could easily speak by voice.
Button(action: {
sendMessage = true
}) {
Image(systemName: "paperplane")
.font(.title)
.accessibilityInputLabels(["send", "reply", "airplane"])
}Tips:
- Your primary string is the first string in the array, and will be the one that Voice Control shows on screen.
- Brevity is key: use short, succinct words.
- Localize your strings using NSLocalizedString and avoid special symbols in your labels.
- When it comes to number of synonyms, add them judiciously. Limit the number of possible strings to a max of 4, as to not overload the recognition system.
You may have multiple elements in your UI that could be described the same way: One example is an image browser, where each image might be described as “Screenshot”. You can rely on Voice Control’s disambiguation feature in these cases to keep your label names short. When someone says “Screenshot”, a list of numbers will appear over all elements named “Screenshot” for someone to choose from.
Shorten label names
If your app already incorporates accessibilityLabel, you’ve done a lot of the work already — but your labels may be too long to speak! You can take advantage of accessibilityUserInputLabels (or, in SwiftUI, .accessibilityInputLabels) to keep the speakable label short, while leaving the valuable information your current accessibilityLabel conveys to an audience that relies on it.
Share your experiences As you add support for Voice Control to your app, share your implementation with the developer community. After you’ve made changes or improvements to your app, take another screenshot of your UI with the “Show names” overlay enabled. Share “before” and “after” screenshots on the Developer Forums. (And don’t forget to add alt text to your screenshot images on platforms that support it!)