Making Apps More Accessible With Custom Actions

Hi, everyone. My name is Dan. I'm an engineer on the Accessibility Team, and I'm super excited to be talking to you about Accessibility Custom Actions and how they can improve the experience for people using assistive technologies with your apps. Let's get started.

Custom actions are really great for two things. The first is reducing clutter in your app.

This is important for everyone that uses your app, but it's especially important for people that use assistive technologies, and I'll explain why. The second is all about improving the convenience and speed of more complex interactions in your app.

Let's start by talking about reducing clutter and to do this I'd like to talk about VoiceOver.

VoiceOver is the built-in screen reader we ship across many of our products for people who are blind and low vision.

People that use VoiceOver can navigate through their apps in a variety of ways but one way that's particularly popular when navigating through a new app or interface is by swiping left and right to move between the next and previous item. I have an app I've been working on to help me track my favorite types of peanut butter. If you were going to use this app without using VoiceOver and we wanted to increase the rating of the second jar of peanut butter in this list, how do we go about doing that? Well, you'd probably visually scan the list until you found the jar you were looking for.

You then might focus on that cell and look for the button that affords the action you want to perform and then tap on it. Now that may seem pretty intuitive but that's because the UI affords that intuition by logically grouping the buttons with the cells, which makes the content easy for us to process and digest, but what if instead the app was laid out a bit differently? Perhaps something like this. Notice the buttons are now underneath the cells, which means we have a lot more content to scroll through.

Additionally, it's much more ambiguous as to which buttons correspond to which cells. This could very well be the experience for someone using VoiceOver with your app. Let me show you what I mean.

Let's take a look at what it's like to use VoiceOver to navigate through one of the cells of this app. Patty's, creamy, Noster rated 1 out of 5. Add to favorites button. Decrease rating button. Increase rating button.

After we swipe past the cell we focused on three buttons, add to favorites, decrease rating and increase rating and those three buttons exist in every single cell in this list.

So if we were trying to find the 10th jar, we would have listened to those buttons 30 times, which can very quickly get repetitive.

Additionally, we have to keep in our head what the last jar of peanut butter we heard was, otherwise we don't know what content is going to be affected when we activate one of those controls.

So let's take a look at the app only this time designed a bit differently. I've changed the buttons so that they're no longer accessible to VoiceOver. I've added custom actions to the cells so that when VoiceOver lands on a cell we can swipe up or down with one finger to choose an action and then double tap to perform it. Let's take a look.

Patty's, creamy, Noster rated 1 out of 5. Anna's, chunky, sugar-free, rated 4 out of 5. Actions available toggle favorite, increase rating.

This time when we swipe past the cell we just got to the next one which made it much easier for us to go through this content and find what we are looking for. Once we found it, we could then drill in and pick an interaction from there, which much more closely matches the experience we talked about earlier of visually scanning a list.

So custom actions can really help us reduce clutter, we remove the repetitive controls, and we also reduced the cognitive load on ourselves while we were using the app. Because we're explicitly picking an interaction after finding the content we want to interact with it's no longer ambiguous what those controls do.

All of this simplified the experience.

Next let's talk about how custom actions can improve the convenience and speed of more complex interactions in your app and to do this I'd like to talk about Switch Control. This is another feature we ship for people with limited mobility and these people can interact with their devices using as few as one or two inputs. I've added some additional functionality into the peanut butter app so that when we long press on a cell an action sheet comes up with additional options including the option to send information about one of the jars of peanut butter to a friend. I have two switches paired with the device.

The orange switch will move Switch Control's focus to the next item and the white switch will select that item.

Keep in mind switches can be designed for a variety of different needs. So, for example, someone might have switches mounted to a wheelchair that they can activate with their head.

Let's take a look at what it's like to Switch Control to send information about that second jar of peanut butter to our friend.

So we can use our orange switch to navigate down to the second cell and then use the white switch to select it. This brings up the Switch Control Menu which provides additional options of ways we can interact with the cell.

We can navigate down to the page control and select it to view more options. From here, we can press select to drill in and then navigate over to gestures.

Selecting this will give us a list of all of the gestures we might have wanted to perform by touching the display.

From here we can navigate down to the second row and press select to drill in and then navigate over to tap and hold.

Selecting this will perform our long press gesture and bring up our action sheet, which we can then navigate through to select send to a friend.

We'll perform the tap and we're done.

Now if you think about it, we did all of that and we never touched the display.

That's pretty cool, right? But you also probably noticed that that interaction normally takes us two taps, one long press and one tap on the action sheet and with Switch Control that took us 18 taps, but custom actions can significantly reduce that number.

So let's take a look again only this time I've added custom actions back into the app.

We'll still use the orange switch to navigate down to the second cell only this time when we select it right away on the first page of the Switch Control Menu are all of the actions of we've implemented. We can drill in, navigate over to send to a friend, press select and we're done.

We didn't even need to navigate through the action sheet that came up the first time. We're right where we need to be. That was a significantly faster interaction. We were able to take those 18 taps and we get all the way down to 6 taps, which is 67% fewer taps than before. So custom actions can provide huge wins for convenience and speed.

We removed the cumbersome gestures which significantly sped up the interaction. Additionally, we improved the prominence and the convenience of those interaction options by placing them on the first page of the Switch Control Menu.

We didn't need to navigate through several submenus and try out different gestures to figure what might be possible. Instead, everything was much more straightforward and direct. Now I bet after hearing all these awesome advantages of custom actions you just can't wait to use them in your apps and luckily it's pretty easy.

You simply override the Accessibility Custom Actions property on your view and return an array of UI accessibility, custom action objects.

Each one takes in a name, a target, and a selector.

Your function simply needs to return a Boolean as to whether or not the action was successful and take in an UI accessibility custom action object as a parameter. New this year we have a version of the API that takes in an action handler instead of a target and selector. So if using a block of code better fits with the design of your app, we encourage you to give this a try.

Yes.

Additionally, if you're going to use a custom action to replace something like an accessory control inside of a cell, you'll often want to set is accessibility element to false on those controls so they're no longer navigable by the assistive technology.

And what we talked about our VoiceOver and Switch Control in our demos today, custom actions can be leveraged across many of the assistive technologies we ship including full keyboard access and voice control which are new this year. I really encourage you to turn some of these features on and play around with your apps to look for ways custom actions can improve the experience. Thank you very much. [ Applause ]