Swan's Quest, Chapter 1: Voices in the dark

Hello and welcome to WWDC.

Hello, WWDC. This is Swan's Quest.

I'm Rob, your host, as we take an inside look at an exciting Swift Playgrounds challenge our crack team has built for you. This was built to remind you that writing code should be fun. Swift Playgrounds is an environment. It's jam-packed with frameworks and APIs that make it simple to experiment with complicated technologies while writing less code. This allows you to focus on what you want to build instead of spending your time figuring out how to build it. In this episode...

we're gonna introduce you to Swan's Quest, which is divided into four chapters.

Each chapter has a specific programming challenge for you that will build on the prior chapters. This first challenge requires you to navigate through a dark and mysterious cave.

You're gonna be armed only with VoiceOver and our easy-to-use APIs for providing descriptive context to on-screen elements. We'll discuss how to turn on VoiceOver for your device, then give you a primer on the Accessibility APIs that we use to make our education content the most accessible programming environment in the world. Finally, we'll give you some clues on how to use these APIs to navigate the cave and meet our Swan.

Are you ready to get started? Swan's Quest is a four-part challenge, where you're gonna use your programming skill to guide our brave Turtle on their epic quest.

Our Turtle's journey sends them into a dark, dark cave in search of a royal Swan. The Swan can grant the Turtle safe passage through their kingdom, but only if the Turtle can win their favor first.

To win favor, the Turtle must complete a series of musical challenges from the Swan. That's where the Turtle needs your help...

and the help of a local Lizard, who helps decode the Swan's clues.

The Lizard gives you hints about which APIs to use along the way.

Swan's Quest consists of four programming exercises, which were built using the same frameworks that we used to deliver education content with Swift Playgrounds. We use this framework to teach beginning programmers how to write code in Swift. But did you know you can remix our content and use the same frameworks for your projects? The best way to get started with our SDK is our starting points, like the three that you see here. These are templated playground books that allow you to create new pages and new Swift modules without leaving Swift Playgrounds. For each of these templates, we include the same modules built to create their guided learning companions.

We make it easy to experiment with our audio APIs in Sonic Create, take advantage of different iPad sensors in Sensor Create, and create your own augmented reality experiences with AR Create, using custom art from Sarah Rothberg.

For Swan's Quest, we created a new, more powerful template that combines API from several of our starting points.

You can download this template, along with Swan's Quest, directly from the Developer app. And like all of our provided content, these books are open-source. That means that you can unpack them on your Mac, and you can see how we use the APIs to build each challenge.

During our four sessions, I'm gonna describe some of the APIs you'll need to complete each challenge. For today's challenge, you're going to use our Accessibility APIs and VoiceOver. And you're gonna need it. You'll need both of them to get out of this dark cave between you and our Swan. Lucky for you, my first guest, Jordyn, is an ace on our assistive technologies. She's here to give you a crash course on VoiceOver. Pay attention, because VoiceOver is the best tool to solve the riddle of our Cave.

Thanks, Rob. Hey, everybody. My name's Jordyn, and I'm an accessibility engineer here at Apple. I not only help build Apple's assistive technology, but I use it every day to keep in touch with my family and friends, order food for my adorable cats, learn to play piano and, of course, write code. Speaking of coding, let's dive right in. To complete the first challenge, you're going to use VoiceOver.

VoiceOver is Apple's screen reader, and it's built into all of our products. It provides information about text, controls, and other elements as you move through an app's interface using text-to-speech, braille, or a combination of the two.

You'll need VoiceOver to navigate through a dark, spooky cave and write code to help light your path. You heard me right. You're gonna write code using VoiceOver. But don't worry. It's not that hard once you get the hang of it. Take it from me. Before we begin, let's set up your Accessibility Shortcut. This will allow you to turn on or off Accessibility Settings quickly by pressing the Home or Sleep/Wake button three times. Go to Settings, Accessibility, and all the way at the bottom, you'll see Accessibility Shortcut. Let's select "VoiceOver" here. Next, let's turn on VoiceOver. If you're on an iPad, you can go into Settings, Accessibility, and VoiceOver. And you'll notice a Toggle switch at the top. To enable it, turn on VoiceOver here or use the Accessibility Shortcut we just set up. You can also ask Siri, "Turn on VoiceOver." First, you'll hear the prompt, "VoiceOver on." You may need to double-tap the OK button, confirming that the gestures will be different once VoiceOver is enabled.

The black rectangle around the selected control is called your VoiceOver cursor, and it confirms visually which item has VoiceOver's focus.

Flick left or right with one finger to move the cursor over different elements on the screen.

Accessibility features Help, Vision, Heading, VoiceOver on, zoom off, VoiceOver on. Button. Alternatively, you can drag your finger around the screen to move the cursor over elements more quickly.

Audio, VoiceOver-- Take a minute now to adjust VoiceOver's speaking rate to your liking by swiping up or down on the Speaking Rate slider with one finger.

Speaking Rate, 45%, 55, 65%, 75%. 85%, 95%, 100%, 95%, 80%, 70%. Many experienced VoiceOver users like to turn the speaking rate all the way up to take in information quickly and navigate efficiently. That's about 720 words per minute. That sounds a little something like this. Speaking rate-- If you're wondering, yes, I can understand VoiceOver that fast. To exit Settings, you'll need to either press the Home button on your iPad, or if your device doesn't have a Home button, slide one finger up from the bottom edge of the screen until you hear the first sound. If you pause there, VoiceOver will say, "Lift for Home." Lift for Home. Once you're on the Home Screen, navigate to the Swift Playgrounds icon and double-tap to open it.

Playgrounds. Double-- Playgrounds. If you're on the Mac, you can use Command plus F5 or triple-click the Touch ID button to activate VoiceOver.

Once activated, you can move the VoiceOver cursor through the screen by pressing Control plus Option plus either the left or right arrow keys.

Likewise, you can navigate up and down by using Control plus Option plus the Up or Down arrow keys. To enter a group of controls or elements, such as the source code editor in Playgrounds, press Control plus Option plus Shift plus the Down arrow to do what we like to call "interacting." And to stop interacting, to get out of a group of controls, press Control plus Option plus Shift plus the Up arrow key.

Use Control plus Option plus the space bar to activate a control. Now that you're familiar with VoiceOver, how to turn it on, and a little bit of the basic gestures and commands, I want to discuss a tragic experience that turns away many blind people from using the amazing apps that you develop.

At Apple, we pride ourselves on best-in-class interface designs for our products. And that same quality is given to accessibility. Accessibility is one of Apple's core values. So, when you turned on VoiceOver, you were greeted with a wealth of context as you moved around Settings and the Home Screen.

Sadly, not all developers pay this much attention to accessibility. And it shows.

When you first download an app, you hear it immediately.

Instead of rich detail, I'm greeted with a ton of unlabeled buttons.

So, when I swipe around the screen, all I hear is "button, button, button." And maybe some image file names thrown in there for good measure. And in this case, when I swipe around the screen, I hear... Seen. Twenty-- Torch two, graphic. Torch three, graphic. Torch six, graphic. I'm left wondering, "What do these torches look like?" "What am I supposed to do with these?" And in the case of the apps, I'm left guessing, "What do these buttons do?" "And how am I supposed to use this?" To be honest, I delete an app with unlabeled buttons within the first minute of opening it. But you have the power to change this experience. You can make a better app interface for everyone, starting by adding labels to your buttons and controls.

Seen. Torch three. Tor-- Torch. A 12-inch rock blazing with a ball of mystical blue flame. Ah. That makes a lot more sense. That sounds great. When people use Swift Playgrounds, they expect a great user experience. Our content must be designed for everyone to learn how to code. No exceptions.

That's why my fellow engineers built a simple-to-use API for building fully accessible Swift Playgrounds content.

To tell you more about these APIs, I'd like to introduce my colleague from the Swift Playgrounds team, Stephen. Thanks, Jordyn. Hello. I'm Stephen, an education engineer on the Swift Playgrounds team. We don't want anyone to have the experience Jordyn described with our content. At Apple, we believe that everyone can code, and that's why we've built a simple-to-use accessibility interface into our Swift Playgrounds content SDK. There are four onscreen elements that form the foundation of our content SDK: Graphic, Button, Sprite, and Label. All of them inherit from BaseGraphic, which contains properties for accessibility support.

Here, we see the optional accessibilityHints property.

Setting that property is how you indicate that this screen element should be read by VoiceOver and makes it visible to other assistive services.

To support VoiceOver, these are the values you need to supply. makeAccessibilityElement tells VoiceOver it should stop the cursor on this item. accessibilityLabel is what VoiceOver reads as it stops on the item.

Let's walk through an example. First, you create an instance of accessibilityHints. We want VoiceOver to stop on this graphic and read, "Activate button to start the party." We then create our graphic and set its accessibilityHints to the hints variable we created in the first step. I hope this example makes sense, because you're gonna need it to complete our first challenge. Thanks, Stephen. As you probably guessed, our first challenge, "Voices in the Dark," is all about making onscreen controls more accessible. When you enter our cave, none of the torches are lit. In fact, you won't be able to see anything. And your character, they're not gonna move until you light enough torches.

To light the torches, you need to set an accessibility label on them.

With VoiceOver, you'll be able to navigate to each of them. And when you land on one, you'll hear a property name like "Torch one" or "Torch three." Once you've added an accessibility label to each torch, your character can pass through the cave and meet our Swan.

In the Playgrounds source editor, you need to replace that unhelpful label with something more descriptive. Use your ears and change the label.

Like you see here, once you do, you'll be off to meet the Swan and get a clue for our next challenge. During this episode, we introduced you to Swan's Quest, an exciting four-part challenge for WWDC attendees.

We introduced you to VoiceOver and how to use Swift Playgrounds to write accessible software. And finally, we gave you some tips on how to navigate the cave on your way to meet our Swan. We'll be back tomorrow to talk about Chapter 2. Until then, good luck, have fun, and drop into the forums to let us know what you thought of our first challenge.