iOS 10 introduces SiriKit, a new framework enabling apps to integrate with Siri. Simply by speaking, your apps can send messages, place calls, send payments, and more. Learn how SiriKit helps you create new experiences on iOS and explore design principles for creating a great Siri experience.
My name is Robby Walker, tonight I will be joined
by Brandon Newendorp and Corey Peterson to talk about SiriKit.
First I'm going to start by talking a little bit
about what SiriKit is, and at a high level, how it works.
Then Brandon's going to talk about building an extension
or two to connect your application to SiriKit.
And then Corey's going to talk about how to build
and design a great conversational experience,
which is meaningfully different
than designing a great graphical user interface.
But first, let's talk about Siri.
Who is Siri?
Siri is your virtual assistant, always ready to help you,
whether it's by adding a bit of humor,
or much more frequently, by helping you get things done,
and alerting you to potential conflicts proactively.
We're going to talk about an example app that integrates
with Siri, and when we built this app,
we built a fake application.
So we wanted to name it after something fake,
but we kind of felt like it was really valuable so we wanted
to name it after a very valuable sounding fake thing.
So we called it Unicorn Chat.
And then because we wanted it to seem even more valuable,
we gave it a really sweet icon.
So you can imagine someone saying something like, "Hey,
Siri, send a Unicorn Chat message."
Siri might respond "To whom?"
Then you might say, "Celestra", which is one
of the Unicorn names of one of my colleagues.
Then you would -- then Siri would ask, "What do you want
to say to Celestra, and then I might say something like,
"Let's add more sparkle transitions,"
because as everyone knows,
Unicorns love sparkle transitions.
But there's a lot of ways to say the same thing to Siri.
Someone else might say it in just two steps,
saying something like, "Send a message
to Celestra in Unicorn Chat."
And then of course, Siri would ask a question
and then the user would fill in the rest of their intent.
Other users might do the exact same thing in only one line,
"Unicorn Chat Celestra, let's add more sparkle transitions."
Somebody else might say an equivalent concept in Chinese,
which I will not attempt to say.
And other users will be incredibly polite and verbose,
and might say something like, "Hey, Siri,
can you please do me a favor and launch the Unicorn Chat app
to send a text with the message let's add more
And despite using 27 words to describe a simple concept,
they actually forgot to say who they were going
to send the message to.
And then even then, they would be incredibly verbose
and say something like, "My very best friend Celestra."
So the point is, there are a lot of ways
to say the same thing to Siri.
And there are also a lot of ways that Siri will respond.
So for example, if you're holding your phone,
and you're pressing the Home button, Siri's going
to provide more visual responses and say less things out loud.
But if you're invoking using Hey Siri,
like our friend Cookie Monster, or if you're in the car,
Siri's going to say more and show you less.
And the Siri experience is also carefully optimized
to meet the accessibility needs of our users and responds
to a lot of different accessibility settings.
So in the end of the day,
there's basically a combinatorial explosion of ways
that people use Siri, both in terms
of how they phrase something to Siri and in terms
of how Siri will respond.
And the question then becomes,
how do we make sure we're building a high quality
experience for our users?
How do we make sure it's really consistent across applications
within a single domain?
And most importantly, for everyone in this room,
how do we make sure it's nice and easy to adopt,
so that as many people as possible can use as many
of their favorite applications in a conversational way?
And the answer is that Siri handles the conversation
and your application just handles the functionality
So I'll go into a little more detail about what that means.
The first thing that Siri does,
is understand what the user said.
So we take audio and we turn it into text.
Then we take that text and we turn it
into a structured representation of what the user is trying
to do, something we call an intent.
Based on that intent, Siri then takes action performing the task
the user wanted to accomplish.
And then Siri provides responses as I said,
both visual and verbal.
And so there's a lot here.
There's a lot of work that goes into these four steps.
There's a lot of deep learning technologies.
There's convolutional neural nets.
There's LSTMs, there's CRFs,
there's SVMs, there's a lot here.
And we wanted to make sure that as app developers,
you could really focus on the things
that your app already does, and there's three parts to that.
The first one is vocabulary.
This is how your application tells Siri words Siri needs
to know about.
And Brandon will tell you more about it in a bit.
The second one is app logic.
This is the functionality of your application.
And then the last one is of course, a great user interface.
So let's go through this example again,
and see a little bit about how it works.
So somebody says, "Hey Siri,
please send a Unicorn Chat message to Celestra."
The first rule that your application plays is
to help Siri understand the speech.
So Siri doesn't have the name Celestra
in its database, but your app does.
So by sharing that information with Siri, you make sure
that Siri understands what the user said.
This also helps Siri identify the fact
that Celestra is a contact
because when you provide this kind vocabulary,
you will provide it by saying these are names of contacts,
or in another example it might be,
these are names of photo albums.
Now Siri understands that in the messaging domain,
to send a message you actually need more than,
of course, just a recipient.
You need a message.
And so Siri will quickly check with your extension to ask,
"Do you actually require content for this message?"
And if your application says yes,
then Siri will ask a follow up question.
So really that's all your application has to do is sort
of specify on an individual parameter basis what are
The next thing that happens is Siri structures all
that information up into an object.
So rather than having to deal with all of the vagaries
of language and phrasing, and all those things that we talked
about earlier, your application really just gets this nice
And then it hands off to your application to help with,
you know, processing the intent and performing the action.
And then of course, along the way,
Siri provides a user interface for the user.
And you have the option to provide a second extension
to customize the user interface to make it look extra Unicorny.
So that's a very, very high level of how SiriKit works.
The idea was to make it incredibly simple to adopt,
but still connect you to the power
of conversational interactions.
And now to tell you a little bit more about how that all works,
is my colleague Brandon Newendorp,
also known to me, as Celestra.
Thanks Robby, I am incredibly excited
to tell you all a little bit more about how SiriKit works,
and how you're going to be speaking Siri's language
when you adopt SiriKit in your own applications.
Now pay close attention because this is some seriously
To get started, I want to talk
about how SiriKit communicates about the world.
At a high level, Siri thinks about everything
in the idea of a domain.
Where a domain is a broad category of things
that Siri understands and knows how to talk about.
For example, things like messaging, placing VOIP calls,
and sending payments, are great examples of domains,
and are some of the domains that we are opening
up with SiriKit this year.
Within a domain, we have an intent.
Where an intent describes a known action
that Siri understands, and can communicate to your application.
Now I'd like to break down an intent a little bit more
with another example, in a little bit more detail
than what Robby was just talking about.
So in this case, we have the Unicorn Pay app.
Hey Siri, send Buttercup 85 bucks
on Unicorn Pay for rainbow polish.
Now Siri understands that this is a request
in the payments domain.
And specifically, we've recognized
that this is a send payment intent.
Now every intent has the idea of parameters.
Where parameters are specific items that Siri can recognize,
understand, and communicate to your application.
And there's a lot of parameters on the send message intent.
We've recognized that the app is Unicorn Pay,
and that the user is trying to send the money
to someone named Buttercup.
We've also recognized an amount of 85, and then Siri is going
to be very smart, and recognize that when the user said "bucks,"
they probably meant US Dollars.
And Siri can use a lot of powerful capabilities
to understand what we think the user means in a case like this,
basing it on things like the user's location,
their Siri locale, and other information available
to our system.
And finally, we recognize that the user is asking
to send a payment for rainbow polish.
So there's this common language that we've created for Sirikit,
of intents that describe actions and intent responses
that are used to describe results coming
from your application back to Siri.
So let's talk a little bit more about an intent.
As I said, an intent is an action
that we're asking your app to perform, and something
that we can communicate
about between Apple and your application.
Every intent will have zero to many parameters, and it's going
to vary based on the individual intent and domain,
and what Siri understands within that domain,
and what we think is relevant to communicate to your application.
And no matter what, an intent will always be classified
into a domain.
So this is something you want to think
about when you're getting started with adopting SiriKit
in your application, is look at the domains that are a part
of SiriKit, and think about which ones are going
to make the most sense for your application.
Now we also have intent responses.
And these are how you tell Siri about the results of the intent
that your app just performed.
You're always going to use one of these
to communicate the result
because there's always an intent response paired with its intent.
Intent responses also can have parameters attached to them,
which are used to describe what your app actually did
with that intent.
Sometimes those parameters are going to be the same as the one
on the intent, and other times they'll be different
because there's something that's only communicated
from your app back to Siri.
We also accept a response code attached to an intent response.
These are used to tell Siri at a broad level
about whether your app was able to complete the request or not,
and there's default ones, such as success, or failure,
or failure that requires an app launch.
Some intents and domains have more specific error codes.
It's very important that you tell Siri an accurate response
code back for your application because we're going to use this
to drive dialogue and make a better user experience
for all of your users.
Finally, we have an NSUserActivity attached
to an intent response.
Now if you're already supporting hand off in your application,
you're familiar with NSUserActivity.
We use NSUserActivities to launch your application
if the user needs to leave the Siri experience and move
onto your application.
Now by default, Siri is going to generate our own user activity
with the intent and intent response to give
to your application, so that you can pick up exactly
where you left off from Siri.
If you'd like to, you can provide your own user activity,
and populate the user info dictionary
with information that's unique to your application.
Just to make sure you create the best possible experience
when users leave Siri and move on to your application.
So those are intents, and intent responses.
They're the fundamental language that Siri uses to talk
to your application
and understand what your application was able to perform
on behalf of the user.
The next thing I'd like to talk about, is how your app
and Siri are going to communicate about intents.
Now there's three steps that we're going to ask you
to go through, while communicating about intents.
The first of those is resolve.
We're going to ask your app
to resolve every parameter on an intent.
And in fact, we will call resolve multiple times
for every parameter and sometimes multiple times
for the same parameter until you and the user agree
on what they requested.
The second step is confirm, which is a chance for your app
to make sure that it's going to be able to handle the intent,
and tell us about the expected outcome.
And finally, most importantly, is the handle step,
where you perform the specific action the user asked you to do.
Let's talk a little bit more about the resolve stages first.
The purpose of the resolve stage,
is for you to help Siri understand what the user said,
and more importantly, what the user meant
when they started asking a request of your application.
This is also your app's big opportunity
to influence Siri's behavior and have some control
over the conversation that we're having
with the user of your application.
In order to do that, your app is going
to provide a resolution response.
And there's a number of different resolution responses
for every type of data that Siri understands, and is going
to ask your app to help us understand.
So let's go back to our example
with sending Buttercup some money on Unicorn Pay
for that rainbow polish.
And specifically, I'd like to go pay attention
to the payee field.
This is used to describe who we are sending money to.
And it's important that Siri, the user, and your app,
all agree on what the user meant when they said Buttercup.
Now hopefully, the most common resolution response
from your app will be success.
Which means that you understood exactly what the user said,
what they specified, and then you're going to tell us
that you successfully understood that and provide the object back
to us to tell us what you resolved to,
what you successfully matched.
In this case, it means
that Unicorn Pay totally understood who Buttercup was.
Now sometimes, your app is going to make some smart choices,
but it wants to make sure
that the user really meant what you think you'd made a smart
So we have a confirmation required option.
This is where you can ask Siri
to have the user double check your resolution result,
and provide a yes or a no response.
So in this case, for Unicorn Pay, Siri's just making sure
that the user meant Sir Buttercup
when they said Buttercup.
Now there's other times that your app is going to need
to ask the user to pick from a short list of options.
And Siri calls this disambiguation.
If you'd like to ask the user to pick from a list of options,
we recommend five to ten at most.
You can provide a disambiguation list to Siri.
And then we'll present the options you told us about.
In the case of this Unicorn Pay example, we're asking the user
if they meant Sir Buttercup,
Buttercup the Bold, or Sunrise Buttercup?
Now one of the powerful parts of Siri, is that we're going
to take care of the interaction with the user on behalf of you.
You don't have to worry about this part.
So users can use a natural way to select one
of these options with Siri.
They could tap on an item in the list, they could speak the name
and say "Sir Buttercup", or they could just refer to one
and simply say, "the second one."
And no matter how the user wants to interact with Siri,
we're going to take care of that for your application for free.
Now there's other times that you need more information
from the user.
Maybe they weren't specific enough,
or maybe your application has 80 people named Buttercup,
and you don't want to present 80 options
of Buttercup to the user.
In these cases, you can say that you need more details,
and Siri will prompt the user to be more specific.
Once the user provides more input,
we'll call the resolve method again.
If the user didn't specify a value, and you need one in order
to proceed, you can tell Siri that you need a value.
And in this case, we'll tell the user, "Hey,
you've got to say something for this because, you know,
Unicorn Pay needs to know who you're sending money to."
You can also tell us that you don't understand the input the
user provided, and that you need a completely new value.
And for that we have unsupported with reason.
And just tell us, you don't support this particular value,
and Siri will ask the user for a new one.
And finally, if there's a value that if a parameter is given,
and your app doesn't require it, or need it, just tell Siri that.
We won't prompt the user for it, and if the user happens
to specify a value, we'll simply ignore them.
So those are the resolve steps.
It's how we make sure that Siri, your app, and the user,
all agree on what they're asking your application
to do before we move on.
The next step is confirm.
So the confirm stage is really about two things.
First, it's a chance for you to tell Siri
about the expected result of handling an intent.
We will want you to provide this response
that we can tell the user what's going to happen.
It's also an opportunity for your app
to check any required state that it might need in order
to complete the request that the user is asking for.
Now no matter what, you're going
to provide an intent response back from the confirm step,
and we want you to populate as much information as you can,
in that intent response, so that we can communicate
that to the user very effectively.
And sometimes, Siri is going to prompt the user for confirmation
for certain intents and certain domains where we've decided
that that's important.
For example, things like sending payments, or requesting rides,
are things that Siri's going to confirm
with the user before we move onto the next step.
So let's think about what we're going to confirm
in the Unicorn Pay example.
Now Unicorn Pay has to be able to talk to our service,
make sure the user signed in, and a number of other things
in order to complete a request.
So we're going to check all of those in the confirm step.
We're going to say, "Can we reach the Unicorn Pay servers?
Is the user signed in to Unicorn Pay?"
And of course, this is expensive rainbow polish,
so we want to make sure the user has enough money
in their account before we send this payment.
Once we've made sure that our app is able
to handle this request, we generate an intent response
with a success result code, and then pass it back to Siri
so we can move onto the final step.
And that's the confirm stage of dealing with an intent.
The final step is handle.
And this is the most important part of working with an intent,
because at this point, you're going to do the thing
that the user asked you to do.
You're going to perform the action on behalf of the user.
And just like in the confirm step, it's important
that you provide as much information
as you have available back to Siri, because we want all
of that information so that we can create the best possible
user experience for all of your app's users.
Sometimes we're going to display this information visually,
and other times we're going to use it as part
of the conversation and speak it back to the user,
so that they understand what your app did.
Now we understand that sometimes your app might need a little bit
of time to complete a request on behalf of the user.
Networking calls might take a while, or maybe there's a human
on the other end of your service that's, you know,
processing transactions or something.
While we're waiting for a response back from your app,
were going to show a waiting UI to the user,
and say that your app is currently handling
We really would like you to provide a response
within a few seconds, two to three seconds at most.
And ideally, you can get back to us with a success
in a very short amount of time.
However, if you know that your app is going
to take a little while to get back to the user with a result,
we'd like you to send an in progress response code.
This allows Siri to tell the user
that your app has received the request with as much information
as you've told us about, and that it's going to continue
to process the request even if they leave Siri.
So in the case of our Unicorn Pay example,
as soon as we get the handle step, we're going to talk
to our servers, we're going to send the payment,
and then give Siri an intent response with a success code
so that Siri can tell the user their money is safely
on the way.
And that's what involved in how we talk
to your application with SiriKit.
We have intents, and intent responses
that are creating this common language
between your app and Siri.
The next thing we're going to talk about is what's involved
in adding SiriKit to your application.
So with SiriKit in iOS 10,
we're creating two new extension points.
An intents extension, and an intents UI extension.
I told you this was some intents stuff.
We also are adding some API
to help you help Siri recognize what the user is saying
for your application.
And we're going that in form
of what we call app vocabulary, and user vocabulary.
I'd like to get started by talking
about the intents extension.
This is what's going to implement the resolve, confirm,
and handle methods that we were just talking about.
Now, at its core, the intents extension is the most important
fundamental part of SiriKit.
This is what we build all of our interactions off
of between Siri and your application.
Your intents extension can support one or more intents,
and we leave it up to you to decide what makes the most sense
for your app and how you'd like to implement your extensions.
Now we encourage you to minimize the number of extensions
so that we aren't launching as many processes
if the user makes multiple requests to your application.
But do what makes sense for you.
It's also important to realize,
that your intent extension is going to run
in the background while Siri is in the foreground.
This allows Siri to maintain a conversation with the user,
and still talk to your app to get data back and forth
and communicate about the user's request.
And as I mentioned, your intent extension is what's implementing
the resolve, confirm, and handle methods for every intent
that your app supports.
Now when we built SiriKit, we put a lot of time into thinking
about security, because security is fundamental
to everything we do at Apple.
Part of Siri's built in behavior is that we're going
to restrict certain behaviors
from being taken while the device is locked.
For example, we wanted to make sure users authenticate
into their device and unlock it before they send a payment
or request a ridesharing vehicle.
And that's a built in behavior we have for certain intents.
Now we also understand that there are some behaviors
that your app would like to increase that security for.
By default, Siri's going to allow things
like sending messages from the lock screen,
but if your app would like to ensure that the user has logged
into their device first, you can tell Siri that you would
like to increase our default security,
and restrict your intent while locked.
And this is simply done through an info plist key.
We've also worked to make sure
that the local authentication framework is available
for your intent extension, so that you can ask the user
for additional authentication,
even while the device is already unlocked.
If you're not familiar with the local authentication framework,
there's a great talk from 2014 that goes into more detail.
And finally, we've made sure that Apple Pay is available
for your intent extension.
So if you'd like to complete a transaction using Apple Pay
with your users, that's available
to your intent extension and the Apple Pay sheet will appear
right above Siri.
If you'd like to learn more about this,
I'm glad someone uses Apply Pay here.
I'm excited about it.
You can learn more in the What's New with Wallet
and Apple Pay talk that just took place yesterday.
So that's your intent extension.
This is a fundamental part of SiriKit and it's how we're going
to communicate with your app to complete the users request.
The next piece is your intents UI extension.
Let's go into a little bit more detail
about what that's all about.
So the intents UI intent extension is a chance
to bring your app's interface into the Siri experience,
and make Siri really feel
like your users are using your application.
You can bring your brand, your visual identity,
as well as other information into the Siri transcript.
And you do that by adding these views
to what we call a Siri snippet.
Your intents UI extension is always going
to return a UI view controller.
And you can add anything you'd
like inside that view controller.
Now it's completely optional to have an intents UI extension.
By default, Siri has an interface for every intent
that we support in SiriKit.
And we think that those will effectively communicate all
of the pieces of information your user wants to know about,
but if you'd like to make a more unique and branded experience,
this is your opportunity.
We've also made intents UI extensions available
for certain intents.
Where we think it makes sense to show an interface to the user.
So there's a number of things you can do
with an intents UI extension.
You've used it to show additional information that goes
above and beyond what's described
in the intent and intent response.
Maybe some information that's unique to your application.
You can also use it to update the user with the status
of a long running intent.
So if you've provided an in progress response code,
as we mentioned earlier, you could continue
to update the user about the status of that intent
in your intents UI extension.
Now no matter what, we're always going
to show your intents UI extension alongside other
So we're always going to show what your app icon
and app name at the top in a sash.
We may choose to show some buttons and other information
at the bottom of the snippet,
as well as other pieces of information.
Your view controller is always part of our Siri snippet.
Now there's a couple of cases where it makes sense
for your app to replace the content
that we're showing in Siri by default.
And this is things like messages, or maps, where we felt
that you should have the opportunity
to replace the default messaging or maps experience with the one
that you feel suits your app and your users best.
So you can tell us that your displaying a map or a message
with your intents UI extension,
and then we will omit certain Siri interface elements
in favor of yours.
Now it's also important to keep in mind,
that just like the new notifications extensions,
intent UI extensions are non-interactive.
So you can draw anything you'd like, you can put animations,
you can play any kind of views you would like,
but we would discourage you from putting controls
in your UI extension, because users will not be able
to interact with them.
So that's the intents UI extension.
Together these two extension points are everything you need
to add your app's capabilities to Siri,
and bring your experience and your user interface inside Siri
to make a better experience for all of your users.
The next two parts of SiriKit,
are all part of your application.
So let's talk about what those all involve next.
Now, all of the responsibilities for your application lie
in the area of vocabulary.
All of your applications have unique ways of talking
to your customers, and that your customers use to refer
to features of your application.
And Siri needs a little bit of your help
in understanding the things that users are going to say
to your app, and the things
that they're accustomed to talking about.
Some of these phrases are part of your application.
They're things that every user
of your app knows and understands.
Other phrases are going to be unique to individual users,
and we have the API to support both of these as part
of SiriKit, and both are very important
to making a great experience.
Let's dig into app specific vocabulary first.
These are words or phrases that are part of your application,
and they're known to every user of your application.
They're things like workout names or vehicle types
that everyone uses to refer to certain features of your app.
Now because they're part of your application, you define these
in a plist that's part of your main app bundle.
And also, because they're part of your main app bundle,
we encourage you to localize these,
so that they can be supported for every locale
that your app supports, and every locale
that Siri's going to be using.
Let's talk about what goes into that vocabulary plist,
because there's a number of parts to a vocabulary plist.
The first of them, are examples of how users are going
to talk to your application.
These are kind of what we call, like,
your hero use cases for your app.
In the case of our new Unicorn Fit app, it'd be an example
like start a workout with Unicorn Fit.
The common things that you think users will say
to talk to your app.
Now in addition to that, there are certain parameters
where you can provide your own vocabulary types.
For the Unicorn Fit application,
we're going to specify our own unique workout names.
Some things that are unique and unique to Unicorn Fit,
but known to all of our users.
Now I'm sure you all are very avid users of Unicorn Fit,
but if you're not, now we're going to learn
about the term Colorlicious.
It's a particular workout name, and it describes a thing
where you run around and you catch bits
of falling glitter as fast as possible.
It's a great workout.
You should all give it a try.
It's a new craze.
So we're going to tell Siri about the phrase "Colorlicious."
It's a unique term for our app.
Now, sometimes these phrases are not the easiest things
They don't look very obvious.
So you're also going to give Siri some
And this is just a simple phonetics spelling of the word
or phrase that you're telling Siri about.
And finally, provide some examples of how you're going
to use that phrase in your app.
"Do a Colorlicious in Unicorn Fit", would be an example
of this particular vocabulary entry.
The other type of vocabulary, is user specific vocabulary.
These are words or phrases that are unique
to individual users of your application.
And because they're unique to individual users of your app,
we're asking you to provide them at runtime
from your main application.
We ask you to provide these in the form
of an ordered set of terms.
Now you might be thinking, and ordered set is kind
of an odd data structure choice.
Why would we be choosing to use that?
Well I promise you there's a really good reason for it.
We really, we really want you --
we'd encourage you to order the vocabulary in a priority
for your users, with the most important terms or the things
that you think users will say the most up front.
Let's talk about that in the context of Unicorn Pay,
where we have an address book
of everyone we can send payments to.
Now we want you to give Siri the names of all of the contacts
in your application, so that if the user says those names,
we'll be able to recognize them.
However, Unicorn Pay keeps track of the people
that we've paid most recently.
So we're going to put our recent contacts ahead
of all the other contacts in our application.
In addition to that, Unicorn Pay has a set of favorites,
which are the most important people to the user.
So we're going to prioritize those first,
and start off our ordered set with our favorites,
followed by our recent contacts, and then wrapping it
up with the other contacts at the end.
And by prioritizing in this way,
your helping Siri better understand your users
and make a better experience for everyone
who uses your app with Siri.
There's a few types of things that we support
with user vocabulary, and those are things like contact names,
workout names, photo albums or photo keywords.
And as an added benefit to your application,
if you rely on the system iOS address book, you don't need
to worry about this part at all for contact names,
because Siri has a built in understanding
of the system address book.
So you only need to provide contact names
if your app keeps its own dedicated address book.
Now, you also should think about what information you're giving
to Siri as part of this vocabulary mechanism,
and only tell us about information
that you your users would expect Siri
to recognize and know about.
So we don't want you to give us things like phone numbers,
or e-mail addresses in your vocabulary data.
Just give us simple common things like your contact names,
the things that users would expect to see come from Siri.
Now it's important that you update Siri promptly
as data changes, because users are accustomed to saying things
to Siri the instant they put them into their phone.
So for example, if the user creates a new contact
in your app, make sure you tell Siri about that as soon
as they hit the save button.
That way we can ingest the data, and be ready to recognize it
as soon as users hold down the home button and talk to Siri.
It's even more important that you delete information
from your Siri vocabulary promptly,
because your users would be very unhappy
if they deleted something from your application,
and then Siri was still recognizing it a few days
or a few weeks later, because you forgot to tell Siri
that the user deleted that particular piece of information.
And finally, make sure that you remove all
of your vocabulary data from Siri if the user chooses
to reset your application, or log out.
Now the other piece of SiriKit has to do with permissions.
So like many other parts of iOS,
users have to approve authorization
for your app to talk to Siri.
And we do this through the standard TCC alerts
that you're used to seeing for things
like the camera, or the microphone.
Now your app is able to provide a string that goes
into this alert to tell users what kind
of information your app is going to be sending to Siri.
This is also a fantastic opportunity to teach your users
about what your app does,
and what cool new features you've just created
for all of them.
We also are encouraging you to request permission
from your app the first time a user uses it
after they upgrade to iOS 10.
We want you to do that so that users know about Siri
and have approved the use
of Siri before they make their first request.
So as soon as they try talking to Siri, and your application,
it's already approved, and they have a fantastic first
So that's what's involved
in adding SiriKit to your application.
These two pieces of extensions, these two new extensions,
and two vocabulary mechanisms.
The last thing I would like to talk about is how SiriKit fits
into the larger world of iOS.
Now Siri is an integral part of our operating system.
It's always accessible to the user via the home button,
via hey Siri, via headset microphones
and many other ways of triggering Siri.
And SiriKit is just as integral
to the operating system as Siri itself.
Now Siri integration is part of this larger intense framework
that we've created this year, which is all
about communicating these common and shared actions and intents
between your app and iOS.
And the intense framework is going beyond just Siri
integration this year.
It's part of the contacts integration,
where you can donate information to the system,
and elevate your application to be the default messaging
or telephoning mechanism for certain contacts.
We've also done very deep integration with CallKit,
to make a great VOIP experience.
The most, the biggest piece of this is with maps.
Maps is adding ridesharing to iOS 10,
where you can book a ride right within the maps application.
Now we worked very closely with the maps team
to create a common extension point.
This intense extension and ridesharing intents are used
for Siri, and maps, to create the same experience.
So if you adopt SiriKit and the intents extensions, you're going
to get the maps integration for free.
We think this is a great way to save you some time,
and make an even better developer and user experience.
The last part I'd like to talk about,
is how users talk to your application.
Now it's very important as part of our security
and privacy model of iOS
and making your customers comfortable with your app
and your data, it's important that they understand when a --
they're going to be talking to your specific application
and when their data is going to your app.
To make sure that all of your users know that,
users are required to say the name of your app
when making a request to Siri.
Specifically, we're going to be looking
at your bundle display name.
This is what's shown on the home screen,
underneath your app icon, to your users.
Now Siri's all about natural language
and having a comfortable fluid conversation with the user.
So to make sure that that's part of how this works,
users are able to say your app name in many parts of speech,
at the beginning, or the end of an utterance, in the middle,
wherever it fits in naturally, because Siri wants
to make a natural fluid experience for talking
to your app, and to your users.
One of the common things that users do
with SiriKit is what we call verbing the app name,
or taking your app name and using it as a verb
to describe a particular action.
As an example of that, they might say,
"Unicorn Chat Celestra order five rainbow rolls please."
And in this case, they're using Unicorn Chat as the verb,
describing the action they're asking us to take.
And this will, of course, work fantastically well with Siri.
In fact, there's a multitude of ways that users can talk
to your app using Siri.
The possibilities are endless.
So that's what's involved
in implementing a great Siri experience for your users.
But there's more than just engineering implantation
involved in creating a good Siri experience.
And to tell you more
about designing a great Siri experience,
I'd like to invite Buttercup, my co-worker Corey, on stage.
I'm Corey Peterson, and I'm a designer on Siri,
and I'm very excited to talk to you
about creating magical experiences with SiriKit.
I've got three things I want to talk about today.
The first is some of the things that Siri does for you
to help facilitate the conversation with your users.
Next, we'll go through some design guidelines
that you should keep in mind as you start to work with SiriKit.
And then, I'll give a few tips of what you can do
to polish the design, and take your app to the next level.
So, let's get started with user interface.
Now when I say user interface, you're probably thinking
about something like this.
A button. Part of a visual interface
that appears on screen.
Now visual interfaces have well understood behaviors
and patterns and
and everyone is pretty familiar with using those.
It's part of an application,
and that application gives the context of what you're trying --
the user is trying to do,
the options that they have available,
and the details about those elements.
And there's pretty much one way to interact with a button.
But, as you start to work with conversation things change,
and one of the biggest differences is
that button is gone.
Now, it can be -- feel strange not having information
on a screen.
So how do we solve this?
Because the user could say whatever they want,
or they might not even know what to say.
So we like to think of it as a conversational interface.
The context of what's happening is established
through the conversation between the person,
Siri, and now your apps.
Now, without a visual reference, the user, or the person has
to keep all this information in their head.
So Siri works hard to prevent --
to present just the relevant critical information
to get a task done, and not overwhelm them with all
of the other information.
And, this interface is shaped by the questions and the responses,
both from the person and from Siri.
And it's important to think
about how Siri asks and responds.
And your application has the ability
to shape this conversation.
And we'll talk about that a little bit later.
So let's go through an example of how this can work.
Let's say, someone has created a message,
and Siri might say something like,
Ready to send it?
Now people can say a lot of different things here.
They might simply say "send it."
Or they might decide not to,
or they might ignore the question entirely,
and say whatever they want.
And that's okay.
These are actually appropriate responses in this situation.
And the great thing is, Siri's going to take care of this
for you, and send the right information back to your app.
And even better than that, Siri will handle the thousands
of other ways that people may say the same thing.
Now earlier, Robby talked about all the different ways
that people can start a conversation with Siri,
and I'm going to focus on a couple of those.
Now everyone's familiar with picking up your device,
pressing the home button, and starting the conversation.
But, another important mode that we have is what call hands free,
where you don't have
to physically touch the device to start talking.
So just by starting your request with "Hey Siri" you're able
to work from across the room, or -- or well, across the room,
if your hands are full, or if you're doing something else.
Let me show you a couple examples
of how the conversational interface changes
to adapt to these situations.
So, here, we're going to say I'm holding onto my phone
and pressing the button.
So I'll say, "Do I have any reminders?"
You have five reminders.
So as you heard, Siri presented a quick answer,
and then gave visual information on the screen
since I'm holding it and I can take a look at that.
Now, listen to how this changes when I activate hands free.
Hey Siri, do I have any reminders?
I found five reminders.
The first one is talk about accessibility, then get Siri
to say hi to people in Cupertino,
then go to the next slide.
Do you want me to read you the rest?
No thanks Siri.
So as you heard, Siri read out that information
and it gave a smaller amount
of information that's easier for me to understand.
And at the end, Siri asks me for the next step so that I'm
in control of the conversation and what happens next.
Now we use these same principles in other places as well.
In car play, where you need to keep your eyes on the road.
And it's also very helpful for people with accessibility needs.
Now Siri is great because you can ask for it to perform a task
and its able to do so, and you don't have
to find an application and navigate through a visual UI
to get something done, but Siri does more.
Siri understands when you have devices connected, such as a set
of headphones, or a Bluetooth hearing aid.
And it adapts what it -- oh, and in another case,
it can recognize settings such as voice over,
which is Apple's technology for reading
out the visual elements on screen.
And it again, adapts its behavior
to help these users complete their tasks.
So, that's a little bit about how Siri helps you.
Here's a couple things that you can do to help Siri.
Make sure that you're testing, as you develop your apps,
with many different requests.
Testing different situations, and especially try
out things hands free.
And throughout the process,
make sure that your app is providing information back
to Siri so that Siri can represent it back to the person.
And that's especially critical in these hands free situations.
All right, let's move on to some design guidelines.
And I want you to think of this
as a conversation with your users.
And as with any important conversation,
you need to make sure that you prepare, that you listen,
that you ask questions, and that you respond to the request.
So, how do we prepare?
The first thing is to recognize
that your users aren't exactly like you.
Now, what do I mean by that?
You've spent a huge amount of time developing
and designing your application, and you have an amount
of knowledge that's greater than anyone else will ever have
with it, and that's going
to influence how you would talk to Siri.
So, you need to go out and find some real people, maybe friends,
significant others, or maybe even people in marketing
and ask them what would you say to Siri?
So I'm going to go with our theme from today,
and talk about an application Unicorn Rides.
So for this fake application, we went and talked
to some real users and they came back with some responses.
Some people might say "Get a ride", and in this case,
Siri is going to ask the user which application they want
to use, and then can continue through the rest of the task.
Some people may provide more information,
and some people may provide a whole lot more information,
some of which may not be possible,
because as we all know, it's really hard to fit more
than two people on a Pegasus.
Once you have all these responses,
make sure that your app is a good listener.
Take all of the information that the user has provided
and use that, but they may not provide all
of the information that you need.
So, you need to find ways to get it.
One way, is to pick a good default,
so if I were requesting a ride, and I didn't say
where to be picked up, Unicorn Rides could pick a good default
that I probably want to be picked up right here,
or maybe outside the building down by the curb.
You can also make educated guesses based on your knowledge
about the user, but avoid bad surprises.
You don't want to pick the most expensive option,
or some other default, or some other guess that a user is going
to be unhappy with later.
So if you have questions and you're not sure
if someone would be unhappy, maybe the better answer is
to ask the question to the user.
But, with this, you need to make sure you keep it simple.
Nobody wants to play a game of 20 questions when they're trying
to get something done.
So, only ask for the things that are absolutely necessary.
And people may provide information
that your app can't handle,
so you need to send those errors back to Siri
so that Siri can present it to the user,
and try to move forward with the task.
There are two types of questions that I want to go discuss more,
and Brandon talked about them earlier.
The first is confirmation.
And this is great when you have one item, that you want
to check with the user.
And this could happen because you made an educated guess,
or if there's an item that would have really big impact.
So if someone is used to making small purchases,
or small payments with your payment app,
and suddenly they make a payment that's much, much larger,
you may want to confirm that with them
to make sure it's what they meant.
The next is disambiguation.
And this is great when you have multiple options,
and you need the user to pick one.
So this can be used when you have a small list,
such as the available ride types at a given time.
Or, if you have a much larger list, say I was going
to send a message to Eric, and I have a number of people
in my contacts, I can filter that down
to just the possible matches for Eric,
and let the user pick that.
Siri can present this list both visually, and through speaking.
And the user can respond by tapping or saying the response.
So before you perform the task, Siri will check
with your extension, to make sure everything's ready.
And it wants to get all the information back
from your extension, so that it can then present it to the user.
Now, again, Siri will present this vocally,
and it will provide a UI on screen to show this information.
If the user decides to continue,
if the user decides to continue,
you should go perform the action, and again,
return those results back to Siri so that Siri can show those
to the user and tell the user.
And in the same way that the user made the request,
they started in a conversation and Siri can reply back
in the conversation as well.
The great thing about that, is it works everywhere.
It works in car play, it works hands free, and it works
when somebody's touching the device.
And Siri will again, provide a visual UI to help out.
But, if you want,
your application can provide a custom UI to bring the feel
of your app into Siri.
I want you to exercise restraint and limit what you show.
The person is trying to get a task done, and you don't want
to distract them
with information that they don't need.
You also need to make sure that you're designing this UI
for different screen sizes, from the smallest phone
up to the biggest iPad.
Here's another example of the custom UI
from our Unicorn Chat application.
And as Brandon said, you should use our API
to hide any redundant information.
As you do so, you need to make sure
that your custom UI represents everything visually,
and that you send all the information back to Siri
so that hands free requests will still work
when they can't see this UI.
In a few cases, Siri will open your application
to finish a request, and we would do
that for a photo search request
where we can show the results in your app.
And this is great because your app can provide the
Now anytime a user goes from Siri to your app,
you don't want to surprise them.
Don't put any interstitial information or alerts
in the way, and make sure
that you are only performing the action that they have requested.
Okay, next, let's do a couple tips
about polishing your design.
So, earlier Brandon talked about the app specific vocabulary,
and the examples that you can give.
And people can find that information by going to Siri
and just asking, "What can you do?"
Siri will show lots of different things that it can do,
including your applications, and the examples that you provide.
So I want to give you some tips about how
to write good examples.
Make sure that your examples include your app name,
so that people can get to your app for your functionality.
Keep your examples short, and focused on what your app can do.
You should provide multiple examples for each intent
that you support, and make sure
that you prioritize both your intents
and the examples within those intents.
If your app works in multiple languages and locales,
make sure that you provide localized examples
in the appropriate places.
And most importantly, make sure that they work.
A couple things to avoid.
You don't need to address Siri in your examples,
and you also don't need any unnecessary pleasantries.
Just focus on the functionality of your app.
And the last thing I want to talk about,
is iterating on the experience.
You want to make sure that you have time to explore
and to experiment in how you shape the conversation.
Make sure that you talk to real people,
not just what they would say at first,
but throughout the process of getting something done.
And again, try it in different situations, hands free,
or out and about, not just testing at a desk.
And throughout, make sure
that you are asking, "Does it feel right?"
If you need to, make a change, and iterate again,
because we want to get to a point
where you have a great Siri experience for your users.
So, let's recap what we've talked about today.
Robby started us out introducing SiriKit, bringing the power
of Siri and your applications to a conversation for your users.
Brandon talked about creating extensions
and the way they communicate with Siri through intents
and intent responses, and the methods
of resolve, confirm, and handle.
And also that your apps need to provide vocabulary
of the common things that people will say to your app.
And then we talked about design.
Introducing the conversational interaction,
the conversational interface,
and talking about the options you can choose
as you're developing your application
that will shape the conversation for your users.
And last we talked about iterating, so that we can get
to a Siri experience that feels effortless, and magical.
We have more information about this session on the web.
We have some related sessions this week,
and I want to especially highlight Extending Your Apps
with SiriKit, which is tomorrow,
where some of our colleagues will go into more details
about the API and show you the code
for an example integration with SiriKit.
So thank you very much, and we can't wait
to see what you create.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.