Build intelligent Siri experiences with App Schemas

Hi, I'm Dan Niemeyer, a software engineer on the Swift Intelligence Frameworks team. Today, I'll show you how to bring your app to Siri using new capabilities powered by Apple Intelligence. In the 27 releases, Siri is more capable, more contextual, and more personal. And App Intents are the foundation that make that possible.

Here's what I'll cover today. Starting with what's new with Siri and App Intents. How Siri understands your app's content, and when Siri understands your content, enabling it to take action.

At the end of the talk, I'll cover use cases for working across apps, such as asking Siri to text my wife her plane ticket, or apply a cinematic filter to this photo. And share some best practices for how to bring your app to Siri by extending the system's understanding of what your app can do.

Alright, there's a lot here, so let's get started.

In the 27 releases, Siri takes a big step forward, powered by Apple Intelligence, our personal intelligence platform that helps people be more creative and productive in the work they do. As a developer, the way you participate in Apple Intelligence is through the App Intents framework.

App Intents is the foundation for integrating your Apple Siri and Apple Intelligence, and provides a structured way to describe what your app can do and the content it manages. Everything I talk about today builds on top of App Intents.

If App Intents is new to you, I highly recommend starting with these videos, which cover the fundamentals in more depth.

This year, Siri becomes more powerful in three key ways. Siri can now access your app's entities, the real meaningful content inside your app. This means people can ask questions like: "When and where is my next meeting?" And Siri can answer directly by understanding what a meeting is in your app, which meeting is relevant, and which properties to return, like time and location.

Siri can take action using your app's intents. For example, someone could say: "Send my latest report to Mary". So, people can send emails just by asking. Intents describe the actions your app supports, the parameters they require, and when they're safe to run. Siri handles the language understanding. Your app focuses on the action.

Siri can also understand on-screen context. For example, people can say things like: "Explain this text", or: "Get me reviews for this product". When you annotate views with entities that describe what's on screen, Siri can then understand what content is meaningful, which entities it represents, and what actions apply. This gives people a more contextual conversational experience that feels natural and incredibly powerful.

So that's a preview of what's new with Siri, how we can find content, take actions, and understand what's on screen. Now it's time to ground all of that in a real app.

For the rest of this video, I'll work in a sample app called UnicornChat.

UnicornChat is a messaging app where people can chat with fictional unicorn characters like Bubbles, Flare, and Glow. It has just enough functionality to demonstrate the concepts I'm covering today.

And while it's a messaging app, the same ideas apply to apps across many other domains. You can download the sample from the Apple Developer website and follow along.

Having explored what's new and the app we'll use to demonstrate it, let's talk about how you can bring these capabilities to your app. At the center of all of these App Intents-powered experiences is a single concept - app entities. An AppEntity is a structured representation of the content inside your app. To make this concrete, think about the data your app already works with every day.

If you have a calendar app, each event is an entity. If you have a mail app, each message is an entity. And if you have a photos app, each photo and each album is an entity.

App entities describe three important things. What the thing is, how it's identified, and which properties matter, like a title, a date, or some text.

They're not a new data model. They're a way of describing your existing content so the system can understand it.

Now, modeling an entity is the first step, but on its own, that's not enough for Siri to be able to find it or talk about it. For Siri to understand what an entity is, what category of thing it represents, your entity needs to conform to an AppSchema.

App schemas give Siri a predefined understanding of common concepts, like messages, contacts, or documents. When your entities conform to a schema, Siri already knows how to reason about them.

Instead of treating your app like a black box, Siri can reason about what the user is talking about. In UnicornChat, the nouns in the app are Contact, Conversation, and Message. All three are modeled as app entities that conform to app schemas.

That's what allows Siri to understand questions like: "Show my last message from Flare", one of our contacts, or "Open UnicornChat with Glow", another unicorn friend.

So once you've modeled your content as app entities, the next question is, how does Siri find the right one? Entity resolution is how Siri resolves what the user says into real app entities inside your app.

For example, when a user says: "Open UnicornChat with Glow", Siri resolves that Glow refers to a specific unicorn, finds the matching contact, and fills in the entity along with the values of its properties, like its name and identifier, so it can be used by the system.

But people don't always ask for things using exact names. They speak in concepts and descriptions. When someone says something like: "The best windsurfing in Carmel", they're not looking for an exact text match. They're expressing meaning. To support that experience, Siri needs more than string matching. It needs semantic search. And that's exactly what IndexedEntity enables.

The primary way to power entity resolution is by adopting IndexedEntity. When you do this, your app's entities are indexed into the system semantic index. This allows Siri to match based on meaning, not just text, understand relationships between entities, and even answer questions over your content. For example: "Show the messages with Flare about movies" - that's not a string match. Siri can find messages that reference movie titles because it's performing a semantic query over UnicornChat's indexed messages.

And the way you get that behavior is by conforming your schematized entity to the IndexedEntity protocol.

The indexingKey tells Spotlight which properties, like the message body, should be searchable.

Once indexed, Siri can search your content, reason over it, and use it to answer questions, not just retrieve items.

IndexedEntity enables the best Siri experience with semantic matching, fewer follow-up questions, and more natural language understanding.

But not everything can be indexed. Your dataset might be large, lives on a server, or changes too frequently to index ahead of time. In those cases, you can use EntityStringQuery.

With a string query, Siri hands you the person's input. Your app is responsible for finding matching entities and returning them.

You don't get semantic understanding, but you do get full control over how you search for and match your app's entities.

So to bring this all together, start by modeling your app's content as app entities.

Conform each entity to an AppSchema so Siri can understand what kind of thing it is. When your data can be indexed, adopt IndexedEntity. Or use EntityStringQuery when indexing isn't feasible. This unlocks powerful experiences like answering questions and finding content instantly. But entities on their own are just information. Where things really get interesting is when you combine entities with actions.

App Intents are how your app exposes actions to the system. But it's important to understand that not all actions are treated the same way. So we're going to break this into two parts.

When you define an app intent, that action can show up across the system in places like Shortcuts, Spotlight, Widgets, and more. This means people can discover and trigger your app's actions in many places, even without Siri.

You describe what the action does, you define its parameters, and you implement the behavior. The system takes care of surfacing it, suggesting it, and wiring it into system experiences. This is incredibly powerful and provides great benefits for your app and the people who use it. But you can provide even more value by taking a few extra steps to bring your actions to Siri.

Just like entities use app schemas to be understood, actions use schemas to become executable by Siri. Think of schemas as a specialization of App Intents. They're still App Intents, but shaped in a way that Siri knows how to process.

Schemas define the kinds of actions Siri understands, the structure it expects, and how those actions map to natural language. This is what allows Siri to confidently handle commands like: "Send a message to Mary", or: "Play my focus Playlist". Individual schemas define individual actions, but apps usually need a complete set. That's why schemas are grouped into AppSchema domains.

Each domain represents a category of tasks, such as mail, photos, messages, and more. When you integrate with a domain, you implement a set of predefined app schemas. You map them to your app's functionality, and Siri immediately knows how to talk about your app in that domain. Think of domains as categories of contracts between your app and Siri. They tell Siri what your app can do, what actions are available, and how the system should respond.

For more details on how app schemas work, check out my video from WWDC24.

Now that we understand the difference between general App Intents and series-specific app schemas, let's bring this to life by adopting one of these domains in UnicornChat and wiring it up end-to-end.

Just like before, everything starts with entities. In UnicornChat, we already modeled the two things we need to send a message.

Contacts which represents the recipient and exposes properties like a name and identifier.

And the message, which captures things like the message body and author.

These are app entities, which tell Siri who the message is going to and what is being sent. Now, let me show you how we connect those entities to an action by adopting an app schema.

In Xcode, we start by typing the schema name. Xcode already knows about all the available app schemas grouped by domain, and Autocomplete lets us pick exactly the one we want.

Since we're building messaging functionality, we select the sendMessage schema from the messages domain.

Instead of inventing a custom intent structure, we're adopting a schema Siri already understands and mapping it to UnicornChat's existing logic. This tells Siri what the action does, which parameters it expects, like the recipient and the message content, and how to guide the customer if something is missing.

Our job is to map those schema parameters onto UnicornChat's existing messaging flow. First, we process the parameters.

Then we pass them into UnicornChat's interface so the message is actually sent.

Finally, we return the newly sent message back to the system as an app entity.

And that's it.

Because this action is implemented as an app intent, it's available throughout the system. And because it conforms to an app schema from the messages domain, Siri can execute it directly without you having to handle natural language yourself. Let's see it in action.

Here I say: "Send a message to Glow in UnicornChat, saying 'What movies do you recommend?'" Siri resolves Glow using our AppEntity query, invokes our intent, and sends it.

All without opening the app.

This is the power of app schemas. Once your entities are in place, you connect them to well-defined actions and Siri handles the rest.

So to recap, App Intents expose actions to the system. App schemas make those actions understandable by Siri. App schema domains like messages package schemas into powerful end-to-end experiences. Once you adopt a domain, Siri can speak your app's language fluently.

So far we focus on how Siri can understand your content and take actions inside your app. But many real-world requests span multiple apps.

They start in one app, continue in another, and finish somewhere else. For example: "Hey Siri, email my wife this reply from Bubbles." These experiences combine two capabilities, understanding what people are looking at and moving that content to another app.

Let's break that down into two parts. First, Siri needs to understand that this reply from Bubbles refers to, that's on-screen awareness. Then, Siri needs to pass that content into another app so an action can be performed. That's content transfer.

We'll look at these one at a time. To enable on-screen awareness, your app connects what's visible on-screen to structured information the system understands. At the core of this is app entities. When views are associated with entities, Siri can resolve references like "this message" or "that conversation" without people naming them explicitly. There are two APIs for annotating on-screen content, and they serve different purposes. Use UserActivity when there's one primary thing on-screen, like viewing a document or composing a message.

Use View annotations when multiple meaningful items are visible at once, like messages in a conversation or items in a list.

Here's how you can achieve this. Each row in this list represents real data in your app. In UnicornChat, that's a message. Each message row is annotated with its corresponding message entity. This explicitly connects what's on screen to the same entities we already use for intents. It allows customers to say things like: "Edit this message", or: "Forward the last one". Siri resolves entities directly from the view, enabling powerful in-app experiences. But the real power comes when you combine annotations with content transfer.

Content transfer is what allows other apps to act on your entities. You enable this by exporting your entities using Transferable. This tells the system how to represent your content in a form other apps can understand.

For example: "Text my wife this conversation", or: "Summarize this message".

You do this by adopting Transferable and providing an IntentValueRepresentation. In UnicornChat, we export a ContactEntity as an IntentPerson. Once exported, the system can pass that content into actions from other apps. It enables use cases such as: "Call this contact". Your app doesn't need to know what happens next. It just needs to describe its content accurately.

When content comes into your app, there are usually two possibilities. Either that content refers to something that already exists, or it represents something entirely new.

You get to decide which path your app takes. If you're matching existing content, use IntentValueQuery. If you're creating something new, use importing on the transferRepresentation.

Use IntentValueQuery when incoming content should resolve to an existing app entity in your app. Conceptually, this is very similar to entity query but scope to intent parameters instead of standalone entity resolution.

In this example, UnicornChat receives an IntentPerson from another app and tries to match it into an existing ContactEntity. This is ideal when the content already exists in your app or you want to select from existing data. You're essentially saying: giving this incoming value, which of my entities does it refer to? Use IntentValueRepresentation importing when incoming content should create something new in your app. Instead of resolving to an existing entity, you convert the incoming value into a brand new app entity. In UnicornChat, this allows us to create a new unicorn from an IntentPerson when needed. From the user's perspective, the content just works. But your app stays in control of how that content is stored and managed.

So when importing content, if the content already exists in your app, resolve it. If it doesn't, import it. Many apps use both, depending on the intent and workflow.

So to recap, on-screen awareness lets Siri understand what the user sees. Content transfer lets that content move across apps. Together, they unlock powerful multi-step experiences, all built on app entities, app intents, and schemas. Now that we've covered entities, actions, and working across apps, let's talk about best practices. Beyond getting things working, there are a few practices that make Siri work better with your app and more resilient over time. Because while you can adopt individual schemas and APIs independently, great Siri experiences often depend on how these pieces work together. And the good news is you don't have to figure this out alone. The tools are designed to help you along the way. Let me show you what I mean with a demo.

Let's jump back into Xcode and pick up where we left off. In our last demo, we adopted the sendMessage schema in UnicornChat. That works great.

People can send messages with Siri. But now, let's try to build.

And we get a build error. Xcode is telling us that while we adopted sendMessage, we haven't adopted the related draftMessage schema. This is important because some Siri scenarios require more than one schema to deliver a complete experience. This isn't just a compiler error, it's a design hint. Xcode knows that if your app can send messages with Siri, it also needs a way to draft messages, especially when confirmation is required. So instead of failing silently at runtime, the build system surfaces this early.

If we click into the error, Xcode offers a fix-it.

Xcode generates a sample adoption of the draftMessage schema. This gives us an intent definition, the required parameters and a stub implementation. All wired correctly. From here, we just need to fill in the app specific pieces. First, we connect the intent to our entities.

Then, we inject our dependencies.

Next, we process the input.

And finally, we open the message creation view.

Since this action mutates UI state, it needs to run on the main actor.

Now, when we build again, the build succeeds.

We've completed the schema adoption, and Siri now has everything it needs to guide people through the full messaging flow.

The important thing to remember is, if a Siri experience depends on multiple schemas, Xcode will tell you, show you what's missing, and help you generate the right remaining steps. This makes it much easier to build complete high-quality integrations.

Once you've adopted your schemas, testing becomes critical. And the best place to start is AppIntentsTesting. This is a new testing framework that lets you exercise your intents entirely in isolation. No Siri involved. You can invoke your intent, pass in parameters, and validate the result, just like any other integration test. This is the fastest and most reliable way to validate your business logic early in development.

Check out the video "Validate your App Intents adoption with AppIntentsTesting" to learn more.

Once your logic is solid, the next step is the Shortcuts app. Shortcuts gives you a structured UI for your intents. Letting you inspect parameters, tweak inputs, and understand how your action is presented to people. This is where you validate the shape of your intent. Not just what it does, but how it's configured and exposed.

Next, move to Spotlight. Spotlight is where you validate your content integration, ensuring your entities are indexed correctly, discoverable, and linkable. This helps you confirm that Siri can find the right data before it ever tries to act on it.

Finally, test a complete experience with Siri. This is where everything comes together. Natural language, entity resolution, on-screen context, and cross-app workflows. Testing end-to-end ensures that everything works together the way your customers expect.

At this point, you've seen how to model your content with app entities. Expose actions using App Intents and app schemas. Enable cross-app workflows. And use tooling to build and test confidently. So let's wrap up with a few concrete next steps.

Here's how to get started. Model and index your entities to Spotlight so Siri can find your content. Adopt app schema domains that match your app's core experiences. Adopt Transferable to enable content import and export. And test early and often using AppIntentsTesting, then Shortcuts, Spotlight, and Siri. All of these APIs are available today, and they're designed to scale with your app as Siri continues to evolve.

Siri is becoming more powerful, more contextual, and more capable. And App Intents are the foundation that make that possible. By bringing your app to Siri, you're not just adding voice support. You're making your app faster, more accessible, and easier to use across the system. Thanks for watching, and we can't wait to see what you build.