Use Case
I’m currently developing , a hiking and cycling app. I’d like to use Siri AI in iOS/watchOS 27 to create an experience like this:
While users are recording an activity in app, they can raise their wrist or invoke Siri on iPhone and simply ask:
“How much farther to the destination?”
“How far have I already gone?”
“How much elevation gain is left?”
“How far is it to the next waypoint?”
…
Siri can then answer based on the app’s current workout context.
Since there are many metrics like these, and it’s not convenient to look at the screen while cycling or trail running, this could be a really valuable use case if Siri AI can support it.
I defined an AppEntity like this, and then associated it with the UI.
But I found that Siri AI doesn’t seem to understand this AppEntity. It can only understand the content it can see on the screen, even though I can see that it did execute ToobooWorkoutSessionQuery, the EntityQuery.
This screen has multiple tabs. Siri understood the on-screen values such as “Distance: 0.21 mi” and “Remaining Distance: 0.14 mi”, but it didn’t know the data from the other tabs on the same page, such as heart rate, average speed, and so on.
So it seems that Siri is directly reading the screen, instead of retrieving the information from the AppEntity I actually provided.
Is this because the AppEntity doesn’t use one of the schemas defined by AppSchemaEntity, so Siri AI can’t understand it?
App Intents without a schema seem to have a similar issue. If the AppIntent I define is not within one of those schemas, Siri won’t directly execute it either.
My current solution:
I defined many Intents and Shortcuts. Siri AI’s ability to parse shortcuts seems to be much smarter in the new system, and it can match them quite easily. Also, shortcuts don’t require the app to be in the foreground.
However, when talking to Siri, the shortcut phrase has to include the app name, which feels a bit unnatural. For example:
“How far have I traveled in xxx app?”