Siri and Voice

RSS for tag

Help users quickly accomplish tasks related to your app using just their voice.

Posts under Siri and Voice tag

42 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

muting SIRI via AppIntents
We're using App Intents to launch are control our app via Siri. Siri's responses have been fairly random, some with a "Done" popup, others with a verbal confirmation, others saying "I'm sorry, there's been a problem". The latter is bogus and doesn't look good to potential investors when the app is actually working fine. There appears to be no way in code that I've been able to find so far that would have been tell Siri to STFU. Let us handle our own errors. Otherwise is there a means to supply Siri with a dictionary of restored messages that could be triggered inside the app?
1
0
193
2d
Siri not calling AppIntents
I am very new to App Intents and I am trying to add them to my On Device LLM ChatBot app so my users can get answers to any questions anywhere in iOS. I have the following code and it is working wonderfully in the Shortcuts app. import AppIntents struct AskAi: AppIntent { static var openAppWhenRun: Bool = false static let title: LocalizedStringResource = "Ask Ai About" static let description = "Gets an answer from Ai for your question." @Parameter(title: "Question") var question: String static var parameterSummary: some ParameterSummary { Summary("Ask Ai About \(\.$question)") } @MainActor func perform() async throws -> some IntentResult & ReturnsValue<String> { let bot: Bot = Bot() await bot.respond(to: self.question) return .result( value: bot.output ) } } class AppShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { AppShortcut( intent: AskAi(), phrases: [ "Ask \(.applicationName) \(\.$question)", "Get \(.applicationName) answer for \(\.$question)", "Open \(\.$question) using \(.applicationName) ", "Using \(.applicationName) get help with \(\.$question)" ], shortTitle: "Ask Ai", systemImageName: "sparkles" ) } } I can create a shortcut for this AppIntent and that allows me say speak the response. I can call my shortcut via iOS 18 Beta 1 by the Shortcut name I set in the Shortcuts app and that allows it to work. It does not work at all by just Asking Siri any of the phrases I have defined. The info.plist has an app name alias defined just to be sure. I even added the Siri capability in Xcode-beta. I also tried using the ProvidesDialog return type too. Whatever I do the AppIntent is invisible to Siri. Siri tries to search the web, looking for my app name in the contacts or have an error Apple Cash which has nothing to do with what I was talking about. Is there anything else I am missing for setting up iOS AppIntents to work with Siri?
3
0
407
2d
Siri not recognizing the parameter in the phrase
I am trying to create an "OpenShowIntent" that allows the user to open a show from outside the app by either invoking the shortcut and provide a showName and/or by asking Siri "Open (.$showName) in (.applicationName)". Currently the shortcut works and but Siri doesn't recognize the showName and keeps on loading when I say the phrase with a show name. I was wondering where I am going wrong. Here are some of the code below: OpenShowIntent.swift: import Foundation import AppIntents import Models @available(iOS 16.0, *) struct OpenShowIntent: AppIntent { static var title: LocalizedStringResource = "Open a Show" static var openAppWhenRun = true @Parameter(title: "Show", optionsProvider: ShowEntityQuery()) var show: ShowEntity? @Parameter(title: "Show Name") var showName: String static var parameterSummary: some ParameterSummary { Summary("Open \(\.$showName) in (APP NAME)") } @MainActor func perform() async throws -> some IntentResult & ProvidesDialog { var showToOpen: ShowEntity if let show = show { showToOpen = show } else { let params = ElasticSearchParams(query: showName) let searchResults = try await IntentsHelper.getShows(searchParams: params) let entities = searchResults.map { show in ShowEntity(id: show.id, name: show.displayName, posterUrl: show.posterImageUrl) } showToOpen = try await $show.requestDisambiguation( among: entities, dialog: "Choose a show from this list?" ) } let deepLink = DeepLink( type: .show( showId: showToOpen.id, showDateStr: nil, voucher: nil, reviewModalParams: nil ) ) let url = try deepLink.createURL(source: nil) IntentsHelper.openLink(url: url) return .result(dialog: "Show '\(showToOpen.name)' opened successfully.") } } ShowEntity.swift: import Foundation import AppIntents import Models @available(iOS 16.0, *) struct ShowEntity: AppEntity { typealias DefaultQuery = ShowEntityQuery var id: Int var name: String var posterUrl: URL? static var typeDisplayRepresentation: TypeDisplayRepresentation = "Show" var displayRepresentation: DisplayRepresentation { var image: DisplayRepresentation.Image? if let imageUrl = posterUrl { image = DisplayRepresentation.Image(url: imageUrl) } return DisplayRepresentation( title: LocalizedStringResource(stringLiteral: name), subtitle: nil, image: image ) } static var defaultQuery = ShowEntityQuery() init(id: Int, name: String, posterUrl: URL?) { self.id = id self.name = name self.posterUrl = posterUrl } } ShowEntityQuery.swift: import Foundation import AppIntents import Models @available(iOS 16.0, *) struct ShowEntityQuery: EntityStringQuery { func entities(for identifiers: [Int]) async throws -> [ShowEntity] { let params = ElasticSearchParams(showIds: identifiers) let searchResult = try await IntentsHelper.getShows(searchParams: params) return searchResult.map { show in ShowEntity(id: show.id, name: show.displayName, posterUrl: show.posterImageUrl) } } func suggestedEntities() async throws -> [ShowEntity] { let params = ElasticSearchParams( showIds: BookmarksManager.sharedManager.getBookmarkShowIdsAsArray() ) let searchResult = try await IntentsHelper.getShows(searchParams: params) return searchResult.map { show in ShowEntity(id: show.id, name: show.displayName, posterUrl: show.posterImageUrl) } } func entities(matching query: String) async throws -> [ShowEntity] { let params = ElasticSearchParams(query: query) print("entities(matching:) called with query: \(query)") let searchResult = try await IntentsHelper.getShows(searchParams: params) return searchResult.map { show in ShowEntity(id: show.id, name: show.displayName, posterUrl: show.posterImageUrl) } } } ShortcutsProvider.swift: import Foundation import AppIntents @available(iOS 16.0, *) struct TTShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { return [ AppShortcut( intent: OpenShowIntent(), phrases: [ "View show in \(.applicationName)", "Open \(\.$showName) in \(.applicationName)", "Show \(\.$showName) in \(.applicationName)", "Find \(\.$showName) in \(.applicationName)", "Search for \(\.$showName) in \(.applicationName)" ], shortTitle: "Open show", systemImageName: "pencil.circle" ) ] } static var shortcutTileColor: ShortcutTileColor = .blue }
0
0
111
3d
Siri can directly launch Map App for navigation
DESCRIPTION OF PROBLEM We discovered that Siri can directly launch Amap and Tencent Maps for navigation. Our app also wants to integrate this feature. After our research, we found that AppIntents cannot accomplish this functionality. Because the parameters of AppIntents are limited. How can we achieve this? Do we need to apply for certain permissions from Apple? STEPS TO REPRODUCE awake siri say "使用高德地图打车去南京南站"
1
0
63
5d
How to add support for Siri / Apple Intelligence to my existing AppEntity?
iOS 18 adds a specific macro for exposing your search app intent, app entities, etc, to siri but how are you meant to add it to your existing objects without removing it entirely from < iOS 18 users? For example, i get the following error: AssistantIntent(schema:) is only available in iOS 18 or newer. Add @available attribute to enclosing struct. I don't want to do that since i still want to support iOS 17 users with my existing shortcuts. Do i need to duplicate my entire shortcuts model to add the new macro?
1
0
541
Jun ’24
Siri enters loop of requesting parameter when running AppIntent
I want to add shortcut and Siri support using the new AppIntents framework. Running my intent using shortcuts or from spotlight works fine, as the touch based UI for the disambiguation is shown. However, when I ask Siri to perform this action, she gets into a loop of asking me the question to set the parameter. My AppIntent is implemented as following: struct StartSessionIntent: AppIntent { static var title: LocalizedStringResource = "start_recording" @Parameter(title: "activity", requestValueDialog: IntentDialog("which_activity")) var activity: ActivityEntity @MainActor func perform() async throws -> some IntentResult & ProvidesDialog { let activityToSelect: ActivityEntity = self.activity guard let selectedActivity = Activity[activityToSelect.name] else { return .result(dialog: "activity_not_found") } ... return .result(dialog: "recording_started \(selectedActivity.name.localized())") } } The ActivityEntity is implemented like this: struct ActivityEntity: AppEntity { static var typeDisplayRepresentation = TypeDisplayRepresentation(name: "activity") typealias DefaultQuery = MobilityActivityQuery static var defaultQuery: MobilityActivityQuery = MobilityActivityQuery() var id: String var name: String var icon: String var displayRepresentation: DisplayRepresentation { DisplayRepresentation(title: "\(self.name.localized())", image: .init(systemName: self.icon)) } } struct MobilityActivityQuery: EntityQuery { func entities(for identifiers: [String]) async throws -> [ActivityEntity] { Activity.all()?.compactMap({ activity in identifiers.contains(where: { $0 == activity.name }) ? ActivityEntity(id: activity.name, name: activity.name, icon: activity.icon) : nil }) ?? [] } func suggestedEntities() async throws -> [ActivityEntity] { Activity.all()?.compactMap({ activity in ActivityEntity(id: activity.name, name: activity.name, icon: activity.icon) }) ?? [] } } Has anyone an idea what might be causing this and how I can fix this behavior? Thanks in advance
3
3
1k
Jun ’24
"Error: Intent of type INStartCallIntent is not supported for this app category"
I am trying to make a voip car play app using siri let assistant = CPAssistantCellConfiguration(position: .top, visibility: .always, assistantAction: .startCall) let siriTmeplate = CPListTemplate(title: "Siri", sections: [sectionItems, loadingSection], assistantCellConfiguration: assistant) siriTmeplate.tabSystemItem = .recents siriTmeplate.showsTabBadge = false Using the above code gives me the error "Error: Intent of type INStartCallIntent is not supported for this app category" on app luanch I have INStartCallIntent in my apps info plist and I have all the entitlements and I have "business" as the app category, I can fine 0 help online with this. what does this error really mean and how can I fix it please
2
0
333
Jun ’24
PTT in the background, cannot activate Siri without unlocking
Hello, We're interested in using the PTT Framework with our PTT capable hardware, as the framework has intended. The problem is activating Siri with any of our specified Intent's doesn't work when the phone is locked. The iPhone always says "You'll have to unlock your iPhone first". Reading up on the problem, it seems pretty common in the fact that Apple doesn't allow Siri Intents to be executed while the phone is locked. It's a sensible precaution by default, but there are countless threads of real use cases that users want to use Siri without unlocking (with PTT, or well, without). There appears to be no options for PTT to enable this, any flags on the Siri Intent to allow benign App actions or queries, user UI configuration through Settings -> Siri & Search to manually allow it even when the phone is locked. Neither are there any entitlements (that I'm aware of) that would allow trivial and non-secure Siri App Intents. The only advice we have for our users (and albeit against the intention of the limitation in the first place) is to: Disable Auto Lock, Disable Face ID and to Disable Passcode. It is in fact 2024, and users do expect a better experience than this with Siri, or am I missing something?
1
0
499
Jun ’24
WatchOS Siri Capability
Hi, I wanted to use Siri Capability for a WatchOS app, however in xcode on a WatchOS project, the option to add Siri is not present. In an IOS project this is visible but if you are not part of the ADP or ADEP you do not have access to it, this message appears in red if you try to select it as a personal team. I am considering paying to join the ADP but I am unsure if it will unlock the ability to use Siri capability on WatchOS. It looks like it is completely unsupported as it cannot be even selected from the capabilities section in xcode , even though Apple states it is supported under ADP and ADEP on their website. I am a little confused. Does anyone else have this issue, or is Siri present under capabilities for you in a WatchOS project?
0
0
325
May ’24
Calling AppIntent Siri Voice Issue
Hello, I have an app with two AppIntents. I invoke first AppIntent with my voice command, then I would like invoke the second AppIntent after the first one. I have implemented the necessary return type for the first AppIntent. My first AppIntent is invoked and executes successfully, however when it calls the second AppIntent, everything inside perform() method in the second AppIntent works but Siri dialog. Siri doesn't say the dialog in the second AppIntent. Below implementation details for my two AppIntents together with AppShortcutsProvider. I appreciate any help. struct MyFirstAppIntent : AppIntent { static let title: LocalizedStringResource = "Show My Usages" func perform() async throws -> some ProvidesDialog & OpensIntent { let dialogStr = IntentDialog(stringLiteral: "You have a package") print("I'm in first AppIntent") return .result(opensIntent: MySecondAppIntent(), dialog: dialogStr) } struct MySecondAppIntent: AppIntent { static let title: LocalizedStringResource = "Show My Usages" func perform() async throws -> some IntentResult & ReturnsValue<String> & ProvidesDialog { print("I'm in second AppIntent") return .result(value: "Listing Packages", dialog: "You have activated Default") } struct MyVoiceShortcutProvider : AppShortcutsProvider { @AppShortcutsBuilder static var appShortcuts: [AppShortcut] { AppShortcut( intent: MyFirstAppIntent(), phrases: ["Call my first intent \(.applicationName)"] ); AppShortcut( intent: MySecondAppIntent(), phrases: ["Call my second intent \(.applicationName)"] ); } }
0
0
306
May ’24
Siri announces GenericHandle Value instead of custom caller Name
Hello fellow developers, I’ve been working with the CallKit framework in iOS, specifically handling incoming calls. One issue I’ve encountered is that Siri when "read caller name" is enabled announces the caller name/surname set via localizedCallerName and then reads the generic handle value (usually alphanumeric) too! Has anyone encountered a similar situation or if there’s a solution to prioritize the localizedCallerName over the generic handle value without using CXHandleType.emailAddress? Alternatively, any insights or workarounds you know would be greatly appreciated. TLDR: even when I correctly configure the localizedCallerName property, Siri persists in reading the CXHandleType.generic. The original Implementation with CXHandleType.generic: The issue arises when using CXHandleType.generic for alphanumeric IDs (or even URLs as stated by documentation https://developer.apple.com/documentation/callkit/cxhandle). Despite correctly setting the localizedCallerName, Siri continues to announce the generic handle value. Expected Behavior: Siri should read only the localizedCallerName when set and ignore the generic handle value when announcing incoming calls. Workaround: Currently, the only workaround is to use CXHandleType.emailAddress for alphanumeric IDs. However, this is not ideal since it repurposes an email-related handle type for a different purpose. Steps to Reproduce: Create a CallKit app that handles incoming calls (example app from documentation can be used too). On incoming call create CXCallUpdate object Create a CXHandle with CXHandleType.generic and an alphanumeric value (e.g., “ABC123”). Pass the CXHandle to the CXCallUpdate objects' remoteHandle. Set the localizedCallerName property of the CXCallUpdate object with a custom caller name/surname. Report the call with reportNewIncomingCallWithUUID Observe that Siri reads both the localizedCallerName and the generic handle value during call announcements. While we are here a Feature Request: Developers should be able to provide a user-friendly caller name without resorting to workarounds like using CXHandleType.emailAddress. I kindly request that Apple consider enhancing Siri’s behavior in the following ways: Allow developers to suppress the reading of generic handle values while still using the correct handle type. Introduce additional type options for call announcements that don't read the generic value. Both of the above. Thank you for your help! 🙌
1
0
277
May ’24
AppShortcutProvider does not launch the shortcut in macOS
I have created an intent using AppIntent and this intent(TWIntent in the code below) can be seen in the shortcuts app as an action to create the shortcut. I am using AppShortcutProvider to create the shortcut so that user can directly make use of it. However, my shortcut does not appear in the shortcut app. The phrases used also do not launch the intent in the spotlight search. Below is my AppShortcutProvider file: @available(macOS 13.0, *) struct TWShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { AppShortcut( intent: TWIntent(), phrases: [ "Open Intent in \(.applicationName)", "Open \(.applicationName) Intent", "Open my Intent in \(.applicationName)", "Open my \(.applicationName) Intent", "Open TWIntent" ], shortTitle: "Open TWIntent", systemImageName: "rectangle.stack.fill" ) } static var shortcutTileColor: ShortcutTileColor = .lightBlue } Is there something I m missing? Also can these phrases be used for siri launch in macOS because the documentation mentions that macOS does not have siri capability? Below is my AppIntent file: import SwiftUI @available(macOS 13, *) struct TWIntent: AppIntent { static let title: LocalizedStringResource = "TWIntent" static var description = IntentDescription("try adding this sample action as your TW shortcut") //launch app on running action static var openAppWhenRun: Bool = false static var parameterSummary: some ParameterSummary { Summary("Get information on \(\.$TWType)") } @Parameter(title: "TWType", description: "The type to get information on.") var TWType: String func perform() async throws -> some IntentResult & ReturnsValue<String> & ProvidesDialog { //perform essential task here to update the application content return .result(value: TWType, dialog: "Logged a 15 minute break.\(TWType)") } }
0
0
302
May ’24
Issue with Siri Intent or App Intent not functioning properly in Speech Framework
Description: Problem Statement: State the problem clearly: The Siri Intent for the "Next","Previous","Repeat" command is not working as expected within the Speech Framework. Steps to Reproduce: Provide a detailed description of the steps to reproduce the issue. For example: Open the Speech Framework application. Tap on the Siri button to activate voice input. Say "Next" to trigger the intended action. Observe that the action is not executed correctly. IN Our Demo App: Steps of my demo application as below: Open SIRI Speak: Check In Response: Open dialog as below: What user wants? One 2) Next 3) Yes 4) Goodbye Speak: Next In Response: SIRI repeat same dialog (Step: 2) 3) Speak: Yes, or One or Goodbye In Response: SIRI goes to next dialog. Expected Behavior: Should be get "Next" Value in siri kit intent or app intent. Actual Behavior: But it give previous user input key word give in siri kit intent and recuresively repeat dialog in app intent. Device versions and Region and Language: Device model: IPhone 11 and OS version: 17.4.1 Region: Us and Language: English(US) Impact: User Cant use Iterative dialog in one context. Additional: How Different command work on app intent and siri kit intent on diffrent diffrent device. you can follow No vise in order. || No || Diffrent Device test on Diffrent sinario || SiriKit intent || app Intent || | 1 | ISG iPhone 11 - Next | Not | Not | | 2 | ISG iPhone 11 - Yes | Not | Yes (But Using Enum) | | 3 | ISG iPhone 11 - GoodBye | Not | Yes (But Using Enum) | | 4 | ISG iPhone 11 - One | Yes | Yes | | 5 | iPad - Next | Not | Not | | 6 | iPad - One | Yes | Yes | | 7 | iPad - GoodBye | Not | Yes | | 8 | iPad - Yes | Not | Yes | | 9 | Simulator - iPhone 15 - Next, Yes, One, GoodBye | Yes | Yes | Please help me in it...
0
0
454
Apr ’24
Detect if user is running Voice Control
Our users are using Apple's native Voice Control feature: https://support.apple.com/en-us/HT210417 We want to enhance our accessibility experience by adding some additional voice controlled dialogs that show up specifically when Voice Control is enabled. It can be determined if other Apple accessibility features are turned on via a check like UIAccessibility.isVoiceOverRunning, however there is no option for Voice Control (note, different than Voice Over). How can I detect if a user is running Voice Control or not?
1
2
766
Mar ’24