App Intents

RSS for tag

Extend your app’s custom functionality to support system-level services, like Siri and the Shortcuts app.

Posts under App Intents tag

177 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Migrating custom intents to App Intents
Hi, I am trying to migrate my custom intents to App Intents, and was running into some issues. My current intentdefinitions file and all intent handling code are in a framework that is shared with my app target. I went through the migration assistant and added the App Intents codes directly to my main app target. When I run a shortcut with the App Intent, it doesn't work ... I get some messages in the console that say: Could not find an intent with identifier MyCustomAddContactIntent, mangledTypeName: Optional("") I guess the old custom intents and new App Intents should both live in the same package to see each other. In this case, I'm not sure if all the existing custom intents file and all the intents handler logic should be moved into the main app bundle (and removed from framework), or should I add the new App Intents handlers into the framework (in addition to the main app)? Also, will the custom framework even be needed or run in iOS16+? Thanks.
0
0
259
Jun ’24
"accelerate everyday tasks" in apps without intents?
From https://www.apple.com/newsroom/2024/06/introducing-apple-intelligence-for-iphone-ipad-and-mac/: Powered by Apple Intelligence, Siri becomes more deeply integrated into the system experience. With richer language-understanding capabilities, Siri is more natural, more contextually relevant, and more personal, with the ability to simplify and accelerate everyday tasks. From https://developer.apple.com/apple-intelligence/: Siri is more natural, more personal, and more deeply integrated into the system. Apple Intelligence provides Siri with enhanced action capabilities, and developers can take advantage of pre-defined and pre-trained App Intents across a range of domains to not only give Siri the ability to take actions in your app, but to make your app’s actions more discoverable in places like Spotlight, the Shortcuts app, Control Center, and more. SiriKit adopters will benefit from Siri’s enhanced conversational capabilities with no additional work. And with App Entities, Siri can understand content from your app and provide users with information from your app from anywhere in the system. Based on this, as well as the video at https://developer.apple.com/videos/play/wwdc2024/10133/ , my understanding is that in order for Siri to be able to execute tasks in applications, those applications must implement the Siri Intents API. Can someone at Apple please clarify: will it be possible for Siri or some other aspect of Apple Intelligence / Core ML / Create ML to take actions in applications which do not support these APIs (e.g. web apps, Citrix apps, legacy apps)? Thank you!
1
1
413
Jun ’24
PTT in the background, cannot activate Siri without unlocking
Hello, We're interested in using the PTT Framework with our PTT capable hardware, as the framework has intended. The problem is activating Siri with any of our specified Intent's doesn't work when the phone is locked. The iPhone always says "You'll have to unlock your iPhone first". Reading up on the problem, it seems pretty common in the fact that Apple doesn't allow Siri Intents to be executed while the phone is locked. It's a sensible precaution by default, but there are countless threads of real use cases that users want to use Siri without unlocking (with PTT, or well, without). There appears to be no options for PTT to enable this, any flags on the Siri Intent to allow benign App actions or queries, user UI configuration through Settings -> Siri & Search to manually allow it even when the phone is locked. Neither are there any entitlements (that I'm aware of) that would allow trivial and non-secure Siri App Intents. The only advice we have for our users (and albeit against the intention of the limitation in the first place) is to: Disable Auto Lock, Disable Face ID and to Disable Passcode. It is in fact 2024, and users do expect a better experience than this with Siri, or am I missing something?
1
0
499
Jun ’24
AppIntents don't work on iOS 16 if app is built with Xcode 15.3
Hi, After upgrading to Xcode 15.3 (15E204a) AppIntents unexpectedly stopped working on iOS 16. System error occurs when you try to run AppIntent on device with iOS 16, Shortcuts app just shows error alert. There are errors in device's logs: error Could not find an intent with identifier , mangledTypeName: Optional("") It seems to be an issue with extract.actionsdata file, generated by appintentsmetadataprocessor. "mangledTypeName" parameter is empty for action descriptors, though it was fulfilled when building with Xcode 14.2 and different version of generator. If I manually set this property in xcarchive and sign it everything works fine. The issue is reproducible with Xcode 15.4 which is currently in beta. Have anyone experienced this or have any solution more solid than editing metadata files after build?
10
3
704
May ’24
Siri permissions being ignored
As described on http://www.openradar.me/45024092 I am facing the same scenario as described on the comment below: As per API the result of the call to https://developer.apple.com/documentation/sirikit/intent_handling_infrastructure/requesting_authorization_to_use_siri INPreferences.requestSiriAuthorization({status in resolve(status.rawValue) }) should be recalled and block execution when it's "denied" but it seems to be ignored. and the feature can be accessed (shortcuts being created etc) Cautious developers would check for the value of the INSiriAuthorizationStatus and block feature access based on the result but that would instead lead to a deadlock as no toggles on Settings-> App -> Siri would trigger a new user prompt for authorisation (as suggested by docs). As per attached screenshot all toggles are enabled besides the user denied the Siri permission request. No documented changes on API I could find on the iOS releases notes > 11.0 Is there a gap on the documentation or is it a bug?
0
0
168
May ’24
iOS Automation could not run because an internal error occurred
I am developing a shortcut for an application that is currently in production. The shortcut essentially involves opening the application and launching a notification so that the AppDelegate initiates a process to scan NFC tags. To achieve this, I have an AppIntent that overrides the variable openAppWhenRun = true and an AppShortcutsProvider to implement this intent. The problem arises when a user updates to the latest version of the application and tries to implement this shortcut through an automation. The following error appears: "When 'Reader' is detected" encountered an error: The action 'Scan DMA tags' could not run because an internal error occurred. This does not always happen, only on some devices. However, if we uninstall and reinstall the application, it works perfectly. But this is not a viable solution since the application uses a database and data loss from frequent uninstallations is not acceptable. Any solution? I have tried to replicate the error but have been unable to do so. This issue has occurred on both iOS 16 and iOS 17.
0
1
305
May ’24
Why I need to press an interactive button two times in a Widget to trigger the AppIntention?
I have a basic Widget with a button to toggle the home lights, the buttons triggers the following AppIntention: import WidgetKit import AppIntents struct ConfigurationAppIntent: WidgetConfigurationIntent { static var title: LocalizedStringResource = "Bulb state" static var description = IntentDescription("This is an example widget.") } struct ToggleStateIntent: AppIntent { static var title: LocalizedStringResource = "Toggle light state" init(){ } func perform() async throws -> some IntentResult { await WizClient.shared.toggleState() return .result() } } The problem is that I must be running the app with xcode (in my phone, not simulator) to work fine, when I stop xcode the button must be pressed two times to trigger the AppIntention. The toggle function works well on the app with a toggle component. Here is the widget: import WidgetKit import SwiftUI struct Provider: AppIntentTimelineProvider { func placeholder(in context: Context) -> SimpleEntry { SimpleEntry(date: Date(), configuration: ConfigurationAppIntent()) } func snapshot(for configuration: ConfigurationAppIntent, in context: Context) async -> SimpleEntry { SimpleEntry(date: Date(), configuration: configuration) } func timeline(for configuration: ConfigurationAppIntent, in context: Context) async -> Timeline<SimpleEntry> { let timeline = Timeline(entries: [SimpleEntry(date: Date(), configuration: configuration)], policy: .atEnd) return timeline } } struct SimpleEntry: TimelineEntry { let date: Date let configuration: ConfigurationAppIntent } struct BulbActionsEntryView : View { var entry: Provider.Entry var body: some View { HStack { Button(intent: ToggleStateIntent()){ Text("Toggle") } } .padding(.vertical) } } struct BulbActions: Widget { let kind: String = "BulbActions" var body: some WidgetConfiguration { AppIntentConfiguration(kind: kind, intent: ConfigurationAppIntent.self, provider: Provider()) { entry in BulbActionsEntryView(entry: entry) .containerBackground(.fill.tertiary, for: .widget) } } } extension ConfigurationAppIntent { fileprivate static var test: ConfigurationAppIntent { let intent = ConfigurationAppIntent() print("Intent -> \(intent)") return intent } } #Preview(as: .systemSmall) { BulbActions() } timeline: { SimpleEntry(date: .now, configuration: .test) }
0
0
297
May ’24
'openAppWhenRun' property causing AppIntentsExtension to fail
I have added an "App Intents Extension" target to my main application in macOS. This generated the below two files: TWAppIntent.swift import AppIntents struct TWAppIntent: AppIntent { static var title: LocalizedStringResource = "TWAppIntentExtension" static var parameterSummary: some ParameterSummary { Summary("Get information on \(\.$TWType)") } //launch app on running action static var openAppWhenRun: Bool = true // we can have multiple parameter of diff types @Parameter(title: "TWType") var TWType: String func perform() async throws -> some IntentResult & ReturnsValue<String> & ProvidesDialog { return .result(value: TWType, dialog: "Logged break.") } } TWAppIntentExtension.swift import AppIntents @main struct TWAppIntentExtension: AppIntentsExtension { } I m able to build the extension target and I my intent action is available in the shortcuts app. However, on launching a shortcut with the above created intent action. I m getting the below popups: I have identified what is causing this error. Setting the openAppWhenRun to true is causing this error. I don't get this when it is set to false. This property is supposed to launch the application, but can someone help me understand why is it happening? This is only causing the error when using this property for AppIntent Extension and not for In app handling for the AppIntent. Can we not launch our application from AppIntent extension?
1
2
307
May ’24
Calling AppIntent Siri Voice Issue
Hello, I have an app with two AppIntents. I invoke first AppIntent with my voice command, then I would like invoke the second AppIntent after the first one. I have implemented the necessary return type for the first AppIntent. My first AppIntent is invoked and executes successfully, however when it calls the second AppIntent, everything inside perform() method in the second AppIntent works but Siri dialog. Siri doesn't say the dialog in the second AppIntent. Below implementation details for my two AppIntents together with AppShortcutsProvider. I appreciate any help. struct MyFirstAppIntent : AppIntent { static let title: LocalizedStringResource = "Show My Usages" func perform() async throws -> some ProvidesDialog & OpensIntent { let dialogStr = IntentDialog(stringLiteral: "You have a package") print("I'm in first AppIntent") return .result(opensIntent: MySecondAppIntent(), dialog: dialogStr) } struct MySecondAppIntent: AppIntent { static let title: LocalizedStringResource = "Show My Usages" func perform() async throws -> some IntentResult & ReturnsValue<String> & ProvidesDialog { print("I'm in second AppIntent") return .result(value: "Listing Packages", dialog: "You have activated Default") } struct MyVoiceShortcutProvider : AppShortcutsProvider { @AppShortcutsBuilder static var appShortcuts: [AppShortcut] { AppShortcut( intent: MyFirstAppIntent(), phrases: ["Call my first intent \(.applicationName)"] ); AppShortcut( intent: MySecondAppIntent(), phrases: ["Call my second intent \(.applicationName)"] ); } }
0
0
306
May ’24
AppShortcutProvider does not launch the shortcut in macOS
I have created an intent using AppIntent and this intent(TWIntent in the code below) can be seen in the shortcuts app as an action to create the shortcut. I am using AppShortcutProvider to create the shortcut so that user can directly make use of it. However, my shortcut does not appear in the shortcut app. The phrases used also do not launch the intent in the spotlight search. Below is my AppShortcutProvider file: @available(macOS 13.0, *) struct TWShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { AppShortcut( intent: TWIntent(), phrases: [ "Open Intent in \(.applicationName)", "Open \(.applicationName) Intent", "Open my Intent in \(.applicationName)", "Open my \(.applicationName) Intent", "Open TWIntent" ], shortTitle: "Open TWIntent", systemImageName: "rectangle.stack.fill" ) } static var shortcutTileColor: ShortcutTileColor = .lightBlue } Is there something I m missing? Also can these phrases be used for siri launch in macOS because the documentation mentions that macOS does not have siri capability? Below is my AppIntent file: import SwiftUI @available(macOS 13, *) struct TWIntent: AppIntent { static let title: LocalizedStringResource = "TWIntent" static var description = IntentDescription("try adding this sample action as your TW shortcut") //launch app on running action static var openAppWhenRun: Bool = false static var parameterSummary: some ParameterSummary { Summary("Get information on \(\.$TWType)") } @Parameter(title: "TWType", description: "The type to get information on.") var TWType: String func perform() async throws -> some IntentResult & ReturnsValue<String> & ProvidesDialog { //perform essential task here to update the application content return .result(value: TWType, dialog: "Logged a 15 minute break.\(TWType)") } }
0
0
302
May ’24
Shortcut doesn't open deep link to the app it belongs to, but widget does
I've created widget that opens app on specific view: .widgetUrl(URL(string: "myapp://Dashboard/SpecialView")) It works alright so i know deep links are working. So after that i created Shortcuts.swift file in my main app and added this: import AppIntents import UIKit @available(iOS 16, *) struct StartRecordingIntent : AppIntent { static var title: LocalizedStringResource = "Special" func perform() async throws -> some IntentResult { let url = URL(string: "myapp://Dashboard/SpecialView") DispatchQueue.main.async { UIApplication.shared.open(url!, options: [:], completionHandler: nil) } return .result() } } This intent created shortcut "Special" in shortcut list, and when run, should open exactly the same view as the widget. But all im getting is error: Failed to open URL myapp://Dashboard/SpecialView: Error Domain=FBSOpenApplicationServiceErrorDomain Code=1 "The request to open "com.myapp" failed." UserInfo={BSErrorCodeDescription=RequestDenied, NSUnderlyingError=0x600000dd7960 {Error Domain=FBSOpenApplicationErrorDomain Code=3 "Application com.myapp is neither visible nor entitled, so may not perform un-trusted user actions." I don't understand what i'm doing wrong and why can't i open app from shortcut exactly like widget does
1
1
380
May ’24
How to contribute to Journaling Suggestions?
Apple's new Journal app was introduced with the iOS 17.2 beta. In the release notes, the following is mentioned: If your app donates activities or interactions to SiriKit or CallKit or if someone authorizes your app to save data to HealthKit, some data might show up as part of Journaling Suggestions. Is there any documentation on how this works exactly? What kind of activities can be featured in Journal? How does the system decide what to feature? For instance, if I have an app that allows the user to create art images, can I somehow make those images appear in the Journaling Suggestions?
1
7
798
Apr ’24
Passing arbitrary strings to AppIntent
I'm using the TrailsSampleApp (in the AppIntentsSampleApp project which is provided by Apple. According to the code, if you use the "Get conditions with TrailsSampleApp" and provide a trail that is NOT in the suggested entities, it should pass control to the following TrailEntityQuery extension. /// An EntityStringQuery extends the capability of an EntityQuery by allowing people to search for an entity with a string. extension TrailEntityQuery: EntityStringQuery { /** To see this method, configure the Get Trail Info intent in the Shortcuts app. A list displays the suggested entities. If you search for an entity not in the suggested entities list, the system passes the search string to this method. - Tag: string_query */ func entities(matching string: String) async throws -> [TrailEntity] { Logger.entityQueryLogging.debug("[TrailEntityQuery] String query for term \(string)") return trailManager.trails { trail in trail.name.localizedCaseInsensitiveContains(string) }.map { TrailEntity(trail: $0) } } } It says if you configure the Get Trail Info section in the Shortcuts App, control will be passed to this entities() function. However, it doesn't say how to configure it. Any suggestions or help?
0
0
251
Apr ’24
suggestedEntities not being called
I'm adding AppIntents to my existing "ShopList" app and I'm having two problems that I think are related. I can't get shortcuts (or Siri) to work when I pass a parameter to an intent. It works if I don't pass a parameter though. My StoreEntityQuery.suggestedEntities function is not being called during initialization. I have an AppIntentsShortcutProvider defined as follows. class ShopListShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { AppShortcut(intent: FindStore(), phrases: [ "Find \(\.$store) store in \(.applicationName)" ], shortTitle: "Find Store", systemImageName: "cart") } } Details below. I would appreciate any suggestions on what I'm missing or doing wrong. In my AppDelegate "didFinishLaunchingWithOptions" function, I call ShopListShortcuts.updateShortcutParameters. The FindStore struct is defined as follows. struct FindStore: AppIntent { static var title: LocalizedStringResource = "Find Store" static var description = IntentDescription("Finds a store.") static var openAppWhenRun: Bool = false @Parameter(title: "Store", description: "The store.") var store: StoreEntity @MainActor func perform() async throws -> some IntentResult { print("performing intent with \(store.id) and \(store.name)") return .result() } } The StoreEntity is defined as follows. struct StoreEntity: AppEntity { ... static var defaultQuery = StoreEntityQuery() var id: Store.ID @Property(title: "Store Name") var name: String ... } And finally the StoreEntityQuery is defined as follows. struct StoreEntityQuery: EntityQuery { func entities(for identifiers: [StoreEntity.ID]) async throws -> [StoreEntity] { print("entities call") return ... } func suggestedEntities(for identifiers: [StoreEntity.ID]) async throws -> [StoreEntity] { print("suggested entities call") return ... } }
1
0
313
Apr ’24
ShareExtension suggestions loosing avatars after a while
Hello, I am developing a Chat application, and I was able to successfully implement INSendMessageIntent to be able to have suggestions in Share Extensions with some of my contacts. I've noticed that after a day or two, avatars are lost, and 2 tested devices are displaying empty background. When I use my app to send a message, this new chat appears with avatar properly, but just after some time it goes away. I am using: let avatarImage = INImage(url: avatarUrl) intent.setImage(avatarImage, forParameterNamed: \.speakableGroupName) where avatarUrl can be a file created in NSTemporaryDirectory or just a file in my shared AppGroup directory. As you can see, Telegram suggestions work fine, suggestions from my app works well as well when I donate the intent, but after some time avatars just goes away Does apple perform some avatar caching, so when I donate the intent avatars are stored for Share sheet somewhere? Or they always use my local url and I need to make sure it's not deleted?
1
0
386
Apr ’24
Appintent and swiftdata
Hi, I'm new to Swift development and encountering an issue that I believe may be due to an error on my part. I have two questions regarding the following code: import AppIntents import SwiftUI import SwiftData @available(iOS 17.0, *) struct CopiarEventoIntent: AppIntent { @Environment(\.modelContext) private var context @Parameter(title: "Nome do Evento") var name: String @Parameter(title: "Data Inicial") var datai: Date @Parameter(title: "Data Final") var dataf: Date @Parameter(title: "Tipo de Evento") var tipo: String @Parameter(title: "Endereço") var endereco: String @Parameter(title: "Lembrete") var reminder: Bool static var title: LocalizedStringResource = "Adicionar Eventos" static var description = IntentDescription("Copiar Eventos e alterar datas", resultValueName: "Resultado") @MainActor func perform() async throws -> some IntentResult & ProvidesDialog { let calData = CalData(title: name, datei: datai, datef: dataf, tipo: tipo, endereco: endereco,reminder: reminder) context.insert(calData) return .result(dialog: "Evento copiado com sucesso!") } } @available(iOS 15.0, *) struct AppShortcuts: AppShortcutsProvider { static var appShortcuts: [AppShortcut] { AppShortcut( intent: CopiarEventoIntent(), phrases: [ "Copiar Evento" ], shortTitle: "Copiar Evento", systemImageName: "square.and.arrow.down" ) } } @available(iOS 15.0, *) struct ShortcutSelectionView: View { @Environment(\.modelContext) private var context @Query(sort: \CalData.title) private var caldatas: [CalData] @State private var selectedEvent: CalData? var body: some View { List(caldatas, id: \.self) { eventData in Button(action: { selectedEvent = eventData handleEventSelection() }) { Text(eventData.title) } } } private func handleEventSelection() { guard selectedEvent != nil else { return } } } When I run the shortcut, gives me the following error: Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Cannot insert 'CalData' in this managed object context because it is not found in the associated managed object model. I attempted to save this using SwiftData with the following model: import SwiftData @Model class CalData { var title : String var datei : Date var datef : Date var tipo : String var endereco : String @Relationship(deleteRule: .cascade) var caldataval = [CalDataVal]() var reminder: Bool = false init(title:String, datei:Date, datef:Date, tipo:String, endereco:String, reminder:Bool){ self.title = title self.datei = datei self.datef = datef self.tipo = tipo self.endereco = endereco self.reminder = reminder } } As for the second question, how can I add a parameter that can have multiple values to save them to EventDataVal, resembling a one-to-many relationship: import SwiftData @Model class CalDataVal { var name : String var value : Double init(name:String, value:Double){ self.name = name self.value = value } } Thanks in advanced
0
0
384
Apr ’24