Apple Intelligence

RSS for tag

Apple Intelligence is the personal intelligence system that puts powerful generative models right at the core of your iPhone, iPad, and Mac and powers incredible new features to help users communicate, work, and express themselves.

Posts under Apple Intelligence tag

200 Posts

Post

Replies

Boosts

Views

Activity

Image Playground API
Hi, WWDC24 videos have a lot of references to an "Image Playground" API, and the "What's New in AppKit" session even shows it in action, with a "ImagePlaygroundViewController". However, there doesn't seem to be any access to the new API, even with Xcode 16.2 beta. Am I missing something, or is that 'coming later'?
5
1
1.4k
Dec ’24
Image Playground Error: Cannot find protocol declaration for 'ImageGenerationViewControllerDelegate'
@available(macCatalyst 18.1, *) @available(iOS 18.1, *) extension CKImageSelectionManager: ImagePlaygroundViewController.Delegate { public func imagePlaygroundViewController(_ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) { } func presentImagePlayground() { let imagePlaygroundVC = ImagePlaygroundViewController() // Set delegate to self to receive the callback imagePlaygroundVC.delegate = self imagePlaygroundVC.isModalInPresentation = true // Prevents dismissal with swipe if needed self.delegate?.presentImageSelectionViewController(imagePlaygroundVC) } } This generates an error in the xcode generated swift header.
3
0
1.3k
Dec ’24
Crash with UITextView intelligenceCollectContent Apple Intelligence
Hello, I am using Xcode 15.3 and doesn't add any support for Apple Intelligence yet as it requires Xcode 16+. But I see many crashlogs related to Apple Intelligence, I assume crash is related to UITextView element of UIKit. Can you please help me to resolve this issue, how can I fix it? There are few lines of the crashlog, they are all similar. Thread 0 name: Thread 0 Crashed: 0 UIKitCore 0x0000000196692d74 specialized UITextView._intelligenceCollectContent(in:collector:) + 2748 (UITextView_IntelligenceSupport.swift:37) 1 UIKitCore 0x0000000196691eb0 @objc UITextView._intelligenceCollectContent(in:collector:) + 60 (<compiler-generated>:17) 2 UIIntelligenceSupport 0x000000026b91f1e4 UIIntelligenceElementCollector.performCollection(_:) + 424 (UIIntelligenceElementCollector.swift:46) 3 UIKitCore 0x0000000196473b70 specialized UIView._intelligenceElement(in:using:transformToRoot:) + 1440 (UIView_IntelligenceSupport.swift:132) 4 UIKitCore 0x000000019647a800 specialized UIView._intelligenceCollectElement(for:in:using:transformToRoot:) + 424 (UIView_IntelligenceSupport.swift:84) 5 UIKitCore 0x00000001964792ec @objc UIView._intelligenceCollectElement(for:in:using:transformToRoot:) + 136 (<compiler-generated>:77) 6 UIKitCore 0x0000000196472c20 closure #1 in UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 256 (UIView_IntelligenceSupport.swift:55) 7 UIIntelligenceSupport 0x000000026b91f728 UIIntelligenceElementCollector.performElementCollection(_:) + 424 (UIIntelligenceElementCollector.swift:61) v 8 UIKitCore 0x00000001964728f4 UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 928 (UIView_IntelligenceSupport.swift:54) 9 UIKitCore 0x0000000196473354 @objc UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 136 (<compiler-generated>:0) 10 UIKitCore 0x0000000196479810 closure #3 in UIView._intelligenceElement(in:using:transformToRoot:) + 256 (UIView_IntelligenceSupport.swift:165) 11 UIIntelligenceSupport 0x000000026b91fb54 UIIntelligenceElementCollector.performElementArrayCollection(_:) + 144 (UIIntelligenceElementCollector.swift:78) ... ... 225 UIIntelligenceSupport 0x000000026b8ee630 closure #1 in IntelligenceCollectionListener.collectFragments(_:) + 192 (IntelligenceCollectionListener.swift:60) 226 UIIntelligenceSupport 0x000000026b91b138 thunk for @escaping @callee_guaranteed () -> () + 36 227 CoreFoundation 0x000000019376b6e4 __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 28 (CFRunLoop.c:1818) 228 CoreFoundation 0x0000000193759910 __CFRunLoopDoBlocks + 356 (CFRunLoop.c:1860) 229 CoreFoundation 0x00000001937595f4 __CFRunLoopRun + 2432 (CFRunLoop.c:3217) 230 CoreFoundation 0x0000000193758830 CFRunLoopRunSpecific + 588 (CFRunLoop.c:3434) 231 GraphicsServices 0x00000001df7381c4 GSEventRunModal + 164 (GSEvent.c:2196) 232 UIKitCore 0x00000001962beeb0 -[UIApplication _run] + 816 (UIApplication.m:3844)
3
0
879
Dec ’24
Making onscreen content available to Siri not requesting my Transferable
Howdy, I'm following along with this sample: https://developer.apple.com/documentation/appintents/making-onscreen-content-available-to-siri-and-apple-intelligence I've got everything up and building. I can confirm that the userActivity modifier is associating my App Intent via EntityIdentifier but my custom Transferable representation (text) is never being called and when Siri is doing the ChatGPT handoff, it's just offering to send a screenshot which is what it does when it has no custom representation. What could I doing wrong? Where should I be looking?
3
0
1.1k
Dec ’24
iOS 18.2 Heavy Delays in Notifications with AI enabled
I am experiencing heavy delays with any push notification if I have AI enabled. Authenticator takes minutes to show up, the app for the doorbell that, needless to say, requires a real-time notification, fails to deliver push notifications in time. I have had to turn off AI in order to have a functioning phone... (I'm on a 16 Pro) Any resolution on this? Thanks
2
1
959
Dec ’24
Does Apple Intelligence Extensions Have an API?
Hi everyone, On the "Apple Intelligence & Siri" settings there's a section titled "Extensions" that specifically mentions ChatGPT. This got me curious—does Apple provide an API or SDK for developers to create custom integrations or use Apple Intelligence Extensions? Or is this currently limited to the Apple/OpenAI partnership? I appreciate any insights or links to relevant documentation. Here's a screenshot of what I mean: https://imgur.com/a/4MuQkIJ
1
2
911
Dec ’24
Unable to Disable/Hide Writing Tools in iOS 18
Hi everyone, I’m working on an app that uses WKWebView. Due to security concerns, our customers want the app to disable copy-paste functionality and similar options (such as Lookup and Share, allowing users to extract text from the app or save it as a file. I was able to disable copy-paste and remove UIMenu options like Lookup and Share. Everything worked fine until the iOS 18.1 beta, which introduced a new menu called Writing Tools (Apple Intelligence) that has its own copy and share buttons I've tried several ways to remove it Remove all UIMenu items using canPerformAction, and it works for all menus but Writing Tools, which still remains override func canPerformAction(_ action: Selector, withSender sender: Any?) -> Bool { return false } Remove it from UIMenuBuilder override func buildMenu(with builder: UIMenuBuilder) { builder.remove(menu: .lookup) builder.remove(menu: .share) } But unfortunately, UIMenuBuilder does not have an identifier for Writing Tools, and it seems like there are no API changes from Xcode 15 to Xcode 16 (https://developer.apple.com/documentation/uikit/uimenu/identifier) It seems like the only way to disable it through MDM configuration (key: allowWritingTools), but we don't use MDM https://developer.apple.com/documentation/devicemanagement/restrictions Environment iOS Version: iOS 18.1 beta4 Device: iPhone 15 Pro App platform: iOS Xcode version: 16.0 MacOS: 15.0 Thank you
11
0
3.0k
Dec ’24
Regression: Heavy Delay of Push Notifications on iOS 18.1 beta (10s and more)
My app one sec uses push notifications to guide the user back to the app from a Screen Time Shield (screenshot attached). On iOS 18.1, notifications are delivered with a delay of 10+ seconds, even though they are classified as time sensitive: notificationContent.interruptionLevel = .timeSensitive notificationContent.relevanceScore = 1.0 The notification trigger is nil, which according to the documentation should show the notification banner immediately: var notificationTrigger: UNTimeIntervalNotificationTrigger? = nil "The condition that causes the system to deliver the notification. Specify nil to deliver the notification right away." In the sysdiagnose I have noticed that activity related to Apple Intelligence Priority classification delays the notification by 10 seconds ("UserNotificationsCore.IntelligenceActor"): [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.373 sec]: Timeout of 10.0 reached. Cancelling work. [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.377 sec]: Calling out to completion with failure(UserNotificationsCore.StepFailure.timedOut(exceeded: 10.0 seconds, summaryStatus: Optional(UserNotificationsServices.NotificationSummaryStatus.inferenceTimedOut), priorityStatus: Optional(UserNotificationsServices.NotificationPriorityStatus.inferenceTimedOut))) from 'scheduleTimeoutToPerform(after:for:)' [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.378 sec]: Step: UserNotificationsCore.IntelligenceActor, index: 0 exceeded 10.0 seconds This seems like a bug to me, time sensitive notifications should be exempted from being analyzed for priority, especially if that comes at the cost of delaying notifications by 10 seconds. Tracked in Radar: FB15255061
20
6
5.1k
Dec ’24
allowedWritingToolsResultOptions has no effect in iOS 18.1 Writing Tools
The UITextView.allowedWritingToolsResultOptions has no effect to how "Writing Tools" feature works. When it is set to empty, it still offer all options in the Writing Tools popup dialog. The result is that it is not possible to limit output results to eg. only plain text, or disable tables in output. let textView = UITextView() textView.isEditable = true textView.writingToolsBehavior = .complete textView.allowedWritingToolsResultOptions = [] resulting Writing Tools has all options available. I Tested with TextKit1 and TextKit 2 setup. tested on iPadOS 18.1 beta (22B5069a) Report: FB15429824
2
0
920
Nov ’24
Can I integrate the new Visual Intelligence feature into my app?
iOS 18.2 includes a new feature called Visual Intelligence. If I hold down the Camera Control on my iPhone, I can take a photo of an object and use Google to look up items similar to what I've photographed. Is there a way to programmatically open this interface within my app? If so, can I see which result the user selects?
Replies
1
Boosts
1
Views
1.2k
Activity
Jan ’25
Apple Intelligence Not Responding
i have iPhone 16 pro and I’ve updated to IOS 18.2. however after I clicked on ‘Get Apple Intelligence’, and then set up button, nothing happened. I’ve tried this process many times still nothing working.
Replies
2
Boosts
2
Views
642
Activity
Jan ’25
Where is Swift Assist?
Why is Apple not releasing Swift Assist?
Replies
1
Boosts
2
Views
789
Activity
Dec ’24
Image Playground API
Hi, WWDC24 videos have a lot of references to an "Image Playground" API, and the "What's New in AppKit" session even shows it in action, with a "ImagePlaygroundViewController". However, there doesn't seem to be any access to the new API, even with Xcode 16.2 beta. Am I missing something, or is that 'coming later'?
Replies
5
Boosts
1
Views
1.4k
Activity
Dec ’24
Image Playground Error: Cannot find protocol declaration for 'ImageGenerationViewControllerDelegate'
@available(macCatalyst 18.1, *) @available(iOS 18.1, *) extension CKImageSelectionManager: ImagePlaygroundViewController.Delegate { public func imagePlaygroundViewController(_ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) { } func presentImagePlayground() { let imagePlaygroundVC = ImagePlaygroundViewController() // Set delegate to self to receive the callback imagePlaygroundVC.delegate = self imagePlaygroundVC.isModalInPresentation = true // Prevents dismissal with swipe if needed self.delegate?.presentImageSelectionViewController(imagePlaygroundVC) } } This generates an error in the xcode generated swift header.
Replies
3
Boosts
0
Views
1.3k
Activity
Dec ’24
Crash with UITextView intelligenceCollectContent Apple Intelligence
Hello, I am using Xcode 15.3 and doesn't add any support for Apple Intelligence yet as it requires Xcode 16+. But I see many crashlogs related to Apple Intelligence, I assume crash is related to UITextView element of UIKit. Can you please help me to resolve this issue, how can I fix it? There are few lines of the crashlog, they are all similar. Thread 0 name: Thread 0 Crashed: 0 UIKitCore 0x0000000196692d74 specialized UITextView._intelligenceCollectContent(in:collector:) + 2748 (UITextView_IntelligenceSupport.swift:37) 1 UIKitCore 0x0000000196691eb0 @objc UITextView._intelligenceCollectContent(in:collector:) + 60 (<compiler-generated>:17) 2 UIIntelligenceSupport 0x000000026b91f1e4 UIIntelligenceElementCollector.performCollection(_:) + 424 (UIIntelligenceElementCollector.swift:46) 3 UIKitCore 0x0000000196473b70 specialized UIView._intelligenceElement(in:using:transformToRoot:) + 1440 (UIView_IntelligenceSupport.swift:132) 4 UIKitCore 0x000000019647a800 specialized UIView._intelligenceCollectElement(for:in:using:transformToRoot:) + 424 (UIView_IntelligenceSupport.swift:84) 5 UIKitCore 0x00000001964792ec @objc UIView._intelligenceCollectElement(for:in:using:transformToRoot:) + 136 (<compiler-generated>:77) 6 UIKitCore 0x0000000196472c20 closure #1 in UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 256 (UIView_IntelligenceSupport.swift:55) 7 UIIntelligenceSupport 0x000000026b91f728 UIIntelligenceElementCollector.performElementCollection(_:) + 424 (UIIntelligenceElementCollector.swift:61) v 8 UIKitCore 0x00000001964728f4 UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 928 (UIView_IntelligenceSupport.swift:54) 9 UIKitCore 0x0000000196473354 @objc UIView._intelligenceCollectSubelements(in:using:transformToRoot:) + 136 (<compiler-generated>:0) 10 UIKitCore 0x0000000196479810 closure #3 in UIView._intelligenceElement(in:using:transformToRoot:) + 256 (UIView_IntelligenceSupport.swift:165) 11 UIIntelligenceSupport 0x000000026b91fb54 UIIntelligenceElementCollector.performElementArrayCollection(_:) + 144 (UIIntelligenceElementCollector.swift:78) ... ... 225 UIIntelligenceSupport 0x000000026b8ee630 closure #1 in IntelligenceCollectionListener.collectFragments(_:) + 192 (IntelligenceCollectionListener.swift:60) 226 UIIntelligenceSupport 0x000000026b91b138 thunk for @escaping @callee_guaranteed () -> () + 36 227 CoreFoundation 0x000000019376b6e4 __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 28 (CFRunLoop.c:1818) 228 CoreFoundation 0x0000000193759910 __CFRunLoopDoBlocks + 356 (CFRunLoop.c:1860) 229 CoreFoundation 0x00000001937595f4 __CFRunLoopRun + 2432 (CFRunLoop.c:3217) 230 CoreFoundation 0x0000000193758830 CFRunLoopRunSpecific + 588 (CFRunLoop.c:3434) 231 GraphicsServices 0x00000001df7381c4 GSEventRunModal + 164 (GSEvent.c:2196) 232 UIKitCore 0x00000001962beeb0 -[UIApplication _run] + 816 (UIApplication.m:3844)
Replies
3
Boosts
0
Views
879
Activity
Dec ’24
Making onscreen content available to Siri not requesting my Transferable
Howdy, I'm following along with this sample: https://developer.apple.com/documentation/appintents/making-onscreen-content-available-to-siri-and-apple-intelligence I've got everything up and building. I can confirm that the userActivity modifier is associating my App Intent via EntityIdentifier but my custom Transferable representation (text) is never being called and when Siri is doing the ChatGPT handoff, it's just offering to send a screenshot which is what it does when it has no custom representation. What could I doing wrong? Where should I be looking?
Replies
3
Boosts
0
Views
1.1k
Activity
Dec ’24
iOS 18.2 Heavy Delays in Notifications with AI enabled
I am experiencing heavy delays with any push notification if I have AI enabled. Authenticator takes minutes to show up, the app for the doorbell that, needless to say, requires a real-time notification, fails to deliver push notifications in time. I have had to turn off AI in order to have a functioning phone... (I'm on a 16 Pro) Any resolution on this? Thanks
Replies
2
Boosts
1
Views
959
Activity
Dec ’24
Does Apple Intelligence Extensions Have an API?
Hi everyone, On the "Apple Intelligence & Siri" settings there's a section titled "Extensions" that specifically mentions ChatGPT. This got me curious—does Apple provide an API or SDK for developers to create custom integrations or use Apple Intelligence Extensions? Or is this currently limited to the Apple/OpenAI partnership? I appreciate any insights or links to relevant documentation. Here's a screenshot of what I mean: https://imgur.com/a/4MuQkIJ
Replies
1
Boosts
2
Views
911
Activity
Dec ’24
Apple Intelligence
Alguem me pode indicar se os devolopers que estão no espaço da união europeia, possam aceder aos serviços apple intelligence ? Obrigado
Replies
1
Boosts
0
Views
962
Activity
Dec ’24
Unable to Disable/Hide Writing Tools in iOS 18
Hi everyone, I’m working on an app that uses WKWebView. Due to security concerns, our customers want the app to disable copy-paste functionality and similar options (such as Lookup and Share, allowing users to extract text from the app or save it as a file. I was able to disable copy-paste and remove UIMenu options like Lookup and Share. Everything worked fine until the iOS 18.1 beta, which introduced a new menu called Writing Tools (Apple Intelligence) that has its own copy and share buttons I've tried several ways to remove it Remove all UIMenu items using canPerformAction, and it works for all menus but Writing Tools, which still remains override func canPerformAction(_ action: Selector, withSender sender: Any?) -> Bool { return false } Remove it from UIMenuBuilder override func buildMenu(with builder: UIMenuBuilder) { builder.remove(menu: .lookup) builder.remove(menu: .share) } But unfortunately, UIMenuBuilder does not have an identifier for Writing Tools, and it seems like there are no API changes from Xcode 15 to Xcode 16 (https://developer.apple.com/documentation/uikit/uimenu/identifier) It seems like the only way to disable it through MDM configuration (key: allowWritingTools), but we don't use MDM https://developer.apple.com/documentation/devicemanagement/restrictions Environment iOS Version: iOS 18.1 beta4 Device: iPhone 15 Pro App platform: iOS Xcode version: 16.0 MacOS: 15.0 Thank you
Replies
11
Boosts
0
Views
3.0k
Activity
Dec ’24
Apple Intelligence "Personality Discovery"
Hey Chat, I'm researching personality analysis using LLMs, and I'm curious about whether Apple’s AI can be allowed access to your messages, Instagram DMs, and similar communications to perform a personality analysis based on your writing style. If anyone has insights on this, I would greatly appreciate your input. Thx a ton
Replies
0
Boosts
0
Views
646
Activity
Dec ’24
Regression: Heavy Delay of Push Notifications on iOS 18.1 beta (10s and more)
My app one sec uses push notifications to guide the user back to the app from a Screen Time Shield (screenshot attached). On iOS 18.1, notifications are delivered with a delay of 10+ seconds, even though they are classified as time sensitive: notificationContent.interruptionLevel = .timeSensitive notificationContent.relevanceScore = 1.0 The notification trigger is nil, which according to the documentation should show the notification banner immediately: var notificationTrigger: UNTimeIntervalNotificationTrigger? = nil "The condition that causes the system to deliver the notification. Specify nil to deliver the notification right away." In the sysdiagnose I have noticed that activity related to Apple Intelligence Priority classification delays the notification by 10 seconds ("UserNotificationsCore.IntelligenceActor"): [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.373 sec]: Timeout of 10.0 reached. Cancelling work. [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.377 sec]: Calling out to completion with failure(UserNotificationsCore.StepFailure.timedOut(exceeded: 10.0 seconds, summaryStatus: Optional(UserNotificationsServices.NotificationSummaryStatus.inferenceTimedOut), priorityStatus: Optional(UserNotificationsServices.NotificationPriorityStatus.inferenceTimedOut))) from 'scheduleTimeoutToPerform(after:for:)' [create, [id=43C0-B333, time=2024-09-27 06:03:26, bundle=***.riedel.one-sec], Time elapsed=10.378 sec]: Step: UserNotificationsCore.IntelligenceActor, index: 0 exceeded 10.0 seconds This seems like a bug to me, time sensitive notifications should be exempted from being analyzed for priority, especially if that comes at the cost of delaying notifications by 10 seconds. Tracked in Radar: FB15255061
Replies
20
Boosts
6
Views
5.1k
Activity
Dec ’24
Apple Intelligence Missing on iPhone 16 Pro with iOS 18.2 Beta – What’s the Fix?
I just got my new iPhone 16 Pro and upgraded to the 18.2 developer beta 4. I've set both Siri and the device language to English (United States), but the Apple Intelligence feature still doesn’t appear in my settings.
Replies
1
Boosts
2
Views
810
Activity
Dec ’24
allowedWritingToolsResultOptions has no effect in iOS 18.1 Writing Tools
The UITextView.allowedWritingToolsResultOptions has no effect to how "Writing Tools" feature works. When it is set to empty, it still offer all options in the Writing Tools popup dialog. The result is that it is not possible to limit output results to eg. only plain text, or disable tables in output. let textView = UITextView() textView.isEditable = true textView.writingToolsBehavior = .complete textView.allowedWritingToolsResultOptions = [] resulting Writing Tools has all options available. I Tested with TextKit1 and TextKit 2 setup. tested on iPadOS 18.1 beta (22B5069a) Report: FB15429824
Replies
2
Boosts
0
Views
920
Activity
Nov ’24
Apple intelligence stop working after 18.2 beta update
After installing IOS 18.2 public beta, the Apple intelligence been stuck on downloading and now I’m Back to the old Siri. Which kinda sucks. lol Any help or tips to get my APPLE INTELLIGENCE back?
Replies
12
Boosts
7
Views
4.3k
Activity
Nov ’24
Image Playground stuck at Early Access requested
Hi I got access to developer beta 18.2 . All the features are working fine but image Playground stuck at Early Access requested. Can someone share how much time it takes before you are able to access image playground? I have been stuck for 3 hours
Replies
241
Boosts
110
Views
69k
Activity
Nov ’24
Please add support for Apple Intelligence features on the simulator
The simulator should support the new Siri Ul and Apple Intelligence features, at least they should work if Apple Intelligence is enabled on the Mac itself. Feedback ID: FB15699827
Replies
0
Boosts
2
Views
741
Activity
Nov ’24
Apple Intelligence is not downloaded?
When i’m update to iOS 18.2 beta 1, Apple intelligence is downloading still now! “Apple Intelligence is downloading. Some Apple Intelligence and Siri features may be unavailable until the download is complete” How can i solve this problem?
Replies
3
Boosts
2
Views
1.4k
Activity
Nov ’24
Playground image creation for no result
I have been to image creation for 2 hours. I can do genmoji, playground well for 10 minutes. After that, when I run the playground and start to generate Animation style of photo, it will stuck in here and no result output. Is there anyone know what's happened...
Replies
1
Boosts
0
Views
1.1k
Activity
Nov ’24