Apple Intelligence

RSS for tag

Apple Intelligence is the personal intelligence system that puts powerful generative models right at the core of your iPhone, iPad, and Mac and powers incredible new features to help users communicate, work, and express themselves.

Posts under Apple Intelligence subtopic

Post

Replies

Boosts

Views

Activity

Image Playground not available for "Designed for iPad" apps?
I'm currently trying to add support for Image Playground to our apps. It seems that it's not working in an app that is "Designed for iPad" and runs on a Mac. The modal just shows a spinner and the following is logged to console: Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> GP extension could not be loaded: Extension (platform: 2) could not be found (in update) dealloc Query controller [C32BA176-6A3E-465D-B3C5-0F8D91068B89] ImagePlaygroundViewController.isAvailable returns true, however. In a "real" Mac Catalyst app, it's working. Just not when the app is actually an iPad app. Is this a bug?
4
2
1.4k
Jan ’25
Will Apple Intelligence Support Third-Party LLMs or Custom AI Agent Integrations?
Hi everyone, I’m an AI engineer working on autonomous AI agents and exploring ways to integrate them into the Apple ecosystem, especially via Siri and Apple Intelligence. I was impressed by Apple’s integration of ChatGPT and its privacy-first design, but I’m curious to know: • Are there plans to support third-party LLMs? • Could Siri or Apple Intelligence call external AI agents or allow extensions to plug in alternative models for reasoning, scheduling, or proactive suggestions? I’m particularly interested in building event-driven, voice-triggered workflows where Apple Intelligence could act as a front-end for more complex autonomous systems (possibly local or cloud-based). This kind of extensibility would open up incredible opportunities for personalized, privacy-friendly use cases — while aligning with Apple’s system architecture. Is anything like this on the roadmap? Or is there a suggested way to prototype such integrations today? Thanks in advance for any thoughts or pointers!
4
0
419
May ’25
Missing parameter prompt for search Assistant Intents
Issue When triggering an App Intent using assistant schemas from Apple Intelligence (voice or text) the App opens without prompting for search criteria. How to repeat This can be repeated in the example provided by Apple here: https://developer.apple.com/documentation/appintents/making-your-app-s-functionality-available-to-siri Download the sample code Build and run on Xcode 16.1 beta 3 Target iPhone 15 Pro Max on iOS 18.1 beta 7 Trigger Apple Intelligence Enter prompt: "Search AssistantSchemasExample" Expected behaviour Apple Intelligence should prompt the user for the criteria and provide this to the App so that the experience is seamless for the end-user. Otherwise Assistant Intents are nothing more than deep links to search screens. Notes The example uses @AssistantIntent(schema: .photos.search) intent. And I've found the issue is also present in other search intents: @AssistantIntent(schema: .system.search) @AssistantIntent(schema: .browser.search) Questions Has anyone managed to get the prompt to appear? Will this only function on iOS 18.2?
3
1
538
Oct ’24
Almost 48 Hours, Still No Access to Playground.
Just wanted to reach out to see if this is the norm. I see several posts saying people are still waiting for the early access playground app, what’s going on? it’s been almost 48 hours and I’ve received nothing. If this is the norm, then so be it…but even when I had to wait for the Apple Intelligence early access that was only a few hours. Hopefully, this will be resolved quickly. I mean what’s the point of being developer beta testers, if we can’t test the beta?
3
2
503
Oct ’24
Image Playground Error: Cannot find protocol declaration for 'ImageGenerationViewControllerDelegate'
@available(macCatalyst 18.1, *) @available(iOS 18.1, *) extension CKImageSelectionManager: ImagePlaygroundViewController.Delegate { public func imagePlaygroundViewController(_ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) { } func presentImagePlayground() { let imagePlaygroundVC = ImagePlaygroundViewController() // Set delegate to self to receive the callback imagePlaygroundVC.delegate = self imagePlaygroundVC.isModalInPresentation = true // Prevents dismissal with swipe if needed self.delegate?.presentImageSelectionViewController(imagePlaygroundVC) } } This generates an error in the xcode generated swift header.
3
0
1.1k
Dec ’24
Guidance Implementing IndexedEntity and CSSearchableItemAttributeSet
I am working to add Spotlight indexing for my app entities as discussed in WWDC24's video "What's New in App Intents". That video goes over the IndexedEntity protocol and the integration with Spotlight via CSSearchableItemAttributeSet. What I'm seeing though does not match the video. In the video, the presenter goes through the sort of progressive approach you can take to getting this data into Spotlight starting with the basics and then expanding to include more support depending on how much the developer wants to do. What I'm seeing is that if you conform to IndexedEntity, your entities will appear in Spotlight using the name derived from public var displayRepresentation: DisplayRepresentation So, that works. Name appears... BUT the next part of the video goes into how to expand your implementation with more metadata for Spotlight via CSSearchableItemAttributeSet. The issue I'm seeing is that once that's implemented, the items disappear from Spotlight, almost like that implementation is overriding the base implementation in a way that no longer functions. My expectation is that an item with custom attributes would use them in Spotlight as appropriate, not disappear from search, i.e. what's shown in the video should work. I've got a sample project here: https://hanchor.s3.amazonaws.com/misc/IndexingTest.zip To reproduce with the sample: Build and run. Indexing is setup in the init() method so it will just run. Go to Spotlight and search for 'Huntersblau', a string included in the content set. At this point you should see a result - good! Stop the app and go back and uncomment the var attributeSet: CSSearchableItemAttributeSet implementation in IndexingTestApp.swift. This will provide custom attributes to Spotlight. Repeat steps 1 and 2 - you'll see now, it no longer appears in the search results - when CSSearchableItemAttributeSet is implemented, the item drops out of Spotlight.
3
0
1.1k
Dec ’24
Image Playground
How long does it usually take to get access to image playground. Its been about a week since I got IOS 18.2 public beta and still am waiting for access to the image playground. When I got apple intelligence only took a few hours.
3
0
551
Nov ’24
Making onscreen content available to Siri not requesting my Transferable
Howdy, I'm following along with this sample: https://developer.apple.com/documentation/appintents/making-onscreen-content-available-to-siri-and-apple-intelligence I've got everything up and building. I can confirm that the userActivity modifier is associating my App Intent via EntityIdentifier but my custom Transferable representation (text) is never being called and when Siri is doing the ChatGPT handoff, it's just offering to send a screenshot which is what it does when it has no custom representation. What could I doing wrong? Where should I be looking?
3
0
857
Dec ’24
lldb issues with Vision
HI, I've been modifying the Camera sample app found here: https://developer.apple.com/tutorials/sample-apps/capturingphotos-camerapreview ... in the processpreview images, I am calling in to the Vision APis to either detect a person or object, then I'm using the segmentation mask to extract the person and composite them onto a different background with some other filters. I am using coreimage to filter the CIImages, and converting and displaying as a SwiftUI Image. When running on my IPhone, it works fine. When running on my Iphone with the debugger, it crashes within a few seconds... Attached is a screenshot. At the top is an EXC_BAD_ACCESS in libRPAC.dylib`std::__1::__hash_table<std::__1::__hash_value_type<long, qos_info_t>, std::__1::__unordered_map_hasher<long, std::__1::__hash_value_type<long, qos_info_t>, std::__1::hash, std::__1::equal_to, true>, std::__1::__unordered_map_equal<long, std::__1::__hash_value_type<long, qos_info_t>, std::__1::equal_to, std::__1::hash, true>, std::__1::allocator<std::__1::__hash_value_type<long, qos_info_t>>>::__emplace_unique_key_args<long, std::__1::piecewise_construct_t const&, std::__1::tuple<long const&>, std::__1::tuple<>>: This was working fine a couple of days ago.. Not sure why it's popping up now. Am I correct in interpreting this as an LLDB issue? How do I fix it?
3
1
93
May ’25
Accessing Apple Intelligence APIs: Custom Prompt Support and Inference Capabilities
Hello Apple Developer Community, I'm exploring the integration of Apple Intelligence features into my mobile application and have a couple of questions regarding the current and upcoming API capabilities: Custom Prompt Support: Is there a way to pass custom prompts to Apple Intelligence to generate specific inferences? For instance, can we provide a unique prompt to the Writing Tools or Image Playground APIs to obtain tailored outputs? Direct Inference Capabilities: Beyond the predefined functionalities like text rewriting or image generation, does Apple Intelligence offer APIs that allow for more generalized inference tasks based on custom inputs? I understand that Apple has provided APIs such as Writing Tools, Image Playground, and Genmoji. However, I'm interested in understanding the extent of customization and flexibility these APIs offer, especially concerning custom prompts and generalized inference. Additionally, are there any plans or timelines for expanding these capabilities, perhaps with the introduction of new SDKs or frameworks that allow deeper integration and customization? Any insights, documentation links, or experiences shared would be greatly appreciated. Thank you in advance for your assistance!
3
0
277
Jun ’25
Python 3.13
Hello, Are there any plans to compile a python 3.13 version of tensorflow-metal? Just got my new Mac mini and the automatically installed version of python installed by brew is python 3.13 and while if I was in a hurry, I could manage to get python 3.12 installed and use the corresponding tensorflow-metal version but I'm not in a hurry. Many thanks, Alan
3
2
875
17h